Large-scale scene three-dimensional reconstruction, updating and editing method based on 3DGS

Abstract

The invention provides a large-scale scene three-dimensional reconstruction, updating and editing method based on 3DGS. The method comprises the following steps: carrying out camera pose calculation on a large-scale picture data set by utilizing an SFM motion structure recovery method based on block registration; a 3D GS three-dimensional reconstruction method based on blocks is used for carrying out three-dimensional reconstruction on a large scene, and high-consistency and low-storage-space modeling under the large scene is achieved. According to the invention, a solution is provided for construction of digital live-action reconstruction; according to the invention, aiming at the thought of carrying out motion structure recovery on a large-scale data set to solve the camera pose, the operation cost and time of the step are greatly reduced, and data support is provided for large-scene three-dimensional reconstruction. The method also supports the functions of updating and editing a dynamic scene, exporting a point cloud format and the like, facilitates the construction of a digital city, provides rich data and model support for many downstream tasks such as automatic driving, and can be widely applied to a plurality of fields and application scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a large-scene three-dimensional reconstruction, updating and editing method based on 3DGS. Background Art

[0002] With the continuous development of digital construction today, it is crucial to build a highly realistic, immersive, and interactive three-dimensional digital virtual reality scene. It is an important part of building digital platforms and digital twins. However, traditional lidar three-dimensional reconstruction methods face many challenges when faced with large-scene reconstruction, such as low modeling precision and lack of color information. To meet this challenge, three-dimensional reconstruction technology based on 3D Gaussian Splatting (3DGS) has become a solution. 3DGS technology can generate fine and continuous scene appearance modeling results while having good new perspective synthesis capabilities, which is helpful for the digital construction of large scenes.

[0003] like Figure 1 As shown in the figure, the picture on the left is a real-world photo, and the picture on the right is a synthetic picture rendered by 3D reconstruction modeling. 3D reconstruction is a technology that recovers the 3D shape of an object by inputting a series of 2D images. It is divided into two stages. The first stage is to restore the camera pose information in the 3D coordinate system through a series of 2D images input; the second stage is to restore the 3D appearance and structure information of the scene through the input image and its corresponding camera pose.

[0004] For 3D reconstruction, some traditional lidar-based methods have been proposed. In terms of urban 3D reconstruction, vehicle-mounted laser scanners are often used to obtain urban ground textures, and the city is reconstructed with drone aerial images. This method can obtain high-quality building structure models. However, this method is inefficient in manually matching images and point clouds, and the errors generated in the registration process can cause large deviations in the 3D model. In addition, the use of radar for 3D reconstruction can often only obtain the structural information of the scene, while the color appearance information of the scene is poorly restored.

[0005] In order to solve the problems of traditional algorithms, researchers generally introduce graphics-based algorithms into the field of 3D reconstruction. At present, the mainstream graphics-based 3D reconstruction algorithms can be roughly divided into two categories: NeRF algorithms based on neural radiation fields and algorithms based on 3DGS architecture. In the category of NeRF algorithms based on neural radiation fields, common algorithms include Instant-NGP, MiPNeRF, BlockNeRF, SwitchNeRF, etc.; and algorithms based on 3DGS architecture include VastGaussian, CityGaussian, etc. At present, mainstream algorithms can obtain better object structure and appearance information in indoor and some small outdoor scenes, but in the face of large outdoor scenes, these methods often cannot balance the modeling range and modeling accuracy well. When the modeling range is expanded, the modeling effect is not ideal. In addition, since these methods often use large models to meet the modeling needs of large scenes, these methods often have slow rendering speeds, cannot meet the needs of real-time rendering, and have poor robustness and cannot adapt to new scenes well. Therefore, although graphics-based methods have made significant progress in the field of 3D reconstruction, there are still certain challenges and limitations when dealing with large scenes.

[0006] Wang Zhangye and others from Zhejiang University have proposed "Unbounded Scene Reconstruction and New Perspective Synthesis Method and System Based on 3DGS" (Patent Application No.: CN202410549901.X). This method provides an unbounded scene reconstruction and new perspective synthesis method and system based on 3DGS, which mainly includes the following steps: SFM initialization, near point cloud rendering, far point cloud rendering, overall rendering, etc. However, this method only optimizes the three-dimensional reconstruction of small outdoor scenes. When the scene is enlarged and the model information is more, it will lead to the problem of rendering failure due to insufficient memory, and the model rendering accuracy is low in large scenes, which makes the three-dimensional reconstruction of large scenes difficult to practice. At the same time, SFM initialization takes a long time to run in the case of large-scale data sets and consumes more resources, which will cause this type of three-dimensional reconstruction method to become costly and difficult to implement. Therefore, although this method proposes an outdoor three-dimensional reconstruction scheme with application value, directly migrating it to a large scene with rich scenes will result in rendering failure, low modeling accuracy, poor robustness, and high training rendering time cost. Summary of the invention

[0007] In view of the defects in the prior art, the object of the present invention is to provide a large scene three-dimensional reconstruction, updating and editing method based on 3DGS.

[0008] According to one aspect of the present invention, a large scene 3D reconstruction method based on 3DGS is provided, comprising:

[0009] The camera pose is solved for large scene image datasets using the SFM motion structure recovery method based on block registration.

[0010] Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks.

[0011] The large-scene three-dimensional model is rendered to achieve large-scene three-dimensional reconstruction.

[0012] Preferably, the method of using the SFM motion structure recovery method based on block registration to perform camera pose calculation on a large-scale image dataset includes:

[0013] Pre-dividing the large scene image data set into blocks;

[0014] Using the motion structure recovery method, the camera pose is solved for each block separately;

[0015] Perform camera pose registration based on the camera pose of each block;

[0016] Based on the registration results between different blocks, the camera poses of all blocks in the world coordinate system are obtained.

[0017] Preferably, the pre-blocking of the large scene image data set includes:

[0018] The large scene image dataset is pre-divided spatially, images with similar characteristics are grouped into the same batch, and it is ensured that there is image overlap between batches; the images in the same batch are called blocks.

[0019] Preferably, the method of using the motion structure recovery algorithm to solve the camera pose for each block includes:

[0020] The SFM motion structure recovery method is used to perform motion structure recovery on each block separately to solve the camera pose information, that is, the internal and external parameter information, of each batch of input images.

[0021] Preferably, the camera pose registration based on the camera pose of each block includes:

[0022] Define the overlapping image between block A and block B as I 1 ,I 2 ,…,I N , N is the total number of overlapping images, are the position information of the camera corresponding to the i-th picture in the A system, B system and world coordinate system, respectively. are the posture information of the camera corresponding to the i-th picture in the A system, B system and world coordinate system, respectively, and there is a conversion relationship:

[0023]

[0024] Among them, R 1 ,R 2 are the rotation matrices from the world coordinate system to the A system and the B system, t 1 ,t 2 are the translation vectors from the world coordinate system to the A system and the B system respectively;

[0025] Based on the relationship between translation, rotation and scaling of the pose data between different blocks, the model of pose transformation between different batches is set as follows:

[0026]

[0027] Among them, ∝ 21 , R 21 , t 21 They are the scaling factor, rotation matrix and translation vector from B system to A system respectively;

[0028] The transformation relationship is calculated in an optimized way:

[0029] Solve the rotation matrix R 21 :

[0030]

[0031] By using the least squares method, we can solve the scaling factor ∝ 21 and the translation vector t 21 ;

[0032] ∝ 21 , R 21 , t 21 It is the pose registration information between different batches.

[0033] Preferably, obtaining the camera poses of all blocks in the world coordinate system based on the registration results between different blocks includes:

[0034] Based on the fact that the pose registration information between different batches is all linear transformation, the transformation between different batches is transferred;

[0035] Based on the transferred transformation, the camera poses of all batches are aligned to the world coordinate system to realize the camera pose solution of the large scene image dataset.

[0036] Preferably, the three-dimensional reconstruction of the large scene is performed based on the calculated camera pose using a block-based 3DGS three-dimensional reconstruction method to obtain a three-dimensional model of the large scene composed of blocks, including:

[0037] The data set consisting of the calculated camera poses is divided into several blocks by adopting a block-by-block approach;

[0038] Perform three-dimensional reconstruction on each block to obtain the point cloud file of each block;

[0039] Combine point cloud files from different blocks into a complete 3D large scene model.

[0040] Preferably, rendering the large scene three-dimensional model to achieve large scene three-dimensional reconstruction includes:

[0041] During initialization, only the models of the four blocks adjacent to the initial view are loaded;

[0042] When the view moves, the rendering model is updated as the view moves, specifically:

[0043] For a given viewing angle, processing is performed according to whether the projection area of ​​the given viewing angle is within the area to be rendered of the large scene 3D model: if the projection area of ​​the given viewing angle has no overlapping area with any block of the large scene 3D model, the background is directly output; if the projection area of ​​the given viewing angle is within the area to be rendered of the large scene 3D model, only the block with the highest overlap with the projection area of ​​the given viewing angle is rendered;

[0044] When the block with the highest overlap with the projection area of ​​the given viewing angle is rendered, the boundary area of ​​the rendering model is optimized using the adjacent blocks of the block.

[0045] According to a second aspect of the present invention, a large scene three-dimensional updating method based on 3DGS is provided, comprising:

[0046] The camera pose is solved for large scene image datasets using the SFM motion structure recovery method based on block registration.

[0047] Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks.

[0048] Updating a single block model of the large scene three-dimensional model;

[0049] The updated single region model is merged with the original large scene 3D model again to obtain a large scene 3D updated model.

[0050] According to a third aspect of the present invention, a large scene three-dimensional editing method based on 3DGS is provided, comprising:

[0051] The camera pose is solved for large scene image datasets using the SFM motion structure recovery method based on block registration.

[0052] Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks.

[0053] Editing the large scene three-dimensional model includes:

[0054] Editing the point cloud file of the block that needs to be edited in the large-scene three-dimensional model, wherein the editing includes deletion, appearance change, and scaling;

[0055] or,

[0056] The point cloud file of a new object is unified into the large-scene three-dimensional model through coordinate transformation and scaling, so as to realize the virtual layout design and dynamic editing layout of the new object.

[0057] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0058] The embodiment of the present invention provides a method for 3DGS to perform 3D reconstruction of large scenes. The idea of ​​performing motion structure recovery and solving camera pose for large scene image data sets greatly reduces the operating cost and time of this step, and provides data support for 3D reconstruction of large scenes.

[0059] The embodiment of the present invention relates to a 3DGS technology, which is used to realize the three-dimensional reconstruction of large-scale scenes and can construct a model with high consistency and low storage space requirements. The technology effectively solves the problem of insufficient memory and failure to successfully render in the process of digital real scene reconstruction as the scale of the scene increases and the amount of model information increases.

[0060] The embodiment of the present invention provides a method for 3DGS to perform large-scene three-dimensional reconstruction, update and editing. It not only provides a full process of block training and rendering of large-scene three-dimensional reconstruction based on 3DGS, but also supports dynamic scene updating, editing and point cloud format export and other functions. It is not only conducive to the construction of digital cities, but also provides rich data and model support for many downstream tasks such as autonomous driving, and can be widely used in multiple fields and application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0062] Figure 1 It is a three-dimensional reconstruction effect diagram in the prior art;

[0063] Figure 2 This is a framework diagram of the rendering steps of a large scene three-dimensional reconstruction method based on 3DGS in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0065] In one embodiment of the present invention, a large scene 3D reconstruction method based on 3DGS is provided, comprising:

[0066] Step 1: Use the SFM motion structure recovery method based on block registration to solve the camera pose of the large scene image dataset;

[0067] Step 2: Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks;

[0068] Step 3: Render the large scene three-dimensional model to obtain a large scene three-dimensional reconstruction model.

[0069] The above embodiments can realize modeling with strong consistency and low storage space in large scenes, and provide a solution for digital scene reconstruction.

[0070] In order to greatly reduce the operating cost and time, in a preferred embodiment of the present invention, the SFM motion structure recovery method based on block registration in step 1 above can be implemented by the following steps:

[0071] S101, pre-dividing a large scene image data set into blocks to obtain a plurality of blocks;

[0072] S102, using the SFM motion structure recovery method, the camera pose is solved for the blocks divided in S101;

[0073] S103, performing camera pose registration on camera poses of different blocks for subsequent acquisition of camera pose data of the entire large scene data.

[0074] S104, based on the registration results between different blocks, the camera poses of all blocks in the world coordinate system are obtained, that is, the camera pose data of the entire large scene is obtained.

[0075] In this embodiment, the idea of ​​performing motion structure recovery and solving camera pose for a large scene image data set greatly reduces the running cost and time of this step, and provides data support for large scene 3D reconstruction.

[0076] Large scene image datasets can usually be obtained by taking outdoor pictures with a camera.

[0077] In order to achieve rapid processing of large scene image data sets, in a preferred embodiment, a preferred operation of step S101 is provided to pre-divide the large scene image data set in space. Specifically, images with similar characteristics are divided into the same batch, and it is ensured that there is image overlap between batches for subsequent processing. Similar characteristics here include, for example, buildings with similar overall height and structure, and scenery with similar appearance, such as woods. By pre-dividing the image data set in space, different batches can be formed (a batch can also be called a block), which is beneficial to the subsequent processing of block images and the final three-dimensional reconstruction.

[0078] Based on the multiple block images obtained by the above pre-division, in the execution of the above S102, the SFM motion structure recovery method is used to perform motion structure recovery on each block to solve the camera pose information, that is, the internal and external parameter information of the input block image. Through this operation, the running time of the feature matching step in the camera solution process can be significantly reduced, and the solution cost and cumulative error can be reduced.

[0079] In order to improve the registration rate between block images, in a preferred embodiment, S103 is implemented to perform block registration between different batches. The specific steps are as follows.

[0080] Assume that the overlapping image between block A and block B is I 1 ,I 2 ,…,I N , N is the total number of overlapping pictures. are the position information of the camera corresponding to the i-th picture in the A system, B system and world coordinate system, respectively. are the posture information of the camera corresponding to the i-th picture in the A system, B system and world coordinate system, respectively, and there is a conversion relationship:

[0081]

[0082] Among them, R 1 ,R 2 are the rotation matrices from the world coordinate system to the A system and the B system, t 1 ,t 2 They are the translation vectors from the world coordinate system to the A system and the B system respectively.

[0083] It is known that the pose data between different blocks may have translation, rotation and scaling relationships. The pose transformation model between different batches can be set as:

[0084]

[0085] Among them, ∝ 21 , R 21 , t21 They are respectively the scaling factor, rotation matrix and translation vector from the B system to the A system.

[0086] Since the camera pose obtained by the existing motion structure recovery has certain errors, the transformation relationship from system A to system B cannot be solved by direct linear transformation. Here, an optimized method is used to calculate the transformation relationship.

[0087] First solve the rotation matrix R 21 :

[0088]

[0089] Then, by the least squares method, we solve the scaling factor ∝ 21 and the translation vector t 21 Thus, the pose registration information between different batches is obtained. In the above embodiment, the transformation relationship calculated in an optimized manner can be solved for large-scale data and large scenes with high accuracy, meeting the needs of modeling.

[0090] In the above S103, due to the transformation between different blocks (position registration information between different batches ∝ 21 , R 21 , t 21 ) are all linear transformations, so in a preferred embodiment, the transformations between different batches can be transferred, and then the camera poses of all batches can be aligned to the world coordinate system, thereby realizing the camera pose solution of large scene data sets.

[0091] Given that the traditional SFM algorithm must match new perspective images with features extracted from all previous images when processing them, and as the number of images increases, errors will accumulate, resulting in increased running time, cost, and reduced solution accuracy. This embodiment demonstrates the complete process of camera pose solution generation for a large scene dataset. For large scene datasets, this method pre-divides the images into different regions, so when a new image is input, it only needs to be matched with features in the same region. By implementing a block strategy, the running time, solution cost, and accumulated error of the feature matching step in the camera solution process are significantly reduced. In addition, block processing allows asynchronous parallel solution of information in different regions, thereby further improving operating efficiency.

[0092] In order to further complete the three-dimensional reconstruction, in one embodiment of the present invention, the block-based 3DGS three-dimensional reconstruction method used in step 2 can be implemented by the following steps to build a large scene three-dimensional model:

[0093] S201, using a block method, divide the camera pose data of the entire large scene obtained in step 1 into several blocks; in this step, the block division method is the same as the block division method of S101.

[0094] S202, performing three-dimensional reconstruction on each block to obtain a point cloud file of each block;

[0095] The three-dimensional reconstruction in this step refers to reconstructing the three-dimensional appearance structure model of the block scene in the form of a three-dimensional Gaussian sphere using the camera image and the corresponding pose data obtained in step 1, and storing it in the format of a point cloud.

[0096] S203, synthesize the point cloud files of different blocks into a complete large scene 3D model.

[0097] In order to more effectively render and reconstruct the three-dimensional model of a large scene, in a preferred embodiment of the present invention, when executing step S3, the three-dimensional reconstruction model in the point cloud format generated in step 2 is first obtained (the point cloud format model stores a three-dimensional Gaussian sphere that can represent the scene appearance structure information), and then the three-dimensional Gaussian sphere is used for rendering to obtain a visualized model. The specific implementation method is as follows:

[0098] (1) During initialization, considering the efficiency of rendering, only the models of the four blocks adjacent to the initial view are loaded (models in point cloud format).

[0099] (2) When the view moves, the model is updated as the view moves. The point cloud file is large. During the processing, in order to prevent memory overflow caused by loading all the files, the loaded models need to be deleted to free up memory. Generally, the models that are far away are deleted, that is, the block models that are more than the threshold distance away from the initial view and the four blocks above, below, left and right. Since the view will not change suddenly, when updating the model, it is ensured that the models of the four blocks adjacent to the view are always loaded. Specifically:

[0100] For a given viewing angle, first determine whether the area projected by the viewing angle is within the entire large scene 3D model to be rendered area. If the projected area does not overlap with any block, then directly output the background. Figure 2 As shown, when the viewing angle range is determined to be out of bounds, the background image is returned and an error is stated.

[0101] For a given viewing angle, when the viewing angle range is determined to be within the rendering range, it is necessary to determine whether the models of the upper, lower, left, and right areas of the given viewing angle have been loaded. If loaded, they are rendered directly; if not loaded, a new model needs to be loaded, and the specified model is deleted (the specified model generally refers to the model loaded in the previous earlier inference viewing angle), and then the model is rendered.

[0102] (3) When rendering, the overlap between the current viewing angle projection area and each block of the large scene 3D model is calculated, and the block with the highest overlap is used for model rendering.

[0103] (4) When the block model with the highest overlap in (3) is rendered, since the effective area of ​​a single model is limited, the boundary area of ​​the rendered model needs to be optimized using the adjacent blocks of the block. The boundary area refers to the part where two blocks are connected, and it is impossible to obtain a more ideal result by relying solely on a single model. Therefore, when the difference in the distance between the rendering perspective and the two blocks is less than a certain threshold, boundary optimization is performed. Boundary optimization means that when rendering, two blocks are placed in the same scale of space and rendered together. A denser and more precise three-dimensional Gaussian sphere is used for blocks at a close distance, while a coarser three-dimensional Gaussian sphere is used for blocks at a long distance, thereby reducing the complexity of rendering.

[0104] Based on the same inventive concept, in other embodiments of the present invention, a large scene 3D update method based on 3DGS is provided. The large scene 3D model formed by the block combination of the large scene 3D reconstruction method based on 3DGS in the above embodiment has the local update capability, and can achieve rapid update when the local area changes. Specifically, the steps of the large scene 3D update method can be as follows:

[0105] S1, using the SFM motion structure recovery method based on block registration to solve the camera pose of large scene image datasets;

[0106] S2, based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks;

[0107] S3: When a building, street or scene changes and the model needs to be updated, there is no need to update the entire large scene model, only the single block model before the model is merged;

[0108] S4, merging the updated single region model with the original large scene 3D model again to obtain a large scene 3D updated model.

[0109] Based on the same inventive concept, in other embodiments of the present invention, a large scene 3D editing method based on 3DGS is provided. The large scene 3D model formed by the block combination of the large scene 3D reconstruction method based on 3DGS in the above embodiment has the ability of scene editing. The large scene 3D editing method based on 3DGS can adopt the following steps:

[0110] S10, using the SFM motion structure recovery method based on block registration to solve the camera pose of large scene image datasets;

[0111] S20, based on the calculated camera pose, the large scene is reconstructed in three dimensions using a block-based 3DGS three-dimensional reconstruction method to obtain a three-dimensional model of the large scene composed of blocks;

[0112] S30, editing the large scene three-dimensional model, including:

[0113] Editing the point cloud file of the block that needs to be edited in the large-scene three-dimensional model, wherein the editing includes deletion, appearance change, and scaling;

[0114] or,

[0115] The point cloud file of a new object is unified into the large-scene three-dimensional model through coordinate transformation and scaling, so as to realize the virtual layout design and dynamic editing layout of the new object.

[0116] Of course, based on the same inventive concept, in other embodiments of the present invention, point cloud format export can also be performed. Specifically, 3DGS solution will obtain a series of three-dimensional Gaussian spheres, and point cloud format export is to convert the three-dimensional Gaussian spheres solved by 3DGS into three-dimensional point clouds and then export them as a PLY point cloud model, which includes the position, color, and transparency information of the three-dimensional point cloud.

[0117] The above-mentioned embodiments not only provide a full process of block training and rendering of large-scene three-dimensional reconstruction based on 3DGS, but also support functions such as dynamic scene updating, editing, and point cloud format export. It is not only conducive to the construction of digital cities, but also provides rich data and model support for many downstream tasks such as autonomous driving, and can be widely used in multiple fields and application scenarios.

[0118] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A large scene 3D reconstruction method based on 3DGS, characterized in that: include: The camera pose is solved for large scene image datasets using the SFM motion structure recovery method based on block registration. Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks. The large-scene three-dimensional model is rendered to achieve large-scene three-dimensional reconstruction.

2. The large scene 3D reconstruction method based on 3DGS according to claim 1, characterized in that: The method of using the SFM motion structure recovery method based on block registration to solve the camera pose of a large-scale image dataset includes: Pre-dividing the large scene image data set into blocks; Using the motion structure recovery method, the camera pose is solved for each block separately; Perform camera pose registration based on the camera pose of each block; Based on the registration results between different blocks, the camera poses of all blocks in the world coordinate system are obtained.

3. The large scene 3D reconstruction method based on 3DGS according to claim 2, characterized in that: The pre-blocking of the large scene image data set includes: The large scene image dataset is pre-divided spatially, images with similar characteristics are grouped into the same batch, and it is ensured that there is image overlap between batches; the images in the same batch are called blocks.

4. The large scene 3D reconstruction method based on 3DGS according to claim 2, characterized in that: The motion structure recovery algorithm is used to solve the camera pose for each block, including: The SFM motion structure recovery method is used to perform motion structure recovery on each block separately to solve the camera pose information, that is, the internal and external parameter information, of each batch of input images.

5. The large scene 3D reconstruction method based on 3DGS according to claim 2, characterized in that: The camera pose registration based on the camera pose of each block includes: Define the overlapping images between block A and block B as I1, I2, …, I N , N is the total number of overlapping images, are the position information of the camera corresponding to the i-th picture in the A system, B system and world coordinate system, respectively. are the posture information of the camera corresponding to the i-th picture in the A system, B system and world coordinate system, respectively, and there is a conversion relationship: Among them, R1 and R2 are the rotation matrices from the world coordinate system to the A system and the B system respectively, and t1 and t2 are the translation vectors from the world coordinate system to the A system and the B system respectively; Based on the relationship between translation, rotation and scaling of the pose data between different blocks, the model of pose transformation between different batches is set as follows: Among them, ∝ 21 , R 21 , t 21 They are the scaling factor, rotation matrix and translation vector from B system to A system respectively; The transformation relationship is calculated in an optimized way: Solve the rotation matrix R 21 : By using the least squares method, we can solve the scaling factor ∝ 21 and the translation vector t 21 ; ∝ 21 , R 21 , t 21 It is the pose registration information between different batches.

6. The large scene 3D reconstruction method based on 3DGS according to claim 2, characterized in that: The camera poses of all blocks in the world coordinate system are obtained based on the registration results between different blocks, including: Based on the fact that the pose registration information between different batches is all linear transformation, the transformation between different batches is transferred; Based on the transferred transformation, the camera poses of all batches are aligned to the world coordinate system to realize the camera pose solution of the large scene image dataset.

7. The large scene 3D reconstruction method based on 3DGS according to claim 1, characterized in that: Based on the calculated camera pose, the large scene is reconstructed in three dimensions using the block-based 3DGS three-dimensional reconstruction method to obtain a large scene three-dimensional model composed of blocks, including: The data set consisting of the calculated camera poses is divided into several blocks by adopting a block-by-block approach; Perform three-dimensional reconstruction on each block to obtain the point cloud file of each block; Combine point cloud files from different blocks into a complete 3D large scene model.

8. The large scene 3D reconstruction method based on 3DGS according to claim 7, characterized in that: The rendering of the large scene three-dimensional model to achieve large scene three-dimensional reconstruction includes: During initialization, only the models of the four blocks adjacent to the initial view are loaded; When the view moves, the rendering model is updated as the view moves, specifically: For a given viewing angle, processing is performed according to whether the projection area of ​​the given viewing angle is within the area to be rendered of the large scene 3D model: if the projection area of ​​the given viewing angle has no overlapping area with any block of the large scene 3D model, the background is directly output; if the projection area of ​​the given viewing angle is within the area to be rendered of the large scene 3D model, only the block with the highest overlap with the projection area of ​​the given viewing angle is rendered; When the block with the highest overlap with the projection area of ​​the given viewing angle is rendered, the boundary area of ​​the rendering model is optimized using the adjacent blocks of the block.

9. A large scene three-dimensional updating method based on 3DGS, characterized in that: include: The camera pose is solved for large scene image datasets using the SFM motion structure recovery method based on block registration. Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks. Updating a single block model of the large scene three-dimensional model; The updated single region model is merged with the original large scene 3D model again to obtain a large scene 3D updated model.

10. A large scene three-dimensional editing method based on 3DGS, characterized in that: include: The camera pose is solved for large scene image datasets using the SFM motion structure recovery method based on block registration. Based on the calculated camera pose, the large scene is reconstructed using the block-based 3DGS 3D reconstruction method to obtain a large scene 3D model composed of blocks. Editing the large scene three-dimensional model includes: Editing the point cloud file of the block that needs to be edited in the large-scene three-dimensional model, wherein the editing includes deletion, appearance change, and scaling; or, The point cloud file of a new object is unified into the large-scene three-dimensional model through coordinate transformation and scaling, so as to realize the virtual layout design and dynamic editing layout of the new object.

Images