Three-dimensional point cloud reconstruction method and device based on Gaussian splash algorithm, and medium

Abstract

The invention discloses a three-dimensional point cloud reconstruction method and device based on a Gaussian splash algorithm, and a medium, and relates to the technical field of three-dimensional point cloud reconstruction, and the method comprises the steps: obtaining a plurality of images; inputting the plurality of images into a pre-trained VGGT model to obtain DPT features obtained by feature coding output by a VIT encoder based on the VGGT model, and camera parameters and global point cloud data output by the VGGT model; based on the DPT features, the camera parameters and the global point cloud data, obtaining Gaussian splashing parameters corresponding to each pixel position of each image; and obtaining target three-dimensional point cloud data based on the Gaussian ball position increment in the Gaussian splashing parameters and the local point cloud data under the camera coordinates. The method is used for solving the problem that in the prior art, when multi-view three-dimensional point cloud reconstruction is carried out, the difference between simulation and a real object is large, and the difference between simulation and the real object is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional point cloud reconstruction, and in particular to a three-dimensional point cloud reconstruction method, device and medium based on a Gaussian splash algorithm. Background Art

[0002] In multi-view Figure 3 During the reconstruction of a 3D point cloud, it is necessary to obtain the ground truth of the point cloud corresponding to the image. Currently, CAD model rendering simulation is used to obtain the rendered RGB image and the ground truth of the point cloud corresponding to the image. However, the rendered RGB image often differs from the actual object due to factors such as lighting and material. This leads to a certain loss of precision when the rendered RGB image is generalized to the actual object image. In addition, the use of rendered RGB images and ground truth point cloud values ​​for model training also results in a certain loss of precision, resulting in a large gap between the simulation and the actual object. Summary of the Invention

[0003] In response to the above problems and technical requirements, the applicant has proposed a three-dimensional point cloud reconstruction method, device and medium based on Gaussian splash algorithm to solve the existing technology in the multi-view Figure 3 When reconstructing the point cloud, there is a large gap between the simulation and the real object, and the gap between the simulation and the real object is reduced.

[0004] The present invention provides a method for reconstructing a three-dimensional point cloud based on a Gaussian splattering algorithm. The method includes:

[0005] Acquiring multiple images, wherein the multiple images are obtained by photographing a preset area at the same time by cameras located at different positions;

[0006] Input multiple images into the pre-trained VGGT model to obtain DPT features based on the feature encoding output by the VIT encoder of the VGGT model, as well as the camera parameters and global point cloud data output by the VGGT model. The VGGT model is trained based on image samples and 3D point cloud samples.

[0007] Based on the DPT features, the camera parameters and the global point cloud data, obtaining Gaussian splash parameters corresponding to each pixel position of each image;

[0008] Based on the Gaussian sphere position increment in the Gaussian splash parameter and the local point cloud data in the camera coordinates, target three-dimensional point cloud data is obtained, wherein the local point cloud data is obtained based on the global point cloud data.

[0009] According to the three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm provided in an embodiment of the present application, based on the DPT feature, the camera parameters and the global point cloud data, the Gaussian splattering parameters corresponding to each pixel position of each image are obtained, including:

[0010] Converting the global point cloud data into the coordinate systems of each camera based on the camera parameters to obtain converted local point cloud data;

[0011] Performing alignment processing on the local point cloud data so that the aligned local point cloud data is aligned with the DPT feature;

[0012] Fusing the DPT features with the aligned local point cloud data to obtain fused point cloud data;

[0013] Convolution interpolation processing is performed on the fused point cloud data to obtain the Gaussian splash parameters.

[0014] According to the three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm provided in an embodiment of the present application, convolution interpolation processing is performed on the fused point cloud data to obtain the Gaussian splattering parameters, including:

[0015] Inputting the fused point cloud data into the CNN network for convolution processing to obtain fused features;

[0016] The fusion feature is upsampled using bilinear interpolation to obtain a target feature, and the Gaussian splash parameter is obtained based on the target feature.

[0017] According to the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided in an embodiment of the present application, target three-dimensional point cloud data is obtained based on the Gaussian sphere position increment in the Gaussian splash parameter and the local point cloud data in the camera coordinates, including:

[0018] Determine a point cloud increment corresponding to a Gaussian sphere position increment in the Gaussian splash parameter;

[0019] Calculating the point cloud sum of the local point cloud data and the point cloud increment in the camera coordinate system;

[0020] A Gaussian rendering operation is performed on the point cloud to obtain the target three-dimensional point cloud data.

[0021] According to the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided by an embodiment of the present application, the Gaussian splash parameters include: transparency, Gaussian sphere position increment, Gaussian sphere deformation parameter, Gaussian sphere rotation parameter and spherical harmonic coefficient.

[0022] According to the three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm provided in an embodiment of the present application, before acquiring multiple images, the method further includes:

[0023] Acquire training sample data, wherein the training sample data includes: image samples and three-dimensional point cloud samples, wherein the image samples include: first image samples carrying camera parameters and second image samples not carrying camera parameters;

[0024] The training sample data is input into the VGGT model. When it is determined that the image sample is the first image sample, the camera parameter prediction loss and the image reconstruction loss are calculated, and the model parameters of the VGGT model are optimized based on the camera parameter prediction loss and the image reconstruction loss. When it is determined that the image sample is the second image sample, the image reconstruction loss is calculated, and the model parameters are optimized based on the image reconstruction loss. The VGGT model training is determined to be completed when the number of iterations reaches a preset number.

[0025] According to the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided in an embodiment of the present application, the VGGT model includes multiple blocks;

[0026] Before training the VGGT model, it also includes:

[0027] A Block is selected from the plurality of Block blocks to add supervision information, wherein the supervision information is used to supervise the VGGT model to predict camera parameters and predict global point cloud data.

[0028] According to the three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm provided by an embodiment of the present application, the first image sample includes: a rendered image obtained by rendering by application software, and the second image sample includes: a real image obtained by camera shooting.

[0029] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the program, the steps of the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm as described in any one of the above items are implemented.

[0030] An embodiment of the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm as described in any one of the above items are implemented.

[0031] The three-dimensional point cloud reconstruction method, device and medium based on the Gaussian splash algorithm provided in the embodiments of the present application obtain multiple images; input the multiple images into a pre-trained VGGT model to obtain DPT features obtained by feature encoding based on the VIT encoder output of the VGGT model, and obtain camera parameters and global point cloud data output by the VGGT model. The present application uses the trained VGGT model to predict camera parameters and global point cloud data, thereby improving the accuracy of point cloud data acquisition; further, based on the DPT features, camera parameters and global point cloud data, Gaussian splash parameters corresponding to each pixel position of each image are obtained; based on the Gaussian sphere position increment in the Gaussian splash parameters and the local point cloud data under the camera coordinates, target three-dimensional point cloud data is obtained. The present application obtains Gaussian splash parameters by analyzing and processing the DPT features, camera parameters and global point cloud data, and then optimizes the global point cloud data according to the Gaussian splash parameters to obtain the final target three-dimensional point cloud data, thereby reducing the gap between simulation and reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 This is one of the flow charts of the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided in the embodiment of the present application;

[0034] Figure 2 This is the second flow chart of the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided in the embodiment of the present application;

[0035] Figure 3 This is the third flow chart of the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided in the embodiment of the present application;

[0036] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] The embodiment of the present application provides a three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm. The method can be applied to smart terminals and can also be applied to servers. The present application uses the method applied to a server as an example for illustration. This is for illustration only and is not intended to limit the scope of protection of the present application. Some other descriptions in the embodiments are also for illustration only and will not be described one by one later. Figure 1 As shown, the method includes:

[0039] Step 101: Acquire multiple images.

[0040] Among them, multiple images are obtained by shooting a preset area at the same time by cameras located at different positions.

[0041] In this case, each camera captures one image at a time. The number of cameras and images captured at the same time is the same. Different cameras are assigned different camera serial numbers, and the images captured by the cameras are position-coded based on the camera serial numbers. Different images are assigned different position codes.

[0042] In step 102, multiple images are input into a pre-trained VGGT model to obtain DPT features obtained by feature encoding based on the VIT encoder output of the VGGT model, as well as camera parameters and global point cloud data output by the VGGT model.

[0043] Among them, the VGGT model is trained based on image samples and three-dimensional point cloud samples.

[0044] Step 103 : Based on the DPT features, camera parameters, and global point cloud data, the Gaussian splash parameters corresponding to each pixel position of each image are obtained.

[0045] The Gaussian splash parameters include transparency, Gaussian sphere position increment, Gaussian sphere deformation parameter, Gaussian sphere rotation parameter and spherical harmonic coefficient.

[0046] Step 104 : Obtain target three-dimensional point cloud data based on the Gaussian sphere position increment in the Gaussian splash parameter and the local point cloud data in the camera coordinates.

[0047] Among them, the local point cloud data is obtained based on the global point cloud data.

[0048] The three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided in the embodiment of the present application obtains multiple images; inputs the multiple images into a pre-trained VGGT model to obtain DPT features obtained by feature encoding based on the VIT encoder output of the VGGT model, and obtains camera parameters and global point cloud data output by the VGGT model. The present application uses the trained VGGT model to predict camera parameters and global point cloud data, thereby improving the accuracy of point cloud data acquisition; further, based on the DPT features, camera parameters and global point cloud data, the Gaussian splash parameters corresponding to each pixel position of each image are obtained; based on the Gaussian sphere position increment in the Gaussian splash parameters and the local point cloud data under the camera coordinates, the target three-dimensional point cloud data is obtained. The present application obtains Gaussian splash parameters by analyzing and processing the DPT features, camera parameters and global point cloud data, and then optimizes the global point cloud data according to the Gaussian splash parameters to obtain the final target three-dimensional point cloud data, thereby reducing the gap between simulation and reality.

[0049] In a specific embodiment, based on DPT features, camera parameters and global point cloud data, the Gaussian splash parameters corresponding to each pixel position of each image are obtained as follows: Figure 2 As shown:

[0050] Step 201 : converting the global point cloud data into the coordinate system of each camera based on the camera parameters to obtain converted local point cloud data.

[0051] Step 202 : performing alignment processing on the local point cloud data so that the aligned local point cloud data is aligned with the DPT features.

[0052] Step 203: Fusing the DPT features and the aligned local point cloud data to obtain fused point cloud data.

[0053] Step 204 : Perform convolution interpolation processing on the fused point cloud data to obtain Gaussian splash parameters.

[0054] Specifically, local point cloud data is input into a CNN network, and the CNN outputs local point cloud data aligned with the DPT features. The CNN has already determined the size of the DPT features. Since the sizes of the local point cloud data and the DPT features are inconsistent, after obtaining the local point cloud data, the CNN is used to resize the local point cloud data to match the size of the DPT features, thereby achieving alignment between the DPT features and the local point cloud data.

[0055] Specifically, the DPT features are merged with the aligned local point cloud data, and the Unet structure is used to perform feature fusion on the merged data to obtain fused point cloud data.

[0056] Specifically, different cameras correspond to different camera serial number codes, and an image carries the camera serial number code, which represents the index number of the image captured by the camera.

[0057] Among them, the position of one camera can be selected from multiple cameras to establish a reference coordinate system, and the position calibration between the cameras can be completed based on the reference coordinate system, and then the relationship between the captured images can be obtained based on the obtained calibration results.

[0058] Among them, in this application, the camera serial number code and the image index number can be simply set to be the same, that is, the camera and the image are one-to-one corresponding.

[0059] Specifically, the VIT encoder generates a position code based on the index number and adds the position code to the feature code of the corresponding image; the feature code with the added position code is input into multiple cross-attention modules, and the feature code of each image and the feature code of other images are cross-attention processed by multiple cross-attention modules to obtain the processed feature code; the processed feature code is fused to obtain the DPT feature.

[0060] Specifically, the VIT encoder adopts the form of absolute position encoding, and each patch has a unique encoding vector, which is generated by linear transformation and added with the generated position encoding to retain the position information.

[0061] Specifically, a preset feature extraction layer determination method is used to determine a target cross-attention module from multiple cross-attention modules; the features output by the target cross-attention module are extracted, and the extracted features are fused to obtain and output DPT features.

[0062] Specifically, the feature extraction layer determination method includes: randomly selecting a preset number of cross-attention modules from multiple cross-attention modules as target cross-attention modules; selecting cross-attention modules from multiple cross-attention modules according to preset intervals as target cross-attention modules; determining the target cross-attention module from multiple cross-attention modules in the manner of 2*N / 4, 3*N / 4, N, where N represents the total number of cross-attention modules; etc.

[0063] Among them, regarding the method of determining the feature extraction layer, the user can choose according to his or her actual needs, and this application does not impose any restrictions.

[0064] In a specific embodiment, the convolution interpolation processing is performed on the fused point cloud data to obtain the Gaussian splash parameters, which includes:

[0065] The fused point cloud data is input into the CNN network for convolution processing to obtain fused features; the fused features are upsampled using bilinear interpolation to obtain target features, and Gaussian splash parameters are obtained based on the target features.

[0066] Specifically, the target features are processed using the sigmoid function to obtain transparency; the target features are processed using the exp function to obtain Gaussian sphere deformation parameters; the target features are normalized by the 2-norm to obtain Gaussian sphere rotation parameters, etc.

[0067] The Gaussian splash parameter may be represented in the form of a Gaussian splash parameter map, and the resolution of the Gaussian splash parameter map is consistent with the resolution of the image.

[0068] In a specific embodiment, based on the Gaussian sphere position increment in the Gaussian splash parameter and the local point cloud data in the camera coordinates, the specific implementation of obtaining the target three-dimensional point cloud data is as follows: Figure 3 As shown:

[0069] Step 301 : Determine the point cloud increment corresponding to the Gaussian sphere position increment in the Gaussian splash parameter.

[0070] Step 302 : Calculate the point cloud sum of the local point cloud data and the point cloud increment in the camera coordinate system.

[0071] Step 303: Perform a Gaussian rendering operation on the point cloud to obtain target three-dimensional point cloud data.

[0072] In a specific embodiment, before acquiring multiple images, a VGGT model is trained.

[0073] The training process includes:

[0074] Obtain training sample data; input the training sample data into the VGGT model, and when it is determined that the image sample is the first image sample, calculate the camera parameter prediction loss and the image reconstruction loss, and optimize the model parameters of the VGGT model based on the camera parameter prediction loss and the image reconstruction loss; when it is determined that the image sample is the second image sample, calculate the image reconstruction loss, and optimize the model parameters based on the image reconstruction loss, until the number of iterations reaches a preset number, determining that the VGGT model training is completed.

[0075] The training sample data includes: image samples and three-dimensional point cloud samples, wherein the image samples include: first image samples carrying camera parameters and second image samples not carrying camera parameters.

[0076] Specifically, the first image sample includes: a rendered image obtained by rendering through application software, and the second image sample includes: a real image obtained by camera shooting (marked with camera parameters).

[0077] Specifically, the VGGT model training process is different depending on the image samples (divided into first image samples and second image samples).

[0078] The first training process for the first image sample involves: requiring the VGGT model to predict camera parameters and global point cloud data, which incurs camera parameter prediction loss. Furthermore, the VGGT model needs to project the Gaussian splatter parameters onto the global coordinate system for rendering based on the predicted camera parameters, which incurs image reconstruction loss. Therefore, the model parameters need to be optimized based on the camera parameter prediction loss and the image reconstruction loss.

[0079] The second training process for the second image sample does not require the VGGT model to predict camera parameters, so there is no need to calculate the camera parameter prediction loss. However, the Gaussian splatter parameters are projected onto the global coordinate system based on the predicted camera parameters for rendering. This process incurs an image reconstruction loss. Therefore, the model parameters need to be optimized based on the image reconstruction loss.

[0080] Specifically, the VGGT model can be pre-trained using the first training process. When the first training process reaches a first preset number of iterations, the VGGT model can be fine-tuned using the second training process. Finally, the first training process and the second training process are collaboratively trained, specifically by using differentiated gradient backpropagation for the first image sample and the second image sample to obtain the final VGGT model.

[0081] In a specific embodiment, the VGGT model includes multiple blocks.

[0082] Specifically, before training the VGGT model, one block is selected from multiple blocks to add supervision information.

[0083] Among them, the supervision information is used to supervise the VGGT model to predict camera parameters and predict global point cloud data.

[0084] Among them, a Block block can be selected from the Block blocks between M / 2 and M to add supervision information, where M is the number of Block blocks.

[0085] Among them, a Block includes multiple attention networks.

[0086] This application adds supervision information to the Block block of the VGGT model to supervise the VGGT model network when extracting camera parameters and point cloud data, which can effectively improve the reconstruction accuracy.

[0087] This application reduces the gap between simulation and reality by predicting the VGGT model and obtaining Gaussian splash parameters based on DPT features, camera parameters and global point cloud data, and then optimizing the global point cloud data according to the Gaussian splash parameters to obtain the final target three-dimensional point cloud data.

[0088] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other via the communication bus 404. The processor 401 may call logic instructions in the memory 403 to execute a three-dimensional point cloud reconstruction method based on a Gaussian splattering algorithm.

[0089] In addition, the logic instructions in the above-mentioned memory 403 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0090] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm provided by the above methods.

[0091] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the three-dimensional point cloud reconstruction method based on the Gaussian splashing algorithm provided in the above embodiments.

[0092] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0093] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0094] Finally, it should be noted that the above description is merely a preferred embodiment of the present application and the present application is not limited to the above embodiments. It is understood that other improvements and variations directly derived or conceived by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included within the scope of protection of the present application.

Claims

1. A 3D point cloud reconstruction method based on Gaussian splashing algorithm, characterized in that: The method comprises: Acquiring multiple images, wherein the multiple images are obtained by photographing a preset area at the same time by cameras located at different positions; Input multiple images into the pre-trained VGGT model to obtain DPT features based on the feature encoding output by the VIT encoder of the VGGT model, as well as the camera parameters and global point cloud data output by the VGGT model. The VGGT model is trained based on image samples and 3D point cloud samples. Based on the DPT features, the camera parameters and the global point cloud data, obtaining Gaussian splash parameters corresponding to each pixel position of each image; Based on the Gaussian sphere position increment in the Gaussian splash parameter and the local point cloud data in the camera coordinates, target three-dimensional point cloud data is obtained, wherein the local point cloud data is obtained based on the global point cloud data.

2. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to claim 1, characterized in that: Based on the DPT features, the camera parameters, and the global point cloud data, the Gaussian splatter parameters corresponding to each pixel position of each image are obtained, including: Converting the global point cloud data into the coordinate systems of each camera based on the camera parameters to obtain converted local point cloud data; Performing alignment processing on the local point cloud data so that the aligned local point cloud data is aligned with the DPT feature; Fusing the DPT features with the aligned local point cloud data to obtain fused point cloud data; Convolution interpolation processing is performed on the fused point cloud data to obtain the Gaussian splash parameters.

3. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to claim 2, characterized in that: Performing convolution interpolation processing on the fused point cloud data to obtain the Gaussian splash parameters includes: Inputting the fused point cloud data into the CNN network for convolution processing to obtain fused features; The fusion feature is upsampled using bilinear interpolation to obtain a target feature, and the Gaussian splash parameter is obtained based on the target feature.

4. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to claim 1, characterized in that: Based on the Gaussian sphere position increment in the Gaussian splash parameter and the local point cloud data in the camera coordinates, the target three-dimensional point cloud data is obtained, including: Determine a point cloud increment corresponding to a Gaussian sphere position increment in the Gaussian splash parameter; Calculating the point cloud sum of the local point cloud data and the point cloud increment in the camera coordinate system; A Gaussian rendering operation is performed on the point cloud to obtain the target three-dimensional point cloud data.

5. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to claim 1, characterized in that: The Gaussian splash parameters include transparency, Gaussian sphere position increment, Gaussian sphere deformation parameter, Gaussian sphere rotation parameter and spherical harmonic coefficient.

6. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to any one of claims 1 to 5, characterized in that: Before getting multiple images, also include: Acquire training sample data, wherein the training sample data includes: image samples and three-dimensional point cloud samples, wherein the image samples include: first image samples carrying camera parameters and second image samples not carrying camera parameters; The training sample data is input into the VGGT model. When it is determined that the image sample is the first image sample, the camera parameter prediction loss and the image reconstruction loss are calculated, and the model parameters of the VGGT model are optimized based on the camera parameter prediction loss and the image reconstruction loss. When it is determined that the image sample is the second image sample, the image reconstruction loss is calculated, and the model parameters are optimized based on the image reconstruction loss. The VGGT model training is determined to be completed when the number of iterations reaches a preset number.

7. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to claim 6, characterized in that: The VGGT model includes multiple Block blocks; Before training the VGGT model, it also includes: A Block is selected from the plurality of Block blocks to add supervision information, wherein the supervision information is used to supervise the VGGT model to predict camera parameters and predict global point cloud data.

8. The three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm according to claim 6, characterized in that: The first image sample includes: a rendered image obtained by rendering through application software, and the second image sample includes: a real image obtained by shooting with a camera.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the three-dimensional point cloud reconstruction method based on the Gaussian splash algorithm as described in any one of claims 1 to 8 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the three-dimensional point cloud reconstruction method based on the Gaussian splattering algorithm as described in any one of claims 1 to 8 are implemented.

Images