Method, device and readable storage medium for reconstructing dense three-dimensional model

Abstract

The present invention discloses a method, device and readable storage medium for reconstructing a dense three-dimensional model. The method for reconstructing a dense three-dimensional model includes the following steps: S110, obtaining an input image, and extracting a multi-scale feature map in the input image; S120, performing a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding feature body; S130, performing multi-level depth estimation on the feature body to obtain a target depth map of the same size as the input image; and S140, fusing the target depth map into a point cloud to reconstruct a dense three-dimensional model of the input image. The technical solution of the present invention can reconstruct a dense three-dimensional model of an input image, and at the same time has the advantages of reducing the complexity of image calculation, requiring less GPU resources, saving costs, having a simple structure, and having high reconstruction efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, electronic device and readable storage medium for reconstructing a dense three-dimensional model. Background Art

[0002] Accurate three-dimensional (3D) city models are essential for digital cities. In recent years, large-scale 3D model reconstruction in remote sensing has been intensively studied. Among the many methods reported in the literature, multi-view stereo (MVS) is one of the most promising methods. In MVS, a 3D model of a scene is reconstructed from a set of images captured from different viewpoints by a camera with known camera parameters. The above-mentioned MVS is capable of reconstructing small objects and large-scale outdoor scenes. For situations with scarce textures, repeated textures, or varying illumination, conventional MVS often fails to obtain accurate reconstruction results. In addition, MVS has high computational requirements. Therefore, it usually takes a long time for MVS to establish the corresponding 3D relationships. This problem is further exacerbated when reconstructing large-scale scenes.

[0003] Driven by the rapid development of machine learning technology, deep learning-based methods have achieved remarkable results in 3D model reconstruction of small objects by mining the features and structures of image data. The success of small 3D model reconstruction indicates that learning-based MVS can potentially be extended to 3D reconstruction in large scenes. The existing technology adopts the approach of first using a feature extraction model to obtain the depth features of the image, and then merging these features to construct the cost volume of the depth map. In this process, MVSNet was proposed to inherit the theoretical basis of stereo matching, while reducing the complexity of the problem by performing depth estimation on only one image at a time. Therefore, MVSNet can effectively improve the impact of the illumination change problem and greatly improve the reconstruction performance in terms of accuracy and generalization. More specifically, images acquired by sensors with known intrinsic parameters can be converted into cost volumes after differentiable homography warping. Afterwards, a 3D convolution operation is performed on the cost volume before generating a depth map for each image by averaging the weighted depth output. Therefore, the correct depth information can be selected for depth prediction by exploiting the photometric consistency between multiple images to obtain a 3D point cloud. Despite their many advantages, these existing MVS methods are limited by their heavy computational requirements and memory consumption. Therefore, it is not easy to actually deploy these methods in large-scale scenarios, especially when using high-resolution image acquisition devices.

[0004] In view of this, it is necessary to propose further improvements to the current techniques for reconstructing dense 3D models. Summary of the invention

[0005] In order to solve at least one of the above technical problems, the main purpose of the present invention is to provide a method, device, electronic device and readable storage medium for reconstructing a dense three-dimensional model.

[0006] To achieve the above object, the first technical solution adopted by the present invention is to provide a method for reconstructing a dense three-dimensional model, comprising:

[0007] Acquire an input image, and extract a multi-scale feature map from the input image;

[0008] Performing a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding feature volume;

[0009] Performing multi-level depth estimation on the feature body to obtain a target depth map having the same size as the input image; and

[0010] The target depth maps are fused into a point cloud to reconstruct a dense three-dimensional model of the input image.

[0011] The multi-scale feature map is obtained by processing a dual U-Net network or a pyramid feature extraction network with an attention mechanism, the multi-scale feature map includes a first group of feature maps, a second group of feature maps, and a third group of feature maps, and extracting the multi-scale feature map in the input image includes:

[0012] Performing feature extraction on the input image to obtain a first set of feature maps of the same size as the input image;

[0013] Performing a first maximum pooling and continuous convolution operation on the input image to increase the number of channels from 3 to 8, thereby obtaining a second set of feature maps having a size of 1 / 4 times that of the input image;

[0014] A second maximum pooling and continuous convolution operation is performed on the input image to increase the number of channels from 8 to 16, and a third set of feature maps with a size of 1 / 16 times the input image are extracted.

[0015] The performing of a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map of the input image into a corresponding feature body includes:

[0016] Performing a differentiable homography transformation on the multi-scale feature map to generate a corresponding plurality of feature quantities;

[0017] Multiple feature quantities of the multi-scale feature map are respectively distorted to the hypothetical plane of the reference camera to obtain a first feature volume, a second feature volume and a third feature volume of different sizes.

[0018] The step of performing multi-level depth estimation on the feature body to obtain a target depth map having the same size as the input image includes:

[0019] The feature information of the first feature body, the second feature body and the third feature body are sliced ​​and fused in the channel dimension to obtain a target depth map.

[0020] The step of slicing and fusing the feature information of the first feature body, the second feature body, and the third feature body in the channel dimension to obtain a target depth map includes:

[0021] Performing depth estimation on the third feature body to obtain a first depth map;

[0022] Upsampling the first depth map and accumulating it with the second set of feature maps, and performing depth estimation on the accumulated feature images to obtain a second depth map;

[0023] Upsampling the second depth map and accumulating it with the first set of feature maps, and performing depth estimation on the accumulated feature images to obtain a third depth map;

[0024] The third depth map is upsampled to obtain a target depth map having the same size as the input image.

[0025] To achieve the above object, the second technical solution adopted by the present invention is: to provide a device for reconstructing a dense three-dimensional model, comprising:

[0026] A feature extraction module, used to obtain an input image and extract a multi-scale feature map in the input image;

[0027] A transformation module, configured to perform a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding reference image;

[0028] a depth estimation module, configured to perform multi-level depth estimation on the reference image to obtain a target depth map having the same size as the input image; and

[0029] The point cloud fusion module is used to fuse the target depth map into a point cloud to reconstruct a dense three-dimensional model of the input image.

[0030] The multi-scale feature map is obtained by processing a dual U-Net network or a pyramid feature extraction network with an attention mechanism, the multi-scale feature map includes a first group of feature maps, a second group of feature maps and a third group of feature maps, and the extraction module is used for:

[0031] Performing feature extraction on the input image to obtain a first set of feature maps of the same size as the input image;

[0032] Performing a first maximum pooling and continuous convolution operation on the input image to increase the number of channels from 3 to 8, thereby obtaining a second set of feature maps having a size of 1 / 4 times that of the input image;

[0033] A second maximum pooling and continuous convolution operation is performed on the input image to increase the number of channels from 8 to 16, and a third set of feature maps with a size of 1 / 16 times the input image are extracted.

[0034] Wherein, the transformation module is used for:

[0035] Performing a differentiable homography transformation on the multi-scale feature map to generate a corresponding plurality of feature quantities;

[0036] Multiple feature quantities of the multi-scale feature map are respectively distorted to the hypothetical plane of the reference camera to obtain a first feature volume, a second feature volume and a third feature volume of different sizes.

[0037] To achieve the above-mentioned purpose, the third technical solution adopted by the present invention is: to provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps in the above-mentioned method are implemented.

[0038] To achieve the above objectives, the fourth technical solution adopted by the present invention is: providing a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are implemented.

[0039] The technical solution of the present invention mainly adopts the steps of first acquiring an input image and extracting a multi-scale feature map in the input image, then performing a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding feature body, and then performing multi-level depth estimation on the feature body to obtain a target depth map of the same size as the input image, and finally fusing the target depth map into a point cloud to reconstruct a dense three-dimensional model of the input image. This method can reduce the complexity of image calculation and has the advantages of requiring less GPU resources, saving costs, having a simple structure, and having high reconstruction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0041] Figure 1 A flowchart of a method for reconstructing a dense three-dimensional model according to a first embodiment of the present invention;

[0042] Figure 2 A specific flow chart of a method for reconstructing a dense three-dimensional model according to a second embodiment of the present invention;

[0043] Figure 3 A module block diagram of an apparatus for reconstructing a dense three-dimensional model according to a third embodiment of the present invention;

[0044] Figure 4 FIG. 4 is a module block diagram of an electronic device according to a fourth embodiment of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that the descriptions of "first", "second", etc. in the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0048] In order to solve the problem of heavy computational requirements and memory consumption in the related art of reconstructing dense three-dimensional models using the MVS (Multiple Video Stereo) method, the present invention provides a method for reconstructing dense three-dimensional models, which is aimed at reconstructing 3D (three dimensional) models in digital cities, and has the advantages of requiring less GPU resources, saving costs, simple structure, and high reconstruction efficiency. Please refer to the following embodiments for the specific steps of the method for reconstructing dense three-dimensional models.

[0049] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for reconstructing a dense three-dimensional model according to a first embodiment of the present invention. In an embodiment of the present invention, the method for reconstructing a dense three-dimensional model is applied to the reconstruction of a 3D model of a digital city. The reconstruction of a dense three-dimensional model specifically includes the following steps:

[0050] S110: Obtain an input image, and extract a multi-scale feature map from the input image.

[0051] Feature extraction plays an important role in multi-view stereo matching reconstruction, because the reconstruction performance depends largely on the effectiveness of feature extraction. For this solution, the multi-scale feature map is processed by a dual U-Net network or a pyramid feature extraction network with an attention mechanism. The dual U-Net network increases the local receptive field of high-frequency areas with rich texture information to obtain better feature information. The dual U-Net network can retain more detailed features, thereby achieving more accurate depth estimation. The above multi-scale feature maps are three feature maps of different sizes from the original image, and the sizes of the three feature maps can be set according to actual requirements.

[0052] S120, performing a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding feature volume.

[0053] After feature extraction forms a multi-scale feature map, a differentiable homography is performed on the feature map to generate multiple feature volumes. The hierarchical approach to developing the cost volume is more memory efficient than building the cost volume on the entire image. Differentiable homography is then applied to warp the 2D feature map into the hypothetical plane of the reference camera to form a feature volume. The multi-scale feature map has corresponding multiple feature volumes.

[0054] S130 , performing multi-level depth estimation on the feature body to obtain a target depth map with the same size as the input image.

[0055] Multiple feature bodies are fused, and multi-level depth estimation is performed on the fused feature bodies using upsampling to obtain a target depth map with the same size as the input image.

[0056] S140, fusing the target depth map into a point cloud to reconstruct a dense three-dimensional model of the input image.

[0057] Each target depth map has a depth value, and the feature information in the target depth map can be projected into three-dimensional space through known coordinates to form a point cloud to reconstruct a dense three-dimensional model of the input image.

[0058] Furthermore, the multi-scale feature map includes a first group of feature maps, a second group of feature maps, and a third group of feature maps, and extracting the multi-scale feature map from the input image includes:

[0059] Perform feature extraction on the input image to obtain a first set of feature maps with the same size as the input image;

[0060] Perform the first maximum pooling and continuous convolution operations on the input image to increase the number of channels from 3 to 8, and obtain a second set of feature maps with a size of 1 / 4 times that of the input image;

[0061] A second maximum pooling and continuous convolution operations are performed on the input image to increase the number of channels from 8 to 16, and a third set of feature maps with a size of 1 / 16 times the input image are extracted.

[0062] Specifically, given an input image of size 512*640, two maximum pooling and continuous convolutions are performed to increase the number of channels from 3 to 8 and then to 32, thereby obtaining feature maps of size 1, 1 / 4, and 1 / 16 of the input image. Among them, the first set of feature maps has the same size as the input image, the second set of feature maps has a size equal to 1 / 4 of the input image, and the third set of feature maps has a size equal to 1 / 16 of the input image.

[0063] Furthermore, a differentiable homography transformation operation is performed on the multi-scale feature map to deform the multi-scale feature map of the input image into a corresponding feature body, including:

[0064] Performing a differentiable homography transformation on the multi-scale feature map to generate corresponding multiple feature quantities;

[0065] Multiple feature quantities of the multi-scale feature map are respectively distorted to the hypothetical plane of the reference camera to obtain a first feature volume, a second feature volume and a third feature volume of different sizes.

[0066] Let d denote the depth hypothesis, and use d to project all pixels of the source image onto the space, and then inversely warp these projected points by the reference camera. Let Hi(d) be the homography between the feature map of the i-th view and the reference feature map at depth d, then

[0067]

[0068] Among them, Ki, Ri, ti represent the camera intrinsic matrix function, rotation parameters and translation parameters of the i-th view respectively, and n1 represents the principal axis of the reference camera. Next, the differentiable homography is applied to warp the 2D feature map into the hypothetical plane of the reference camera to form the corresponding feature volume. The above-mentioned feature volume includes the first feature volume, the second feature volume and the third feature volume.

[0069] Furthermore, multi-level depth estimation is performed on the feature body to obtain a target depth map with the same size as the input image, including:

[0070] The feature information of the first feature body, the second feature body and the third feature body are sliced ​​and fused in the channel dimension to obtain a target depth map.

[0071] Specifically, the feature information of the first feature body, the second feature body, and the third feature body are sliced ​​and fused in the channel dimension to obtain a target depth map, including:

[0072] Performing depth estimation on the third feature body to obtain a first depth map;

[0073] Upsampling the first depth map and accumulating it with the second set of feature maps, and performing depth estimation on the accumulated feature images to obtain a second depth map;

[0074] Upsampling the second depth map and accumulating it with the first set of feature maps, and performing depth estimation on the accumulated feature images to obtain a third depth map;

[0075] The third depth map is upsampled to obtain a target depth map of the same size as the input image. The above upsampling process is designed to preserve the details and spatial dimensions of the object, while accumulation (side connection) can better repair the details of the target. The side connection is to accumulate the features of the previous layer with the features of this layer.

[0076] Please refer to Figure 3 , Figure 3 This is a block diagram of a module of a device for reconstructing a dense three-dimensional model according to a third embodiment of the present invention. In an embodiment of the present invention, the device for reconstructing a dense three-dimensional model comprises:

[0077] The feature extraction module 110 is used to obtain an input image and extract a multi-scale feature map in the input image;

[0078] A transformation module 120, configured to perform a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding reference image;

[0079] A depth estimation module 130 is used to perform multi-level depth estimation on the reference image to obtain a target depth map of the same size as the input image; and

[0080] The point cloud fusion module 140 is used to fuse the target depth map into a point cloud to reconstruct a dense three-dimensional model of the input image.

[0081] The multi-scale feature map is obtained by processing a dual U-Net network or a pyramid feature extraction network with an attention mechanism. The multi-scale feature map includes a first group of feature maps, a second group of feature maps, and a third group of feature maps. The extraction module is used for:

[0082] Perform feature extraction on the input image to obtain a first set of feature maps with the same size as the input image;

[0083] Perform the first maximum pooling and continuous convolution operations on the input image to increase the number of channels from 3 to 8, and obtain a second set of feature maps with a size of 1 / 4 times that of the input image;

[0084] A second maximum pooling and continuous convolution operations are performed on the input image to increase the number of channels from 8 to 16, and a third set of feature maps with a size of 1 / 16 times the input image are extracted.

[0085] The transformation module 120 is used for:

[0086] Performing a differentiable homography transformation on the multi-scale feature map to generate corresponding multiple feature quantities;

[0087] Multiple feature quantities of the multi-scale feature map are respectively distorted to the hypothetical plane of the reference camera to obtain a first feature volume, a second feature volume and a third feature volume of different sizes.

[0088] Wherein, the depth estimation module 130 is used to:

[0089] The feature information of the first feature body, the second feature body and the third feature body are sliced ​​and fused in the channel dimension to obtain a target depth map.

[0090] The depth estimation module 130 is specifically used for:

[0091] Performing depth estimation on the third feature body to obtain a first depth map;

[0092] Upsampling the first depth map and accumulating it with the second set of feature maps, and performing depth estimation on the accumulated feature images to obtain a second depth map;

[0093] Upsampling the second depth map and accumulating it with the first set of feature maps, and performing depth estimation on the accumulated feature images to obtain a third depth map;

[0094] The third depth map is upsampled to obtain a target depth map having the same size as the input image.

[0095] See also Figure 4 , Figure 4 FIG. 4 is a block diagram of a fourth embodiment of an electronic device of the present invention. The electronic device can be used to implement the method for reconstructing a dense three-dimensional model in the above-mentioned embodiment. Figure 4 As shown, the electronic device mainly includes: a memory 301, a processor 302, a bus 303, and a computer program stored in the memory 301 and executable on the processor 302, and the memory 301 and the processor 302 are connected via the bus 303. When the processor 302 executes the computer program, the method for reconstructing a dense three-dimensional model in the aforementioned embodiment is implemented. The number of processors can be one or more.

[0096] The memory 301 may be a high-speed random access memory (RAM) memory, or a non-volatile memory, such as a disk memory. The memory 301 is used to store executable program codes, and the processor 302 is coupled to the memory 301 .

[0097] Furthermore, an embodiment of the present invention further provides a readable storage medium, which may be disposed in the electronic device in the above embodiments. The readable storage medium may be the above Figure 4 Memory in the illustrated embodiment.

[0098] The readable storage medium stores a computer program, and when the program is executed by the processor, the method for reconstructing a dense three-dimensional model in the aforementioned embodiment is implemented. Furthermore, the computer storable medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0099] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0100] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.

[0102] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk and other media that can store program codes.

[0103] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0104] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0105] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings under the technical solution of the present invention, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A method for reconstructing a dense three-dimensional model, characterized in that: The method for reconstructing a dense three-dimensional model comprises: An input image is obtained, and a multi-scale feature map in the input image is extracted, wherein the multi-scale feature map is obtained by processing a dual U-Net network or a pyramid feature extraction network with an attention mechanism, and the multi-scale feature map includes a first group of feature maps, a second group of feature maps, and a third group of feature maps, and extracting the multi-scale feature map in the input image includes: Performing feature extraction on the input image to obtain a first set of feature maps of the same size as the input image; Performing a first maximum pooling and continuous convolution operation on the input image to increase the number of channels from 3 to 8, thereby obtaining a second set of feature maps having a size of 1 / 4 times that of the input image; Performing a second maximum pooling and continuous convolution operation on the input image to increase the number of channels from 8 to 16, and extracting a third set of feature maps with a size of 1 / 16 times the input image; Performing a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding feature body, including: performing a differentiable homography transformation on the multi-scale feature map to generate a corresponding plurality of feature quantities; Warping multiple feature quantities of the multi-scale feature map to the hypothetical plane of the reference camera respectively to obtain a first feature volume, a second feature volume and a third feature volume of different sizes; Performing multi-level depth estimation on the feature body to obtain a target depth map having the same size as the input image, including: slicing and fusing feature information of the first feature body, the second feature body, and the third feature body in a channel dimension to obtain a target depth map; and The target depth maps are fused into a point cloud to reconstruct a dense three-dimensional model of the input image.

2. The method for reconstructing a dense three-dimensional model according to claim 1, characterized in that: The step of slicing and fusing the feature information of the first feature body, the second feature body, and the third feature body in the channel dimension to obtain a target depth map includes: Performing depth estimation on the third feature body to obtain a first depth map; Upsampling the first depth map and accumulating it with the second set of feature maps, and performing depth estimation on the accumulated feature images to obtain a second depth map; Upsampling the second depth map and accumulating it with the first set of feature maps, and performing depth estimation on the accumulated feature images to obtain a third depth map; The third depth map is upsampled to obtain a target depth map having the same size as the input image.

3. A device for reconstructing a dense three-dimensional model, characterized in that: The device for reconstructing a dense three-dimensional model comprises: A feature extraction module is used to obtain an input image and extract a multi-scale feature map in the input image, wherein the multi-scale feature map is obtained by processing a dual U-Net network or a pyramid feature extraction network with an attention mechanism, wherein the multi-scale feature map includes a first group of feature maps, a second group of feature maps, and a third group of feature maps. The feature extraction module is used to: Performing feature extraction on the input image to obtain a first set of feature maps of the same size as the input image; Performing a first maximum pooling and continuous convolution operation on the input image to increase the number of channels from 3 to 8, thereby obtaining a second set of feature maps having a size of 1 / 4 times that of the input image; Performing a second maximum pooling and continuous convolution operation on the input image to increase the number of channels from 8 to 16, and extracting a third set of feature maps with a size of 1 / 16 times the input image; A transformation module is used to perform a differentiable homography transformation operation on the multi-scale feature map to deform the multi-scale feature map into a corresponding reference image, and the transformation module is used to: Performing a differentiable homography transformation on the multi-scale feature map to generate a corresponding plurality of feature quantities; Warping multiple feature quantities of the multi-scale feature map to the hypothetical plane of the reference camera respectively to obtain a first feature body, a second feature body and a third feature body of different sizes; a depth estimation module, configured to perform multi-level depth estimation on the reference image to obtain a target depth map having the same size as the input image, wherein the depth estimation module is configured to: slice and fuse feature information of the first feature body, the second feature body, and the third feature body in a channel dimension to obtain a target depth map; and The point cloud fusion module is used to fuse the target depth map into a point cloud to reconstruct a dense three-dimensional model of the input image.

4. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to claim 1 or 2 when executing the computer program.

5. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method of claim 1 or 2 are implemented.

Images