Three-dimensional reconstruction method, system and equipment based on satellite remote sensing image and medium

Abstract

The invention discloses a three-dimensional reconstruction method, system and device based on a satellite remote sensing image and a medium, and relates to the technical field of three-dimensional modeling, and the method comprises the steps: generating a digital surface model (DSM) of a target region according to the remote sensing image; generating a digital terrain model DTM according to the digital surface model DSM, and performing difference calculation on the digital surface model DSM and the digital terrain model DTM to obtain elevation information of the building; clustering roofs of the building according to the semantic segmentation map to obtain a corresponding instance segmentation map; performing polygon fitting on the building according to the instance segmentation map, and constructing a geometric model of the building in combination with elevation information; rendering the geometric model according to a preset PBR material model, and predicting material parameters of each point of the building in the geometric model according to a preset MLP model; and optimizing MLP parameters in the MLP model according to a pre-trained text graph diffusion model. The method has the effect of improving the accuracy and integrity of the building three-dimensional model.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional modeling, and in particular to a three-dimensional reconstruction method, system, equipment and medium based on satellite remote sensing images. Background Art

[0002] In current map query software, users can usually only see the front projection or partial side view of the building, but cannot obtain the complete three-dimensional model of the building. This limitation leads to a large difference between the appearance of the building displayed in the software and the actual situation, which affects the accuracy of users in navigation and positioning, and thus restricts the promotion and use of map query software.

[0003] In order to solve the problem of 3D modeling of buildings, the existing technologies mainly use traditional measurement methods and LiDAR methods. However, these two methods have their own defects in practical applications. Traditional measurement methods require a lot of manpower and material resources, the data acquisition process is long, the update cycle is long, and it is difficult to meet the needs of the rapidly changing urban environment. In contrast, although the LiDAR method can provide high-precision elevation data, it is limited to obtaining a single elevation value and lacks a comprehensive record of the building's appearance and ground object information, resulting in the generated 3D model being relatively poor in information.

[0004] Therefore, the current technology for generating three-dimensional building models has problems with insufficient accuracy and completeness, which in turn limits the improvement of user experience and the effect of practical applications. Summary of the invention

[0005] In order to improve the accuracy and completeness of a three-dimensional model of a building, the present application provides a three-dimensional reconstruction method, system, device and medium based on satellite remote sensing images.

[0006] In the first aspect, the present application provides a three-dimensional reconstruction method based on satellite remote sensing images, which adopts the following technical solutions:

[0007] The three-dimensional reconstruction method based on satellite remote sensing images includes:

[0008] Acquire a remote sensing image of a target area, and generate a digital surface model (DSM) of the target area according to the remote sensing image;

[0009] Generate a digital terrain model DTM according to the digital surface model DSM, and perform difference calculation between the digital surface model DSM and the digital terrain model DTM to obtain elevation information of the building;

[0010] Acquire a semantic segmentation map corresponding to the remote sensing image, and cluster the roofs of the buildings according to the semantic segmentation map to obtain a corresponding instance segmentation map;

[0011] Performing polygon fitting on the building according to the instance segmentation map, and constructing a geometric model of the building in combination with the elevation information;

[0012] Rendering the geometric model according to a preset PBR material model, and predicting material parameters of each point of the building in the geometric model according to a preset MLP model;

[0013] The MLP parameters in the pre-set MLP model are optimized by distillation fraction sampling SDS and according to the pre-trained Vincent graph diffusion model.

[0014] By adopting the above technical scheme, a remote sensing image of the target area is first obtained, and a digital surface model DSM of the target area is generated according to the remote sensing image, and then a digital terrain model DTM is generated according to the digital surface model DSM, and the difference calculation is performed between the digital surface model DSM and the digital terrain model DTM to obtain the elevation information of the building, and then the semantic segmentation map corresponding to the remote sensing image is obtained, and the roofs of the buildings are clustered according to the semantic segmentation map to obtain the corresponding instance segmentation map, and then the building is polygonally fitted according to the instance segmentation map, and the geometric model of the building is constructed in combination with the elevation information, and then the geometric model is rendered according to the preset PBR material model, and the material parameters of each point of the building in the geometric model are predicted according to the preset MLP model, and then the MLP parameters in the preset MLP model are optimized by the distillation fraction sampling SDS method and the pre-trained Vincent map diffusion model; the data in the geometric model of the building obtained by the above method is richer, and the geometric model of the building is closer to the appearance of the actual building, which improves the accuracy and completeness of the three-dimensional model of the building, thereby improving the user experience and the actual application effect.

[0015] Optionally, the step of optimizing the MLP parameters in the pre-set MLP model by distillation fraction sampling SDS and according to the pre-trained Vincent graph diffusion model includes:

[0016] For a building in the geometric model, randomly sample observation angles around the building, and render the image x under the observation angles;

[0017] The MLP parameter γ in the preset MLP model is updated by using the gradient calculation formula of the preset SDS loss function, and the gradient calculation formula of the preset SDS loss function is:

[0018]

[0019] in, is the pre-trained Wensheng graph diffusion model, φ is the pre-training parameter, and w(t) is the weight that changes with time t.

[0020] By adopting the above technical solution, in order to optimize the MLP parameters in the preset MLP model, for the buildings in the geometric model, observation perspectives are randomly sampled around the buildings, and the image x under the observation perspective is rendered, and then the MLP parameters γ in the preset MLP model are updated through the gradient calculation formula of the preset SDS loss function.

[0021] Optionally, the step of generating a digital surface model DSM of the target area according to the remote sensing image comprises:

[0022] Fine-tune the pre-set Depth Anything model through LoRA low-rank adaptation to obtain a fine-tuned Depth Anything model;

[0023] For a single remote sensing image, input the remote sensing image into the fine-tuned Depth Anything model to obtain first depth information of the target area;

[0024] For a common viewing area of ​​at least two remote sensing images, inputting at least two remote sensing images into the fine-tuned Depth Anything model to obtain at least two corresponding depth values;

[0025] An average value of the at least two depth values ​​is used as second depth information, and a digital surface model DSM of the target area is generated according to the first depth information and the second depth information.

[0026] By adopting the above technical solution, in order to generate a digital surface model DSM of the target area, the pre-set Depth Anything model is first fine-tuned by LoRA low-rank adaptation to obtain a fine-tuned Depth Anything model. For a single remote sensing image, the remote sensing image is input into the fine-tuned Depth Anything model to obtain the first depth information of the target area. For a common viewing area of ​​at least two remote sensing images, at least two remote sensing images are input into the fine-tuned Depth Anything model to obtain at least two corresponding depth values. Finally, the average of the at least two depth values ​​is used as the second depth information, and the digital surface model DSM of the target area is generated according to the first depth information and the second depth information.

[0027] Optionally, the fine-tuned Depth Anything model uses the Transformer encoder of Vision Transformer as a feature extraction module and uses the decoder part of DPT.

[0028] Optionally, the step of generating a digital terrain model DTM according to the digital surface model DSM comprises:

[0029] Using a preset number of evenly distributed scan lines to search for a minimum value on the digital surface model DSM within a predefined filter window, to obtain a ground point;

[0030] Interpolating the ground points to obtain an initial digital terrain model DTM;

[0031] Compare the initial digital terrain model DTM and the digital surface model DSM pixel by pixel to obtain ground pixels and non-ground pixels;

[0032] Non-ground pixels are deleted from the initial digital terrain model DTM, and holes in the initial digital terrain model DTM are filled to obtain a digital terrain model DTM.

[0033] By adopting the above technical scheme, in order to generate a digital terrain model DTM according to the digital surface model DSM, a preset number of evenly distributed scan lines are used to perform a minimum search on the digital surface model DSM within a predefined filter window to obtain ground points, and then the ground points are interpolated to obtain an initial digital terrain model DTM, and then the initial digital terrain model DTM and the digital surface model DSM are compared pixel by pixel to obtain ground pixels and non-ground pixels, and finally the non-ground pixels are deleted from the initial digital terrain model DTM, and the holes in the initial digital terrain model DTM are filled to obtain the digital terrain model DTM.

[0034] Optionally, the preset MLP model is:

[0035] (k d , k rm , k n )=μ(β(p);γ)

[0036] Among them, k d is the diffuse reflection parameter, k rm is the roughness and metalness, k n is the normal vector, β is the position encoding function, and γ is the MLP parameter.

[0037] Optionally, before the step of acquiring a remote sensing image of the target area and generating a digital surface model DSM of the target area according to the remote sensing image, the method further includes:

[0038] Acquire an initial remote sensing image of the target area, and preprocess the initial remote sensing image to obtain a corresponding remote sensing image;

[0039] Determining a target image from the remote sensing image, and semantically annotating the target image to obtain a corresponding semantically annotated image;

[0040] Training a pre-created semantic segmentation model according to the semantically annotated image to obtain a trained semantic segmentation model;

[0041] The remote sensing image is input into a trained semantic segmentation model to obtain a corresponding semantic segmentation map.

[0042] By adopting the above technical scheme, in order to realize the semantic segmentation of remote sensing images and generate corresponding semantic segmentation maps, the initial remote sensing image of the target area is first obtained, and the initial remote sensing image is preprocessed to obtain the corresponding remote sensing image, and then the target image is determined from the remote sensing image, and the target image is semantically annotated to obtain the corresponding semantically annotated image, and then the pre-created semantic segmentation model is trained according to the semantically annotated image to obtain the trained semantic segmentation model, and finally the remote sensing image is input into the trained semantic segmentation model to obtain the corresponding semantic segmentation map.

[0043] In the second aspect, the present application also provides a three-dimensional reconstruction system based on satellite remote sensing images, which adopts the following technical solutions:

[0044] The 3D reconstruction system based on satellite remote sensing images includes:

[0045] A DSM generation module is used to obtain a remote sensing image of a target area and generate a digital surface model DSM of the target area according to the remote sensing image;

[0046] An elevation information generating module is used to generate a digital terrain model DTM according to the digital surface model DSM, and perform difference calculation between the digital surface model DSM and the digital terrain model DTM to obtain elevation information of the building;

[0047] An instance segmentation module is used to obtain a semantic segmentation map corresponding to the remote sensing image, and cluster the roofs of the buildings according to the semantic segmentation map to obtain a corresponding instance segmentation map;

[0048] A geometric model generation module, used for performing polygon fitting on the building according to the instance segmentation map, and constructing a geometric model of the building in combination with the elevation information;

[0049] A material parameter generation module, used for rendering the geometric model according to a preset PBR material model, and predicting the material parameters of each point of the building in the geometric model according to a preset MLP model;

[0050] The MLP parameter optimization module is used to optimize the MLP parameters in the pre-set MLP model by means of distillation fraction sampling SDS and according to the pre-trained Vincent graph diffusion model.

[0051] In a third aspect, the present application also provides a computer device, which adopts the following technical solution:

[0052] A computer device comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method described in the first aspect when executing the computer program.

[0053] In a fourth aspect, the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0054] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the method described in the first aspect.

[0055] In summary, the present application includes at least the following beneficial technical effects: first, a remote sensing image of the target area is obtained, and a digital surface model DSM of the target area is generated according to the remote sensing image, then a digital terrain model DTM is generated according to the digital surface model DSM, and the digital surface model DSM and the digital terrain model DTM are differenced to obtain the elevation information of the building, then a semantic segmentation map corresponding to the remote sensing image is obtained, and the roofs of the buildings are clustered according to the semantic segmentation map to obtain the corresponding instance segmentation map, and then polygon fitting is performed on the building according to the instance segmentation map, and a geometric model of the building is constructed in combination with the elevation information, and then the geometric model is rendered according to a preset PBR material model, and the material parameters of each point of the building in the geometric model are predicted according to a preset MLP model, and then the MLP parameters in the preset MLP model are optimized by distillation fraction sampling SDS and according to the pre-trained Vincent map diffusion model; the data in the geometric model of the building obtained by the above method is richer, and the geometric model of the building is closer to the appearance of the actual building, which improves the accuracy and completeness of the three-dimensional model of the building, thereby improving the user experience and practical application effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the overall process of an embodiment of the present application.

[0057] Figure 2 It is a schematic diagram of the structure of the system of this application.

[0058] Figure 3 It is a structural block diagram of the computer device of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-3 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The embodiment of the present application discloses a three-dimensional reconstruction method based on satellite remote sensing images.

[0061] Reference Figure 1 , a three-dimensional reconstruction method based on satellite remote sensing images, including:

[0062] Step S11, acquiring a remote sensing image of the target area, and generating a digital surface model DSM of the target area according to the remote sensing image.

[0063] Step S12, generating a digital terrain model DTM according to the digital surface model DSM, and performing difference calculation between the digital surface model DSM and the digital terrain model DTM to obtain elevation information of the building.

[0064] Step S13, obtaining a semantic segmentation map corresponding to the remote sensing image, and clustering the roofs of the buildings according to the semantic segmentation map to obtain a corresponding instance segmentation map.

[0065] Step S14, polygon fitting is performed on the building according to the instance segmentation map, and a geometric model of the building is constructed in combination with the elevation information.

[0066] Step S15, rendering the geometric model according to the preset PBR material model, and predicting the material parameters of each point of the building in the geometric model according to the preset MLP model.

[0067] It can be understood that by introducing the PBR material model, the rendering of buildings can be made more realistic.

[0068] Step S16, optimizing the MLP parameters in the pre-set MLP model by means of distillation fraction sampling SDS and according to the pre-trained Vincent graph diffusion model.

[0069] In the above implementation, a remote sensing image of the target area is first obtained, and a digital surface model DSM of the target area is generated according to the remote sensing image, and then a digital terrain model DTM is generated according to the digital surface model DSM, and the difference calculation is performed between the digital surface model DSM and the digital terrain model DTM to obtain the elevation information of the building, and then a semantic segmentation map corresponding to the remote sensing image is obtained, and the roofs of the buildings are clustered according to the semantic segmentation map to obtain the corresponding instance segmentation map, and then polygon fitting is performed on the buildings according to the instance segmentation map, and a geometric model of the building is constructed in combination with the elevation information, and then the geometric model is rendered according to a preset PBR material model, and the material parameters of each point of the building in the geometric model are predicted according to a preset MLP model, and then the MLP parameters in the preset MLP model are optimized according to the pre-trained Vincent map diffusion model by using the distillation fraction sampling SDS method; the data in the geometric model of the building obtained by the above method is richer, and the geometric model of the building is closer to the appearance of the actual building, which improves the accuracy and completeness of the three-dimensional model of the building, thereby improving the user experience and the actual application effect.

[0070] As a further implementation of the three-dimensional reconstruction method, the step of optimizing the MLP parameters in the pre-set MLP model by distilling fractional sampling SDS and according to the pre-trained Vincent graph diffusion model includes:

[0071] Step S21 : for a building in the geometric model, randomly sample observation angles around the building, and render the image x under the observation angles.

[0072] Step S22, updating the MLP parameter γ in the preset MLP model by using the preset SDS loss function gradient calculation formula.

[0073] Among them, the gradient calculation formula of the preset SDS loss function is:

[0074]

[0075] in, is the pre-trained Wensheng graph diffusion model, φ is the pre-training parameter, and w(t) is the weight that changes with time t.

[0076] In the above embodiment, in order to optimize the MLP parameters in the preset MLP model, for the buildings in the geometric model, observation perspectives are randomly sampled around the buildings, and the image x under the observation perspective is rendered, and then the MLP parameters γ in the preset MLP model are updated through the gradient calculation formula of the preset SDS loss function.

[0077] As a further implementation of the three-dimensional reconstruction method, the step of generating a digital surface model DSM of the target area according to the remote sensing image includes:

[0078] Step S31, fine-tuning the preset Depth Anything model by LoRA low-rank adaptation to obtain a fine-tuned Depth Anything model.

[0079] Among them, the fine-tuned Depth Anything model uses the Transformer encoder of Vision Transformer as the feature extraction module and uses the decoder part of DPT.

[0080] It should be noted that in this embodiment, the LoRA low-rank adaptation technology is used to fine-tune the Depth Anything model based on the Depth Anything model to adapt the Depth Anything model to satellite remote sensing images. The Depth Anything model uses the Transformer encoder of the Vision Transformer as a feature extraction module and uses the decoder part of the DPT. During the working process of the Depth Anything model, the pre-trained weights of the Depth Anything model will be used and the weights will be frozen during the fine-tuning process. On this basis, a trainable low-rank matrix is ​​added to the pre-trained weights for learning, that is, for the pre-trained weight matrix The constraint update matrix is ​​a low-rank matrix, which is expressed as:

[0081] W0+ΔW=W0+BA

[0082] in, r<<min(d,k);

[0083] Finally, the Depth Anything model is fine-tuned on the remote sensing image with depth annotation to obtain the fine-tuned Depth Anything model.

[0084] Step S32: for a single remote sensing image, input the remote sensing image into the fine-tuned Depth Anything model to obtain first depth information of the target area.

[0085] Step S33: for the common viewing area of ​​at least two remote sensing images, input the at least two remote sensing images into the fine-tuned Depth Anything model to obtain at least two corresponding depth values.

[0086] Step S34: taking the average of the at least two depth values ​​as the second depth information, and generating a digital surface model DSM of the target area according to the first depth information and the second depth information.

[0087] In the above embodiment, in order to generate a digital surface model DSM of the target area, the pre-set Depth Anything model is first fine-tuned by LoRA low-rank adaptation to obtain a fine-tuned Depth Anything model. For a single remote sensing image, the remote sensing image is input into the fine-tuned Depth Anything model to obtain the first depth information of the target area. For a common viewing area of ​​at least two remote sensing images, at least two remote sensing images are input into the fine-tuned Depth Anything model to obtain at least two corresponding depth values. Finally, the average of the at least two depth values ​​is used as the second depth information, and the digital surface model DSM of the target area is generated according to the first depth information and the second depth information.

[0088] As a further implementation of the three-dimensional reconstruction method, the step of generating a digital terrain model DTM according to the digital surface model DSM includes:

[0089] Step S41 : Use a preset number of uniformly distributed scan lines to perform a minimum search on the digital surface model DSM within a predefined filter window to obtain ground points.

[0090] Step S42: interpolate the ground points to obtain an initial digital terrain model DTM.

[0091] Step S43, performing pixel-by-pixel comparison on the initial digital terrain model DTM and the digital surface model DSM to obtain ground pixels and non-ground pixels.

[0092] Step S44, deleting non-ground pixels from the initial digital terrain model DTM and filling the holes in the initial digital terrain model DTM to obtain the digital terrain model DTM.

[0093] In the above embodiment, in order to generate a digital terrain model DTM according to the digital surface model DSM, a preset number of evenly distributed scan lines are used to perform a minimum search on the digital surface model DSM within a predefined filter window to obtain ground points, and then the ground points are interpolated to obtain an initial digital terrain model DTM. Then, the initial digital terrain model DTM and the digital surface model DSM are compared pixel by pixel to obtain ground pixels and non-ground pixels. Finally, the non-ground pixels are deleted from the initial digital terrain model DTM, and the holes in the initial digital terrain model DTM are filled to obtain the digital terrain model DTM.

[0094] As a further implementation of the three-dimensional reconstruction method, the pre-set MLP model is:

[0095] (k d , k rm , kn )=μ(β(p);γ)

[0096] Among them, k d is the diffuse reflection parameter, k rm is the roughness and metalness, k n is the normal vector, β is the position encoding function, and γ is the MLP parameter.

[0097] As a further implementation of the three-dimensional reconstruction method, before the step of acquiring a remote sensing image of the target area and generating a digital surface model DSM of the target area according to the remote sensing image, the method further includes:

[0098] Step S51, acquiring an initial remote sensing image of the target area, and preprocessing the initial remote sensing image to obtain a corresponding remote sensing image.

[0099] It should be noted that, in this embodiment, a high-resolution satellite (such as WorldView, GeoEye, or Spot) is used to photograph the target area to obtain an initial remote sensing image of the target area; the preprocessing operations include geometric correction, atmospheric correction, and noise removal.

[0100] Step S52, determining a target image from the remote sensing image, and semantically annotating the target image to obtain a corresponding semantically annotated image.

[0101] It is understandable that the target image is usually a group of images in the remote sensing image; the semantic annotation method can be manual annotation, automatic annotation using machine learning and deep learning algorithms, or other annotation methods. The semantic labels corresponding to the semantic annotation include building roofs, building facades, building shadows, and backgrounds. Since building facades and building shadows are easily affected by occlusion (such as occlusion by trees), resulting in incomplete shapes, it is necessary to maintain the geometric regularity and integrity of the building during the annotation process so that the regularity of artificial buildings can be learned in the subsequent network or model.

[0102] Step S53: training the pre-created semantic segmentation model according to the semantically annotated image to obtain a trained semantic segmentation model.

[0103] It should be noted that the semantic segmentation model of this embodiment is designed based on U-Net, and its encoding layer is replaced by a Transformer-based feature encoding module to enhance the feature extraction capability of remote sensing images, and the enhanced U-Net (i.e., the semantic segmentation model) is trained according to the semantically annotated images, and the training loss function uses a pixel-by-pixel softmax function.

[0104] Step S54, inputting the remote sensing image into the trained semantic segmentation model to obtain a corresponding semantic segmentation map.

[0105] In the above-mentioned embodiment, in order to realize the semantic segmentation of the remote sensing image and generate the corresponding semantic segmentation map, the initial remote sensing image of the target area is first obtained, and the initial remote sensing image is preprocessed to obtain the corresponding remote sensing image, and then the target image is determined from the remote sensing image, and the target image is semantically annotated to obtain the corresponding semantically annotated image, and then the pre-created semantic segmentation model is trained according to the semantically annotated image to obtain the trained semantic segmentation model, and finally the remote sensing image is input into the trained semantic segmentation model to obtain the corresponding semantic segmentation map.

[0106] The embodiment of the present application also discloses a three-dimensional reconstruction system based on satellite remote sensing images.

[0107] refer to Figure 2 , a 3D reconstruction system based on satellite remote sensing images, including:

[0108] A DSM generation module is used to obtain a remote sensing image of a target area and generate a digital surface model DSM of the target area according to the remote sensing image;

[0109] The elevation information generation module is used to generate a digital terrain model DTM according to the digital surface model DSM, and perform difference calculation between the digital surface model DSM and the digital terrain model DTM to obtain the elevation information of the building;

[0110] The instance segmentation module is used to obtain the semantic segmentation map corresponding to the remote sensing image, and cluster the roofs of the buildings according to the semantic segmentation map to obtain the corresponding instance segmentation map;

[0111] The geometric model generation module is used to perform polygon fitting on the building according to the instance segmentation map and construct the geometric model of the building in combination with the elevation information;

[0112] A material parameter generation module is used to render the geometric model according to a preset PBR material model and predict the material parameters of each point of the building in the geometric model according to a preset MLP model;

[0113] The MLP parameter optimization module is used to optimize the MLP parameters in the pre-set MLP model by sampling the distillation fraction SDS according to the pre-trained Vincent graph diffusion model.

[0114] The three-dimensional reconstruction system based on satellite remote sensing images of the present invention can implement any one of the three-dimensional reconstruction methods based on satellite remote sensing images, and the specific working process of the three-dimensional reconstruction system based on satellite remote sensing images of the present invention can refer to the corresponding process in the above-mentioned three-dimensional reconstruction method based on satellite remote sensing images.

[0115] The embodiment of the present application also discloses a computer device.

[0116] refer to Figure 3 A computer device includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, any of the above-mentioned three-dimensional reconstruction methods based on satellite remote sensing images is implemented.

[0117] The embodiment of the present application also discloses a computer-readable storage medium.

[0118] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any one of the above three-dimensional reconstruction methods based on satellite remote sensing images.

[0119] Among them, computer-readable storage media can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0120] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A three-dimensional reconstruction method based on satellite remote sensing images, characterized in that: include: Acquire a remote sensing image of a target area, and generate a digital surface model (DSM) of the target area according to the remote sensing image; Generate a digital terrain model DTM according to the digital surface model DSM, and perform difference calculation between the digital surface model DSM and the digital terrain model DTM to obtain elevation information of the building; Acquire a semantic segmentation map corresponding to the remote sensing image, and cluster the roofs of the buildings according to the semantic segmentation map to obtain a corresponding instance segmentation map; Performing polygon fitting on the building according to the instance segmentation map, and constructing a geometric model of the building in combination with the elevation information; Rendering the geometric model according to a preset PBR material model, and predicting material parameters of each point of the building in the geometric model according to a preset MLP model; The MLP parameters in the pre-set MLP model are optimized by distillation fraction sampling SDS and according to the pre-trained Vincent graph diffusion model.

2. The three-dimensional reconstruction method based on satellite remote sensing images according to claim 1, characterized in that: The step of optimizing the MLP parameters in the pre-set MLP model by using the distillation fraction sampling SDS method and according to the pre-trained Vincent graph diffusion model comprises: For a building in the geometric model, randomly sample observation angles around the building, and render the image x under the observation angles; The MLP parameter γ in the preset MLP model is updated by using the gradient calculation formula of the preset SDS loss function, and the gradient calculation formula of the preset SDS loss function is: in, is the pre-trained Wensheng graph diffusion model, φ is the pre-training parameter, and w(t) is the weight that changes with time t.

3. The three-dimensional reconstruction method based on satellite remote sensing images according to claim 1, characterized in that: The step of generating a digital surface model DSM of the target area according to the remote sensing image comprises: Fine-tune the pre-set Depth Anything model through LoRA low-rank adaptation to obtain a fine-tuned Depth Anything model; For a single remote sensing image, input the remote sensing image into the fine-tuned Depth Anything model to obtain first depth information of the target area; For a common viewing area of ​​at least two remote sensing images, inputting at least two remote sensing images into the fine-tuned DepthAnything model to obtain at least two corresponding depth values; An average value of the at least two depth values ​​is used as second depth information, and a digital surface model DSM of the target area is generated according to the first depth information and the second depth information.

4. The three-dimensional reconstruction method based on satellite remote sensing images according to claim 3 is characterized in that: The fine-tuned Depth Anything model uses the Transformer encoder of the Vision Transformer as the feature extraction module and the decoder part of the DPT.

5. The three-dimensional reconstruction method based on satellite remote sensing images according to claim 1, characterized in that: The step of generating a digital terrain model DTM according to the digital surface model DSM comprises: Using a preset number of evenly distributed scan lines to search for a minimum value on the digital surface model DSM within a predefined filter window, to obtain a ground point; Interpolating the ground points to obtain an initial digital terrain model DTM; Compare the initial digital terrain model DTM and the digital surface model DSM pixel by pixel to obtain ground pixels and non-ground pixels; Non-ground pixels are deleted from the initial digital terrain model DTM, and holes in the initial digital terrain model DTM are filled to obtain a digital terrain model DTM.

6. The three-dimensional reconstruction method based on satellite remote sensing images according to claim 1, characterized in that: The pre-set MLP model is: (k d ,k rm ,k n )=T(β(p);γ) Among them, k d is the diffuse reflection parameter, k rm is the roughness and metalness, k n is the normal vector, β is the position encoding function, and γ is the MLP parameter.

7. The three-dimensional reconstruction method based on satellite remote sensing images according to claim 1, characterized in that: Before the step of acquiring a remote sensing image of the target area and generating a digital surface model DSM of the target area according to the remote sensing image, the method further includes: Acquire an initial remote sensing image of the target area, and preprocess the initial remote sensing image to obtain a corresponding remote sensing image; Determining a target image from the remote sensing image, and semantically annotating the target image to obtain a corresponding semantically annotated image; Training a pre-created semantic segmentation model according to the semantically annotated image to obtain a trained semantic segmentation model; The remote sensing image is input into a trained semantic segmentation model to obtain a corresponding semantic segmentation map.

8. A three-dimensional reconstruction system based on satellite remote sensing images, characterized in that: include: A DSM generation module is used to obtain a remote sensing image of a target area and generate a digital surface model DSM of the target area according to the remote sensing image; An elevation information generating module is used to generate a digital terrain model DTM according to the digital surface model DSM, and perform difference calculation between the digital surface model DSM and the digital terrain model DTM to obtain elevation information of the building; An instance segmentation module is used to obtain a semantic segmentation map corresponding to the remote sensing image, and cluster the roofs of the buildings according to the semantic segmentation map to obtain a corresponding instance segmentation map; A geometric model generation module, used for performing polygon fitting on the building according to the instance segmentation map, and constructing a geometric model of the building in combination with the elevation information; A material parameter generation module, used for rendering the geometric model according to a preset PBR material model, and predicting the material parameters of each point of the building in the geometric model according to a preset MLP model; The MLP parameter optimization module is used to optimize the MLP parameters in the pre-set MLP model by means of distillation fraction sampling SDS and according to the pre-trained Vincent graph diffusion model.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.

Images