2D satellite remote sensing image building resolution improving method based on unmanned aerial vehicle 3D building model

Abstract

The invention relates to a 2D satellite remote sensing image building resolution improving method based on an unmanned aerial vehicle 3D building model, and the method specifically comprises the steps: carrying out the preprocessing of a satellite remote sensing image in a target region, obtaining the boundary information of a building in the satellite remote sensing image in the target region, and carrying out the orthographic projection of the boundary information to the surface of an earth model; performing feature extraction on the building roof in the satellite remote sensing image; calculating an inclination angle alpha of the satellite remote sensing image; performing 3D unmanned aerial vehicle remote sensing image shooting on the building in the target area to obtain a 3D model of the building in the target area; and selecting an image with the same angle as alpha in the 3D model of the building, extracting building roof features in the image, performing 2D projection on the building according to the building roof, and fusing the projection with the projection of the satellite remote sensing image, thereby realizing the improvement of the image resolution of the target area, and realizing the improvement of the satellite remote sensing data evaluation result.

Description

Technical Field

[0001] The invention relates to a method for improving the building resolution of a 2D satellite remote sensing image based on a 3D building model of an unmanned aerial vehicle, and belongs to the technical field of remote sensing image processing. Background Art

[0002] In the field of distributed rooftop photovoltaic extraction, accurate extraction of building roof information is a very important step. However, due to the limitation of satellite resolution by existing satellite technology, the general resolution of public civil data is only 0.5 meters. For a 100-square-meter building, such as a 10-meter × 10-meter square building, if it is represented by pixels, it is only 20 pixels × 20 pixels. Assuming that there is an error in the pixel value when extracting the building roof, the area of ​​the building we identify may change from 90.25 square meters to 110.25 square meters, which is a relatively large error. For drones, the current drone data can reach an accuracy of 0.1 meters. When identifying a 100-square-meter building area, the monitoring area is calculated to be 98.01 square meters to 101 square meters according to the error of 1 pixel. It can be seen that although satellite remote sensing can realize the extraction of building roof area under economic conditions, the accuracy is poor; on the contrary, drone data can achieve high-precision identification, but the price is too expensive to be used on a large scale. Summary of the invention

[0003] In order to solve the problems existing in the prior art, the present invention provides a method for fusing unmanned aerial vehicle image features with satellite remote sensing image features to thereby improve image resolution.

[0004] In order to achieve the above object, the technical solution proposed in the present invention is: a method for improving the building resolution of a 2D satellite remote sensing image based on a UAV 3D building model, comprising the following steps:

[0005] Step 1: preprocess the satellite remote sensing image in the target area, obtain the building boundary information in the satellite remote sensing image in the target area, and orthographically project the boundary information onto the surface of the earth model;

[0006] Step 2: Refine the roof information of the projected building and extract features of the building roof in the satellite remote sensing image;

[0007] Step 3, calculating the tilt angle α of the satellite remote sensing image according to the time when the satellite remote sensing image was taken and the altitude information of the satellite;

[0008] Step 4: Take 3D drone remote sensing images of the buildings in the target area to obtain 3D models of the buildings in the target area;

[0009] Step 5: Select an image with the same angle as α in the 3D model of the building, extract the building roof features in the image, perform 2D projection of the building based on the building roof, and fuse it with the projection of the satellite remote sensing image to improve the image resolution of the target area.

[0010] A further design of the above technical solution is as follows: the preprocessing in step 1 includes radiation correction, geometric correction, orthorectification, mosaicking and cutting.

[0011] In step 1, the building boundary information is obtained by using semantic segmentation technology.

[0012] The inclination angle α is calculated as follows:

[0013] α=arctan(h / L)

[0014] Where h is the altitude of the satellite, and L is the distance from the satellite sub-satellite point to the end point of the building's shadow when shooting.

[0015] The step 4 is specifically as follows:

[0016] Use GIS software or drone planning software to set the drone’s flight altitude, speed, and overlap rate;

[0017] Accurately calibrate the laser scanner on the drone, including time synchronization, angle calibration, and distance calibration;

[0018] According to the planned flight route, the drone is started to collect laser point cloud data. During the flight of the drone, the laser scanner will emit laser pulses to the ground and receive the reflected signals. By calculating the time difference and angle information, point cloud data in three-dimensional space is generated;

[0019] The point cloud data collected from different flights are spliced ​​together to form a complete point cloud model of the building.

[0020] The point cloud data collected by the UAV is filtered to remove noise points in the point cloud data.

[0021] Through the FPN network and SNAKE algorithm, the building roofs in satellite remote sensing images are extracted for feature extraction.

[0022] The beneficial effects of the present invention are:

[0023] The present invention extracts building roof information by constructing a drone model. When the drone shooting angle is the same as the satellite shooting angle α, it is fused with the satellite remote sensing image features, thereby achieving the purpose of using the building roof information extracted by small-area 3D high-resolution drone data to improve the resolution of building information extracted by large-area satellite remote sensing data, thereby improving the evaluation results of satellite remote sensing data, and can be further used for the evaluation of distributed rooftop photovoltaics. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic flow diagram of the method of the present invention;

[0025] Figure 2 is a schematic diagram of projection angle α;

[0026] Figure 3 A flowchart of the method of the present invention;

[0027] Figure 4 3D modeling display of the roof of Wujin National High-tech Industrial Development Zone

[0028] Figure 5 Schematic diagram of feature extraction for satellite remote sensing images and UAV 3D images. DETAILED DESCRIPTION

[0029] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Embodiment 1

[0031] like Figure 1 As shown, the method for improving the resolution of a 2D satellite remote sensing image based on a 3D building model of an unmanned aerial vehicle in this embodiment includes the following steps:

[0032] Step 1: preprocess the satellite remote sensing image in the target area, obtain the building boundary information in the satellite remote sensing image in the target area, and orthographically project the boundary information onto the surface of the earth model;

[0033] Step 2: Refine the projected building roof information to remove building roof information errors caused by shadows, and extract features from the building roof in the satellite remote sensing image;

[0034] Step 3: Calculate the tilt angle α of the satellite remote sensing image based on the time when the satellite remote sensing image was taken and the satellite altitude information recorded in the satellite data metadata; Figure 2 As shown, the inclination angle α is calculated as follows:

[0035] α=arctan(h / L)

[0036] Where h is the altitude of the satellite, and L is the distance from the satellite sub-satellite point to the end point of the building's shadow when shooting.

[0037] Step 4: Take 3D drone remote sensing images of the buildings in the target area to obtain 3D models of the buildings in the target area;

[0038] Step 5: Select an image with the same angle as α in the 3D model of the building, extract the building roof features in the image, perform 2D projection of the building based on the building roof, and fuse it with the projection of the satellite remote sensing image to improve the image resolution of the target area.

[0039] Embodiment 2

[0040] This embodiment takes Changzhou City, Jiangsu Province as an example to verify and illustrate the invention. Figure 3 As shown, first select Jiangsu Changzhou 0.5 satellite remote sensing data as the processing object:

[0041] Step 1: Preprocess the remote sensing images in the study area, mainly including radiation correction, geometric correction, orthorectification, mosaicking and cutting.

[0042] Step 2: Obtain the boundary information of buildings in the area, mainly through semantic segmentation technology, to extract the boundary information of buildings, and orthographically project all boundary information onto the surface of the earth model;

[0043] Step 3: Refine the projected building roof information to remove the building roof information errors caused by shadows; and extract the features (patterns) of the building roofs in the satellite remote sensing image;

[0044] Step 4: Calculate the building's tilt angle α based on the time when the satellite image was taken and the satellite's altitude information recorded in the satellite data metadata.

[0045] Step 5: Take Wujin District High-tech Development Zone as the 3D modeling area, select suitable drone platforms and laser scanner equipment, formulate a safe flight plan and obtain necessary flight permits. Plan a reasonable drone flight route according to the shape, size and surrounding environment of the building to be modeled. Ensure that the flight path can fully cover all facades of the building, avoid interference from obstacles, and ensure the integrity and safety of data collection. Use GIS software or drone-specific planning software to set parameters such as flight altitude, speed, and overlap rate. Before formal collection, accurately calibrate the laser scanner carried by the drone, including time synchronization, angle calibration, distance calibration, etc., to ensure that the collected point cloud data is accurate. According to the planned flight route, start the drone to collect laser point cloud data. During the flight of the drone, the laser scanner will continuously emit laser pulses to the ground and receive the reflected signals. By calculating the time difference and angle information, point cloud data in three-dimensional space is generated. The point cloud data collected from different flight sorties are spliced ​​to form a complete point cloud model of the building. Use filtering algorithms to remove noise points in the point cloud data, such as point cloud data generated by reflections from non-target objects such as leaves and pedestrians.

[0046] Step 6: Identify the edge information in the building point cloud (such as through CNN) to provide a basis for subsequent 3D modeling. Use the plane fitting algorithm to extract the various planes of the building (such as walls, roofs, etc.) from the point cloud data to provide structural information for subsequent 3D modeling. Based on the extracted building features, use 3D modeling software (such as AutoCAD, 3ds Max, Revit, etc.) for geometric modeling to construct the three-dimensional structure of the building. You can also take photos of the building through drones and use texture mapping technology to attach the photos to the 3D model to enhance the realism of the model. Simplify the 3D model, reduce the number of polygons in the model, and improve the rendering efficiency and loading speed of the model. Enhance the details of key parts of the model, such as windows, door openings, decorative lines, etc., to make the model more refined. Export the optimized 3D model to the OBJ format suitable for this project, and finally display the results as follows. Figure 4 shown.

[0047] Step 7: Select the image with the same angle as α in the 3D model of the building, and extract the building roof features (patterns) in the image. Perform 2D projection of the building based on the building roof and fuse it with the projection of the satellite remote sensing image. By fusing the high-resolution image of the drone with the low-resolution image of the satellite, a mask that can be used to improve the resolution of the satellite remote sensing image is generated, thereby improving the global resolution of the satellite remote sensing image.

[0048] like Figure 5As shown, in this embodiment, the satellite remote sensing image is subjected to feature extraction. First, the feature extraction is performed by fusing the FPN and SNAKE algorithms, where FPN (Feature Pyramid Networks) is a multi-scale feature extraction network. Four downsamplings are used here, and each time the image is reduced by 1 / 4 compared to the previous one. Each time the image is reduced, a SNAKE algorithm is used to extract the features of the image. The SNAKE algorithm here is an active contour model, which converts the image segmentation problem into a problem of solving the minimum value of the energy functional. The main idea is to construct an energy functional, and after algorithm iteration, the contour curve gradually approaches the image edge that minimizes the energy function (or local minimum) from the initial position, and finally segments the target. Where E T is the total energy function curve of a building boundary line in the image, E Int (v(s)) is the internal energy of the curve, and v(s) is a set of points used to describe the curve of the building roof, which can be expressed as:

[0049] v(s)=[x(s),y(s)], s∈[0,1]

[0050] These points are connected to form a closed contour line. Where x(s)x(s)x(s) and y(s)y(s)y(s) represent the coordinate position of each control point in the image, and s is the independent variable that describes the boundary in the form of Fourier transform, which can also be understood as the arc length.

[0051] E ex (v(s)) is the external energy of the point set v(s), which mainly includes image energy E im and E con External constraint energy composition. Eim is mainly composed of lines, edges, and ends in the image; E con The main external constraints include fixed points, such as the corners of buildings.

[0052] After extracting the features of the satellite remote sensing image, the building roof information is extracted from the constructed drone model. When its shooting angle is the same as the satellite's shooting angle α, the features of the two can be fused to generate a high-resolution mask.

[0053] This high-resolution mask can be fused with the previously extracted satellite remote sensing image features to achieve the purpose of using the building roof information extracted from small-area 3D high-resolution UAV data to improve the building local information extracted from large-area satellite remote sensing data.

[0054] The technical solutions of the present invention are not limited to the above-mentioned embodiments, and any technical solutions obtained by equivalent replacement methods fall within the scope of protection required by the present invention.

Claims

1. A method for improving the building resolution of 2D satellite remote sensing images based on UAV 3D building models, characterized in that: The steps include: Step 1: preprocess the satellite remote sensing image in the target area, obtain the building boundary information in the satellite remote sensing image in the target area, and orthographically project the boundary information onto the surface of the earth model; Step 2: Refine the roof information of the projected building and extract features of the building roof in the satellite remote sensing image; Step 3, calculating the tilt angle α of the satellite remote sensing image according to the time when the satellite remote sensing image was taken and the altitude information of the satellite; Step 4: Take 3D drone remote sensing images of the buildings in the target area to obtain 3D models of the buildings in the target area; Step 5: Select an image with the same angle as α in the 3D model of the building, extract the building roof features in the image, perform 2D projection of the building based on the building roof, and fuse it with the projection of the satellite remote sensing image to improve the image resolution of the target area.

2. The method for improving the building resolution of 2D satellite remote sensing images based on the 3D building model of an unmanned aerial vehicle according to claim 1, characterized in that: The preprocessing in step 1 includes radiation correction, geometric correction, orthorectification, mosaicking and clipping.

3. The method for improving the resolution of 2D satellite remote sensing images based on the 3D building model of an unmanned aerial vehicle according to claim 2, characterized in that: In step 1, the building boundary information is obtained by using semantic segmentation technology.

4. The method for improving the building resolution of 2D satellite remote sensing images based on the 3D building model of an unmanned aerial vehicle according to claim 3, characterized in that: The inclination angle α is calculated as follows: α=arctan(h / L) Where h is the altitude of the satellite, and L is the distance from the satellite sub-satellite point to the end point of the building's shadow when shooting.

5. The method for improving the building resolution of 2D satellite remote sensing images based on the 3D building model of an unmanned aerial vehicle according to claim 4, characterized in that: The step 4 is specifically as follows: Use GIS software or drone planning software to set the drone’s flight altitude, speed, and overlap rate; Accurately calibrate the laser scanner on the drone, including time synchronization, angle calibration, and distance calibration; According to the planned flight route, the drone is started to collect laser point cloud data. During the flight of the drone, the laser scanner will emit laser pulses to the ground and receive the reflected signals. By calculating the time difference and angle information, point cloud data in three-dimensional space is generated; The point cloud data collected from different flights are spliced ​​together to form a complete point cloud model of the building.

6. The method for improving the building resolution of 2D satellite remote sensing images based on the 3D building model of an unmanned aerial vehicle according to claim 5, characterized in that: The point cloud data collected by the UAV is filtered to remove noise points in the point cloud data.

7. The method for improving the building resolution of 2D satellite remote sensing images based on the 3D building model of an unmanned aerial vehicle according to claim 1, characterized in that: Through the FPN network and SNAKE algorithm, the building roofs in satellite remote sensing images are extracted for feature extraction.

Images