Urban real scene three-dimensional construction method based on high-resolution satellite image

Abstract

The invention relates to an urban live-action three-dimensional construction method based on a high-resolution satellite image. The method comprises the following steps: carrying out stereo image pair screening and stereo image pair preprocessing on a high-resolution multi-view satellite image; carrying out adjustment optimization based on a rational function model on the satellite stereo image pair; generating an epipolar ray image based on the optimized rational function model; performing dense matching on the epipolar line image by combining building edge straight line segment constraint and an MGM algorithm to generate a disparity map; generating a three-dimensional point cloud and a three-dimensional model formed by a plurality of stereo image pairs through forward intersection based on the disparity map and the optimization parameters; generating a digital elevation model, a digital surface model and a three-dimensional Mesh model based on the three-dimensional point cloud; and carrying out precision evaluation and integrity evaluation by adopting pixel proportions of plane precision, elevation precision and height difference. According to the method, the urban live-action three-dimensional model and the corresponding digital product can be constructed quickly, efficiently and accurately by using the high-resolution satellite image.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method for constructing a three-dimensional urban scene based on high-resolution satellite images. Background Art

[0002] With the acceleration of urbanization and the continuous advancement of science and technology, the demand for an accurate and comprehensive understanding and representation of the urban environment is growing. Urban 3D, which provides a realistic, three-dimensional, and time-series representation of the spatiotemporal information of human production, living, and ecological space, is a crucial component of the country's new infrastructure development and is of great significance to the comprehensive understanding, planning, and management of China's cities in the future.

[0003] Currently, the construction of real-world 3D urban scenes primarily relies on oblique photogrammetry and lidar methods, as demonstrated in Chinese invention patents CN119665918A and CN120147563A. However, with the continuous advancement of Earth observation technology, the resolution and accuracy of optical mapping satellite imagery have continued to improve, making it possible to construct real-world 3D urban scenes using high-resolution satellite imagery. Furthermore, compared to traditional real-world modeling using oblique photogrammetry data, high-resolution satellite imagery offers advantages such as low cost, wide observation range, no airspace restrictions, and short revisit cycles. This makes it more suitable for large-scale real-world 3D modeling and the rapid construction of urban 3D models.

[0004] By capturing the same target area from satellites at different angles, multi-view images can be obtained. Three-dimensional reconstruction of these multi-view images of the target area can provide high-precision three-dimensional geographic information about the Earth's surface and reconstruct the target area's three-dimensional geographic environment. However, due to the large amount of data and the difficulty in deploying control points in high-resolution satellite imagery, constructing a three-dimensional geographic environment using high-resolution satellite imagery presents challenges such as low scene accuracy, poor reliability, insufficient scene integrity, and inefficient construction. Furthermore, because satellite imaging utilizes push-broom technology, unlike conventional array imaging, existing, established oblique photography methods cannot be directly applied to satellite image processing. Summary of the Invention

[0005] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method for constructing real-life three-dimensional urban scenes based on high-resolution satellite images. By utilizing high-resolution satellite images, a real-life three-dimensional model of the urban scene and corresponding digital products can be constructed quickly, efficiently and accurately.

[0006] To achieve the above-mentioned object, the present invention provides a method for constructing a three-dimensional urban scene based on high-resolution satellite images, comprising the following steps:

[0007] Step S1, screening and preprocessing stereo pairs of high-resolution multi-view satellite images;

[0008] Step S2, the satellite stereo image is carried out to the adjustment optimization based on the rational function model;

[0009] Step S3, generating an epipolar image based on the optimized rational function model;

[0010] Step S4, combining the building edge straight line segment constraint with the MGM algorithm to perform dense matching on the epipolar image to generate a disparity map;

[0011] Step S5: generating a three-dimensional point cloud and a three-dimensional model formed by multiple stereo image pairs through forward intersection based on the disparity map and the optimized parameters;

[0012] Step S6: generating a digital elevation model, a digital surface model, and a three-dimensional mesh model based on the three-dimensional point cloud;

[0013] Step S7: Use plane accuracy, elevation accuracy, and pixel ratio of height difference to perform accuracy and completeness evaluation.

[0014] According to a technical solution of the present invention, step S1 specifically includes:

[0015] Step S11: Acquire auxiliary imaging information of high-resolution multi-view satellite images;

[0016] Step S12, selecting the best stereo image pair based on the maximum incident angle α of the image, the intersection angle β between the images, and the time interval t between the acquisition of the images;

[0017] Step S13: histogram normalization is performed on the selected image pairs to achieve brightness equalization.

[0018] According to a technical solution of the present invention, in step S12, according to the maximum incident angle α<40°, the intersection angle 15°≤β≤40° and the minimum time interval t in the time interval t, min Or filter stereo pairs in the same season.

[0019] According to a technical solution of the present invention, step S2 specifically includes:

[0020] Step S21: extracting image feature points based on the SIFT operator and performing matching to obtain accurate feature matching point pairs;

[0021] Step S22: Optimizing rational function model parameters by using an uncontrolled or controlled bundle adjustment method for the two-view or multi-view stereo image based on the rational function model.

[0022] According to a technical solution of the present invention, step S3 specifically includes:

[0023] Step S31, calculating the basic matrix H of coordinate transformation based on the same-name points of the stereo image pair;

[0024] Step S32: Perform SVD decomposition on the basic matrix to obtain affine transformation matrices H1 and H2;

[0025] Step S33 : Apply H1 and H2 to perform geometric transformation on the left and right images of the stereoscopic image pair respectively, to generate left and right epipolar images with a vertical parallax of approximately zero.

[0026] According to a technical solution of the present invention, in step S5, a forward intersection is performed based on the disparity map generated by dense matching and the rational function model after corresponding adjustment optimization to generate the three-dimensional coordinates of the ground points of all matching point pairs, forming a dense matching point cloud, and at the same time, high-precision alignment and fusion are performed on the three-dimensional model formed by multiple stereo image pairs.

[0027] According to a technical solution of the present invention, in step S6, based on the dense matching point cloud generated in step S5, processing is performed to obtain a digital elevation model, a digital surface model and a real-city three-dimensional mesh model;

[0028] The generated dense matching point cloud is ground filtered and projected according to the geographic grid to obtain a digital elevation model;

[0029] The generated dense matching point cloud is directly projected onto the geographic grid to obtain a digital surface model;

[0030] Based on the generated dense matching point cloud, Delaunay triangulation is used to reconstruct the surface and generate a 3D mesh model of the real city scene.

[0031] According to a technical solution of the present invention, step S7 specifically includes:

[0032] Extract multiple distinctive features evenly within the generated digital surface model, extract the corresponding distinctive features from the high-precision reference data, and calculate the plane accuracy RMSE of the digital surface model. Plane and elevation absolute accuracy RMSE Height evaluation results.

[0033] According to a technical solution of the present invention, the following formula is used to calculate the plane accuracy RMSE: Plane and elevation accuracy RMSE Height :

[0034]

[0035] Among them, n represents the number of high-precision detection points, X i ',Y i ',Height i ' represents the i-th feature salient point extracted from the generated digital surface model, Xi ,Y i ,Height i Represents the corresponding salient feature points extracted from high-precision reference data.

[0036] According to a technical solution of the present invention, in step S7, the percentage of pixels whose actual height difference between the generated digital surface model and the high-precision reference data is less than 1 meter in the total effective pixels is calculated to complete the integrity evaluation, which is expressed as:

[0037]

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] This paper proposes a method for constructing real-world 3D cities based on high-resolution satellite imagery. Aiming at high-resolution satellite imagery data, a complete set of 3D reconstruction techniques is designed and proposed, which has important practical application value for urban planning and resource management. At the same time, the feature extraction and matching algorithms in the satellite 3D reconstruction process are optimized, solving common problems such as low positioning accuracy of satellite image modeling, large matching noise, missing ground geometry, and structural adhesion. The generated building edge point cloud is more complete, and the structural outline of the constructed real-world 3D Mesh model is clearer.

[0040] This invention converts high-resolution satellite imagery from two-dimensional to three-dimensional spatial information, enabling the rapid, accurate, and automated construction of large-scale, city-level surface 3D models. This invention provides new insights and methods for the application of real-world 3D modeling of urban scenes, particularly for large-scale 3D reconstruction of target areas of interest abroad. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0042] Figure 1 Flowchart of the steps of a method for constructing a 3D urban scene based on high-resolution satellite images according to an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the maximum incident angle α and the intersection angle β between images for screening the best stereo image pair in an embodiment of the present invention;

[0044] Figure 3Schematic diagram of a process for adjusting a satellite stereo image pair based on a rational function model in an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of satellite 3D reconstruction data using original stereo images of Jacksonville in an embodiment of the present invention;

[0046] Figure 5 The real-scene 3D white model Mesh and the real-scene 3D texture Mesh are generated by performing satellite 3D reconstruction of the original stereo image of Jacksonville in an embodiment of the present invention;

[0047] Figure 6 In the embodiment of the present invention, a typical elevation and plane point schematic diagram is extracted from the real digital surface model and the digital surface model generated by 3D reconstruction, which is used to quantitatively evaluate the accuracy of 3D reconstruction. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0049] like Figures 1 to 6 As shown, the present invention provides a method for constructing a three-dimensional urban scene based on high-resolution satellite images, comprising the following steps:

[0050] Step S1: Screen and pre-process the high-resolution multi-view satellite images into stereo pairs. Figure 2 As shown, specifically including:

[0051] Step S11: Acquire auxiliary imaging information of high-resolution multi-view satellite images;

[0052] Step S12, selecting the best stereo image pair based on the maximum incident angle α of the image, the intersection angle β between the images, and the time interval t between the acquisition of the images;

[0053] Step S13: histogram normalization is performed on the selected image pairs to achieve brightness equalization;

[0054] In some embodiments of the present invention, in step S12 , the stereo image pairs are selected based on the maximum incident angle α<40°, the intersection angle 15°≤β≤40°, and the minimum time interval tmin in the time interval t or the same season.

[0055] By limiting α<40°, β=15°-40°, and determining the minimum time interval t from the selected stereo pairs min , thereby determining the best stereo pair, overcoming the geometric distortion (>2 pixels) of satellite images caused by large incident angles and the occlusion problem caused by large intersection angles. At the same time, histogram normalization greatly reduces the brightness difference of stereo pairs, thereby reducing the subsequent matching error rate and solving the problem of failed matching of shadow pairs.

[0056] In addition, by limiting time intervals and seasons, elevation errors caused by changes in vegetation cover (such as the exposure of deciduous forests in winter) can be eliminated, the accuracy of the connection between buildings and the surface can be improved, and the consistency of surface texture during cross-season modeling can be ensured.

[0057] Step S2: perform adjustment optimization based on rational function model on the satellite stereo image pair, such as Figure 3 As shown, specifically including:

[0058] Step S21: extracting image feature points based on the SIFT operator and matching them to obtain accurate feature matching point pairs;

[0059] Step S22: Optimizing rational function model parameters by using an uncontrolled or controlled bundle adjustment method for the two-view or multi-view stereo image based on the rational function model.

[0060] SIFT feature matching combined with RFM adjustment can optimize the positioning accuracy of satellite images. At the same time, controlled adjustment can further improve the accuracy to meet the needs of urban component-level modeling.

[0061] Step S3, generating an epipolar image based on the optimized rational function model, specifically includes:

[0062] Step S31, calculating the basic matrix H of coordinate transformation based on the same-name points of the stereo image pair;

[0063] Step S32: Perform SVD decomposition on the basic matrix to obtain affine transformation matrices H1 and H2;

[0064] Step S33 : Apply H1 and H2 to perform geometric transformation on the left and right images of the stereoscopic image pair respectively, to generate left and right epipolar images with a vertical parallax of approximately zero.

[0065] Among them, the stereo image pair consists of two images, a left image and a right image. Usually, the left and right images are set according to the shooting time, and the shooting time of the left image is earlier than that of the right image. The kernel line resampling method of the basic matrix SVD decomposition makes the vertical parallax lower, thereby eliminating the offset of the top of high-rise buildings and ensuring that the vertical structure is not distorted.

[0066] Step S4, combining the building edge straight line segment constraint with the MGM algorithm to perform dense matching on the epipolar image to generate a disparity map;

[0067] For urban scenes containing a large number of buildings, building edge information is the most abundant feature. Therefore, we combine building edge straight line segments with the classic MGM (More Global Matching, a variant of the SGM algorithm) to perform stereo matching processing on high-resolution satellite optical images. This achieves more robust dense matching of epipolar images with fewer matching misses, and generates a dense matching disparity map.

[0068] Step S5: generating a three-dimensional point cloud and a three-dimensional model formed by multiple stereo image pairs through forward intersection based on the disparity map and the optimized parameters;

[0069] Based on the disparity map generated by dense matching and the rational function model after corresponding adjustment optimization, a forward intersection is performed to generate the three-dimensional coordinates of the ground points of all matching point pairs, forming a dense matching point cloud. At the same time, high-precision alignment and fusion are performed on the three-dimensional models formed by multiple stereo image pairs to achieve the generation of a large range of digital products and three-dimensional models.

[0070] Step S6: generating a digital elevation model, a digital surface model, and a three-dimensional mesh model based on the three-dimensional point cloud;

[0071] Based on the dense matching point cloud generated in step S5, the digital elevation model (DEM), digital surface model (DSM) and urban real scene 3D mesh model are processed respectively. The specific implementation is as follows: ground filtering is performed on the generated dense matching point cloud to obtain a ground point cloud, and then a certain geographic coordinate and projection grid size are set to perform ground projection to obtain a digital elevation model; based on the generated dense matching point cloud, according to the certain geographic coordinate and projection grid size, the ground projection is directly performed to obtain a digital surface model; based on the generated dense matching point cloud, the Delaunay triangulation method is used to perform surface reconstruction to generate a urban real scene 3D mesh model; Figure 4 and Figure 5 are the input and output of satellite 3D reconstruction, where Figure 4 For the best stereo pair selected according to step S1, Figure 5 To perform 3D reconstruction on the screened stereo image pairs, a 3D Mesh model of the real city scene is generated.

[0072] Step S7: Use plane accuracy, elevation accuracy, and pixel ratio of height difference to perform accuracy and completeness evaluation.

[0073] The 3D model accuracy evaluation mainly evaluates the generated digital surface model, and its accuracy evaluation consists of geometric accuracy and scene integrity accuracy. The geometric accuracy of the digital surface model is divided into elevation and plane absolute accuracy. Multiple feature-significant points (such as building turning points) can be evenly extracted from the generated digital surface model, and then the corresponding feature-significant points can be extracted from the high-precision reference data to calculate the plane accuracy RMSE of the digital surface model. Plane and elevation absolute accuracy RMSE Height The completeness evaluation method is to calculate the percentage of pixels whose difference from the true height is less than 1 meter among all valid pixels.

[0074] In some embodiments of the present invention, the following formula is used to calculate the plane accuracy RMSE: Plane and elevation accuracy RMSE Height :

[0075]

[0076] Among them, n represents the number of high-precision detection points, X i ',Y i ',Height i ' represents the i-th feature salient point extracted from the generated digital surface model, X i ,Y i ,Height i Represents the corresponding salient feature points extracted from high-precision reference data.

[0077] In some embodiments of the present invention, the completeness evaluation is expressed as:

[0078]

[0079] like Figure 6 As shown in the figure, typical elevation and plane point schematic diagrams are extracted from the real digital surface model and the digital surface model generated by 3D reconstruction, which are used to quantitatively evaluate the accuracy of 3D reconstruction.

[0080] The present invention provides a method for constructing real-world 3D cities based on high-resolution satellite imagery: first, a complete set of 3D reconstruction technology methods is designed and proposed for high-resolution satellite imagery data, which has important practical application value for urban planning and resource management; second, the feature extraction and matching algorithms in the satellite 3D reconstruction process are optimized, solving common problems such as low positioning accuracy of satellite image modeling, large matching noise, missing ground object geometry, and structural adhesion, thereby achieving a more complete generated building edge point cloud and a clearer structural outline of the constructed real-world 3D mesh model.

[0081] This invention converts high-resolution satellite imagery from two-dimensional to three-dimensional spatial information, enabling the rapid, accurate, and automated construction of large-scale, city-level surface 3D models. This invention provides new insights and methods for the application of real-world 3D modeling of urban scenes, particularly for large-scale 3D reconstruction of target areas of interest abroad.

[0082] In the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not preclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0083] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A method for constructing a three-dimensional urban scene based on high-resolution satellite images, characterized in that: The following steps are involved: Step S1, screening and preprocessing stereo pairs of high-resolution multi-view satellite images; Step S2, the satellite stereo image is carried out to the adjustment optimization based on the rational function model; Step S3, generating an epipolar image based on the optimized rational function model; Step S4, combining the building edge straight line segment constraint with the MGM algorithm to perform dense matching on the epipolar image to generate a disparity map; Step S5: generating a three-dimensional point cloud and a three-dimensional model formed by multiple stereo image pairs through forward intersection based on the disparity map and the optimized parameters; Step S6: generating a digital elevation model, a digital surface model, and a three-dimensional mesh model based on the three-dimensional point cloud; Step S7: Use plane accuracy, elevation accuracy, and pixel ratio of height difference to perform accuracy and completeness evaluation.

2. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 1, characterized in that: The step S1 specifically includes: Step S11: Acquire auxiliary imaging information of high-resolution multi-view satellite images; Step S12, selecting the best stereo image pair based on the maximum incident angle α of the image, the intersection angle β between the images, and the time interval t between the acquisition of the images; Step S13: histogram normalization is performed on the selected image pairs to achieve brightness equalization.

3. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 2, characterized in that: In step S12, according to the maximum incident angle α<40°, the intersection angle 15°≤β≤40° and the minimum time interval t among the time intervals t min Or filter stereo pairs in the same season.

4. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 1, wherein: The step S2 specifically includes: Step S21: extracting image feature points based on the SIFT operator and matching them to obtain accurate feature matching point pairs; Step S22: Optimizing rational function model parameters by using an uncontrolled or controlled bundle adjustment method for the two-view or multi-view stereo image based on the rational function model.

5. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 1, characterized in that: The step S3 specifically includes: Step S31, calculating the basic matrix H of coordinate transformation based on the same-name points of the stereo image pair; Step S32: Perform SVD decomposition on the basic matrix to obtain affine transformation matrices H1 and H2; Step S33 : Apply H1 and H2 to perform geometric transformation on the left and right images of the stereoscopic image pair respectively, to generate left and right epipolar images with a vertical parallax of approximately zero.

6. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 1, characterized in that: In step S5, a forward intersection is performed based on the disparity map generated by dense matching and the rational function model after corresponding adjustment optimization to generate the three-dimensional coordinates of the ground points of all matching point pairs, forming a dense matching point cloud, and high-precision alignment and fusion are performed on the three-dimensional model formed by multiple stereo image pairs.

7. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 1, characterized in that: In step S6, based on the dense matching point cloud generated in step S5, processing is performed to obtain a digital elevation model, a digital surface model and a city scene three-dimensional mesh model; The generated dense matching point cloud is ground filtered and projected according to the geographic grid to obtain a digital elevation model; The generated dense matching point cloud is directly projected onto the geographic grid to obtain a digital surface model; Based on the generated dense matching point cloud, Delaunay triangulation is used to reconstruct the surface and generate a 3D mesh model of the real city scene.

8. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 1, wherein: The step S7 specifically includes: Extract multiple distinctive features evenly within the generated digital surface model, extract the corresponding distinctive features from the high-precision reference data, and calculate the plane accuracy RMSE of the digital surface model. Plane and elevation absolute accuracy RMSE Height evaluation results.

9. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 8, characterized in that: The following formula is used to calculate the plane accuracy RMSE Plane and elevation accuracy RMSE Height : Among them, n represents the number of high-precision detection points, X i ',Y i ',Height i ' represents the i-th feature salient point extracted from the generated digital surface model, X i ,Y i ,Height i Represents the corresponding salient feature points extracted from high-precision reference data.

10. The method for constructing a 3D urban scene based on high-resolution satellite images according to claim 9, characterized in that: In step S7, the percentage of pixels whose actual height difference between the digital surface model and the high-precision reference data is less than 1 meter in the total effective pixels is calculated to complete the integrity evaluation, which is expressed as:

Images