Three-dimensional image modeling method and device based on plane data

Abstract

The invention provides a three-dimensional image modeling method and device based on plane data, and the method comprises the steps: carrying out the full-color / multispectral band fusion, bit depth adjustment, thin cloud removal, image enhancement, stripe removal and light and color uniformity of a satellite surveying and mapping image, and achieving the real color recovery of the image; performing splicing preprocessing on the plurality of small orthographic satellite surveying and mapping images after real color recovery, performing image registration on the to-be-registered image and a reference image, performing image geometric correction on the registered image, and performing image mosaic processing on the plurality of images after image geometric correction to obtain a single large-scene image; and three-dimensional image matching, building mask extraction, three-dimensional building modeling and three-dimensional building post-processing are carried out on a single large scene image to obtain a three-dimensional digital surface model, and ground three-dimensional real image modeling is completed. By applying the technical scheme of the invention, the technical problem that the existing two-dimensional surveying and mapping data is difficult to meet the increasing application requirements of three-dimensional scenes is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for stereoscopic image modeling based on planar data. Background Technology

[0002] With the rapid development of aerospace, sensor, and communication technologies, satellite remote sensing has made tremendous progress, becoming an important means of Earth observation and environmental monitoring. Relying on the efficient and wide-area observations provided by satellite remote sensing, geographic mapping technology has achieved leaps and bounds. Satellite remote sensing can cover vast geographical areas in a short time, greatly improving the efficiency and scope of mapping; satellite remote sensing data updates quickly, providing the latest surface information, which is particularly important for rapidly changing regions; the optical resolution of satellite remote sensing images is constantly improving, allowing for more detailed observation and inversion of ground scenes; and the increasing richness of satellite multispectral data gives it a crucial advantage in refined Earth analysis. To date, after more than half a century of development, countries around the world have collected a large amount of remote sensing Earth observation data, producing numerous two-dimensional mapping results such as digital elevation, orthophotos, and digital line maps, accumulating a vast amount of two-dimensional mapping data and resources.

[0003] However, with technological advancements and expanding application demands, two-dimensional mapping data is struggling to meet the growing application needs of three-dimensional scenarios. Three-dimensional geographic data holds greater value in areas such as terrain analysis, flood simulation, urban planning, emergency disaster relief, situational awareness, and sophisticated simulation. Therefore, the question of how to construct realistic three-dimensional scene models based on two-dimensional mapping data such as remote sensing imagery and topographic maps to achieve automated, high-quality, and efficient production is becoming an increasingly popular and urgent application requirement. Summary of the Invention

[0004] This invention provides a method and apparatus for stereoscopic image modeling based on planar data, which can solve the technical problem that existing two-dimensional surveying data is difficult to meet the growing application needs of three-dimensional scenes.

[0005] This invention provides a method for stereo image modeling based on planar data. The method includes: Step 1, performing panchromatic / multispectral band fusion, depth adjustment, thin cloud removal, image enhancement, stripe removal, and light and color homogenization on satellite survey images to restore the true colors of the satellite survey images; Step 2, performing stitching preprocessing on multiple small orthophoto satellite survey images after true color restoration, performing image registration between the preprocessed image to be registered and the reference image, performing image geometric correction on the registered image, and performing image mosaicking on the multiple geometrically corrected images to obtain a single large-scene image; Step 3, performing stereo image matching, building mask extraction, 3D building modeling, and 3D building post-processing on the single large-scene image to obtain a 3D digital surface model, thus completing the stereo image modeling based on planar data.

[0006] Furthermore, the fusion methods used for panchromatic / multispectral band fusion of satellite mapping images include IHS transform, principal component transform, weighted product, ratio transform, wavelet transform, high-pass filtering, BROVERY, and the PANSHARP fusion method combining GRB and IHS transform.

[0007] Furthermore, adjusting the bit depth of satellite mapping images specifically includes processing raw data of different bit depths into 8-bit images that are easy to display, which requires maximizing the retention of data information based on the statistical distribution of the data.

[0008] Furthermore, stripe removal in satellite mapping images specifically includes: introducing an edge weight factor on the constraint of the vertical direction of the stripes, and solving and optimizing the proposed model through the alternating direction multiplier method (ADMM) to complete the stripe removal of satellite mapping images.

[0009] Furthermore, in step two, deep learning methods are used for image registration. These deep learning methods include PointNet and OA-NET, as well as the SuperGlue algorithm based on the attention mechanism and the Transformer-based LofTR, CoTR, and ClusterGNN.

[0010] Furthermore, in step two, image geometric correction is performed using methods such as RPC model (rational polynomial) correction and feature point fine correction.

[0011] Further, in step two, the image mosaicking process performed on multiple geometrically corrected images to obtain a single large-scene image specifically includes: stitching together multiple geometrically corrected small orthophoto images according to pixel and geographical location relationships to obtain an original large mosaic image; downsampling the original large mosaic image according to pyramid levels, calculating the downsampling level under the condition of a specified working image size, and calculating the downsampled thumbnail of the original large mosaic image; performing mosaic line calculation, mask calculation, and color gradient smoothing on the downsampled thumbnail to generate a small-size mosaic preview image; performing color processing on the small-size mosaic preview image to obtain preview images with different style colors; using Laplacian pyramid color fusion and multi-band fusion techniques, and employing a block processing method, mapping the colors from the preview image onto the original large mosaic image to obtain a single large-scene image.

[0012] Further, step three specifically includes: based on satellite image stereo pairs and SRTM elevation data, performing stereo image matching on a single large-scene image to generate an idealized surface model LOD0; based on the idealized surface model LOD0, extracting building masks, adding building heights to the building masks to generate a simple building model LOD1; based on the simple building model LOD1, combining it with an Open Street Map (OSM) to generate a refined building model LOD2; performing 3D modeling post-processing on the refined building model LOD2 to generate a 3D surface model.

[0013] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps of the stereoscopic image modeling method based on planar data as described above.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the stereoscopic image modeling method based on planar data as described above.

[0015] The present invention provides a method for stereoscopic image modeling based on planar data. The method includes steps such as restoration of the true colors of satellite survey images, combination processing of satellite survey images, and three-dimensional modeling of two-dimensional survey data. This method addresses the problem of insufficient realism of two-dimensional remote sensing images in three-dimensional simulation by performing stitching and enhancement processing on two-dimensional image data from satellite surveying, thereby improving the visualization effect of remote sensing images and obtaining a high-fidelity three-dimensional ground model. Attached Figure Description

[0016] The accompanying drawings, which form part of this specification, are provided to further illustrate embodiments of the invention and, together with the textual description, explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0017] Figure 1 shows the overall flowchart of the stereo image modeling method based on planar data provided according to a specific embodiment of the present invention;

[0018] Figure 2 shows a schematic diagram of satellite mapping image combination processing according to a specific embodiment of the present invention;

[0019] Figure 3 A flowchart of remote sensing data fusion processing according to a specific embodiment of the present invention is shown;

[0020] Figures 4a to 4c A schematic diagram of Pensharpen band fusion according to a specific embodiment of the present invention is shown (4a is panchromatic, 4b is multispectral, and 4c is the fusion result).

[0021] Figure 5a and Figure 5b A schematic diagram of bit depth adjustment color recovery provided according to a specific embodiment of the present invention is shown;

[0022] Figure 6a and Figure 6b A schematic diagram of color restoration for removing thin clouds / haze according to a specific embodiment of the present invention is shown;

[0023] Figure 7a and Figure 7b A schematic diagram of stripe removal data recovery provided according to a specific embodiment of the present invention is shown;

[0024] Figure 8a and Figure 8b This diagram illustrates a multi-image color restoration process according to a specific embodiment of the present invention.

[0025] Figures 9a to 9c This diagram illustrates a comparison of deep learning registration and feature detection according to a specific embodiment of the present invention.

[0026] Figure 10 A schematic diagram of the distribution of orthophoto correction control points according to a specific embodiment of the present invention is shown;

[0027] Figure 11 A schematic diagram of a mosaic-mesh flow processing method according to a specific embodiment of the present invention is shown;

[0028] Figure 12 A flowchart of a three-dimensional modeling process according to a specific embodiment of the present invention is shown; Figure 13 A three-dimensional reconstruction effect diagram provided according to a specific embodiment of the present invention is shown. Detailed Implementation

[0029] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0031] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0032] like Figure 1As shown in the figure, a stereo image modeling method based on planar data is provided according to a specific embodiment of the present invention. The stereo image modeling method based on planar data includes: Step 1, performing panchromatic / multispectral band fusion, depth adjustment, thin cloud removal, image enhancement, stripe removal, and light and color homogenization on satellite survey images to achieve true color restoration of satellite survey images; Step 2, performing stitching preprocessing on multiple small orthophoto satellite survey images after true color restoration, performing image registration on the image to be registered and the reference image after stitching preprocessing, performing image geometric correction on the registered image, and performing image mosaicking on the multiple images after image geometric correction to obtain a single large scene image; Step 3, performing stereo image matching, building mask extraction, 3D building modeling, and 3D building post-processing on the single large scene image to obtain a 3D digital surface model, thus completing the stereo image modeling based on planar data.

[0033] This configuration provides a method for stereoscopic image modeling based on planar data. The method includes steps such as restoration of the true colors of satellite survey images, combination processing of satellite survey images, and three-dimensional modeling of two-dimensional survey data. This method addresses the problem of insufficient realism of two-dimensional remote sensing images in three-dimensional simulation by performing stitching and enhancement processing on satellite survey two-dimensional image data to improve the visualization effect of remote sensing images and obtain a high-fidelity three-dimensional ground model.

[0034] The stereo image modeling method based on planar data provided by this invention specifically includes the following steps:

[0035] 1. True color restoration of satellite mapping images

[0036] The main focus is on image preprocessing algorithms, thin cloud removal, and color balancing to address the challenge of restoring true color from raw satellite data and obtaining true-color remote sensing imagery. Image preprocessing algorithms primarily include: panchromatic / multispectral band fusion, bit depth adjustment, thin cloud removal, light and color balancing, image enhancement, and stripe removal. Thin cloud removal technology utilizes image processing techniques to ensure that cloud-covered areas maintain consistent brightness, saturation, and hue with cloudless areas, thereby enhancing the contrast and clarity of ground features and improving the efficiency of color balancing. Color balancing aims to eliminate color differences in the image, resulting in an image with consistent overall brightness and uniform hue. Color differences can originate from a single image or from multiple images.

[0037] Among them, the fusion methods used for panchromatic / multispectral band fusion of satellite mapping images include IHS transform, principal component transform, weighted product, ratio transform, wavelet transform, high-pass filtering, BROVERY, and PANSHARP fusion method combining GRB and IHS transform.

[0038] The specific steps for adjusting the bit depth of satellite mapping images include: processing raw data of different bit depths into 8-bit images that are easy to display, which requires maximizing the retention of data information based on the statistical distribution of the data.

[0039] The stripe removal process for satellite mapping images specifically includes: introducing an edge weight factor on the constraint of the vertical direction of the stripes, and solving and optimizing the proposed model using the Alternating Direction Multiplier Method (ADMM) to complete the stripe removal of satellite mapping images.

[0040] 2. Satellite mapping image combination processing

[0041] In the application of remote sensing imagery, when the study area is located at the intersection of several image sets, or when the study area is large and requires multiple image sets to cover it, it is necessary to register and stitch together the relevant image sets within the coverage area. This allows for better unified processing, interpretation, analysis, and research. The combined processing flow is shown in the attached figure. Figure 2 As shown.

[0042] Preprocessing for image stitching: Basic digital image processing operations such as histogram matching, smoothing filtering, and enhancement transformation are performed on the original image to prepare it for the next step of image stitching.

[0043] Image registration: Image registration is the core of the entire image stitching process, and its accuracy determines the stitching quality. The basic idea is to first find the corresponding positions of templates or feature points between the image to be registered and the reference image. Then, based on the correspondence, a mathematical model for the transformation between the reference and image to be registered is established, transforming the image to be registered into the coordinate system of the reference image and determining the overlapping area between the two images. The key to accurate registration is finding a data model that can well describe the transformation relationship between the two images. In step two, deep learning methods are used for image registration. These methods include PointNet and OA-NET, as well as the SuperGlue algorithm based on the attention mechanism and Transformer-based algorithms such as LofTR, CoTR, and ClusterGNN.

[0044] Image geometric correction: Image geometric correction includes methods such as RPC (Rational Polynomial) correction and feature point fine correction. Polynomial correction assumes the image has no significant distortion and achieves geometric correction through the correlation of control points, making it suitable for images of plains areas. Satellite imagery involves side-view angles, which can cause significant distortion within the image, especially in areas with large terrain undulations. In such cases, polynomial correction produces substantial errors. Fine correction, on the other hand, uses ground control points (GCPs) combined with a digital elevation model (DEM) and satellite attitude parameter control to eliminate distortion caused by terrain undulations, thus improving correction accuracy. Figure 10 As shown.

[0045] Image mosaicking: After determining the transformation relationship model between two images, i.e., the overlapping area, it is necessary to mosaic the images to be stitched into a visually feasible panoramic image based on the information of the overlapping area. Due to factors such as slight differences in terrain or different shooting conditions causing differences in image grayscale or brightness, or the existence of certain registration errors in the image registration results, it is necessary to select an appropriate image mosaicking strategy in order to minimize the impact of residual deformation or brightness or grayscale differences between images on the mosaicking result.

[0046] In step two, the process of mosaicking multiple geometrically corrected images to obtain a single large-scene image specifically includes: stitching together multiple geometrically corrected small orthophoto images according to pixel and geographical location relationships to obtain an original large mosaic image; downsampling the original large mosaic image according to pyramid levels, calculating the downsampling level under the condition of a specified working image size, and calculating the downsampled thumbnail of the original large mosaic image; performing mosaic line calculation, mask calculation, and color gradient smoothing on the downsampled thumbnail to generate a small-size mosaic preview image; performing color processing on the small-size mosaic preview image to obtain preview images with different style colors; using Laplacian pyramid color fusion and multi-band fusion techniques, and employing a block processing method, mapping the colors from the preview image to the original large mosaic image to obtain a single large-scene image.

[0047] 3. Three-dimensional modeling of two-dimensional surveying data

[0048] 3D modeling of remote sensing images involves converting two-dimensional remote sensing images of the same area from multiple perspectives into a three-dimensional digital surface model. The main steps include: stereo image matching, building mask extraction, 3D building modeling, and 3D building post-processing.

[0049] In this invention, step three specifically includes: based on satellite image stereo pairs and SRTM elevation data, performing stereo image matching on a single large-scene image to generate an idealized surface model LOD0; based on the idealized surface model LOD0, extracting building masks, adding building heights to the building masks to generate a simple building model LOD1; based on the simple building model LOD1, combining it with an Open Street Map (OSM) to generate a refined building model LOD2; performing 3D modeling post-processing on the refined building model LOD2 to generate a 3D surface model.

[0050] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps of the stereoscopic image modeling method based on planar data as described above.

[0051] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the stereoscopic image modeling method based on planar data as described above.

[0052] To gain a further understanding of the present invention, the following description is provided in conjunction with... Figures 1 to 13 The method for stereo image modeling based on planar data provided by this invention will be described in detail.

[0053] like Figures 1 to 13 As shown, a typical area is illustrated using 3D modeling for a detailed explanation.

[0054] 1. True color restoration of satellite mapping images

[0055] True color restoration employs radiometric correction methods, which include steps such as full-color / multi-spectral band fusion, bit depth adjustment, de-clouding, color mapping, stripe removal, and uniform light and color mixing.

[0056] 1.1 Panchromatic / Multispectral Band Fusion

[0057] Panchromatic / multispectral band information fusion, also known as color image synthesis, involves fusing high-resolution panchromatic and low-resolution multispectral band data to generate a single high-resolution color image, thereby improving the geometric resolution and spectral fidelity of the image. Common fusion methods include IHS transform, principal component transform, weighted product, ratio transform, wavelet transform, high-pass filtering, Brovy, and PANSHARP fusion combining GRB and IHS transform, among others. IHS transform and PANSHARP fusion methods show particularly good performance in image fusion.

[0058] The process of restoring true colors through panchromatic / multi-band data fusion is as follows: Figure 3 As shown.

[0059] Data fusion methods vary depending on the circumstances and the type of data source. However, they generally involve the following steps: pre-fusion image processing (including image enhancement and contrast improvement), selection of fusion units, selection and implementation of the optimal fusion algorithm, post-fusion processing (e.g., image sharpening), and effect checking (visual inspection).

[0060] Figures 4a to 4c This is a PanSharpen band fusion algorithm. The middle image is a 2.4m spatial resolution color image, the left image is a 0.6m spatial resolution panchromatic image, and the right image is a high-resolution color image generated using PanSharpen.

[0061] 1.2-bit deep adjustment

[0062] To provide higher spectral resolution, the bit depth of raw data acquired by satellite sensors is typically higher than 8 bits. Bit depth adjustment involves processing raw data at different bit depths into 8-bit images that are easy to display, and this requires maximizing the retention of data information based on the statistical distribution of the data.

[0063] 1.3 Remove the thin clouds

[0064] Thin clouds in the atmosphere can reduce the contrast of the colors of ground features, decrease clarity, and interfere with color uniformity. Thin clouds are mainly composed of two parts: ground feature signals and noise. Methods for removing thin clouds by separating noise and signals are divided into spatial domain and frequency domain methods.

[0065] 1.4 Stripe Removal

[0066] Stripe noise is mainly caused by inconsistencies in the electrical response of multi-element detectors, and commonly occurs in swing-broom and push-broom remote sensing imaging systems. Stripe noise can be removed and data restored by analyzing its structural properties and separating the stripe components. In the optimization model, L1-norm-based regularization represents the global sparsity of the stripes; difference-based constraints describe the smoothness along the stripe direction and the discontinuity along the stripe's vertical direction. To better preserve image details, an edge weight factor is introduced into the constraints along the stripe's vertical direction. Finally, the proposed model is solved and optimized using the Alternating Direction Multiplier Method (ADMM). Stripe removal and data restoration are achieved, as shown in the image. Figure 7a and Figure 7b As shown.

[0067] 1.5 Even light and color

[0068] The goal of uniform light and color processing in visible light remote sensing images is to eliminate color differences and obtain images with consistent overall brightness and uniform tone. Color difference problems in visible light remote sensing images can be categorized into three types. For color restoration of multiple images, the effect is as follows... Figure 8a and Figure 8b As shown.

[0069] 2. Satellite mapping image combination processing

[0070] The goal of satellite mapping image assembling processing is to generate single large-scene images that are correctly positioned, geometrically aligned, and color-balanced. By studying key technologies, an intelligent mosaicking system is developed, which mainly includes image registration, orthorectification, and image mosaicking.

[0071] 2.1 Deep Learning Registration

[0072] Deep learning is increasingly being applied to image registration, such as PointNet and OA-NET, as well as attention-based algorithms like SuperGlue and Transformer-based algorithms like LofTR, CoTR, and ClusterGNN. These methods combine feature point coordinates and descriptors to construct new matching descriptors using graph neural networks, treating feature matching as an optimal transfer problem and leveraging the differentiability of the Sinkhom algorithm to achieve end-to-end training. Through attention mechanisms, long-distance interactions between features can be achieved spatially, thus overcoming the limitations of local feature perception and enhancing feature matching accuracy.

[0073] like Figures 9a to 9c As shown, image pairs from four different scenarios are configured. In non-rigid matching scenarios, deep learning-based methods exhibit better robustness and obtain more correctly matched feature pairs, thus providing an option for the convergence of the RANSAC method and the entire matching process. The drawback is that current deep learning registration methods have relatively low computational performance.

[0074] 2.2 Image Geometric Correction

[0075] Image geometric correction includes methods such as RPC (Rational Polynomial) correction and feature point fine correction. Polynomial correction assumes the image has no significant distortion and achieves geometric correction through the correlation of control points, making it suitable for images of plains areas. However, satellite imagery involves side-view angles, which can cause significant distortion within the image, especially in areas with large terrain undulations. In such cases, polynomial correction produces substantial errors. Fine correction, on the other hand, uses ground control points (GCPs) combined with a digital elevation model (DEM) and satellite attitude parameter control to eliminate distortion caused by terrain undulations, thus improving correction accuracy. Figure 10 As shown.

[0076] 2.3 Image Mosaic

[0077] Image mosaicking involves stitching together multiple small orthophotos based on pixel and geographic location relationships to obtain a large scene image. These small orthophotos may have been captured using different sensors and at different times, resulting in variations in color and brightness. Even slight differences in brightness and hue at the edges of adjacent areas can produce noticeable stitching blocks, creating a patchwork effect.

[0078] To obtain a clear and seamless mosaic map, it is necessary to (1) ensure the geometric continuity of man-made features at the seam locations and (2) resolve color and brightness differences between images. Therefore, the mosaic line method should be used for overlapping orthophotos.

[0079] Block processing is applicable to multiple mosaicking steps such as mosaicking line calculation, color mapping, and large image output. It helps to save memory consumption while driving multiple CPU computing cores to carry out parallel computing, effectively improving the timeliness of large-scale image mosaicking.

[0080] To further improve the color quality and processing performance of mosaicking, this paper proposes a mesh flow processing technique. This technique integrates a large number of color balancing and color mapping techniques. The method is as follows: First, the original large mosaic image is downsampled according to the pyramid level. Under the condition of a specified working image size (usually 500-1000 pixels), the downsampling level is calculated, and the downsampled thumbnail of the original single piece is calculated. Then, mosaicking lines, mask calculation, color gradient smoothing, and a small mosaic preview image are performed on this small thumbnail. Next, color processing is performed on the preview image, such as using a color mapping algorithm to obtain images with different color styles. Finally, Laplacian pyramid color fusion and multi-band fusion techniques are used, and a block processing method is employed to map the colors from the processed preview image to the large mosaic image.

[0081] 3. Three-dimensional modeling of two-dimensional surveying data

[0082] 3D model reconstruction from satellite remote sensing imagery involves generating a 3D surface model, or high-fidelity digital surface model (DSM), from 2D satellite remote sensing imagery. This includes both traditional digital terrain (DEM) models and detailed 3D building models. There are three types of digital surface models: LOD0 (idealized surface), LOD1 (with simple building models), and LOD2 (with detailed buildings). This application includes four main steps: stereo image matching, building mask generation, 3D building modeling, and 3D building post-processing. Stereo image matching generates the LOD0 model, building masks combined with building heights create the LOD1 model, and finally, the LOD2 model is generated. Based on satellite imagery stereo pairs and SRTM elevation data, a single large-scene image is matched to generate a LOD0 model. Building masks are extracted from the LOD0 model, and building heights are added to the building masks to generate a LOD1 model. Based on the LOD1 model, an LOD2 model is generated by combining an Open Street Map (OSM). The LOD2 model is then processed to generate a 3D surface model.

[0083] In the building mask extraction part, deep learning-based semantic analysis is used for image segmentation, and deep learning-based building detection is used to detect individual buildings. Segmentation-level fusion is used to obtain the initial building mask for building fragment fusion.

[0084] The 3D building modeling section mainly includes processes such as building polygon extraction, building rectangle fusion, graph-based algorithm-based building rectangle orientation optimization, OSM-based building rectangle orientation optimization, and 3D model fitting. The purpose of building polygon extraction is to vectorize building boundaries into line segments for optimizing the building model fit. Building rectangle fusion is carried out using mesh and rectangle decomposition algorithms. Graph-based algorithms optimize the building rectangle orientation, and OSM-based optimization (OSM being OpenStreetMap, a publicly available crowdfunded vector database) is also performed. For 3D model fitting, the extracted building polygons are rectangles, which can be easily fitted using a simplex building model. Five types of building models are considered: flat roof, triangular roof, ridge roof, pyramid roof, and double-sloped roof.

[0085] like Figure 12 As shown, the data sources are mainly satellite imagery stereo pairs, SRTM elevation data, and OpenStreetMap (OSM). Stereo pairs are satellite remote sensing images of the same area from multiple perspectives, generally requiring consistent sensor data with short time intervals; SRTM is open global 90-meter grid elevation data; OpenStreetMap is open-source, crowdfunded vector map data, characterized by high accuracy and frequent updates.

[0086] In summary, this invention provides a method for stereoscopic image modeling based on planar data, including steps such as true color restoration of satellite survey images, satellite survey image combination processing, and three-dimensional modeling of two-dimensional survey data. It can achieve true color restoration of satellite survey images, mainly through image preprocessing algorithms, thin cloud removal, and color equalization, to solve the problem of true color restoration from raw satellite data, obtaining true-color remote sensing images. It can achieve satellite survey image combination processing; for specific areas located at the intersection of several images, or for large areas requiring multiple images for coverage, related images of the coverage area need to be registered and stitched together, thus facilitating better unified processing, interpretation, analysis, and research. It can achieve three-dimensional modeling based on two-dimensional survey data, converting two-dimensional remote sensing images of the same area from multiple perspectives into a three-dimensional digital surface model, obtaining a high-fidelity three-dimensional ground model. The main steps include stereoscopic image matching, building mask generation, three-dimensional building modeling, and three-dimensional building post-processing.

[0087] This invention relates to a stereoscopic image modeling technology based on planar data. Its feature is that, in order to address the problem of insufficient realism in two-dimensional remote sensing images in three-dimensional simulation, it performs stitching and enhancement processing on two-dimensional image data obtained from satellite mapping to improve the visualization effect of remote sensing images and obtain a high-fidelity three-dimensional ground model.

[0088] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0089] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for stereo image modeling based on planar data, characterized in that, The stereo image modeling method based on planar data includes: Step 1: Perform panchromatic / multispectral band fusion, depth adjustment, thin cloud removal, image enhancement, stripe removal, and light and color homogenization on the satellite mapping image to achieve true color restoration of the satellite mapping image; Step 2: Perform image stitching preprocessing on multiple small orthophoto satellite images after real color restoration; perform image registration on the image to be registered and the reference image after stitching preprocessing; perform image geometric correction on the registered image; and perform image mosaic processing on multiple images after image geometric correction to obtain a single large scene image. Step 3: Perform stereo image matching, building mask extraction, 3D building modeling, and 3D building post-processing on the single large scene image to obtain a 3D digital surface model, thus completing stereo image modeling based on planar data.

2. The stereo image modeling method based on planar data according to claim 1, characterized in that, The fusion methods used for panchromatic / multispectral band fusion of satellite mapping images include IHS transform, principal component transform, weighted product, ratio transform, wavelet transform, high-pass filtering, BROVERY, and PANSHARP fusion method which combines GRB and IHS transform.

3. The stereo image modeling method based on planar data according to claim 2, characterized in that, The specific steps for adjusting the bit depth of satellite mapping images include: processing raw data of different bit depths into 8-bit images that are easy to display, which requires maximizing the retention of data information based on the statistical distribution of the data.

4. The stereo image modeling method based on planar data according to claim 3, characterized in that, The stripe removal process for satellite mapping images specifically includes: introducing an edge weight factor on the constraint of the vertical direction of the stripes, and solving and optimizing the proposed model using the Alternating Direction Multiplier Method (ADMM) to complete the stripe removal of satellite mapping images.

5. The stereo image modeling method based on planar data according to any one of claims 1 to 4, characterized in that, In step two, deep learning methods are used for image registration. These deep learning methods include PointNet and OA-NET, as well as the SuperGlue algorithm based on the attention mechanism and the Transformer-based LofTR, CoTR, and ClusterGNN.

6. The stereo image modeling method based on planar data according to claim 5, characterized in that, In step two, image geometric correction is performed using methods such as RPC model (rational polynomial) correction and feature point fine correction.

7. The stereo image modeling method based on planar data according to claim 6, characterized in that, In step two, the process of mosaicking multiple geometrically corrected images to obtain a single large-scene image specifically includes: Multiple geometrically corrected small orthophoto images are stitched together according to pixel and geographical location relationships to obtain the original large mosaic image; The original large mosaic image is downsampled according to the pyramid level. Under the condition of a specified working image size, the downsampling level is calculated, and the downsampled thumbnail of the original large mosaic image is calculated. The mosaicking line calculation, mask calculation, and color gradient smoothing are performed on the downsampled thumbnail to generate a small-sized mosaicking preview image. Color processing is performed on the small-sized mosaic preview image to obtain preview images with different style colors; Using Laplacian pyramid color fusion and multi-band fusion techniques, and employing a block-based processing method, colors are mapped from the preview image onto the original large mosaic image to obtain a single large scene image.

8. The method for stereo image modeling based on planar data according to claim 7, characterized in that, Step three specifically includes: Based on satellite image stereo pairs and SRTM elevation data, stereo image matching is performed on a single large-scene image to generate an idealized surface model LOD0. Based on the idealized ground model LOD0, building masks are extracted, and building heights are added to the building masks to generate a simple building model LOD1. Based on the simple building model L0D1, a more detailed building model L0D2 is generated by combining an OpenStreetMap (OSM). The detailed building model L0D2 is processed to generate a 3D surface model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the stereo image modeling method based on planar data as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the stereo image modeling method based on planar data as described in any one of claims 1 to 8.

Images