High-resolution satellite image three-dimensional reconstruction system for smart city
By combining the two-dimensional EMD algorithm with the denoising process of adaptive Gaussian filtering and the matching window of the adaptive NCC algorithm, the problems of noise and matching difficulty in the three-dimensional reconstruction of satellite imagery are solved, and high-precision urban three-dimensional reconstruction effect is achieved.
Patent Information
- Application Number
- CN202610576928.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-28
- Publication Date
- 2026-07-07
AI Technical Summary
Traditional satellite image 3D reconstruction methods suffer from noise introduction, exposure differences, and matching difficulties in complex scenes in high-resolution remote sensing data, resulting in insufficient accuracy in 3D reconstruction, especially in urban scenes with dense buildings and many shadows, which cannot meet the high-precision requirements.
A high-resolution satellite imagery 3D reconstruction system for smart cities is adopted. Noise reduction is performed by combining the 2D EMD algorithm with adaptive Gaussian filtering. High-frequency energy coefficients are constructed to optimize the number of decomposition layers. The matching window of the NCC algorithm is adjusted by combining texture repetition features and illumination intensity features. Sparse reconstruction and dense reconstruction techniques are used to generate a LOD 3D reconstruction model.
It effectively filters out complex noise in urban images, improves image matching accuracy, generates more accurate 3D reconstruction models, and meets the high-resolution reconstruction needs of complex urban scenes.
Smart Images
Figure CN122347642A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of satellite image 3D reconstruction technology, specifically to a high-resolution satellite image 3D reconstruction system for smart cities. Background Technology
[0002] In recent years, the number of remote sensing satellites has grown rapidly, and high-resolution satellite image sensor technology has advanced by leaps and bounds. With the ever-increasing demands for accuracy in 3D terrain visualization, traditional binocular image 3D reconstruction methods are no longer sufficient to meet these growing needs. Therefore, 3D reconstruction from satellite imagery has gradually become an effective way to improve the accuracy of digital surface models and address various complex application scenarios.
[0003] Currently, due to various factors such as differences in satellite sensor hardware, different observation times, and diverse shooting angles, noise is inevitably introduced during the data acquisition stage, leading to serious errors in the subsequent analysis and use of remote sensing satellite imagery. Furthermore, the diversity of shooting angles can easily cause exposure differences between different satellite images, especially in scenes with dense buildings, many shadows, or complex terrain features, which further increases the difficulty of image matching and results in insufficient accuracy in the final 3D reconstruction of cities from satellite imagery. Summary of the Invention
[0004] To address the aforementioned technical problems, the purpose of this application is to provide a high-resolution satellite imagery 3D reconstruction system for smart cities, and the specific technical solution adopted is as follows: This application proposes a high-resolution satellite imagery 3D reconstruction system for smart cities, the system comprising: Data acquisition module: Acquires high-resolution satellite imagery of the target city from multiple perspectives; Denoising module: It splits each satellite image into high-frequency and low-frequency images, constructs high-frequency energy coefficients based on the energy distribution characteristics of high-frequency and low-frequency images, and obtains the stopping criteria when decomposing high-frequency images using the two-dimensional EMD algorithm; it denoises each IMF component after decomposition and combines it with the low-frequency image to obtain the denoised satellite images. Image matching module: Divide each denoised satellite image into several image blocks. Based on the texture repetition features and local illumination variation uniformity of each image block, construct the first feature value of each image block. Combine the illumination intensity and grayscale distribution features of each image block to construct the second feature value of each image block. This adjusts the matching window in the NCC algorithm to obtain matching feature points between any two satellite images. 3D modeling module: Based on the matching feature points between all satellite images, a digital terrain model is constructed, which in turn generates a LOD 3D reconstruction model.
[0005] Preferably, the method for constructing the high-frequency energy coefficient is as follows: The power spectra and their corresponding energies of the high-frequency and low-frequency images are obtained respectively. Calculate the sum of the energies of the power spectra of the high-frequency image and the low-frequency image, and record the ratio between the energy of the power spectrum of the high-frequency image and the sum as the high-frequency energy coefficient of the high-frequency image.
[0006] Preferably, the formula for calculating the stopping criterion when using the two-dimensional EMD algorithm to decompose high-frequency images is as follows: In the formula, Here, q represents the stopping criterion for high-frequency images; q and p represent the preset maximum and minimum values of the stopping criterion, respectively. This represents the high-frequency energy coefficient of the high-frequency image.
[0007] Preferably, the method for acquiring the denoised satellite images is as follows: The high-frequency images of each satellite image are used as inputs to the two-dimensional EMD algorithm. The stopping criterion is set to the stopping criterion of the corresponding high-frequency image, and several IMF component images and residual trend images are output. Denoise the image for each IMF component; The denoised IMF component images corresponding to the high-frequency images of each satellite image are reconstructed with the residual trend image. The reconstructed images are then added to the corresponding pixels of the low-frequency images to obtain the denoised satellite images.
[0008] Preferably, the method for constructing the first feature value of each image block is as follows: Perform sliding window sampling within each image block; The LBP encoded values of all pixels in each sampling window are arranged in order of pixel position from top to bottom and from left to right, forming a one-dimensional sequence of each sampling window; Calculate the Pearson correlation coefficient between each sampling window and the one-dimensional sequence of all its neighboring windows, and record the mean of all the Pearson correlation coefficients as the texture repetition value of each sampling window; Calculate the mean value of texture repetition for all sampling windows in each image block; Calculate the variance of the gradient magnitude of all pixels in each image block; The first feature value of each image block is positively correlated with the mean and negatively correlated with the variance.
[0009] Preferably, the method for constructing the second feature value of each image block is as follows: The grayscale histogram of each image block is obtained, and the kernel density estimation algorithm is used to obtain the kernel density estimation curve of the grayscale histogram, thereby obtaining the total number of maximum points in the kernel density estimation curve. Calculate the average grayscale value of each image block; The second feature value of each image block is negatively correlated with the total number and the average gray value, and positively correlated with the first feature value.
[0010] Preferably, the specific formula for adjusting the matching window in the NCC algorithm is as follows: Calculate the second feature value of all image blocks in each satellite image and normalize them. The mean of all normalized values is recorded as the adjustment factor for each satellite image. The formula for adjusting the matching window in the NCC algorithm is as follows: In the formula, Let be the size of the matching window for the i-th satellite image; is the adjustment factor for the i-th satellite image; , These are the preset minimum and maximum values for the matching window size, respectively; This is the rounding function.
[0011] Preferably, the method for obtaining the matching feature points between any two satellite images is as follows: The algorithm takes any two denoised satellite images as input, uses the adaptively acquired matching window of each satellite image as the corresponding matching window size, and outputs the matching feature points between any two satellite images.
[0012] Preferably, the construction process of the digital terrain model is as follows: Based on the matching feature points between any two satellite images, the camera pose is recovered using sparse reconstruction techniques, thereby generating sparse point clouds from various viewpoints. Dense point clouds are generated from sparse point clouds at various viewpoints using dense reconstruction techniques; The TSDF algorithm is used to fuse dense point clouds from multiple perspectives to generate a digital terrain model.
[0013] Preferably, the specific process of generating the LOD 3D reconstruction model is as follows: An initial triangular mesh was constructed for the digital terrain model using the Delaunay triangulation algorithm; The initial triangular mesh is optimized by using point cloud normal vector information to guide the smooth transition of the surface and repair small holes, using edge folding simplification to reduce the amount of data, and using Laplacian smoothing to eliminate reconstruction noise. The optimized initial triangular mesh is used as input for continuous LOD generation technology to generate a LOD 3D reconstruction model.
[0014] This application has the following beneficial effects: This application proposes a high-resolution satellite imagery 3D reconstruction system for smart cities. In satellite imagery denoising, a combination of a 2D EMD algorithm and adaptive Gaussian filtering is employed. Based on the high and low frequency energy distribution characteristics of satellite images, the number of decomposition layers is optimized. This approach effectively filters out complex noise in urban images while maximizing the preservation of key details, thus improving the accuracy of subsequent image matching. In the image matching module, a second feature value is constructed through in-depth analysis of texture repetition features, brightness uniformity variations, and illumination intensity characteristics in the satellite images. Based on this, the matching window size of the NCC algorithm is adaptively optimized, improving the matching accuracy between images and helping to compensate for the shortcomings of high-resolution satellite imagery 3D reconstruction accuracy. Attached Figure Description
[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a block diagram of a high-resolution satellite imagery 3D reconstruction system for smart cities provided in one embodiment of this application; Figure 2 This is a flowchart illustrating the process of obtaining matching feature points between any two satellite images, as provided in one embodiment of this application. Detailed Implementation
[0017] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the high-resolution satellite imagery 3D reconstruction system for smart cities proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0019] The specific scheme of the high-resolution satellite imagery 3D reconstruction system for smart cities provided in this application is described in detail below with reference to the accompanying drawings.
[0020] Please see Figure 1The diagram illustrates a block diagram of a high-resolution satellite imagery 3D reconstruction system for smart cities according to an embodiment of this application. The system includes: a data acquisition module, a noise reduction module, an image matching module, and a 3D modeling module.
[0021] Data acquisition module: Acquire high-resolution satellite imagery of the target city from multiple perspectives.
[0022] High-resolution remote sensing data provides crucial support for the realization of concepts such as digital cities and smart cities; therefore, reconstructing high-resolution 3D models of cities is essential. This application obtains multi-view high-resolution satellite imagery of the target city from a satellite operation service provider. Each acquired satellite image has undergone radiometric and geometric correction. Radiometric correction eliminates radiation distortion caused by sensor characteristics and solar altitude angle, restoring the true reflectivity of ground features. Geometric correction corrects geometric distortions caused by sensor attitude, terrain undulations, and Earth curvature.
[0023] Denoising module: It splits each satellite image into high-frequency and low-frequency images, constructs high-frequency energy coefficients based on the energy distribution characteristics of the high-frequency and low-frequency images, and obtains the stopping criteria when decomposing the high-frequency image using the two-dimensional EMD algorithm; it denoises each IMF component after decomposition and combines it with the low-frequency image to obtain the denoised satellite image.
[0024] Satellite imagery inevitably introduces noise during reception and acquisition, which severely impacts subsequent dense matching and other analytical processing. Therefore, in the denoising process of remote sensing image data, it is crucial to preserve the original satellite image details while removing noise.
[0025] Urban satellite imagery is characterized by dense buildings, complex textures, rich edges, and uneven noise distribution. Traditional denoising methods struggle to balance noise removal and detail preservation. However, the 2D EMD algorithm can adaptively decompose the image into IMF components of different frequencies, achieving frequency band separation between noise and signal. Therefore, this application employs a combination of the 2D EMD algorithm and adaptive Gaussian filtering for denoising processing.
[0026] Satellite imagery of cities is composed of different frequency components. The high-frequency component represents a wealth of detail and noise from the original image data, while the low-frequency component represents the macroscopic structure and overall information of the image. Therefore, based on this characteristic, this application takes the i-th satellite image as an example. First, it uses this satellite image as the input to a low-pass filter, outputting both high-frequency and low-frequency images. Since noise signals are mainly concentrated in the high-frequency component, the low-frequency image is kept unchanged, and only the high-frequency image is decomposed using a two-dimensional EMD algorithm. Considering the varying levels of detail in satellite imagery of different urban areas—for example, densely built commercial centers with complex road networks have more detailed satellite imagery, while sparsely built suburbs or new urban areas with mostly bare land or vegetation have fewer detailed features—the number of decomposition layers in the two-dimensional EMD algorithm is adaptively obtained, but the stopping criterion... The value of the stopping criterion has a significant impact on the final number of decomposition layers. This value directly affects the overall denoising performance of the image and the quality of the subsequently reconstructed image. Specifically, when the stopping criterion value is small, the frequency components in the same IMF decomposed will be relatively simple, which is suitable for processing images containing a lot of detailed information and requiring fine extraction of high-frequency information. When the stopping criterion value is large, the IMF components will retain more original features, which is suitable for processing images dominated by low-frequency components.
[0027] Therefore, this application adapts the stopping criteria of the two-dimensional EMD algorithm based on the energy ratio characteristics between the obtained high-frequency and low-frequency images to improve denoising performance. Specifically, Fourier transform technology is used to obtain the power spectra of the high-frequency and low-frequency images respectively, and the energy of the corresponding power spectra is calculated respectively; the Fourier transform technology and the calculation method of the power spectrum energy are well known technologies, and the specific process will not be described in detail. Then, the sum of the energy of the power spectra of the high-frequency and low-frequency images is calculated, and the ratio between the energy of the power spectrum of the high-frequency image and the sum is recorded as the high-frequency energy coefficient of the high-frequency image; the larger the obtained high-frequency energy coefficient, the more detailed information the high-frequency image contains, so when using the two-dimensional EMD algorithm to process the high-frequency image, a smaller stopping criterion should be set. During the ratio calculation, when the denominator is 0, a preset minimum positive number (0.001 in this embodiment) is added to the denominator as a parameter tuning factor to prevent the calculation from crashing due to the denominator being 0.
[0028] Furthermore, this application sets the range of the stopping criterion in the two-dimensional EMD algorithm to [p, q]. In this embodiment, p is 0.1 and q is 0.5. The specific calculation formula for the stopping criterion of the high-frequency image is as follows: In the formula, Here, q represents the stopping criterion for high-frequency images; q and p represent the preset maximum and minimum values of the stopping criterion, respectively. This represents the high-frequency energy coefficient of the high-frequency image.
[0029] Then, the high-frequency image of the i-th satellite image is used as the input to the two-dimensional EMD algorithm, and the stopping criterion is set as follows. The system outputs several IMF component images and residual trend images. In this embodiment, the stopping criterion refers to the standard deviation (SD) threshold during the screening process.
[0030] An adaptive Gaussian filtering algorithm is used to denoise each IMF component image. The adaptive Gaussian filtering algorithm is a well-known technique, and the specific process will not be described in detail. All the denoised IMF component images and the residual trend image are used to reconstruct the image. The reconstructed image is recorded as the high-frequency reconstructed image. The corresponding pixels of the obtained high-frequency reconstructed image and the low-frequency image are added together to obtain the denoised i-th satellite image.
[0031] Similarly, the same method is used to denoise the satellite images from each shooting angle to obtain denoised satellite images from all shooting angles.
[0032] The above methods can effectively filter out complex noise in urban images while preserving key details such as building outlines and roads. This improves the accuracy of subsequent image matching and processing, and ultimately leads to a more accurate 3D reconstruction model.
[0033] Image matching module: Divide each denoised satellite image into several image blocks. Based on the texture repetition features and local illumination uniformity of each image block, construct the first feature value of each image block. Combine the illumination intensity and grayscale distribution features of each image block to construct the second feature value of each image block. This adjusts the matching window in the NCC algorithm to obtain matching feature points between any two satellite images.
[0034] Furthermore, considering the complex and varied content of satellite imagery in urban scenes, and given the good adaptability of the NCC (Normalized Cross Correlation) algorithm to various complex scenes, this application adopts this algorithm to perform matching processing on any two satellite images. However, due to differences in observation time and shooting angles, the illumination intensity of urban satellite images varies. Non-uniform changes in illumination intensity affect the matching accuracy of the algorithm, and urban satellite images have obvious texture repetition features, which easily leads to ambiguous matching. The matching result of the NCC algorithm is affected by the size of the region window. If the selected region matching window is too small, a large amount of redundant calculation will be generated for regions with obvious texture repetition features or uniform illumination intensity, resulting in low computational efficiency of the algorithm. Conversely, if the selected region matching window is too large, key information in the image may not be well identified for regions with complex texture features or uneven illumination intensity, which can easily lead to mismatches.
[0035] Therefore, taking the denoised i-th satellite image as an example, the satellite image is first converted into a grayscale image and divided into multiple non-overlapping image blocks. In this embodiment, the size of each image block is 64×64 pixels. It should be noted that when the size of an image block located at the edge of the satellite image is less than 64×64 pixels, the actual size is used as the size of the corresponding image block. To obtain the texture variation features and texture repetition features in each image block, sliding window sampling is performed within each image block. In this embodiment, the size of the sampling window is set to [size missing]. The sliding step size is 3, and the sampling window slides in each image block in a top-to-bottom and left-to-right order. Taking the j-th image block as an example, the LBP (Local Binary Pattern) codes of all pixels in each sampling window of the image block are calculated. The obtained LBP codes represent the micro-texture structure features within each sampling window. The LBP code values of all pixels in each sampling window are arranged in order of pixel position from top to bottom and left to right to form a one-dimensional sequence of each sampling window. The Pearson correlation coefficient between each sampling window and the one-dimensional sequences of all its adjacent windows is calculated. The mean of all the Pearson correlation coefficients is taken as the texture repetition value of each sampling window. The mean of the texture repetition feature values of all sampling windows in the j-th image block is also calculated. The larger the mean value, the more significant the texture repetition feature of the j-th image block. Specifically, when the variance of the one-dimensional sequences of two sampling windows is 0 (i.e., a flat region), if the mean values of the one-dimensional sequences of the two sampling windows are the same, the Pearson correlation coefficient between the one-dimensional sequences of these two sampling windows is recorded as 1; otherwise, it is recorded as 0. This application uses the Sobel operator to extract the gradient magnitude of each pixel in the denoised satellite image; then, the variance of the gradient magnitude of all pixels in the j-th image block is denoted as... The result This indicates the uniformity of local illumination variation in the j-th image block. The smaller the value, the more uniform the local illumination variation in the image block.
[0036] Therefore, as a preferred embodiment, based on the texture repetition features and local illumination uniformity of each image block, a first feature value is constructed for each image block to characterize the significance of the texture repetition features and the uniformity of illumination variation in each image block. The method for constructing the first feature value of each image block is as follows: statistically analyze the texture repetition value of each sampling window; calculate the mean of the texture repetition values of all sampling windows in each image block; calculate the variance of the gradient magnitude of all pixels in each image block; the first feature value of each image block is positively correlated with the mean and negatively correlated with the variance. The positive correlation means that the dependent variable increases (decreases) as the independent variable increases (decreases), and the negative correlation means that the dependent variable decreases (increases) as the independent variable increases (decreases).
[0037] In this embodiment, the first feature value of the j-th image block is denoted as... Its specific expression is: In the formula, The first feature value of the j-th image block; The mean of the texture repetition feature values of all sampling windows in the j-th image block; Let be the variance of the gradient magnitude of all pixels in the j-th image block; As a normalization function, this embodiment uses the minimum-maximum normalization method for normalization, and the maximum and minimum values are determined based on the corresponding parameter set of all image blocks in all satellite images; This is a preset constant used to prevent the expression from being meaningless due to a denominator of 0. Its value range is [0.01, 0.1]. The value is very small and its impact on the calculation result is negligible. In this embodiment, it is taken as 0.1.
[0038] In the above expression, the pair in the molecule Add 1 and multiply The purpose is to make the range [-1, 1] Mapped to [0,1]; the result The larger the value, the more significant the texture repetition features and uniform illumination intensity variation features of the j-th image block.
[0039] Furthermore, high-resolution urban satellite imagery itself is highly non-uniform, with significant differences in lighting conditions across different image patches. This means that the processing strategies required for dense matching should also differ. Specifically, in areas with high building density, the regular or semi-regular grid structure formed by the arrangement of buildings, along with the high-frequency edges created by roads and shadows, typically results in a multi-peak histogram with a relatively high average gray level. In contrast, image patches in suburban areas are more homogeneous, with similar gray levels among adjacent pixels. Gray level variations are significantly reduced compared to areas with high building density, resulting in a typically unimodal gray level histogram with a gentler change in surface reflectivity and a relatively low average gray level.
[0040] Therefore, taking the j-th image block as an example, firstly, the grayscale histogram of the image block is obtained, and the kernel density estimation algorithm is used to obtain the kernel density estimation curve of the grayscale histogram. The bandwidth of the kernel density estimation is set to h (h=1.0 in this embodiment). Then, the derivative calculation method is used to obtain the maximum points in the kernel density estimation curve. The ratio of the number of obtained maximum points to the preset maximum number (10 in this embodiment) is recorded as the multimodal coefficient of the j-th image block. The larger the multimodal coefficient, the more obvious the multimodal distribution characteristics are in the grayscale histogram of the j-th image block, and the more likely the image block is a building area with uneven illumination distribution. Then, in order to take into account the distribution characteristics of its average gray level, the ratio between the average gray level of the j-th image block and the maximum gray range of the satellite image gray map (255 in this embodiment) is calculated and denoted as the gray level factor of the j-th image block. The larger the gray level factor, the larger the average gray level of the j-th image block, and the more likely the image block is a building area with uneven illumination distribution.
[0041] In a preferred embodiment, a brightness uniformity coefficient is constructed for each image block based on its illumination intensity and grayscale distribution characteristics. This coefficient characterizes the uniformity of illumination distribution in each image block. The method for constructing the brightness uniformity coefficient for each image block is as follows: a grayscale histogram of each image block is obtained, and a kernel density estimation algorithm is used to obtain the kernel density estimation curve of the grayscale histogram. The total number of maxima in the kernel density estimation curve is then obtained. The mean grayscale value of each image block is calculated. The brightness uniformity coefficient of each image block is negatively correlated with both the total number of maxima and the mean grayscale value.
[0042] In this embodiment, the sum of the product of the multimodal coefficient and the gray factor of the j-th image block and the preset minimum normal number (0.01 in this embodiment) is calculated. The reciprocal of the sum is used as the brightness uniformity coefficient of the j-th image block, which is used to characterize the uniformity of the illumination intensity distribution of the j-th image block. The larger the value, the higher the uniformity of the illumination distribution of the j-th image block.
[0043] In a preferred embodiment, a second feature value is constructed for each image block based on its first feature value and brightness uniformity coefficient. This second feature value characterizes the probability that each image block belongs to a region with repetitive texture and uniform light intensity. The method for constructing the second feature value of each image block is as follows: obtain the first feature value and brightness uniformity coefficient of each image block. The second feature value of each image block is positively correlated with both the brightness uniformity coefficient and the first feature value.
[0044] In this embodiment, the second feature value of the j-th image block is denoted as... The specific formula is as follows: In the formula, The second feature value of the j-th image block; The first feature value of the j-th image block; Let be the brightness uniformity coefficient of the j-th image block; As a normalization function, this embodiment uses the minimum-maximum normalization method for normalization, and the maximum and minimum values are determined based on the corresponding parameter set of all image blocks in all satellite images.
[0045] income The larger the value, the more significant the texture repetition features and uniform light intensity distribution features of the j-th image block.
[0046] The first feature value measures the matching ambiguity caused by texture repetition features and texture variation features at the texture state level, while the brightness uniformity coefficient reflects the uniformity of illumination distribution at the brightness distribution level. The sum of the two can more accurately evaluate the significance of texture repetition features and brightness distribution uniformity features of each image block, enabling the NCC algorithm to dynamically adjust the matching window size parameters in a targeted manner.
[0047] Furthermore, the second feature value of all image blocks in the denoised i-th satellite image is calculated and normalized. The mean of all normalized values is recorded as the adjustment factor of the i-th satellite image. The larger the value, the greater the degree of texture repetition and the more uniform the light intensity distribution in the i-th satellite image. Therefore, in the subsequent image matching process of this satellite image, a larger matching window needs to be set to include more texture context information, thereby eliminating matching ambiguities caused by weak or repetitive textures.
[0048] Based on the above analysis, this application sets the matching window size in the NCC algorithm to the range of [g, h]. In this embodiment, g and h are set to 7 and 21, respectively. The matching window size for each satellite image is calculated based on its adjustment factor. The specific calculation formula is as follows: In the formula, Let be the size of the matching window for the i-th satellite image; is the adjustment factor for the i-th satellite image; , These are the preset minimum and maximum values for the matching window size, respectively; This is the rounding function. The calculation logic of the above formula ensures that the calculated matching window size is an odd number within the range [g, h].
[0049] The NCC algorithm takes any two denoised satellite images as input, uses the adaptively acquired matching window size for each satellite image as the corresponding matching window size, and outputs the matching feature points between any two satellite images. The flowchart for obtaining the matching feature points between any two satellite images is shown below. Figure 2 As shown.
[0050] 3D modeling module: Based on the matching feature points between all satellite images, a digital terrain model is constructed, which in turn generates a LOD 3D reconstruction model.
[0051] Furthermore, based on the matching feature points between any two satellite images, the camera pose is recovered using sparse reconstruction technology, thereby generating sparse point clouds from each viewpoint. Then, dense reconstruction technology is used to generate dense point clouds from the sparse point clouds from each viewpoint. Finally, the TSDF algorithm is used to fuse the dense point clouds from multiple viewpoints to generate a digital terrain model (DTM).
[0052] Finally, surface modeling is performed. First, an initial triangular mesh is constructed for the digital terrain model using the Delaunay triangulation algorithm. The initial triangular mesh is then optimized, including using point cloud normal vector information to guide the smooth transition of the surface and repair small holes, using edge folding to reduce the amount of data, and using Laplacian smoothing to eliminate reconstruction noise. The optimized initial triangular mesh is then used as input for continuous LOD generation technology to generate a LOD 3D reconstruction model with realistic textures.
[0053] The above methods for 3D reconstruction of satellite imagery can help compensate for the lack of accuracy in 3D reconstruction of satellite imagery and provide directly accessible 3D digital assets for subsequent applications such as urban planning and virtual simulation.
[0054] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0055] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0056] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.
Claims
1. A high-resolution satellite imagery 3D reconstruction system for smart cities, characterized in that, The system includes: Data acquisition module: Acquires high-resolution satellite imagery of the target city from multiple perspectives; Denoising module: It splits each satellite image into high-frequency and low-frequency images, constructs high-frequency energy coefficients based on the energy distribution characteristics of high-frequency and low-frequency images, and obtains the stopping criteria when decomposing high-frequency images using the two-dimensional EMD algorithm; it denoises each IMF component after decomposition and combines it with the low-frequency image to obtain the denoised satellite images. Image matching module: Divide each denoised satellite image into several image blocks. Based on the texture repetition features and local illumination variation uniformity of each image block, construct the first feature value of each image block. Combine the illumination intensity and grayscale distribution features of each image block to construct the second feature value of each image block. This adjusts the matching window in the NCC algorithm to obtain matching feature points between any two satellite images. 3D modeling module: Based on the matching feature points between all satellite images, a digital terrain model is constructed, which in turn generates a LOD 3D reconstruction model.
2. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The method for constructing the high-frequency energy coefficient is as follows: The power spectra and their corresponding energies of the high-frequency and low-frequency images are obtained respectively. Calculate the sum of the energies of the power spectra of the high-frequency image and the low-frequency image, and record the ratio between the energy of the power spectrum of the high-frequency image and the sum as the high-frequency energy coefficient of the high-frequency image.
3. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The formula for calculating the stopping criterion when using the two-dimensional EMD algorithm to decompose high-frequency images is as follows: In the formula, Here, q represents the stopping criterion for high-frequency images; q and p represent the preset maximum and minimum values of the stopping criterion, respectively. This represents the high-frequency energy coefficient of the high-frequency image.
4. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The method for acquiring the denoised satellite images is as follows: The high-frequency images of each satellite image are used as inputs to the two-dimensional EMD algorithm. The stopping criterion is set to the stopping criterion of the corresponding high-frequency image, and several IMF component images and residual trend images are output. Denoise the image for each IMF component; The denoised IMF component images corresponding to the high-frequency images of each satellite image are reconstructed with the residual trend image. The reconstructed images are then added to the corresponding pixels of the low-frequency images to obtain the denoised satellite images.
5. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The method for constructing the first feature value of each image block is as follows: Perform sliding window sampling within each image block; The LBP encoded values of all pixels in each sampling window are arranged in order of pixel position from top to bottom and from left to right, forming a one-dimensional sequence of each sampling window; Calculate the Pearson correlation coefficient between each sampling window and the one-dimensional sequence of all its neighboring windows, and record the mean of all the Pearson correlation coefficients as the texture repetition value of each sampling window; Calculate the mean value of texture repetition for all sampling windows in each image block; Calculate the variance of the gradient magnitude of all pixels in each image block; The first feature value of each image block is positively correlated with the mean and negatively correlated with the variance.
6. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The method for constructing the second feature value of each image block is as follows: The grayscale histogram of each image block is obtained, and the kernel density estimation algorithm is used to obtain the kernel density estimation curve of the grayscale histogram, thereby obtaining the total number of maximum points in the kernel density estimation curve. Calculate the average grayscale value of each image block; The second feature value of each image block is negatively correlated with the total number and the average gray value, and positively correlated with the first feature value.
7. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The specific formula for adjusting the matching window in the NCC algorithm is as follows: Calculate the second feature value of all image blocks in each satellite image and normalize them. The mean of all normalized values is recorded as the adjustment factor for each satellite image. The formula for adjusting the matching window in the NCC algorithm is as follows: In the formula, Let be the size of the matching window for the i-th satellite image; is the adjustment factor for the i-th satellite image; , These are the preset minimum and maximum values for the matching window size, respectively; This is the rounding function.
8. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The method for obtaining the matching feature points between any two satellite images is as follows: The algorithm takes any two denoised satellite images as input, uses the adaptively acquired matching window of each satellite image as the corresponding matching window size, and outputs the matching feature points between any two satellite images.
9. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The construction process of the digital terrain model is as follows: Based on the matching feature points between any two satellite images, the camera pose is recovered using sparse reconstruction techniques, thereby generating sparse point clouds from various viewpoints. Dense point clouds are generated from sparse point clouds at various viewpoints using dense reconstruction techniques; The TSDF algorithm is used to fuse dense point clouds from multiple perspectives to generate a digital terrain model.
10. The high-resolution satellite imagery 3D reconstruction system for smart cities as described in claim 1, characterized in that, The specific process for generating the LOD 3D reconstruction model is as follows: An initial triangular mesh was constructed for the digital terrain model using the Delaunay triangulation algorithm; The initial triangular mesh is optimized by using point cloud normal vector information to guide the smooth transition of the surface and repair small holes, using edge folding simplification to reduce the amount of data, and using Laplacian smoothing to eliminate reconstruction noise. The optimized initial triangular mesh is used as input for continuous LOD generation technology to generate a LOD 3D reconstruction model.