Satellite image ground object height calculation method and system based on monocular disparity estimation

Abstract

The application provides a satellite image ground object height calculation method based on monocular disparity estimation, comprising: accurately calculating the azimuth and height disparity ratio when the image is taken by using the RPC model of the satellite image; extracting the disparity information along the vertical line direction in the image by using the trained monocular disparity estimation network model, wherein the monocular disparity estimation network is composed of an encoder, a decoder, a layer-by-layer optimization fusion module, a ground classification module and a disparity optimization module; and combining the disparity estimation value, the azimuth and the elevation angle to restore the height information of each position point in the image. By ensuring the consistency of the image and the height information in the same phase, the application is particularly suitable for time-sensitive monitoring of the ground height change, the monocular disparity estimation method is adopted, the object geometric position distortion problem caused by the angle difference is solved, and therefore the accuracy and stability of the height restoration are significantly improved, so that the technology shows a wide application prospect in the monocular oblique satellite image height restoration field.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for calculating the height of ground features in satellite imagery based on monocular parallax estimation. Background Technology

[0002] Traditional 3D reconstruction methods primarily rely on technologies such as Multiple View Stereo (MVS) and Laser Radar (LiDAR). While these methods can provide high-precision reconstruction results, they often require a large number of multi-view images or depend on expensive equipment and complex sensor systems. Over the past three decades, optical satellite remote sensing technology has undergone a leapfrog development, achieving significant progress in key indicators such as spatial resolution, imaging quality, and geometric accuracy. High-resolution satellite imagery not only captures details of the Earth's surface but also provides precise geometric information. By introducing advanced geometric correction models, such as the Rational Polynomial Coefficient (RPC) model, the positioning accuracy and geometric consistency of the imagery have been further improved. This advancement has made the application of monocular satellite imagery in 3D reconstruction possible.

[0003] In close-range applications such as drones and indoor environments, monocular depth estimation has become an important method for reconstructing 3D scenes from single-view images. Satellite imagery is typically taken at altitudes above 500 kilometers. Due to the high angle of view, the scale of objects changes significantly, making it difficult for depth estimation models to accurately infer the depth information of objects. Furthermore, the complex imaging mechanism of satellite imagery (three-line array pushbroom imaging model) further increases the difficulty of depth calculation. Therefore, in 3D reconstruction based on single satellite imagery, height estimation is usually chosen to avoid the uncertainties introduced by long-distance depth estimation. Monocular height estimation refers to the process of constructing a monocular height estimation model using imagery and a Digital Surface Model (DSM) or Above Ground Level (AGL) to achieve the conversion from image information to 3D information. Although depth estimation and height estimation differ in their tasks, the rich methods and theories of depth estimation remain a valuable knowledge base for 3D reconstruction using single satellite imagery. Conventional monocular height estimation methods often face the problem of solution ambiguity, stemming from the fact that the same 2D image can be generated from different 3D scenes. For example, when a shorter building is photographed from a lower elevation angle, its image appearance may be similar to that of a taller building photographed from a higher elevation angle. This lack of height information due to the non-unique mapping between the imaging viewpoint and the target height is one of the main challenges of monocular height estimation. Summary of the Invention

[0004] This invention provides a method and system for calculating the height of ground features in satellite imagery based on monocular parallax estimation, in order to solve the defects in the height recovery of single tilted satellite imagery in the prior art.

[0005] In a first aspect, the present invention provides a method for calculating the height of ground features in satellite imagery based on monocular parallax estimation, comprising:

[0006] Based on ground control point or known ground feature height data, combined with DTM and satellite imagery RPC model, the pixel positions of the top and bottom of the ground feature corresponding to the ground control point in the image are determined, and the azimuth angle is calculated.

[0007] By using ground control points or known feature height data, determine the proportional relationship between the parallax of features along the vertical direction in the image and their actual height.

[0008] The disparity values ​​from the top to the bottom of ground features along the vertical direction are extracted from the oblique satellite image using a trained monocular disparity estimation network model to generate a disparity map.

[0009] Based on the azimuth angle, the scaling relationship, and the disparity map, a height point cloud is obtained;

[0010] The elevation point cloud is pre-processed to generate an elevation map that matches the geographic range of a single tilted satellite image.

[0011] The present invention provides a method for calculating the height of ground features in satellite imagery based on monocular parallax estimation. Based on ground control points or known feature height data, and combining a DTM and satellite imagery RPC model, the method determines the pixel positions of the top and bottom of the features corresponding to the ground control points in the image, and calculates the azimuth angle, including:

[0012]

[0013]

[0014]

[0015] in, , , , These are the polynomial functions of the numerator and denominator in the RPC model, respectively. , The scaling factor in the normalization parameters provided for the RPC model. , Image coordinate offsets in the normalization parameters provided for the RPC model. The coordinates of the ground control points, This represents the surface elevation of the location of the feature. , , and Calculate the azimuth angle respectively Process variables.

[0016] The present invention provides a method for calculating the height of ground features in oblique satellite imagery based on monocular parallax estimation. This method determines the proportional relationship between the parallax of a ground feature along the vertical direction in the image and its actual height using ground control points or known feature height data. The method includes:

[0017]

[0018] in, The height of the ellipsoid at the location of the feature. This represents the surface elevation of the location of the feature, obtained through DTM interpolation. It is a proportional relationship.

[0019] The present invention provides a method for calculating the height of ground features in satellite imagery based on monocular disparity estimation. This method utilizes a trained monocular disparity estimation network model to extract disparity values ​​from the top to the bottom of ground features along the vertical direction from oblique satellite imagery, generating a disparity map. The method includes:

[0020] Obtain a raw dataset consisting of multiple oblique satellite images, and divide the raw dataset into a training set, a validation set, and a test set. The training set and validation set contain oblique satellite image patches, image azimuth information, and label data, while the test set contains oblique satellite image patches and image azimuth information.

[0021] Using the large-scale depth estimation model Depth Anything as the basic framework, a monocular disparity estimation network model is constructed.

[0022] A loss function is constructed based on the disparity prediction loss component, the ground loss component, and the orientation gradient loss component;

[0023] Establish preset error evaluation indicators based on the network prediction results;

[0024] The training set is input into the monocular disparity estimation network model, the gradient descent optimization algorithm is used to update the model parameters, the learning rate is adjusted according to the convergence of the loss function during training, the model performance is monitored in real time using the validation set to prevent overfitting, and through multiple rounds of iteration, the trained monocular disparity estimation network model is obtained.

[0025] The performance of the trained network model is evaluated on the validation set. The disparity prediction ability of the model is quantitatively analyzed by a preset error evaluation index, and the consistency between the network prediction results and the real results is verified by visualization.

[0026] The test set data is input into the trained network model to generate disparity prediction results. The disparity prediction results are then post-processed to obtain an optimized disparity map.

[0027] According to the present invention, a method for calculating the height of ground features in satellite imagery based on monocular parallax estimation is provided. This method uses a large-scale depth estimation model, Depth Anything, as its basic framework to construct a monocular parallax estimation network model, including:

[0028] The input image is encoded using the encoder of the large depth estimation model Depth Anything, and the DPT Head module of Depth Anything and the last four layers of the DINO-v2 pre-trained model are unfrozen for feature reconstruction and prediction.

[0029] A layer-by-layer optimization and fusion module for the decoding layer is constructed. This module comprises five layers, with the first four layers having the same number of channels, and their sizes increasing sequentially to the size of the original image.

[0030] Layers 1 / 16, 1 / 8, 1 / 4, and 1 / 2, with the fifth layer having 32 channels and the same size as the original image, are used. For the first four layers, each layer refines the features of the unfrozen decoded layer using a ResNet module. Through progressive upsampling, convolution, concatenation, and convolution operations, the fused feature layer is obtained as follows:

[0031]

[0032] in, For the first Decoding feature map of the layer To fuse feature layers, , and In order, they are residuals, upsampling, and convolution;

[0033] Construct a ground classification module, using the fifth layer of features in the decoding layer. Then, a ground classification module was added, which uses two 1x1 convolutional layers to classify the ground. The process involves halving the number of input channels in the first convolutional layer and improving training stability through batch normalization. The second convolutional layer compresses the number of channels to 1, outputting a single-channel feature map. Each pixel value represents the probability that the corresponding location is a ground point. A sigmoid activation function is used to enhance features and map the classification probability to the range [0, 1] to achieve ground point classification. Specifically:

[0034]

[0035] in, This is the first convolutional layer. This is the second convolutional layer. For batch normalization operations, For activation functions;

[0036] Construct a disparity optimization module based on the fusion feature layer Constructing location-encoded features :

[0037]

[0038] in, and This indicates that arrays W and H, each containing an element uniformly distributed from 0 to 1, are generated respectively. This means stacking the x and y coordinates along a new dimension to obtain a shape like... tensor; This indicates that the tensor will be copied. This yields the final normalized position code, with a size of [value missing]. The first channel represents the x-coordinate, and the second channel represents the y-coordinate.

[0039] The feature map and the ground classification map are resized using bilinear interpolation to ensure consistency. Then, location features and the ground classification map are encoded separately using convolutional layers to obtain the location-encoded features. and classification coding features ;

[0040] Location-encoded features are obtained through addition. With fusion feature layer Addition, combining classification coding features through multiplication operations. Adjusting parallax prediction :

[0041]

[0042] in, This represents the element-wise addition operation between two feature vectors at corresponding positions. The operation of multiplying two feature vectors element by element at corresponding positions;

[0043] The optimized disparity map is output through the convolutional layer.

[0044] According to the present invention, a method for calculating the height of ground features in satellite imagery based on monocular disparity estimation is provided. A loss function is constructed based on disparity prediction loss components, ground loss components, and orientation gradient loss components, including:

[0045] The disparity prediction loss components are:

[0046]

[0047] in, Let be the disparity prediction loss, representing the mean squared error between the predicted disparity map and the true disparity map. To predict disparity maps The disparity value corresponding to the location, For true disparity maps The disparity value corresponding to the location, These are the height and width of the image, respectively;

[0048] The ground loss component is:

[0049]

[0050] in, For ground classification loss, represents the binary cross-entropy loss between the predicted classification probability and the classification label. For classification diagram Category value of location, Predicted classification probability map The probability value of the location. These are the height and width of the image, respectively;

[0051] The directional gradient loss components are:

[0052]

[0053]

[0054] in, Let be the disparity directional gradient loss, representing the gradient loss function between the predicted disparity map and the true disparity map along the azimuth direction. For the direction vector directional derivative, direction vector Expressed in azimuth angle as , To predict disparity maps The disparity value corresponding to the location, For true disparity maps The disparity value corresponding to the location, It is expressed as mean squared error and is used to measure the difference between the predicted gradient and the true gradient.

[0055] Loss component predicted by disparity Ground loss component and directional gradient loss components Construct the overall loss function. :

[0056]

[0057] in, , , These are the corresponding loss weights.

[0058] According to the present invention, a method for calculating the height of ground features in satellite imagery based on monocular parallax estimation is provided. For network prediction results, a preset error evaluation index is established, including:

[0059] Establish root mean square error :

[0060]

[0061] in, This represents the number of samples in the validation set. For the first The true value of each sample For the first Predicted values ​​for each sample;

[0062] Establish the root mean square error of classification :

[0063]

[0064] in, For category index, To verify that the set belongs to the category The number of samples, To belong to category The The true value of each sample To belong to category The Predicted values ​​for each sample;

[0065] Establish the correlation coefficient R:

[0066]

[0067] in, This represents the number of samples in the validation set. For the first The true value of each sample For the first The predicted value for each sample, for The mean of the true values ​​of each sample for The mean of the predicted values ​​for each sample.

[0068] According to the present invention, a method for calculating the height of ground features in satellite imagery based on monocular parallax estimation is provided, which obtains a height point cloud based on the azimuth angle, the scale relationship, and the parallax map, including:

[0069]

[0070]

[0071] in, For proportional relationships, It is the azimuth angle. The original image coordinates, These are the correct image coordinates after parallax compensation. coordinates The disparity value at the corresponding location, For parallax compensation position The corresponding height value.

[0072] Secondly, the present invention also provides a satellite imagery feature height calculation system based on monocular parallax estimation, comprising:

[0073] The first calculation module is used to determine the pixel positions of the top and bottom of the ground features corresponding to the ground control points in the image based on the ground control point or known ground feature height data, combined with the RPC model of DTM and satellite imagery, and to calculate the azimuth angle.

[0074] The second calculation module is used to determine the proportional relationship between the parallax of a ground feature along the vertical direction in the image and its actual height, using ground control points or known ground feature height data.

[0075] The third calculation module is used to extract the disparity values ​​from the top to the bottom of ground features along the vertical direction from the oblique satellite image using a trained monocular disparity estimation network model, and generate a disparity map.

[0076] The fourth calculation module is used to obtain the height point cloud based on the azimuth angle, the proportional relationship, and the disparity map;

[0077] The fifth calculation module is used to perform preset processing on the elevation point cloud to generate an elevation map that is consistent with the geographical range of a single oblique satellite image.

[0078] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the satellite imagery feature height calculation method based on monocular parallax estimation as described above.

[0079] The method and system for calculating the height of ground features in satellite imagery based on monocular parallax estimation provided by this invention ensures the consistency of imagery and height information within the same temporal phase, making it suitable for time-sensitive monitoring of changes in ground height. By employing the monocular parallax estimation method, it solves the problem of geometric position distortion of objects caused by differences in viewing angle, thereby significantly improving the accuracy and stability of height recovery. This technology shows broad application prospects in the field of height recovery of monocular oblique satellite imagery. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0081] Figure 1 This is one of the flowcharts illustrating the satellite imagery feature height calculation method based on monocular parallax estimation provided by the present invention;

[0082] Figure 2 This is the second flowchart of the satellite imagery feature height calculation method based on monocular parallax estimation provided by the present invention;

[0083] Figure 3 This is a schematic diagram of the monocular depth estimation network structure provided by the present invention;

[0084] Figure 4 This is a schematic diagram comparing the results of the monocular disparity estimation algorithm provided by the present invention;

[0085] Figure 5 This is a schematic diagram of the result of restoring a height map from monocular parallax provided by the present invention.

[0086] Figure 6 This is a schematic diagram of the structure of the satellite image ground feature height calculation system based on monocular parallax estimation provided by the present invention;

[0087] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0089] Figure 1 This is one of the flowcharts illustrating the satellite imagery feature height calculation method based on monocular parallax estimation provided in this embodiment of the invention, such as... Figure 1 As shown, it includes:

[0090] Step 100: Based on ground control point or known ground feature height data, and combined with the RPC model of DTM and satellite imagery, determine the pixel positions of the top and bottom of the ground feature corresponding to the ground control point in the image, and calculate the azimuth angle;

[0091] Step 200: Determine the proportional relationship between the parallax of the ground feature along the vertical direction in the image and its actual height using ground control points or known ground feature height data;

[0092] Step 300: Use the trained monocular disparity estimation network model to extract the disparity values ​​from the top to the bottom of the ground features along the vertical direction from the oblique satellite imagery to generate a disparity map;

[0093] Step 400: Based on the azimuth angle, the scaling relationship, and the disparity map, obtain the elevation point cloud;

[0094] Step 500: Perform preset processing on the elevation point cloud to generate an elevation map that is consistent with the geographic range of a single oblique satellite image.

[0095] Understandably, previous monocular height estimation research has begun to focus on the concept of "geocentric attitude." Geocentric attitude describes the direction and height of a feature's center of gravity relative to the Earth's surface. In-depth analysis has revealed that the positional offset of a feature along the center of gravity direction in an image (i.e., the vertical offset from the top to the bottom of the feature), called "centroid parallax," is more suitable for monocular 3D reconstruction tasks using oblique satellite imagery than directly estimating feature height. The main reasons for this adaptability include the following:

[0096] (1) One-to-one stable relationship: There is a one-to-one correspondence between the centroid parallax and the image, which fundamentally avoids the many-to-one ill-conditioned problem common in traditional height estimation and improves the stability of the solution space of the task.

[0097] (2) Simple 3D reconstruction: Under the imaging model of satellite imagery with approximate parallel projection, an image corresponds to only a set of fixed azimuth and elevation angle parameters. Therefore, the 3D point cloud information of ground objects can be directly reconstructed through centroid parallax without the need for a complex nonlinear optimization process.

[0098] (3) Elevation deviation compensation: In a single tilted satellite image, the distortion of the geographical location of objects above the ground surface will cause deviation in the predicted elevation. The centroid parallax effectively compensates for this error caused by the imaging tilt by estimating the relative position of the top and bottom of the ground features, thereby improving the accuracy of elevation estimation.

[0099] (4) Transferability of the method: Compared with traditional monocular height estimation, the monocular estimation method based on centroid parallax is more likely to borrow the technical framework of monocular depth estimation. The difference is that depth estimation relies on perspective geometry, and the light direction of each pixel in the image is affected by different azimuth and elevation angles; while centroid parallax estimation is based on the principle of parallel projection, and the light direction of all pixels in the image shares the same azimuth and elevation angles, which significantly reduces the model complexity.

[0100] Based on the above advantages, this invention proposes a deep learning-based method for height restoration of tilted satellite images based on monocular parallax estimation. This method estimates the centroid parallax in the image instead of direct elevation estimation, and combines the RPC model of the satellite image to calculate the azimuth and elevation angle information, thereby accurately restoring the height information of each location point in the image.

[0101] Specifically, such as Figure 2 As shown, it includes the following steps:

[0102] Step 1, Azimuth Calculation: Based on ground control points or known feature height data, and combining a Digital Terrain Model (DTM) and an RPC model from satellite imagery, determine the pixel positions of the top and bottom of the features corresponding to the ground control points in the imagery. Calculate the azimuth angle. Azimuth, or the direction from the top of the feature to the bottom (also the direction of the plumb line in the image), provides a spatial reference for subsequent parallax analysis and height restoration. The specific calculation formula is as follows:

[0103] (1)

[0104] (2)

[0105] (3)

[0106] In equations (1), (2), and (3), , , , Let be the polynomial functions of the numerator and denominator in the RPC model. , The scaling factor in the normalization parameters provided for the RPC model. , Image coordinate offsets in the normalization parameters provided by the RPC model; these parameters are all provided by the RPC model. The coordinates of the ground control points, That is the surface elevation of the location of the feature. , , and Calculate the azimuth angle respectively Process variables.

[0107] Step 2: Parallax and Height Ratio Calculation. Using ground control points or known feature height data, determine the ratio k between the parallax D of a feature along the vertical direction in the image and its actual height H. The specific formula for calculating the ratio k is:

[0108] (4)

[0109] In equation (4), The height of the ellipsoid at the location of the feature; This is the surface elevation of the location of the feature, which can be obtained through DTM interpolation; and The results are obtained from equations (2) and (3) respectively.

[0110] Step 3: Disparity estimation information extraction. Using a trained monocular disparity estimation network model, the disparity values ​​from the top to the bottom of ground features along the vertical direction are automatically extracted from a single oblique satellite image, generating a disparity map D. Specific steps include:

[0111] Step 3.1. Dataset Preprocessing. The original dataset is divided into a training set, a validation set, and a test set. The training set and the validation set both include oblique satellite image patches and image azimuth information as input data, as well as disparity maps of the images along the vertical direction as label data. The test set only contains oblique satellite image patches and image azimuth information as input data.

[0112] This invention uses the DFC19 dataset as the basic data source for model training and validation. The original data includes: oblique satellite image patches of size 2048×2048, a ground truth map of disparity along the vertical direction with the same size as the image, azimuth information corresponding to the image, and the ratio information of disparity to height values, used to calculate terrain height from disparity. In addition, the DFC19 dataset also contains high-density 3D point cloud data. Through point cloud filtering and interpolation operations, a ground elevation map aligned with the image was generated. To adapt to the model training requirements, the original data underwent processing such as cropping, resampling, and splitting into training and test sets. The cropped image size was adapted to the model input requirements, resampling ensured data consistency, and the data was strictly divided into training and test sets according to the random partitioning principle. Finally, the DFC19 dataset contains 6450 sets of training data and 1500 sets of test data, with a preprocessed dataset size of 512×512.

[0113] Step 3.2. Design the Network Model. The network model uses the large-scale depth estimation model Depth Anything as its basic framework, leveraging its encoder to enhance the network's disparity estimation capability for oblique satellite imagery. The DPT Head in its decoding module is unfrozen, and a layer-by-layer optimization fusion module is designed within the decoding layer to extract high-resolution feature representations. Furthermore, a ground classification module is integrated at the end of the decoding module to predict ground point probabilities, and a disparity optimization module is designed by combining location encoding and ground classification information to further improve the accuracy of disparity prediction. The overall network structure is as follows: Figure 3 As shown:

[0114] Module 1. Encoder Module. This method uses a large-scale depth estimation model, Depth Anything, as the encoder to enhance the generalization ability of the disparity estimation task, and unfreezes the DPT Head module in its decoder, enabling the large-scale depth estimation model feature extraction network to achieve efficient feature reconstruction and prediction in monocular disparity estimation tasks of oblique satellite imagery.

[0115] Module 2. Decoding Layer Layer-by-Layer Optimization and Fusion Module. The decoding module mentioned in Module 1 consists of five layers. The first four layers have the same number of channels, and their sizes increase sequentially to 1 / 16, 1 / 8, 1 / 4, and 1 / 2 of the original image size. The fifth layer has 32 channels and its size is the same as the original image. For the first four layers, each layer uses a ResNet module to refine the features of the unfrozen decoding layer, resulting in feature maps. Indicates the first The decoded features of the layer, after the following operations, yield the fused features:

[0116] (5)

[0117] in, For the first Decoding feature map of the layer To fuse feature layers, , and The steps are residual, upsampling, and convolution, respectively; deep feature fusion and nonlinear mapping further enhance the network's ability to express details at different scales and improve its adaptability to complex terrain.

[0118] Module 3. Ground Classification Module. This module obtains the fifth layer features from the decoding layer described in Module 2. Next, a ground classification module is added. This module uses two 1x1 convolutional layers to... The process involves several steps. The first convolutional layer halves the number of input channels and uses batch normalization to improve training stability. The second convolutional layer compresses the number of channels to 1, outputting a single-channel feature map. Each pixel value represents the probability that the location is a ground point. Finally, the Sigmoid activation function is used to enhance the features and map the classification probability to the range [0, 1], thus achieving ground point classification. The specific implementation steps are as follows:

[0119] (6)

[0120] in, This is the first convolutional layer. This is the second convolutional layer. For batch normalization operations, This is the Sigmoid activation function.

[0121] Module 4. Disparity Optimization Module. This module assists in disparity estimation by optimizing disparity prediction through the combination of location coding and ground classification information.

[0122] First, based on the fusion features described in Module 2, position coding features are constructed. :

[0123] (7)

[0124] in, and This indicates that arrays W and H, each containing an element uniformly distributed from 0 to 1, are generated respectively. This means stacking the x and y coordinates along a new dimension to obtain a shape like... tensor; This indicates that the tensor will be copied. This yields the final normalized position code, with a size of [value missing]. The first channel represents the x-coordinate, and the second channel represents the y-coordinate.

[0125] Then, the size of the feature map and the ground classification map is adjusted to be consistent through bilinear interpolation, and the location features and the ground classification map are encoded separately through convolutional layers to obtain the location-encoded features. and classification coding features .

[0126] Next, the position-encoded features are combined with the fused features obtained from module 2 through an addition operation. The values ​​are added together, and the disparity prediction is further adjusted by combining ground classification features through multiplication operations. :

[0127] (8)

[0128] in, This represents the element-wise addition operation between two feature vectors at corresponding positions. The operation of multiplying two feature vectors element by element at corresponding positions;

[0129] Finally, an optimized disparity map is output through a convolutional layer.

[0130] Step 3.3: Construct the loss function. The loss function consists of several components, including: disparity prediction loss, which calculates the difference between the predicted disparity map and the true disparity map using mean squared error (MSE); ground classification loss, which uses a binary cross-entropy loss function to constrain the ground classification result; and directional gradient loss, which constrains the predicted disparity map along the feature direction based on the orientation information of the input image, thereby optimizing the structural consistency of the disparity map in one step. In this embodiment of the invention, the weights of the disparity prediction loss, ground classification loss, and directional gradient loss function are typically set to 1, 10, and 1, respectively.

[0131] The disparity prediction loss components are:

[0132] (9)

[0133] in, Let be the disparity prediction loss, representing the mean squared error between the predicted disparity map and the true disparity map. To predict disparity maps The disparity value corresponding to the location, For true disparity maps The disparity value corresponding to the location, These are the height and width of the image, respectively;

[0134] The ground loss component is:

[0135] (10)

[0136] in, For ground classification loss, represents the binary cross-entropy loss between the predicted classification probability and the classification label. For classification diagram Category value of location, Predicted classification probability map The probability value of the location. These are the height and width of the image, respectively;

[0137] The directional gradient loss components are:

[0138] (11)

[0139] (12)

[0140] in, Let be the disparity directional gradient loss, representing the gradient loss function between the predicted disparity map and the true disparity map along the azimuth direction. For the direction vector directional derivative, direction vector Expressed in azimuth angle as , To predict disparity maps The disparity value corresponding to the location, For true disparity maps The disparity value corresponding to the location, It is expressed as mean squared error and is used to measure the difference between the predicted gradient and the true gradient.

[0141] The overall loss function consists of the three loss components mentioned above, namely:

[0142] (13)

[0143] in, , , These are the corresponding loss weights.

[0144] Step 3.4: Establish error evaluation metrics. Based on the network prediction results, the established error evaluation metrics include: Root Mean Square Error (RMSE), which measures the overall deviation between the predicted disparity map and the true disparity map; Correlation Coefficient (R), which measures the linear correlation between the predicted disparity map and the true disparity map; and Classification Disparity Error, which categorizes disparities into three classes: low (disparity value less than 5), medium (disparity value between 5 and 15), and high (true disparity value greater than 15). The RMSE is calculated for each class of disparity to measure the model's prediction deviation across different disparity categories, comprehensively evaluating the model's adaptability and performance. The specific formula is:

[0145] Establish root mean square error :

[0146] (14)

[0147] in, This represents the number of samples in the validation set. For the first The true value of each sample For the first Predicted values ​​for each sample;

[0148] Establish the root mean square error of classification :

[0149] (15)

[0150] in, For category index, To verify that the set belongs to the category The number of samples, To belong to category The The true value of each sample To belong to category The Predicted values ​​for each sample;

[0151] Establish the correlation coefficient R:

[0152] (16)

[0153] in, This represents the number of samples in the validation set. For the first The true value of each sample For the first The predicted value for each sample, for The mean of the true values ​​of each sample for The mean of the predicted values ​​for each sample.

[0154] Step 3.5. Network Model Training. Input the training data into the network model and update the model parameters using the gradient descent optimization algorithm. Adjust the learning rate based on the convergence of the loss function during training, and monitor model performance in real time using the validation set to prevent overfitting. Through multiple iterations, the final model is obtained.

[0155] Step 3.6. Network Model Validation. Evaluate the performance of the trained network model on the validation set. Quantitatively analyze the model's disparity prediction ability using evaluation metrics, and verify the consistency between the network prediction results and the actual results using visualization methods.

[0156] Step 3.7. Network Model Testing. Input the test set data into the trained network model to generate disparity prediction results. Post-process the prediction results (such as denoising, interpolation, etc.) to optimize the final disparity map.

[0157] After completing the network design, the preprocessed training set is input, and the loss function is calculated using the backpropagation algorithm. Parameter learning is performed by iteratively reducing the error to obtain the optimal weight model. In actual training, the PyTorch Lightening structure is used, with a maximum iteration count of 200, a batch processing parameter of 4, and the Adam optimizer with a learning rate of 0.0001. The pre-trained weight model is loaded, and the test set is input into the network model to obtain a monocular disparity map. The obtained disparity map is compared with the ground truth disparity map to evaluate the weight model. It is also compared with five other methods (img2height, MQTransformer, geocentric pose, zoedepth, and depthanything). The comparison results are shown in Table 1. The disparity map comparison results are displayed as follows: Figure 4 As shown.

[0158] Table 1 Comparison of Monocular Disparity Estimation Results for DFC19 Dataset

[0159]

[0160] As shown in Table 1, in the monocular disparity estimation task, this invention outperforms the other five methods in all aspects, especially in complex scenes where the ground truth disparity is greater than 10. The accuracy can be further improved by using a large model (vitl) and gradient directional enhancement (grad), demonstrating better accuracy and robustness. Figure 4 The results of five comparative methods and the present invention using different models are presented, which also demonstrate the advantages of the present invention.

[0161] Step 4, height point cloud extraction, based on the azimuth angle obtained in step 1.1. Based on the scaling factor k obtained in step 1.2, and combined with the disparity map D estimated by the network model in step 1.3, the height point cloud is obtained. The specific calculation formula is as follows:

[0162] (17)

[0163] (18)

[0164] in, For proportional relationships, It is the azimuth angle. The original image coordinates, These are the correct image coordinates after parallax compensation. coordinates The disparity value at the corresponding location, For parallax compensation position The corresponding height value.

[0165] Step 5: Post-processing of the elevation point cloud. The obtained elevation point cloud is filtered, an irregular triangular network is constructed and interpolated, etc., to finally generate a elevation map consistent with the geographic range of the image.

[0166] Figure 5 The process of recovering a height map from a disparity map is demonstrated: First, by combining the azimuth angle and the disparity-to-height ratio, the disparity map is recovered into a height point cloud; then, the height point cloud is filtered to remove point clouds with a density of less than 5 points in a spatial range of 2, resulting in a filtered height point cloud; finally, interpolation is performed to obtain the height map. The height map recovered from the disparity map is compared with the results of four training methods directly using the ground truth height map values. The comparison results are shown in Table 2. The "invention" mentioned in Table 2 refers to the result of converting the disparity result into a height using the Vitl model and the directional gradient loss (grad).

[0167] Table 2 Comparison results of height maps in the DFC19 dataset

[0168]

[0169] As shown in Table 2, the present invention performs better in scenarios with a true parallax greater than 10, and its overall performance surpasses that of other methods. This indicates that the present invention not only possesses higher accuracy when handling complex scenarios with large parallax, but also has better adaptability and application potential.

[0170] The satellite imagery feature height calculation system based on monocular parallax estimation provided by this invention is described below. The satellite imagery feature height calculation system based on monocular parallax estimation described below can be referred to in correspondence with the satellite imagery feature height calculation method based on monocular parallax estimation described above.

[0171] Figure 6 This is a schematic diagram of the structure of the satellite imagery ground feature height calculation system based on monocular parallax estimation provided in an embodiment of the present invention, as shown below. Figure 6 As shown, it includes: a first calculation module 61, a second calculation module 62, a third calculation module 63, a fourth calculation module 64, and a fifth calculation module 65, wherein:

[0172] The first calculation module 61 is used to determine the pixel positions of the top and bottom of the ground control point corresponding to the ground feature in the image based on ground control point or known ground feature height data, combined with the DTM and satellite image RPC model, and calculate the azimuth angle; the second calculation module 62 is used to determine the proportional relationship between the parallax of the ground feature in the image along the vertical direction and the actual height through ground control point or known ground feature height data; the third calculation module 63 is used to extract the parallax value from the top to the bottom of the ground feature along the vertical direction from the oblique satellite image using a trained monocular parallax estimation network model, and generate a parallax map; the fourth calculation module 64 is used to obtain the height point cloud based on the azimuth angle, the proportional relationship and the parallax map; the fifth calculation module 65 is used to perform preset processing on the height point cloud to generate a height map consistent with the geographical range of a single oblique satellite image.

[0173] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call logic instructions in the memory 730 to execute a satellite image feature height calculation method based on monocular disparity estimation. This method includes: determining the pixel positions of the top and bottom of the feature corresponding to the ground control point in the image based on ground control point or known feature height data, combined with a DTM and satellite image RPC model, and calculating the azimuth angle; determining the proportional relationship between the disparity of the feature along the vertical direction in the image and its actual height using the ground control point or known feature height data; extracting the disparity value from the top to the bottom of the feature along the vertical direction from the oblique satellite image using a trained monocular disparity estimation network model, and generating a disparity map; obtaining a height point cloud based on the azimuth angle, the proportional relationship, and the disparity map; and performing pre-processing on the height point cloud to generate a height map consistent with the geographical range of a single oblique satellite image.

[0174] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calculating the height of ground features in satellite imagery based on monocular parallax estimation, characterized in that, include: Based on ground control point or known ground feature height data, combined with the digital terrain model (DTM) and the rational polynomial coefficient (RPC) model of satellite imagery, the pixel positions of the top and bottom of the ground features corresponding to the ground control points in the imagery are determined, and the azimuth angle is calculated. By using ground control points or known feature height data, determine the proportional relationship between the parallax of features along the vertical direction in the image and their actual height. The disparity values ​​from the top to the bottom of ground features along the vertical direction are extracted from the oblique satellite image using a trained monocular disparity estimation network model to generate a disparity map. Based on the azimuth angle, the scaling relationship, and the disparity map, a height point cloud is obtained; The elevation point cloud is pre-processed to generate a elevation map that matches the geographic range of a single oblique satellite image; Based on the azimuth angle, the scaling relationship, and the disparity map, a height point cloud is obtained, including: in, For proportional relationships, It is the azimuth angle. The original image coordinates, These are the correct image coordinates after parallax compensation. coordinates The disparity value at the corresponding location, For parallax compensation position The corresponding height value.

2. The method for calculating the height of ground features in satellite imagery based on monocular parallax estimation according to claim 1, characterized in that, Based on ground control point or known feature height data, and combined with the RPC model of DTM and satellite imagery, the pixel positions of the top and bottom of the features corresponding to the ground control points in the imagery are determined, and the azimuth angle is calculated, including: in, , , , These are the polynomial functions of the numerator and denominator in the RPC model, respectively. , The scaling factor in the normalization parameters provided for the RPC model. , Image coordinate offsets in the normalization parameters provided for the RPC model. The coordinates of the ground control points, This represents the surface elevation of the location of the feature. , , and Calculate the azimuth angle respectively Process variables.

3. The method for calculating the height of ground features in satellite imagery based on monocular parallax estimation according to claim 2, characterized in that, Using ground control points or known feature height data, determine the proportional relationship between the parallax of a feature along the vertical direction in the image and its actual height, including: in, This represents the ellipsoidal height of the location of the corresponding ground feature in the coordinates of the ground control point. This represents the surface elevation of the location of the feature, obtained through DTM interpolation. It is a proportional relationship.

4. The method for calculating the height of ground features in satellite imagery based on monocular parallax estimation according to claim 1, characterized in that, Using a trained monocular disparity estimation network model, disparity values ​​along the vertical direction from the top to the bottom of ground features are extracted from oblique satellite imagery to generate a disparity map, including: Obtain a raw dataset consisting of multiple oblique satellite images, and divide the raw dataset into a training set, a validation set, and a test set. The training set and validation set contain oblique satellite image patches, image azimuth information, and label data, while the test set contains oblique satellite image patches and image azimuth information. Using the large-scale depth estimation model Depth Anything as the basic framework, a monocular disparity estimation network model is constructed. A loss function is constructed based on the disparity prediction loss component, the ground loss component, and the orientation gradient loss component; Establish preset error evaluation indicators based on the network prediction results; The training set is input into the monocular disparity estimation network model, the gradient descent optimization algorithm is used to update the model parameters, the learning rate is adjusted according to the convergence of the loss function during training, the model performance is monitored in real time using the validation set to prevent overfitting, and through multiple rounds of iteration, the trained monocular disparity estimation network model is obtained. The performance of the trained network model is evaluated on the validation set. The disparity prediction ability of the model is quantitatively analyzed by a preset error evaluation index, and the consistency between the network prediction results and the real results is verified by visualization. The test set data is input into the trained network model to generate disparity prediction results. The disparity prediction results are then post-processed to obtain an optimized disparity map.

5. The method for calculating the height of ground features in satellite imagery based on monocular parallax estimation according to claim 4, characterized in that, Using the large-scale depth estimation model Depth Anything as the basic framework, a monocular disparity estimation network model is constructed, including: The input image is encoded using the encoder of the large-scale depth estimation model Depth Anything. The DPT Head module of Depth Anything and the last four layers of the DINO-v2 pre-trained model are then unfrozen for feature reconstruction and prediction. A layer-by-layer optimization and fusion module for the decoding layer is constructed. This module comprises five layers, with the first four layers having the same number of channels, and their sizes increasing sequentially to the size of the original image. Layers 1 / 16, 1 / 8, 1 / 4, and 1 / 2, with the fifth layer having 32 channels and the same size as the original image, are used. For the first four layers, each layer refines the features of the unfrozen decoded layer using a ResNet module. Through progressive upsampling, convolution, concatenation, and convolution operations, the fused feature layer is obtained as follows: in, For the first Decoding feature map of the layer To fuse feature layers, , and In order, they are residuals, upsampling, and convolution; Construct a ground classification module, using the fifth layer of features in the decoding layer. Then, a ground classification module was added, which uses two 1x1 convolutional layers to classify the ground. The process involves halving the number of input channels in the first convolutional layer and improving training stability through batch normalization. The second convolutional layer compresses the number of channels to 1, outputting a single-channel feature map. Each pixel value represents the probability that the corresponding location is a ground point. A sigmoid activation function is used to enhance features and map the classification probability to the [0,1] range to achieve ground point classification. Specifically: in, This is the first convolutional layer. This is the second convolutional layer. For batch normalization operations, Use the Sigmoid activation function; Construct a disparity optimization module based on the fusion feature layer Constructing location-encoded features : in, and This indicates that arrays W and H, each containing an element uniformly distributed from 0 to 1, are generated respectively. This means stacking the x and y coordinates along a new dimension to obtain a shape like... tensor; This indicates that the tensor will be copied. This yields the final normalized position code, with a size of [value missing]. The first channel represents the x-coordinate, and the second channel represents the y-coordinate. The feature map and the ground classification map are resized using bilinear interpolation to ensure consistency. Then, location features and the ground classification map are encoded separately using convolutional layers to obtain the location-encoded features. and classification coding features ; Location-encoded features are obtained through addition. With fusion feature layer Addition, combining classification coding features through multiplication operations. Adjusting parallax prediction : in, This represents the element-wise addition operation between two feature vectors at corresponding positions. The operation of multiplying two feature vectors element by element at corresponding positions; The optimized disparity map is output through the convolutional layer.

6. The method for calculating the height of ground features in satellite imagery based on monocular parallax estimation according to claim 4, characterized in that, Based on the disparity prediction loss component, the ground loss component, and the orientation gradient loss component, a loss function is constructed, including: The disparity prediction loss components are: in, Let be the disparity prediction loss, representing the mean squared error between the predicted disparity map and the true disparity map. To predict disparity maps The disparity value corresponding to the location, For true disparity maps The disparity value corresponding to the location, These are the height and width of the image, respectively; The ground loss component is: in, For ground classification loss, represents the binary cross-entropy loss between the predicted classification probability and the classification label. For classification diagram Category value of location, Predicted classification probability map The probability value of the location. These are the height and width of the image, respectively; The directional gradient loss components are: in, Let be the disparity directional gradient loss, representing the gradient loss function between the predicted disparity map and the true disparity map along the azimuth direction. For the direction vector directional derivative, direction vector Expressed in azimuth angle as , To predict disparity maps The disparity value corresponding to the location, For true disparity maps The disparity value corresponding to the location, It is expressed as mean squared error and is used to measure the difference between the predicted gradient and the true gradient. Loss component predicted by disparity Ground loss component and directional gradient loss components Construct the overall loss function. : in, , , These are the corresponding loss weights.

7. The method for calculating the height of ground features in satellite imagery based on monocular parallax estimation according to claim 4, characterized in that, Based on the network prediction results, pre-defined error evaluation indicators are established, including: Establish root mean square error : in, This represents the number of samples in the validation set. For the first The true value of each sample For the first Predicted values ​​for each sample; Establish the root mean square error of classification : in, For category index, To verify that the set belongs to the category The number of samples, To belong to category The The true value of each sample To belong to category The Predicted values ​​for each sample; Establish the correlation coefficient R: in, This represents the number of samples in the validation set. For the first The true value of each sample For the first The predicted value for each sample, for The mean of the true values ​​of each sample for The mean of the predicted values ​​for each sample.

8. A system for calculating the height of ground features in satellite imagery based on monocular parallax estimation, comprising the method for calculating the height of ground features in satellite imagery based on monocular parallax estimation as described in any one of claims 1 to 7, characterized in that, include: The first calculation module is used to determine the pixel positions of the top and bottom of the ground features corresponding to the ground control points in the image based on the ground control point or known ground feature height data, combined with the RPC model of DTM and satellite imagery, and to calculate the azimuth angle. The second calculation module is used to determine the proportional relationship between the parallax of a ground feature along the vertical direction in the image and its actual height, using ground control points or known ground feature height data. The third calculation module is used to extract the disparity values ​​from the top to the bottom of ground features along the vertical direction from the oblique satellite image using a trained monocular disparity estimation network model, and generate a disparity map. The fourth calculation module is used to obtain the height point cloud based on the azimuth angle, the proportional relationship, and the disparity map; The fifth calculation module is used to perform preset processing on the elevation point cloud to generate an elevation map that is consistent with the geographical range of a single oblique satellite image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the satellite imagery feature height calculation method based on monocular parallax estimation as described in any one of claims 1 to 7.

Images