Remote sensing image RPC parameter repair method, system and electronic equipment based on deep learning

Abstract

This invention belongs to the field of remote sensing image processing technology and discloses a deep learning-based RPC parameter restoration method, comprising: S1 acquiring a sample set, the sample set including multiple data sample groups, each data sample group including source sample image, target sample image and corresponding ground control point sample data; and preprocessing the sample set to obtain a preprocessed sample set; S2: constructing an initial RPC restoration network, the initial RPC restoration network including a first sub-network and a second sub-network; S3 training the network model, using the preprocessed sample set to train the initial RPC restoration network to obtain a trained RPC restoration network; S4 using the trained RPC restoration network to perform PPC parameter restoration on the image to be restored to obtain restored RPC parameters; the image to be restored includes a source image and a target image; S5 performing geometric correction on the image to be restored based on the restored RPC parameters to obtain the restored image.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and particularly relates to a method, system and electronic device for repairing RPC parameters of remote sensing images based on deep learning. Background Technology

[0002] In the era of remote sensing big data, the volume of satellite remote sensing data continues to grow, with data sources becoming increasingly diversified and the update speed accelerating. The rapid and accurate processing and analysis of this massive amount of remote sensing imagery data is becoming increasingly important. Geometric correction is a method of refining the positional accuracy of satellite imagery. Its purpose is to correlate each pixel in the image with its corresponding geographical location on the Earth's surface, eliminating the influence of factors such as the Earth's curvature and topographical changes on the image's positional accuracy, and providing accurate geographic information for subsequent analysis and application of remote sensing data.

[0003] Rational Polynomial Coefficients (RPC) models are a widely used method for geometric correction of satellite imagery. The RPC model is built using a "terrain-independent" approach, meaning a rigorous geometric model is established using satellite orbit parameters determined by onboard GPS and attitude parameters measured by a star camera and inertial measurement unit. This rigorous geometric model then generates a large number of uniformly distributed virtual ground control points, which are used to calculate the RPC parameters. Since the RPC model is generated from these uniformly distributed virtual ground control points based on the rigorous geometric model, any errors in estimating the virtual control points during the geometric process will directly lead to errors in the parameters fitted by the RPC model. This results in an inaccurate RPC model, ultimately causing image distortion when orthorectifying using the RPC model. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method for repairing RPC parameters of remote sensing images based on deep learning. By constructing an RPC repair network model, the original RPC parameters are corrected or estimated, eliminating problems such as reduced geometric positioning accuracy caused by model errors, and improving the geometric positioning accuracy of high-resolution satellite images.

[0005] To achieve the above technical objectives, this invention provides a method for repairing RPC parameters in remote sensing images based on deep learning, comprising the following steps:

[0006] S1 acquires a sample set, which includes multiple data sample groups, each of which includes source sample images, target sample images, and corresponding ground control point sample data; and preprocesses the sample set to obtain a preprocessed sample set.

[0007] S2: Construct an initial RPC repair network, which includes a first subnetwork and a second subnetwork;

[0008] S3 network model training: The initial RPC repair network is trained using a preprocessed sample set to obtain the trained RPC repair network.

[0009] S4 uses a trained PRC inpainting network to perform PPC parameter inpainting on the image to be inpainted, and obtains the inpainted RPC parameters; the image to be inpainted includes the source image and the target image.

[0010] S5 performs geometric correction on the image to be repaired based on the repaired RPC parameters to obtain the repaired image.

[0011] Specifically, in step S2, the first and second sub-networks of the PRC restoration network both include a feature extraction module, a correlation calculation module, and an iterative RPC estimation module; the second sub-network also includes an image distortion module.

[0012] Specifically, the feature extraction module in step S2 includes a first feature extraction submodule and a second feature extraction submodule, used to extract features from the image and obtain feature maps; the correlation calculation module is used to calculate the correlation between feature maps; the iterative RPC estimation module is used to obtain the iterative RPC parameters; and the image distortion module is used to distort the image to obtain a distorted image.

[0013] Specifically, the iterative RPC estimation module in step S2 includes a coordinate projection submodule, a correlation update submodule, a global motion aggregation submodule, and an RPC update submodule.

[0014] Specifically, step S4 further includes:

[0015] S41 inputs the source image I and the target image G into the first feature extraction submodule and the second feature extraction submodule in the first subnetwork, respectively, to obtain the feature map F of the first source image. I_1 Feature map F of the first target image G_1 ;

[0016] S42 will generate the feature map F of the source image. I_1 Feature map F of the target image G_1 The correlation between feature maps is calculated by inputting them into the correlation calculation module of the first sub-network to obtain the first correlation volume C1 and C1. 1 / 2 ;

[0017] S43 will use the first relevant volume C1 and C1 1 / 2 Input the RPC estimation module in the first sub-network to obtain the RPC parameters P1 updated in the first iteration;

[0018] S44 inputs the preprocessed target image into the image warping module in the second sub-network for warping processing, resulting in a distorted target image. ;

[0019] S45 will process the pre-processed source image and the distorted target image. The first and second feature extraction submodules in the second subnetwork are respectively input to obtain the feature maps F of the second source image. I_2 Feature map F of the second target image G_2 ;

[0020] S46 will use the feature map F of the second source image I_2 Feature map F of the second target image G_2 The correlation calculation module in the second sub-network is used to calculate the correlation between feature maps, resulting in the second correlation volume C. 1_2 and C 1_2 1 / 2 ;

[0021] S47 will use the second related volume C1 and C1 1 / 2 Input the iterative RPC estimation module in the second sub-network to obtain the RPC parameters P2 updated in the second iteration;

[0022] S48 calculates the repaired RPC parameters using the geometric consistency constraint method based on the RPC parameters P1 updated in the first iteration and P2 updated in the second iteration.

[0023] Specifically, sub-step S43 in step S4 further includes:

[0024] S431 uses a coordinate projection submodule to project the feature map F of the first source image. I_1 The coordinate set is mapped to the feature map F of the first target image. G_1 The corresponding projected coordinate set X1' k Above, and based on the coordinate set X1 and the projected coordinate set X1' k The pixel-by-pixel displacement vector F1 is obtained from the calculation. k ;

[0025] S432 will use the first relevant volume C1 and C1 1 / 2 The input correlation update submodule uses the projected coordinate set X1' k For the first relevant volumes C1 and C1 1 / 2 Sampling is performed to generate the first correlation slice S1. k ;

[0026] S433 will slice the correlation into S1 k and pixel-by-pixel displacement vector F1 kInput the global motion aggregation submodule to estimate the first residual displacement vector ∆D1 k ;

[0027] S434 will use the first residual displacement vector ∆D1 k Input the RPC update submodule to obtain the first updated RPC parameter P1. k +1 ;

[0028] S435 Repeat steps S441-S444 until the first residual displacement vector ∆D1 is obtained. k Once the system stabilizes, the RPC parameters P1 for the first iteration update are obtained.

[0029] Specifically, the preprocessing in step S1 includes normalizing and standardizing the ground control point data, cropping the image data, and converting the target image to grayscale; wherein the ground control point data consists of the pixel coordinates of the image and the corresponding geographic coordinates.

[0030] The second objective of this invention is to provide a deep learning-based RPC parameter repair system, characterized in that the system comprises:

[0031] The sample set construction module is configured to acquire a sample set, which includes multiple data sample groups, each of which includes source sample images, target sample images, and corresponding ground control point sample data; and to preprocess the sample set to obtain a preprocessed sample set.

[0032] An initial RPC repair network construction module is configured such that the initial RPC repair network includes a first sub-network and a second sub-network;

[0033] The network model training module is configured to train the initial RPC repair network using a preprocessed sample set to obtain the trained RPC repair network.

[0034] The PRC parameter repair module is configured to use a trained PRC repair network to repair the PPC parameters of the image to be repaired, thereby obtaining the repaired PPC parameters; the image to be repaired includes the source image and the target image.

[0035] The image restoration module is configured to perform geometric correction on the image to be restored based on the restored RPC parameters, thereby obtaining the restored image.

[0036] A third objective of this invention is to provide an electronic device, characterized in that it comprises:

[0037] A memory is used to store computer control instructions; a processor is connected to the memory and is used to retrieve and execute the computer control instructions so that the electronic device performs the deep learning-based RPC parameter repair method as described above.

[0038] Compared with existing technologies, the advantages and beneficial effects of this invention are as follows: By constructing an RPC repair network model containing two parallel sub-networks, this network model includes a feature extraction module, a correlation calculation module, and an iterative RPC estimation module; through this network model, automatic iterative updates of RPC parameters are achieved, obtaining RPC parameters at different resolution scales; and the fused RPC parameters are obtained using a geometric consistency constraint method, thereby improving the accuracy of the RPC model. This eliminates problems such as reduced geometric positioning accuracy caused by model errors, thus improving the geometric positioning accuracy of high-resolution satellite imagery. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart of a remote sensing image RPC parameter repair method based on deep learning provided in an embodiment of the present invention;

[0041] Figure 2 This is a model structure diagram of the RPC repair network provided in an embodiment of the present invention;

[0042] Figure 3 This is a structural diagram of the iterative RPC estimation module in the RPC repair network provided in this embodiment of the invention;

[0043] Figure 4 A framework diagram of a deep learning-based remote sensing image RPC parameter repair system provided in an embodiment of the present invention;

[0044] Figure 5 A framework diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0046] An embodiment of the present invention provides a method for repairing RPC parameters of remote sensing images based on deep learning, comprising the following steps:

[0047] S1 acquires a sample set, which includes multiple data sample groups, each of which includes source sample images, target sample images, and corresponding ground control point sample data; and preprocesses the sample set to obtain a preprocessed sample set.

[0048] Specifically, the preprocessing in step S1 includes normalizing and standardizing the ground control point data, cropping the image data, and converting the target image to grayscale; wherein the ground control point data consists of the pixel coordinates of the image and the corresponding geographic coordinates.

[0049] S2: Construct an initial RPC repair network, which includes a first subnetwork and a second subnetwork;

[0050] In the embodiments of the present invention, the first and second sub-networks of the PRC restoration network in step S2 both include a feature extraction module, a correlation calculation module, and an iterative RPC estimation module; the second sub-network also includes an image distortion module. The feature extraction module includes a first feature extraction sub-module and a second feature extraction sub-module, used to extract features from the image and obtain feature maps; the correlation calculation module is used to calculate the correlation between feature maps; the feature extraction sub-modules are all CNN networks, specifically including: 1 convolutional block (7×7), 2 basic units, and 1 linear convolutional block (1×1); the basic unit includes 1 max pooling layer (stride of 2) and 2 residual blocks. The correlation calculation module calculates the correlation between the extracted feature maps, i.e., the correlation volume C. The correlation volume is a multi-dimensional data structure generated by calculating the pairwise correlation between the feature maps of the source image and the target image. Specifically, the correlation volume contains the correlation information between each position on the feature map of the source image and all positions on the feature map of the target image, indexed by two-dimensional coordinate positions. The correlation calculation module also includes a flat pooling layer, which performs average pooling on the obtained correlation volume C to obtain another correlation volume C. 1 / 2 The iterative RPC estimation module is used to obtain the RPC parameters after iteration; the image distortion module is used to distort the image to obtain a distorted image.

[0051] In an embodiment of the present invention, the iterative RPC estimation module in step S2 includes a coordinate projection submodule, a correlation update submodule, a global motion aggregation submodule, and an RPC update submodule. The coordinate projection submodule maps the coordinate set X of the source image feature map to the corresponding projected coordinate set X' on the target grayscale image feature map, and obtains a pixel-by-pixel displacement vector F. The correlation update submodule uses the projected coordinates X'... k For the first relevant volumes C and C 1 / 2 Sampling is performed to generate the first correlation slice S. The global motion aggregation submodule estimates the residual displacement vector ∆D based on the correlation slice S and the pixel-by-pixel displacement vector F, where the displacement vector corresponds to the four corner points of the image. The global motion aggregation submodule includes multiple basic units and one convolutional block; each basic unit includes: one 3×3 convolutional block, one set of normalized + ReLU, and one max pooling layer (stride of 2). The basic units are used in series to continuously reduce the spatial resolution of the input features until it reaches 2×2. The residual displacement vector is a correction to the current RPC parameter estimate, used to update the RPC parameters to approximate the true RPC correction parameters. The RPC update submodule updates the RPC parameters based on the obtained residual displacement vector ∆D, gradually bringing them closer to the true RPC correction parameters.

[0052] S3 network model training: The initial RPC repair network is trained using a preprocessed sample set to obtain the trained RPC repair network.

[0053] In an embodiment of the present invention, the training process of the RPC parameter repair network includes:

[0054] The source and target sample images are sequentially input into the global motion aggregation submodules of the first and second subnetworks of the initial RPC parameter inpainting network to obtain the first estimated residual displacement D1 and the second estimated residual displacement D2. Based on the first and second estimated residual displacements D1 and D2, the parameters of the first and second subnetworks are updated according to the loss functions L1, L2, and the comprehensive loss function L, respectively. This process is iterated until both loss functions L1 and L2 converge, resulting in the trained RPC inpainting network. The loss function is calculated by measuring the actual ground displacement D in each iteration. gt The difference between the estimated residual displacement D and the actual residual displacement D is obtained, and the specific calculation method is as follows:

[0055] The loss function L1 for the first subnetwork:

[0056]

[0057] The loss function L2 of the second subnetwork:

[0058]

[0059] Comprehensive loss function L:

[0060] L = γL1 + (1-γ)L2

[0061] Where K represents the total number of iterations at one resolution scale; k represents the number of iterations, ranging from [0, K-1]; α and β are weight parameters, where α and β < 1; D1 k+1 and D2 k+1 D represents the residual displacement cubes obtained by the first and second subnetworks after (k+1) iterations, respectively; gt This indicates the actual displacement of the image.

[0062] S4 uses a trained PRC inpainting network to perform PPC parameter inpainting on the image to be inpainted, and obtains the inpainted RPC parameters; the image to be inpainted includes the source image and the target image.

[0063] Specifically, step S4 further includes:

[0064] S41 inputs the source image I and the target image G into the first feature extraction submodule and the second feature extraction submodule in the first subnetwork, respectively, to obtain the feature map F of the first source image. I_1 (F) I_1 ∈R D×H×W ) and the feature map F of the first target image G_1 (F) G_1 ∈R D×H×W );

[0065] Where D represents the dimension of the feature image, and H and W represent the height and width of the feature image.

[0066] S42 will generate the feature map F of the source image. I_1 Feature map F of the target image G_1 The correlation between feature maps is calculated by inputting them into the correlation calculation module of the first sub-network to obtain the first correlation volume C1 and C1. 1 / 2 The specific process includes:

[0067] (1) For the feature map F' of the source image I_1 The position x of each point on I_1 (The coordinate position is represented as (u) I_1 ,v I_1 )) and the feature map F' of the target image G_1 The position x of each point on G_1 (The coordinate position is represented as (u) G_1 ,v G_1The correlation between them is calculated by multiplying the feature vectors at the two locations and then applying a ReLU activation function to the result for non-linear processing, thereby enhancing the response to significant matching relationships. The mathematical expression is as follows:

[0068]

[0069] in, Indicates the feature map position x in the source image I_1 Feature map position x of the feature map of the target image G_1 Correlation score between them; x I_1 and x G_1 This indicates the coordinate positions of the feature maps of the source and target images.

[0070] (2) By traversing the source image feature map F I_1 All positions x on I_1 and target gray image feature map F G-1 All positions x on G_1 Calculate all possible pair of locations (x) I_1 ,x G_1 The correlation scores between the features are arranged according to their coordinates on the feature map, forming a four-dimensional data structure, namely the first correlation volume C1; the obtained correlation volume C1 is input into the average pooling layer, and another first correlation volume C1... 1 / 2 One of the first correlated volumes C1 has dimensions of H×W×H×W, and the other first correlated volume C1... 1 / 2 The dimensions are H×W×H / 2×W / 2, where each position (u I_1 ,v I_1 ,u G_1 ,v G_1 A correlation score corresponds to the source image feature map F. I_1 Position on (u) I_1 ,v I_1 ) and target image feature map F G-1 Position on (u) G_1 ,v G_1 The similarity between ) is calculated. Average pooling yields the second correlation volume C1. 1 / 2 The specific steps are as follows: Perform average pooling on the last dimension (width) of the first relevant volume C1 with a stride of 2, reducing the original width W to W / 2. Similarly, perform average pooling on the second-to-last dimension (height), reducing the original height H to H / 2. Thus, the original four-dimensional first relevant volume C1 has dimensions H×W×H×W. After pooling, the newly generated pooled first relevant volume C1... 1 / 2 The dimensions become H×W×H / 2×W / 2.

[0071] S43 will use the first relevant volume C1 and C1 1 / 2 The input is fed into the iterative RPC estimation module in the first sub-network to obtain the RPC parameters P1 updated in the first iteration; the specific process is as follows:

[0072] In this embodiment of the invention, step S43 further includes:

[0073] S431 uses a coordinate projection submodule to project the feature map F of the first source image. I_1 The coordinate set is mapped to the feature map F of the first target image. G_1 The corresponding projected coordinate set X1' k Above, and based on the coordinate set X1 and the projected coordinate set X1' k The pixel-by-pixel displacement vector F1 is obtained from the calculation. k ;

[0074] The RPC parameter P1 is used in the coordinate projection submodule. k (k=0,1,...,n) The source image feature map F G-1 The coordinate set X1 (x1=(u1,v1), x1∈X1) is mapped to the target grayscale image feature map F. G-1 The corresponding projected coordinate set X1' k (x1') k =(u1' k ,v1' k ), x1' k ∈X1' k ), and obtain the pixel-by-pixel displacement vector F1 k The mathematical expression is:

[0075] X1' k =P1 k • X1, F1 k =X1' k - X1

[0076] Among them, the pixel-by-pixel displacement vector F1 k This indicates the difference between the predicted and actual locations of feature points in the source image and their corresponding locations in the target image.

[0077] S432 will use the first relevant volume C1 and C1 1 / 2 The input correlation update submodule uses the projected coordinate set X1' k For the first relevant volumes C1 and C1 1 / 2 Sampling is performed to generate the first correlation slice S1. k ;

[0078] The first relevant volumes C1 and C1 1 / 2The input correlation update submodule uses the projected coordinate set X1' k For the first relevant volumes C1 and C1 1 / 2 Sampling is performed to generate the first correlation slice S1. k The specific process is as follows:

[0079]

[0080] in, Indicates x1' k A local square grid with radius r centered at r.

[0081] The first correlation slice refers to a two-dimensional slice cut from the multidimensional correlation volume. It represents the set of correlation values ​​calculated within the search window corresponding to the target feature map after projecting a certain position on the source feature map according to the currently estimated RPC parameters in a specific iteration step. For each position x1 on the feature map corresponding to the source sample image, the first correlation slice S1... k It records the correlation scores of all pixels in a neighborhood of size r×r centered at the projection point x1' on the feature map corresponding to the target sample image with respect to the source position x1.

[0082] S433 will slice the correlation into S1 k and pixel-by-pixel displacement vector F1 k Input the global motion aggregation submodule to estimate the first residual displacement vector ∆D1 k ;

[0083] Slice the first correlation segment S1 k and pixel-by-pixel displacement vector F1 k Input the global motion aggregation submodule to estimate the residual displacement vector ∆D1 k Obtain the residual displacement vector ∆D1 k The specific process is as follows: slice the first correlation slice S1 k and pixel-by-pixel displacement vector F1 k Input the global motion aggregation submodule; slice the correlation of the input S1 through a series of convolution and pooling operations. k and pixel-by-pixel displacement vector F1 k Deep feature extraction is performed to extract features that can characterize the global motion pattern, resulting in feature map F. L_1 When feature map F L_1 When the spatial resolution is reduced to 2×2, another convolutional block maps the feature map into a 2×2×2 cube, i.e., the residual displacement vectors ∆D1 at the four corner points. k .

[0084] S434 will use the first residual displacement vector ∆D1 kInput the RPC update submodule to obtain the first updated RPC parameter P1. k +1 ;

[0085] The residual displacement vector ∆D1 k Input the RPC update submodule to obtain the updated RPC parameters P1. k+1 Get the updated RPC parameters P1. k+1 The specific process is as follows: The residual displacement vector ∆D1 obtained from the global motion aggregation submodule is... k The input RPC update submodule is based on the input residual displacement vector ∆D1. k According to formula D1 k+1 = D1 k + ∆D1 k Update the displacement cube D1; use methods such as least squares or direct linear transformation based on the updated D. k+1 The RPC parameter P is calculated. k+1 ;

[0086] S435 Repeat steps S441-S444 until the first residual displacement vector ∆D1 is obtained. k Once the system stabilizes, the RPC parameters P1 for the first iteration update are obtained.

[0087] The updated RPC parameter P k+1 In the next iteration, it is input into the coordinate projector and used in the next round of iterations to calculate new point correspondences and correlation slices S1. k+1 And further residual displacement vector estimation ∆D1 k +1 The update process is repeated in each iteration, ultimately yielding the iteratively updated RPC parameters P1. Initially, the displacement cube D1 is set. 0 =0, RPC parameter P1 0 Given the existing PRC parameters, the PRC parameters are updated with each iteration. k+1 All will be better than the previous round's P k It is closer to the actual RPC correction parameters.

[0088] S44 inputs the preprocessed target image into the image warping module in the second sub-network for warping processing, resulting in a distorted target image. The specific representation is as follows:

[0089]

[0090] Warp(·,·) represents the warp operation of the image warp module.

[0091] S45 will process the pre-processed source image and the distorted target image. The first and second feature extraction submodules in the second subnetwork are respectively input to obtain the feature maps F of the second source image. I_2 (F) I_2 ∈R D×H×W Feature map F of the second target image G_2 (F) G_2 ∈R D×H×W );

[0092] S46 will use the feature map F of the second source image I_2 Feature map F of the second target image I_2 The correlation calculation module in the second sub-network is used to calculate the correlation between feature maps, resulting in the second correlation volume C. 1_2 and C 1_2 1 / 2 ;

[0093] S47 will use the second relevant volume C 1_2 and C 1_2 1 / 2 Input the iterative RPC estimation module in the second sub-network to obtain the second iteratively updated RPC parameter P; obtain the iteratively updated RPC parameter P2 by following the same steps as the iteratively updated RPC parameter P1 obtained in the first sub-network (i.e., steps S441-S444);

[0094] S48 calculates the repaired RPC parameters using the geometric consistency constraint method based on the RPC parameters P1 updated in the first iteration and P2 updated in the second iteration.

[0095] Based on the RPC parameters P1 updated in the first iteration and P2 updated in the second iteration at different scales, the repaired RPC parameters P are calculated. The specific calculation method is as follows:

[0096]

[0097] Where d(·,·) represents the distance metric function, and λ represents the weighting parameter that balances the importance of the two scales.

[0098] S5 performs geometric correction on the image to be repaired based on the repaired RPC parameters to obtain the repaired image.

[0099] An embodiment of the present invention also provides a deep learning-based RPC parameter repair system 100, which includes:

[0100] The sample set construction module 101 is configured to acquire a sample set, which includes multiple data sample groups, each of which includes source sample images, target sample images and corresponding ground control point sample data; and to preprocess the sample set to obtain a preprocessed sample set.

[0101] The initial RPC repair network construction module 102 is configured such that the initial RPC repair network includes a first sub-network and a second sub-network.

[0102] The network model training module 103 is configured to train the initial RPC repair network using a preprocessed sample set to obtain the trained RPC repair network.

[0103] The PRC parameter repair module 104 is configured to use the trained PRC repair network to repair the PPC parameters of the image to be repaired, and obtain the repaired PPC parameters; the image to be repaired includes the source image and the target image.

[0104] The image restoration module 105 is configured to perform geometric correction on the image to be restored based on the restored RPC parameters to obtain the restored image.

[0105] An embodiment of the present invention also provides an electronic device 40, which includes:

[0106] The memory 402 is used to store computer control instructions; the processor 401 is connected to the memory and is used to retrieve and execute the computer control instructions so that the electronic device performs the deep learning-based RPC parameter repair method as described above.

Claims

1. A deep learning-based RPC parameter repair method, characterized in that, The method includes the following steps: S1 acquires a sample set, which includes multiple data sample groups, each of which includes source sample images, target sample images, and corresponding ground control point sample data; and preprocesses the sample set to obtain a preprocessed sample set. S2: Construct an initial RPC repair network, which includes a first subnetwork and a second subnetwork; S3 network model training: The initial RPC repair network is trained using a preprocessed sample set to obtain the trained RPC repair network. S4 uses a trained PRC inpainting network to perform PPC parameter inpainting on the image to be inpainted, and obtains the inpainted RPC parameters; the image to be inpainted includes the source image and the target image. S5 performs geometric correction on the image to be repaired based on the repaired RPC parameters to obtain the repaired image.

2. The method according to claim 1, characterized in that, The first and second sub-networks in the PRC restoration network described in step S2 both include a feature extraction module, a correlation calculation module, and an iterative RPC estimation module; the second sub-network also includes an image distortion module.

3. The method according to claim 2, characterized in that, The feature extraction module in step S2 includes a first feature extraction submodule and a second feature extraction submodule, used to extract features from the image and obtain feature maps; the correlation calculation module is used to calculate the correlation between feature maps; the iterative RPC estimation module is used to obtain the iterative RPC parameters; and the image distortion module is used to distort the image to obtain a distorted image.

4. The method according to claim 2, characterized in that, The iterative RPC estimation module in step S2 includes a coordinate projection submodule, a correlation update submodule, a global motion aggregation submodule, and an RPC update submodule.

5. The method according to claim 4, characterized in that, Step S4 further includes: S41 inputs the source image I and the target image G into the first feature extraction submodule and the second feature extraction submodule in the first subnetwork, respectively, to obtain the feature map F of the first source image. I_1 Feature map F of the first target image G_1 ; S42 will generate the feature map F of the source image. I_1 Feature map F of the target image G_1 The correlation between feature maps is calculated by inputting them into the correlation calculation module of the first sub-network to obtain the first correlation volume C1 and C1. 1 / 2 ; S43 will use the first relevant volume C1 and C1 1 / 2 Input the RPC estimation module in the first sub-network to obtain the RPC parameters P1 updated in the first iteration; S44 inputs the preprocessed target image into the image warping module in the second sub-network for warping processing, resulting in a distorted target image. ; S45 will process the pre-processed source image and the distorted target image. The first and second feature extraction submodules of the second subnetwork are respectively input to obtain the feature maps F of the second source image. I_2 Feature map F of the second target image G_2 ; S46 will use the feature map F of the second source image I_2 Feature map F of the second target image G_2 The correlation calculation module in the second sub-network is used to calculate the correlation between feature maps, resulting in the second correlation volume C. 1_2 and C 1_2 1 / 2 ; S47 will use the second related volume C1 and C1 1 / 2 Input the iterative RPC estimation module in the second sub-network to obtain the RPC parameters P2 updated in the second iteration; S48 calculates the repaired RPC parameters using the geometric consistency constraint method based on the RPC parameters P1 updated in the first iteration and P2 updated in the second iteration.

6. The method according to claim 5, characterized in that, Step S43 in step S4 further includes: S431 uses a coordinate projection submodule to project the feature map F of the first source image. I_1 The coordinate set is mapped to the feature map F of the first target image. G_1 The corresponding projected coordinate set X1' k Above, and based on the coordinate set X1 and the projected coordinate set X1' k The pixel-by-pixel displacement vector F1 is obtained from the calculation. k ; S432 will use the first relevant volume C1 and C1 1 / 2 The input correlation update submodule uses the projected coordinate set X1' k For the first relevant volumes C1 and C1 1 / 2 Sampling is performed to generate the first correlation slice S1. k ; S433 will slice the correlation into S1 k and pixel-by-pixel displacement vector F1 k Input the global motion aggregation submodule to estimate the first residual displacement vector ∆D1 k ; S434 will use the first residual displacement vector ∆D1 k Input the RPC update submodule to obtain the first updated RPC parameter P1. k+1 ; S435 Repeat steps S441-S444 until the first residual displacement vector ∆D1 is obtained. k Once the system stabilizes, the RPC parameters P1 updated in the first iteration are obtained.

7. The method according to claim 1, characterized in that, The preprocessing in step S1 includes normalizing and standardizing the ground control point data, cropping the image data, and converting the target image to grayscale; wherein the ground control point data consists of the pixel coordinates of the image and the corresponding geographic coordinates.

8. A deep learning-based RPC parameter repair system, characterized in that, The system includes: The sample set construction module is configured to acquire a sample set, which includes multiple data sample groups, each of which includes source sample images, target sample images, and corresponding ground control point sample data; and to preprocess the sample set to obtain a preprocessed sample set. An initial RPC repair network construction module is configured such that the initial RPC repair network includes a first sub-network and a second sub-network; The network model training module is configured to train the initial RPC repair network using a preprocessed sample set to obtain the trained RPC repair network. The PRC parameter repair module is configured to use a trained PRC repair network to repair the PPC parameters of the image to be repaired, thereby obtaining the repaired PPC parameters; the image to be repaired includes the source image and the target image. The image restoration module is configured to perform geometric correction on the image to be restored based on the restored RPC parameters, thereby obtaining the restored image.

9. An electronic device, characterized in that, include: Memory, used to store computer control instructions; A processor, connected to the memory, is configured to retrieve and execute the computer control instructions to cause the electronic device to perform the deep learning-based RPC parameter repair method as described in any one of claims 1-7.

Images