A method and system for producing regional spaceborne optical and SAR imagery DOM

Abstract

The present invention provides a method and system for producing a DOM (Domain Positioning System) for regional spaceborne optical and SAR images. The method comprises: performing RPC modeling on the RD model of the original SAR image, suppressing speckle noise in the SAR image through filtering, and obtaining a SAR image with suppressed speckle noise; performing image control point matching between the original spaceborne optical image and the digital orthophoto image to obtain control points of the original spaceborne optical image; performing image control point matching between the SAR image with suppressed speckle noise and the digital orthophoto image to obtain control points of the filtered SAR image; performing joint block adjustment and orthorectification processing based on the SAR image with suppressed speckle noise and the original spaceborne optical image and the RPC positioning model, performing automatic light and color balancing on the orthorectified optical and SAR images, and mosaicking the light and color balancing optical and SAR images to obtain mosaicked optical and SAR orthorectified products. The present invention effectively guarantees the product quality of the joint DOM of regional optical and SAR images in actual production.

Description

Technical Field

[0001] The present invention belongs to the field of geographic information digitalization, and in particular relates to a method and system for producing regional satellite-borne optical and SAR images DOM. Background Art

[0002] Digital orthorectified maps (DOMs) are a crucial component of my country's digital geographic information infrastructure. Their production process involves digitally correcting, fusing, and mosaicking satellite images, then cropping them within a specific map area to create a digital orthophoto collection that combines both map geometric accuracy and imagery characteristics. Because DOMs combine both map geometric accuracy and imagery characteristics, they offer the advantages of high accuracy, rich information, intuitive realism, and a short production cycle. They can serve as background control information for evaluating the accuracy, currency, and integrity of other data, and can also be used to extract information on natural resources and socioeconomic development, providing a reliable basis for applications such as dynamic updates of basic data, disaster prevention and control, and public infrastructure planning.

[0003] In recent years, with the rapid development of domestic optical and synthetic aperture radar (SAR) satellites and related surveying and mapping technologies, my country has acquired the capability to independently develop and control 1:50,000 SAR and optical DOM systems. Current research on digital orthophoto production, both domestically and internationally, primarily focuses on processing optical or SAR satellite data independently, without comprehensively considering both payloads and relying excessively on manual control point selection.

[0004] ① SAR image orthorectification method based on satellite control point library and DEM (CN 107341778A): The present invention discloses a SAR image orthorectification method based on satellite control point library and DEM. The method utilizes existing external DEM data and Resource No. 3 control point data, and through SAR image simulation technology and image precise matching, obtains the image point coordinates of Resource No. 3 control point in the SAR image. Subsequently, the high-precision Resource No. 3 control point participates in SAR orthorectification to obtain a DOM image after SAR orthorectification. The present invention can automatically utilize the Resource No. 3 control point library to participate in SAR image orthorectification, reduce the inefficiency caused by manual intervention, and reduce the cost of obtaining control points. However, this invention only processes a single SAR payload and does not comprehensively consider the processing requirements of both optical and SAR payloads.

[0005] ② A method for generating digital orthophoto maps by a city low-altitude unmanned aerial vehicle system (CN 102506824A): The present invention discloses a method for generating digital orthophoto maps by a city low-altitude unmanned aerial vehicle system, comprising the following steps: data preparation, the data including control point data, camera calibration files, and attitude data; aerial triangulation of images to generate epipolar images; image matching to generate digital surface models (DSMs); construction of irregular triangulated networks (TINs) and digital elevation models (DEMs) based on urban building features for interpolation; digital differential correction to generate digital orthophoto maps (DOMs); mosaicking of city low-altitude digital orthophoto maps, including stitching line selection, uniform light and color, and boundary clipping. The present invention can fully utilize the high-resolution advantage of images acquired at low altitudes in cities to efficiently and accurately produce DOMs. However, this invention is mainly aimed at low-altitude unmanned aerial vehicle systems and does not consider the processing requirements of satellite images.

[0006] ③ An automatic registration technology for SAR and visible light band images (CN108038873A): The present invention discloses an automatic registration technology for SAR and visible light band images, comprising the following steps: data preprocessing, edge extraction, edge connection, target extraction, and region segmentation: target attribute calculation, target matching, extracting the current from the reference image to obtain control points, affine transformation of target extraction, and finally resampling to obtain the result. The present invention is not affected by time and space images, and normally analyzes and processes the collected data to obtain accurate numerical values, and the result is accurate. However, this invention mainly focuses on the SAR and visible light automatic registration steps, and does not consider the registration-related strategies.

[0007] ④ SAR Image Speckle Noise Suppression Method Based on Gamma-Lee Filtering (CN111861905B): This invention discloses a SAR image speckle noise suppression method based on Gamma Lee filtering. By performing a Gamma transformation on the original image to enhance its dark details, this method solves the problem of poor edge detail preservation in filtered SAR images. A translation-invariant local window is used to traverse each pixel of the measured image, and Lee filtering is implemented by calculating the mean and variance of the covered pixels and the center pixel value. The method combines the concept of negative feedback to adjust the Gamma coefficient and Gamma exponent to gradually adjust and enhance the suppression of SAR image speckle noise. Finally, simulation experimental results show that this invention solves the problem of poor edge detail preservation in existing SAR image filtering methods, while also improving the speckle noise suppression effect. This method is superior to the original Lee filtering method in both noise suppression and edge detail preservation, and can be used in the field of SAR image processing. However, this invention focuses on suppressing noise and edge details, and does not consider the overall SAR image quality.

[0008] Defects of the existing technology:

[0009] 1) Manual intervention will cause low efficiency, and manual point selection will bring problems such as punctum error. The present invention makes full use of the existing resource No. 3 optical reference base map to automatically match optical and SAR images to obtain the control points required for regional block adjustment, realizing automatic and high-precision correction processing of regional spaceborne optical and SAR images.

[0010] 2) Due to differences in satellite image payload type, imaging time, and incident angle, there are significant color and tonal differences between images taken over the same area at different times. Furthermore, current research lacks comprehensive consideration of both optical and SAR sensors, resulting in a lack of comprehensive processing methods. Therefore, an automated and comprehensive DOM processing method for regional spaceborne optical and SAR imagery was established. This method can achieve joint processing of spaceborne optical and SAR heterogeneous DOM imagery in terms of control point extraction, light and color uniformity, and mosaic line editing. This method is of great significance for guiding the production of regional digital orthophotos of spaceborne optical and SAR imagery. Summary of the Invention

[0011] In order to solve the above technical problems, the present invention proposes a technical solution of a method and system for producing regional spaceborne optical and SAR images DOM to solve the above technical problems.

[0012] A first aspect of the present invention discloses a method for producing a regional spaceborne optical and SAR image DOM, the method comprising:

[0013] Step S1, selecting the original spaceborne optical images and RPC models, original spaceborne SAR images and RD models, digital orthophoto image library and external reference DEM data within the survey area;

[0014] Step S2, performing RPC modeling on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and suppressing speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise;

[0015] Step S3: matching image control points between the original spaceborne optical image and the digital orthophoto based on the principle of consistency of optical image features to obtain the original spaceborne optical image control points;

[0016] Step S4: using the similarity of heterogeneous images in the deep convolutional neural network space, image control point matching is performed between the SAR image after suppressing speckle noise and the digital orthophoto image to obtain the SAR image control points after filtering;

[0017] Step S5, respectively utilize described original spaceborne optical image control points and the SAR image control points after filtering process, carry out joint block adjustment and orthorectification process based on the SAR image after described suppression of speckle noise and original spaceborne optical image and RPC positioning model, obtain orthorectified rear optical and SAR image;

[0018] Step S6: performing automatic light and color homogenization processing on the orthorectified optical and SAR images to obtain light and color homogenization processed optical and SAR images;

[0019] Step S7, mosaicking the optical and SAR images after the uniform light and color processing to obtain mosaicked optical and SAR orthorectified products;

[0020] Step S8: The mosaicked optical and SAR orthorectified products are cropped according to standard framing to obtain standard framing optical and SAR image DOM products.

[0021] Preferably, in step S2, performing RPC modeling on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and suppressing speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise includes:

[0022] Step S21: combining the characteristics of the optical image RPC model, using the original SAR image strict imaging geometry model to establish the correspondence between the three-dimensional spatial grid of ground points and the image surface as control points to solve the RPC parameters, and obtaining the fitted RPC model, that is, the RPC positioning model of the SAR image;

[0023] Step S22: Based on the output DOM and the resolution of the optical image involved in data processing, perform multi-look processing in azimuth or range on the original spaceborne SAR image single-look slant range complex data (SLC) to preliminarily suppress speckle noise and obtain a SAR image with preliminarily suppressed speckle noise; based on the multi-look processing, use a SAR image speckle noise suppression network to filter the SAR image with preliminarily suppressed speckle noise to obtain a SAR image with suppressed speckle noise.

[0024] Preferably, in step S2, the SAR image after the speckle noise is initially suppressed is filtered using a SAR image speckle noise suppression network, and a specific method for obtaining the SAR image after the speckle noise is suppressed includes:

[0025] 1) The backbone network of the SAR image speckle noise suppression network is UNet, which consists of two parts: encoder and decoder. The encoder uses an 18-layer ResNet instead of a four-fold downsampling CBR module stack (Conv+BN+ReLU), and the decoder uses a four-fold upsampling CBR module stack;

[0026] 2) Calculating the intensity of the sample and the SAR image to be filtered, and converting the multiplicative speckle noise into additive noise through logarithmic (log) operation; obtaining an additive noise image, i.e., the sample additive noise image and the SAR image additive noise image;

[0027] The SAR image to be filtered is the SAR image after the speckle noise is initially suppressed; the sample is the SAR image with superimposed synthetic speckle noise, which is also the input data used for noise suppression network training;

[0028] Through the true value θ of the model parameter to be estimated and the mapping relationship f θ The noise-free image intensity is estimated by The relationship between the noise-free image intensity x and the noisy image observation value y is as follows:

[0029]

[0030] in:

[0031] x: noise-free image intensity;

[0032] Estimation of noise-free image intensity;

[0033] y: observed value of x, containing additive noise;

[0034] For the two observed values y1 and y2 of x, the denoising objective function is established using the following formula:

[0035]

[0036] Estimation of the model parameters to be estimated

[0037] E: Mathematical expectation

[0038] f θ (·): The observation value y is filtered and denoised to obtain a noise-free image intensity estimate. The mapping relationship

[0039] Get an estimate of θ so that E[·] takes the minimum value;

[0040] Among them, l is the loss function, which has the following form:

[0041]

[0042] Among them, y1 and y2 are two independent observation values of x, and k is the pixel number;

[0043] 3) The model training is performed on the superimposed synthetic speckle noise SAR images as samples; each iteration generates two sets of independent synthetic speckle noise SAR images, which are used as input and for calculating the loss function respectively;

[0044] The samples were divided into 256×256 slices with a step size of 32 to form the training set. The number of training epochs was not less than 50. The training network used the Adam optimizer with an initial learning rate of 0.001, which was reduced to 1 / 10 of the initial learning rate after 10 epochs.

[0045] The training network is pre-trained and re-trained. The pre-training process generates downsampled denoised SAR images to compensate for changes in SAR images at different periods, and the downsampling is used to reduce the correlation of synthetic noise. The re-training updates the pre-trained weights, and the number of epochs is not less than 20.

[0046] Preferably, in step S4, the method of performing image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space to obtain the control points of the filtered SAR image includes:

[0047] Step S41: using the image obtained by matching the speckle noise suppressed SAR image with the optical image template as a positive sample and the non-matching speckle noise suppressed SAR image as a negative sample, using a convolutional network to extract the feature tensor of the optical image template as a similarity measure for the registration between the speckle noise suppressed SAR image and the optical image template;

[0048] Step S42: A deep matching network is used to drive the positive and negative samples. The loss function is designed as follows: under a given threshold, the overall distance between the positive sample and the optical image template in the feature space is made smaller than that between the negative sample, so that the weights obtained through training have the ability to determine whether the SAR image after suppressing speckle noise matches the optical image template during the inference process.

[0049] Preferably, in step S4, the method of performing image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space to obtain the control points of the filtered SAR image includes:

[0050] 1) Selecting spots with obvious point and line features from optical and radar images as control point samples, using optical image feature spots as anchor points, radar spots with the same name as positive samples, and radar spots with different names as negative samples. The optical image is converted into a feature tensor through a convolutional network, and the loss function is established based on this as follows:

[0051] Loss=max(0,Δ + -Δ - +μ)

[0052] Among them, Loss is the loss function, u is the distance to distinguish whether it is matched or not, Δ + and Δ - are the Euclidean distances between the positive and negative samples and the optical spot anchor feature tensor:

[0053] Δ + =||F(t)-F(g)||2,Δ - =||F(t)-F(b)||2

[0054] F(t), F(g), and F(b) are the feature tensors of the optical anchor, positive sample, and negative sample, respectively;

[0055] 2) The deep matching network consists of a 6-layer convolutional network. The first 5 layers are stacked CBR modules (Conv+BN+ReLU). The last layer uses 8 convolutional connections and uses L2 regularization and 0.3 dropout to avoid overfitting. The deep matching network input is a 64×64 image slice and the output is a 128-dimensional unit vector.

[0056] 3) Based on the above loss function, the deep matching network selects all negative spots with close Euclidean distance of feature tensors in the optical and radar sample images as samples, sets the edge distance u between positive and negative spots in the feature space, and if Δ + +u>Δ - , then readjust the weight until Δ + <Δ - ,;

[0057] 4) The network is pre-trained using medium-resolution (10m) optical and radar imagery. The network uses the Adam optimizer with a learning rate of 0.1 and a minimum of 50 epochs. High-resolution imagery (1m) is then re-trained with a learning rate of 0.001 and a minimum of 100 epochs.

[0058] Preferably, in step S5, the method of performing joint block adjustment and orthorectification processing based on the SAR image after speckle noise suppression, the original satellite optical image, and the RPC positioning model using the satellite optical image control points and the filtered SAR image control points respectively to obtain the orthorectified optical and SAR images comprises:

[0059] Step S51: using the original satellite-borne optical image control points and the SAR image control points after multi-look filtering processing, and compensating the system error of the RPC model through the constraint relationship between the control points of the images;

[0060]

[0061] Where (x, y) is the measured coordinates on the original spaceborne optical image and the SAR image after multi-look filtering processing, (sample, line) is the projection value of the ground control point onto the image surface using the RPC parameters;

[0062] List the following linear equation for each control point and perform bundle adjustment:

[0063] V=At+BX-1

[0064]

[0065] t=(Δpx0Δpx1Δpx2Δpy0Δpy1Δpy2) T ,

[0066]

[0067] Where V is the residual, A is the coefficient matrix of the affine parameters, B is the coefficient matrix of the object coordinates, t is the affine parameter correction number, X is the object coordinate correction number, and 1 is the constant term.

[0068] Among them, the RPC model is formed by fitting the original optical image RPC model and the SAR image;

[0069] Step S52: Based on the external reference DEM data, the spaceborne optical data and the RPC parameters after the SAR image block adjustment, orthorectify the spaceborne optical and multi-look filtering processed SAR image.

[0070] Preferably, in step S7, the method of performing mosaic processing on the optical and SAR images after the light and color uniformity processing to obtain mosaicked optical and SAR orthorectified products includes:

[0071] Adding avoidance points within a predetermined area of the optical and SAR images after the uniform light and color processing, and adjusting the tones of the images in the local areas on both sides of the mosaic line; by adding avoidance points and editing the mosaic line, the tones between the images in the predetermined area are uniformly transitioned;

[0072] In step S8, the minimum bounding rectangle of the coordinates of the center points of the pixels of the four corner points of the outline in the standard frame is expanded outward by a preset value P pixels. The coordinates of the center points of the pixels of the four corner points of the outline in the standard frame are calculated as follows:

[0073] X min =int[min(X1,X2,X3,X4) / d]×dP×d

[0074] Y min =int[min(Y1, Y2, Y3, Y4) / d]×dP×d

[0075] X max =it[max(X1,X2,X3,X4) / d+1]×d+P×d

[0076] Y max =int[max(Y1, Y2, Y3, Y4) / d+1]×d+P×d

[0077] Where (X1, Y1), (X2, Y2), (X3, Y3), and (X4, Y4) are the coordinates of the four outline points, and the unit of coordinate is meters; d is the ground resolution of the orthophoto, int[·] rounds the number down, max(·) returns the maximum value in the parameter list, and min(·) returns the minimum value in the parameter list.

[0078] A second aspect of the present invention discloses a system for producing regional spaceborne optical and SAR images DOM, the system comprising:

[0079] The first processing module is configured to select the original spaceborne optical image and RPC model, the original spaceborne SAR image and RD model, the digital orthophoto library and the external reference DEM data within the survey area;

[0080] The second processing module is configured to perform RPC modeling on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and suppress speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise;

[0081] The third processing module is configured to match the image control points between the original spaceborne optical image and the digital orthophoto based on the principle of consistency of optical image features to obtain the control points of the original spaceborne optical image;

[0082] a fourth processing module configured to perform image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space, and obtain the control points of the filtered SAR image;

[0083] a fifth processing module configured to respectively use the original spaceborne optical image control points and the filtered SAR image control points, and perform joint block adjustment and orthorectification processing based on the speckle noise suppressed SAR image and the original spaceborne optical image and the RPC positioning model to obtain orthorectified optical and SAR images;

[0084] a sixth processing module configured to perform automatic light and color homogenization processing on the orthorectified optical and SAR images to obtain light and color homogenized optical and SAR images;

[0085] A seventh processing module is configured to perform mosaic processing on the optical and SAR images after the uniform light and color processing to obtain mosaicked optical and SAR orthorectified products;

[0086] The eighth processing module is configured to perform cropping processing on the mosaicked optical and SAR orthorectified products according to standard framing to obtain standard framing optical and SAR image DOM products.

[0087] A third aspect of the present invention discloses an electronic device comprising a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of any one of the methods for producing a regional spaceborne optical and SAR image DOM according to the first aspect of the present disclosure.

[0088] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods for producing regional spaceborne optical and SAR images DOM according to the first aspect of the present disclosure.

[0089] The proposed solution effectively utilizes the existing No. 3 optical reference basemap to automatically match optical and SAR imagery to obtain the control points required for regional block adjustment. This reduces the inefficiency caused by manual intervention or the puncture errors caused by manual point selection, enabling automatic and high-precision correction of regional spaceborne optical and SAR imagery. This solution addresses the significant color and tonal variations between images taken at different times in the same region due to differences in satellite imaging time, angle of incidence, and atmospheric conditions, as well as the differences between optical and SAR sensors. By leveraging proven uniform light and color mixing and mosaic line editing strategies, the quality of the combined DOM of regional optical and SAR imagery is effectively guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0091] Figure 1 Flowchart of a method for producing a regional spaceborne optical and SAR image DOM according to an embodiment of the present invention;

[0092] Figure 2 A SAR image speckle noise suppression network according to an embodiment of the present invention;

[0093] Figure 3a This is a result image of Hainan Province jointly orthorectified using optical and SAR data according to an embodiment of the present invention;

[0094] Figure 3b A vertical track map of the optical and SAR data jointly orthorectified image of Hainan Province according to an embodiment of the present invention;

[0095] Figure 3c A track-wise image of Hainan Province jointly orthorectified with optical and SAR data according to an embodiment of the present invention;

[0096] Figure 4a This is a result image of the joint orthorectification of optical and SAR data of Jiangsu Province according to an embodiment of the present invention;

[0097] Figure 4b A vertical track map of an image jointly orthorectified with optical and SAR data of Jiangsu Province according to an embodiment of the present invention;

[0098] Figure 4c A track-wise image of the Jiangsu Province image jointly orthorectified with optical and SAR data according to an embodiment of the present invention;

[0099] Figure 5a An along-track diagram showing the accuracy comparison of optical and SAR orthorectification products according to an embodiment of the present invention;

[0100] Figure 5b A vertical trajectory diagram showing the accuracy comparison of optical and SAR orthorectification products according to an embodiment of the present invention;

[0101] Figure 6a The image before editing the mosaic lines after uniforming according to an embodiment of the present invention;

[0102] Figure 6b Editing images for mosaic lines and dodging points according to embodiments of the present invention;

[0103] Figure 6c An image after editing with mosaic lines and dodging points according to an embodiment of the present invention;

[0104] Figure 7a This is a mosaicked image of an optical orthorectification product according to an embodiment of the present invention;

[0105] Figure 7b This is a mosaicked image of a radar orthorectification product after uniform light emission according to an embodiment of the present invention;

[0106] Figure 7c This is a mosaic result of optical and radar uniform light according to an embodiment of the present invention;

[0107] Figure 8 The optical digital orthophoto result according to the embodiment of the present invention;

[0108] Figure 9 The SAR digital orthophoto results according to the embodiment of the present invention;

[0109] Figure 10 A convolutional network for expressing image matching features according to an embodiment of the present invention;

[0110] Figure 11 The deep matching network training process and loss function according to an embodiment of the present invention;

[0111] Figure 12 A structural diagram of a system for producing regional spaceborne optical and SAR images DOM according to an embodiment of the present invention;

[0112] Figure 13 FIG. 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0113] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0114] The first aspect of the present invention discloses a method for producing regional spaceborne optical and SAR images DOM. Figure 1 FIG. 1 is a flow chart of a method for producing a regional spaceborne optical and SAR image DOM according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0115] Step S1, selecting the original spaceborne optical image and RPC model, the original spaceborne SAR image and RD model, the digital orthophoto library and the external reference DEM data in the survey area; wherein the digital orthophoto library can generally be the Resource No. 3 digital orthophoto library;

[0116] Step S2, performing RPC modeling on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and suppressing speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise;

[0117] Step S3: Based on the principle of consistency of optical image features, matching of image control points between the original spaceborne optical image and the ZY-3 digital orthophoto image is performed to obtain the control points of the original spaceborne optical image;

[0118] Step S4: using the similarity of heterogeneous images in the deep convolutional neural network space, image control point matching is performed between the SAR image after suppressing speckle noise and the digital orthophoto image to obtain the SAR image control points after filtering;

[0119] Step S5, respectively utilize described original spaceborne optical image control points and the SAR image control points after filtering process, carry out joint block adjustment and orthorectification process based on the SAR image after described suppression of speckle noise and original spaceborne optical image and RPC positioning model, obtain orthorectified rear optical and SAR image;

[0120] Step S6: performing automatic light and color homogenization processing on the orthorectified optical and SAR images to obtain light and color homogenization processed optical and SAR images;

[0121] Step S7: mosaicking the optical and SAR images after the uniform light and color processing to obtain mosaicked optical and SAR orthorectified products; Step S8: cropping the mosaicked optical and SAR orthorectified products according to standard framing to obtain standard framing optical and SAR image DOM products.

[0122] In step S1, the original spaceborne optical images and RPC models, the original spaceborne SAR images and RD models, the digital orthophoto library and the external reference DEM data within the survey area are selected.

[0123] In some embodiments, in step S1, the method further includes:

[0124] The data coverage and image quality of the original spaceborne optical images and original spaceborne SAR images are checked and screened, and the degree of matching between the digital orthophoto library and the external reference DEM data and the original spaceborne optical images and the original spaceborne SAR images is determined.

[0125] Specifically, raw data and control data are collected. The raw data includes raw spaceborne optical images and corresponding RPC models, raw spaceborne SAR images and corresponding RD models. The control data includes digital orthophoto libraries, external reference DEM data, and other control data. Simultaneously, the data coverage and image quality of the raw spaceborne optical and SAR images are checked and screened, and the degree of matching between the digital orthophoto library and external reference DEM data and the raw spaceborne optical and SAR images is determined.

[0126] In step S2, RPC modeling is performed on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and speckle noise of the SAR image is suppressed by filtering to obtain a SAR image after suppressing speckle noise;

[0127] In some embodiments, the method of performing RPC modeling on the RD model of the original SAR image to obtain the RPC positioning model of the SAR image, and suppressing the speckle noise of the SAR image by filtering to obtain the SAR image after suppressing the speckle noise includes:

[0128] Step S21: combining the characteristics of the optical image RPC model, using the original SAR image strict imaging geometry model to establish the correspondence between the three-dimensional spatial grid of ground points and the image surface as control points to solve the RPC parameters, and obtaining the fitted RPC model, that is, the RPC positioning model of the SAR image;

[0129] Step S22: performing multi-look processing in azimuth or range on the original spaceborne SAR image single-look slant range complex data (SLC) based on the output DOM and the resolution of the optical image involved in data processing to preliminarily suppress speckle noise and obtain a SAR image with preliminarily suppressed speckle noise; based on the multi-look processing, performing filtering processing on the SAR image with preliminarily suppressed speckle noise using a SAR image speckle noise suppression network to obtain a SAR image with suppressed speckle noise;

[0130] like Figure 2 As shown in FIG, the backbone network of the SAR image speckle noise suppression network is UNet, ResNet is used as the encoder, and the training data consists of multi-temporal SAR images and simulated noise.

[0131] The method of suppressing speckle noise of SAR images by filtering and obtaining a SAR image with suppressed speckle noise is as follows:

[0132] The SAR image speckle noise suppression network, namely UNet+ResNet, removes speckle noise as follows:

[0133] 1) The backbone network of the SAR image speckle noise suppression network is UNet, which consists of two parts: encoder and decoder. The encoder uses an 18-layer ResNet instead of a four-fold downsampling CBR module stack (Conv+BN+ReLU), and the decoder uses a four-fold upsampling CBR module stack;

[0134] 2) Calculating the intensity of the sample and the SAR image to be filtered, and converting the multiplicative speckle noise into additive noise through logarithmic (log) operation; obtaining an additive noise image, i.e., the sample additive noise image and the SAR image additive noise image;

[0135] This section is used to explain the objective function of network training calculation, namely the loss function. Network training is based on this function to obtain the model weight parameters.

[0136] Network training input: sample images and synthetic speckle noise;

[0137] Network training output: model parameters θ to be estimated

[0138] The SAR image to be filtered is the SAR image after the speckle noise is initially suppressed; the sample is the SAR image with superimposed synthetic speckle noise, which is also the input data used for noise suppression network training;

[0139] Through the true value θ of the model parameter to be estimated and the mapping relationship f θ The noise-free image intensity is estimated by The relationship between the noise-free image intensity x and the noisy image observation value y is as follows:

[0140]

[0141] in:

[0142] x: noise-free image intensity, which refers to the SAR image intensity without noise;

[0143] Estimation of noise-free image intensity;

[0144] y: observed value of x, containing additive noise;

[0145] For the two observation values y1 and y2 of x, that is, the two measurements of x, y1 and y2 respectively, y1 = x + n1, n1 is speckle noise; y2 = x + n2, n2 is speckle noise; use the following formula to establish the denoising objective function:

[0146]

[0147] The estimated value of the model parameter to be estimated; where the estimated model parameter θ is the true value;

[0148] E: Mathematical expectation

[0149] f θ (·): The observation value y is filtered and denoised to obtain a noise-free image intensity estimate. The mapping relationship

[0150] Get an estimate of θ so that E[·] takes the minimum value;

[0151] Among them, l is the loss function, which has the following form:

[0152]

[0153] Among them, y1 and y2 are two independent observation values of x, and k is the pixel number; this loss function no longer needs to simulate or estimate the noise-free image x; the above process is the loss function of network training and the goal of training calculation convergence; after training, the network will evaluate the quality of the model through the above loss function. The greater the loss, the worse the model parameters, and vice versa.

[0154] 3) The model training is performed on the superimposed synthetic speckle noise SAR images as samples; each iteration generates two sets of independent synthetic speckle noise SAR images, which are used as input and for calculating the loss function respectively;

[0155] The samples were divided into 256×256 slices with a step of 32 to form a training set. The number of training epochs was no less than 50. The Adam optimizer was used for training the network with an initial learning rate of 0.001, which dropped to 1 / 10 of the initial learning rate after 10 epochs.

[0156] The training network is pre-trained and re-trained. The pre-training process generates downsampled denoised SAR images to compensate for changes in SAR images at different periods, and the downsampling is used to reduce the correlation of synthetic noise. The re-training updates the pre-trained weights, and the number of epochs is not less than 20.

[0157] The noise suppression network generates synthetic speckle noise, which is then subtracted from the image to produce the denoised result. This means that the SAR image speckle noise suppression network is used to filter the SAR image after initial speckle noise suppression, resulting in a speckle-suppressed SAR image. To ensure sufficient generalization, the model is trained on SAR image samples superimposed with synthetic speckle noise. Each iteration generates two independent sets of synthetic noise images, one for input and the other for loss function calculation.

[0158] To improve performance on real-world images, the network undergoes pre-training and re-training. The pre-training process generates downsampled denoised SAR images to compensate for variations in the time series imagery, while the downsampling process reduces the correlation of the synthesized noise. The re-training process updates the pre-trained weights, with a minimum of 20 epochs.

[0159] In step S3, based on the principle of consistency of optical image features, image control points are matched between the original spaceborne optical image and the Resource-3 digital orthophoto image to obtain the original spaceborne optical image control points.

[0160] In some embodiments, in step S3, the method of matching image control points between the original spaceborne optical image and the Resource-3 digital orthophoto based on the principle of consistency of optical image features to obtain the control points of the original spaceborne optical image includes:

[0161] The original satellite optical image and the digital orthophoto library are used, based on the principle of feature consistency between optical images, a preset number of same-name points are obtained through the correlation coefficient method, SIFT algorithm and SGM image matching algorithm to obtain the control points of the original satellite optical image.

[0162] In step S4, the method of performing image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space to obtain the control points of the filtered SAR image includes:

[0163] like Figure 10 As shown in the figure, the image of the SAR image after suppressing speckle noise and the image matching the optical image template is used as the positive sample, and the non-matching SAR image is used as the negative sample. The convolutional network is used to extract the feature tensor of the template image as the similarity measure for the registration of the SAR image after suppressing speckle noise and the optical image; the deep matching network is driven by the above positive and negative samples, and its loss function is designed as follows: under a given threshold condition, the overall distance between the positive sample and the template in the feature space is made smaller than that of the negative sample, so that the weight obtained by training has the ability to determine whether the SAR image and the optical template match during the inference process, as shown in the figure. Figure 11 shown.

[0164] Among them, the deep network optical radar registration process:

[0165] 1) Selecting spots with obvious point and line features from optical and radar images as control point samples, using optical image feature spots as anchor points, radar spots with the same name as positive samples, and radar spots with different names as negative samples. The optical image is converted into a feature tensor through a convolutional network, and the loss function is established based on this as follows:

[0166] Loss=max(0,Δ + -Δ- +μ)

[0167] Among them, Loss is the loss function, u is the distance to distinguish whether it is matched or not, Δ + and Δ - are the Euclidean distances between the positive and negative radar samples and the optical pattern anchor point feature tensors:

[0168] Δ + =||F(t)-F(g)||2,Δ - =||F(t)-F(b)||2

[0169] F(t), F(g), and F(b) are the feature tensors of the optical anchor, positive, and negative radar samples, respectively.

[0170] 2) The deep matching network consists of a 6-layer convolutional network. The first 5 layers are all CBR module stacks (Conv+BN+ReLU), and the last layer uses 8 convolution connections. L2 regularization and 0.3 dropout are used to avoid overfitting. The deep matching network input is a 64×64 image slice and the output is a 128-dimensional unit vector.

[0171] 3) Based on the above loss function, the deep matching network selects all negative spots with close Euclidean distance of feature tensors in the optical and radar sample images as samples, sets the edge distance u between positive and negative spots in the feature space, and if Δ + +u>Δ - , then readjust the weight until Δ + <Δ - ,.

[0172] 4) The network is pre-trained using medium-resolution (10m-level) optical and radar imagery. The network uses the Adam optimizer with a learning rate of 0.1 and a minimum of 50 epochs. For high-resolution imagery (1m-level), a secondary training session is performed with a learning rate of 0.001 and a minimum of 100 epochs.

[0173] In step S5, the method of performing joint block adjustment and orthorectification processing based on the SAR image after the speckle noise is suppressed, the original spaceborne optical image, and the RPC positioning model using the original spaceborne optical image control points and the filtered SAR image control points to obtain the orthorectified optical and SAR images includes: Figures 3a to 3c , Figures 4a to 4c and 5a-5b.

[0174] In some embodiments, in step S5, the method of respectively using the original spaceborne optical image control points and the filtered SAR image control points, performing joint block adjustment and orthorectification processing based on the SAR image after speckle noise suppression and the original spaceborne optical image and the RPC positioning model to obtain the orthorectified optical and SAR images includes:

[0175] Step S51: using the original satellite-borne optical image control points and the SAR image control points after multi-look filtering processing, and compensating the system error of the RPC model through the constraint relationship between the control points of the images;

[0176] List the following linear equation for each control point and perform bundle adjustment:

[0177] VAt+BX—

[0178]

[0179] t=(Apx0 Apx1 Apx2 Apy0 Apy1 Apy2) T ,

[0180] X=(Δlat Δlon Δheitht) T ,

[0181] Among them, the RPC model is formed by fitting the original optical image RPC model and the SAR image;

[0182] in,

[0183] Step S52: Based on the external reference DEM data, the original spaceborne optical data and the RPC parameters after block adjustment of the SAR images, orthorectification is performed on the original spaceborne optical data and the SAR images after multi-look filtering processing.

[0184] In step S6, automatic light and color homogenization processing is performed on the orthorectified optical and SAR images to obtain light and color homogenized optical and SAR images.

[0185] Specifically, due to differences in imaging time, incident angle, and atmospheric conditions, there are obvious color or hue differences between optical and SAR images after orthorectification. These color or hue differences can be eliminated through automatic light and color uniformity methods, specifically the Advance-bundle method.

[0186] By selecting the optimal hue for adjacent areas of the SAR image and the optical image, histogram information of the image hue is generated. This information is then used to simulate the image's brightness distribution and used as an approximate representation of the background image. The uneven background image is then subtracted from the original image. Finally, contrast enhancement is performed to suppress the uneven background, enhance the contrast of image details, and generate a final image with moderate contrast. The hue histograms of the SAR and optical images are made as consistent as possible, that is, by forcing the image's hue range to achieve the goal of uniform, smooth, and consistent hue for the overall color of the two images. The final uniforming model can be simplified to formula (2):

[0187] f dst (x,y)=f src (x, y)-f blur (x,y)+f aver (x, y)

[0188] Where, f src (x, y) represents the hue value at position (x, y) on the original image, f blur (x,y) is the approximate background correction value at the corresponding position (x,y) after processing the hue histogram information. dst (x,y) represents the corrected target pixel hue value, f aver (x,y) The mean or median hue of the pixels in the local window.

[0189] In step S7, the optical and SAR images after the light and color uniformity processing are mosaicked to obtain mosaicked optical and SAR orthorectified products.

[0190] In some embodiments, in step S7, the method of performing mosaicking on the optical and SAR images after light and color uniformity processing to obtain mosaicked optical and SAR orthorectified products includes:

[0191] In the preset area of the optical and SAR images after the uniform light and color processing, dodge points are added to adjust the color tones of the local areas on both sides of the mosaic line; by adding dodge points and editing the mosaic line, the color tones between the images in the preset area are uniformly transitioned, and the local image color differences between adjacent images can be adjusted, so that the tones of satellite images at different imaging times are closer, thereby achieving the best mosaic effect. Figures 6a to 6c and Figures 7a to 7c shown.

[0192] In step S8, the mosaicked optical and SAR orthorectified products are cropped according to standard framing to obtain standard framing optical and SAR image DOM products, such as Figure 8 and Figure 9 shown.

[0193] In some embodiments, in step S8, the minimum bounding rectangle of the coordinates of the center points of the pixels of the four corner points of the outline within the standard frame is expanded outward by a preset value P, specifically P can be 200 pixels. The coordinates of the center points of the pixels of the four corner points of the outline within the standard frame are calculated as follows:

[0194] X min =int[min(X1,X2,X3,X4) / d]×d-200×d

[0195] Y min =int[min(Y1,Y2,Y3,Y4) / d]×d-200×d

[0196] X max =int[max(X1,X2,X3,X4) / d+1]×d+200×d

[0197] Y max =int[max(Y1,Y2,Y3,Y4) / d+1]×d+200×d

[0198] Where (X1, Y1), (X2, Y2), (X3, Y3), and (X4, Y4) are the coordinates of the four outline points, and the unit of coordinate is meters; d is the ground resolution of the orthophoto, int[·] rounds the number down, max(·) returns the maximum value in the parameter list, and min(·) returns the minimum value in the parameter list.

[0199] In summary, the solution proposed in the present invention can address the characteristics of traditional optical and SAR digital orthophoto production methods that are overly dependent on manual labor and low efficiency, and make relevant improvements to the conventional digital orthophoto processing process. The Resource-3 optical benchmark map is used based on the principle of feature consistency to automatically match the control points required for optical regional network adjustment; on the basis of multi-view processing, the SAR image speckle noise suppression network is used to filter the image to improve the image quality; the deep convolutional neural network is used to express the features of heterogeneous images to match the image control points between the filtered SAR image and the Resource-3 digital orthophoto. At the same time, the present invention solves the large color and hue differences between images of the same area at different times due to different satellite image payloads, imaging times and incident angles, as well as the differences between optical and SAR sensors, and proposes a new Advance-bundle method. It also uses the avoidance point editing strategy to provide a technical route that can support the joint uniform light and color and mosaic line editing of regional optical and SAR satellite images, effectively ensuring the product quality of the joint DOM of regional optical and SAR images in actual production.

[0200] A second aspect of the present invention discloses a system for producing regional spaceborne optical and SAR images DOM. Figure 12 FIG. 1 is a structural diagram of a system for producing regional spaceborne optical and SAR images DOM according to an embodiment of the present invention; FIG. Figure 12 As shown, the system 100 includes:

[0201] The first processing module 101 is configured to select original spaceborne optical images and RPC models, original spaceborne SAR images and RD models, digital orthophoto libraries, and external reference DEM data within the survey area;

[0202] The second processing module 102 is configured to perform RPC modeling on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and suppress speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise;

[0203] The third processing module 103 is configured to match the image control points between the original spaceborne optical image and the digital orthophoto based on the principle of consistency of optical image features to obtain the control points of the original spaceborne optical image;

[0204] The fourth processing module 104 is configured to perform image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space, and obtain the control points of the filtered SAR image;

[0205] The fifth processing module 105 is configured to respectively use the original spaceborne optical image control points and the filtered SAR image control points, and perform a joint block adjustment and orthorectification process based on the speckle noise suppressed SAR image and the original spaceborne optical image and the RPC positioning model to obtain orthorectified optical and SAR images;

[0206] A sixth processing module 106 is configured to perform automatic light and color homogenization processing on the orthorectified optical and SAR images to obtain light and color homogenized optical and SAR images;

[0207] The seventh processing module 107 is configured to perform mosaic processing on the optical and SAR images after the uniform light and color processing to obtain mosaicked optical and SAR orthorectified products;

[0208] The eighth processing module 108 is configured to perform cropping processing on the mosaicked optical and SAR orthorectified products according to standard framing to obtain standard framing optical and SAR image DOM products.

[0209] According to the second aspect of the present invention, the second processing module 102 is configured to perform RPC modeling on the RD model of the original SAR image to obtain an RPC positioning model of the SAR image, and suppress speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise, including:

[0210] Combining the characteristics of the optical image RPC model, the correspondence between the three-dimensional spatial grid of ground points and the image surface is established using the strict imaging geometric model of the original SAR image as the control points to solve the RPC parameters, and the RPC model after fitting is obtained, that is, the RPC positioning model of the SAR image;

[0211] According to the output DOM and the resolution of the optical images involved in data processing, the original spaceborne SAR image single-look slant range complex data (SLC) is subjected to multi-look processing in azimuth or range to preliminarily suppress speckle noise and obtain a SAR image with preliminarily suppressed speckle noise. Based on the multi-look processing, the SAR image with preliminarily suppressed speckle noise is filtered using a SAR image speckle noise suppression network to obtain a SAR image with suppressed speckle noise.

[0212] According to the second aspect of the present invention, the second processing module 102 is configured to use a SAR image speckle noise suppression network to perform filtering processing on the SAR image after preliminary speckle noise suppression, and obtaining the SAR image after speckle noise suppression specifically includes:

[0213] 1) The backbone network of the SAR image speckle noise suppression network is UNet, which consists of two parts: encoder and decoder. The encoder uses an 18-layer ResNet instead of a four-fold downsampling CBR module stack (Conv+BN+ReLU), and the decoder uses a four-fold upsampling CBR module stack;

[0214] 2) Calculating the intensity of the sample and the SAR image to be filtered, and converting the multiplicative speckle noise into additive noise through logarithmic (log) operation; obtaining an additive noise image, i.e., the sample additive noise image and the SAR image additive noise image;

[0215] The SAR image to be filtered is the SAR image after the speckle noise is initially suppressed; the sample is the SAR image with superimposed synthetic speckle noise, which is also the input data used for noise suppression network training;

[0216] Through the true value θ of the model parameter to be estimated and the mapping relationship f θ The noise-free image intensity is estimated by The relationship between the noise-free image intensity x and the noisy image observation value y is as follows:

[0217]

[0218] in:

[0219] x: noise-free image intensity;

[0220] Estimation of noise-free image intensity;

[0221] y: observed value of x, containing additive noise;

[0222] For the two observed values y1 and y2 of x, the denoising objective function is established using the following formula:

[0223]

[0224] Estimation of the model parameters to be estimated

[0225] E: Mathematical expectation

[0226] f θ (·): The observation value y is filtered and denoised to obtain a noise-free image intensity estimate. The mapping relationship

[0227] Get an estimate of θ so that E[·] takes the minimum value;

[0228] Among them, l is the loss function, which has the following form:

[0229]

[0230] Among them, y1 and y2 are two independent observation values of x, and k is the pixel number;

[0231] 3) The model training is performed on the superimposed synthetic speckle noise SAR images as samples; each iteration generates two sets of independent synthetic speckle noise SAR images, which are used as input and for calculating the loss function respectively;

[0232] The samples were divided into 256×256 slices with a step size of 32 to form the training set. The number of training epochs was not less than 50. The training network used the Adam optimizer with an initial learning rate of 0.001, which was reduced to 1 / 10 of the initial learning rate after 10 epochs.

[0233] The training network is pre-trained and re-trained. The pre-training process generates downsampled denoised SAR images to compensate for changes in SAR images at different periods, and the downsampling is used to reduce the correlation of synthetic noise. The re-training updates the pre-trained weights, and the number of epochs is not less than 20.

[0234] According to the second aspect of the present invention, the fourth processing module 104 is configured to perform image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space, and obtain the filtered SAR image control points including:

[0235] The image obtained by matching the speckle noise suppressed SAR image with the optical image template is used as a positive sample, and the image obtained by matching the speckle noise suppressed SAR image with the optical image template is used as a negative sample. A convolutional network is used to extract the feature tensor of the optical image template as a similarity measure for the registration between the speckle noise suppressed SAR image and the optical image template.

[0236] A deep matching network is used to drive the above positive and negative samples, and its loss function is designed as follows: under a given threshold condition, the overall distance between the positive sample and the optical image template in the feature space is made smaller than that of the negative sample, so that the weights obtained through training have the ability to determine whether the SAR image after suppressing speckle noise matches the optical image template during the inference process.

[0237] According to the second aspect of the present invention, the fourth processing module 104 is configured to use the similarity of heterogeneous images in the deep convolutional neural network space to perform image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image, and obtain the SAR image control points after filtering, specifically including:

[0238] 1) Selecting spots with obvious point and line features from optical and radar images as control point samples, using optical image feature spots as anchor points, radar spots with the same name as positive samples, and radar spots with different names as negative samples. The optical image is converted into a feature tensor through a convolutional network, and the loss function is established based on this as follows:

[0239] Loss=max(0,Δ + -Δ _ +μ)

[0240] Among them, Loss is the loss function, u is the distance to distinguish whether it is matched or not, Δ + and Δ - are the Euclidean distances between the positive and negative samples and the optical spot anchor feature tensor:

[0241] Δ + =||F(t)-F(g)||2,Δ - =||F(t)-F(b)||2

[0242] F(t), F(g), and F(b) are the feature tensors of the optical anchor, positive sample, and negative sample, respectively;

[0243] 2) The deep matching network consists of a 6-layer convolutional network. The first 5 layers are stacked CBR modules (Conv+BN+ReLU). The last layer uses 8 convolutional connections and uses L2 regularization and 0.3 dropout to avoid overfitting. The deep matching network input is a 64×64 image slice and the output is a 128-dimensional unit vector.

[0244] 3) Based on the above loss function, the deep matching network selects all negative spots with close Euclidean distance of feature tensors in the optical and radar sample images as samples, sets the edge distance u between positive and negative spots in the feature space, and if Δ + +u>Δ - , then readjust the weight until Δ + <Δ - ,;

[0245] 4) The network is pre-trained using medium-resolution (10m) optical and radar imagery. The network uses the Adam optimizer with a learning rate of 0.1 and a minimum of 50 epochs. High-resolution imagery (1m) is then re-trained with a learning rate of 0.001 and a minimum of 100 epochs.

[0246] According to the second aspect of the present invention, the fifth processing module 105 is configured to respectively use the spaceborne optical image control points and the filtered SAR image control points, and perform joint block adjustment and orthorectification processing based on the SAR image after suppressing speckle noise, the original spaceborne optical image, and the RPC positioning model, to obtain the orthorectified optical and SAR images including:

[0247] Using the original spaceborne optical image control points and the SAR image control points after multi-look filtering processing, the system error of the RPC model is compensated through the constraint relationship between the control points of the images;

[0248]

[0249] Where (x, y) is the measured coordinates on the original spaceborne optical image and the SAR image after multi-look filtering processing, (sample, line) is the projection value of the ground control point onto the image surface using the RPC parameters;

[0250] List the following linear equation for each control point and perform bundle adjustment:

[0251] V=At+BX-1

[0252]

[0253] t=(Δpx0 ΔpxX1 Δpx2 Δpy0 Δpy1 Δpy2) T,

[0254] X=(Alat Δlon Aheitht) T ,

[0255] Where V is the residual, A is the coefficient matrix of the affine parameters, B is the coefficient matrix of the object coordinates, t is the affine parameter correction number, X is the object coordinate correction number, and l is the constant term.

[0256] Among them, the RPC model is formed by fitting the original optical image RPC model and the SAR image;

[0257] Based on the external reference DEM data, the spaceborne optical data and the RPC parameters after the SAR image block adjustment, the spaceborne optical and multi-look filtering processed SAR images are orthorectified.

[0258] According to the second aspect of the present invention, the seventh processing module 107 is configured to perform mosaic processing on the optical and SAR images after the uniform light and color processing to obtain the mosaicked optical and SAR orthorectified products, including:

[0259] Adding avoidance points within a predetermined area of the optical and SAR images after the uniform light and color processing, and adjusting the tones of the images in the local areas on both sides of the mosaic line; by adding avoidance points and editing the mosaic line, the tones between the images in the predetermined area are uniformly transitioned;

[0260] According to the second aspect of the present invention, the eighth processing module 108 is configured to expand the minimum bounding rectangle of the coordinates of the center points of the pixels of the four corner points of the outline within the standard frame by a preset value P pixels. The coordinates of the center points of the pixels of the four corner points of the outline within the standard frame are calculated using the following formula:

[0261] X min =int[min(X1,X2,X3,X4) / d]×dP×d

[0262] Y min =int[min(Y1, Y2, Y3, Y4)d]×dP×d

[0263] X max =int[max(X1,X2,X3,X4) / d+1]×d+P×d

[0264] Y max =int[max(Y1, Y2, Y3, Y4) / d+1]×d+P×d

[0265] Where (X1, Y1), (X2, Y2), (X3, Y3), and (X4, Y4) are the coordinates of the four outline points, and the unit of coordinate is meters; d is the ground resolution of the orthophoto, int[·] rounds the number down, max(·) returns the maximum value in the parameter list, and min(·) returns the minimum value in the parameter list.

[0266] The present invention discloses a method and system for producing a regional satellite-borne optical and SAR image DOM. Based on the Resource-3 digital orthophoto and external existing DEM data, the principle of feature consistency matching between the satellite-borne optical image and the digital orthophoto is used to obtain the corresponding image point coordinates between the optical and digital orthophoto libraries. At the same time, based on multi-view processing, a SAR image speckle noise suppression network is used to filter the image to improve image quality. The feature representation capability of a deep convolutional neural network for heterogeneous images is utilized to match image control points between the filtered SAR image and the Resource-3 digital orthophoto. Finally, the extracted image control points and external existing DEM data are used for optical and SAR joint regional block adjustment and orthorectification processing. At the same time, the large color and hue differences between images taken at different times in the same area due to different satellite image payloads, imaging times, and incident angles, as well as the differences between optical and SAR sensors, are solved. A new advance-bundle method is proposed. A dodging point editing strategy is used to provide a technical route that can support joint light and color grading and mosaic line editing of regional optical and SAR satellite images, effectively ensuring the product quality of regional optical and SAR image joint DOM in actual production.

[0267] 1) The present invention effectively utilizes the existing resource No. 3 optical reference base map to automatically match optical and SAR images to obtain the control points required for regional block adjustment, reducing the inefficiency caused by manual intervention or the point error caused by manual point selection, and can realize automatic and high-precision correction processing of regional spaceborne optical and SAR images.

[0268] 2) Based on multi-view processing, the SAR image speckle noise suppression network is used to filter the image. The backbone network is UNet, and ResNet is used as the encoder. The training data consists of multi-phase SAR images and simulated noise, which improves the image quality.

[0269] 3) Leveraging the deep convolutional neural network's ability to represent heterogeneous imagery, we performed image control point matching between filtered SAR images and the Resource-3 digital orthophoto. Using images that matched the SAR and optical image templates as positive samples and non-matching SAR images as negative samples, we used a convolutional network to extract the template image's feature tensor, which served as a similarity measure for SAR and optical image registration. This yielded the control points for the filtered SAR images.

[0270] This paper addresses the significant color and hue discrepancies between images of the same region taken at different times due to differences in satellite image payload, imaging time, and angle of incidence, as well as differences between optical and SAR sensors. A new advance-bundle method is proposed to address these discrepancies in hue caused by differences in optical and SAR imaging mechanisms. To eliminate these hue inconsistencies, a corresponding dodging algorithm is developed to estimate the optimal hue for adjacent regions. This approach leverages proven dodging and color matching techniques and mosaic line editing to effectively ensure the quality of the combined DOM (Domain Mapping) of regional optical and SAR imagery.

[0271] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the electronic device implements the steps of any one of the methods for producing regional spaceborne optical and SAR images (DOM) disclosed in the first aspect of the present invention.

[0272] Figure 13 FIG. 1 is a structural diagram of an electronic device according to an embodiment of the present invention. Figure 13 As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the electronic device housing, or an external keyboard, touchpad or mouse.

[0273] Those skilled in the art will understand that Figure 13 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0274] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for producing a regional spaceborne optical and SAR image DOM according to any one of the first aspects of the present invention.

[0275] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application. The description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of this application, several variations and improvements can be made, which all fall within the scope of protection of this application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A method for producing regional spaceborne optical and SAR images DOM, characterized in that: The method comprises: Step S1, selecting the original spaceborne optical image and its RPC model, the original spaceborne SAR image and its RD model, the digital orthophoto library and the external reference DEM data in the survey area; Step S2, performing RPC modeling on the RD model in the original spaceborne SAR image to obtain an RPC positioning model of the SAR image, and suppressing speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise; Step S21: In combination with the characteristics of the optical image RPC model, the original SAR image strict imaging geometry model is used to establish the correspondence between the three-dimensional spatial grid of ground points and the image surface as control points to solve the RPC parameters, and the RPC model after fitting is obtained, that is, the RPC positioning model of the SAR image is obtained; Step S22: performing multi-look processing in azimuth or range on the original spaceborne SAR image single-look slant range complex data (SLC) based on the output DOM and the resolution of the optical image involved in data processing to preliminarily suppress speckle noise and obtain a SAR image with preliminarily suppressed speckle noise; based on the multi-look processing, performing filtering processing on the SAR image with preliminarily suppressed speckle noise using a SAR image speckle noise suppression network to obtain a SAR image with suppressed speckle noise; In step S22, the SAR image speckle noise suppression network is used to filter the SAR image after the speckle noise is initially suppressed. The specific method for obtaining the SAR image after the speckle noise is suppressed includes: 1) The backbone network of the SAR image speckle noise suppression network is UNet, which consists of two parts: encoder and decoder. The encoder uses an 18-layer ResNet instead of a four-fold downsampling CBR module stacked with Conv+BN+ReLU, and the decoder uses a four-fold upsampling CBR module stack; 2) Obtain the intensity of the sample and the SAR image to be filtered, and use the logarithm log The operation converts the multiplicative speckle noise into additive noise; the additive noise image is obtained, namely the sample additive noise image and the SAR image additive noise image; The SAR image to be filtered is the SAR image after the speckle noise is initially suppressed; the sample is the SAR image with superimposed synthetic speckle noise, which is also the input data used for noise suppression network training; The true value of the model parameter to be estimated θ Mapping relationship The noise-free image intensity is estimated by , noise-free image intensity x The relationship between the noisy image observation y is as follows: ； in: x : noise-free image intensity; : estimation of noise-free image intensity; y: x The observations of contain additive noise; right x Two observations of y 1 and y 2. Use the following formula to establish the denoising objective function: ; : Estimation of the model parameters to be estimated E: Mathematical Expectation :Observed value y Obtain noise-free image intensity estimation through filtering and denoising The mapping relationship : Get an estimate of θ so that E[·] takes the minimum value; in, is the loss function, which has the following form: in, y 1 and y 2 are x Two independent observations of k is the pixel number; 3) The model is trained on the SAR images with the superimposed synthetic speckle noise as samples. Each iteration generates two independent sets of synthetic speckle noise SAR images, which are used as input and for calculating the loss function respectively. The samples were divided into 256×256 slices with a step size of 32 to form the training set. The number of training epochs was not less than 50. The training network used the Adam optimizer with an initial learning rate of 0.001, which was reduced to 1 / 10 of the initial learning rate after 10 epochs. The training network is pre-trained and re-trained, wherein the pre-training process generates down-sampled denoised SAR images to compensate for changes in SAR images at different periods, and the downsampling is used to reduce the correlation of synthetic noise; the re-training process updates the pre-trained weights, and the number of epochs is not less than 20; Step S3: matching image control points between the original spaceborne optical image and the digital orthophoto based on the principle of consistency of optical image features to obtain the original spaceborne optical image control points; Step S4: using the similarity of heterogeneous images in the deep convolutional neural network space, image control point matching is performed between the SAR image after suppressing speckle noise and the digital orthophoto image to obtain the SAR image control points after filtering; Step S41: using the image obtained by matching the speckle noise suppressed SAR image with the optical image template as a positive sample and the non-matching speckle noise suppressed SAR image as a negative sample, using a convolutional network to extract the feature tensor of the optical image template as a similarity measure for the registration between the speckle noise suppressed SAR image and the optical image template; Step S42: A deep matching network is used to drive the positive and negative samples, and its loss function is designed as: Under a given threshold, the overall distance between the positive sample and the optical image template in the feature space is made smaller than that between the negative sample, so that the weight obtained by training has the ability to determine whether the SAR image after suppressing speckle noise matches the optical image template during the inference process. 1) Select patches with obvious point and line features from optical and radar images as control point samples. Use the optical image feature patches as anchor points, radar patches with the same name as positive samples, and radar patches with different names as negative samples. The optical image is converted into a feature tensor through a convolutional network, and the loss function is established based on this: ; in, is the loss function, To distinguish the distance between matching and non-matching, Δ + and Δ - are the Euclidean distances between the positive and negative samples and the optical spot anchor feature tensor: ; F ( t ), F ( g )and F ( b ) are the feature tensors of the optical anchor point, positive sample, and negative sample respectively; 2) The deep matching network consists of a 6-layer convolutional network. The first 5 layers are CBR modules stacked with Conv+BN+ReLU. The last layer uses 8 convolutional connections and uses L2 regularization and 0.3 dropout to prevent overfitting. The deep matching network input is a 64×64 image slice and the output is a 128-dimensional unit vector. 3) Based on the above loss function, the deep matching network selects all negative spots with close Euclidean distance of feature tensors in the optical and radar sample images as samples, and sets the edge distance between positive and negative spots in the feature space. , if Δ + + >Δ - , then readjust the weight until Δ + < Δ - ; 4) The network is pre-trained using medium-resolution optical and radar imagery at the order of 10 m. The network uses the Adam optimizer with a learning rate of 0.1 and a minimum of 50 epochs. For high-resolution imagery at the order of 1 m, secondary training is performed with a learning rate of 0.001 and a minimum of 100 epochs. Step S5, respectively utilize described original spaceborne optical image control points and the SAR image control points after filtering process, carry out joint block adjustment and orthorectification process based on the SAR image after described suppression of speckle noise and original spaceborne optical image and RPC positioning model, obtain orthorectified rear optical and SAR image; Step S51: using the original satellite-borne optical image control points and the SAR image control points after multi-look filtering processing, and compensating the system error of the RPC model through the constraint relationship between the control points of the images; ; Where, are the measured coordinates on the original spaceborne optical image and the SAR image after multi-look filtering processing, The projection value of the ground control point projected onto the image surface using the RPC parameters; List the following linear equation for each control point and perform bundle adjustment: ; Where V is the residual, A is the coefficient matrix of the affine parameters, B is the coefficient matrix of the object coordinates, t is the affine parameter correction number, X is the object coordinate correction number, and l is the constant term. Among them, the RPC model is formed by fitting the original optical image RPC model and the SAR image; Step S52, based on the RPC parameters after the external reference DEM data, spaceborne optics and SAR image block adjustment, orthorectifies the spaceborne optics and multi-look filtering processed SAR image; Step S6: performing automatic light and color homogenization processing on the orthorectified optical and SAR images to obtain light and color homogenization processed optical and SAR images; Step S7, mosaicking the optical and SAR images after the uniform light and color processing to obtain mosaicked optical and SAR orthorectified products; Adding avoidance points within a predetermined area of the optical and SAR images after the uniform light and color processing, and adjusting the tones of the images in the local areas on both sides of the mosaic line; by adding avoidance points and editing the mosaic line, the tones between the images in the predetermined area are uniformly transitioned; Step S8, cropping the mosaicked optical and SAR orthorectified products according to standard framing to obtain standard framing optical and SAR image DOM products; In step S8, the minimum bounding rectangle of the coordinates of the center points of the pixels of the four corner points of the outline in the standard frame is expanded outward by a preset value P pixels. The coordinates of the center points of the pixels of the four corner points of the outline in the standard frame are calculated as follows: ; Where: (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4) are the coordinates of the four outline points, and the coordinate unit is meter; d is the ground resolution of the orthophoto, int[•] rounds the number down, max(•) returns the maximum value in the parameter list, and min(•) returns the minimum value in the parameter list.

2. A system for producing regional spaceborne optical and SAR images DOM, the system adopting the method of claim 1, characterized in that: The system comprises: The first processing module is configured to select the original spaceborne optical image and its RPC model, the original spaceborne SAR image and its RD model, the digital orthophoto library and the external reference DEM data within the survey area; The second processing module is configured to perform RPC modeling on the RD model in the original spaceborne SAR image to obtain an RPC positioning model of the SAR image, and suppress speckle noise of the SAR image by filtering to obtain a SAR image after suppressing speckle noise; The third processing module is configured to match the image control points between the original spaceborne optical image and the digital orthophoto based on the principle of consistency of optical image features to obtain the control points of the original spaceborne optical image; a fourth processing module configured to perform image control point matching between the SAR image after suppressing speckle noise and the digital orthophoto image by utilizing the similarity of heterogeneous images in the deep convolutional neural network space, and obtain the control points of the filtered SAR image; a fifth processing module configured to respectively use the original spaceborne optical image control points and the filtered SAR image control points, and perform joint block adjustment and orthorectification processing based on the speckle noise suppressed SAR image and the original spaceborne optical image and the RPC positioning model to obtain orthorectified optical and SAR images; a sixth processing module configured to perform automatic light and color homogenization processing on the orthorectified optical and SAR images to obtain light and color homogenized optical and SAR images; A seventh processing module is configured to perform mosaic processing on the optical and SAR images after the uniform light and color processing to obtain mosaicked optical and SAR orthorectified products; The eighth processing module is configured to perform cropping processing on the mosaicked optical and SAR orthorectified products according to standard framing to obtain standard framing optical and SAR image DOM products.

3. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the method for producing regional spaceborne optical and SAR images DOM according to claim 1 are implemented.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for producing regional spaceborne optical and SAR images DOM according to claim 1 are implemented.

Images