Spatial radiation normalization method, system and device for large-area multi-source data

Abstract

The present invention discloses a method, system and device for normalizing spatial radiation for multi-source data in a large area. The method includes: generating a multi-cause radiation normalization image database based on a remote sensing observation image set and forming a sample image pair; constructing a generative adversarial network, fusing the sample image pair through a super-resolution module to form a fused definition image, forming a radiation consistent image through a style transfer module, optimizing the radiation consistent image through a normalization module to form a normalized image to complete the optimization, and obtaining a normalized adversarial pre-training model; training the normalized adversarial pre-training model through a multi-cause radiation normalization database to obtain a normalized adversarial model; processing the spatial radiation image to be normalized based on the normalized adversarial model to obtain the corresponding spatial radiation normalized image. The present invention can deeply mine the correlation between the spatiotemporal spectrum of data and solve the problem of complex nonlinear radiation inconsistency and global overall optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing data processing, and in particular to a method, system and device for normalizing spatial radiation of multi-source data in a large area. Background Art

[0002] Massive satellite remote sensing earth observation data provides rich data resources for various applications. It has powerful service capabilities. Therefore, remote sensing classification can make the data of large-area geographic census and natural resource survey more accurate. However, there are certain limitations in the actual application process. For example, although there is a large amount of data, there are often problems of data shortage, lack of spatiotemporal integrity and comparability in the application process, which restricts the in-depth research and development of technology and the promotion and application of methods. The crux of the contradiction is the incompatibility and inconsistency between multi-source data, which limits data integration and collaborative application. At the same time, this crux also affects the generalization ability of classification methods. Deep learning technology performs well in classification tasks due to its advantages in feature mining and nonlinear problem solving. However, the distribution consistency between test / training data is the premise of model generalization application. The radiation inconsistency between multi-source data will aggravate the data set offset and reduce the migration and generalization ability of deep learning models across time and space and across sensors, which is more prominent in large-area classification tasks.

[0003] There are many reasons for the inconsistency of remote sensing data radiation, such as data source problems, a large increase in data, or insufficient data quality. When applied to large areas, the increase in data volume and inconsistent data quality exacerbate the complexity of the problem. Different from the radiation normalization between two pairs of images, the overall consistency requirements are relatively high in large-area applications. However, due to the availability of overlapping areas and pseudo-invariant pixels, the association between multi-source data cannot be guaranteed, and radiation normalization residuals may occur, resulting in a lack of overall consistency.

[0004] In other words, developing a radiation normalization method for multi-source data in large areas, ensuring the accurate characterization of complex radiation conversion relationships, and achieving global radiation consistency among multi-source data in large areas are key issues that need to be addressed at present. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method, system and device for normalizing spatial radiation of multi-source data in a large area.

[0006] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0007] A spatial radiation normalization method for large-area multi-source data includes the following steps:

[0008] Generate a multi-genetic radiation normalized image database based on the remote sensing observation image set, randomly select two multi-genetic radiation normalized images to form a sample image pair, one of which is used as a reference image, and the other is used as the image to be normalized;

[0009] Constructing a generative adversarial network, wherein the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module, fusing the sample image pairs through the super-resolution module to form a fused definition image, inputting the fused definition image into the style transfer module to form a radiation consistent image, optimizing the radiation consistent image through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model;

[0010] The normalized adversarial pre-training model is trained through the multi-cause radiation normalization database to obtain a normalized adversarial model;

[0011] Based on the normalized adversarial model, the spatial radiation image to be normalized is processed to obtain the corresponding spatial radiation normalized image.

[0012] As an implementable method, the method of forming a uniform definition image from a multi-genesis radiation normalized image by a super-resolution module comprises the following steps:

[0013] Encoding is performed at the same position of the sample image pair respectively, and a geometrically aligned sample image pair is obtained based on the relevant information of the encoding;

[0014] Extracting features from the geometrically aligned sample image pairs respectively to obtain a first feature and a second feature and form a feature pair;

[0015] The optical flow field information of the feature pair is extracted through the optical flow alignment module to obtain the motion information of the pixel;

[0016] Based on the optical flow field information, the pixels are adjusted according to the motion information, and the features between the sample image pairs are aligned;

[0017] The texture information of the reference image is fused into the image to be normalized through the attention mechanism to obtain a fused clarity image.

[0018] As an implementable method, the step of inputting the fused definition image into the style transfer module to form a radiation consistent image includes the following steps:

[0019] The fused definition image is changed into a preset size image, and the preset size image is processed through a convolution layer to obtain pixel-by-pixel features;

[0020] Analyze the correlation between pixel-by-pixel features to obtain the spatial information matrix;

[0021] Preserving the features of ground object information based on the spatial information matrix;

[0022] Based on the fused definition image that retains the information features of the ground objects, a radiation consistent image is reconstructed.

[0023] As an implementable method, the fused definition image is changed into an image of a preset size, specifically:

[0024] Input the image of preset size into the convolution layer to obtain the pixel-by-pixel feature vector;

[0025] Keeping the feature map unchanged, the pixel-by-pixel feature vectors between different channels are fused, and the position encoding is added to restore the original sequence arrangement, where the pixel-by-pixel feature vector is obtained by adding the feature vector and the position encoding.

[0026] As an implementable method, obtaining the spatial information matrix includes the following steps:

[0027] The self-attention unit analyzes the self-similar attention weight matrix of the pixel-by-pixel features, and converts the pixel-by-pixel features into corresponding query embedding, key embedding, and value embedding through the projection matrix;

[0028] The attention score is calculated based on the query embedding and key embedding, and is scaled according to a preset ratio, converted through a softmax function, and normalized for similarity;

[0029] The spatial position information of the position encoding is used to restore and combine the attention scores of the pixel-by-pixel feature vectors to be consistent with the spatial dimension of the input image, and a normalized weight matrix is ​​generated to obtain a radiation consistent image.

[0030] It is expressed as follows:

[0031]

[0032]

[0033]

[0034]

[0035] in, , and They represent the projection matrix, represents query embedding, represents key embedding, represents value embedding, represents the attention score, Indicates the preset ratio, An input sequence representing pixel-wise features.

[0036] As an implementable method, the step of optimizing the radiation consistent image by a normalization module to form a normalized image includes the following steps:

[0037] Normalizing the radiation consistent image to obtain normalized multi-source time-series image data, wherein the multi-source time-series image data at least includes a large scene image and other large scene images;

[0038] Based on the normalized multi-source time-series image data, the overlapping areas are matched with non-locally similar pixels through a variational framework.

[0039] Construct a weight matrix, construct a pre-condition regularization prior model and retain the gradient consistency constraint, so that the weight of non-local similar pixels in the condition regularization prior result reaches the minimum value, and obtain the best normalized image;

[0040] The conditional regularization prior model is expressed as follows:

[0041]

[0042] in, , Respectively represent the normalized multi-source time series image data, represents the gradient operator, B represents the number of bands, Represents the entire space of normalized multi-source time series image data, , represent the constraint coefficients, Represents the weight of non-locally similar pixels.

[0043] As an implementable method, generating a normalized database of multi-genesis radiation includes the following steps:

[0044] Based on the atmospheric radiation transfer model, the atmospheric radiation transfer process is simulated to obtain remote sensing observation data under different atmospheric conditions;

[0045] The data between different sensors are converted based on the spectral response function to obtain a converted remote sensing observation image data set, specifically: obtaining the spectral response function, converting the response value of one sensor to the band of another sensor based on the spectral response function, and calculating the spectral matching conversion factor between the two sensors by comparing the spectral response functions of the two sensors; applying the conversion factor to the remote sensing observation data to obtain a converted remote sensing observation image data set;

[0046] The converted remote sensing observation image data set is resampled by area weighting to generate a multi-genetic radiation normalized database;

[0047] Among them, image resampling is performed by area weighting, specifically: calculating the actual ground area of ​​each pixel in the converted remote sensing observation image;

[0048] The weight of each pixel is obtained by dividing the actual ground area of ​​each pixel by the sum of the areas of all pixels, and then the weighted average is obtained through the weight of each pixel to achieve image resampling.

[0049] As an implementable method, the following steps are also included:

[0050] The spatial domain radiation normalized image is preprocessed to obtain weight file information, wherein the weight file information is applied in an affine reconstruction model.

[0051] A spatial radiation normalization system for large-area multi-source data, including a database construction module, a model construction module, a model training module and a model reasoning module;

[0052] The database construction module is used to generate a multi-genetic radiation normalized image database based on a remote sensing observation image set, and randomly select two multi-genetic radiation normalized images to form a sample image pair, wherein one multi-genetic radiation normalized image is used as a reference image, and the other multi-genetic radiation normalized image is used as an image to be normalized;

[0053] The model construction module is used to construct a generative adversarial network, and the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module. The sample image pair is fused through the super-resolution module to form a fused definition image, the fused definition image is input into the style transfer module to form a radiation consistent image, and the radiation consistent image is optimized through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model;

[0054] The model training module is used to train the normalized adversarial pre-training model through the multi-cause radiation normalization database to obtain a normalized adversarial model;

[0055] The model reasoning module processes the spatial radiation image to be normalized based on the normalized adversarial model to obtain a corresponding spatial radiation normalized image.

[0056] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the following method:

[0057] Generate a multi-genetic radiation normalized image database based on the remote sensing observation image set, randomly select two multi-genetic radiation normalized images to form a sample image pair, one of which is used as a reference image, and the other is used as the image to be normalized;

[0058] Constructing a generative adversarial network, wherein the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module, fusing the sample image pairs through the super-resolution module to form a fused definition image, inputting the fused definition image into the style transfer module to form a radiation consistent image, optimizing the radiation consistent image through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model;

[0059] The normalized adversarial pre-training model is trained through the multi-cause radiation normalization database to obtain a normalized adversarial model;

[0060] Based on the normalized adversarial model, the spatial radiation image to be normalized is processed to obtain the corresponding spatial radiation normalized image.

[0061] A device for normalizing spatial radiation of multi-source data in a large area, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following method when executing the computer program:

[0062] Generate a multi-genetic radiation normalized image database based on the remote sensing observation image set, randomly select two multi-genetic radiation normalized images to form a sample image pair, one of which is used as a reference image, and the other is used as the image to be normalized;

[0063] Constructing a generative adversarial network, wherein the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module, fusing the sample image pairs through the super-resolution module to form a fused definition image, inputting the fused definition image into the style transfer module to form a radiation consistent image, optimizing the radiation consistent image through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model;

[0064] The normalized adversarial pre-training model is trained through the multi-cause radiation normalization database to obtain a normalized adversarial model;

[0065] Based on the normalized adversarial model, the spatial radiation image to be normalized is processed to obtain the corresponding spatial radiation normalized image.

[0066] The present invention has significant technical effects due to the adoption of the above technical solution:

[0067] The present invention constructs a generative adversarial network, which includes a super-resolution module, a style transfer module and a normalization module, and optimizes the generative adversarial network to obtain a normalized adversarial pre-training model, and then trains the normalized adversarial pre-training model through a multi-cause radiation normalization database to obtain a normalized adversarial model, and then infers the normalized image to obtain a spatial domain radiation normalized image. The present invention can deeply mine the correlation relationship of the spatiotemporal spectrum of data, and solves the problem of complex nonlinear radiation inconsistency and global overall optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0069] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;

[0070] Figure 2 It is a schematic diagram of the overall structure of the system of the present invention;

[0071] Figure 3 It is a schematic diagram of some technical processes of a specific embodiment of the present invention;

[0072] Figure 4 It is a flowchart of spatial interactive attention according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0073] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.

[0074] Embodiment 1:

[0075] A spatial radiation normalization method for multi-source data in large areas, such as Figure 1 As shown, the following steps are included:

[0076] S100, generating a multi-genetic radiation normalized image database based on a remote sensing observation image set, randomly selecting two multi-genetic radiation normalized images to form a sample image pair, wherein one multi-genetic radiation normalized image is used as a reference image, and the other multi-genetic radiation normalized image is used as an image to be normalized;

[0077] S200, constructing a generative adversarial network, wherein the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module, fusing the sample image pair through the super-resolution module to form a fused definition image, inputting the fused definition image into the style transfer module to form a radiation consistent image, optimizing the radiation consistent image through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model;

[0078] S300, training a normalized adversarial pre-training model using a multi-cause radiation normalization database to obtain a normalized adversarial model;

[0079] S400: Process the spatial radiation image to be normalized based on the normalized adversarial model to obtain a corresponding spatial radiation normalized image.

[0080] By constructing a generative adversarial network, the generative adversarial network includes a super-resolution module, a style transfer module and a normalization module, and optimizing the generative adversarial network to obtain a normalized adversarial pre-trained model, and then training the normalized adversarial pre-trained model through a multi-cause radiation normalization database to obtain a normalized adversarial model, and then reasoning is performed on the normalized image to obtain a spatial domain radiation normalized image.

[0081] The normalized adversarial models of the super-resolution module, style transfer module and normalization module are trained, and global consistent optimization of large-area images is achieved based on the normalized adversarial model; the whole process can mine the time-spectral information of time-spectral sequence data, learn self-similarity weights, screen and enhance stable features, and coordinate self-similarity weights, time-spectral autoregression and similarity to obtain a spatial domain radiation normalized image; the present invention can deeply mine the correlation between the spatiotemporal spectrum of data, and solve the problems of complex nonlinear radiation inconsistency and global overall optimization.

[0082] The existing spatial radiation normalization methods basically ignore the scale diversity of images and the differences of sensors themselves, which will cause the error transmission of geometric deviation and topological relationship mapping in the radiation normalization based on a single radiation normalization model. Therefore, the core of the present invention lies in the optimization of the generative adversarial network. The constructed generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module. The sample image pairs are fused through the super-resolution module to form a fused clarity image, and the fused clarity image is input into the style transfer module to form a radiation consistent image. The radiation consistent image is optimized through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model. The detailed process is shown in the following specific steps:

[0083] Firstly, the sample image pairs are fused through the super-resolution module to form a fused clarity image, that is: encoding is performed at the same position of the sample image pair respectively, and a geometrically aligned sample image pair is obtained based on the relevant information of the encoding; feature extraction is performed on the geometrically aligned sample image pair respectively to obtain the first feature and the second feature and form a feature pair; the optical flow field information of the feature pair is extracted through the optical flow alignment module to obtain the motion information of the pixel; based on the optical flow field information, the pixels are adjusted according to the motion information, and the features between the sample image pairs are aligned; the texture information of the reference image is fused into the image to be normalized through the attention mechanism to obtain a fused clarity image.

[0084] Firstly, coupled optical flow alignment is used to construct a generative adversarial super-resolution structure, and the encoding is performed at the same position of the sample image pair. Here, one multi-genesis radiation normalized image is used as the reference image, and the other multi-genesis radiation normalized image is used as the image to be normalized. The input reference image and the image to be normalized are respectively subjected to multiple convolutional layers to extract multi-scale reference features, and the corresponding position encoding is given to the paired reference image and the image to be normalized to achieve geometric alignment of the features; the optical flow field information between the paired input features is extracted, and the optical flow field is used to estimate and adjust the pixels according to the motion information based on the optical flow field information to achieve feature alignment between the reference image and the image to be normalized, and the attention effect is further used to better transfer the texture information of the reference image to the image to be normalized to obtain a fused clarity image.

[0085] In the optical flow alignment processing module, a 4-layer branch network framework is designed. Multiple optical flow estimates at different frame intervals are used to perform motion compensation on the low-resolution video frame images of the entire input network in sequence, and then deep features are extracted and corrected through multi-layer dense residual blocks. Among them, the optical flow field estimation is built by the optical flow spatial pyramid network (SPyNet) estimation method. SPyNet has lightweight model parameters and has excellent accuracy and speed during model training. Optical flow field estimation first needs to calculate the optical flow field information between adjacent frames in the image sequence. The optical flow field information referred to in this field refers to the pattern of pixel movement on the surface of an object in an image sequence, and the speed and direction of movement are estimated by tracking feature points or pixels in the image. After the optical flow field information is calculated, the feature points in the image are tracked to estimate their movement between consecutive frames; according to the optical flow field information, each pixel in the reference image is adjusted. For example, if a pixel is located at (i, j) in the first frame and moves to (i+u, j+v) in the second frame due to motion, the pixel position needs to be adjusted according to the offset. Feature alignment: By adjusting the pixels, feature points can be aligned between different images; after feature alignment, some post-processing steps may be required, such as removing artifacts caused by inaccurate motion estimation or enhancing the quality of the aligned image.

[0086] Next, in one embodiment, the fused clarity image is input into a style transfer module to form a radiation consistent image, specifically: the fused clarity image is changed into an image of a preset size, and the preset size image is processed through a convolution layer to obtain pixel-by-pixel features; the correlation between the pixel-by-pixel features is analyzed to obtain a spatial information matrix; the ground object information features are retained based on the spatial information matrix; and the radiation consistent image is reconstructed based on the fused clarity image with the retained ground object information features.

[0087] Among them, the fused clarity image is changed into an image of a preset size, specifically: the image of the preset size is input into the convolution layer to obtain a pixel-by-pixel feature vector; the feature map is kept unchanged, the pixel-by-pixel feature vectors between different channels are fused, and the position code is added to restore the original sequence arrangement, wherein the pixel-by-pixel feature vector is obtained by adding the feature vector and the position code.

[0088] The spatial information matrix used in this embodiment is obtained in the following way:

[0089] The self-similar attention weight matrix of the pixel-by-pixel features is analyzed through the self-attention unit, and the pixel-by-pixel features are converted into corresponding query embedding, key embedding and value embedding through the projection matrix; the attention score is calculated based on the query embedding and key embedding, and scaled according to the preset ratio, converted through the softmax function, and the similarity is normalized; the attention score of the pixel-by-pixel feature vector is restored and combined into a consistent spatial dimension with the input image using the spatial position information of the position encoding, and a normalized weight matrix is ​​generated to obtain a radiation consistent image;

[0090] It is expressed as follows:

[0091]

[0092]

[0093]

[0094]

[0095] in, , and They represent the projection matrix, represents query embedding, represents key embedding, represents value embedding, represents the attention score, Indicates the preset ratio, An input sequence representing pixel-wise features.

[0096] In this embodiment, spatial interactive attention is used to capture the correlation between the aligned feature vectors, and three independent expansion and deformation operation networks are used to extract the spatial information matrix of each frame for the input fused definition image. , , , by processing the spatial information matrix to retain the information characteristics of the ground objects.

[0097] The image of size M*N*(C*T) is passed through a 1*1 convolutional layer to obtain a pixel-by-pixel feature vector, the features between different channels are fused without changing the size of the feature map, and position or spatial encoding is added to restore the original sequence arrangement. The representation vector of the pixel-by-pixel feature The output feature is obtained by adding the feature vector and the position code, and the self-attention block learns the self-similar attention weight matrix, that is, the input embedding First, through the learnable projection matrix , and , converted into a query ,key Sum Embed. Utilize and Embedding dot multiplication to calculate attention score ,according to After being scaled by the ratio, it is transformed through the softmax function, the similarity is normalized, and the spatial position information provided by the position encoding is used to restore and combine the pixel-by-pixel attention scores to be consistent with the spatial dimension of the input image, generating a normalized weight matrix.

[0098] In order to reduce the impact of diverse changes in ground objects in remote sensing images of different periods on the quality of pseudo high-resolution image reconstruction, and eliminate noise information in the reference image and the image to be normalized, the present invention incorporates an improved convolutional attention module into the residual network of the generator full convolution connection to reduce the normalized image generation error, and makes the interdependence between features adaptively adjust the feature weights.

[0099] The multi-source time series image data obtained by spatial radiation normalization still has problems such as spatial consistency in overlapping areas and residual radiation differences in non-local spatial similarities in non-overlapping areas, which makes it difficult to meet the high-precision and sophisticated application requirements of large-area time series. That is, it is necessary to optimize the radiation-consistent image through the normalization module to form a normalized image, specifically:

[0100] Normalizing the radiation consistent image to obtain normalized multi-source time-series image data, wherein the multi-source time-series image data at least includes a large scene image and other large scene images;

[0101] Based on the normalized multi-source time-series image data, the overlapping areas are matched with non-locally similar pixels through a variational framework.

[0102] Construct a weight matrix, construct a pre-condition regularization prior model and retain the gradient consistency constraint, so that the weight of non-local similar pixels in the condition regularization prior result reaches the minimum value, and obtain the best normalized image;

[0103] The conditional regularization prior model is expressed as follows:

[0104]

[0105] in, , Respectively represent the normalized multi-source time series image data, represents the gradient operator, B represents the number of bands, Represents the entire space of normalized multi-source time series image data, , represent the constraint coefficients, Represents the weight of non-locally similar pixels.

[0106] The non-local similarity regularization constraint term is constructed between the non-overlapping areas of the images. First, the non-local similar pixels of the non-overlapping images are matched and the weights are established. Image blocks are randomly sampled in the radiation-position space, and their non-local similar image blocks are searched on other images. The spatial distance and structural similarity are integrated to construct the similar pixel weights. As a precondition of non-local constraints, a conditional regularization prior model is constructed to make full use of the radiation complementary relationship and spatial consistency between the bands of optical remote sensing images to maintain the complete and true structural information of the image.

[0107] In one embodiment, generating a normalized multi-genesis radiation database comprises the following steps:

[0108] Based on the atmospheric radiation transfer model, the atmospheric radiation transfer process is simulated to obtain remote sensing observation data under different atmospheric conditions;

[0109] The data between different sensors are converted based on the spectral response function to obtain a converted remote sensing observation image data set, specifically: obtaining the spectral response function, converting the response value of one sensor to the band of another sensor based on the spectral response function, and calculating the spectral matching conversion factor between the two sensors by comparing the spectral response functions of the two sensors; applying the conversion factor to the remote sensing observation data to obtain a converted remote sensing observation image data set;

[0110] The converted remote sensing observation image data set is resampled by area weighting to generate a multi-genetic radiation normalized database;

[0111] Among them, image resampling is performed by area weighting, specifically: calculating the actual ground area of ​​each pixel in the converted remote sensing observation image;

[0112] The weight of each pixel is obtained by dividing the actual ground area of ​​each pixel by the sum of the areas of all pixels, and then the weighted average is obtained through the weight of each pixel to achieve image resampling.

[0113] In this embodiment, the atmospheric radiation transmission 6S model is used to simulate the atmospheric radiation transmission process, and the remote sensing observation data under different atmospheric conditions are simulated. The 6S model is mainly used to simulate the transmission process of solar radiation in the ground-atmosphere system and calculate the radiation brightness at the satellite entrance pupil. It can consider the absorption of water vapor, CO2, O3 and O2, the scattering of molecules and aerosols, and the influence of non-uniform ground and bidirectional reflectivity distribution under the assumption of a cloudless atmosphere. The input parameters of the 6S model include: the geometric conditions of the satellite, including the solar zenith angle, the solar azimuth angle, the observation zenith angle, the observation azimuth angle, etc.; the atmospheric model defines the basic composition of the atmosphere and the temperature and humidity profile, and you can choose a preset mode or customize the input measured data; the aerosol model defines the main global aerosol parameters, such as aerosol phase function, asymmetry factor and single scattering albedo; the spectral characteristics of the sensor define the spectral response function of the sensor channel; the surface reflectivity defines the reflectivity model of the surface, including two cases of uniform surface and non-uniform surface.

[0114] Based on the spectral response function, the data conversion between different sensors is realized by calculating the spectral matching conversion factor between two sensors;

[0115] Data resampling is the process of converting a dataset from one resolution or format to another. Resampling can be used to enlarge (upsampling) or reduce (downsampling) a dataset.

[0116] For example, in this embodiment, the following resampling method may be used, specifically:

[0117] from osgeo import gdal, osr

[0118] def resample(input_file, output_file, new_width, new_height,resampling_method):

[0119] # Open the remote sensing observation dataset

[0120] src_ds = gdal.Open(input_file)

[0121] srcband = src_ds.GetRasterBand(1)

[0122] # Create target dataset

[0123] driver = gdal.GetDriverByName('GTiff')

[0124] out_ds = driver.Create(output_file, new_width, new_height, 1,srcband.DataType)

[0125] out_ds.SetProjection(src_ds.GetProjection())

[0126] out_transform = src_ds.GetGeoTransform()

[0127] out_ds.SetGeoTransform(out_transform)

[0128] # Calculate new geographic transformation parameters

[0129] x_res = out_transform[1] float(new_width) / src_ds.RasterXSize

[0130] y_res = out_transform[5] float(new_height) / src_ds.RasterYSize

[0131] out_transform[1] = x_res

[0132] out_transform[5] = y_res

[0133] out_ds.SetGeoTransform(out_transform)

[0134] # Perform resampling

[0135] gdal.ReprojectImage(src_ds, out_ds, src_ds.GetProjection(), out_ds.GetProjection(), resampling_method)

[0136] # Close the dataset

[0137] src_ds = None

[0138] out_ds = None

[0139] # Resample using bilinear interpolation

[0140] resample('input.tif', 'output.tif', 500, 500, gdal.GRA_Bilinear)

[0141] In a specific embodiment, data resampling may be performed by area weighting, and the following code may be used in this embodiment:

[0142] from osgeo import gdal

[0143] import numpy as np

[0144] # Open the raster dataset

[0145] dataset = gdal.Open('path_to_your_raster.tif')

[0146] band = dataset.GetRasterBand(1)

[0147] values ​​= band.ReadAsArray()

[0148] # Get geographic transformation parameters

[0149] geo_transform = dataset.GetGeoTransform()

[0150] lat_res = geo_transform[5]

[0151] lon_res = geo_transform[1]

[0152] # Earth's radius (meters)

[0153] R = 6371007.181

[0154] # Calculate the latitude and longitude range of each pixel

[0155] lat_edges = np.linspace(geo_transform[3], geo_transform[3]+ lat_res values.shape[0], values.shape[0]+ 1)

[0156] lon_edges = np.linspace(geo_transform[0], geo_transform[0]+ lon_res values.shape[1], values.shape[1]+ 1)

[0157] # Calculate the area of ​​each pixel

[0158] area = (lat_edges[1] - lat_edges[0]) (np.pi / 180) R 2 np.abs(np.sin(np.radians(lat_edges[:-1]))) (lon_edges[1] - lon_edges[0])

[0159] # Calculate weights

[0160] weights = area / area.sum()

[0161] # Calculate weighted average

[0162] weighted_mean = np.sum(values weights[:, np.newaxis]) / weights.sum()

[0163] print("Weighted mean:", weighted_mean)

[0164] The whole process is basically to calculate the actual ground area of ​​each pixel in the converted remote sensing observation image, divide the actual ground area of ​​each pixel by the sum of the areas of all pixels, obtain the weight of each pixel, and then obtain the weighted average through each pixel weight to achieve image resampling.

[0165] In a specific embodiment, the obtained spatial radiation normalized image will be used later. The image processed by spatial radiation normalization will make the result of the affine reconstruction model more accurate in subsequent applications. The spatial radiation normalized image is preprocessed, for example, information conversion and other processing are performed to obtain weight file information, wherein the weight file information is applied in the affine reconstruction model.

[0166] In the actual experiment, after generating the multi-cause radiation normalized image database, the present invention Figure 3and attached Figure 4 The technical flow shown in the figure has been used for related experiments, and the experimental results are attached. Figure 3 It can also be seen that the experimental results have achieved global overall optimization and have eliminated the problem of complex nonlinear radiation inconsistency.

[0167] Embodiment 2:

[0168] A spatial radiation normalization system for multi-source data in large areas, such as Figure 2 As shown, it includes a database construction module 100, a model construction module 200, a model training module 300 and a model reasoning module 400;

[0169] The database construction module 100 is used to generate a multi-genetic radiation normalized image database based on a remote sensing observation image set, and randomly select two multi-genetic radiation normalized images to form a sample image pair, wherein one multi-genetic radiation normalized image is used as a reference image, and the other multi-genetic radiation normalized image is used as an image to be normalized;

[0170] The model construction module 200 is used to construct a generative adversarial network. The generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module. The sample image pair is fused through the super-resolution module to form a fused definition image, the fused definition image is input into the style transfer module to form a radiation consistent image, and the radiation consistent image is optimized through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model;

[0171] The model training module 300 is used to train the normalized adversarial pre-training model through the multi-cause radiation normalization database to obtain a normalized adversarial model;

[0172] The model inference module 400 processes the spatial radiation image to be normalized based on the normalized adversarial model to obtain a corresponding spatial radiation normalized image.

[0173] Also included is a conversion application module 500:

[0174] The spatial domain radiation normalized image is preprocessed to obtain weight file information, wherein the weight file information is applied in an affine reconstruction model.

[0175] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0176] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0178] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0180] It should be noted that:

[0181] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0182] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A spatial radiation normalization method for multi-source data in a large area, characterized by: The following steps are involved: Generate a multi-genetic radiation normalized image database based on the remote sensing observation image set, randomly select two multi-genetic radiation normalized images to form a sample image pair, one of which is used as a reference image, and the other is used as the image to be normalized; Constructing a generative adversarial network, wherein the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module, fusing the sample image pairs through the super-resolution module to form a fused definition image, inputting the fused definition image into the style transfer module to form a radiation consistent image, optimizing the radiation consistent image through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model; The normalized adversarial pre-training model is trained through the multi-cause radiation normalization database to obtain a normalized adversarial model; The spatial radiation image to be normalized is processed based on the normalized adversarial model to obtain the corresponding spatial radiation normalized image; The step of optimizing the radiation consistent image by a normalization module to form a normalized image includes the following steps: Normalizing the radiation consistent image to obtain normalized multi-source time-series image data, wherein the multi-source time-series image data at least includes a large scene image and other large scene images; Based on the normalized multi-source time-series image data, the overlapping areas are matched with non-locally similar pixels through a variational framework. Construct a weight matrix, construct a pre-condition regularization prior model and retain the gradient consistency constraint, so that the weight of non-local similar pixels in the condition regularization prior result reaches the minimum value, and obtain the best normalized image; The conditional regularization prior model is expressed as follows: in, , Respectively represent the normalized multi-source time series image data, represents the gradient operator, B represents the number of bands, Represents the entire space of normalized multi-source time series image data, , represent the constraint coefficients, Represents the weight of non-locally similar pixels.

2. The spatial radiation normalization method for large-area multi-source data according to claim 1 is characterized in that: The method of forming a uniform definition image from a multi-origin radiation normalized image by a super-resolution module comprises the following steps: Encoding is performed at the same position of the sample image pair respectively, and a geometrically aligned sample image pair is obtained based on the relevant information of the encoding; Extracting features from the geometrically aligned sample image pairs respectively to obtain a first feature and a second feature and form a feature pair; The optical flow field information of the feature pair is extracted through the optical flow alignment module to obtain the motion information of the pixel; Based on the optical flow field information, the pixels are adjusted according to the motion information, and the features between the sample image pairs are aligned; The texture information of the reference image is fused into the image to be normalized through the attention mechanism to obtain a fused clarity image.

3. The spatial radiation normalization method for large-area multi-source data according to claim 1 is characterized in that: The step of inputting the fused definition image into the style migration module to form a radiation consistent image comprises the following steps: The fused definition image is changed into a preset size image, and the preset size image is processed through a convolution layer to obtain pixel-by-pixel features; Analyze the correlation between pixel-by-pixel features to obtain the spatial information matrix; Preserving the features of ground object information based on the spatial information matrix; Based on the fused definition image that retains the information features of the ground objects, a radiation consistent image is reconstructed.

4. The spatial radiation normalization method for large-area multi-source data according to claim 3 is characterized in that: The step of changing the fused definition image into an image of a preset size is specifically as follows: Input the image of preset size into the convolution layer to obtain the pixel-by-pixel feature vector; Keeping the feature map unchanged, the pixel-by-pixel feature vectors between different channels are fused, and the position encoding is added to restore the original sequence arrangement, where the pixel-by-pixel feature vector is obtained by adding the feature vector and the position encoding.

5. The spatial radiation normalization method for large-area multi-source data according to claim 3 is characterized in that: The step of obtaining the spatial information matrix comprises the following steps: The self-attention unit analyzes the self-similar attention weight matrix of the pixel-by-pixel features, and converts the pixel-by-pixel features into corresponding query embedding, key embedding, and value embedding through the projection matrix; The attention score is calculated based on the query embedding and key embedding, and is scaled according to a preset ratio, converted through a softmax function, and normalized for similarity; The spatial position information of the position encoding is used to restore and combine the attention scores of the pixel-by-pixel feature vectors to be consistent with the spatial dimension of the input image, and a normalized weight matrix is ​​generated to obtain a radiation consistent image. It is expressed as follows: in, , and They represent the projection matrix, represents query embedding, represents key embedding, represents value embedding, represents the attention score, Indicates the preset ratio, An input sequence representing pixel-wise features.

6. The spatial radiation normalization method for large-area multi-source data according to claim 1 is characterized in that: The generating of the normalized multi-genesis radiation database comprises the following steps: Based on the atmospheric radiation transfer model, the atmospheric radiation transfer process is simulated to obtain remote sensing observation data under different atmospheric conditions; The data between different sensors are converted based on the spectral response function to obtain a converted remote sensing observation image data set, specifically: obtaining the spectral response function, converting the response value of one sensor to the band of another sensor based on the spectral response function, and calculating the spectral matching conversion factor between the two sensors by comparing the spectral response functions of the two sensors; applying the conversion factor to the remote sensing observation data to obtain a converted remote sensing observation image data set; The converted remote sensing observation image data set is resampled by area weighting to generate a multi-genetic radiation normalized database; Among them, image resampling is performed by area weighting, specifically: calculating the actual ground area of ​​each pixel in the converted remote sensing observation image; The weight of each pixel is obtained by dividing the actual ground area of ​​each pixel by the sum of the areas of all pixels, and then the weighted average is obtained through the weight of each pixel to achieve image resampling.

7. The spatial radiation normalization method for large-area multi-source data according to claim 1 is characterized in that: The following steps are also included: The spatial domain radiation normalized image is preprocessed to obtain weight file information, wherein the weight file information is applied in an affine reconstruction model.

8. A spatial radiation normalization system for multi-source data in a large area, characterized by: Including database construction module, model construction module, model training module and model reasoning module; The database construction module is used to generate a multi-genetic radiation normalized image database based on a remote sensing observation image set, and randomly select two multi-genetic radiation normalized images to form a sample image pair, wherein one multi-genetic radiation normalized image is used as a reference image, and the other multi-genetic radiation normalized image is used as an image to be normalized; The model construction module is used to construct a generative adversarial network, and the generative adversarial network is provided with a super-resolution module, a style transfer module and a normalization module. The sample image pair is fused through the super-resolution module to form a fused definition image, the fused definition image is input into the style transfer module to form a radiation consistent image, and the radiation consistent image is optimized through the normalization module to form a normalized image, thereby completing the optimization of the generative adversarial network and obtaining a normalized adversarial pre-training model; The model training module is used to train the normalized adversarial pre-training model through the multi-cause radiation normalization database to obtain a normalized adversarial model; The model inference module processes the spatial radiation image to be normalized based on the normalized adversarial model to obtain a corresponding spatial radiation normalized image; The step of optimizing the radiation consistent image by a normalization module to form a normalized image includes the following steps: Normalizing the radiation consistent image to obtain normalized multi-source time-series image data, wherein the multi-source time-series image data at least includes a large scene image and other large scene images; Based on the normalized multi-source time-series image data, the overlapping areas are matched with non-locally similar pixels through a variational framework. Construct a weight matrix, construct a pre-condition regularization prior model and retain the gradient consistency constraint, so that the weight of non-local similar pixels in the condition regularization prior result reaches the minimum value, and obtain the best normalized image; The conditional regularization prior model is expressed as follows: in, , Respectively represent the normalized multi-source time series image data, represents the gradient operator, B represents the number of bands, Represents the entire space of normalized multi-source time series image data, , represent the constraint coefficients, Represents the weight of non-locally similar pixels.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A spatial radiation normalization device for large-area multi-source data, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images