Remote sensing satellite image data management method and system

Abstract

The invention provides a remote sensing satellite image data management method and system, and the method comprises the steps: constructing a denoising and cloud and mist correction model based on a deep learning network CNN and a GAN, and carrying out the combination to generate a preprocessing model; image data are preprocessed through the model, and then through image enhancement, multi-degree processing and feature extraction, feature images are obtained and stored in a classified mode. During user retrieval, the sum of the difference between the feature image and the demand, the sparse regular term and the complex regular term is used as an optimization formula, an alternating direction multiplier method is used for solving, and accurate matching is achieved. According to the method, denoising and correction models are integrated, data integrity and collaboration preprocessing is achieved, feature images are obtained through further processing, and classified storage is carried out. During retrieval, solution is carried out through an optimization formula and an alternating direction multiplier method, and the management and application efficiency of remote sensing satellite image data is improved.

Description

Technical Field

[0001] This application relates to the field of data management, and particularly to a method and system for managing remote sensing satellite image data. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing satellite image data is playing an increasingly important role in many fields such as environmental monitoring, resource investigation, disaster warning, and urban planning. However, remote sensing satellite image data is often affected by various factors during the acquisition and transmission processes, such as sensor noise, atmospheric cloud and fog occlusion, and changes in lighting conditions. These factors will seriously reduce the quality of the images, thereby affecting subsequent data analysis and application effects.

[0003] Traditional methods for processing remote sensing satellite image data mainly include steps such as filtering and denoising, image enhancement, and cloud and fog removal. However, most of these methods are based on manually designed features and algorithms, with limited processing effects and difficulty in adapting to complex and changing image conditions. In recent years, the rise of deep learning technology has provided new ideas for the processing of remote sensing satellite image data. Deep learning networks, especially models such as convolutional neural networks (CNNs) and conditional adversarial networks (GANs), have achieved remarkable results in fields such as image denoising and cloud and fog removal by virtue of their powerful feature extraction and image generation capabilities.

[0004] However, existing deep learning models still have some deficiencies in the processing of remote sensing satellite image data. For example, when using a CNN or GAN model alone for denoising or cloud and fog correction, it is often difficult to achieve both ideal denoising effects and cloud and fog removal effects at the same time. At the same time, in traditional processing workflows, denoising and cloud and fog correction are usually carried out as independent steps, lacking overall optimization and combination. In addition, the processed remote sensing satellite image data also needs to be further subjected to image enhancement, multi-scale processing, and feature extraction for subsequent classification storage and retrieval. There is a lack of close connection and collaborative optimization between these steps, resulting in low overall processing efficiency and difficulty in fully exploring and utilizing the useful information in the image data. Summary of the Invention

[0005] The purpose of this application is to overcome the above-mentioned existing technologies and provide a method and system for managing remote sensing satellite image data.

[0006] This application provides a method for managing remote sensing satellite image data, including:

[0007] Constructing a denoising model and a cloud and fog correction model based on the deep learning networks CNN and GAN;

[0008] Taking the output of the denoising model as the input of the cloud and fog correction model, combining the denoising model and the cloud and fog correction model to generate a preprocessing model;

[0009] Preprocess the remote sensing satellite image data through the preprocessing model to obtain preprocessed data;

[0010] The preprocessed data is successively processed by an image enhancement function, a multi-degree processing function, and a feature extraction function to obtain a feature image;

[0011] Classify and store the feature images;

[0012] When a user retrieves, the difference between the feature image and the user's requirements is used as an optimization formula together with the sum of a sparse regularization term and a complex regularization term, and the optimization formula is solved by the alternating direction method of multipliers to obtain a matching result.

[0013] Optionally, the preprocessed data is successively processed by an image enhancement function, a multi-degree processing function, and a feature extraction function to obtain a feature image.

[0014] Optionally, the feature extraction sub-function ψj includes at least one of statistical features, transform features, texture features, morphological features, edge features, multi-scale features, spectral features, and spatial features.

[0015] Optionally, in the classification and storage of the feature images, the classification steps include:

[0016] Use machine learning or deep learning algorithms for classification.

[0017] Optionally, in the classification and storage of the feature images, the storage steps include:

[0018] Use distributed storage or cloud storage technology.

[0019] This application also provides a remote sensing satellite image data management system, including:

[0020] A construction module for constructing a denoising model and a cloud and fog correction model based on the deep learning networks CNN and GAN;

[0021] A preprocessing module for using the output of the denoising model as the input of the cloud and fog correction model to combine the denoising model and the cloud and fog correction model to generate a preprocessing model; preprocess the remote sensing satellite image data through the preprocessing model to obtain preprocessed data;

[0022] A classification module for classifying the extracted features;

[0023] A storage module for storing the classified data;

[0024] A query module, when a user retrieves, takes the difference between the feature image and the user's needs, and the sum of the sparse regularization term and the complex regularization term as an optimization formula, and solves the optimization formula by the alternating direction multiplier method to obtain a matching result.

[0025] Optionally, the preprocessed data is sequentially processed by an image enhancement function, a multi-degree processing function, and a feature extraction function to obtain a feature image.

[0026] Optionally, the feature extraction sub-function ψj includes at least one of statistical features, transform features, texture features, morphological features, edge features, multi-scale features, spectral features, and spatial features.

[0027] Optionally, when classifying and storing the feature image, the classification steps include:

[0028] Using machine learning or deep learning algorithms for classification.

[0029] Optionally, when classifying and storing the feature image, the storage steps include:

[0030] Using distributed storage or cloud storage technology.

[0031] The beneficial effects of this application are:

[0032] Inventive point 1, the preprocessing model

[0033] Inventive point 2, the retrieval method

[0034] The present application provides a method for managing remote sensing satellite image data, including: constructing a denoising model and a cloud and fog correction model based on the deep learning networks CNN and GAN; combining the denoising model and the cloud and fog correction model by using the output of the denoising model as the input of the cloud and fog correction model to generate a preprocessing model; preprocessing remote sensing satellite image data through the preprocessing model to obtain preprocessed data; successively processing the preprocessed data through an image enhancement function, a multi-degree processing function, and a feature extraction function to obtain a feature image; classifying and storing the feature image; when a user retrieves, taking the difference between the feature image and the user's requirement, and the sum of a sparse regularization term and a complex regularization term as an optimization formula, and solving the optimization formula through the alternating direction multiplier method to obtain a matching result. By constructing a denoising model and a cloud and fog correction model based on the deep learning networks CNN and GAN, and combining these two models to generate a preprocessing model, the overall and collaborative preprocessing of remote sensing satellite image data is realized. The preprocessed data is further processed through an image enhancement function, a multi-degree processing function, and a feature extraction function to obtain a feature image, which is then classified and stored. When a user retrieves, by constructing an optimization formula containing a sparse regularization term and a complex regularization term, and using the alternating direction multiplier method to solve it, an accurate match between the feature image and the user's requirement is realized, thereby improving the management and application efficiency of remote sensing satellite image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 FIG. is a schematic diagram of the management process of remote sensing satellite image data in the present application;

[0036] Figure 2 FIG. is a schematic diagram of the management device of remote sensing satellite image data in the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The following further describes the present application with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it.

[0038] The following content are all examples of the specific implementation process provided to detail the technical solution to be protected by the present application. However, the present application can also be implemented in other ways different from the descriptions herein. Those skilled in the art can implement the present application by using different technical means under the guidance of the concept of the present application. Therefore, the present application is not limited by the following specific embodiments.

[0039] Please refer to Figure 1 as shown, the present application provides a method for managing remote sensing satellite image data, including:

[0040] S101. Construct a denoising model and a cloud and fog correction model based on the deep learning networks CNN and GAN.

[0041] During the acquisition and transmission of satellite images, various noises can interfere, such as sensor noise, transmission noise, etc. Such noises not only reduce the clarity of the images but also obscure important information. Therefore, noise removal is the primary task in satellite image preprocessing.

[0042] A denoising network based on CNN is adopted to address this challenge. CNN has powerful feature learning capabilities and can automatically extract the feature patterns of noises from a large amount of training data. During the training process, the network continuously adjusts its weight parameters to minimize the difference between the denoised image and the original noise-free image. This self-learning feature enables the CNN denoising network to handle various complex types of noises while retaining the detailed information of the images.

[0043] The denoising model (CNN) includes: an input layer that accepts the original remote sensing satellite image data with a size of 256x256 pixels and the number of channels being the number of bands of the satellite image (e.g., 3 for RGB and more for multispectral images). A convolutional layer that uses a multi-layer convolutional structure, with each layer containing a certain number of convolutional kernels, such as 3x3 or 5x5 in size, a stride of 1, and an edge padding strategy to keep the size of the feature map unchanged. Each layer is followed by a ReLU activation function. A pooling layer that, after the convolutional layer, uses a max pooling layer with a size of 2x2 and a stride of 2 to reduce the dimension of the feature map and the computational amount. A transposed convolutional layer (or deconvolutional layer) for upsampling to gradually restore the size of the image. The parameter settings of the transposed convolutional layer correspond to those of the convolutional layer. An output layer that outputs the denoised image data with the same size as the input.

[0044] The noise removal formula is expressed as:

[0045] I denoised (x,y) = CNN denoise (I(x, y))

[0046] where I(x, y) represents the original image data, and I denoised (x, y) represents the denoised image data, and CNN denoise represents the trained denoising convolutional neural network model.

[0047] According to the characteristics of the noise, appropriate parameters such as the number of network layers, the size of the convolutional kernels, and the stride are selected to improve the denoising effect.

[0048] Cloud and fog occlusion is one of the common interference factors in satellite images. It not only reduces the visibility of the images but also obscures the important features of the ground surface.

[0049] This application uses a GAN-based cloud removal network to address this challenge. A GAN consists of two parts: a generator and a discriminator. The generator is responsible for generating the cloud-removed image, while the discriminator is used to evaluate whether the generated image is real. Through training, the generator can learn how to effectively remove clouds from the image while preserving the original structure and details of the image.

[0050] The cloud correction model includes: a generator that accepts the denoised image data as input. Its network structure is similar to the denoising model but contains more convolutional layers and a more complex feature extraction structure to better learn the characteristics of cloud removal. A discriminator that is used to distinguish the generated cloud-removed image from the real cloud-free image. The discriminator usually adopts a convolutional neural network structure, and the last layer is a fully connected layer that outputs a scalar value representing the probability of authenticity.

[0051] The cloud correction formula is expressed as:

[0052] C corr (x,y) = GAN decloud (I denoised (x,y))

[0053] Where, I denoised (x, y) represents the denoised image data, C corr (x, y) represents the corrected image data, and GAN decloud represents the trained cloud removal generative adversarial network model.

[0054] S102. Use the output of the denoising model as the input of the cloud correction model, combine the denoising model and the cloud correction model, and generate a preprocessing model.

[0055] Using the output of the denoising model as the input of the cloud correction model realizes the seamless docking of the two models. Specifically, the denoising model first processes the original image to generate the denoised image; then, this denoised image is fed into the cloud correction model for further cloud removal processing. Finally, the comprehensively preprocessed image data is obtained.

[0056] Combining the noise removal and cloud correction formulas, a comprehensive preprocessing formula is obtained:

[0057] I preprocessd (x, y) = GAN decloud (CNN dencise (I(x,y)))

[0058] Where, I preprocessed(x, y) represents the preprocessed image data, and I(x, y) represents the original satellite image data. This formula clearly shows the entire preprocessing process: First, the original image is denoised by a CNN denoising model, then the denoised image is further processed to remove clouds by a GAN cloud correction model, and finally the preprocessed image is obtained.

[0059] Furthermore, in the satellite image data preprocessing process, adding a verification step before using the output of the denoising model as the input of the cloud correction model is a key link to ensure the preprocessing quality.

[0060] Use algorithms based on image statistical features, such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics, to objectively evaluate the denoised image.

[0061] To implement the verification of intermediate results and ensure that the denoised image meets the quality standards, a loop processing flow is designed.

[0062] Loop condition: The denoised image does not meet the quality standards, and the maximum number of loops or the maximum processing time has not been reached.

[0063] End condition: The denoised image meets the quality standards, or the maximum number of loops / maximum processing time has been reached, and the last denoised image still does not meet the quality standards.

[0064] The denoising model and the cloud correction model are trained simultaneously, and their synergy is enhanced by sharing layers or parameters.

[0065] According to the characteristics of specific tasks and datasets, select some layers in network structures such as shared convolutional layers and fully connected layers, or share some parameters. This is verified through experiments to find the best combination method. For example: Select the first three convolutional layers of the denoising model and the first three convolutional layers of the cloud correction model for sharing.

[0066] Design reasonable loss functions and training strategies to ensure that the two models can be optimized simultaneously. Use methods such as alternating training and joint loss functions to balance the training processes of the two models.

[0067] Design a joint loss function that simultaneously considers the outputs of the denoising model and the cloud correction model. For example, add the loss of the denoising model (such as mean squared error MSE) and the loss of the cloud correction model (such as cross-entropy loss) to obtain a total loss function. In this way, during the training process, both models will be constrained by the loss function and thus optimized synergistically.

[0068] Adopt an alternating training method. In each iteration, first fix the parameters of one model and update the parameters of the other model. Then, fix the parameters of the other model and update the parameters of the current model. In this way, the two models learn from each other during the training process and gradually optimize their own parameters.

[0069] S103. Preprocess the remote sensing satellite image data through the preprocessing model to obtain preprocessed data.

[0070] S104. The preprocessed data is successively processed by an image enhancement function, a multi-scale processing function, and a feature extraction function to obtain a feature image.

[0071] Through image enhancement, the key features in the image become more prominent.

[0072] By filtering and segmenting the image at different scales, features of different sizes and shapes in the image are extracted.

[0073] The feature extraction function ψ is a composite function that combines multiple image processing techniques and algorithms to extract features. ψ takes the multi-scale processed image Imulti-scale as input and outputs a feature set F containing multiple feature vectors.

[0074] To extract complex and useful feature information from satellite images, the feature extraction formula provided in this application is:

[0075] F = {f 1 , f 2 ,..., f n} = Ψ(Multi-Scale Process(Enhanced Image(I'(x, y))))

[0076] Ψ(I multi-scale ) = {ψ 1 (I multi-scale ), ψ 2 (I multi-scale ),..., ψ m (I multi-scale )}

[0077] Among them, Imulti-scale is the image after multi-scale processing and image enhancement.

[0078] Among them, F is the feature set, f iis the i-th eigenvector or eigen-set, Ψ is the composite feature extraction function, Multi-ScaleProcess is the multi-scale processing function, Enhancedlmage is the image enhancement function, I′(x, y) is the preprocessed satellite image, which is the input of the feature extraction process, and Imulti-scale = Multi-ScaleProcess(Enhancedlmage(I′(x, y))) is the image after multi-scale processing and image enhancement. Ψ i is the j-th specific feature extraction sub-function.

[0079] S105. Classify and store the belonging feature images.

[0080] Before classifying and storing the feature images, formulate clear classification criteria. The criteria are based on factors such as the type of features (such as edge features, texture features, shape features, etc.), the application scenarios of features (such as face recognition, object detection, scene classification, etc.), the sources of feature images (such as images taken by different sensors, at different time periods, etc.).

[0081] According to the formulated classification criteria, store the feature images in different folders, database tables or data structures. For example, store the edge feature images in a folder named "EdgeFeatures" and the texture feature images in another folder named "TextureFeatures".

[0082] S106. When the user retrieves, take the difference between the feature image and the user's needs, and the sum of the sparse regularization term and the complex regularization term as the optimization formula, and solve the optimization formula by the alternating direction method of multipliers to obtain the matching result.

[0083] Define a non-linear optimization algorithm formula:

[0084]

[0085] subject to g i (x,y) ≤ 0, i = 1, 2,..., p

[0086] h j (x,y) = 0, j = 1, 2,..., q

[0087] x ∈ X, y ∈ Y

[0088] This formula contains multiple variables (x and y), constraints, and regularization terms.

[0089] x ∈ R n and y ∈ R mis the vector to be solved, representing different features respectively. These features come from the feature extraction step and are used to describe the relationship between user requirements and the feature image. C(x, y) is a non-linear function that maps x and y to the observation value space and is compared with the observation value vector d. Here, d represents the quantitative representation of user requirements, while C(x, y) represents the mapping relationship between the feature image and user requirements. is the data fitting term, used to measure the difference between the model prediction value (i.e., C(x, y)) and the observation value (i.e., d).

[0090] μ||D(x)-e|| 1 is the sparse regularization term, where D(x) is another non-linear function and e is the corresponding observation value or target value of D(x).

[0091] Encourages the algorithm to select fewer features or parameters during the solution process, thereby improving the generalization ability and interpretability of the model. λΩ(x, y) is the complex regularization term, used to control the complexity and stability of the solution, including various regularization techniques such as L2 norm (for smoothing the solution) and total variation (for preserving edge information).

[0092] By introducing the complex regularization term, it avoids overfitting of the algorithm or generating unstable solutions, thereby improving the robustness and accuracy of the model.

[0093] g i (x,y)≤0 and h j (x,y)=0 are the constraint conditions, used to limit the solution space.

[0094] By introducing the constraint conditions, it ensures that the algorithm meets specific requirements or conditions during the solution process, thereby obtaining a solution that better meets the actual application requirements.

[0095] X and Y are the feasible regions of x and y, containing various boundary conditions and restrictions. The boundary conditions and restrictions come from the physical properties of the data, the parameter settings of the algorithm, or the actual application requirements.

[0096] To solve the above non-linear optimization algorithm formula, the Alternating Direction Method of Multipliers (ADMM) is adopted. ADMM is an effective iterative algorithm that approximates the optimal solution by decomposing the original problem into multiple sub-problems and solving them alternately.

[0097] Specifically, initialize the initial values of variables x and y and the initial values of the multipliers (used to handle the constraint conditions). The initial values are determined by random selection, empirical setting, or heuristic algorithms.

[0098] In each iteration, update the values of variables x and y and the multiplier according to the iteration rules of ADMM. Specifically, first fix the value of one variable (such as x), and then solve for the optimal value of the other variable (such as y); then fix the value of y and solve for the optimal value of x. This process is repeated multiple times until the stopping condition is met (such as reaching the maximum number of iterations, converging to a certain threshold, etc.).

[0099] After each iteration, check whether the updated variable values satisfy the constraint conditions. If not, use the multiplier to adjust the variable values to ensure that they satisfy the constraint conditions.

[0100] When the iteration process stops, obtain the values of the optimal solution (or approximate optimal solution) x and y. These values are used to represent the feature image that best matches the user's requirements.

[0101] As Figure 2 shown, the present application also provides a remote sensing satellite image data management system, including:

[0102] A construction module for constructing a denoising model and a cloud and fog correction model based on the deep learning networks CNN and GAN;

[0103] A preprocessing module for combining the output of the denoising model as the input of the cloud and fog correction model to combine the denoising model and the cloud and fog correction model to generate a preprocessing model; preprocessing the remote sensing satellite image data through the preprocessing model to obtain preprocessing data;

[0104] A classification module for classifying the extracted features;

[0105] A storage module for storing the classified data;

[0106] A query module for, when the user retrieves, taking the difference between the feature image and the user's requirements, and the sum of the sparse regularization term and the complex regularization term as an optimization formula, and solving the optimization formula by the alternating direction multiplier method to obtain a matching result.

[0107] Optionally, the preprocessing data is successively processed by an image enhancement function, a multi-scale processing function, and a feature extraction function to obtain a feature image, which is expressed by the formula:

[0108] F = {f 1 , f 2 ,..., f n} = Ψ(Multi-Scale Process(Enhanced Image(I′(x, y))))

[0109] Ψ(I multi-scale ) = {ψ 1 (I multi-scale ), ψ2 (I multi-scale ),..., ψ m (I multi-scale )}

[0110] Among them, lmulti-scale is the image after multi-scale processing and image enhancement, F is the feature set, fi is the i-th feature vector or feature set, Ψ is the composite feature extraction function, Multi-ScaleProcess is the multi-scale processing function, EnhancedImage is the image enhancement function, I′(x, y) is the preprocessed satellite image, which is the input of the feature extraction process, and lmulti-scale = Multi-ScaleProcess(Enhancedlmage(l′(x, y))) is the image after multi-scale processing and image enhancement. Ψ i is the j-th specific feature extraction sub-function.

[0111] Optionally, the feature extraction sub-function Ψj includes at least one of statistical features, transform features, texture features, morphological features, edge features, multi-scale features, spectral features, and spatial features.

[0112] Optionally, in the classification storage of the feature images, the classification steps include:

[0113] Using machine learning or deep learning algorithms for classification.

[0114] Optionally, in the classification storage of the feature images, the storage steps include:

[0115] Using distributed storage or cloud storage technology.

Claims

1. A remote sensing satellite image data management method, characterized in that: include: Build denoising model and cloud correction model based on deep learning network CNN and GAN; The output of the denoising model is used as the input of the cloud correction model to combine the denoising model and the cloud correction model to generate a preprocessing model; Preprocessing the remote sensing satellite image data by using the preprocessing model to obtain preprocessed data; The pre-processed data is processed by an image enhancement function, a multi-degree processing function and a feature extraction function in sequence to obtain a feature image; Classify and store the characteristic images; When a user searches, the difference between the feature image and the user's needs and the sum of the sparse regularization term and the complex regularization term are used as an optimization formula, and the optimization formula is solved by the alternating direction multiplier method to obtain a matching result.

2. A remote sensing satellite image data management method according to claim 1, characterized in that: The pre-processed data is processed in sequence by an image enhancement function, a multi-degree processing function and a feature extraction function to obtain a feature image.

3. A remote sensing satellite image data management method according to claim 2, characterized in that: The feature extraction subfunction ψ j It includes at least one of statistical features, transformation features, texture features, morphological features, edge features, multi-scale features, spectral features and spatial features.

4. A remote sensing satellite image data management method according to claim 1, characterized in that: In classifying and storing the characteristic images, the classification step includes: Use machine learning or deep learning algorithms for classification.

5. The remote sensing satellite image data management method according to claim 1, characterized in that: In classifying and storing the characteristic images, the steps of storing include: Use distributed storage or cloud storage technology.

6. A remote sensing satellite image data management system, characterized in that: include: Building modules for building denoising and cloud correction models based on deep learning networks CNN and GAN; A preprocessing module, used for combining the denoising model and the cloud correction model by taking the output of the denoising model as the input of the cloud correction model to generate a preprocessing model; Preprocessing the remote sensing satellite image data by using the preprocessing model to obtain preprocessed data; Classification module, used to classify the extracted features; A storage module, used for storing classified data; The query module is used for user retrieval, and the difference between the feature image and the user demand and the sum of the sparse regularization term and the complex regularization term are used as the optimization formula, and the optimization formula is solved by the alternating direction multiplier method to obtain the matching result.

7. A remote sensing satellite image data management system according to claim 6, characterized in that: The pre-processed data is processed in sequence by an image enhancement function, a multi-degree processing function and a feature extraction function to obtain a feature image.

8. A remote sensing satellite image data management system according to claim 7, characterized in that: The feature extraction subfunction ψ j It includes at least one of statistical features, transformation features, texture features, morphological features, edge features, multi-scale features, spectral features and spatial features.

9. A remote sensing satellite image data management system according to claim 6, characterized in that: In classifying and storing the characteristic images, the classification step includes: Use machine learning or deep learning algorithms for classification.

10. A remote sensing satellite image data management system according to claim 6, characterized in that: In classifying and storing the characteristic images, the steps of storing include: Use distributed storage or cloud storage technology.

Images