Remote sensing image interpretation method based on deep learning

Abstract

The invention provides a remote sensing image interpretation method based on deep learning. The method comprises the following steps: S1, preprocessing a remote sensing image; s2, constructing a deep learning model, and training the deep learning model by using the remote sensing image data sample; s3, interpreting the preprocessed remote sensing image by using the trained deep learning model and outputting a result; and S4, evaluating the output result. The remote sensing image intelligent interpretation method can effectively solve the problems of complex space-time heterogeneity of the remote sensing image and scarcity and imbalance of interpretation samples, thereby promoting application of the remote sensing image intelligent interpretation method in large-range and complex scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image interpretation, and in particular to a remote sensing image interpretation method based on deep learning. Background Art

[0002] Since remote sensing images have complex temporal and spatial heterogeneity, that is, the image features of the same object at different times, locations, and sensors may vary greatly, resulting in low temporal and spatial transferability of interpretation samples, that is, it is difficult to use samples from a certain region or period for image interpretation in other regions or periods. This has resulted in the scarcity and imbalance of remote sensing interpretation samples, limiting the application of remote sensing image intelligent interpretation methods in large-scale and complex scenarios. Therefore, a remote sensing image interpretation method based on deep learning is proposed to solve the above problems. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a remote sensing image interpretation method based on deep learning to at least solve the above problems.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A remote sensing image interpretation method based on deep learning, the method comprising the following steps:

[0006] S1. Preprocessing remote sensing images;

[0007] S2. Building a deep learning model and training it using remote sensing image data samples;

[0008] S3, using the trained deep learning model to interpret the pre-processed remote sensing images and output the results;

[0009] S4. Evaluate the output results.

[0010] Furthermore, in step S1, the remote sensing image is preprocessed by performing radiation correction on the remote sensing image data, performing geometric correction on the radiation corrected image, performing atmospheric correction on the geometric corrected image, performing image enhancement on the atmospheric corrected image, and performing standardization on the atmospheric corrected image.

[0011] Furthermore, in step S1, the remote sensing image data includes source domain image data and target domain image data.

[0012] Furthermore, in step S2, a deep learning model is constructed and trained using remote sensing image data samples. Specifically:

[0013] When the deep learning model is trained on remote sensing image data samples, the features of the source domain image data and the target domain image data are extracted, and based on the migration strategy in the deep learning model, higher weights are given to source domain samples with high similarity to the target domain.

[0014] Furthermore, in step S2, the features of the source domain image data and the target domain image data are extracted as follows:

[0015] Feature extraction is performed in the convolutional neural network, and optimization is performed through the convolution layer. The optimization calculation formula is:

[0016]

[0017] in is the value of the l-th layer convolution output feature map at position (i, j), is the weight of the convolution kernel of the lth layer, is the value of the feature map of the previous layer at the corresponding position, b (l) is the bias term;

[0018] The maximum pooling is performed through the pooling layer. The maximum pooling calculation formula is:

[0019]

[0020] in is the value of the l-th layer pooling output feature map at position (i, j), R ij is the area covered by the pooling window, It is the value of the previous feature map at position (m,n) in the window.

[0021] Furthermore, step 2.5 is also included:

[0022] S2.5. Input the extracted features of the source domain image data and the target domain image data into the deep learning model for training, and define the cross entropy loss function according to the task objectives. The cross entropy loss function formula is as follows:

[0023]

[0024] Among them, L is the loss value, N is the number of samples, C is the number of categories, and y ic is the true label, is the predicted probability;

[0025] When extracting input features for training through a deep learning model, the SGD optimization algorithm is selected to update the model parameters. The SGD optimization algorithm is as follows:

[0026] theta (t+1) =θ (t) -α·▽θ L(θ (t) )

[0027] Among them, θ (t) is the model parameter of the t-th iteration, α is the learning rate, ▽ θ L(θ (t) ) is the gradient of the loss function with respect to the model parameters.

[0028] Furthermore, in step S3, the following sub-steps are also included:

[0029] S31, for the target domain remote sensing image data, the same processing operation as step S1 is performed;

[0030] S32, extract features through steps S2 and S2.5, and match them with the deep learning model trained in step S3;

[0031] S33, according to the matching results, selecting a total interpretation model and several sub-interpretation models for combination;

[0032] S34, using the combined model to adaptively interpret the new image data and generate a classification result;

[0033] S35. Process the classification results, including color labeling, shape labeling and boundary delineation, and output the interpretation results in the form of images, tables or vector layers.

[0034] Furthermore, the deep learning model also includes an attention mechanism module, which enhances the focus on important features during the feature extraction process. The important features include color, texture, and shape.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention proposes a remote sensing image interpretation method based on deep learning. This method can improve the image quality through preprocessing by combining the advantages of deep learning and transfer learning, use the deep learning model for feature extraction and interpretation, and improve the performance of the model on the target domain data through transfer learning, thereby improving the accuracy of interpretation and the generalization ability on the target domain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1It is a schematic diagram of the overall process of a remote sensing image interpretation method based on deep learning proposed in an embodiment of the present invention.

[0039] Figure 2 It is a flowchart of step S3 of a remote sensing image interpretation method based on deep learning proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0041] Reference Figure 1 The present invention provides a remote sensing image interpretation method based on deep learning, the method comprising the following steps:

[0042] S1. Preprocessing remote sensing images;

[0043] S2. Building a deep learning model and training it using remote sensing image data samples;

[0044] S3, using the trained deep learning model to interpret the pre-processed remote sensing images and output the results;

[0045] S4. Evaluate the output results.

[0046] In step S1, the remote sensing image is preprocessed by performing radiation correction on the remote sensing image data, performing geometric correction on the radiation corrected image, performing atmospheric correction on the geometric corrected image, performing image enhancement on the atmospheric corrected image, and performing standardization on the atmospheric corrected image.

[0047] Exemplarily, in order to effectively solve the complex spatiotemporal heterogeneity of remote sensing images and the scarcity and imbalance of interpretation samples, and provide high-quality input data for subsequent feature extraction and model training, remote sensing images can be corrected by radiation correction to correct radiation errors caused by sensor performance, atmospheric conditions and other factors. For geometric correction, the geometric distortion of remote sensing images can be eliminated to make them conform to the geographic coordinate system. For atmospheric correction, the authenticity and accuracy of images can be improved by eliminating the influence of the atmosphere on the propagation of electromagnetic waves. For image enhancement, the contrast and clarity of images can be improved to facilitate subsequent feature extraction and analysis. For standardization, the generalization ability of the model can be improved by eliminating the differences in illumination and scale between different images. The feature extraction of standardized image data is performed by constructing a deep learning model. The deep learning model is constructed and trained with remote sensing image data samples. The preprocessed remote sensing images are interpreted and the results are output by using the trained deep learning model. The output results are evaluated to achieve the interpretation of remote sensing images.

[0048] In step S1, the remote sensing image data includes source domain image data and target domain image data.

[0049] For example, in response to the complex spatiotemporal heterogeneity of remote sensing images and the scarcity and imbalance of interpretation samples, remote sensing image data can be divided into source domain image data and target domain image data to facilitate subsequent feature extraction and feature alignment, thereby reducing the distribution offset between the source domain and the target domain and improving the generalization ability of the model in the target domain.

[0050] In step S2, a deep learning model is constructed and trained using remote sensing image data samples as follows:

[0051] When the deep learning model is trained on remote sensing image data samples, the features of the source domain image data and the target domain image data are extracted, and based on the migration strategy in the deep learning model, higher weights are given to source domain samples with high similarity to the target domain.

[0052] For example, by aligning the features of source domain image data and target domain image data, and using the source domain data to train the basic model so that it can learn effective feature representation, it is possible to capture key information such as texture, shape, and color. Domain adaptation technology is then applied to reduce the differences in feature space between the source domain and the target domain to ensure that the features extracted from the two domains are closer in representation. A transfer learning strategy is then used to give higher weights to source domain samples that have a high similarity to the target domain, so as to emphasize samples that are more useful to the target domain.

[0053] In step S2, the features of the source domain image data and the target domain image data are extracted as follows:

[0054] Feature extraction is performed in the convolutional neural network, and optimization is performed through the convolution layer. The optimization calculation formula is:

[0055]

[0056] in is the value of the l-th layer convolution output feature map at position (i, j), is the weight of the convolution kernel of the lth layer, is the value of the feature map of the previous layer at the corresponding position, b (l) is the bias term;

[0057] The maximum pooling is performed through the pooling layer. The maximum pooling calculation formula is:

[0058]

[0059] in is the value of the l-th layer pooling output feature map at position (i, j), R ij is the area covered by the pooling window, It is the value of the previous feature map at position (m,n) in the window.

[0060] For example, in a convolutional neural network, a convolution kernel is used to slide on an image to extract local features. The weight of the convolution kernel is continuously optimized through a training process to learn a more effective feature representation. The kernel slides on the input feature map, and for each position, the maximum value of all values ​​in the window is calculated as the output value of that position. This process reduces the size of the feature map and helps the model learn a more robust feature representation.

[0061] Also includes step 2.5:

[0062] S2.5. Input the extracted features of the source domain image data and the target domain image data into the deep learning model for training, and define the cross entropy loss function according to the task objectives. The cross entropy loss function formula is as follows:

[0063]

[0064] Among them, L is the loss value, N is the number of samples, C is the number of categories, and y ic is the true label, is the predicted probability;

[0065] When extracting input features for training through a deep learning model, the SGD optimization algorithm is selected to update the model parameters. The SGD optimization algorithm is as follows:

[0066] theta (t+1) =θ (t) -α·▽ θ L(θ (t) )

[0067] Among them, θ (t) is the model parameter of the t-th iteration, α is the learning rate, ▽ θ L(θ (t) ) is the gradient of the loss function with respect to the model parameters.

[0068] Exemplarily, the cross entropy loss function is selected as the loss function according to the task objective to measure the difference between the model prediction result and the true label. Then, the SGD (stochastic gradient descent) optimization algorithm is used to update the model parameters to minimize the loss value and improve the interpretation performance of the model.

[0069] In step S3, the following sub-steps are also included:

[0070] S31, for the target domain remote sensing image data, the same processing operation as step S1 is performed;

[0071] S32, extract features through steps S2 and S2.5, and match them with the deep learning model trained in step S3;

[0072] S33, according to the matching results, selecting a total interpretation model and several sub-interpretation models for combination;

[0073] S34, using the combined model to adaptively interpret the new image data and generate a classification result;

[0074] S35. Process the classification results, including color labeling, shape labeling and boundary delineation, and output the interpretation results in the form of images, tables or vector layers.

[0075] Exemplarily, the new target domain remote sensing image data needs to go through the same preprocessing process as in step S1, namely, including radiation correction, geometric correction, atmospheric correction and image enhancement, and final standardization processing. The purpose of the above processing steps is to ensure the data quality of the new image, remove interference factors, and make it consistent with the data used in training the model, thereby improving the accuracy and reliability of the interpretation. In the feature extraction stage, the preprocessed target domain image is extracted by convolutional neural network and the like using the same method as steps S2 and S2.5, and then input into the deep learning model trained in step S3. Based on the mapping relationship from features to interpretation results in the deep learning model, the newly extracted features can be matched, and preliminary interpretation results can be generated based on these features. According to the matching result obtained in step S32, that is, the preliminary interpretation result, one or more most suitable sub-interpretation models can be selected for combination. The combination method can be a simple weighted average or a more complex set. In addition to the learning method, there may also be an overall interpretation model, which makes a comprehensive judgment based on the output of all sub-models to generate the final interpretation result. After determining the model combination to be used, the model can be used to adaptively interpret the new remote sensing image data. Adaptive interpretation means that the model can automatically adjust its interpretation strategy according to the specific features of the new image to generate accurate classification results. The classification results may include different surface cover types such as forests, water bodies, cities, building distribution, road networks, etc. The generated classification results are processed and visualized, and different classification results are marked with different colors to more intuitively display the different features in the image; specific shapes are marked, such as the outline of buildings; and the boundaries of the classification results are depicted for spatial analysis and map making. The processed results can be directly displayed in the form of images, or converted into tables or vector layers for further data analysis or geographic information system (GIS) integration.

[0076] The deep learning model also includes an attention mechanism module, which enhances the focus on important features during feature extraction. Important features include color, texture, and shape.

[0077] For example, the introduction of an attention mechanism module in the deep learning model can enhance the focus on important features such as color, texture, shape, etc. during the feature extraction process, thereby improving the model's sensitivity to key information and further improving the accuracy and efficiency of interpretation.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A remote sensing image interpretation method based on deep learning, characterized in that: The method comprises the following steps: S1. Preprocessing remote sensing images; S2. Building a deep learning model and training it using remote sensing image data samples; S3, using the trained deep learning model to interpret the pre-processed remote sensing images and output the results; S4. Evaluate the output results.

2. The remote sensing image interpretation method based on deep learning according to claim 1, characterized in that: In step S1, the remote sensing image is preprocessed by performing radiation correction on the remote sensing image data, performing geometric correction on the radiation corrected image, performing atmospheric correction on the geometric corrected image, performing image enhancement on the atmospheric corrected image, and performing standardization on the atmospheric corrected image.

3. The remote sensing image interpretation method based on deep learning according to claim 2, characterized in that: In step S1, the remote sensing image data includes source domain image data and target domain image data.

4. The remote sensing image interpretation method based on deep learning according to claim 3, characterized in that: In step S2, a deep learning model is constructed and trained using remote sensing image data samples as follows: When the deep learning model is trained on remote sensing image data samples, the features of the source domain image data and the target domain image data are extracted, and based on the migration strategy in the deep learning model, higher weights are given to source domain samples with high similarity to the target domain.

5. The remote sensing image interpretation method based on deep learning according to claim 4, characterized in that: In step S2, the features of the source domain image data and the target domain image data are extracted as follows: Feature extraction is performed in the convolutional neural network, and optimization is performed through the convolution layer. The optimization calculation formula is: in is the value of the l-th layer convolution output feature map at position (i, j), is the weight of the convolution kernel of the lth layer, is the value of the feature map of the previous layer at the corresponding position, b (l) is the bias term; The maximum pooling is performed through the pooling layer. The maximum pooling calculation formula is: in is the value of the l-th layer pooling output feature map at position (i, j), R ij is the area covered by the pooling window, It is the value of the previous feature map at position (m,n) in the window.

6. The remote sensing image interpretation method based on deep learning according to claim 5, characterized in that: Also includes step 2.5: S2.

5. Input the extracted features of the source domain image data and the target domain image data into the deep learning model for training, and define the cross entropy loss function according to the task objectives. The cross entropy loss function formula is as follows: Among them, L is the loss value, N is the number of samples, C is the number of categories, and y ic is the true label, is the predicted probability; When extracting input features for training through a deep learning model, the SGD optimization algorithm is selected to update the model parameters. The SGD optimization algorithm is as follows: Among them, θ (t) is the model parameter of the t-th iteration, α is the learning rate, ▽ θ L(θ (t) ) is the gradient of the loss function with respect to the model parameters.

7. The remote sensing image interpretation method based on deep learning according to claim 6, characterized in that: In step S3, the following sub-steps are also included: S31, for the target domain remote sensing image data, the same processing operation as step S1 is performed; S32, extract features through steps S2 and S2.5, and match them with the deep learning model trained in step S3; S33, according to the matching results, selecting a total interpretation model and several sub-interpretation models for combination; S34, using the combined model to adaptively interpret the new image data and generate a classification result; S35. Process the classification results, including color labeling, shape labeling and boundary delineation, and output the interpretation results in the form of images, tables or vector layers.

8. The remote sensing image interpretation method based on deep learning according to claim 1, characterized in that: The deep learning model also includes an attention mechanism module, which enhances the focus on important features during feature extraction. Important features include color, texture, and shape.

Images