Remote sensing image object classification method combining multi-scale information and encoding and decoding network
By introducing multi-scale downsampling features into the U-net network, the problems of classification accuracy and edge accuracy of high-resolution remote sensing images are solved, and higher classification accuracy and speed are achieved, suitable for multi-source remote sensing data.
Patent Information
- Application Number
- CN202010834935.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-19
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2040-08-19
AI Technical Summary
The accuracy of geographic classification and edge accuracy of high-resolution remote sensing images are limited by traditional methods, especially due to the problem of ‘mutual spectrum, heterospectral and heterospectral’ and the problem of loss of shallow features in deep learning methods.
Using a method combining multi-scale information and codec network, the low-level edge features of the image are enhanced by adding multi-scale downsampling features of the original image in the U-net network, and the cross entropy loss function and stochastic gradient descent optimization model are used to improve classification. Accuracy and edge accuracy.
It significantly improves the accuracy and speed of remote sensing images, is suitable for multi-source data, is not restricted by seasons or regions, and is more accurate in segmentation of edges.
Smart Images
Figure CN112001293B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of remote sensing images, and in particular relates to a remote sensing image ground object classification method combining multi-scale information and a coding and decoding network. Background Art
[0002] Remote sensing images are high-information images that contain the temporal and spatial characteristics of various ground objects. By analyzing the spectral, texture and shape characteristics of the objects in the images, the types of objects can be quickly distinguished, providing applications in many fields such as environmental ecology, land resources, and urban planning. In recent years, with the development of remote sensing technology, the resolution of remote sensing images has become higher and higher, the acquisition cycle has become shorter and shorter, and the application scenarios of remote sensing technology have become more and more extensive.
[0003] Compared with low- and medium-resolution remote sensing images, high-resolution remote sensing images contain more detailed information, but are also affected by many factors such as sensor location, atmospheric conditions, topography and climate. The problem of "same object, different spectra, different objects, same spectra" is more serious. Traditional remote sensing image land object classification methods are difficult to achieve high land object classification accuracy.
[0004] Automatically processing remote sensing images with the help of deep learning methods and autonomously learning features through deep convolutional neural networks can help to fully tap the large amount of useful information in remote sensing images, enhance the robustness of classifiers, and effectively improve the accuracy and speed of high-resolution remote sensing image classification.
[0005] Traditional remote sensing image classification methods are mainly divided into pixel-oriented methods and object-oriented methods. The smallest unit of the pixel-oriented method is a single pixel, which only uses the spectral information characteristics, and the utilization rate of rich spatial information is almost zero. Although the traditional object-oriented method is based on spectral information, geometric information and structural information for analysis, the processing speed is slow and the accuracy needs to be improved.
[0006] Most of the existing deep learning-based semantic segmentation methods use the classic encoding and decoding structure to generate a segmentation map with the same size as the original image. Although they extract high-level features of the image and are more robust than traditional methods, they often lose too many shallow features, resulting in the edges of the final segmentation map being not precise enough. Summary of the invention
[0007] The purpose of the present invention is to provide a remote sensing image ground object classification method combining multi-scale information and a coding and decoding network, which is conducive to improving the accuracy of remote sensing image ground object classification.
[0008] To achieve the above object, the technical solution adopted by the present invention is: a remote sensing image object classification method combining multi-scale information and encoding and decoding network, comprising the following steps:
[0009] 1) Label the object categories of the collected remote sensing image datasets;
[0010] 2) The U-net network is used as the main structure. In the encoding process, multi-scale information based on the original image is added to strengthen the edge features of the low-level image. The improved network structure is obtained. In addition to convolution and pooling the input image to obtain the feature map of each stage, different scales of downsampling are directly performed on the original image, and then it is spliced with the convolution feature map of the corresponding stage and sent to the next stage of feature extraction. The decoding process keeps the original structure unchanged, and the feature map is deconvoluted to gradually restore the feature map to the size of the original image.
[0011] 3) In the feature map of the last layer of the decoding process, the softmax function is used to activate each pixel, and then the cross entropy loss function is calculated and optimized using the stochastic gradient descent method to obtain the trained optimal model;
[0012] 4) When predicting the image to be classified, the image is first preprocessed;
[0013] 5) Divide the pre-processed image into blocks of fixed size, record the starting point coordinates of each block of the image, and store them in sequence;
[0014] 6) The divided image blocks to be classified are input into the trained optimal model in batches for prediction, and the optimal model outputs the segmentation map of each image block;
[0015] 7) The segmentation atlas is stitched in order according to the coordinate information recorded before segmentation, and finally a segmentation map of the entire image is formed.
[0016] Furthermore, in step 1, the method for labeling the object categories of the remote sensing image is: using image processing software to outline the edges of the objects in the remote sensing image, filling in color labels corresponding to different object categories, and then converting the color labels into corresponding digital labels.
[0017] Furthermore, the land feature categories include six categories: buildings, farmlands, water bodies, forests, grasslands and undefined; in the digital labels, 0 represents the undefined category, and 1 to 5 represent buildings, farmlands, water bodies, forests, and grasslands respectively.
[0018] Furthermore, in step 2, downsampling of different scales is performed on the original image, and the downsampling is respectively performed to 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the size of the original image.
[0019] Furthermore, in step 3, the calculation formula of the cross entropy loss function is as follows:
[0020]
[0021] in,l ( x ) represents pixels x The corresponding feature category, ω ( l ( x )) represents pixels x The weights corresponding to the feature categories are mainly used to solve the problem of uneven distribution of samples of each feature category. p l(x) ( x ) represents pixels x In category l ( x ) on the activation value;
[0022] The model with the smallest loss E on the validation set within the number of iterations is stored as the optimal model F.
[0023] Furthermore, in step 4, the method for preprocessing the image is: using the Radiometric Correction algorithm of the ENVI platform to perform radiation calibration, and automatically completing the calibration after setting the radiation calibration parameters; and then using the FLAASH Atmospheric Correction algorithm in the platform to perform atmospheric correction.
[0024] Compared with the prior art, the present invention has the following beneficial effects: the present invention adopts a deep learning codec network model for object classification, which is more robust than traditional pixel-oriented and object-oriented methods, is applicable to multi-source remote sensing image data sources, is not limited to specific seasons or specific regions, and has significant improvements in classification accuracy and speed. At the same time, the classic U-net codec structure is improved. In addition to the original convolution feature extraction during the encoding process, multiple scale downsampling is performed on the original image. On the one hand, the model is better adapted to different scales of the same object, and on the other hand, low-level features such as edges are strengthened, so that the object classification results of the model are more accurate and the segmentation edges are more precise. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a flow chart of the method implementation of the embodiment of the present invention.
[0026] Figure 2 It is a schematic diagram of the improved network structure in an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0028] The present invention provides a remote sensing image object classification method that combines multi-scale information and a codec network. The method increases the feature map of the original image by directly downsampling it, strengthens the edge features of the lower layer of the image, and improves the accuracy of remote sensing image object classification. Figure 1 As shown, the method specifically comprises the following steps:
[0029] 1) Divide the collected GF-2 remote sensing image dataset into training set and test set, and label the object categories.
[0030] In this embodiment, the object categories include buildings, farmland, water bodies, forests, grasslands and undefined. The lasso tool of Photoshop software is used to outline the edges of the objects in the remote sensing image, fill in the color labels corresponding to the different object categories, and then convert the color labels into corresponding digital labels. In the digital labels, 0 represents the undefined category, and buildings, farmland, water bodies, forests, and grasslands are 1 to 5 respectively.
[0031] 2) Using the U-net network as the main structure, adding multi-scale information based on the original image during the encoding process, strengthening the edge features of the low-level image, and obtaining the improved encoding and decoding network structure as follows Figure 2 As shown in the figure (the downward arrow is the encoding process, and the upward arrow is the decoding process). In the encoding process, in addition to convolving and pooling the input image downward to obtain the feature map of each stage, different scales of downsampling are performed directly on the original image, and the downsampling is respectively downsampled to 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image, and then spliced with the convolution feature map of the corresponding stage and sent to the next stage of feature extraction. The decoding process keeps the original structure unchanged, and deconvolves the feature map to gradually restore the feature map to the original size.
[0032] 3) In the feature map of the last layer of the decoding process, the softmax function is used to activate each pixel, and then the cross entropy loss function is calculated and optimized using the stochastic gradient descent method to obtain the trained optimal model.
[0033] Among them, the calculation formula of the cross entropy loss function is as follows:
[0034]
[0035] in, l ( x ) represents pixels x The corresponding feature category, ω ( l ( x )) represents pixels x The weights corresponding to the feature categories are mainly used to solve the problem of uneven distribution of samples of each feature category. p l(x) (x ) represents pixels x In category l ( x ) on the activation value;
[0036] The model with the smallest loss E on the validation set within the number of iterations is stored as the optimal model F.
[0037] 4) When predicting the image to be classified, the image is first preprocessed. Specifically, the Radiometric Correction algorithm of the ENVI platform is used for radiometric calibration, and the calibration is automatically completed after the radiometric calibration parameters are set; then the FLAASH Atmospheric Correction algorithm in the platform is used for atmospheric correction.
[0038] 5) Divide the preprocessed image into blocks of fixed size, record the starting point coordinates of each block, and store them in sequence.
[0039] 6) The divided image blocks to be classified are input into the trained optimal model in batches for prediction, and the optimal model outputs the segmentation map of each image block.
[0040] 7) The segmentation atlas is stitched in order according to the coordinate information recorded before segmentation, and finally a segmentation map of the entire image is formed.
[0041] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.
Claims
1. A remote sensing image object classification method combining multi-scale information and encoding and decoding network, characterized in that: The following steps are involved: 1) Label the object categories of the collected remote sensing image datasets; 2) The U-net network is used as the main structure. In the encoding process, multi-scale information based on the original image is added to strengthen the edge features of the low-level image. The improved network structure is obtained. In addition to convolution and pooling the input image to obtain the feature map of each stage, different scales of downsampling are directly performed on the original image, and then it is spliced with the convolution feature map of the corresponding stage and sent to the next stage of feature extraction. The decoding process keeps the original structure unchanged, and the feature map is deconvoluted to gradually restore the feature map to the size of the original image. 3) In the feature map of the last layer of the decoding process, the softmax function is used to activate each pixel, and then the cross entropy loss function is calculated and optimized using the stochastic gradient descent method to obtain the trained optimal model; 4) When predicting the image to be classified, the image is first preprocessed; 5) Divide the pre-processed image into blocks of fixed size, record the starting point coordinates of each block of the image, and store them in sequence; 6) The divided image blocks to be classified are input into the trained optimal model in batches for prediction, and the optimal model outputs the segmentation map of each image block; 7) The segmentation atlas is stitched in order according to the coordinate information recorded before segmentation, and finally a segmentation map of the entire image is formed; In the step 1), the method for labeling the ground object categories of the remote sensing image is: using image processing software to outline the ground object edges of the remote sensing image, filling in color labels corresponding to different ground object categories, and then converting the color labels into corresponding digital labels; The ground feature categories include buildings, farmland, water bodies, forests, grasslands and undefined categories; in the digital labels, 0 represents undefined categories, and 1 to 5 represent buildings, farmland, water bodies, forests, and grasslands respectively; In the step 2), downsampling of different scales is performed on the original image, and the downsampling is respectively 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image; In step 3), the calculation formula of the cross entropy loss function is as follows: in, l ( x ) represents pixels x The corresponding feature category, ω ( l ( x )) represents pixels x The weights corresponding to the feature categories are used to solve the problem of uneven distribution of samples of each feature category. p l(x) ( x ) represents pixels x In category l ( x ) on the activation value; The model with the smallest loss E on the validation set within the number of iterations is stored as the optimal model F.
2. The remote sensing image object classification method combining multi-scale information and encoding and decoding network according to claim 1 is characterized in that: In step 4), the method for preprocessing the image is: using the RadiometricCorrection algorithm of the ENVI platform to perform radiometric calibration, and automatically completing the calibration after setting the radiometric calibration parameters; and then using the FLAASH Atmospheric Correction algorithm in the platform to perform atmospheric correction.