Multi-source remote sensing land cover classification method based on improved U-Net network

Abstract

The invention discloses a multi-source remote sensing land cover classification method based on an improved U-Net network, and the method comprises the steps: S1, fusing the spectral information of a plurality of remote sensing data sources, and obtaining a multi-spectral fused multi-source remote sensing image; s2, preprocessing the multi-source remote sensing image obtained in the S1, and processing in ArcGIS by taking remote sensing image data of the acquisition area as a training vector to obtain a sample image and a sample label; s3, cutting the sample image training set obtained in the step S2; s4, carrying out improvement based on a U-Net network model, adding a convolutional layer and a Dropout layer, introducing an attention mechanism, and carrying out semantic segmentation on the multi-source remote sensing image by utilizing the improved U-Net network and the improved U-Net network and realizing category judgment; and S5, performing land cover classification on the multi-source remote sensing image of the remote sensing image acquisition area. The invention provides a method for realizing multi-source remote sensing image land cover classification through an improved U-Net network model aiming at the problems that remote sensing images are complex in background, more in interference and difficult to extract useful information features.

Description

Technical Field

[0001] The invention relates to the field of geospatial information technology, and in particular to a multi-source remote sensing land cover classification method based on an improved U-Net network. Background Art

[0002] Remote sensing images have been widely used in land cover classification in recent years, and land cover classification plays a vital role in land resource management, urban planning, environmental protection and other applications. Therefore, in the context of rapid population growth and rapid economic and industrial development, it is necessary to evaluate the accuracy of land cover classification for various purposes, especially in developing countries where land cover is changing rapidly.

[0003] By using high-resolution remote sensing images acquired by resource satellites, combined with relevant remote sensing image processing software and adopting reasonable information extraction methods, the corresponding resource information can be obtained quickly and accurately. Since the single remote sensing image data source obtains limited information features of ground objects, it is not conducive to complex ground object classification. In addition, due to the complex background of remote sensing images and the interference of various redundant backgrounds, it is difficult for machine learning and traditional classification algorithms to extract deep features of remote sensing images. Therefore, deep learning came into being. Deep learning is a deep machine learning algorithm that can explore deeper features of images and describe abstract features, and has developed rapidly in the fields of visual analysis, image and text recognition.

[0004] Through the investigation of the current research status in this field at home and abroad, firstly, most of the remote sensing images used for land cover classification use a single remote sensing image; secondly, although the traditional classification algorithm has achieved good results, for remote sensing images with complex objects and high resolution, there is still a problem of low classification accuracy, and samples need to be manually determined. Therefore, how to provide a multi-source remote sensing land cover classification method based on an improved U-Net network is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] One purpose of the present invention is to propose a multi-source remote sensing land cover classification method based on an improved U-Net network. The present invention aims at the complex background of remote sensing images, more interference, and difficulty in extracting useful information features. A method for realizing multi-source remote sensing image land cover classification by improving the U-Net network model is proposed.

[0006] A multi-source remote sensing land cover classification method based on an improved U-Net network according to an embodiment of the present invention comprises the following steps:

[0007] S1, fusing the spectral information of multiple remote sensing data sources to obtain a multi-spectral fused multi-source remote sensing image;

[0008] S2, preprocessing the multi-source remote sensing images acquired in S1, using the remote sensing image data of the acquisition area as training vectors, processing them in ArcGIS, obtaining sample images and sample labels, and dividing them into a sample image training set and a sample image test set in a ratio of 7:3;

[0009] S3, cropping the sample image training set obtained in S2;

[0010] S4, based on the U-Net network model, the convolution layer, Dropout layer and attention mechanism are added, and the semantic segmentation of multi-source remote sensing images and category determination are realized by using the improved U-Net network and U-Net network;

[0011] S5. Classify the land cover of multi-source remote sensing images in the remote sensing image acquisition area and divide the land features into five categories: forest land, water land, wasteland, road, and building land.

[0012] Optionally, the specific implementation method of acquiring the multi-source remote sensing image is:

[0013] S11, fusion of Landsat8 panchromatic image and Landsat8 multispectral image;

[0014] S12, performing radiation correction and atmospheric correction on the fused image;

[0015] S13, orthorectify the GF-2 panchromatic image;

[0016] S14, perform image registration and image fusion on S12 and S13 to obtain a multi-source remote sensing image.

[0017] Optionally, the image fusion uses a Gram-schmidt orthogonalization algorithm:

[0018] V1=u1;

[0019]

[0020] …………;

[0021]

[0022] Among them, the vector group {u1, u2, ..., u n} are independent of each other.

[0023] Optionally, the key steps of the Gram-schmidt algorithm to achieve remote sensing image fusion are as follows:

[0024] Use the original low spatial resolution image to simulate a full-color image;

[0025] The panchromatic image is used as the first component of the GS transformation to perform the GS transformation on the low spatial resolution image. The specific formula is:

[0026]

[0027] Among them, GS T represents the Tth orthogonal component after GS transformation, B T represents the Tth band of the original low spatial resolution remote sensing image; μT represents the mean gray value of the pixel in the Tth band of the original low spatial resolution remote sensing image; φ(B T , G.S. i ) is the original low spatial resolution image from the T band and GS i The covariance between them; i and j represent the number of rows and columns of the original low spatial resolution image respectively; M and N represent the number of rows and columns of the entire image;

[0028] Replace the first component after GS transformation with a high spatial resolution image;

[0029] Finally, perform the GS inverse transform to complete the image fusion and obtain the multi-source remote sensing image:

[0030]

[0031] Optionally, the 30-meter resolution multispectral image and the 15-meter resolution panchromatic image of Landsat8 are fused to obtain a 15-meter resolution multispectral image, and radiation correction and atmospheric correction are performed. The 0.8-meter resolution GF-2 panchromatic image that has been orthorectified is combined for image registration, fusion and cropping to obtain a 0.8-meter resolution multispectral fused multi-source image, and finally a land cover map is obtained based on the obtained multi-source remote sensing image.

[0032] Optionally, improvements based on the U-Net network model include:

[0033] Four layers of convolution are added, and the convolution method is depth-separable convolution;

[0034] Added 2 Dropout layers, and added Dropout layers to the 4th and 5th groups;

[0035] The scSE module is introduced after each set of convolutional layers in the decoding part.

[0036] Optionally, the specific process of using the improved U-Net model to perform semantic segmentation and category determination on objects in multi-source remote sensing images in step 4 is as follows:

[0037] By adding convolutional layers and Dropout layers to the original U-Net network model and introducing the scSE module, a new U-Net multi-source remote sensing image semantic segmentation model is constructed. The pixels are segmented according to the different semantic meanings expressed in the image, and the labels of different objects in the multi-source remote sensing images are obtained. The large-scale region of interest (ROI) is retained as a label for category determination.

[0038] The land cover category of multi-source remote sensing images is determined by using resnet50 as the land cover classification model. According to the set five categories, a similar number of multi-source remote sensing images are selected for each category. The ROI area labels obtained in the previous step are used for training to obtain the selected model. The trained model parameters are used to obtain the category attributes of the test set.

[0039] The beneficial effects of the present invention are:

[0040] The present invention fuses the spectral information of multiple remote sensing data to make up for the missing data, obtain rich land feature features, lay the foundation for land cover classification, and improve the final classification accuracy. It improves the existing U-Net network and proposes adding network layers, Dropout layers and introducing scSE modules, which effectively improves the problem of difficulty in extracting complex land feature information features, solves the problem of difficulty in extracting useful information features, improves the network training speed, and obtains land cover classification results with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0042] Figure 1 A flowchart of a multi-source remote sensing land cover classification method based on an improved U-Net network proposed by the present invention;

[0043] Figure 2 A remote sensing data preprocessing flow chart of a multi-source remote sensing land cover classification method based on an improved U-Net network proposed by the present invention;

[0044] Figure 3 The invention proposes an improved U-Net network model framework for a multi-source remote sensing land cover classification method based on an improved U-Net network. DETAILED DESCRIPTION

[0045] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0046] refer to Figure 1-3 , a multi-source remote sensing land cover classification method based on an improved U-Net network, comprising the following steps:

[0047] S1, fusing the spectral information of multiple remote sensing data sources to obtain a multi-spectral fused multi-source remote sensing image;

[0048] S2, preprocessing the multi-source remote sensing images acquired in S1, using the remote sensing image data of the acquisition area as training vectors, processing them in ArcGIS, obtaining sample images and sample labels, and dividing them into a sample image training set and a sample image test set in a ratio of 7:3;

[0049] S3, cropping the sample image training set obtained in S2;

[0050] S4, based on the U-Net network model, the convolution layer, Dropout layer and attention mechanism are added, and the semantic segmentation of multi-source remote sensing images and category determination are realized by using the improved U-Net network and U-Net network;

[0051] S5. Classify the land cover of multi-source remote sensing images in the remote sensing image acquisition area and divide the land features into five categories: forest land, water land, wasteland, road, and building land.

[0052] In this implementation, the specific implementation method of multi-source remote sensing image acquisition is:

[0053] S11, fusion of Landsat8 panchromatic image and Landsat8 multispectral image;

[0054] S12, performing radiation correction and atmospheric correction on the fused image;

[0055] S13, orthorectify the GF-2 panchromatic image;

[0056] S14, perform image registration and image fusion on S12 and S13 to obtain a multi-source remote sensing image.

[0057] In this embodiment, the image fusion adopts Gram- sc Hmidt orthogonalization algorithm:

[0058] V1=u1;

[0059]

[0060] …………;

[0061]

[0062] Among them, the vector group {u1, u2, ..., u n} are independent of each other.

[0063] In this implementation, the key steps of the Gram-schmidt algorithm to achieve remote sensing image fusion are as follows:

[0064] Use the original low spatial resolution image to simulate a full-color image;

[0065] The panchromatic image is used as the first component of the GS transformation to perform the GS transformation on the low spatial resolution image. The specific formula is:

[0066]

[0067]

[0068]

[0069] Among them, GS T represents the Tth orthogonal component after GS transformation, B T Indicates the T band of the original low spatial resolution remote sensing image; μ T represents the mean gray value of the pixel in the T band of the original low spatial resolution remote sensing image; φ(B T , G.S. i ) is the original low spatial resolution image from the T band and GS i The covariance between them; i and i represent the number of rows and columns of the original low spatial resolution image respectively; M and N represent the number of rows and columns of the entire image;

[0070] Replace the first component after GS transformation with a high spatial resolution image;

[0071] Finally, perform the GS inverse transform to complete the image fusion and obtain the multi-source remote sensing image:

[0072]

[0073] In this implementation, the 30-meter resolution multispectral image and the 15-meter resolution panchromatic image of Landsat8 are fused to obtain a 15-meter resolution multispectral image, and radiation correction and atmospheric correction are performed. The 0.8-meter resolution GF-2 panchromatic image that has been orthorectified is combined for image registration, fusion and cropping to obtain a 0.8-meter resolution multispectral fused multi-source image, and finally a land cover map is obtained based on the obtained multi-source remote sensing image.

[0074] In this implementation, the improvements based on the U-Net network model specifically include:

[0075] Four layers of convolution are added, and the convolution method is depth-separable convolution;

[0076] Added 2 Dropout layers, and added Dropout layers to the 4th and 5th groups;

[0077] The scSE module is introduced after each set of convolutional layers in the decoding part.

[0078] In this implementation, the specific process of using the improved U-Net model in step 4 to perform semantic segmentation and category determination on objects in multi-source remote sensing images is as follows:

[0079] By adding convolutional layers and Dropout layers to the original U-Net network model and introducing the scSE module, a new U-Net multi-source remote sensing image semantic segmentation model is constructed. The pixels are segmented according to the different semantic meanings expressed in the image, and the labels of different objects in the multi-source remote sensing images are obtained. The large-scale region of interest (ROI) is retained as a label for category determination.

[0080] The land cover category of multi-source remote sensing images is determined by using resnet50 as the land cover classification model. According to the set five categories, a similar number of multi-source remote sensing images are selected for each category. The ROI area labels obtained in the previous step are used for training to obtain the selected model. The trained model parameters are used to obtain the category attributes of the test set.

[0081] Embodiment 1:

[0082] Step 1: Image preprocessing

[0083] The 30-meter resolution multispectral image and the 15-meter resolution panchromatic image of Landsat8 are fused to obtain a 15-meter resolution multispectral image;

[0084] Perform radiation correction and atmospheric correction;

[0085] Image registration, fusion and cropping are performed with the 0.8-meter-resolution GF-2 panchromatic image that has been orthorectified to obtain a 0.8-meter-resolution multi-spectral fused multi-source image. Finally, training samples are produced based on the obtained multi-source remote sensing image.

[0086] Step 2: Improve the U-Net network model

[0087] Aiming at the problems existing in the U-Net network in the study of remote sensing images, the network is improved. First, a layer of depth-separable convolution is added after the 3rd, 4th, 6th and 7th convolution layers of the U-Net network to make the network simpler and more universal, and solve the problem of long network training time. Secondly, Dropout layers are added to the 4th and 5th groups of the decoding part to reduce the number of network parameters and prevent the network from overfitting. Finally, the scSE module is introduced after each group of convolution layers in the decoding part to enhance the network's extraction of useful information and suppress the acquisition of useless feature information.

[0088] Step 3: Use the improved U-Net network and U-Net network to perform semantic segmentation on the multi-source remote sensing images obtained after preprocessing in step 1 and realize category determination;

[0089] Step 4: Classify the land cover of the multi-source remote sensing images in the remote sensing image acquisition area, and divide the land features into five categories: forest land, water land, wasteland, road, and building land. Obtain the remote sensing land cover classification results and perform accuracy evaluation.

[0090] The present invention fuses the spectral information of multiple remote sensing data to make up for the missing data, obtain rich land feature features, lay the foundation for land cover classification, and improve the final classification accuracy. It improves the existing U-Net network and proposes adding network layers, Dropout layers and introducing scSE modules, which effectively improves the problem of difficulty in extracting complex land feature information features, solves the problem of difficulty in extracting useful information features, improves the network training speed, and obtains land cover classification results with higher accuracy.

[0091] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-source remote sensing land cover classification method based on an improved U-Net network, characterized in that: The steps include: S1, fusing the spectral information of multiple remote sensing data sources to obtain a multi-spectral fused multi-source remote sensing image; S2, preprocessing the multi-source remote sensing images acquired in S1, using the remote sensing image data of the acquisition area as training vectors, processing them in ArcGIS, obtaining sample images and sample labels, and dividing them into a sample image training set and a sample image test set in a ratio of 7:3; S3, cropping the sample image training set obtained in S2; S4, based on the U-Net network model, the convolution layer, Dropout layer and attention mechanism are added, and the semantic segmentation of multi-source remote sensing images and category determination are realized by using the improved U-Net network and U-Net network; S5. Classify the land cover of multi-source remote sensing images in the remote sensing image acquisition area and divide the land features into five categories: forest land, water land, wasteland, road, and building land.

2. According to claim 1, a multi-source remote sensing land cover classification method based on an improved U-Net network is characterized in that: The specific implementation method of the multi-source remote sensing image acquisition is: S11, fusion of Landsat8 panchromatic image and Landsat8 multispectral image; S12, performing radiation correction and atmospheric correction on the fused image; S13, orthorectify the GF-2 panchromatic image; S14, perform image registration and image fusion on S12 and S13 to obtain a multi-source remote sensing image.

3. The multi-source remote sensing land cover classification method based on the improved U-Net network according to claim 2 is characterized in that: The image fusion adopts the Gram-schmidt orthogonalization algorithm: V1=u1; …………； Among them, the vector group {u1, u2, ..., u n } are independent of each other.

4. The multi-source remote sensing land cover classification method based on the improved U-Net network according to claim 3 is characterized in that: The key steps of the Gram-schmidt algorithm to achieve remote sensing image fusion are as follows: Use the original low spatial resolution image to simulate a full-color image; The panchromatic image is used as the first component of the GS transformation to perform the GS transformation on the low spatial resolution image. The specific formula is: Among them, GS T represents the Tth orthogonal component after GS transformation, B T Indicates the T band of the original low spatial resolution remote sensing image; μ T represents the mean gray value of the pixel in the T band of the original low spatial resolution remote sensing image; φ(B T ,GS i ) is the original low spatial resolution image from the T band and GS i The covariance between them; i and j represent the number of rows and columns of the original low spatial resolution image respectively; M and N represent the number of rows and columns of the entire image; Replace the first component after GS transformation with a high spatial resolution image; Finally, perform the GS inverse transform to complete the image fusion and obtain the multi-source remote sensing image:

5. The multi-source remote sensing land cover classification method based on the improved U-Net network according to claim 4 is characterized in that: The 30-meter resolution multispectral image and the 15-meter resolution panchromatic image of Landsat8 were fused to obtain a 15-meter resolution multispectral image, and radiation correction and atmospheric correction were performed. The 0.8-meter resolution GF-2 panchromatic image that had been orthorectified was combined for image registration, fusion and cropping to obtain a 0.8-meter resolution multispectral fused multi-source image. Finally, the land cover map was obtained based on the multi-source remote sensing image.

6. The multi-source remote sensing land cover classification method based on the improved U-Net network according to claim 1 is characterized in that: The improvements based on the U-Net network model include: Four layers of convolution are added, and the convolution method is depth-separable convolution; Added 2 Dropout layers, and added Dropout layers to the 4th and 5th groups; The scSE module is introduced after each set of convolutional layers in the decoding part.

7. The multi-source remote sensing land cover classification method based on the improved U-Net network according to claim 6 is characterized in that: The specific process of using the improved U-Net model to perform semantic segmentation and category determination on objects in multi-source remote sensing images in step 4 is as follows: By adding convolutional layers and Dropout layers to the original U-Net network model and introducing the scSE module, a new U-Net multi-source remote sensing image semantic segmentation model is constructed. The pixels are segmented according to the different semantic meanings expressed in the image, and the labels of different objects in the multi-source remote sensing images are obtained. The large-scale region of interest (ROI) is retained as a label for category determination. The land cover category of multi-source remote sensing images is determined by using resnet50 as the land cover classification model. According to the set five categories, a similar number of multi-source remote sensing images are selected for each category. The ROI area labels obtained in the previous step are used for training to obtain the selected model. The trained model parameters are used to obtain the category attributes of the test set.

Images