City region ground object classification method based on scene analysis and slice image combination

Abstract

The application discloses a city region ground object classification method based on scene analysis and slice image combination, belongs to the technical field of city region ground objects, and comprises the following steps: collecting remote sensing images, and performing preliminary classification on the remote sensing images by using a semantic segmentation algorithm; performing slice processing on the remote sensing images after preliminary classification to obtain slice images; training and constructing a deep convolutional neural network model by using the obtained slice images, and performing scene environment identification on the slice images by using the trained and constructed deep convolutional neural network model. The application removes a full connection layer and performs end-to-end training, effectively combines context information, reduces noise in high-resolution image classification, and provides a new possibility for high-resolution remote sensing image classification. Meanwhile, by selecting a suitable band combination according to a ground object scene environment, the classification effect is further improved, and the processing difficulty of the complexity and large-scale range of high-resolution remote sensing images is overcome.

Description

Technical Field

[0001] This invention discloses a method for classifying urban features based on scene analysis and sliced ​​images, belonging to the field of urban feature technology. Background Technology

[0002] Remote sensing Earth observation technology is widely used in land surveys, environmental protection, disaster monitoring, urban planning, and mapping. In recent decades, remote sensing technology has developed rapidly, with continuously improving image spatial resolution. On the one hand, high-resolution remote sensing images show more complex details and richer spectral features; on the other hand, the spectral differences among similar features have increased. Therefore, traditional classification methods based on spectral statistical features, as well as machine learning methods such as artificial neural networks, support vector machines, and genetic algorithms, are not suitable for classifying complex remote sensing images and have poor generalization ability. Deep convolutional neural networks (DCNNs), developed based on neural networks, have been widely used in remote sensing image classification due to their superior ability to fully extract deeper information from data and handle complex samples.

[0003] Semantic segmentation is an image segmentation method that learns by removing fully connected layers from a convolutional neural network and training it end-to-end. This algorithm can effectively reduce or even eliminate noise in high-resolution image classification by combining contextual information with features at different scales and levels of the target, providing new possibilities for high-resolution remote sensing image classification. However, semantic segmentation algorithms have not been applied to high-resolution remote sensing image classification. This is partly because high-resolution remote sensing images contain more complex details and spectral information than natural photographs; and partly because remote sensing images have a large scale range, and segmentation algorithms cannot directly process them due to GPU memory limitations. Furthermore, different band feature combinations in remote sensing images, as well as feature combinations derived from band operations, can highlight the attributes of different land features. Therefore, selecting appropriate band combinations based on the scene environment of the land features can improve classification results. Summary of the Invention

[0004] The purpose of this invention is to provide a method for classifying urban features based on scene analysis and sliced ​​images, so as to solve the problem of poor classification effect of urban features in the prior art.

[0005] Urban area feature classification methods based on scene analysis and tile imagery include: S1 acquires remote sensing images and uses the DeepLabV3+ semantic segmentation algorithm to perform preliminary classification of the remote sensing images; S2 performs patch processing on the pre-classified remote sensing images to obtain patch images; S3 uses the patch image to train and construct a deep convolutional neural network (DCNN), and uses the trained and constructed DCNN to perform scene environment recognition on the patch image. S4 divides the patch image into multiple scene environments based on the scene recognition results of DCNN; S5 performs environmental scene analysis on the various scene environments obtained in S4, and selects the optimal band features as the input data for classification in different scene environments; S6 performs semantic segmentation on each patch image after processing by S5 using its own optimal band features, and performs fusion processing on the entire patch based on the coordinates of the patch image to complete the classification of the complete remote sensing image.

[0006] The DeepLabV3+ semantic segmentation algorithm includes an Xception network skeleton, which is combined with a spatial pyramid pooling module. The spatial pyramid pooling module contains a 1×1 convolution and three dilated convolutions with different dilation rates. The DeepLabV3+ semantic segmentation algorithm performs the encoding and decoding stages through the Xception network skeleton and the pyramid pooling module; the three different dilation rates are 6, 12, and 18. The Xception network skeleton consists of three parts: input stream, intermediate stream, and output stream. The Xception skeleton network model is optimized multiple times using the DeepLabV3+ semantic segmentation algorithm. The optimization types include increasing the number of network layers, increasing the intermediate stream repetition to 16 times, replacing max pooling in the Xception network with depthwise separable convolution with stride, and adding normalization and corrected linear ReLU after the 3×3 depthwise separable convolution. The depthwise separable convolution decomposes the convolution into two steps: depthwise convolution and pointwise convolution. The depthwise convolution performs convolution on each channel of the image input separately, and the pointwise convolution merges the results of the depthwise convolution.

[0007] The encoding stage of the DeepLabV3+ semantic segmentation algorithm includes: The dilated convolution and spatial pyramid pooling modules are used to capture contextual information. Dilated convolution increases the receptive field of the convolution kernel by introducing a dilation rate to capture image features at a larger scale, while spatial pyramid pooling obtains global information through pooling operations at different scales. The final output of the encoding is passed through a 1x1 convolution operation with 256 kernels. This 1x1 convolution operation linearly combines the features at each pixel position of the feature map to generate a new feature map.

[0008] The decoding stage of the DeepLabV3+ semantic segmentation algorithm includes: Bilinear interpolation was used to upsample by 4 times, and then fused with low-level features of the same spatial resolution. Before fusion, a 1x1 convolution operation is used to reduce the number of channels in the encoded output feature map to 256; The fused features are subjected to a 3×3 convolution operation to improve feature performance. The feature map is upsampled by a factor of 4 using a bilinear interpolation method, restoring its size to the same as the input image.

[0009] Dilated convolution increases the receptive field of the convolution kernel by introducing a dilation rate on top of standard convolution, thereby capturing image features at a larger scale. The expression is: ; in, It is the output feature map of dilated convolution. It is a dilated convolution input feature map. It is a convolution filter. Here, r is the filter length and r is the dilation rate; when r=1, it is a standard convolution. Indicates the first Each feature value.

[0010] Patch processing divides high-resolution remote sensing images into patches of 600x600 pixels in size, while retaining a 50-pixel overlap area.

[0011] Scene recognition is based on DCNN, which includes 2 convolutional layers, 2 pooling layers, and 2 fully connected layers. The steps for scene recognition using DCNN are as follows: The input feature for each patch after rectangular segmentation is 600x600x3. The first convolutional layer is 3x3x64, resulting in 600x600x64 convolutional features. The first max pooling layer is 2x2, resulting in a 300x300x64 pooling layer. The second convolutional layer is 3x3x128, resulting in 300x300x128 convolutional features. The second max pooling layer is 2x2, resulting in a 150x150x128 pooling layer. The first and second fully connected layers use modified linear ReLU as the activation function, Softmax as the classifier to divide the scene environment into seven categories, and cross-entropy as the loss function. The expression is: ; In the formula, n is the characteristic number. The actual label value. For the predicted values, the Adam optimizer is finally used to update the network weights Wi and bias b parameters. The scene environment to which the patch image belongs is output based on the obtained results.

[0012] The various scene environments obtained in step S4 include, but are not limited to, tall building areas, vegetation areas, surface areas, and combinations thereof, where a single scene may include tall building areas. Vegetation area and surface area The two-way combination scenes are tall building area and vegetation area, tall building area and ground area, and vegetation area and ground area; the three-way combination scene is a combination of tall building area, vegetation area and ground area. Finally, based on the area ratio of ground features in the patch image, different scene environment rules are defined.

[0013] The classification process in S5 involves analyzing the environmental scene of the patch image, selecting the optimal band features as the classification input data under different scene environments, and obtaining the band combination that maximizes the overall classification accuracy. The expression is: ; Where OA represents the overall classification accuracy. Let N be the number of pixels in the image, representing the j-th scene environment. Number of land cover categories denoted by the diagonal values ​​in the confusion matrix, where p represents the accuracy.

[0014] The steps for semantic segmentation of each patch image after S5 processing using its optimal band combination, followed by fusion processing of the entire patch based on its coordinates to complete the classification of the complete remote sensing image are as follows: Environmental scene analysis is performed on each patch image, and the optimal band combination is selected based on the scene environment; The DeepLabV3+ semantic segmentation algorithm is used to classify the selected optimal band combination to obtain the classification result of each patch; Based on the coordinates of the patch images, the classification results are fused, retaining a 50-pixel overlapping area. During merging, 25 pixels from each image are selected for stitching, ultimately forming a complete high-resolution remote sensing image classification result.

[0015] Compared to existing technologies, this invention offers the following advantages: DCNN semantic segmentation, by removing fully connected layers and performing end-to-end training, effectively combines contextual information, reducing noise in high-resolution image classification and providing new possibilities for high-resolution remote sensing image classification. Simultaneously, by selecting appropriate band combinations based on the ground scene environment, the classification effect is further improved, overcoming the challenges of processing the complexity and large scale of high-resolution remote sensing images. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the classification structure of the complete high-resolution remote sensing image of the present invention; Figure 2 This is a diagram showing the comparison of classification accuracy in a single scene using slices. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] Urban area feature classification methods based on scene analysis and tile imagery include: S1 acquires remote sensing images and uses the DeepLabV3+ semantic segmentation algorithm to perform preliminary classification of the remote sensing images; S2 performs patch processing on the pre-classified remote sensing images to obtain patch images; S3 uses the patch image to train and construct a deep convolutional neural network (DCNN), and uses the trained and constructed DCNN to perform scene environment recognition on the patch image. S4 divides the patch image into multiple scene environments based on the scene recognition results of DCNN; S5 performs environmental scene analysis on the various scene environments obtained in S4, and selects the optimal band features as the input data for classification in different scene environments; S6 performs semantic segmentation on each patch image after processing by S5 using its own optimal band features, and performs fusion processing on the entire patch based on the coordinates of the patch image to complete the classification of the complete remote sensing image.

[0019] The DeepLabV3+ semantic segmentation algorithm includes an Xception network skeleton, which is combined with a spatial pyramid pooling module. The spatial pyramid pooling module contains a 1×1 convolution and three dilated convolutions with different dilation rates. The DeepLabV3+ semantic segmentation algorithm performs the encoding and decoding stages through the Xception network skeleton and the pyramid pooling module; the three different dilation rates are 6, 12, and 18. The Xception network skeleton consists of three parts: input stream, intermediate stream, and output stream. The Xception skeleton network model is optimized multiple times using the DeepLabV3+ semantic segmentation algorithm. The optimization types include increasing the number of network layers, increasing the intermediate stream repetition to 16 times, replacing max pooling in the Xception network with depthwise separable convolution with stride, and adding normalization and corrected linear ReLU after the 3×3 depthwise separable convolution. The depthwise separable convolution decomposes the convolution into two steps: depthwise convolution and pointwise convolution. The depthwise convolution performs convolution on each channel of the image input separately, and the pointwise convolution merges the results of the depthwise convolution.

[0020] The encoding stage of the DeepLabV3+ semantic segmentation algorithm includes: The dilated convolution and spatial pyramid pooling modules are used to capture contextual information. Dilated convolution increases the receptive field of the convolution kernel by introducing a dilation rate to capture image features at a larger scale, while spatial pyramid pooling obtains global information through pooling operations at different scales. The final output of the encoding is passed through a 1x1 convolution operation with 256 kernels. This 1x1 convolution operation linearly combines the features at each pixel position of the feature map to generate a new feature map.

[0021] The decoding stage of the DeepLabV3+ semantic segmentation algorithm includes: Bilinear interpolation was used to upsample by 4 times, and then fused with low-level features of the same spatial resolution. Before fusion, a 1x1 convolution operation is used to reduce the number of channels in the encoded output feature map to 256; The fused features are subjected to a 3×3 convolution operation to improve feature performance. The feature map is upsampled by a factor of 4 using a bilinear interpolation method, restoring its size to the same as the input image.

[0022] Dilated convolution increases the receptive field of the convolution kernel by introducing a dilation rate on top of standard convolution, thereby capturing image features at a larger scale. The expression is: ; in, It is the output feature map of dilated convolution. It is a dilated convolution input feature map. It is a convolution filter. Here, r is the filter length and r is the dilation rate; when r=1, it is a standard convolution. Indicates the first Each feature value.

[0023] Patch processing divides high-resolution remote sensing images into patches of 600x600 pixels in size, while retaining a 50-pixel overlap area.

[0024] Scene recognition is based on DCNN, which includes 2 convolutional layers, 2 pooling layers, and 2 fully connected layers. The steps for scene recognition using DCNN are as follows: The input feature for each patch after rectangular segmentation is 600x600x3. The first convolutional layer is 3x3x64, resulting in 600x600x64 convolutional features. The first max pooling layer is 2x2, resulting in a 300x300x64 pooling layer. The second convolutional layer is 3x3x128, resulting in 300x300x128 convolutional features. The second max pooling layer is 2x2, resulting in a 150x150x128 pooling layer. The first and second fully connected layers use modified linear ReLU as the activation function, Softmax as the classifier to divide the scene environment into seven categories, and cross-entropy as the loss function. The expression is: ; In the formula, n is the characteristic number. The actual label value. For the predicted values, the Adam optimizer is finally used to update the network weights Wi and bias b parameters. The scene environment to which the patch image belongs is output based on the obtained results.

[0025] The various scene environments obtained in step S4 include, but are not limited to, tall building areas, vegetation areas, surface areas, and combinations thereof, where a single scene may include tall building areas. Vegetation area and surface area The two-way combination scenes are tall building area and vegetation area, tall building area and ground area, and vegetation area and ground area; the three-way combination scene is a combination of tall building area, vegetation area and ground area. Finally, based on the area ratio of ground features in the patch image, different scene environment rules are defined.

[0026] The classification process in S5 involves analyzing the environmental scene of the patch image, selecting the optimal band features as the classification input data under different scene environments, and obtaining the band combination that maximizes the overall classification accuracy. The expression is: ; Where OA represents the overall classification accuracy. Let N be the number of pixels in the image, representing the j-th scene environment. For the number of land cover categories, denoted by the diagonal values ​​in the confusion matrix, where p represents the accuracy.

[0027] The steps for semantic segmentation of each patch image after S5 processing using its optimal band combination, followed by fusion processing of the entire patch based on its coordinates to complete the classification of the complete remote sensing image are as follows: Environmental scene analysis is performed on each patch image, and the optimal band combination is selected based on the scene environment; The DeepLabV3+ semantic segmentation algorithm is used to classify the selected optimal band combination to obtain the classification result of each patch; Based on the coordinates of the patch images, the classification results are fused, retaining a 50-pixel overlapping area. During merging, 25 pixels from each image are selected for stitching, ultimately forming a complete high-resolution remote sensing image classification result.

[0028] The complete high-resolution remote sensing image classification structure of this invention is as follows: Figure 1 As shown, the classification accuracy in a single scene with slices is compared to... Figure 2 As shown. In this embodiment of the invention, S1 acquires remote sensing images and uses the DeepLabV3+ semantic segmentation algorithm to perform preliminary classification of the remote sensing images. The remote sensing images are acquired through high-resolution remote sensing equipment such as satellites or drones. The resolution of the remote sensing images should be high enough to contain detailed information about urban features, such as buildings, roads, and vegetation. Furthermore, the acquired remote sensing images need to undergo radiometric correction, geometric correction, and noise reduction to ensure image quality and consistency. The classification results include the feature category of each pixel, such as buildings, roads, and vegetation.

[0029] In step S2, the pre-classified remote sensing images are patched to obtain patched images. Since high-resolution remote sensing images are usually large in size, direct processing would consume a lot of computing resources and be limited by hardware memory constraints. By dividing the image into smaller patches, the computational requirements for each processing run can be reduced, and processing efficiency can be improved.

[0030] In S5, environmental scene analysis is performed on the scene environment obtained in S4. The optimal band features are selected as the input data for classification under different scene environments. Different land cover types and scene environments exhibit different spectral characteristics in different bands of remote sensing images. Selecting the optimal band features as the input data for classification can significantly improve classification accuracy. The spectral characteristics of buildings are prominent in the visible and near-infrared bands, vegetation has significant characteristics in the red-edge and near-infrared bands, and surface areas have significant characteristics in the visible and short-wave infrared bands. For combined scenes containing multiple land cover types, the band combination with the best overall performance is selected by analyzing the band characteristics of each type of land cover.

[0031] The steps to obtain optimal band data are as follows: collect band data from different scene environments, perform spectral analysis, and determine the optimal band features for each scene environment. Then, use statistical methods and machine learning algorithms (such as feature selection algorithms and principal component analysis) to select the band combination most favorable for classification.

[0032] In step S6, each patch image processed in step S5 is semantically segmented using its optimal band combination. Then, the entire patch is fused based on its coordinates to complete the classification of the complete remote sensing image. The classification results are applicable to various scene classification tasks for high-resolution remote sensing images, such as urban planning, environmental monitoring, and land use surveys.

[0033] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for classifying urban land cover based on scene analysis and tiled imagery, characterized in that, include: S1 acquires remote sensing images and uses the DeepLabV3+ semantic segmentation algorithm to perform preliminary classification of the remote sensing images; S2 performs patch processing on the pre-classified remote sensing images to obtain patch images; S3 uses the patch image to train and construct a deep convolutional neural network (DCNN), and uses the trained and constructed DCNN to perform scene environment recognition on the patch image. S4 divides the patch image into multiple scene environments based on the scene recognition results of DCNN; S5 performs environmental scene analysis on the various scene environments obtained in S4, and selects the optimal band features as the input data for classification in different scene environments; S6 performs semantic segmentation on each patch image after processing by S5 using its own optimal band features, and performs fusion processing on the entire patch based on the coordinates of the patch image to complete the classification of the complete remote sensing image. The various scene environments obtained in step S4 include, but are not limited to, tall building areas, vegetation areas, surface areas, and combinations thereof, where a single scene may include tall building areas. Vegetation area and surface area The two-way combination scenes are tall building area and vegetation area, tall building area and ground area, and vegetation area and ground area; the three-way combination scene is a combination of tall building area, vegetation area and ground area. Finally, based on the area ratio of ground features in the patch image, different scene environment rules are defined; The classification process in S5 involves analyzing the environmental scene of the patch image, selecting the optimal band features as the classification input data under different scene environments, and obtaining the band combination that maximizes the overall classification accuracy. The expression is: ； Where OA represents the overall classification accuracy. For the scene environment, N is the number of pixels in the image, and r is the number of land cover categories. These are the diagonal values ​​in the confusion matrix; The steps for semantic segmentation of each patch image after S5 processing using its optimal band combination, followed by fusion processing of the entire patch based on its coordinates to complete the classification of the complete remote sensing image are as follows: Environmental scene analysis is performed on each patch image, and the optimal band combination is selected based on the scene environment; The DeepLabV3+ semantic segmentation algorithm is used to classify the selected optimal band combination to obtain the classification result of each patch; Based on the coordinates of the patch images, the classification results are fused, retaining a 50-pixel overlapping area. During the merge process, 25 pixels from each image are selected and stitched together to form a complete high-resolution remote sensing image classification result.

2. The urban area feature classification method based on scene analysis and slice imagery as described in claim 1, characterized in that, The DeepLabV3+ semantic segmentation algorithm includes an Xception network skeleton, which is combined with a spatial pyramid pooling module. The spatial pyramid pooling module contains a 1×1 convolution and three dilated convolutions with different dilation rates. The DeepLabV3+ semantic segmentation algorithm performs the encoding and decoding stages through the Xception network skeleton and the pyramid pooling module. The three different dilation rates are 6, 12, and 18. The Xception network skeleton consists of three parts: input stream, intermediate stream, and output stream. The Xception skeleton network model is optimized multiple times using the DeepLabV3+ semantic segmentation algorithm. The optimization types include increasing the number of network layers, increasing the intermediate stream repetition to 16 times, replacing max pooling in the Xception network with depthwise separable convolution with stride, and adding normalization and corrected linear ReLU after the 3×3 depthwise separable convolution. The depthwise separable convolution decomposes the convolution into two steps: depthwise convolution and pointwise convolution. The depthwise convolution performs convolution on each channel of the image input separately, and the pointwise convolution merges the results of the depthwise convolution.

3. The urban area feature classification method based on scene analysis and slice imagery as described in claim 2, characterized in that, The encoding stage of the DeepLabV3+ semantic segmentation algorithm includes: The dilated convolution and spatial pyramid pooling modules are used to capture contextual information. Dilated convolution increases the receptive field of the convolution kernel by introducing a dilation rate to capture image features at a larger scale, while spatial pyramid pooling obtains global information through pooling operations at different scales. The final output of the encoding is passed through a 1x1 convolution operation with 256 kernels. This 1x1 convolution operation linearly combines the features at each pixel position of the feature map to generate a new feature map.

4. The urban area feature classification method based on scene analysis and slice imagery as described in claim 3, characterized in that, The decoding stage of the DeepLabV3+ semantic segmentation algorithm includes: Bilinear interpolation was used to upsample by 4 times, and then fused with low-level features of the same spatial resolution. Before fusion, a 1x1 convolution operation is used to reduce the number of channels in the encoded output feature map to 256; The fused features are subjected to a 3×3 convolution operation to improve feature performance. The feature map is upsampled by a factor of 4 using a bilinear interpolation method, restoring its size to the same as the input image.

5. The urban area feature classification method based on scene analysis and slice imagery as described in claim 4, characterized in that, Dilated convolution increases the receptive field of the convolution kernel by introducing a dilation rate on top of standard convolution, thereby capturing image features at a larger scale. The expression is: ； in, It is the output feature map of dilated convolution. It is a dilated convolution input feature map. It is a convolutional filter, k is the filter length, r is the dilation rate, and when r=1 it is a standard convolution.

6. The urban area feature classification method based on scene analysis and slice imagery as described in claim 5, characterized in that, Patch processing divides high-resolution remote sensing images into patches of 600x600 pixels in size, while retaining an overlap area of ​​50 pixels.

7. The urban area feature classification method based on scene analysis and slice imagery as described in claim 6, characterized in that, The scene recognition is based on DCNN, including 2 convolutional layers, 2 pooling layers, and 2 fully connected layers. The steps for scene recognition using DCNN are as follows: The input feature for each patch after rectangular segmentation is 600x600x3. The first convolutional layer is 3x3x64, resulting in 600x600x64 convolutional features. The first max pooling layer is 2x2, resulting in a 300x300x64 pooling layer. The second convolutional layer is 3x3x128, resulting in 300x300x128 convolutional features. The second max pooling layer is 2x2, resulting in a 150x150x128 pooling layer. The first and second fully connected layers use modified linear ReLU as the activation function and Softmax as the classifier to divide the scene environment into seven categories. Cross-entropy is used as the loss function, specifically: ； In the formula, The actual label value. For the predicted values, the Adam optimizer is finally used to update the network weights Wi and bias b parameters. The scene environment to which the patch image belongs is output based on the obtained results.

Images