A land parcel type classification method based on image

Abstract

The present invention belongs to the field of computer technology application, and specifically relates to an image-based land parcel type classification method. The present invention uses a neural network to distinguish the types of land parcel units divided from original satellite images with road network information as boundaries. The neural network consists of a feature extractor, an adaptive pyramid pooling layer, and a classifier. The feature extractor specifically includes convolution, batch normalization, ReLU, and maximum pooling processing; the adaptive pyramid pooling layer uses pooling windows of different numbers and sizes to perform maximum pooling operations on feature data obtained by the feature extractor, and then splices the pooling results; the classifier maps the obtained feature vector to the actual label space of the input data. The trained network model predicts the type of land parcel data.

Description

Technical Field

[0001] The invention belongs to the field of computer technology application, and in particular relates to an image-based land parcel type classification method. Background Art

[0002] The research on high-resolution satellite images has developed into many fields. Among them, the classification of land parcels in urban satellite images is a very important and promising task. Its results can be well applied to valuable directions in life such as urban rail transit site selection, and can also provide certain data support for urban planning and construction.

[0003] Convolutional neural networks have powerful representation capabilities and can be naturally introduced into the scene classification of satellite images. However, due to the existence of fully connected layers, traditional neural network architectures can only receive single-scale images as input data. In the task of satellite image plot classification, all plot information is obtained based on road network information, so most of them are irregular and have large variations in shape, specifications and size. When faced with the classification problem of multi-scale image data such as satellite image plot classification, traditional convolutional neural networks are obviously no longer applicable. Moreover, in actual satellite images, there may be a high degree of similarity between plots of different categories, such as areas where buildings are relatively concentrated in plots of different categories; and there may be great differences in plots of the same category, such as plots in the same residential area may have different roof colors and different building location distributions. At the same time, semantic recognition of satellite images also needs to consider some related interference factors, such as satellite images have large differences in spectral reflectance, contrast and shadows.

[0004] In order to solve the above problems, the present invention uses convolutional neural networks as the entry point and residual network architecture as the basis to extract deep semantic information. By introducing an adaptive pyramid pooling layer, the entire network architecture can process land parcel data in satellite images of any scale, thereby obtaining a solution for processing satellite image land parcel classification problems. This research scheme can divide urban land into individual land parcel data for specific type classification by segmenting land parcel data in satellite images. Summary of the invention

[0005] The purpose of the present invention is to propose a land parcel type classification method based on images.

[0006] The present invention proposes an image-based land parcel type classification method, wherein the land parcel data is land parcel units divided by road network information as boundaries in the original satellite image, and the present invention uses a neural network to distinguish the types of these land parcel units.

[0007] Based on the residual network and combined with the specific data characteristics, this paper proposes a land parcel type classification method in urban satellite images. The entire neural network consists of three parts: feature extractor, adaptive pyramid pooling layer and classifier. Among them:

[0008] Feature extractor; First, all the plot image data are obtained by combining the road network information in large-scale high-resolution satellite images, so all images have large differences in size and aspect ratio. Secondly, the roads in the city are not parallel, so the plot data directly obtained must be irregular in shape. In addition, all data inevitably carry boundary information, and these boundary information are almost meaningless black pixels. If they are used directly for classification without processing, they will inevitably have a certain impact on the final classification result. Therefore, after analyzing the original architecture of the residual network, the present invention decides to use the original residual network as the basis, add a maximum pooling layer between its residual blocks, and retain the most significant features in a certain area of ​​the feature map in a maximum pooling manner for subsequent calculations, so that at the boundary, a part of the influence caused by the boundary information can be discarded to a certain extent. When performing feature extraction, the input image will first pass through a structure called an initialization layer before entering the first residual block. The function of this initialization layer is to perform some initialization processing on the input data before performing convolution processing on it to extract features, specifically including convolution, batch normalization, ReLU and maximum pooling processing.

[0009] Among them, the step size of the convolution of the original image is 2, and the convolution kernel size is 7*7, with a total of 64. After that, batch normalization is performed on the extracted features. Then the ReLU operation is introduced to enhance the nonlinear processing capability of the network architecture. Finally, for the output result of the ReLU function, a 3*3 pooling window is used with a moving step of 2 pixels for pooling operation. It can be calculated that the size of the original image data after the initial layer (the total number of pixels on a single channel) is only 1 / 16 of the original size. However, since the convolution operation uses 64 convolution kernels for feature extraction, the output result of the initialization layer has up to 64 channels. After the initialization layer is a series of convolution layers, the number of convolution kernels used in these convolution layers will increase accordingly as the network deepens. Therefore, all convolution layers can be divided into four residual blocks according to the number of convolution kernels. In each residual block, every three convolution layers form a residual unit, and the sizes of the convolution kernels of these three convolution layers are [1*1, 3*3, 1*1] respectively. The first residual block can directly receive the output result of the initialization layer as input data, and generate output data that can be directly used in subsequent network blocks. In order to reduce the influence of boundary information in the input image and further enhance the effectiveness of the features obtained by the entire feature extractor, the present invention adds a maximum pooling layer between every two residual blocks. The combination of each residual block and the maximum pooling layer can reduce the size of the input data (the total number of pixels on a single channel) to 1 / 4 of the original. Therefore, the data size (the total number of pixels on a single channel) after passing through all convolutional layers is only 1 / 64 of the input data size of the network block, but has a channel number of up to 2048. The combination of an initialization layer, four residual blocks and a maximum pooling layer together constitutes the feature extractor of the network architecture designed by the present invention.

[0010] Adaptive pyramid pooling layer: The adaptive pyramid pooling layer uses pooling windows of different numbers and sizes to perform maximum pooling operations on the feature data obtained by the feature extractor, and then concatenates the pooling results.

[0011] When the pyramid layer of adaptive pyramid pooling layer works, it is necessary to set the number of pooling windows parameter [n0, n1, ...]. This parameter is a sequence. Each parameter in the sequence represents the number of pooling windows used by the pooling layer corresponding to the parameter subscript. In addition, since the feature map is not completely square, the pooling window is not completely square either, but will be adaptively adjusted according to the size of the feature map, and the horizontal and vertical movement steps of the pooling window are also different during the pooling operation. The calculation method of the pooling window size and the movement step is as follows:

[0012]

[0013]

[0014] Formula 1 and Formula 2 respectively represent the x*y sliding window size used when performing n*n pooling operation on feature map of size a*b and the calculation method of the horizontal and vertical moving step of the pooling window. Where [z] represents the rounding up of the value z, Indicates the rounding down of the value z, and n is a specific parameter in the preset adaptive pyramid pooling layer parameter sequence. The convolutional neural network that introduces the adaptive pyramid pooling layer has two main advantages. First, after the introduction of the adaptive pyramid pooling layer, the entire network architecture can take plot image data of any scale and size as input, and provide a fixed-length feature vector for the fully connected layer for classification; second, the adaptive pyramid pooling layer can merge features at different scales, thereby obtaining richer feature data, which is helpful for the subsequent classification of the classifier.

[0015] Classifier; The function of the classifier part is to map the feature vector obtained after the convolution layer and the pooling layer to the actual label space of the input data. The core operation of this level is to map the output of the previous layer to another probability distribution by means of vector product. After passing through the fully connected layer, the feature vector obtained by the adaptive pyramid convolution is mapped into a feature vector with a dimension equal to the number of categories, and the feature vector is then passed through a Softmax function to obtain the confidence probability of the input data for each category. In the classifier part of the present invention, the fully connected layer can receive the output generated by the adaptive pyramid pooling layer as input data, and then pass through a Dropout layer introduced to prevent overfitting. The feature vector after Dropout will be mapped into a feature vector with a dimension equal to the classification category, and then pass through the Softmax function to calculate the confidence probability for each category, and finally achieve the distinction between categories. The calculation method of the Softmax function is as follows:

[0016]

[0017] After the training is completed, the trained model can be used to predict the type of the land parcel data. During the prediction, in order to further improve the prediction accuracy, the present invention will perform multiple predictions on different locations of the same land parcel data.

[0018] The satellite image plot classification network designed by the present invention consists of three parts: a feature extractor, an adaptive pyramid pooling layer and a classifier. The feature extractor is based on the residual network architecture, and a maximum pooling layer is added to reduce the influence of boundary information. By introducing the adaptive pyramid pooling layer, feature data under different dimensions can be combined together and a feature vector of uniform length is generated. In the classifier, a Dropout layer is introduced to prevent overfitting, and finally the confidence probability of the input image data about each category is obtained by the Softmax function. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The overall architecture of the classification network designed by the present invention is demonstrated.

[0020] Figure 2 The composition of the feature extractor designed by the present invention is demonstrated.

[0021] Figure 3 Demonstrates how the adaptive pyramid pooling layer works.

[0022] Figure 4 The composition of the classifier designed by the present invention is shown.

[0023] Figure 5 The working principle of the present invention in predicting the data type of a plot of land is demonstrated. DETAILED DESCRIPTION

[0024] The specific implementation of the present invention will be described below in conjunction with the accompanying drawings.

[0025] Figure 1 The overall architecture of the classification network designed by the present invention is shown. In the specific implementation of the present invention, the input plot image will first pass through the feature extractor for initial feature extraction. Plot data of different sizes will generate feature maps of different sizes after passing through the feature extractor. For example Figure 1 As shown in , if the plot images of size [256*256, 224*224, 192*192] are used as input, then after the feature extractor, feature maps of size [8*8, 7*7, 6*6] with 2048 channels will be output respectively. After these feature maps of different sizes pass through the adaptive pyramid pooling layer, the feature data of different dimensions they contain will be further combined and generate feature vectors of size 1*1*22624. Finally, these feature vectors will be distinguished by the classifier.

[0026] Figure 2In the figure, the number under each module indicates the dimension of the feature map output by the module, such as "256*256*3" indicates a three-channel 256*256 pixel image. The symbols in each convolution block indicate the size and number of convolution kernels, and "conv3-64" means that the convolution layer uses 64 convolution kernels of size 3*3 for convolution operations. For example, Figure 2 When a plot image of size 256*256*3 is used as the original input, the image will first pass through the initialization layer, which will perform some initialization processing on the input data, including convolution, batch normalization, ReLU and maximum pooling, and then the data size will become 64*64*64. Next, the data will pass through the processing unit consisting of three groups of residual blocks and the maximum pooling layer in sequence, and the size of the feature map has become 16*16*1024. Finally, this feature map will be processed by a residual block again, and finally generate feature data of size 8*8*2048.

[0027] The adaptive pyramid pooling layer processes data as follows: Figure 3 As shown in the figure, after a certain original plot data passes through the feature extractor, a feature map of size 8*8*2048 is obtained, and this feature map will pass through the adaptive pyramid pooling layer. At this time, the number of pooling windows is set to [3,2,1]. Then the feature data will be subjected to the maximum pooling operation by the pooling windows of number 3*3, 2*2 and 1*1 in turn, and the corresponding pooling window sizes are [3*3,4*4,8*8], and the step sizes are [2,4,8] respectively. After flattening the results of these three layers of maximum pooling operations, three feature vectors of size 1*1*18432, 1*1*8192 and 1*1*2048 will be obtained respectively. These three feature vectors are then concatenated to obtain a feature vector of size 1*1*22634, which is used for classification in the fully connected layer. It is worth noting that even if feature maps of different sizes are input into the adaptive pyramid pooling layer, the same operation will be performed, and feature vectors of the same dimension will be obtained in the end.

[0028] like Figure 4 After receiving the feature vector of size 1*1*22634, the classifier will perform dimension mapping on the feature data by calculating the vector outer product, and finally generate a feature vector of the length of the classification category. For example, when the classification category is 3, a 1*1*3 vector will be generated. This final vector will pass through the softmax function to generate the probability of the corresponding category. For example, if the final result is [0.9, 0.2, 0.1], the image category is 1.

[0029] Figure 5 It shows how to make multiple predictions for different locations of a specific plot. Figure 5The plot data shown will be predicted 6 times, namely the entire plot, its central area and its four corners. Figure 5 When the method of the present invention is used to predict the plot shown, the results of 6 predictions are [1, 1, 1, 1, 3, 2] respectively. Therefore, the category of the plot is finally determined to be 1.

Claims

1. A land parcel type classification method based on an image, wherein the land parcel data is a land parcel unit divided by road network information as a boundary in the original satellite image, characterized in that: A neural network is used to distinguish the types of these plot units; The neural network consists of three parts, namely, a feature extractor, an adaptive pyramid pooling layer and a classifier; wherein: The feature extractor, after analyzing the original architecture of the residual network, takes the original residual network as the basis, adds a maximum pooling layer between its residual blocks, and retains the most significant features in a certain area of ​​the feature map for subsequent calculations in a maximum pooling manner, and then discards part of the influence caused by the boundary information at the boundary; when performing feature extraction, the input image first passes through a structure called an initialization layer before entering the first residual block; the function of this initialization layer is to perform some initialization processing on the input data before performing convolution processing on it to extract features; specifically, it includes convolution, batch normalization, ReLU and maximum pooling processing; Among them, the step size of the convolution on the original image is 2, and the convolution kernel size is 7*7, totaling 64; then, batch normalization is performed on the extracted features; then the ReLU operation is introduced to enhance the nonlinear processing capability of the network architecture; finally, for the output result of the ReLU function, a 3*3 pooling window is used with a moving step of 2 pixels for pooling operation; the size of the original image data after passing through the initial layer is calculated to be only 1 / 16 of the original size; since the convolution operation uses 64 convolution kernels for feature extraction, the output result of the initialization layer has up to 64 channels; the initialization layer is followed by a series of convolution layers, and the number of convolution kernels used in these convolution layers increases accordingly as the network deepens; therefore, all convolution layers are divided into four residual blocks according to the number of convolution kernels; in each residual In the block, every three convolutional layers form a residual unit, and the sizes of the convolution kernels of these three convolutional layers are [1*1, 3*3, 1*1] respectively; the first residual block directly receives the output result of the initialization layer as input data, and generates output data that can be directly used in subsequent network blocks; in order to weaken the influence of boundary information in the input image and further enhance the effectiveness of the features obtained by the entire feature extractor; a maximum pooling layer is added between every two residual blocks; the combination of each residual block and the maximum pooling layer can reduce the size of the input data to 1 / 4 of the original; therefore, the data size after all convolutional layers is only 1 / 64 of the input data size of the network block; but it has a channel number of up to 2048; the combination of an initialization layer, four residual blocks and a maximum pooling layer together constitute the feature extractor of the designed network architecture; The adaptive pyramid pooling layer uses pooling windows of different numbers and sizes to perform a maximum pooling operation on the feature data obtained by the feature extractor, and then performs a splicing process on the pooling results; When the pyramid layer works, the number of pooling windows is set [n0, n1, ...]. This parameter is a sequence. Each parameter in the sequence represents the number of pooling windows used by the pooling layer corresponding to the parameter subscript. In addition, since the feature map is not completely square, the pooling window is not completely square either. It is adaptively adjusted according to the size of the feature map. The calculation method of the pooling window size and the moving step size is as follows: Formula 1 and Formula 2 respectively represent the x*y sliding window size used when performing n*n pooling operation on the feature map of size a*b and the calculation method of the horizontal and vertical moving step length of the pooling window; Indicates that the value z is rounded up. Indicates that the value z is rounded down, and n is a specific parameter in the preset adaptive pyramid pooling layer parameter sequence; The classifier is used to map the feature vector obtained after the convolution layer and the pooling layer to the actual label space of the input data. The operation at this level is to map the output of the previous layer to another probability distribution by means of vector product. After the fully connected layer, the feature vector obtained by the adaptive pyramid convolution is mapped into a feature vector with a dimension equal to the number of categories. The feature vector is then passed through a Softmax function to obtain the confidence probability of the input data for each category. In the classifier part, the fully connected layer can receive the output generated by the adaptive pyramid pooling layer as input data, and then pass through a Dropout layer introduced to prevent overfitting; the feature vector after Dropout is mapped into a feature vector with the dimension of the classification category, and then passes through the Softmax function to calculate the confidence probability of each category, and finally realizes the classification; the calculation method of the Softmax function is as follows: After the training is completed, the trained model is used to predict the type of land parcel data.

Images