Building migration extraction method in remote sensing images based on anomaly detection
Through anomaly detection and reclassification technology, the problem of deep learning models' dependence on high-quality samples and low accuracy in building extraction is solved, and high-precision building extraction and model migration improvement is achieved.
Patent Information
- Application Number
- CN202210176333.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-02-25
AI Technical Summary
Existing deep learning models require a large number of high-quality target area samples and rely on manual annotation data when extracting buildings, resulting in low accuracy, high computational difficulty, and limited generalization capabilities.
Using a remote sensing image building migration and extraction method based on anomaly detection, the open data set is trained through a deep semantic segmentation network model, combined with an isolated forest algorithm and a supervision classifier, the building extraction results are preprocessed and reclassified, and abnormal objects are eliminated. , improve building sample accuracy and extraction accuracy.
The high-precision extraction of buildings from high-resolution remote sensing images is achieved, which improves the overall accuracy of building extraction and the mobility of models, and reduces the dependence on high-quality samples.
Smart Images

Figure CN114694019B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to remote sensing image detection, in particular to a remote sensing image building migration extraction method based on anomaly detection. Background Art
[0002] Buildings are one of the most important land features that change frequently. Therefore, automatic, accurate and efficient extraction of buildings is of great significance in urban planning, data updating and disaster assessment. The development of remote sensing technology and the gradual maturity of computer image processing technology have promoted the intelligent extraction of land features, and building extraction has also become a hot topic for scholars.
[0003] A large number of studies have been conducted on extracting buildings from high spatial resolution images. Traditional work mainly focuses on the integrated use of various supervised classifiers and extraction methods designed using highly discriminative ground features. Jin et al. combined the spectral information, structural information, and contextual information in the image and used mathematical morphological methods to extract buildings. Wegner et al. proposed combining the algorithmic advantages of regional segmentation and Markov random fields to perform image scene modeling and building extraction.
[0004] In recent years, in view of the excellent performance of convolutional neural networks in computer vision target classification and image segmentation, many scholars have applied CNN-based semantic segmentation algorithms to the field of remote sensing and gradually obtained satisfactory experimental results. Long et al. proposed a fully convolutional neural network (FCN) by combining downsampling and upsampling inside the convolutional neural network, and successfully achieved semantic segmentation of images. Ji Shunping et al. designed the SUNet model based on UNet and FPN to a certain extent, which solved the multi-scale problem when extracting buildings from high spatial resolution images. Wu Muyu et al. combined the dilated convolution, attention mechanism and multi-scale prediction on the LinkNet framework, and proposed ADLinkNet to further refine the extraction results of buildings. However, the ground feature method based on deep learning requires a large number of samples to train the model. When Wu Hua et al. tested the improved PSPNet model that fused multiple features, they manually annotated the buildings and segmented them to obtain 30,000 576×576 pixel images. Luo Liyan et al. used ArcGIS to manually draw vector maps of buildings, and the entire area was segmented into 53,220 256×256 pixel tiles without overlap, which facilitated the application of their deep learning network. When training the model, Guo Wen et al. selected 200 high-resolution satellite remote sensing images to produce building sample data. The sample labels were obtained by synthesizing the building-related patches in the results of geographic national conditions monitoring, and the accuracy was lower than that of the manually labeled data set. However, when the constructed deep learning model is used in areas outside the training data set, it still relies on high-quality manually labeled data for fine-tuning or even re-training, otherwise its actual accuracy still does not meet the requirements. Therefore, scholars have also begun to focus on research directions such as small sample learning and transfer learning. Zhang et al. used the Massachusetts aerial image building data set for training to address the problem of limited generalization ability of deep convolutional neural network models, and combined a small number of target area building image data samples to fine-tune the model parameters and extract buildings. Xia et al. used a semi-supervised method with a small number of labeled samples and a large number of unlabeled images for joint training, trained an expert-level semantic edge segmentation model based on labeled samples, guided the unlabeled images to automatically generate pseudo labels, and used the automatically generated label set and manually labeled samples together to fine-tune the semantic edge model. Li et al. generated pseudo masks from image-level labels, applied fully connected conditional random fields to improve the accuracy of pseudo masks, and then trained segmentation networks with pseudo masks to apply weakly supervised semantic segmentation to high-resolution remote sensing images with image-level labels. Na et al. combined domain adaptive transfer and adversarial attack methods to transfer the input image distribution to the target image distribution, and at the same time converted the image into adversarial examples for the target network to overcome the performance degradation caused by the domain gap. Although there are many model transfer methods, there are still problems such as the need for a certain number of high-quality target area samples, excessive computational difficulty, and the credibility of model transferability, that is, low accuracy. Summary of the invention
[0005] The main problem to be solved by the present invention is to provide a remote sensing image building migration extraction method based on anomaly detection, which can extract buildings from high-resolution remote sensing images and ensure the overall accuracy of building extraction.
[0006] In order to solve the above technical problems, the present invention provides a remote sensing image building migration extraction method based on anomaly detection, comprising the following steps:
[0007] S1. Using an open data set to train a deep semantic segmentation network model, inputting the remote sensing image into the trained deep semantic segmentation network model to obtain the output result of the remote sensing image;
[0008] S2, extracting the building prediction results and the road prediction results in the output results, and preprocessing the building prediction results to obtain the building layer, the non-building layer and the to-be-layered layer;
[0009] S3, performing anomaly detection on the building layer, removing abnormal objects to obtain building samples, and taking the non-building layer as non-building samples;
[0010] S4, erasing the road prediction result from the to-be-stratified object, combining the building samples and non-building samples to reclassify the to-be-stratified object, and obtaining a reclassified building classification result;
[0011] S5. Superimpose the building classification result with the building sample to obtain a building migration extraction result.
[0012] Preferably, in S2, the preprocessing performs layered processing on the building prediction results according to the prediction probability threshold.
[0013] Further preferably, the building layer, the non-building layer and the sample objects in the to-be-layered layer are processed by image segmentation, and the algorithm of the image segmentation processing is a classification network evolution algorithm.
[0014] Preferably, the building prediction result is determined by a deep semantic segmentation network model trained by an input remote sensing image and an open data set. By corresponding the output result to the pixel coordinates in the input remote sensing image, the probability that the pixels corresponding to the output result and the input remote sensing image belong to a building is obtained. The value range of each pixel is [0,1], indicating the predicted probability that each pixel is a building.
[0015] Further preferably, the building layer is a pixel area with a probability greater than 0.6, the non-building layer is a pixel area with a probability less than 0.2, and the to-be-layered layer is a pixel area with a probability between 0.2 and 0.6.
[0016] Preferably, the anomaly detection in S3 includes the following sub-steps:
[0017] S31, calculating the spectrum and texture feature values of objects in the building layer;
[0018] S32. Using an anomaly detection algorithm to remove abnormal objects in the building layer according to the calculated characteristic values, to obtain building samples.
[0019] Further preferably, the anomaly elimination adopts an isolation forest algorithm to eliminate abnormal objects in the building sample. The isolation forest algorithm normalizes the path length from the leaf node to the root node in the isolation tree constructed by the algorithm through the following formula. The specific formula is as follows:
[0020]
[0021] H(i)=ln(i)+0.577
[0022] Where: p(x,n) is the anomaly score of node x, and the anomaly score p(x,n) is a monotonically decreasing function of l(x), and the value range of p is [0, 1]; E(l(x)) is the expected value of l(x) of node x in multiple isolated trees; C(n) is the average path length after a given number of samples n, which is used to standardize the path length of sample x; H(i) is the harmonic number; 0.577 is the Euler constant.
[0023] Preferably, the reclassification uses a supervised classifier to reclassify the objects in the to-be-classified layer.
[0024] Further preferably, the reclassification in S4 comprises the following sub-steps:
[0025] S41, removing the road prediction results in the to-be-layered layer;
[0026] S42, adding the abnormal objects removed from the building layer to the layer to be layered;
[0027] S43, extracting feature values from building samples and non-building samples and inputting the extracted feature values into a supervised classifier as a reference object;
[0028] S44, inputting the objects to be stratified into a supervised classifier for reclassification, and obtaining the building classification result.
[0029] Preferably, in S43, the feature values include the color and texture features of the building and the index features related to the building, wherein the calculation formula for reclassifying the objects in the to-be-layered layer according to the color features is as follows:
[0030]
[0031]
[0032] where fij is the color value of the i-th color component of the j-th image pixel; N is the total number of pixels in the image; μ i , σ i , γ i (i=1, 2, 3) represents the mean, standard deviation and skewness of each channel of the image respectively.
[0033] Through the above technical scheme, the present invention provides a remote sensing image building migration extraction method based on anomaly detection, an open data set is trained through a deep semantic segmentation network model to obtain a basic building extraction model, and then the remote sensing image is input into the model for building extraction, and an anomaly detection step is added during the extraction process, and the output prediction results are layered, and then the layered building remote sensing images are anomaly eliminated to improve the sample accuracy of the building. In addition, it is necessary to superimpose the object to be classified with the abnormal object eliminated in the building layer, and use the building sample and the non-building sample as reference objects, and improve the extraction accuracy of the building by reclassifying the sample to be classified, and then merge the building object after anomaly detection and the building object obtained by reclassification, so as to obtain the final high-precision extraction result of the building.
[0034] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic diagram of the extraction method of the present invention;
[0036] Figure 2 A schematic diagram of a specific processing process of the extraction method of the present invention;
[0037] Figure 3 The building prediction results of the training data and the experimental data of the present invention;
[0038] Figure 4 The building prediction results and hierarchical display of the present invention;
[0039] Figure 5 This is the building extraction result of Experiment 1 of the present invention. DETAILED DESCRIPTION
[0040] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.
[0041] An embodiment of the remote sensing image building migration extraction method based on anomaly detection of the present invention is as follows: Figure 1 to Figure 2 As shown, it mainly includes the following steps:
[0042] S1. Using an open data set to train a deep semantic segmentation network model, inputting the remote sensing image into the trained deep semantic segmentation network model to obtain the output result of the remote sensing image;
[0043] S2, extracting the building prediction results and the road prediction results in the output results, and preprocessing the building prediction results to obtain the building layer, the non-building layer and the to-be-layered layer;
[0044] S3, performing anomaly detection on the building layer, removing abnormal objects to obtain building samples, and taking the non-building layer as non-building samples;
[0045] S4, erasing the road prediction result from the to-be-stratified object, combining the building samples and non-building samples to reclassify the to-be-stratified object, and obtaining a reclassified building classification result;
[0046] S5. Superimpose the building classification result with the building sample to obtain a building migration extraction result.
[0047] Specifically, in S1, semantic segmentation is to assign a corresponding object category identifier to each image pixel to achieve pixel-level classification. The introduction of the fully convolutional neural network (FCN) realized end-to-end image semantic segmentation for the first time. Therefore, in the field of semantic segmentation of remote sensing objects, the current mainstream method is to use image data containing a large number of label maps to supervise the training of the fully convolutional neural network (FCN). The fully convolutional neural network (FCN) adopts the structure of the convolutional neural network (CNN). As the number of network layers increases, the low-dimensional features of shallow learning can be approximately understood as low- and medium-level features such as color, texture, and geometric shape, and the high-dimensional features of deep learning can be understood as abstract semantic features. The deep semantic segmentation network model used in this paper is the UNet network model. This network model draws on the idea of FCN to fuse shallow features and deep features, and introduces the idea of encoding and decoding to form a deep network model. In this deep network model, it includes an encoding structure and a decoding structure. The encoding structure includes four downsampling layers, and the input image is reduced in dimension through the four downsampling layers, thereby reducing the number of parameters that the network model needs to learn, preventing overfitting, and improving the processing efficiency of the network model; the decoding structure can restore the encoded result to the input image size through the above sampling operation.
[0048] When extracting buildings, the above-mentioned UNet deep semantic segmentation network model is first trained using an open data set to obtain a trained deep semantic segmentation network model, and the remote sensing image is input into the trained deep semantic segmentation network model to obtain an output result. The last layer of the UNet deep semantic segmentation network model used in the present invention uses a 1×1 convolutional layer and adds a sigmoid activation function to process the output result to ensure that the image of the output result is consistent with the size of the input image.
[0049] Specifically, in S2, the building prediction results are pre-processed and layered according to the prediction probability threshold. After layering, the building prediction results are divided into building layer, non-building layer and to-be-layered layer, and the building layer, non-building layer and to-be-layered layer are subjected to image segmentation processing to form objects. The algorithm for image segmentation processing is a classification network evolution algorithm, wherein the layering basis of the building layer, non-building layer and to-be-layered layer is determined by the probability that the pixel in the output image and the corresponding pixel in the input image belong to the building. In some methods of extracting land features using deep learning networks, the prediction probability of the foreground and background or the generation of a unique hot code is used to judge. Specifically, when the foreground prediction probability value of a pixel is greater than or equal to 0.5, it is determined that the pixel belongs to the foreground target element, and when it is less than 0.5, it belongs to the background, thereby distinguishing and classifying the two to finally obtain the extraction result. In the present invention, Inria Aerial Image Labeling Dataset is used as training data, UNet is selected as the deep semantic segmentation network model, and the experimental objects are as follows: Figure 3 As shown, Figure 3 -a is the test data from the training data set, Figure 3 -d is a remote sensing image from a certain area in Shenzhen, with a spatial resolution of 0.5m. By inputting the two images into the UNet semantic segmentation network model, the prediction results of the two images are obtained respectively. Figure 3 -b and Figure 3 -e, then Figure 3 -b and Figure 3 -e Perform layer processing respectively to obtain the corresponding building extraction results Figure 3 -c and Figure 3 -f, the specific processing flow of layering the prediction results is as follows Figure 4 As shown, Figure 4 -a is the original image, Figure 4 -b is the prediction result, Figure 4-c is the stratification result. When the building probability value in the prediction result is less than 0.2, there is almost no building object in the prediction result. Under the condition that the experimental data and the training data are quite different, the part with a probability value greater than 0.5 is regarded as a building, but there may be a certain number of non-buildings therein. Therefore, in the present invention, 0.2 and 0.6 are used as thresholds, and the pixel area with a probability greater than 0.6 is divided into a building layer; the pixel area with a probability less than 0.2 is divided into a non-building layer; and the pixel area between 0.2 and 0.6 is divided into the layer to be stratified.
[0050] Specifically, anomaly detection in S3 includes the following steps:
[0051] S31, calculating the spectrum and texture feature values of objects in the building layer;
[0052] S32. Using an anomaly detection algorithm to remove abnormal objects in the building layer according to the calculated characteristic values, to obtain building samples.
[0053] In S32, the isolation forest algorithm is used to remove the non-building parts of the building layer. The isolation forest algorithm uses a random hyperplane to cut the data space to form two subspaces, and then cuts the subspaces until there is only one data node in each subspace. The isolated tree formed in this way contains only one data node in each leaf node of each isolated tree. The data density at the outlier point is very low, so it will soon stop in a subspace. The length of the path l(x) from the leaf node x to the root node is used to determine whether the data x is an outlier. The isolation forest consists of multiple isolated trees. For a data set containing n data points, the maximum height of the constructed isolated tree is n-1, and the minimum is log(n). The maximum possible height of the path l(x) increases linearly with n, and the average height increases with log(n). According to the similarity between the isolation tree and the binary search tree, the path length l(x) from the leaf node to the root node is normalized using the following formula. The specific formula is as follows:
[0054]
[0055] H(i)=ln(i)+0.577
[0056] Where: p(x,n) is the anomaly score of node x, and the anomaly score p(x,n) is a monotonically decreasing function of l(x), and the value range of p is [0, 1]; E(l(x)) is the expected value of l(x) of node x in multiple isolated trees; C(n) is the average path length after a given number of samples n, which is used to standardize the path length of sample x; H(i) is the harmonic number; 0.577 is the Euler constant.
[0057] In S4, the reclassification uses a supervised classifier to reclassify the objects in the target layer, and the supervised classifier can be a random forest classifier. Specifically, the reclassification includes the following steps:
[0058] S41, removing the road prediction results in the to-be-layered layer;
[0059] S42, adding the abnormal objects removed from the building layer to the layer to be layered;
[0060] S43, extracting feature values from building samples and non-building samples and inputting the extracted feature values into a supervised classifier as a reference object;
[0061] S44, inputting the objects to be stratified into a supervised classifier for reclassification, and obtaining the building classification result.
[0062] By inputting the building samples and non-building samples as a whole as reference objects into the to-be-stratified layer and comparing them with the feature information of the objects in the to-be-stratified layer, the road prediction results in the to-be-stratified layer are first eliminated, which can effectively improve the extraction efficiency of buildings in the to-be-stratified samples. In addition, the building samples and non-building samples are used as reference objects to reclassify the buildings and non-buildings in the to-be-stratified samples, which ensures the extraction accuracy of buildings in the to-be-stratified samples. Since each land feature corresponds to a unique feature in the image, its category can be classified through the feature description of the segmented object, and different extraction schemes can be established based on various image features of buildings in high-resolution remote sensing images (color, shape, texture, spatial relationship and other index features related to buildings).
[0063] In the present invention, the image object features are mainly expressed by color and texture. Color is one of the most important features of the image. The present invention uses the mean, standard deviation and skewness in the color moment to express the color features of the object. Therefore, the calculation formula for reclassifying the sample objects in the layer according to the color features is as follows:
[0064]
[0065]
[0066] where f ij is the color value of the i-th color component of the j-th image pixel; N is the total number of pixels in the image; μ i , σ i , γ i (i=1, 2, 3) represents the mean, standard deviation and skewness of each channel of the image respectively.
[0067] Similarly, texture is one of the main features of the human visual system in identifying and interpreting targets. The gray level co-occurrence matrix calculation method adopted in the present invention can count whether the gray value in a certain direction changes regularly. The expression of the gray level co-occurrence matrix is as follows:
[0068]
[0069] Among them, p(i,j) represents the gray value. In addition, in addition to traditional features, index features for specific objects can also be introduced into the method of extracting ground features. The shadow part in the segmented image patch can be removed by the morphological shadow index, so as to improve the accuracy of building extraction. In addition, since the present invention combines the index feature and the shadow feature of the building, the distinction between the building category and other categories is further enhanced, making the extraction of the building simpler and more effective.
[0070] An embodiment of the remote sensing image building migration extraction method based on anomaly detection of the present invention is as follows: Figure 5 As shown, in the experiment of the present invention, three experimental areas are designed using three semantic segmentation networks, UNet, PSPNet, and LinkNet, respectively, to verify the applicability of the algorithm in different scenarios. Since some deep semantic segmentation network models are highly sensitive to the scale and spatial resolution of the experimental object, when predicting an over-high-resolution image, the problem of only extracting the edge area of the building may occur. Therefore, the image can be resampled to appropriately reduce the spatial resolution so that the high-resolution image is close to the normal level of the training data set to solve the problem. In addition, when extracting objects to be layered in the prediction results, it is necessary to segment the remote sensing image so that the layers to be layered can be reclassified and the non-building layer and the building layer after the abnormality is eliminated can be used to automatically select building and non-building samples.
[0071] The present invention mainly adopts the classification network evolution algorithm as the image segmentation algorithm of this paper, in which the smoothness and compactness are set according to the default settings, and the segmentation scale is appropriately selected according to different image conditions. Due to the different segmentation scales, it is easy to cause over-segmentation or under-segmentation. Therefore, in order to make the samples and objects to be classified more accurately reflect the characteristics of the corresponding ground objects themselves, the segmentation scale of the experiment needs to be appropriately reduced. Based on the above reasons, Figure 5 The building extraction accuracy results of the three areas shown in different models are shown in Table 1:
[0072] Table 1. Building extraction accuracy in Experiment 1
[0073]
[0074] It can be seen from Table 1 that compared with the original UNet, PSPNet and LinkNet networks, the building migration extraction results of the present invention have greatly improved accuracy in recall, precision and F1. Among them, the F1 index of building extraction in the three regions has increased by an average of 12.11%. Therefore, the method of the present invention can extract buildings more accurately.
[0075] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited thereto. Within the technical concept of the present invention, the technical solution of the present invention can be subjected to a variety of simple modifications, including the combination of various specific technical features in any appropriate manner. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations. However, these simple modifications and combinations should also be regarded as the contents disclosed by the present invention and belong to the protection scope of the present invention.
Claims
1. A remote sensing image building migration extraction method based on anomaly detection, characterized in that: The following steps are involved: S1. Using an open data set to train a deep semantic segmentation network model, inputting the remote sensing image into the trained deep semantic segmentation network model to obtain the output result of the remote sensing image; S2, extracting the building prediction results and the road prediction results in the output results, and preprocessing the building prediction results to obtain the building layer, the non-building layer and the to-be-layered layer; S3, performing anomaly detection on the building layer, removing abnormal objects to obtain building samples, and taking the non-building layer as non-building samples; S4, erasing the road prediction result from the to-be-stratified object, combining the building samples and non-building samples to reclassify the to-be-stratified object, and obtaining a reclassified building classification result; S5, superimposing the building classification result with the building sample to obtain a building migration extraction result; The reclassification in S4 includes the following sub-steps: S41, removing the road prediction results in the to-be-layered layer; S42, adding the abnormal objects removed from the building layer to the layer to be layered; S43, extracting feature values from building samples and non-building samples and inputting the extracted feature values into a supervised classifier as a reference object; S44, inputting the objects to be stratified into a supervised classifier for reclassification, and obtaining the building classification result; In S43, the feature values include the color and texture features of the building and the index features related to the building, wherein the calculation formula for reclassifying the objects in the to-be-layered layer according to the color features is as follows: where f ij is the color value of the i-th color component of the j-th image pixel; N is the total number of pixels in the image; μ i , σ i , γ i (i=1, 2, 3) represent the mean, standard deviation and skewness of each channel of the image respectively.
2. The extraction method according to claim 1, characterized in that In S2, the preprocessing performs layered processing on the building prediction results according to the prediction probability threshold.
3. The extraction method according to claim 1, characterized in that The building layer, the non-building layer and the objects in the to-be-layered layer are processed by image segmentation, and the algorithm of the image segmentation processing is a classification network evolution algorithm.
4. The extraction method according to claim 1, characterized in that The building prediction result is determined by a deep semantic segmentation network model trained with an input remote sensing image and an open data set. By corresponding the output result to the pixel coordinates in the input remote sensing image, the probability that the pixel corresponding to the output result in the input remote sensing image belongs to a building is obtained. The value range of each pixel is [0,1], indicating the predicted probability that each pixel is a building.
5. The extraction method according to claim 4, characterized in that The building layer is a pixel area with a probability greater than 0.6, the non-building layer is a pixel area with a probability less than 0.2, and the to-be-layered layer is a pixel area with a probability between 0.2 and 0.
6.
6. The extraction method according to claim 1, characterized in that Anomaly detection in S3 consists of the following steps: S31, calculating the spectrum and texture feature values of objects in the building layer; S32. Using an anomaly detection algorithm to remove abnormal objects in the building layer according to the calculated characteristic values, to obtain building samples.
7. The extraction method according to claim 6, characterized in that The anomaly elimination adopts the isolation forest algorithm to eliminate the abnormal objects in the building sample. The isolation forest algorithm normalizes the path length from the leaf node to the root node in the isolation tree formed by the algorithm through the following formula. The specific formula is as follows: H(i)=ln(i)+0.577 Where: p(x,n) is the anomaly score of node x, and the anomaly score p(x,n) is a monotonically decreasing function of l(x), and the value range of p is [0, 1]; E(l(x)) is the expected value of l(x) of node x in multiple isolated trees; C(n) is the average path length after a given number of samples n, which is used to standardize the path length of sample x; H(i) is the harmonic number; 0.577 is the Euler constant.
8. The extraction method according to claim 1, characterized in that The reclassification uses a supervised classifier to reclassify the objects in the target layer.
