An optimization method for building extraction based on multi-temporal remote sensing images

Abstract

The present invention discloses a building extraction optimization method based on multi-temporal remote sensing images, and belongs to the field of satellite remote sensing image processing and application technology. First, the multi-temporal remote sensing images are subjected to an initial binary classification with an optimized precision rate to realize the binary classification process of buildings and non-buildings, and texture, geometry, and area features are used to reduce misclassification in the binary classification process; secondly, the initial binary extraction results of multiple phases are subjected to an object intersection and union operation, and then the intersection results are subjected to a union operation to traverse all phase extraction results; finally, the neighborhood relative boundary index is used to optimize the non-building class, and the hole removal and edge filling in the building class are reduced. It can be seen from the experimental results of three-phase high-resolution remote sensing images that the present invention can improve the recognition rate of buildings, reduce the noise interference of non-buildings, have better extraction results, and have a significant improvement in extraction accuracy compared to single-body recognition.

Description

Technical Field

[0001] The invention belongs to the technical field of satellite remote sensing image processing and application, and in particular relates to a building extraction optimization method based on multi-temporal remote sensing images. Background Art

[0002] It is of great significance to use remote sensing images for automatic and semi-automatic building extraction. On the one hand, in the face of the massive increase in remote sensing image update data, it can more quickly and efficiently update the building categories, thereby realizing the needs of digital earth and smart cities for building category visualization and statistics. On the other hand, it can reduce the traditional work mode of manual visual interpretation and reduce manpower and labor costs.

[0003] However, there are still many difficulties in building extraction using existing methods. One of the most significant difficulties is the occlusion of shadows from trees and tall buildings, which makes the results obtained using the spectral difference extraction of building roofs incomplete and easily confused with surface shadows. Among the existing methods, traditional supervised classification and shadow migration algorithms are more based on single images to improve accuracy. Without introducing more data sources, the upper limit of accuracy for single image extraction is limited. Therefore, how to improve the upper limit of building extraction has become a difficult point in research. Summary of the invention

[0004] In view of the problems existing in the prior art, the technical problem to be solved by the present invention is to provide an optimization method for building extraction based on multi-temporal images. The method is based on the optimization of three steps: "initial binary classification of buildings with optimized precision", "intersection and union operation of multi-temporal objects", and "hole removal using neighborhood relative boundary index", thereby achieving improved building extraction accuracy.

[0005] In order to solve the above problems, the technical solution adopted by the present invention is as follows:

[0006] An optimization method for building extraction based on multi-temporal images. First, the multi-temporal remote sensing images are subjected to initial binary classification with optimized precision to achieve binary classification of buildings and non-buildings, and texture, geometry and area features are used to reduce misclassification in the binary classification process; secondly, the initial binary extraction results of multiple temporal phases are subjected to object intersection and union operations, and the intersection operation is performed based on the two-by-two results, and then the intersection results are combined to traverse all the temporal extraction results; finally, the neighborhood relative boundary index is used to optimize the non-building class to reduce the removal of holes and edge filling in the building class.

[0007] The building extraction optimization method based on multi-temporal images and the building initial binary classification method include the following steps:

[0008] Step 1: Image segmentation and supervised classification: Perform multi-resolution segmentation on the input remote sensing image, and perform supervised classification on the segmented objects to obtain two categories: building and non-building;

[0009] Step 2: Using texture features to improve the precision of building classes: The remote sensing image of the building class obtained in step 1 contains noise, which is removed by using the spectral gray-level co-occurrence matrix method; the building classes are spatially merged, and the building classes with GLCM entropy higher than 0.21 are classified back to non-building classes;

[0010] Step 3: Use geometric and area features to improve the precision of building searches: Continue to optimize the results obtained in step 2, classify buildings with an area of ​​less than 100 square meters as non-buildings, and classify irregular buildings with an aspect ratio greater than 10 as non-buildings.

[0011] The building extraction optimization method based on multi-temporal images performs multi-resolution segmentation on the input remote sensing image, the segmentation parameter is set to 10, the weight of each band is set to equal weight, the shape weight is 0.1, and the compactness weight is 0.5.

[0012] In the said building extraction optimization method based on multi-temporal images, in said step 2, the trees with similar colors to the building tops are removed by using the spectral grayscale co-occurrence matrix method, and the texture features formed by the branches of the trees are more significant.

[0013] The building extraction optimization method based on multi-temporal images uses the multi-temporal classification results to perform intersection operations, using the following formula:

[0014] R nT =(R1∩R2)∪(R1∩R3)∪……(R1∩R n )∪(R2∩R3)∪(R2∩R4)∪……(R2∩R n )∪……(R n-1 ∩R n );

[0015] Among them, R nT is the target result of the building in n phases, R1, R2, ... R n These are the initial classification results of phase 1, phase 2, ... phase n.

[0016] The building extraction optimization method based on multi-temporal images uses the relative border to neighbor objects (RBNO) index to calculate non-buildings. The basic calculation formula of RBNO is as follows:

[0017]

[0018] Among them, l public It refers to the length of the common boundary between the object and other objects in the neighborhood. total Refers to the total length of the object's boundaries.

[0019] The building extraction optimization method based on multi-temporal images classifies adjacent non-building categories with an RBNO index greater than or equal to 0.5 as building categories.

[0020] The building extraction optimization method based on multi-temporal images, the multi-temporal remote sensing images are worldview satellite images with a spatial resolution of 0.23 meters.

[0021] Beneficial effects: Compared with the existing technology, the advantages of the present invention include:

[0022] (1) The present invention performs an initial binary classification of multi-phase remote sensing images with optimized precision. This step is to realize the binary classification process of buildings and non-buildings, and use texture, geometry, and area features to reduce misclassification in the binary classification process to prevent subsequent error amplification. Then, the initial binary extraction results of multiple phases are subjected to object intersection and union operations, and intersection operations are performed based on the results of each two, and then the intersection results are combined to achieve traversal of all phase extraction results, prevent extraction errors of a single image, and achieve leak detection and gap filling of multi-phase stable results. Finally, the neighborhood relative boundary index is used to optimize the non-building class, reduce the removal of holes and edge filling in the building class.

[0023] (2) The present invention is suitable for extracting relatively low buildings, which may be blocked by tree canopies and shadows of other buildings, making the target roof extraction result incomplete. The present invention can improve the recognition rate of buildings, reduce noise interference from non-buildings, and has better extraction results, and has a significant improvement in extraction accuracy compared to single-body recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a method flow chart;

[0025] Figure 2 Schematic diagram of objects with different RBNO indices;

[0026] Figure 3 It is a schematic diagram of three-phase data;

[0027] Figure 4 Schematic diagram of the initial binary classification of buildings to optimize the precision rate;

[0028] Figure 5 This is a schematic diagram of the result of the intersection and union operation of multiple temporal objects;

[0029] Figure 6Schematic diagram of hole fusion based on domain relative boundary index. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with specific embodiments.

[0031] Example 1

[0032] A building extraction optimization method based on multi-temporal images. First, the multi-temporal remote sensing images are subjected to an initial binary classification with an optimized precision rate. This step is to realize the binary classification process of buildings and non-buildings, and texture, geometry, and area features are used to reduce misclassification in the binary classification process to prevent subsequent error amplification. Secondly, the initial binary extraction results of multiple phases are subjected to object intersection and union operations, and intersection operations are performed based on the results of each pair, and then the intersection results are combined to traverse all the extraction results of the phases, prevent extraction errors of a single image, and realize the omission and filling of multi-temporal stable results. Finally, the neighborhood relative boundary index is used to optimize the non-building class to reduce the removal of holes and edge filling in the building class. The method flow chart is shown in the figure below. Figure 1 The following is a detailed introduction:

[0033] 1. Initial binary classification of buildings with optimized precision

[0034] The present invention uses remote sensing images of multiple time phases to obtain multiple initial building classification results. If the building contains too much noise and errors, it will bring about an error amplification effect on the subsequent multi-time intersection and operation. Therefore, it is necessary to emphasize the precision rate of the initial classification.

[0035] In the existing research, the representative methods of this step are the novel offset shadow analysis method and the traditional supervised classification method. Among them, the most important thing about the offset shadow analysis method is that there must be more significant building shadows, which need to be automatically identified, and then the basic process of "building shadow-finding offset-building" is formed based on the geometric relationship between the shadow and the related building. However, in multi-temporal images, the probability of shadows cannot be controlled. According to the existing Google Earth images, the number of images with significant shadows is less than 50%, which makes the offset shadow analysis method not universally applicable. In addition, the shadow offset method cannot remove the confusion process of the same spectrum heterogeneous objects on the roof of the building and the impermeable surface of the ground. The traditional supervised classification method has good universality, but if only a single image is used, it is difficult to obtain a satisfactory recognition accuracy based on the "sample-training-classification" method process. Through the preliminary experiments in the early stage, it was found that in the face of complex surface scenes, the extraction accuracy was generally less than 80%. The most important factor was the phenomenon of the same spectrum heterogeneous objects, and the roof of the building and the impermeable surface of the ground (mainly concrete) were easily confused. Therefore, in this step, the present invention uses supervised classification as the basic method, and adds three features (texture, geometry, and area) to optimize the accuracy, thereby meeting the initial classification requirements of the buildings of the present invention.

[0036] The present invention requires that the initial classification precision rate should be emphasized rather than the recall rate, that is, the extraction strategy should meet the accuracy requirements as much as possible and minimize the false recognition rate. It is better to extract fewer buildings than to extract more buildings and bring more noise. Based on this principle, the initial two-level classification of buildings in the present invention is mainly divided into three steps, namely:

[0037] Step 1 is image segmentation and supervised classification. Multi-resolution segmentation is performed on the input remote sensing image. The usual segmentation parameter is set to 10, the weight of each band is set to equal weight, the shape weight is 0.1, and the compactness weight is 0.5. Supervised classification is performed on the segmented objects, and the sampled samples should follow the principle of typicality and comprehensiveness to obtain two categories: building and non-building.

[0038] Step 2 is to use texture features to improve the precision of building classes. The building classes obtained in step 1 contain a lot of noise, which is removed using the spectral gray-level co-occurrence matrix (GLCM). In this step, the main thing to remove is trees with a color similar to the top of the building. The texture features formed by the branches of the trees are more significant. The building classes are spatially merged, and the building classes with GLCM entropy higher than 0.21 are classified back to the non-building class. Here, the gray-level co-occurrence matrix used is the most commonly used feature of texture features.

[0039] Step 3 is to improve the precision of the building class by using geometric and area features. The results obtained in step 2 are further optimized, and buildings with an area of ​​less than 100 square meters are classified as non-buildings (here, it is assumed that the area of ​​a normal building is greater than or equal to 100 square meters, and buildings with an area of ​​less than 100 square meters are eliminated); and particularly long buildings are classified as non-buildings (here, it is assumed that buildings are usually regular and the length-to-width ratio is not greater than 10:1).

[0040] 2. Object Intersect-Union

[0041] According to the initial binary classification results of buildings with optimized precision, the next key step is to use the multi-temporal classification results to perform object intersect-union operations. The purpose of the object intersect-union operation is to check for omissions and fill in the gaps; at the same time, the object of the object intersect-union operation is also required to prevent the image results of a single phase from being misclassified.

[0042] Take the three-phase as an example. The three-phase refers to the remote sensing images and initial classification results of the same location at three times. The intersection and union operation used here is shown in Formula 1. Among them, R 3T It refers to the target result of the three-phase building, R1 refers to the initial classification result of phase 1, R2 refers to the initial classification result of phase 2, and so on.

[0043] R 3T =(R1∩R2)∪(R1∩R3)∪(R2∩R3) Formula 1

[0044] Taking the four phases as an example, the intersection and union operation formula of the four phases is shown in Formula 2. 4T refers to the target result of the three-phase building, R1 refers to the initial classification result of phase 1, and so on.

[0045] R 4T =(R1∩R2)∪(R1∩R3)∪(R1∩R4)∪(R2∩R3)∪(R2∩R4)∪(R3∩R4) Formula 2

[0046] The following is a specific explanation using the three-phase formula 1 as an example. First, the reason for performing the intersection operation of two phases is to obtain a stable result in the initial building binary classification to prevent the initial classification result of a single phase image from being misclassified. Only when the results are obtained after the two phases are intersected can the classification result be considered stable and reliable. Second, each phase needs to be traversed and intersected with another phase. The purpose of this is to traverse all phase results to ensure that the stable extraction results in the initial classification results are retained to the greatest extent. Third, the results of the traversed intersection operation are combined to merge the stable results in the initial classification, thereby achieving the goal of checking and filling in the gaps in the stable building extraction results. Intersection and union operations on the initial classification results of multi-phase images can reduce non-building noise, optimize the integrity of buildings, and improve the recognition accuracy of buildings.

[0047] 3. Hole fusion based on relative boundaries of neighborhoods

[0048] After the intersection and union operation of multi-temporal objects, multi-temporal building targets are obtained. However, due to the existence of holes inside the building targets, it is necessary to perform a hole fusion operation on the extraction results to reduce the holes inside the buildings and the vacancies on the edges.

[0049] The present invention uses the relative boundary index based on the neighborhood for hole fusion. The basic operation is to calculate the non-buildings based on the relative boundary of the neighborhood (Relative Border to Neighbor Objects, RBNO) index, and then classify the adjacent non-building classes greater than or equal to 0.5 as building categories. The basic calculation formula of RBNO can be shown in Formula 3:

[0050]

[0051] Among them, lpublic refers to the length of the common boundary between the object and the neighborhood and the building class, and ltotal refers to the total length of the boundary of the object. In this paper, eCognition software is used for calculation.

[0052] In order to further illustrate the effectiveness of the RBNO index, the objects of different RBNO indexes are described. Figure 2 As shown in the figure, the gray area represents the building category, and the dark boundary represents the object adjacent to the building category, and the corresponding RBNO indexes are 0.3, 0.5, and 0.7 respectively. Figure 2It can be seen that as the RBNO index increases, it has more common boundaries with buildings. Therefore, it can also be simplified as the ratio of the length of the common boundary with the building class to its own length. The higher the value, the longer the common boundary between the object and the building class; the lower the value, the shortest common boundary between the object and the building class. When the RBNO index is equal to 1, it means that the object is completely surrounded by the building; when the RBNO index is 0, it means that there is no building class in the neighborhood of the object. Here, the RBNO index is designed to be equal to 0.5 as the threshold. When it is greater than or equal to 0.5, the non-building class is converted to the building class, thereby reducing the void phenomenon inside the building and reducing the vacancies at the edge of the building.

[0053] 4. Experiments and Results

[0054] In order to verify the effectiveness of the method of the present invention, three-phase Worldview satellite images are used for experiments. The spatial resolution of Worldview satellite images is 0.23 meters, and the three phases are March 3, 2018 ( Figure 3 -a) May 31, 2020 ( Figure 3 -b), August 31, 2020 ( Figure 3 -c), the true color image of the data can be found in Figure 3 The house in this embodiment is relatively low and surrounded by many green trees, which will affect the extraction of the house roof.

[0055] The initial two-classification of buildings with optimized precision is performed. Figure 3 -b, that is, the data of May 31, 2020 is used as an example for the experiment.

[0056] First, the segmentation and supervised classification in step 1. The parameter used for segmentation is 10, the weight of each band is set to the same weight, the shape weight is 0.1, and the compactness weight is 0.5. The result after segmentation is shown in the figure Figure 4 -a. Then, the sample selection is performed on the segmentation result map, and the classification system is building and non-building. Figure 4 -b, the white area is the sample of the building class, and the gray area is the sample of the non-building class. Then, the nearest neighbor supervised classification is performed using the eCognition software. The feature space used is the spectral feature space. The initial classification results are shown in Figure 4 -c.

[0057] In order to achieve precision optimization, the initial binary classification results need to be optimized. The first step is to optimize the precision based on texture features. In this step, the trees with similar colors to the top of the building are mainly removed. The texture features formed by the branches of the trees are more significant. The building class is spatially merged, and the building class with GLCM entropy higher than 0.21 is classified back to the non-building class. The results are shown in Figure 4 -d. From the results, we can see that some trees are misclassified as buildings ( Figure 4 -c black circle) is optimized. Secondly, the precision of the building class is improved by using geometric and area features. The objects that obviously do not conform to the building style are removed and optimized by using the aspect ratio feature in the eCognition software. Buildings with an aspect ratio greater than or equal to 10:1 are classified as non-buildings. The results are shown in Figure 4 -e. Compared to Figure 4 The black circles in -d indicate that these misclassified buildings have been removed. Next, we optimize based on the area of ​​the building. An assumption here is that the area of ​​a single building is generally greater than or equal to 100 square meters, so objects with an area attribute (Area) less than or equal to 100 are set as non-buildings. The results are shown in Figure 4 -f. Figure 4 -The small area of ​​the black circle in e is optimized.

[0058] After obtaining the initial classification results of buildings in each time phase according to the previous step "initial binary classification of buildings with optimized precision", it is necessary to perform multi-time phase object intersection-union operation on the initial classification results. The purpose of intersection is to obtain the results of at least two time phases extracted together, thereby preventing the instability of the extraction results of a single image, and the purpose of union is to fill in the gaps of stable results of multiple time phases and maximize the integrity of stable buildings. The operation process can be seen in Figure 5 shown. Figure 5 The input is the extraction results of different phases, namely R1 (phase March 3, 2018), R2 (phase May 31, 2020), and R3 (phase August 31, 2020). By intersecting them two by two, a total of three results are obtained, namely R1∩R2, R1∩R3, and R2∩R3. The results are shown in Figure 5 As shown on the right. From the result graph, we can see that the most intuitive effect of the intersection graphs of different phases is to reduce the impact of non-buildings and provide a stable extraction result for the common parts of the two result graphs. Then, R1∩R2, R1∩R3, and R2∩R3 were combined, and the result is Figure 5The corresponding graph of (R1∩R2)∪(R1∩R3)∪(R2∩R3) in the figure. It can be seen that the result of the AND operation can fuse multiple objects, playing a role in filling in the gaps. Its most significant effect is that the building blocked by the tree in the lower left corner enhances the integrity of the building.

[0059] After the intersection and union operation of multi-temporal objects, multi-temporal building targets are obtained. However, due to the existence of holes inside the building targets, it is necessary to perform a hole fusion operation on the extraction results to reduce the holes inside the building and the vacancies on the edges. The experimental results are shown in the figure below. Figure 6 shown. Figure 6 -a is the image before post-processing. From the black circle in the image, we can see that there are holes and gaps inside and around the building. We use the RBNO index, i.e. the neighborhood relative boundary index, to merge holes. Non-buildings with values ​​greater than or equal to 0.5 are classified as buildings. The results are shown in the figure. Figure 6 -b. The results show that Figure 6 -The holes in the black circles in a are effectively removed and filled, thus achieving further optimization of building extraction.

[0060] Through the experimental results of three-phase high-resolution remote sensing images, it can be seen that the method of the present invention can improve the recognition rate of buildings, reduce noise interference from non-buildings, have better extraction results, and have a significant improvement in extraction accuracy compared to single-body recognition. In this embodiment, the accuracy of extracting R1 (phase March 3, 2018), R2 (phase May 31, 2020), and R3 (phase August 31, 2020) buildings separately is 73.7%, 82.18%, and 83.76%, respectively, while the accuracy of the buildings extracted by the method of this patent is 91.78%, which is significantly improved compared to single-body recognition efficiency.

Claims

1. A building extraction optimization method based on multi-temporal images, characterized in that: Firstly, the initial binary classification of multi-temporal remote sensing images with optimized precision is performed to achieve binary classification of buildings and non-buildings, and texture, geometry, and area features are used to reduce misclassification in the binary classification process; secondly, the initial binary extraction results of multiple temporal phases are subjected to object intersection and union operations, and the intersection operation is performed based on the two-by-two results, and then the intersection results are combined to traverse all the temporal extraction results; Finally, the neighborhood relative boundary index is used to optimize the non-building class; The initial binary classification method of buildings comprises the following steps: Step 1: Image segmentation and supervised classification: Perform multi-resolution segmentation on the input remote sensing image, and perform supervised classification on the segmented objects to obtain two categories: building and non-building; Step 2: Using texture features to improve the precision of building classes: The remote sensing image of the building class obtained in step 1 contains noise, which is removed by using the spectral gray-level co-occurrence matrix method; the building classes are spatially merged, and the building classes with GLCM entropy higher than 0.21 are classified back to non-building classes; Step 3: Use geometric and area features to improve the precision of building search: Continue to optimize the results obtained in step 2, classify buildings with an area of ​​less than 100 square meters as non-buildings, and classify irregular buildings with an aspect ratio greater than 10 as non-buildings; The multi-temporal classification results are used for intersection and union operations, using the following formula: R nT =(R1∩R2)∪(R1∩R3)∪……(R1∩R n )∪(R2∩R3)∪(R2∩R4)∪……(R2∩R n )∪……(R n-1 ∩R n ); Among them, R nT is the target result of the building in n phases, R1, R2, ... R n They are the initial classification results of phase 1, phase 2, ... phase n respectively; The non-buildings are calculated based on the neighborhood relative boundary index. The calculation formula of RBNO is as follows: Among them, l public It refers to the length of the common boundary between the object and other objects in the neighborhood. total Refers to the total length of the object's boundaries.

2. The building extraction optimization method based on multi-temporal images according to claim 1 is characterized in that: The input remote sensing image is segmented at multiple resolutions, the segmentation parameter is set to 10, the weight of each band is set to equal weight, the shape weight is 0.1, and the compactness weight is 0.

5.

3. The building extraction optimization method based on multi-temporal images according to claim 1 is characterized in that: In step 2, trees with a color similar to that of the building tops are removed, and the texture features formed by the branches of the trees are more obvious.

4. The building extraction optimization method based on multi-temporal images according to claim 1 is characterized in that: Adjacent non-building categories with RBNO index greater than or equal to 0.5 are classified as building categories.

5. The building extraction optimization method based on multi-temporal images according to claim 1 is characterized in that: The multi-temporal remote sensing image is a worldview satellite image with a spatial resolution of 0.23 meters.

Images