Building attribute extraction method based on high-resolution hyperspectral remote sensing image

Abstract

The invention discloses a building attribute extraction method based on a high-resolution hyperspectral remote sensing image, and relates to the technical field of remote sensing building attribute extraction, and the method comprises the steps: obtaining large-scene high-resolution satellite image data of a town region to be measured and hyperspectral remote sensing image data in the same scene; fusing the two images to obtain a high-resolution hyperspectral fused image, inputting the high-resolution hyperspectral fused image into a Unet deep learning network model, and cutting to obtain a plurality of single building sub-images; and inputting the single building sub-images into an overall edge attention converter to obtain a roof surface edge structure line, matching the spectral data of roof surface pixels with the spectrum database of the building roof material in the to-be-detected town area to obtain building attributes, and synthesizing all the sub-images to obtain the building attributes of the to-be-detected town. According to the invention, building roof extraction, material identification and purpose determination can be conveniently and rapidly realized.

Description

Technical Field

[0001] The invention relates to a building attribute extraction method based on high-resolution hyperspectral remote sensing images, and belongs to the technical field of remote sensing building attribute extraction. Background Art

[0002] With the continuous advancement of urbanization, the workload of investigating and monitoring buildings has increased dramatically. Traditional building attribute extraction methods require a lot of manpower and material resources to conduct field investigations on the building's outline, material, use and other attributes. Remote sensing technology can initially realize the automatic extraction of building roof structures from high-resolution aerial images, but other important attributes such as building materials and uses often require traditional manual investigations to determine, which increases a lot of manual workload. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a building attribute extraction method based on high-resolution hyperspectral remote sensing images, which can conveniently and quickly realize building roof extraction, material identification and purpose determination.

[0004] To achieve the above object, the present invention is implemented by adopting the following technical solutions: The present invention provides a method for extracting building attributes based on high-resolution hyperspectral remote sensing images, comprising: Obtain large-scale high-resolution satellite image data of the urban area to be tested and hyperspectral remote sensing image data of the same scene; The high-resolution and high-spectral fused images are obtained by fusing large-scene high-resolution satellite image data and the high-spectral remote sensing image data of the same scene. The high-resolution hyperspectral fusion image is input into the pre-trained Unet deep learning network model to detect and extract buildings, and multiple sub-images of individual buildings are output after cropping. The single building sub-image is input into the overall edge attention converter to extract the outer contour of the building roof, and the edge structure line of the roof surface is output; According to the pixel position coordinates of the edge structure line of the roof surface in the sub-image of the single building and the topological connection relationship of the edge structure line, the spectral data of the roof surface pixels are extracted; Match the spectral data of the roof surface pixels with the spectral database of the roof materials of the buildings in the urban area to be tested, identify the roof materials of the buildings, and infer the building attributes based on the roof materials of the buildings; According to the location information of the individual building sub-images, all the individual building sub-images are spliced ​​and fused to obtain the building attributes of the urban area to be tested.

[0005] Furthermore, the spectrum database of roof materials of buildings in the town to be tested is obtained by performing spectrum detection on commonly used roof materials in the town to be tested.

[0006] Furthermore, the image fusion of the large-scene high-resolution satellite image data and the hyperspectral remote sensing image data of the same scene to obtain a high-resolution hyperspectral fused image includes: Convert large-scene high-resolution satellite image data from RGB space to HIS space to obtain large-scene high-resolution satellite image data in HIS space; The intensity information of the hyperspectral remote sensing image data in the same scene is used to replace the HIS intensity of the large-scene high-resolution satellite image data in the HIS space to obtain the replaced high-resolution satellite image data; The replaced hyperspectral remote sensing image data is converted into RGB space to obtain a high-resolution hyperspectral fusion image.

[0007] Furthermore, the Unet deep learning network model is a binary classification Unet network.

[0008] Furthermore, the pre-training method of the Unet deep learning network model includes: S1, select part of the high-resolution hyperspectral fusion image, and perform building attribute annotation on the high-resolution hyperspectral fusion image in QGIS software, wherein the building attribute annotation includes annotating the building roof with a polygon, and annotating the roof material and the house use attribute with a text label, so as to obtain a high-resolution hyperspectral fusion image with building attribute annotation; S2, dividing the high-resolution hyperspectral fusion images with building attribute annotations into a training set and a validation set; S3, using the training set as input and the background and building roof distinguishing images as output, training the Unet deep learning network model until the evaluation index reaches the preset target value, thereby obtaining a trained Unet deep learning network model; S4, using the validation set as input and the background and building roof distinguishing images as output, to verify the Unet deep learning network model. If the evaluation index does not reach the preset target value, S3 to S4 are repeated until the evaluation index reaches the preset target value, thereby obtaining a pre-trained Unet deep learning network model. The evaluation indicators include pixel accuracy and intersection-over-union ratio.

[0009] Furthermore, the size of the single building sub-image is 256×256.

[0010] Furthermore, the overall edge attention converter includes a backbone network and a corner detector, the backbone network is Resnet, the output end of which is connected to the input end of the image feature encoder, the output end of the corner detector is connected to the input end of the position encoder, the output end of the position encoder and the output end of the image feature encoder are connected to the input end of the feature fusion module, the output end of the feature fusion module is connected to the input end of the edge filter, and the output ends of the edge filters are respectively connected to the image perception decoders; The corner point detector is used to obtain a set of candidate corner points from the high-resolution hyperspectral fusion image, wherein each pair of candidate corner points constitutes a candidate edge; The position encoder is used to initialize the candidate edge to obtain the edge position feature of the candidate edge; The feature fusion module includes a connected additive normalization layer and a feedforward network model, which is used to fuse the edge position features of the candidate edge and the image features output by the image encoder, and use the fused candidate edge as a fusion node; The edge filter is used to calculate the confidence score of the fused node, and screen the fused nodes whose confidence score is higher than a preset threshold according to the confidence score; The image perception decoder is used to use the fusion nodes with confidence scores higher than a preset threshold as Transfomer nodes, and combine all Transfomer node outputs to obtain the roof surface edge structure line.

[0011] Furthermore, the spectral data of the roof surface pixels are obtained by extracting pixels of a grid based on polygonal coordinates.

[0012] Furthermore, the spectral data of the roof surface pixels are matched with the spectral database of roof materials of buildings in the urban area to be tested, the roof material of the building is identified, and the building attributes are inferred according to the roof material of the building, including: The spectral data is regarded as a vector in a multidimensional space, and the spectral angle between the spectral data of the roof surface and the spectral data of the known roof material in the spectral database of the roof material of the urban area to be tested is calculated; The known roof material spectrum data with the smallest spectrum angle with the spectrum data vector of the roof surface is the same type of roof material, and the roof material is obtained; The building attributes are obtained by inferring the correlation characteristics between the roof material and the house use; The building attributes include a roof outline, a roof material label, and a house use attribute inference label.

[0013] Furthermore, the calculation expression of the spectral angle is: ; in, represents the spectral data vector of the roof surface, It represents the known spectral data vector in the spectral database of the roof materials of the urban area to be tested, represents the spectral angle.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention utilizes the fact that the roof material of buildings in urban-rural fringe areas has a strong correlation with the purpose of the buildings. Through a deep learning network, the background and buildings in high-resolution hyperspectral images are distinguished. The overall edge attention converter is used to extract the roof outline topological structure. Finally, the spectral angle method is combined for material recognition, and then the building attributes are inferred, which can effectively reduce the manual workload required to determine the building attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of a method for extracting building attributes based on high-resolution hyperspectral remote sensing images in an embodiment of the present invention; Figure 2 Schematic diagram of the structure of the overall edge attention converter of the building attribute extraction method based on high-resolution hyperspectral remote sensing imagery in one embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a method for extracting building attributes based on high-resolution hyperspectral remote sensing images, comprising the following steps: The commonly used roof materials in cities and towns in Hubei Province were sampled, and spectral detection was performed using a spectrometer. A spectral database of roof materials of cities and towns in Hubei Province was created in EMVI5.3.

[0018] Download multiple L1A-level multispectral true-color high-resolution satellite images taken by Wuhan-1 satellite in Hubei Province on June 15, use the full-band airborne hyperspectral imaging system to perform spectral scanning on the same scene, and obtain the hyperspectral remote sensing image data of the same scene.

[0019] The HIS fusion algorithm is used to fuse multispectral true color high-resolution satellite images with hyperspectral remote sensing images of the same scene, including: The multispectral true color high-resolution satellite image is converted from RGB space to HIS space, and the HIS intensity of the multispectral true color high-resolution satellite image is replaced by the intensity information of the hyperspectral remote sensing image. Finally, the replaced image data is converted back to RGB space to obtain a high-resolution hyperspectral fusion image.

[0020] Construct a Unet deep learning network. In this embodiment, the Unet deep learning network is a binary classification Unet network, which is a classic image segmentation network. The image data is downsampled four times and then upsampled four times for output. There is a jump connection between each upsampling layer and downsampling layer.

[0021] Pre-train the Unet deep learning network. The pre-training methods include: S1. Select some high-resolution hyperspectral fusion images, and annotate the high-resolution hyperspectral fusion images with building attributes in QGIS software. The building attribute annotation includes annotating the building roof with polygons, and annotating the roof material and house use attributes with text labels, so as to obtain a high-resolution hyperspectral fusion image with building attribute annotations.

[0022] S2. Divide the high-resolution hyperspectral fusion images with building attribute annotations into a training set and a validation set.

[0023] S3. Take the training set as input and the background and building roof distinguishing images as output, and train the Unet deep learning network model until the evaluation index reaches the preset target value. In this embodiment, the evaluation indexes are pixel accuracy and intersection-over-union ratio to obtain a trained Unet deep learning network model.

[0024] S4. Use the validation set as input and the background and building roof distinction images as output to verify the Unet deep learning network model. If the evaluation index does not reach the preset target value, repeat S3~S4 until the evaluation index reaches the preset target value to obtain the pre-trained Unet deep learning network model.

[0025] The high-resolution hyperspectral fusion image is used as input, and the pre-trained Unet deep learning network model is used to detect and extract it, separate the background from the building roof, and then obtain the pixel position information of the building roof based on the semantic segmentation result. Finally, it is cropped into multiple single building sub-images of a fixed size of 256×256. In this embodiment, the size of the single building sub-image is 256×256.

[0026] The overall edge attention converter is used to extract the outer contour of the building roof for each single building sub-image. Specifically: like Figure 2As shown, the overall edge attention converter is a neural network based on the attention mechanism, which uses Resnet50 as the backbone network and also includes a corner detector. The output end of the backbone network is connected to the input end of the image feature encoder, the output end of the corner detector is connected to the input end of the position encoder, the output end of the position encoder and the output end of the image feature encoder are connected to the input end of the feature fusion module, the output end of the feature fusion module is connected to the input end of the edge filter, and the output ends of the edge filters are respectively connected to the image perception decoders.

[0027] In this embodiment, Harris is selected as the corner detector, the feature fusion module is composed of a standard additive normalization layer and a feed-forward network (FFN), and the edge filter is composed of two layers of MLP and a sigmoid function.

[0028] The single building sub-image obtains a set of candidate corner points through the corner point detector. Each pair of candidate corner points constitutes a candidate edge. Then, the position encoder performs 256-dimensional trigonometric function position encoding initialization to obtain the edge position features of the candidate edge.

[0029] The backbone network and the image feature encoder form a three-level image feature pyramid, output image feature information, fuse the edge position features of the candidate edges with the image feature information in the fusion feature module, and use the candidate edges after feature fusion as fusion nodes.

[0030] The fused nodes filter out bad candidates by calculating confidence scores through edge filters, and the remaining fused nodes proceed to the decoder part.

[0031] It should be noted that the overall edge attention converter includes two decoders in the training stage, namely the image perception decoder and the pure geometry decoder. The image perception decoder is used to take the fusion nodes with confidence scores higher than the preset threshold as Transfomer nodes, and combine the outputs of all Transfomer nodes to obtain the roof edge structure line. The pure geometry decoder is used to disable image feature information and use pure geometric position information for learning, but the overall edge attention converter only needs the image perception decoder during the use stage.

[0032] According to the pixel position coordinates of the roof edge structure line in the single building sub-image and the topological connection relationship between the candidate edges, the spectral data of the roof surface pixels can be directly extracted based on the pixels of the polygonal coordinate extraction grid.

[0033] The spectral data of the roof surface pixels are matched with the spectral database of the roof materials of the buildings in the urban area to be tested, the roof materials of the buildings are identified, and the building attributes are inferred based on the roof materials of the buildings, including: The spectral data is regarded as a vector in multidimensional space, and the spectral angle between the spectral data of the roof surface and the known spectral data in the spectral database of the roof material of the urban area to be tested is calculated. The calculation expression is: ; in, represents the spectral data vector of the roof surface, It represents the known spectral data vector in the spectral database of the roof materials of the urban area to be tested, represents the spectral angle.

[0034] Since the calculated spectral angle represents the similarity, the smaller the similarity, the more similar the two are. Therefore, it is considered that the known roof material spectral data with the smallest spectral angle with the spectral data vector of the roof surface is the same type of roof material, that is, the roof material corresponding to the known spectral data is the roof material of the building to be tested in the sub-image of the single building.

[0035] Then, based on the correlation characteristics between the roof material and the house use, the building attributes are finally derived. The building attributes include the roof outline, the roof material label and the house use attribute inference label.

[0036] According to the location information of the individual building sub-images, all the individual building sub-images are spliced ​​and fused to obtain the building attributes in the urban areas within Hubei Province.

[0037] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for extracting building attributes based on high-resolution hyperspectral remote sensing images, characterized in that: include: Obtain large-scale high-resolution satellite image data of the urban area to be tested and hyperspectral remote sensing image data of the same scene; The high-resolution and high-spectral fused images are obtained by fusing large-scene high-resolution satellite image data and the high-spectral remote sensing image data of the same scene. The high-resolution hyperspectral fusion image is input into the pre-trained Unet deep learning network model to detect and extract buildings, and multiple sub-images of individual buildings are output after cropping. The single building sub-image is input into the overall edge attention converter to extract the outer contour of the building roof, and the edge structure line of the roof surface is output; According to the pixel position coordinates of the edge structure line of the roof surface in the sub-image of the single building and the topological connection relationship of the edge structure line, the spectral data of the roof surface pixels are extracted; Match the spectral data of the roof surface pixels with the spectral database of the roof materials of the buildings in the urban area to be tested, identify the roof materials of the buildings, and infer the building attributes based on the roof materials of the buildings; According to the location information of the individual building sub-images, all the individual building sub-images are spliced ​​and fused to obtain the building attributes of the urban area to be tested.

2. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1 is characterized in that: The spectrum database of roof materials of buildings in the town to be tested is obtained by performing spectrum detection on common roof materials in the town to be tested.

3. The building attribute extraction method based on high-resolution hyperspectral remote sensing imagery according to claim 1 is characterized in that: The method of fusing large-scene high-resolution satellite image data with hyperspectral remote sensing image data of the same scene to obtain a high-resolution hyperspectral fused image includes: Convert large-scene high-resolution satellite image data from RGB space to HIS space to obtain large-scene high-resolution satellite image data in HIS space; The intensity information of the hyperspectral remote sensing image data in the same scene is used to replace the HIS intensity of the large-scene high-resolution satellite image data in the HIS space to obtain the replaced high-resolution satellite image data; The replaced hyperspectral remote sensing image data is converted into RGB space to obtain a high-resolution hyperspectral fusion image.

4. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that: The Unet deep learning network model is a binary classification Unet network.

5. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 4 is characterized in that: The pre-training method of the Unet deep learning network model includes: S1, select part of the high-resolution hyperspectral fusion image, and perform building attribute annotation on the high-resolution hyperspectral fusion image in QGIS software, wherein the building attribute annotation includes annotating the building roof with a polygon, and annotating the roof material and the house use attribute with a text label, so as to obtain a high-resolution hyperspectral fusion image with building attribute annotation; S2, dividing the high-resolution hyperspectral fusion images with building attribute annotations into a training set and a validation set; S3, using the training set as input and the background and building roof distinguishing images as output, training the Unet deep learning network model until the evaluation index reaches the preset target value, thereby obtaining a trained Unet deep learning network model; S4, using the validation set as input and the background and building roof distinguishing images as output, to verify the Unet deep learning network model. If the evaluation index does not reach the preset target value, S3 to S4 are repeated until the evaluation index reaches the preset target value, thereby obtaining a pre-trained Unet deep learning network model. The evaluation indicators include pixel accuracy and intersection-over-union ratio.

6. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that: The size of the single building sub-image is 256×256.

7. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that: The overall edge attention converter includes a backbone network and a corner detector, the backbone network is Resnet, the output end of which is connected to the input end of the image feature encoder, the output end of the corner detector is connected to the input end of the position encoder, the output end of the position encoder and the output end of the image feature encoder are connected to the input end of the feature fusion module, the output end of the feature fusion module is connected to the input end of the edge filter, and the output ends of the edge filters are respectively connected to the image perception decoders; The corner point detector is used to obtain a set of candidate corner points from the high-resolution hyperspectral fusion image, wherein each pair of candidate corner points constitutes a candidate edge; The position encoder is used to initialize the candidate edge to obtain the edge position feature of the candidate edge; The feature fusion module includes a connected additive normalization layer and a feedforward network model, which is used to fuse the edge position features of the candidate edge and the image features output by the image encoder, and use the fused candidate edge as a fusion node; The edge filter is used to calculate the confidence score of the fused node, and screen the fused nodes whose confidence score is higher than a preset threshold according to the confidence score; The image perception decoder is used to use the fusion nodes with confidence scores higher than a preset threshold as Transfomer nodes, and combine the outputs of all Transfomer nodes to obtain the roof surface edge structure line.

8. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that: The spectral data of the roof surface pixels are obtained by extracting the pixels of the grid based on the polygonal coordinates.

9. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that: The spectral data of the roof surface pixels are matched with the spectral database of the roof materials of the buildings in the urban area to be tested, the roof materials of the buildings are identified, and the building attributes are inferred according to the roof materials of the buildings, including: The spectral data is regarded as a vector in a multidimensional space, and the spectral angle between the spectral data of the roof surface and the spectral data of the known roof material in the spectral database of the roof material of the urban area to be tested is calculated; The known roof material spectrum data with the smallest spectrum angle with the spectrum data vector of the roof surface is the same type of roof material, and the roof material is obtained; The building attributes are obtained by inferring the correlation characteristics between the roof material and the house use; The building attributes include a roof outline, a roof material label, and a house use attribute inference label.

10. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 9, characterized in that: The calculation expression of the spectral angle is: ； in, represents the spectral data vector of the roof surface, It represents the known spectral data vector in the spectral database of the roof materials of the urban area to be tested, Represents the spectral angle.

Images