A method for extracting building attributes based on high-resolution hyperspectral remote sensing images

Abstract

The present invention discloses a method for extracting building attributes based on high-resolution hyperspectral remote sensing images, which relates to the technical field of remote sensing building attribute extraction. The method includes obtaining large-scene high-resolution satellite image data of a town area to be measured and hyperspectral remote sensing image data of the same scene, fusing the two to obtain a high-resolution hyperspectral fused image, inputting the high-resolution hyperspectral fused image into a Unet deep learning network model, and obtaining multiple single-building sub-images after cropping; inputting the single-building sub-images into an overall edge attention transformer to obtain roof surface edge structure lines, matching the spectral data of the roof surface pixels with the spectral database of building roof materials in the town area to be measured to obtain building attributes, and comprehensively obtaining the building attributes of the town to be measured from all sub-images. The present invention can conveniently and quickly realize roof extraction, material identification and use determination of buildings.

Description

Technical Field

[0001] The present invention relates to a method for extracting building attributes based on high - resolution hyperspectral remote sensing images, belonging to the technical field of remote sensing building attribute extraction. Background Technique

[0002] With the continuous advancement of the urbanization process, the workload of investigating and monitoring buildings has increased dramatically. Traditional methods for extracting building attributes require a large amount of manpower and material resources for on - site investigations of building outlines, materials, uses, and other attributes. Using remote sensing technology can initially achieve the automatic extraction of building roof structures from high - resolution aerial images, but for other important attributes such as building materials and uses, traditional manual investigations are often still required to determine, increasing a lot of manual work. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method for extracting building attributes based on high - resolution hyperspectral remote sensing images, which can conveniently and quickly achieve building roof extraction, material recognition, and use determination.

[0004] To achieve the above - mentioned purpose, the present invention is implemented by the following technical solutions:

[0005] The present invention provides a method for extracting building attributes based on high - resolution hyperspectral remote sensing images, including:

[0006] Obtain large - scene high - resolution satellite image data of the urban area to be measured and hyperspectral remote sensing image data of the same scene;

[0007] Perform image fusion on the large - scene high - resolution satellite image data and the hyperspectral remote sensing image data of the same scene to obtain a high - resolution hyperspectral fusion image;

[0008] Input the high - resolution hyperspectral fusion image into a pre - trained Unet deep - learning network model for building detection and extraction, and after cropping, output multiple single - building sub - images;

[0009] Input the single - building sub - images into an overall edge attention transformer to extract the outer contour of the building roof, and output the roof surface edge structure line;

[0010] According to the pixel position coordinates of the roof surface edge structure line in the single - building sub - images and the topological connection relationship of the edge structure line, extract the spectral data of the roof surface pixels;

[0011] Match the spectral data of the roof surface pixels with the spectral database of building roof materials in the urban area to be measured, identify the building roof materials, and infer the building attributes based on the building roof materials;

[0012] According to the position information of the sub-images of individual buildings, all the sub-images of individual buildings are stitched and fused to obtain the building attributes of the urban area to be measured.

[0013] Furthermore, the spectral database of the roof materials of the buildings in the urban area to be measured is obtained by performing spectral detection on the common roof materials in the urban area to be measured.

[0014] Furthermore, the process of fusing the large-scene high-resolution satellite image data and the hyperspectral remote sensing image data of the same scene to obtain the high-resolution hyperspectral fused image includes:

[0015] Converting the large-scene high-resolution satellite image data from the RGB space to the HIS space to obtain the large-scene high-resolution satellite image data in the HIS space;

[0016] Using the intensity information of the hyperspectral remote sensing image data of the same scene to replace the HIS intensity of the large-scene high-resolution satellite image data in the HIS space to obtain the replaced high-resolution satellite image data;

[0017] Converting the replaced hyperspectral remote sensing image data to the RGB space to obtain the high-resolution hyperspectral fused image.

[0018] Furthermore, the Unet deep learning network model is a binary classification Unet network.

[0019] Furthermore, the pre-training method of the Unet deep learning network model includes:

[0020] S1. Selecting some high-resolution hyperspectral fused images, and performing building attribute annotation on the high-resolution hyperspectral fused images in the QGIS software. The building attribute annotation includes polygon annotation for the building roofs, and text label annotation for the roof materials and the house usage attributes, to obtain the high-resolution hyperspectral fused images with building attribute annotation;

[0021] S2. Dividing the high-resolution hyperspectral fused images with building attribute annotation into a training set and a validation set;

[0022] S3. Using the training set as the input and the image for distinguishing the background and the building roofs as the output, training the Unet deep learning network model until the evaluation index reaches the preset target value to obtain the trained Unet deep learning network model;

[0023] S4. Using the validation set as the input and the image for distinguishing the background and the building roofs as the output, validating the Unet deep learning network model. If the evaluation index does not reach the preset target value, then repeat S3 - S4 until the evaluation index reaches the preset target value to obtain the pre-trained Unet deep learning network model;

[0024] The evaluation indicators include pixel accuracy and intersection-over-union ratio.

[0025] Furthermore, the size of the single building sub-image is 256×256.

[0026] Furthermore, the overall edge attention converter includes a backbone network and a corner detector, the backbone network is Resnet, the output end of which is connected to the input end of the image feature encoder, the output end of the corner detector is connected to the input end of the position encoder, the output end of the position encoder and the output end of the image feature encoder are connected to the input end of the feature fusion module, the output end of the feature fusion module is connected to the input end of the edge filter, and the output ends of the edge filters are respectively connected to the image perception decoder;

[0027] The corner detector is used to obtain a set of candidate corner points from the high-resolution hyperspectral fusion image, wherein each pair of candidate corner points constitutes a candidate edge;

[0028] The position encoder is used to initialize the candidate edge to obtain the edge position feature of the candidate edge;

[0029] The feature fusion module includes a connected additive normalization layer and a feedforward network model, which is used to fuse the edge position features of the candidate edge and the image features output by the image encoder, and use the fused candidate edge as a fusion node;

[0030] The edge filter is used to calculate the confidence scores of the fused nodes, and screen out fused nodes whose confidence scores are higher than a preset threshold according to the confidence scores;

[0031] The image perception decoder is used to use the fusion nodes with confidence scores higher than a preset threshold as Transformer nodes, and combine the outputs of all Transformer nodes to obtain the roof surface edge structure line.

[0032] Furthermore, the spectral data of the roof surface pixels are obtained by extracting pixels of a grid based on polygonal coordinates.

[0033] Furthermore, the spectral data of the roof surface pixels are matched with the spectral database of roof materials of buildings in the urban area to be measured, the roof material of the building is identified, and the building attributes are inferred based on the roof material of the building, including:

[0034] The spectral data is regarded as a vector in a multidimensional space, and the spectral angle between the spectral data of the roof surface and the spectral data of the known roof materials in the spectral database of the roof materials of the urban area to be measured is calculated;

[0035] The known roof material spectrum data with the smallest spectral angle with the spectrum data vector of the roof surface is the same type of roof material, and the roof material is obtained;

[0036] The building attributes are obtained by inferring the correlation characteristics between the roof material and the house use;

[0037] The building attributes include a roof outline, a roof material label, and a house usage attribute inference label.

[0038] Furthermore, the calculation expression of the spectral angle is:

[0039] ;

[0040] in, represents the spectral data vector of the roof surface, represents the known spectral data vector in the spectral database of roof materials of buildings in the urban area to be tested, represents the spectral angle.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention utilizes the fact that the roof material of buildings in urban-rural fringe areas has a strong correlation with the purpose of the buildings. Through a deep learning network, the background and buildings in high-resolution hyperspectral images are distinguished. The overall edge attention converter is used to extract the roof outline topology structure. Finally, the spectral angle method is combined to perform material recognition, and then the building attributes are inferred, which can effectively reduce the manual workload required to determine the building attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of a flow chart of a method for extracting building attributes based on high-resolution hyperspectral remote sensing images in one embodiment of the present invention;

[0044] Figure 2 Schematic diagram of the structure of the overall edge attention converter of the building attribute extraction method based on high-resolution hyperspectral remote sensing imagery in one embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0046] like Figure 1 As shown, an embodiment of the present invention provides a method for extracting building attributes based on high-resolution hyperspectral remote sensing images, comprising the following steps:

[0047] Samples of common roof materials in urban areas within Hubei Province were taken, spectral detection was carried out using a spectrometer, and a spectral database of roof materials of urban buildings within Hubei Province was created in EMVI 5.3.

[0048] Downloaded multiple L1A-level multispectral true-color high-resolution satellite images taken by Wuhan-1 satellite on June 15 within Hubei Province, and used an airborne hyperspectral imaging system with all bands to perform spectral scanning on the same scene to obtain hyperspectral remote sensing image data of the same scene.

[0049] The HIS fusion algorithm was used to fuse the multispectral true-color high-resolution satellite image with the hyperspectral remote sensing image of the same scene, specifically including:

[0050] The multispectral true-color high-resolution satellite image was converted from the RGB space to the HIS space, the intensity information of the hyperspectral remote sensing image was used to replace the HIS intensity of the multispectral true-color high-resolution satellite image, and finally the replaced image data was reconverted to the RGB space to obtain a high-resolution hyperspectral fusion image.

[0051] A Unet deep learning network was constructed. The Unet deep learning network in this embodiment is a binary-class Unet network, which is a classic image segmentation network. The image data undergoes four downsamplings and then four upsamplings for output, and there is a skip connection between each upsampling layer and downsampling layer.

[0052] The Unet deep learning network was pre-trained, and the pre-training method includes:

[0053] S1. Select a part of the high-resolution hyperspectral fusion images, and perform building attribute annotation on the high-resolution hyperspectral fusion images in QGIS software. The building attribute annotation includes polygon annotation for the building roof, and text labels for the roof material and the building usage attribute, to obtain high-resolution hyperspectral fusion images with building attribute annotation.

[0054] S2. The high-resolution hyperspectral fusion images with building attribute annotation were divided into a training set and a validation set.

[0055] S3. Using the training set as the input and the image that distinguishes the background from the building roof as the output, the Unet deep learning network model was trained until the evaluation index reached the preset target value. In this embodiment, the evaluation indexes selected were pixel accuracy and intersection over union, to obtain a trained Unet deep learning network model.

[0056] S4. Use the validation set as the input and the image that differentiates the background from the building roof as the output to verify the Unet deep learning network model. If the evaluation metrics do not reach the preset target value, repeat S3 - S4 until the evaluation metrics reach the preset target value, and obtain the pre-trained Unet deep learning network model.

[0057] Use the high-resolution hyperspectral fused image as the input, and use the pre-trained Unet deep learning network model to detect and extract it, separate the background from the building roof, then obtain the pixel position information of the building roof according to the semantic segmentation result, and finally crop it into multiple single-building sub-images of a fixed size of 256×256. In this embodiment, the size of the single-building sub-image is 256×256.

[0058] Use the global edge attention transformer to extract the outer contour of the building roof for each single-building sub-image. Specifically:

[0059] As Figure 2 shown, the global edge attention transformer is a neural network based on the attention mechanism. It uses Resnet50 as the backbone network and also includes a corner detector. The output end of the backbone network is connected to the input end of the image feature encoder, the output end of the corner detector is connected to the input end of the position encoder, the output end of the position encoder and the output end of the image feature encoder are connected to the input end of the feature fusion module, the output end of the feature fusion module is connected to the input end of the edge filter, and the output end of the edge filter is respectively connected to the image perception decoder.

[0060] In this embodiment, the Harris corner detector is selected. The feature fusion module consists of a standard addition normalization layer and a feed-forward network (FFN). The edge filter consists of two layers of MLP and a sigmoid function.

[0061] The single-building sub-image obtains a set of candidate corners through the corner detector. Each pair of candidate corners forms a candidate edge, and then it is initialized with 256-dimensional trigonometric function position encoding by the position encoder to obtain the edge position feature of the candidate edge.

[0062] The backbone network and the image feature encoder form a three-level image feature pyramid to output image feature information. In the fusion feature module, the edge position feature of the candidate edge and the image feature information are feature-fused, and the candidate edge after feature fusion is used as the fusion node.

[0063] The fusion node calculates the confidence score through the edge filter to filter out bad candidates, and the remaining fusion nodes enter the decoder part.

[0064] It should be noted that the overall edge attention converter includes two decoders in the training stage, namely the image perception decoder and the pure geometry decoder. The image perception decoder is used to take the fusion nodes with confidence scores higher than the preset threshold as Transfomer nodes, and combine the outputs of all Transfomer nodes to obtain the roof surface edge structure line. The pure geometry decoder is used to disable image feature information and use pure geometric position information for learning, but the overall edge attention converter only needs the image perception decoder during the use stage.

[0065] According to the pixel position coordinates of the roof surface edge structure line in the single building sub-image and the topological connection relationship between the candidate edges, the spectral data of the roof surface pixels can be directly extracted based on the pixels of the polygonal coordinate extraction grid.

[0066] The spectral data of the roof surface pixels are matched with the spectral database of roof materials of the buildings in the urban area to be tested, the roof material of the building is identified, and the building attributes are inferred based on the roof material of the building, including:

[0067] The spectral data is regarded as a vector in multidimensional space, and the spectral angle between the spectral data of the roof surface and the known spectral data in the spectral database of the roof materials of the urban area to be measured is calculated. The calculation expression is:

[0068] ;

[0069] in, represents the spectral data vector of the roof surface, represents the known spectral data vector in the spectral database of roof materials of buildings in the urban area to be tested, represents the spectral angle.

[0070] Since the calculated spectral angle represents the similarity, the smaller the similarity, the more similar the two are. Therefore, it is considered that the known roof material spectral data with the smallest spectral angle with the spectral data vector of the roof surface is the same type of roof material, that is, the roof material corresponding to the known spectral data is the roof material of the building to be tested in the sub-image of the single building.

[0071] Then, based on the correlation characteristics between the roof material and the house use, the building attributes are finally derived. The building attributes include the roof outline, the roof material label and the house use attribute inference label.

[0072] According to the location information of the individual building sub-images, all the individual building sub-images are spliced and fused to obtain the building attributes in urban areas within Hubei Province.

[0073] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for extracting building attributes based on high-resolution hyperspectral remote sensing images, characterized in that, Including: Obtain high-resolution satellite image data of the large-scale scene of the urban area to be measured and hyperspectral remote sensing image data of the same scene; Perform image fusion on the high-resolution satellite image data of the large-scale scene and the hyperspectral remote sensing image data of the same scene to obtain a high-resolution hyperspectral fusion image; Input the high-resolution hyperspectral fusion image into a pre-trained Unet deep learning network model for building detection and extraction, and output multiple single-building sub-images after cropping; Input the single-building sub-images into the overall edge attention transformer for extracting the outer contour of the building roof, and output the roof surface edge structure line. Among them, the overall edge attention transformer includes a backbone network and a corner detector. The backbone network is Resnet, and its output end is connected to the input end of the image feature encoder. The output end of the corner detector is connected to the input end of the position encoder. The output end of the position encoder and the output end of the image feature encoder are connected to the input end of the feature fusion module. The output end of the feature fusion module is connected to the input end of the edge filter, and the output end of the edge filter is respectively connected to the image perception decoder; The corner detector is used to obtain a set of candidate corners from the high-resolution hyperspectral fusion image, and each pair of candidate corners forms a candidate edge; The position encoder is used to initialize the candidate edge to obtain the edge position feature of the candidate edge; The feature fusion module includes a connected addition normalization layer and a feed-forward network model, which is used to fuse the edge position feature of the candidate edge and the image feature output by the image encoder, and use the fused candidate edge as a fusion node; The edge filter is used to calculate the confidence score of the fusion node, and filter out the fusion nodes with a confidence score higher than a preset threshold according to the confidence score; The image perception decoder is used to use the fusion nodes with a confidence score higher than the preset threshold as Transfomer nodes, and combine all Transfomer nodes to output the roof surface edge structure line; According to the pixel position coordinates of the roof surface edge structure line in the single-building sub-image and the topological connection relationship of the edge structure line, extract the spectral data of the roof surface pixels; Match the spectral data of the roof surface pixels with the spectral database of building roof materials in the urban area to be measured, identify the building roof materials, and infer the building attributes based on the building roof materials. Specifically, it includes: Regard the spectral data as a vector in a multi-dimensional space, and calculate the spectral angle between the spectral data of the roof surface and the spectral data of the known roof materials in the spectral database of building roof materials in the urban area to be measured; The known roof material spectral data with the smallest spectral angle to the spectral data vector of the roof surface is the same type of roof material, and the roof material is obtained; Infer according to the correlation characteristics between the roof material and the house use, and obtain the building attributes. The building attributes include roof contour, roof material label, and house use attribute inference label; According to the location information of the individual building sub-images, all the individual building sub-images are spliced and fused to obtain the building attributes of the urban area to be measured.

2. The building attribute extraction method based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that, The spectrum database of roof materials of buildings in the town to be tested is obtained by performing spectrum detection on common roof materials in the town to be tested.

3. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, wherein The method of fusing large-scene high-resolution satellite image data and hyperspectral remote sensing image data of the same scene to obtain a high-resolution hyperspectral fused image includes: Convert large-scale high-resolution satellite image data from RGB space to HIS space to obtain large-scale high-resolution satellite image data in HIS space; The intensity information of the hyperspectral remote sensing image data under the same scene is used to replace the HIS intensity of the large-scene high-resolution satellite image data in the HIS space to obtain the replaced hyperspectral remote sensing image data; The replaced hyperspectral remote sensing image data is converted into RGB space to obtain a high-resolution hyperspectral fusion image.

4. The method for extracting building attributes based on high - resolution hyperspectral remote sensing images according to claim 1, wherein, The Unet deep learning network model is a binary classification Unet network.

5. The building attribute extraction method based on high-resolution hyperspectral remote sensing imagery according to claim 4 is characterized in that: The pre-training method of the Unet deep learning network model includes: S1. Select a portion of the high-resolution hyperspectral fusion image and perform building attribute annotation on the high-resolution hyperspectral fusion image in QGIS software. The building attribute annotation includes annotating the building roof with a polygon and annotating the roof material and the building use attribute with a text label to obtain a high-resolution hyperspectral fusion image with building attribute annotations; S2, dividing the high-resolution hyperspectral fusion image with building attribute annotations into a training set and a validation set; S3. Using the training set as input and the background and building roof distinguishing images as output, the Unet deep learning network model is trained until the evaluation index reaches the preset target value, thereby obtaining a trained Unet deep learning network model. S4: Use the validation set as input and the background and building roof distinction image as output to verify the Unet deep learning network model. If the evaluation index does not reach the preset target value, repeat S3 to S4 until the evaluation index reaches the preset target value, and obtain the pre-trained Unet deep learning network model. The evaluation indicators include pixel accuracy and intersection-over-union ratio.

6. The building attribute extraction method based on high-resolution hyperspectral remote sensing imagery according to claim 1, characterized in that: The size of the single building sub-image is 256×256.

7. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, wherein The spectral data of the roof surface pixels are obtained by extracting pixels of a grid based on polygonal coordinates.

8. The method for extracting building attributes based on high-resolution hyperspectral remote sensing images according to claim 1, characterized in that The calculation expression of the spectral angle is: ； in, represents the spectral data vector of the roof surface, represents the known spectral data vector in the spectral database of roof materials of buildings in the urban area to be tested, represents the spectral angle.

Images