Building roof information extraction method and device, electronic equipment and storage medium

Through the combination of semantic segmentation and image classification model, building roof information is extracted, which solves the problems of high extraction difficulty and imaging quality in the prior art, and improves the accuracy of building roof information.

CN119006820APending Publication Date: 2024-11-22STATE NUCLEAR ELECTRIC POWER PLANNING DESIGN & RES INST CO LTD +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411140968.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art has problems with high difficulty and influence of remote sensing image imaging quality in the extraction of roof information of buildings, resulting in low extraction accuracy.

Method used

By obtaining semantic segmentation sample data, training semantic segmentation model and post-processing, obtaining roof image data sets, training roof image classification model, and finally inputting multiple single roof image data into roof image classification model, obtaining multi-classification of building roofs information.

Benefits of technology

It reduces the overall difficulty of building roof information extraction task, reduces the impact of remote sensing images, and improves the accuracy of building roof information extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006820A_ABST
    Figure CN119006820A_ABST
Patent Text Reader

Abstract

The invention provides a building roof information extraction method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the specific implementation scheme is as follows: obtaining first semantic segmentation sample data; training a first model based on the first semantic segmentation sample data to obtain a semantic segmentation model; inputting the first original remote sensing image data into a semantic segmentation model to obtain first semantic segmentation image data, and performing post-processing on the first semantic segmentation image data to obtain multiple pieces of single roof image data; training a second model based on the first roof image data set to obtain a roof image classification model; inputting a plurality of pieces of single roof image data into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data, so that the overall difficulty of a building roof information extraction task can be reduced, the imaging quality influence of the remote sensing image is reduced, and the extraction efficiency is improved. Therefore, the accuracy of building roof information extraction is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for extracting building roof information. Background Art

[0002] With the continuous advancement of sensor technology, the clarity of remote sensing equipment is also constantly improving. Remote sensing imaging technology is an imaging technology that uses remote sensing equipment to obtain surface information. It has been widely used in agriculture, urban planning, environmental monitoring and other fields. Through high-resolution image capture, remote sensing imaging technology can not only provide more detailed surface observations, but also provide decision makers with dynamically changing surface information through multi-phase and multi-band data analysis. Summary of the invention

[0003] The present invention provides a method, device, electronic device and storage medium for extracting building roof information.

[0004] According to a first aspect of the present disclosure, a method for extracting building roof information is provided, comprising:

[0005] Acquire first semantic segmentation sample data; the first semantic segmentation sample data includes first image slice data and binary image data corresponding to the first image slice data; the first image slice data is image slice data obtained by randomly sampling the first original remote sensing image data; the binary image data is obtained by marking the roofs of buildings in the first image slice data;

[0006] The first model is trained based on the first semantic segmentation sample data to obtain a semantic segmentation model; the first model takes the remote sensing image data as input and takes the semantic segmentation image data as output;

[0007] Inputting the first original remote sensing image data into the semantic segmentation model to obtain first semantic segmentation image data, and post-processing the first semantic segmentation image data to obtain a plurality of single roof image data;

[0008] The second model is trained based on the first roof image data set to obtain a roof image classification model; the second model takes a single roof image data as input and takes roof multi-classification information as output; the first roof image data set includes at least one single roof image data among multiple single roof image data and its corresponding building roof multi-classification information; the building roof multi-classification information is obtained by multi-classification labeling of the building roof corresponding to the single roof image data;

[0009] A plurality of single roof image data are input into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0010] According to a second aspect of the present disclosure, a device for extracting building roof information is provided, comprising:

[0011] An acquisition module is used to acquire first semantic segmentation sample data; the first semantic segmentation sample data includes first image slice data and binary image data corresponding to the first image slice data; the first image slice data is image slice data obtained by randomly sampling the first original remote sensing image data; the binary image data is obtained by marking the roofs of buildings in the first image slice data;

[0012] A first training module is used to train a first model based on first semantic segmentation sample data to obtain a semantic segmentation model; the first model takes remote sensing image data as input and takes semantic segmentation image data as output;

[0013] A post-processing module, used for inputting the first original remote sensing image data into the semantic segmentation model to obtain first semantic segmentation image data, and post-processing the first semantic segmentation image data to obtain a plurality of single roof image data;

[0014] The second training module is used to train the second model based on the first roof image data set to obtain a roof image classification model; the second model takes a single roof image data as input and takes roof multi-classification information as output; the first roof image data set includes at least one single roof image data among multiple single roof image data and its corresponding building roof multi-classification information; the building roof multi-classification information is obtained by multi-classification labeling of the building roof corresponding to the single roof image data;

[0015] The classification module is used to input multiple single roof image data into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to at least one processor; wherein,

[0019] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the method of the first aspect mentioned above.

[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the first aspect.

[0021] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program, which implements the steps of any one of the methods in the first aspect when executed by a processor.

[0022] The technical solution disclosed in the present invention comprises the following steps: obtaining first semantic segmentation sample data; training a first model based on the first semantic segmentation sample data to obtain a semantic segmentation model; inputting first original remote sensing image data into the semantic segmentation model to obtain first semantic segmentation image data, and post-processing the first semantic segmentation image data to obtain multiple single roof image data; training a second model based on the first roof image data set to obtain a roof image classification model; inputting multiple single roof image data into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data, thereby reducing the overall difficulty of the task of extracting building roof information and reducing the impact of the imaging quality of the remote sensing image, so as to improve the accuracy of extracting building roof information.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0025] Figure 1 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure;

[0026] Figure 2 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure;

[0027] Figure 3 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure;

[0028] Figure 4 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure;

[0029] Figure 5 is a schematic diagram of an execution method for extracting building roof information provided according to an embodiment of the present disclosure;

[0030] Figure 6 is a block diagram of a building roof information extraction device provided according to an embodiment of the present disclosure;

[0031] Figure 7 is a block diagram of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] Figure 1 FIG. 1 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure. Figure 1 As shown, the building roof information extraction method includes but is not limited to the following steps:

[0034] In step S101, first semantic segmentation sample data is obtained.

[0035] It should be noted that, in some embodiments, the first original remote sensing image data is obtained by shooting in the study area by a remote sensing device. The first original remote sensing image data includes geographic coordinate information. Exemplarily, multiple sub-region original image data shot by the first remote sensing device in the first study area are obtained; radiation calibration and correction processing is performed on each of the multiple sub-region original image data based on the geographic coordinate information to obtain multiple corrected sub-region original image data; multiple corrected sub-region original image data are spliced ​​to obtain spliced ​​original image data; and the spliced ​​original image data are subjected to noise reduction processing to obtain the first original remote sensing image data.

[0036] Optionally, in some embodiments, the first image slice data is image slice data obtained by randomly sampling the first original remote sensing image data. Exemplarily, the first original remote sensing image data is randomly cropped to obtain multiple first image slice data with the same resolution.

[0037] It should be noted that, in some embodiments, the plurality of first image slice data is a group of image slice data with the same resolution and consistent spatial range.

[0038] Optionally, in some embodiments, the binary image data is obtained by annotating the roof of the building in the first image slice data. Exemplarily, the annotation method of the binary image data is a binary classification annotation method, and the label information of the binary image data is roof or non-roof; the building roof of each first image slice data in the first image slice data is annotated by binary classification using a manual method or an artificial intelligence algorithm to obtain multiple binary image data.

[0039] It should be noted that, in some embodiments, the first semantic segmentation sample data includes first image slice data and binary image data corresponding to the first image slice data.

[0040] In step S102, a first model is trained based on first semantic segmentation sample data to obtain a semantic segmentation model.

[0041] It should be noted that, in some embodiments, the first model takes remote sensing image data as input and semantic segmentation image data as output. Optionally, in some embodiments, the first model is an image segmentation model with edge filling operations and batch normalization layers added to the U-Net image segmentation network.

[0042] It is understandable that the U-Net image segmentation network is a deep learning network for image semantic segmentation. Adding edge padding operations to the compression and expansion paths of the U-Net image segmentation network keeps the image size consistent before and after convolution processing; adding batch normalization layers to the U-Net image segmentation network can reduce the gradient explosion phenomenon during training, so as to improve the learning effect of the U-Net image segmentation network.

[0043] Optionally, in some embodiments, the first semantic segmentation sample data is divided into a first semantic segmentation sample training set and a first semantic segmentation sample verification set; the first model is trained based on the first semantic segmentation sample training set and the first semantic segmentation sample verification set to obtain a semantic segmentation model.

[0044] In step S103, the first original remote sensing image data is input into a semantic segmentation model to obtain first semantic segmentation image data, and the first semantic segmentation image data is post-processed to obtain a plurality of single roof image data.

[0045] Optionally, in some embodiments, the first original remote sensing image data is divided into multiple first original remote sensing sub-image data having spatial coordinates; for each first original remote sensing sub-image data in the multiple first original remote sensing sub-image data, the first original remote sensing sub-image data is input into a semantic segmentation model to obtain first semantic segmentation sub-image data corresponding to the first original remote sensing sub-image data; and the multiple first semantic segmentation sub-image data are spliced ​​based on the geographic coordinate information in the first original remote sensing image data to obtain first semantic segmentation image data.

[0046] It should be noted that, in some embodiments, the mathematical morphology method is an image processing method that extracts information from the geometric shape of an image. The spatial clustering algorithm is an algorithm that groups sample data according to spatial features. Optionally, in some embodiments, the first semantic segmentation image data is post-processed using the mathematical morphology method and the spatial clustering algorithm to obtain multiple single roof image data.

[0047] In step S104, the second model is trained based on the first roof image dataset to obtain a roof image classification model.

[0048] Optionally, in some embodiments, the first roof image data set includes at least one single roof image data among multiple single roof image data and its corresponding multi-classification information of the building roof; the multi-classification information of the building roof is obtained by multi-classification labeling of the building roof of the corresponding single roof image data.

[0049] Optionally, in some embodiments, the second model takes a single roof image data as input and outputs multi-classification information of the roof.

[0050] In step S105, a plurality of single roof image data are input into a roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0051] It should be noted that, in some embodiments, the multi-classification information of each building roof in the first original remote sensing image data includes at least one of the following: roof type information; roof shape information; roof use information; roof material information. Exemplarily, the roof type information includes but is not limited to flat roof, single-slope roof, double-slope roof; the roof shape information includes but is not limited to rectangular roof, polygonal roof, special-shaped roof; the roof use information includes but is not limited to placing communication equipment, parking lot, renewable energy generation, storage space; the roof material information includes but is not limited to blue color steel, concrete.

[0052] In an embodiment of the present disclosure, first semantic segmentation sample data is obtained; a first model is trained based on the first semantic segmentation sample data to obtain a semantic segmentation model; first original remote sensing image data is input into the semantic segmentation model to obtain first semantic segmentation image data, and the first semantic segmentation image data is post-processed to obtain multiple single roof image data; a second model is trained based on the first roof image data set to obtain a roof image classification model; multiple single roof image data are input into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data, which can reduce the overall difficulty of the task of extracting building roof information and reduce the impact of the imaging quality of the remote sensing image, so as to improve the accuracy of extracting building roof information.

[0053] Figure 2 FIG. 1 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure. Figure 2 As shown, the building roof information extraction method includes but is not limited to the following steps:

[0054] In step S201, first semantic segmentation sample data is obtained.

[0055] In the embodiments of the present disclosure, step S201 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0056] In step S202, data enhancement processing is performed on the first semantic segmentation sample data to obtain first semantic segmentation sample data after data enhancement.

[0057] It should be noted that, in some embodiments, the method of performing data enhancement processing on the first semantic segmentation sample data includes but is not limited to random image rotation, random image flipping, random image cropping, and random image color change. Exemplarily, the first semantic segmentation sample data is subjected to data enhancement processing by using the image random rotation method to obtain the first semantic segmentation sample data after data enhancement, which can expand the number of sample data, improve the quality of sample data and the generalization of the first semantic segmentation sample data.

[0058] In step S203, the first model is trained based on the first semantic segmentation sample data after data enhancement to obtain a semantic segmentation model.

[0059] Optionally, in some embodiments, the first semantic segmentation sample data after data enhancement is divided into a first semantic segmentation sample training set and a first semantic segmentation sample verification set; the first model is trained based on the first semantic segmentation sample training set and the first semantic segmentation sample verification set to obtain a semantic segmentation model.

[0060] In step S204, the first original remote sensing image data is input into a semantic segmentation model to obtain first semantic segmentation image data, and the first semantic segmentation image data is post-processed to obtain a plurality of single roof image data.

[0061] In the embodiments of the present disclosure, step S204 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0062] In step S205, the second model is trained based on the first roof image data set to obtain a roof image classification model.

[0063] In the embodiments of the present disclosure, step S205 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0064] In step S206, a plurality of single roof image data are input into a roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0065] In the embodiments of the present disclosure, step S206 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0066] In an embodiment of the present disclosure, data enhancement processing is performed on the first semantic segmentation sample data to obtain the first semantic segmentation sample data after data enhancement; the first model is trained based on the first semantic segmentation sample data after data enhancement to obtain a semantic segmentation model, which can expand the number of sample data, improve the quality of the sample data and the generalization of the first semantic segmentation sample data, so as to improve the accuracy of the semantic segmentation model.

[0067] Figure 3 FIG. 1 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure. Figure 3 As shown, the building roof information extraction method includes but is not limited to the following steps:

[0068] In step S301, first semantic segmentation sample data is obtained.

[0069] In the embodiments of the present disclosure, step S301 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0070] In step S302, a first model is trained based on first semantic segmentation sample data to obtain a semantic segmentation model.

[0071] In the embodiments of the present disclosure, step S302 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0072] In step S303, the first original remote sensing image data is input into a semantic segmentation model to obtain first semantic segmentation image data.

[0073] Optionally, in some embodiments, a first sliding window is used to obtain multiple first original remote sensing sub-image data from the first original remote sensing image data; for each first original remote sensing sub-image data in the multiple first original remote sensing sub-image data, the first original remote sensing sub-image data is input into a semantic segmentation model to obtain first semantic segmentation sub-image data; and the multiple first semantic segmentation sub-image data are spliced ​​to obtain first semantic segmentation image data.

[0074] Optionally, in some embodiments, a model accuracy evaluation method is used to evaluate the semantic segmentation model based on the first semantic segmentation image data to obtain semantic segmentation model accuracy evaluation information. Exemplarily, the semantic segmentation model accuracy evaluation information includes but is not limited to at least one of the following: pixel accuracy information, precision information, recall rate information, intersection-over-union information, and F1 index information.

[0075] In step S304, a mathematical morphology processing method is used to smooth the first semantic segmentation image data to obtain intermediate semantic segmentation image data.

[0076] It can be understood that by processing the first semantic segmentation image data using a mathematical morphology processing method, noise in the first semantic segmentation image data can be removed.

[0077] Optionally, in some embodiments, the mathematical morphology processing method includes but is not limited to at least one of the following: an erosion method, an expansion method, an opening operation method, and a closing operation method. Exemplarily, an opening operation is performed on the first semantic segmentation image data based on a preset convolution kernel to obtain intermediate semantic segmentation image data.

[0078] In step S305, a connected domain analysis is performed on the intermediate semantic segmentation image data to obtain at least one first connected domain image data.

[0079] It should be noted that, in some embodiments, a connected domain refers to an area in an image consisting of pixels with the same pixel value and adjacent positions. Optionally, in some embodiments, a connected domain labeling algorithm based on depth-first search is used to perform connected domain analysis on the intermediate semantic segmentation image data to obtain at least one first connected domain image data.

[0080] In step S306, an image clustering algorithm is used to segment the first connected domain image data to obtain a plurality of single roof image data.

[0081] Optionally, in some embodiments, for each first connected domain image data in at least one first connected domain image data, a spatial clustering algorithm is used to smooth the first connected domain image data to obtain smoothed first connected domain image data; the edge pixel value in the smoothed first connected domain image data is determined as the first background value to obtain multiple single roof image data.

[0082] It should be noted that, in some embodiments, opening operation is performed on multiple single roof image data based on a preset convolution kernel, so that the noise caused by the roof segmentation process can be removed.

[0083] In step S307, the second model is trained based on the first roof image data set to obtain a roof image classification model.

[0084] In the embodiments of the present disclosure, step S307 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0085] In step S308, a plurality of single roof image data are input into a roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0086] In the embodiments of the present disclosure, step S308 may be implemented in any of the ways in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0087] In an embodiment of the present disclosure, first original remote sensing image data is input into a semantic segmentation model to obtain first semantic segmentation image data; a mathematical morphology processing method is used to smooth the first semantic segmentation image data to obtain intermediate semantic segmentation image data; a connected domain analysis is performed on the intermediate semantic segmentation image data to obtain at least one first connected domain image data; an image clustering algorithm is used to segment the first connected domain image data to obtain multiple individual roof image data, which can further improve the accuracy of the roof segmentation processing, so as to improve the accuracy of the building roof information extraction.

[0088] Figure 4 FIG. 1 is a flow chart of a method for extracting building roof information according to an embodiment of the present disclosure. Figure 4 As shown, the building roof information extraction method includes but is not limited to the following steps:

[0089] In step S401, first semantic segmentation sample data is obtained.

[0090] In the embodiments of the present disclosure, step S401 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0091] In step S402, a first model is trained based on first semantic segmentation sample data to obtain a semantic segmentation model.

[0092] In the embodiments of the present disclosure, step S402 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0093] In step S403, the first original remote sensing image data is input into a semantic segmentation model to obtain first semantic segmentation image data, and the first semantic segmentation image data is post-processed to obtain a plurality of single roof image data.

[0094] In the embodiments of the present disclosure, step S403 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0095] In step S404, data enhancement processing is performed on the first roof image dataset to obtain a data enhanced first roof image dataset.

[0096] It should be noted that, in some embodiments, the data enhancement processing of the first roof image dataset includes, but is not limited to, random rotation of the image, random flipping of the image, random cropping of the image, and random color transformation of the image. The data enhancement processing of the first roof image dataset is performed to obtain the data-enhanced first roof image dataset, which can expand the number of sample data, improve the quality of the sample data and the generalization of the first roof image dataset.

[0097] In step S405, the second model is trained based on the data-enhanced first roof image dataset to obtain a roof image classification model.

[0098] It should be noted that, in some embodiments, the second model is a ResNet50 image classification model with a binary cross entropy function as a loss function. Exemplarily, the second model is a ResNet50 image classification model with a single roof image data as input and multi-classification information of the roof as output; the second model is trained based on the first roof image data set after data enhancement to obtain a roof image classification model.

[0099] In step S406, a plurality of single roof image data are input into a roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0100] In the embodiments of the present disclosure, step S406 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0101] In an embodiment of the present disclosure, data enhancement processing is performed on the first roof image dataset to obtain the data enhanced first roof image dataset; the second model is trained based on the data enhanced first roof image dataset to obtain a roof image classification model, which can expand the number of sample data, improve the quality of sample data and the generalization of the first roof image dataset, so as to further improve the accuracy of extracting building roof information.

[0102] Figure 5 FIG. 1 is a schematic diagram of an execution of a method for extracting building roof information according to an embodiment of the present disclosure. Figure 5As shown, a plurality of sub-region original image data taken by a first remote sensing device in a first study area are obtained (S501); radiation calibration and correction processing is performed on each of the plurality of sub-region original image data based on geographic coordinate information to obtain a plurality of corrected sub-region original image data (S502); a plurality of corrected sub-region original image data are spliced ​​to obtain spliced ​​original image data (S503); a noise reduction processing is performed on the spliced ​​original image data to obtain first original remote sensing image data (S504); a random cropping of the first original remote sensing image data is performed Cut to obtain multiple first image slice data with the same resolution (S505); perform binary classification annotation on the building roof of each first image slice data in the first image slice data to obtain multiple binary image data (S506); determine first semantic segmentation sample data based on the multiple first image slice data and the multiple binary image data (S507); use the image random rotation method to perform data enhancement processing on the first semantic segmentation sample data to obtain first semantic segmentation sample data after data enhancement; train the first model based on the first semantic segmentation sample data after data enhancement to obtain a semantic segmentation model (S508). The first original remote sensing image data is divided into a plurality of first original remote sensing sub-image data with spatial coordinates (S509); for each of the plurality of first original remote sensing sub-image data, the first original remote sensing sub-image data is input into a semantic segmentation model to obtain first semantic segmentation sub-image data corresponding to the first original remote sensing sub-image data (S510); based on the geographic coordinate information in the first original remote sensing image data, the plurality of first semantic segmentation sub-image data are spliced ​​to obtain first semantic segmentation image data (S511); the first semantic segmentation image data is smoothed by a mathematical morphology processing method to obtain intermediate semantic segmentation image data; a connected domain analysis is performed on the intermediate semantic segmentation image data to obtain at least one first connected domain image data; for each of the at least one first connected domain image data, a spatial clustering algorithm is used to smooth the first connected domain image data to obtain smoothed first connected domain image data; the edge pixel value in the smoothed first connected domain image data is determined as a first background value to obtain a plurality of single roof image data (S512). The second model is trained based on the first roof image data set to obtain a roof image classification model (S513); multiple single roof image data are input into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data (S514).

[0103] Figure 6 is a block diagram of a building roof information extraction device provided according to an embodiment of the present disclosure. Figure 6As shown, the building roof information extraction device includes but is not limited to: an acquisition module 601, a first training module 602, a post-processing module 603, a second training module 604, and a classification module 605.

[0104] The acquisition module 601 is used to acquire first semantic segmentation sample data; the first semantic segmentation sample data includes first image slice data and binary image data corresponding to the first image slice data; the first image slice data is image slice data obtained by randomly sampling the first original remote sensing image data; the binary image data is obtained by marking the roofs of the buildings in the first image slice data;

[0105] A first training module 602 is used to train a first model based on first semantic segmentation sample data to obtain a semantic segmentation model; the first model takes remote sensing image data as input and semantic segmentation image data as output;

[0106] A post-processing module 603 is used to input the first original remote sensing image data into a semantic segmentation model to obtain first semantic segmentation image data, and post-process the first semantic segmentation image data to obtain a plurality of single roof image data;

[0107] The second training module 604 is used to train the second model based on the first roof image data set to obtain a roof image classification model; the second model takes a single roof image data as input and takes roof multi-classification information as output; the first roof image data set includes at least one single roof image data among multiple single roof image data and its corresponding building roof multi-classification information; the building roof multi-classification information is obtained by multi-classification labeling of the building roof corresponding to the single roof image data;

[0108] The classification module 605 is used to input multiple single roof image data into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

[0109] As an example, the first training module 602 is also used to perform data enhancement processing on the first semantic segmentation sample data to obtain the first semantic segmentation sample data after data enhancement; and train the first model based on the first semantic segmentation sample data after data enhancement to obtain a semantic segmentation model.

[0110] As an example, the first model in the first training module 602 is an image segmentation model that adds edge filling operations and batch normalization layers based on the U-Net image segmentation network.

[0111] As an example, the post-processing module 603 is also used to use a mathematical morphological processing method to smooth the first semantic segmentation image data to obtain intermediate semantic segmentation image data; perform a connected domain analysis on the intermediate semantic segmentation image data to obtain at least one first connected domain image data; and use an image clustering algorithm to segment the first connected domain image data to obtain multiple single roof image data.

[0112] As an example, the post-processing module 603 is also used to perform a connected domain analysis on the intermediate semantic segmentation image data to obtain at least one first connected domain information; for each first connected domain information in the at least one first connected domain information, determine the first connected domain image data corresponding to the first connected domain information from the first original image data.

[0113] As an example, the second training module 604 is also used to perform data enhancement processing on the first roof image data set to obtain the data enhanced first roof image data set; and train the second model based on the data enhanced first roof image data set to obtain a roof image classification model.

[0114] As an example, the multi-classification information of each building roof in the first original remote sensing image data in the building roof information extraction device includes at least one of the following: roof type information; roof shape information; roof use information; roof material information.

[0115] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0116] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a storage medium, and a computer program product.

[0117] Figure 7 is a block diagram of an electronic device provided according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0118] like Figure 7As shown, the electronic device includes: one or more processors 701, a memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 701 is taken as an example.

[0119] The memory 702 is a non-transitory computer-readable storage medium provided by the present disclosure. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the building roof information extraction method provided by the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions, which are used to enable a computer to perform the building roof information extraction method provided by the present disclosure.

[0120] The memory 702 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the building roof information extraction method in the embodiment of the present disclosure (for example, the attached Figure 6 The processor 601 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 602, that is, the building roof information extraction method in the above method embodiment is implemented.

[0121] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The electronic device may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0123] The input device 703 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 704 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0124] Optionally, in some embodiments, a computer program product is also provided, including a computer program, which, when executed by one or more processors 701, can implement the building roof information extraction method provided in the embodiments of the present disclosure.

[0125] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0129] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for extracting building roof information, characterized in that: The following steps are involved: Acquire first semantic segmentation sample data; the first semantic segmentation sample data includes first image slice data and binary image data corresponding to the first image slice data; the first image slice data is image slice data obtained by randomly sampling the first original remote sensing image data; the binary image data is obtained by marking the roofs of buildings in the first image slice data; Training a first model based on the first semantic segmentation sample data to obtain a semantic segmentation model; the first model takes remote sensing image data as input and takes semantic segmentation image data as output; Inputting the first original remote sensing image data into the semantic segmentation model to obtain first semantic segmentation image data, and post-processing the first semantic segmentation image data to obtain a plurality of single roof image data; The second model is trained based on the first roof image data set to obtain a roof image classification model; the second model takes a single roof image data as input and takes roof multi-classification information as output; the first roof image data set includes at least one of the multiple single roof image data and its corresponding building roof multi-classification information; The multi-classification information of the building roof is obtained by multi-classification labeling of the building roof of the corresponding single roof image data; The plurality of single roof image data are input into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

2. The method according to claim 1, characterized in that The step of training the first model based on the first semantic segmentation sample data to obtain a semantic segmentation model includes: Performing data enhancement processing on the first semantic segmentation sample data to obtain first semantic segmentation sample data after data enhancement; The first model is trained based on the first semantic segmentation sample data after the data enhancement to obtain the semantic segmentation model.

3. The method according to claim 2, characterized in that The first model is an image segmentation model that adds edge filling operations and batch normalization layers on the basis of the U-Net image segmentation network.

4. The method according to claim 1, characterized in that The post-processing of the first semantic segmentation image data to obtain a plurality of single roof image data includes: Using a mathematical morphology processing method to smooth the first semantic segmentation image data to obtain intermediate semantic segmentation image data; Performing a connected domain analysis on the intermediate semantic segmentation image data to obtain at least one first connected domain image data; The first connected domain image data is segmented using an image clustering algorithm to obtain the plurality of single roof image data.

5. The method according to claim 4, characterized in that The performing a connected domain analysis on the intermediate semantic segmentation image data to obtain at least one first connected domain image data includes: Performing a connected domain analysis on the intermediate semantic segmentation image data to obtain at least one first connected domain information; For each first connected domain information in the at least one first connected domain information, first connected domain image data corresponding to the first connected domain information is determined from the first original image data.

6. The method according to claim 1, characterized in that The training of the second model based on the first roof image dataset to obtain a roof image classification model includes: Performing data enhancement processing on the first roof image dataset to obtain a data enhanced first roof image dataset; The second model is trained based on the data-enhanced first roof image data set to obtain the roof image classification model.

7. The method according to any one of claims 1 to 6, characterized in that: The multi-classification information of each building roof in the first original remote sensing image data includes at least one of the following: Roof type information; Roof shape information; Roof usage information; Roof material information.

8. A device for extracting building roof information, characterized in that: include: An acquisition module is used to acquire first semantic segmentation sample data; the first semantic segmentation sample data includes first image slice data and binary image data corresponding to the first image slice data; the first image slice data is image slice data obtained by randomly sampling the first original remote sensing image data; the binary image data is obtained by marking the roofs of buildings in the first image slice data; A first training module is used to train a first model based on the first semantic segmentation sample data to obtain a semantic segmentation model; the first model takes remote sensing image data as input and takes semantic segmentation image data as output; A post-processing module, used for inputting the first original remote sensing image data into the semantic segmentation model to obtain first semantic segmentation image data, and post-processing the first semantic segmentation image data to obtain a plurality of single roof image data; A second training module is used to train a second model based on the first roof image data set to obtain a roof image classification model; the second model takes a single roof image data as input and takes roof multi-classification information as output; the first roof image data set includes at least one of the multiple single roof image data and its corresponding building roof multi-classification information; The multi-classification information of the building roof is obtained by multi-classification labeling of the building roof of the corresponding single roof image data; The classification module is used to input the multiple single roof image data into the roof image classification model to obtain multi-classification information of each building roof in the first original remote sensing image data.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Filtering method for non-building area of remote sensing image based on deep learning

    CN111291608A

  • Image multi-label classification method based on semi-supervised learning

    CN112115995A

  • Hyperspectral image classification method based on twin network

    CN114821164A

  • Traditional village building roof damage identification and damage rate calculation method

    CN117523405A

  • Photovoltaic developable roof resource evaluation and carbon reduction potential evaluation method based on satellite image recognition

    CN118246763A