A method and device for automatically identifying roof information based on high-definition satellite images

Abstract

The present application provides a method and device for automatically identifying roof information based on high-definition satellite images, comprising: collecting high-definition satellite images without roof information labeled, preprocessing the images to obtain high-definition satellite raster images, and obtaining an orthophoto and a digital surface map based on the high-definition satellite raster images; identifying a roof contour range based on the orthophoto and the digital surface map in combination with high-definition satellite comparison images, extracting a set of roof contour coordinates from the roof contour range to obtain a second plane equation of a roof contour fitting plane; using the second plane equation, identifying roof contour information on the high-definition satellite comparison images, and obtaining a roof contour plane model based on the roof contour information; obtaining a roof contour comparison model using the pixel values ​​of each pixel point in the target roof contour fitting plane corresponding to the second plane equation; and obtaining and deriving a roof information recognition model through the high-definition satellite comparison images, the roof contour plane model, and the roof contour comparison model.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a method and device for automatically identifying roof information based on high-definition satellite images. Background Art

[0002] The roof is the outer covering of a building, such as a house. Automatic roof information recognition is the process of identifying and detecting information on the roof of a building. It is of great significance to land use analysis, urban planning changes, geographic data updates, and natural disaster relief.

[0003] With the continuous progress and development of science and technology, the application of remote sensing identification technologies such as high-definition satellite imagery is becoming more and more extensive. Among them, high-definition satellite imagery is a kind of data that uses satellites equipped with various sensors to obtain comprehensive, true and objective data reflecting the surface characteristics. These data can be processed by remote sensing technology to become images with high-precision geographic coordinate information.

[0004] High-definition satellite imagery is also used for automatic rooftop identification due to its high resolution, wide coverage, short update cycles, and rich spectral characteristics. However, due to the diverse natural environments, regional cultures, and architectural styles of different regions, this process is prone to missed and misidentified rooftops, resulting in poor accuracy.

[0005] Therefore, in order to solve the above technical problems, it is necessary to provide a method and device for automatically identifying roof information based on high-definition satellite images. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a method and device for automatically identifying rooftop information based on high-definition satellite images. The technical solution is as follows:

[0007] In a first aspect, the present application provides a method for automatically identifying rooftop information based on high-definition satellite images, the method comprising:

[0008] collecting high-definition satellite images without roof information labeled, preprocessing the high-definition satellite images to obtain high-definition satellite raster images, and obtaining orthophotos and digital surface maps based on the high-definition satellite raster images;

[0009] Based on the orthophoto and the digital surface map, in combination with a pre-built high-definition satellite image library, identifying the roof outline range of the high-definition satellite image, and extracting a roof outline coordinate set from the roof outline range, wherein the high-definition satellite image library stores high-definition satellite comparison images with different roof features;

[0010] Obtaining a first plane equation of the roof profile fitting plane according to the roof profile coordinate set, and correcting the first plane equation of the roof profile fitting plane to obtain a second plane equation of the roof profile fitting plane;

[0011] Using the second plane equation, identifying roof profile information on the high-definition satellite comparison image, and obtaining a roof profile plane model based on the roof profile information;

[0012] Collecting pixel values ​​of each pixel point in the target roof outline fitting plane corresponding to the second plane equation to obtain a roof outline color set, and fitting the target roof outline fitting plane using the roof outline color set to obtain a roof outline comparison model;

[0013] Comparing the roof profile plane model and the roof profile comparison model using the high-definition satellite comparison image to obtain a roof information recognition model;

[0014] The roof information recognition model is derived, and the roof information recognition model is used to recognize roof information of the high-definition satellite image.

[0015] Optionally, obtaining an orthophoto and a digital surface map based on the high-definition satellite raster image includes:

[0016] Calling a roof comparison model from a pre-built roof comparison model library to perform feature extraction on the high-definition satellite raster image to obtain key points in the high-definition satellite raster image, and determining landmark points matching the key points in the high-definition satellite raster image, wherein the roof comparison model is trained based on building features of at least one region, and different roof comparison models correspond to different features of some buildings;

[0017] When the high-definition satellite grid image is determined to be a key frame image based on the landmark points, optimizing the local map information corresponding to the key frame image based on a preset spherical positioning system error and a reprojection error;

[0018] Determining the orthotropy of the key frame image based on the optimized local map information;

[0019] When the orthotropy is greater than a preset threshold, projecting the key frame image onto a ground plane, and dividing the projected key frame image into a plurality of image tiles;

[0020] fusing each image tile and the orthophoto value tile of each image tile to generate an orthophoto image of each image tile, wherein the orthophoto image of each image tile can constitute an orthophoto map of the key frame image;

[0021] The high-definition satellite raster image is processed according to the image block matching technology and the semi-global matching algorithm to obtain the digital surface image.

[0022] Optionally, the preprocessing of the high-definition satellite image includes:

[0023] Image filtering and image cropping are performed on the high-definition satellite image, wherein the image filtering is used to eliminate noise in the high-definition satellite image, and the image cropping is used to remove interference contours in the high-definition satellite image, and one of the image filtering and the image cropping is performed before the other.

[0024] Optionally, the identifying the roof outline range of the high-definition satellite image based on the orthophoto and the digital surface map in combination with a pre-built high-definition satellite image library includes:

[0025] selecting a high-definition satellite comparison image from a pre-built high-definition satellite image library based on the orthophoto and the digital surface map;

[0026] Using the selected high-definition satellite comparison images, a roof outline recognition model with the ability to recognize the range of roof outlines is trained;

[0027] Inputting a high-definition satellite image into the roof outline recognition model to obtain an initial roof outline range of the high-definition satellite image output by the roof outline recognition model;

[0028] performing binarization processing on the high-definition satellite image using the initial roof outline range to obtain a binarized raster image of the high-definition satellite image;

[0029] Vectorization is performed on the binary raster map of the high-definition satellite image to obtain a raster vector map, and the boundary of the raster vector map is optimized to obtain the roof outline range.

[0030] Optionally, the selecting a high-definition satellite comparison image from a pre-built high-definition satellite image library based on the orthophoto and the digital surface map includes:

[0031] For each pixel point in the digital surface map, using the coordinates of the pixel point, obtain the RGB information of the pixel point from the orthophoto map;

[0032] By obtaining the RGB information of all pixels from the orthophoto, a high-definition satellite comparison image that matches the RGB information of all pixels is selected from the high-definition satellite image library.

[0033] Optionally, obtaining a first plane equation of a roof profile fitting plane according to the roof profile coordinate set includes:

[0034] Determining pixel coordinates of corner points of the roof outline according to the roof outline coordinate set;

[0035] A first plane equation of the roof outline fitting plane is calculated according to the corner point pixel coordinates.

[0036] Optionally, the modifying the first plane equation of the roof profile fitting plane to obtain the second plane equation of the roof profile fitting plane includes:

[0037] Obtaining multi-extended pixel coordinates corresponding to the focus pixel coordinates according to the corner pixel coordinates and a set constant value;

[0038] Substituting each extended pixel coordinate into the first plane equation to obtain a calculation result of the extended pixel coordinate, and calculating an error value of the calculation result;

[0039] When the error value is smaller than the standard error value, the first plane equation is corrected using the extended pixel coordinates whose error value is smaller than the standard error value.

[0040] Optionally, the using the second plane equation to identify roof profile information on the high-definition satellite comparison image, and obtaining a roof profile plane model based on the roof profile information includes:

[0041] Using the plane equation to identify the roof outline information in the high-definition satellite comparison image, the roof outline information is identified from the high-definition satellite comparison image;

[0042] Comparing the identified roof profile information with the roof profile information annotated in the high-definition satellite comparison image to determine the roof profile information for training the roof profile plane model;

[0043] The roof profile plane model is trained using the determined roof profile information.

[0044] Optionally, comparing the roof profile plane model and the roof profile comparison model using the high-definition satellite comparison image to obtain a roof information recognition model includes:

[0045] Inputting the high-definition satellite comparison image into the roof profile plane model and the roof profile comparison model to obtain roof profile coordinates outputted by the roof profile plane model and the roof profile comparison model respectively;

[0046] calculating the error between the roof profile coordinates;

[0047] If the error is within the standard deviation range, the roof profile comparison model is used as the roof information recognition model;

[0048] If the error is not within the standard deviation range, it is prohibited to use the roof profile comparison model as the roof information recognition model.

[0049] In a second aspect, the present application provides a device for automatically identifying rooftop information based on high-definition satellite images, the device comprising:

[0050] An image processing module is used to collect high-definition satellite images without roof information labeled, pre-process the high-definition satellite images to obtain high-definition satellite raster images, and obtain orthophotos and digital surface maps based on the high-definition satellite raster images;

[0051] a contour recognition module for identifying the roof contour range of the high-definition satellite image based on the orthophoto and the digital surface map, and extracting a set of roof contour coordinates from the roof contour range in combination with a pre-built high-definition satellite image library, wherein the high-definition satellite image library stores high-definition satellite comparison images with different roof features;

[0052] a plane equation processing module, configured to obtain a first plane equation of the roof profile fitting plane according to the roof profile coordinate set, and to modify the first plane equation of the roof profile fitting plane to obtain a second plane equation of the roof profile fitting plane;

[0053] a roof profile plane model acquisition module, configured to identify roof profile information on the high-definition satellite comparison image using the second plane equation, and obtain a roof profile plane model based on the roof profile information;

[0054] a roof outline comparison model acquisition module, configured to collect pixel values ​​of each pixel point in the target roof outline fitting plane corresponding to the second plane equation to obtain a roof outline color set, and fit the target roof outline fitting plane using the roof outline color set to obtain a roof outline comparison model;

[0055] a roof information recognition model acquisition module, configured to compare the roof profile plane model and the roof profile comparison model using the high-definition satellite comparison image to obtain a roof information recognition model;

[0056] The export module is used to export the roof information recognition model, and the roof information recognition model is used to recognize the roof information of the high-definition satellite image.

[0057] In a third aspect, the present application provides a device for automatically identifying roof information, comprising: a processor and a memory;

[0058] The memory is used to store computer program codes, which include computer instructions. When the processor executes the computer instructions, the roof information automatic identification device performs the above method.

[0059] In a fourth aspect, the present application provides a storage medium for storing a computer program, which, when executed, is specifically used to implement the above method.

[0060] Compared with the existing technology, the above technical solution provided by this application has the following advantages:

[0061] The high-definition satellite image library can store high-definition satellite comparison images with different roof features. Using these high-definition satellite comparison images, the above method is used to obtain a roof information recognition model that can be applied to different roof features. The roof information of high-definition satellite images with different roof features is automatically recognized using this roof information recognition model, which greatly reduces the possibility of missed recognition or misidentification during the automatic recognition of roof information using high-definition satellite images, and significantly improves the accuracy of automatic recognition of roof information using high-definition satellite images. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0063] Figure 1 This is a flow chart of a method for automatically identifying rooftop information based on high-definition satellite images provided in an embodiment of the present application;

[0064] Figure 2 is a schematic diagram of a high-definition satellite image library provided in an embodiment of the present application;

[0065] Figure 3 This is a flowchart of generating an orthophoto provided by an embodiment of the present application;

[0066] Figure 4 This is a schematic diagram of automatic roof information recognition provided by an embodiment of the present application;

[0067] Figure 5 This is a schematic diagram of another method for automatically identifying rooftop information provided by an embodiment of the present application;

[0068] Figure 6 This is a structural diagram of a device for automatically identifying rooftop information based on high-definition satellite images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0070] See Figure 1 , which shows the process of a method for automatically identifying rooftop information based on high-definition satellite images provided by an embodiment of the present application, which may include the following steps:

[0071] S11. Pre-build a high-definition satellite image library, collect building features in various regions, and pre-build a roof comparison model library based on the building features in various regions.

[0072] The high-definition satellite image library is used to store high-definition satellite comparison images. High-definition satellite comparison images are pre-collected high-definition satellite images with roof information annotated. High-definition satellite comparison images can be applied to roof information recognition in high-definition satellite images that do not have roof information annotated. A high-definition satellite comparison image can include at least one type of building with the same building features, and the roof information of each building is annotated. This allows the high-definition satellite image library to include multiple high-definition satellite images with various building features and annotated roof information, thereby enriching the building features in the high-definition satellite image library. By using numerous high-definition satellite comparison images to record building features in different regions, when automatically identifying roof information using the high-definition satellite image library, matching high-definition satellite comparison images can be found from the numerous high-definition satellite comparison images, reducing missed or misidentified information and improving accuracy.

[0073] The roof comparison model library is used to store roof comparison models, which are used to generate roof information recognition models. This model is used to perform roof information recognition on HD satellite imagery without roof information labels, thereby identifying (extracting) roof outlines from the HD satellite imagery. A roof comparison model can identify features of one type of building, or it can identify features of multiple types of buildings.

[0074] The rooftop comparison model is trained using building features from various regions. For example, building features are extracted from high-definition satellite comparison images. The extracted building features and rooftop information annotated in the high-definition satellite comparison images are then used to train a rooftop comparison model. This allows the model to learn building features from different regions using at least one rooftop comparison model in the rooftop comparison model library. This model is then used to generate a rooftop information recognition model capable of identifying building features from different regions. This allows the model to identify rooftops with different building features, reducing missed or misidentified features and improving accuracy.

[0075] In some examples, building features include shape features, color features, and area features, which describe at least the roof information of the building. Shape features include square, rectangle, trapezoid, and combination shapes, and color features include red, blue, white, and other types. Area features can be used to indicate the level of building area, such as the level less than 10m 2 (square meters), 10-100m 2 , 100-500m 2 , 500m-1000m 2 , 1000m 2 above.

[0076] In this embodiment, the area features can be classified according to the shape features. For example, high-definition satellite comparison images with the same shape features and the same area features are classified into the same category. The roof comparison model is trained using high-definition satellite comparison images of the same category. Then, after obtaining high-definition satellite images without roof information labeled, the roof comparison model can be selected according to the area size of the high-definition satellite images, thereby improving the efficiency of subsequent automatic recognition of roof information. For example, the area of ​​the high-definition satellite image is 60m 2 , then the building area is less than or equal to 60m 2 , thus selecting the corresponding level of 10-100m for the high-definition satellite image 2 Roof comparison model.

[0077] Furthermore, during the training process for the rooftop comparison model, a masking algorithm is used to process the HD satellite comparison images, converting them into black and white images. Specifically, HD satellite comparison images are binary images with 0s and 1s, where 1 represents a building and 0 represents a non-building. This reduces the data volume of the HD satellite comparison images and improves processing efficiency. Furthermore, after the masking algorithm is applied, the coloring of the HD satellite comparison images is uniform, with 1 representing a building and 0 representing a non-building. This reduces the impact of color on the rooftop comparison model and thus reduces interference with the automatic recognition of rooftop information. Each HD satellite comparison image in the HD satellite image library can be replaced with an image processed by the masking algorithm, further reducing interference with the automatic recognition of rooftop information.

[0078] In some examples, the high-definition satellite image library includes an image classification library, an analysis and comparison image library, and an image update training library, such as Figure 2 The image classification library uses a classification comparison algorithm to classify HD satellite comparison images with the same roof features. Roof features include regional features, building features, and style features. By classifying HD satellite comparison images, it is easier to select and compare HD satellite comparison images based on roof features, thereby improving the efficiency of comparative analysis using the HD satellite image library.

[0079] The regional characteristics are used to indicate the region to which the high-definition satellite comparison image belongs. For example, the regional characteristics include information such as the northern plains region, the southern region, and the northern plateau region. The image classification library can classify the high-definition satellite comparison image based on the region, such as the northern region, the southern region, and the northern plateau region. The style characteristics are used to indicate at least one of the style and building type of the buildings in the high-definition satellite comparison image. In this embodiment, the high-definition satellite comparison image can be classified based on at least one of the regional characteristics, the building characteristics, and the style characteristics.

[0080] The analysis and comparison library uses an analysis and comparison algorithm to analyze and compare the HD satellite comparison images with the roof profile fitting plane to select HD satellite comparison images that match the roof profile indicated by the roof profile fitting plane. The HD satellite comparison images can be stored in the analysis and comparison library, and the image classification library can store the classifications of the HD satellite comparison images, such as the image identifier and classification identifier of the HD satellite comparison images. The image identifier can be an image number, and the classification identifier can be the type of the HD satellite comparison image or a classification number, with each classification number corresponding to a specific type.

[0081] The image update training library uses a replacement algorithm to update the high-definition satellite comparison images in the analysis and comparison library. The replacement algorithm directly replaces the same type of stored data with the updated data. The updated data can be the latest high-definition satellite comparison image. Because over time, there are situations such as demolition, new construction, and renovation of buildings in the same place, which cause the high-definition satellite comparison images of the place to be modified. Therefore, the image update training library can use the latest high-definition satellite comparison images to replace the old high-definition satellite comparison images, so that the high-definition satellite comparison images in the analysis and comparison library can be continuously updated and improved.

[0082] It's important to note that the pre-built HD satellite image library and rooftop comparison model library are used after acquiring HD satellite imagery without rooftop information. Once these libraries are built, simply update the HD satellite comparison chart and rooftop comparison model. Therefore, whenever HD satellite imagery without rooftop information is acquired, the pre-built HD satellite image library and rooftop comparison model library can be used, eliminating the need to build these libraries before each acquisition.

[0083] S12. Collect high-definition satellite images without roof information labeled. High-definition satellite images, also known as high-definition satellite images, can be captured by a high-definition satellite camera.

[0084] S13. Preprocess the high-definition satellite image to obtain a high-definition satellite raster image. The high-definition satellite raster image still does not have roof information marked.

[0085] In some examples, preprocessing includes at least one of image framing, image filtering, and image cropping. Image framing is used to perform block processing on high-definition satellite images. Image filtering is used to eliminate noise in high-definition satellite images. For example, image framing divides high-definition satellite images into multiple image blocks, and image filtering can eliminate noise in each image block. Multiple image blocks can be processed in parallel to improve noise removal efficiency. Image filtering can also improve the image quality of high-definition satellite images, such as at least improving image clarity, thereby reducing errors in subsequent contour extraction.

[0086] Image cropping is used to remove interfering contours from high-definition satellite imagery. For example, for each image block undergoing noise elimination, interfering contours are removed. Interfering contours can be the outlines of objects similar to buildings, such as the outlines of terrain features with similar textures and colors to buildings, or the outlines of hardened surfaces like roads. Removing interfering contours ensures accurate extraction of building outlines. Thus, preprocessing high-definition satellite imagery reduces interference and improves the accuracy of rooftop information recognition.

[0087] S14. Based on the high-definition satellite raster image, an orthophoto and a digital surface map are obtained. Each pixel in the orthophoto contains the RGB information of the region, and each pixel in the digital surface map contains the corresponding coordinate information. That is, each pixel in the orthophoto records the RGB information of the pixel, and each pixel in the digital surface map records the coordinate information of the pixel. The coordinates A (x i ,y i ) and coordinates A(x i ,y i )’s RGB value (r i ,g i ,b i ).

[0088] In some examples, the orthophoto generation process is as follows Figure 3 As shown, the following steps may be included:

[0089] S141. Perform feature extraction on the high-definition satellite raster image to obtain key points in the high-definition satellite raster image, and determine landmark points matching the key points in the high-definition satellite raster image.

[0090] In this embodiment, feature extraction is performed on the high-definition satellite raster image through the roof comparison model in the roof comparison model library. For example, the high-definition satellite raster image is input into the roof comparison model to obtain the features output by the roof comparison model. The features can be the roof outline output by the roof comparison model, etc., and each point representing the roof outline is used as a key point.

[0091] S142. Based on the landmark points, determine whether the high-definition satellite raster image is a key frame image. If so, execute step S143; if not, execute step S147.

[0092] S143. If the high-definition satellite raster image is a keyframe image, optimize the local map information corresponding to the keyframe image based on preset GPS (Global Positioning System) errors and reprojection errors. The local map information includes all keyframe images, all landmarks, and a similarity transformation from the visual coordinate system to the geographic coordinate system on the local map. The local map is the map to which the local map information points.

[0093] S144: Determine the orthorectification of the key frame image based on the optimized local map information. The key frame image is the high-definition satellite raster image determined as the key frame image based on the landmark points in step S142 above.

[0094] S145 : If the orthotropy is greater than a preset threshold, project the key frame image onto the ground plane, and divide the projected key frame image into a plurality of image tiles.

[0095] S146 , fusing each image tile and the orthophoto value tile of each image tile to generate an orthophoto image of each image tile. The orthophoto image of each image tile can constitute an orthophoto map of the key frame image.

[0096] In this embodiment, fusing each image tile and its orthophoto tile refers to fusing the image tile and its orthophoto tile, taking each image tile as a unit, to generate an orthophoto image of the image tile. After generating the orthophoto images of all image tiles, the orthophoto images of all image tiles are assembled to obtain an orthophoto map of the key image, thereby achieving an orthophoto map based on a high-definition satellite raster image. During the orthophoto image assembly process, the assembly can be performed according to the order in which the key frame image is segmented into image tiles.

[0097] S147: If the high-definition satellite raster image is a key frame image, the processing ends.

[0098] One point that needs to be explained here is that if image cropping and image filtering are performed by framing a high-definition satellite image to obtain multiple image blocks, then the high-definition satellite raster image includes each image block that constitutes the high-definition satellite image. Each image block can be processed according to the above steps S141 to S146 to obtain an orthophoto of each image block when each image block is used as a key frame image. After assembling the orthophotos of each image block, an orthophoto of the high-definition satellite raster image is obtained.

[0099] In some examples, the digital surface map can be generated by dense matching using image block matching technology and a semi-global matching algorithm, the process of which will not be described in this embodiment.

[0100] S15. Based on the orthophoto and digital surface map, combined with a pre-built high-definition satellite image library, identify the roof outline range of the high-definition satellite image, and extract the roof outline coordinate set Im from the roof outline range, where m is the number of roofs. The roof outline range is used to indicate the roof outline of the building.

[0101] In some examples, the process of identifying rooftop contours in high-definition satellite imagery can include: selecting a high-definition satellite comparison image from a pre-built high-definition satellite image library based on an orthophoto and a digital surface map; using the selected high-definition satellite comparison image to train a rooftop contour recognition model capable of identifying rooftop contours; inputting the high-definition satellite image into the rooftop contour recognition model to obtain an initial rooftop contour range in the high-definition satellite image, which is output by the model. The initial rooftop contour range indicates the approximate area where the rooftop contour is located in the high-definition satellite image; binarizing the high-definition satellite image using the initial rooftop contour range to obtain a binary raster image of the high-definition satellite image; and vectorizing the binary raster image of the high-definition satellite image to obtain a raster-vector map. The boundaries of the raster-vector map are optimized to obtain the rooftop contour range.

[0102] One way to select the high-definition satellite comparison image may be: for each pixel point in the digital surface map, using the coordinates of the pixel point, obtain the RGB information of the pixel point from the orthophoto; using the RGB information of all pixels obtained from the orthophoto, obtain a high-definition satellite comparison image that matches the RGB information of these pixels from the high-definition satellite image library.

[0103] The HD satellite image library contains a large number of HD satellite comparison images. To improve efficiency, the HD satellite image library can be preliminarily screened using at least one of regional and stylistic characteristics. Then, based on orthophotos and digital surface maps, selection can be made from these preliminarily screened HD satellite comparison images. For example, when HD satellite images are captured, they may record regional information, allowing selection of HD satellite comparison images from the HD satellite image library whose regional characteristics match that information.

[0104] S16. Forming a plane equation (ie, a first plane equation) of the roof profile fitting plane for the roof profile coordinate set Im.

[0105] One feasible way is to determine the pixel coordinates of the corner points of the roof outline according to the roof outline coordinate set Im, which are (x min ,y min )、(x min ,y max )、(x max ,y min )、(x max ,y max ), calculate the plane equation corresponding to the roof outline fitting plane according to the pixel coordinates of the corner points.

[0106] In addition, when the roof contour range indicates that the roof contour is a polygon, the plane equation of the roof contour fitting plane can be calculated by selecting the pixel coordinates of multiple corner points. For example, the multiple corner points can be five or more, and the inflection points (also called vertices) of the polygon are generally selected as corner points.

[0107] One thing that needs to be explained here is that the roof outline coordinate set Im is a coordinate set of m roof outlines. For each roof outline, the pixel coordinates of the corner points of the roof outline can be determined, and the plane equation corresponding to the roof outline fitting plane can be calculated based on the focus pixel coordinates of the roof outline.

[0108] In some examples, in the process of calculating the plane equation of the roof profile fitting plane, the corrected pixel coordinates can be added, and the plane equation of the roof profile fitting plane is fitted using the least squares method based on the corner pixel coordinates and the corrected pixel coordinates. The corrected pixel coordinates can be obtained based on the SIFT (Scale-Invariant Feature Transform) algorithm, or the corrected pixel coordinates can be screened by setting a step size l. Specifically, a pixel coordinate is selected from the corner pixel coordinates as the initial pixel coordinate, and at least four corrected pixel coordinates around the initial pixel coordinate are determined based on the initial pixel coordinate and the set step size l. Assuming that the initial pixel coordinates are (x, y), the four corrected pixel coordinates determined are (x+l, y+l), (x+l, yl), (xl, y+l), and (xl, yl). The set step size l can be set in advance, and the value is not limited.

[0109] S17. Correcting the plane equation of the roof profile fitting plane to obtain a corrected plane equation of the roof profile fitting plane (ie, a second plane equation).

[0110] In some examples, the plane equation of the roof profile fitting plane is corrected using the corrected pixel coordinates. In some examples, the corrected pixel coordinates are obtained as follows:

[0111] Based on the pixel coordinates of the corner points, the pixel coordinates of multiple points (x1, y1), (x2, y2)…(x i ,y i ); Substitute any extended pixel coordinate into the plane equation to obtain the calculation result of the extended pixel coordinate, and calculate the error value of the calculation result. When it is judged that the error value is less than the standard error value, the extended pixel coordinate can be used as the corrected pixel coordinate, and the plane equation is corrected using the extended pixel coordinate.

[0112] In addition, when executing the above steps as the extended pixel coordinates of the corrected pixel coordinates, the collection can be terminated until the boundary point of the roof contour boundary is collected, that is, when the extended pixel coordinates of the corrected pixel coordinates are the coordinates of a boundary point, and the plane equation of the roof contour fitting plane corrected by using multiple extended pixel coordinates is the plane equation of the roof contour fitting plane.

[0113] S18. Using the plane equation of the corrected roof profile fitting plane, identify roof profile information on the high-definition satellite comparison image in the high-definition satellite image library, and obtain a roof profile plane model based on the identified roof profile information.

[0114] In this embodiment, the plane equation of the modified roof profile fitting plane is used to identify roof profile information in the HD satellite comparison image. Roof profile information, such as roof profile coordinates, is identified from the HD satellite comparison image. Furthermore, this embodiment can perform roof profile identification on HD satellite comparison images of different roof types to enrich the roof profile information.

[0115] After identifying the roof profile information of any high-definition satellite comparison image using the plane equation of the roof profile fitting plane, the identified roof profile information is compared with the roof profile information annotated in the high-definition satellite comparison image to determine the roof profile information used to train the roof profile plane model. If the identified roof profile information is identical or similar to the roof profile information in the annotated roof profile information, the identified roof profile information can be used to train the roof profile plane model.

[0116] S19, collect the RGB value of each pixel in the corrected roof profile fitting plane (r i ,g i ,b i ), forming a roof outline color set Kn, where n is the number of roofs, and the corrected roof outline fitting plane is the roof outline fitting plane corresponding to the plane equation of the corrected roof outline fitting plane.

[0117] The RGB value of each pixel (r i ,g i ,b i ) can be obtained by setting the step size for screening. For details, please refer to the process of obtaining the corrected pixel coordinates in step S16.

[0118] S20. Fitting the corrected roof profile to a plane using the roof profile color set Kn to obtain a roof profile comparison model.

[0119] S21. Compare the roof profile plane model and the roof profile comparison model using high-definition satellite comparison images in the high-definition satellite image library to obtain a roof information recognition model.

[0120] One feasible approach is to compare the roof profile plan model with the roof profile comparison model using a screening comparison algorithm. High-definition satellite comparison images are input into the roof profile plan model and the roof profile comparison model to obtain the roof profile coordinates output by each model. The error between the roof profile coordinates is calculated. If the error is within a standard deviation, the roof profile comparison model is used as the roof information recognition model. If the error is not within the standard deviation, the roof profile comparison model is not used as the roof information recognition model. The standard deviation value can be adjusted based on accuracy.

[0121] S22. Export a roof information recognition model. The roof information recognition model is used to recognize roof information from high-definition satellite images, such as identifying roof outline coordinates and roof color.

[0122] It can be seen from the above technical solutions that this application has the following beneficial effects:

[0123] The high-definition satellite image library can store high-definition satellite comparison images with different roof features. Using these high-definition satellite comparison images, the above method is used to obtain a roof information recognition model that can be applied to different roof features. The roof information of high-definition satellite images with different roof features is automatically recognized using this roof information recognition model, which greatly reduces the possibility of missed recognition or misidentification during the automatic recognition of roof information using high-definition satellite images, and significantly improves the accuracy of automatic recognition of roof information using high-definition satellite images.

[0124] In addition, the roof comparison model in the roof comparison model library can identify the characteristics of different buildings. In the process of obtaining the roof information recognition model, the roof comparison model can be called to extract the building characteristics of different buildings from the high-definition satellite raster image, so that the orthophoto can accurately record the RGB information of different buildings and the digital surface map can accurately record the coordinate information of different buildings; the orthophoto and the digital surface map can be used to accurately identify the roof contour range of the high-definition satellite image, improve the accuracy of the roof contour range, and then extract the roof contour coordinate set from the roof contour range, and use the above steps S16 to S22 to obtain the roof information recognition model, so that the roof information recognition model can learn the roof contour information of different buildings, so that the roof information recognition model can automatically recognize the roof information of high-definition satellite images with different roof characteristics, greatly reducing the occurrence of missed recognition or misrecognition in the process of automatic recognition of roof information using high-definition satellite images, and significantly improving the accuracy of automatic recognition of roof information using high-definition satellite images.

[0125] like Figure 4 and Figure 5 As shown, Figure 4 (1) is the collected high-definition satellite image. The high-definition satellite image does not have roof information marked. The roof information recognition model is used to process the high-definition satellite image to obtain roof information. Figure 4 The recognition result shown in (2) enables the high-definition satellite image to mark the roof information of each building. Similarly, the above roof information recognition model is used to identify Figure 5 The high-definition satellite image shown in (1) is processed to identify roof information, so that the high-definition satellite image can be Figure 5 As shown in (2), the roof information of each building is marked, and the roof information of high-definition satellite images with different roof features can be automatically recognized by using the roof information recognition model.

[0126] For the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0127] See Figure 6, which shows an optional structure of an automatic roof information recognition device based on high-definition satellite images provided in an embodiment of the present application, which may include: an image processing module 10, a contour recognition module 20, a plane equation processing module 30, a roof contour plane model acquisition module 40, a roof contour comparison model acquisition module 50, a roof information recognition model acquisition module 60 and an export module 70.

[0128] The image processing module 10 is used to collect high-definition satellite images without roof information labeled, pre-process the high-definition satellite images to obtain high-definition satellite raster images, and obtain orthophotos and digital surface maps based on the high-definition satellite raster images.

[0129] In some examples, the image processing module 10 pre-processes the high-definition satellite imagery by performing image filtering and image cropping on the high-definition satellite imagery. Image filtering is used to eliminate noise in the high-definition satellite imagery, and image cropping is used to remove interfering contours in the high-definition satellite imagery. Image filtering and image cropping are performed before the other. Furthermore, the image processing module 10 may also perform image framing on the high-definition satellite imagery, for example, before image filtering and image cropping.

[0130] In some examples, the image processing module 10 obtains the orthophoto map and the digital surface map in the following process:

[0131] A roof comparison model in a pre-built roof comparison model library is called to perform feature extraction on the high-definition satellite raster image to obtain key points in the high-definition satellite raster image, and landmark points matching the key points are determined in the high-definition satellite raster image. The roof comparison model is trained based on building features in at least one region, and different roof comparison models correspond to different building features. When the high-definition satellite raster image is determined to be a key frame image based on the landmark points, local map information corresponding to the key frame image is optimized based on preset spherical positioning system errors and reprojection errors. Based on the optimized local map information, the orthophoto of the key frame image is determined. When the orthophoto is greater than a preset threshold, the key frame image is projected onto the ground plane and the projected key frame image is divided into multiple image tiles. Each image tile and its orthophoto value tile are fused to generate an orthophoto image of each image tile. The orthophoto images of each image tile can form an orthophoto map of the key frame image. The high-definition satellite raster image is processed according to image block matching technology and a semi-global matching algorithm to obtain a digital surface image.

[0132] The contour recognition module 20 is configured to identify the roof contour range in the HD satellite image based on the orthophoto and digital surface map, in conjunction with a pre-built HD satellite image library, and extract a set of roof contour coordinates from the roof contour range. The HD satellite image library stores HD satellite comparison images with different roof features. The process by which the contour recognition module 20 identifies the roof contour range in the HD satellite image is as follows:

[0133] Based on the orthophoto and digital surface map, a high-definition satellite comparison image is selected from a pre-built high-definition satellite image library; the selected high-definition satellite comparison image is used to train a roof contour recognition model capable of recognizing the range of roof contours; the high-definition satellite image is input into the roof contour recognition model to obtain the initial roof contour range of the high-definition satellite image output by the roof contour recognition model; the high-definition satellite image is binarized using the initial roof contour range to obtain a binary raster map of the high-definition satellite image; the binary raster map of the high-definition satellite image is vectorized to obtain a raster vector map, and the boundary of the raster vector map is optimized to obtain the roof contour range.

[0134] The method of selecting a high-definition satellite comparison image from a pre-built high-definition satellite image library includes: for each pixel point in the digital surface map, using the coordinates of the pixel point to obtain the RGB information of the pixel point from the orthophoto map; and selecting a high-definition satellite comparison image that matches the RGB information of all pixels points from the high-definition satellite image library based on the RGB information of all pixels points obtained from the orthophoto map.

[0135] The plane equation processing module 30 is used to obtain a first plane equation of the roof profile fitting plane according to the roof profile coordinate set, and to modify the first plane equation of the roof profile fitting plane to obtain a second plane equation of the roof profile fitting plane.

[0136] For example, the plane equation processing module 30 determines the pixel coordinates of the corner points of the roof outline based on the roof outline coordinate set; and calculates a first plane equation for the roof outline fitting plane based on the pixel coordinates of the corner points. The correction process for the first plane equation includes: obtaining multiple extended pixel coordinates corresponding to the focus pixel coordinates based on the pixel coordinates of the corner points and a set constant value; substituting each extended pixel coordinate into the first plane equation to obtain a calculated result of the extended pixel coordinates, and calculating an error value of the calculated result; and when the error value is less than a standard error value, correcting the first plane equation using the extended pixel coordinates whose error value is less than the standard error value.

[0137] The roof profile plane model acquisition module 40 is used to identify roof profile information on the high-definition satellite comparison image using the second plane equation, and obtain a roof profile plane model based on the roof profile information.

[0138] The process of obtaining the roof contour plane model by the roof contour plane model acquisition module 40 includes: using the plane equation to identify the roof contour information in the high-definition satellite comparison image, and identifying the roof contour information from the high-definition satellite comparison image; comparing the identified roof contour information with the roof contour information in the roof information marked in the high-definition satellite comparison image to determine the roof contour information used to train the roof contour plane model; and using the determined roof contour information to train the roof contour plane model.

[0139] The roof outline comparison model acquisition module 50 is used to collect the pixel values ​​of each pixel point in the target roof outline fitting plane corresponding to the second plane equation, obtain a roof outline color set, and fit the target roof outline fitting plane through the roof outline color set to obtain a roof outline comparison model.

[0140] The roof information recognition model acquisition module 60 is used to compare the roof profile plane model and the roof profile comparison model using high-definition satellite comparison images to obtain a roof information recognition model.

[0141] For example, a high-definition satellite comparison image is input into a roof contour plane model and a roof contour comparison model to obtain the roof contour coordinates output by the roof contour plane model and the roof contour comparison model respectively; the error between the roof contour coordinates is calculated; if the error is within the standard deviation value range, the roof contour comparison model is used as a roof information recognition model; if the error is not within the standard deviation value range, it is prohibited to use the roof contour comparison model as a roof information recognition model.

[0142] The export module 70 is used to export a roof information recognition model, which is used to recognize roof information from high-definition satellite images.

[0143] An embodiment of the present application also provides an automatic roof information identification device, including: a processor and a memory; the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the automatic roof information identification device executes the above method.

[0144] An embodiment of the present application also provides a storage medium for storing a computer program, which is specifically used to implement the above method when executed.

[0145] It should be noted that the various embodiments in this specification may be described in a progressive manner, and the features described in the various embodiments may be interchanged or combined. Each embodiment focuses on the differences from other embodiments, and similarities between the various embodiments may be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simple, and relevant details may be referred to the description of the method embodiments.

[0146] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0147] The above description of the disclosed embodiments will enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

[0148] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for automatically identifying rooftop information based on high-definition satellite images, characterized in that: The method comprises: collecting high-definition satellite images without roof information labeled, preprocessing the high-definition satellite images to obtain high-definition satellite raster images, and obtaining orthophotos and digital surface maps based on the high-definition satellite raster images; Based on the orthophoto and the digital surface map, in combination with a pre-built high-definition satellite image library, identifying the roof outline range of the high-definition satellite image, and extracting a roof outline coordinate set from the roof outline range, wherein the high-definition satellite image library stores high-definition satellite comparison images with different roof features; Obtaining a first plane equation of the roof profile fitting plane according to the roof profile coordinate set, and correcting the first plane equation of the roof profile fitting plane to obtain a second plane equation of the roof profile fitting plane; Using the second plane equation, identifying roof profile information on the high-definition satellite comparison image, and obtaining a roof profile plane model based on the roof profile information; Collecting pixel values ​​of each pixel point in the target roof outline fitting plane corresponding to the second plane equation to obtain a roof outline color set, and fitting the target roof outline fitting plane using the roof outline color set to obtain a roof outline comparison model; Comparing the roof profile plane model and the roof profile comparison model using the high-definition satellite comparison image to obtain a roof information recognition model; The roof information recognition model is derived, and the roof information recognition model is used to recognize roof information of the high-definition satellite image.

2. The method according to claim 1, characterized in that The obtaining of an orthophoto and a digital surface map based on the high-definition satellite grid image comprises: Calling a roof comparison model from a pre-built roof comparison model library to perform feature extraction on the high-definition satellite raster image to obtain key points in the high-definition satellite raster image, and determining landmark points matching the key points in the high-definition satellite raster image, wherein the roof comparison model is trained based on building features of at least one region, and different roof comparison models correspond to different features of some buildings; When the high-definition satellite grid image is determined to be a key frame image based on the landmark points, optimizing the local map information corresponding to the key frame image based on a preset spherical positioning system error and a reprojection error; Determining the orthotropy of the key frame image based on the optimized local map information; When the orthotropy is greater than a preset threshold, projecting the key frame image onto a ground plane, and dividing the projected key frame image into a plurality of image tiles; fusing each image tile and the orthophoto value tile of each image tile to generate an orthophoto image of each image tile, wherein the orthophoto image of each image tile can constitute an orthophoto map of the key frame image; The high-definition satellite raster image is processed according to the image block matching technology and the semi-global matching algorithm to obtain the digital surface image.

3. The method according to claim 1, characterized in that The pre-processing of the high-definition satellite image comprises: Image filtering and image cropping are performed on the high-definition satellite image, wherein the image filtering is used to eliminate noise in the high-definition satellite image, and the image cropping is used to remove interference contours in the high-definition satellite image, and one of the image filtering and the image cropping is performed before the other.

4. The method according to claim 1, wherein The identifying of the roof outline range of the high-definition satellite image based on the orthophoto and the digital surface map in combination with a pre-built high-definition satellite image library includes: selecting a high-definition satellite comparison image from a pre-built high-definition satellite image library based on the orthophoto and the digital surface map; Using the selected high-definition satellite comparison images, a roof outline recognition model with the ability to recognize the range of roof outlines is trained; Inputting a high-definition satellite image into the roof outline recognition model to obtain an initial roof outline range of the high-definition satellite image output by the roof outline recognition model; performing binarization processing on the high-definition satellite image using the initial roof outline range to obtain a binarized raster image of the high-definition satellite image; Vectorization is performed on the binary raster map of the high-definition satellite image to obtain a raster vector map, and the boundary of the raster vector map is optimized to obtain the roof outline range.

5. The method according to claim 4, characterized in that The step of selecting a high-definition satellite comparison image from a pre-built high-definition satellite image library based on the orthophoto and the digital surface map comprises: For each pixel point in the digital surface map, using the coordinates of the pixel point, obtain the RGB information of the pixel point from the orthophoto map; By obtaining the RGB information of all pixels from the orthophoto, a high-definition satellite comparison image that matches the RGB information of all pixels is selected from the high-definition satellite image library.

6. The method according to claim 1, characterized in that Obtaining a first plane equation of the roof profile fitting plane according to the roof profile coordinate set includes: Determining pixel coordinates of corner points of the roof outline according to the roof outline coordinate set; A first plane equation of the roof outline fitting plane is calculated according to the corner point pixel coordinates.

7. The method according to claim 6, characterized in that The step of correcting the first plane equation of the roof profile fitting plane to obtain the second plane equation of the roof profile fitting plane includes: Obtaining multi-extended pixel coordinates corresponding to the corner point pixel coordinates according to the corner point pixel coordinates and a set constant value; Substituting each extended pixel coordinate into the first plane equation to obtain a calculation result of the extended pixel coordinate, and calculating an error value of the calculation result; When the error value is smaller than the standard error value, the first plane equation is corrected using the extended pixel coordinates whose error value is smaller than the standard error value.

8. The method according to claim 1, characterized in that The step of identifying roof profile information on the high-definition satellite comparison image using the second plane equation and obtaining a roof profile plane model based on the roof profile information includes: Using the plane equation to identify the roof outline information in the high-definition satellite comparison image, the roof outline information is identified from the high-definition satellite comparison image; Comparing the identified roof profile information with the roof profile information annotated in the high-definition satellite comparison image to determine the roof profile information for training the roof profile plane model; The roof profile plane model is trained using the determined roof profile information.

9. The method according to claim 1, characterized in that The step of comparing the roof profile plane model with the roof profile comparison model using the high-definition satellite comparison image to obtain a roof information recognition model includes: Inputting the high-definition satellite comparison image into the roof profile plane model and the roof profile comparison model to obtain roof profile coordinates outputted by the roof profile plane model and the roof profile comparison model respectively; calculating the error between the roof profile coordinates; If the error is within the standard deviation range, the roof profile comparison model is used as the roof information recognition model; If the error is not within the standard deviation range, it is prohibited to use the roof profile comparison model as the roof information recognition model.

10. A device for automatically identifying rooftop information based on high-definition satellite images, characterized in that: The device comprises: An image processing module is used to collect high-definition satellite images without roof information labeled, pre-process the high-definition satellite images to obtain high-definition satellite raster images, and obtain orthophotos and digital surface maps based on the high-definition satellite raster images; a contour recognition module for identifying the roof contour range of the high-definition satellite image based on the orthophoto and the digital surface map, and extracting a set of roof contour coordinates from the roof contour range in combination with a pre-built high-definition satellite image library, wherein the high-definition satellite image library stores high-definition satellite comparison images with different roof features; a plane equation processing module, configured to obtain a first plane equation of the roof profile fitting plane according to the roof profile coordinate set, and to modify the first plane equation of the roof profile fitting plane to obtain a second plane equation of the roof profile fitting plane; a roof profile plane model acquisition module, configured to identify roof profile information on the high-definition satellite comparison image using the second plane equation, and obtain a roof profile plane model based on the roof profile information; a roof outline comparison model acquisition module, configured to collect pixel values ​​of each pixel point in the target roof outline fitting plane corresponding to the second plane equation to obtain a roof outline color set, and fit the target roof outline fitting plane using the roof outline color set to obtain a roof outline comparison model; a roof information recognition model acquisition module, configured to compare the roof profile plane model and the roof profile comparison model using the high-definition satellite comparison image to obtain a roof information recognition model; The export module is used to export the roof information recognition model, and the roof information recognition model is used to recognize the roof information of the high-definition satellite image.

Images