Building roof structure vectorization reasoning method

Abstract

The invention discloses a vectorization reasoning method for a building roof structure, and belongs to the technical field of computer vision. The method comprises the steps of extracting a building in an aerial image through a pre-trained building semantic segmentation model, obtaining a semantic segmentation result of the building, cutting and filling the aerial image according to the semantic segmentation result of the building, and obtaining a plurality of sub-images containing single buildings; and inputting the sub-images into a pre-trained roof structure vectorization perception model to obtain single building roof structure vectorization topological graphs, and merging the single building roof structure vectorization topological graphs. In the semantic extraction stage, soft mask filtering is utilized to expand the receptive field of spatial domain convolution, the recognition and positioning capability of buildings with different sizes and shapes is enhanced, a decoder module for semantic edge signal transmission is designed, edge information is mined to assist in building contour positioning and extraction, and the accuracy of building extraction is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to a vectorized reasoning method for a building roof structure. Background Art

[0002] Buildings are the main features in cities. They occupy a large area, are frequently updated, and are closely related to human activities. The structured 3D model of a building has the advantages of rich topological information, lightweight, and easy calculation and analysis. It can be well adapted to a wide range of application scenarios, such as urban land planning, photovoltaic design, landscape display, etc. It is an important part of the construction of digital twin cities and real 3D China. The vectorized reasoning of the building roof structure is a key part of the building structure modeling. Figure 1 As shown in the figure, the vectorized reasoning of the building roof structure focuses on the extraction of the internal geometric structure and topological information of the roof, which can better reflect the actual structural situation of the building and has important application value.

[0003] With the continuous development of deep learning, it has become possible to use deep neural networks to perform topologically consistent vector perception of building roofs in high-resolution aerial imagery, thereby recovering the roof structures of large areas. Existing deep learning-based methods typically first extract all key corner points of the roof from the imagery and then infer the true roof structure from several candidate line segments formed by these corner points. Given the rich variety of objects in aerial imagery and the complex geometric features and neighborhood relationships between buildings and other objects, these technical solutions generally suffer from low corner point extraction efficiency, high computational complexity in structural inference, and low accuracy. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a building roof structure vectorization reasoning method that can improve the accuracy of vector extraction of building roof structures in aerial images under complex scenes.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] The present invention provides a vectorized reasoning method for a building roof structure, comprising:

[0007] Extracting buildings from aerial images using a pre-trained building semantic segmentation model to obtain building semantic segmentation results, wherein the building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission;

[0008] The aerial image is cropped and filled according to the semantic segmentation result of the building to obtain multiple sub-images containing individual buildings;

[0009] Inputting the sub-image into a pre-trained roof structure vectorization perception model to obtain a vectorized topological map of the roof structure of a single building, wherein the roof structure vectorization perception model includes a feature extractor module, a corner point generation module, and a topological structure reasoning module;

[0010] The vectorized topological maps of the roof structures of the plurality of individual buildings are merged to generate a vectorized topological map of the roof structures of the buildings in the aerial image.

[0011] Furthermore, the soft mask filtering based encoder module includes a soft mask filter unit and an adaptive feature extractor unit;

[0012] The operations of the encoder module include:

[0013] The aerial image is input into the soft mask filter unit to obtain a multi-level feature map ;

[0014] The feature map Input to the adaptive feature extractor unit to obtain the feature map .

[0015] Furthermore, the aerial image is input into the soft mask filter unit to obtain a multi-level feature map ,include:

[0016] Using the ResNet50 model, the aerial images are mapped into spatial sizes of 、 、 and Multi-level feature maps ;in, is the spatial size of the aerial image;

[0017] The feature map The frequency characteristics are obtained by transferring the original space to Fourier space through two-dimensional discrete cosine transform ;

[0018] Through a convolution with a convolution kernel size of 3×3, the frequency features Learning to obtain adaptive soft masks ;

[0019] The adaptive soft mask and frequency characteristics Perform dot multiplication for adaptive frequency filtering;

[0020] The filtered features are transferred back to the original space through inverse discrete cosine transform to obtain multi-level feature maps .

[0021] Furthermore, the adaptive feature extractor unit includes a triangular position encoding layer and six cascaded deformable self-attention encoding layers;

[0022] The feature map Input to the adaptive feature extractor unit to obtain the feature map ,include:

[0023] Feature maps of different scales The spatial dimension of is flattened into a one-dimensional vector, and the channels are spliced ​​to obtain the feature sequence ; Among them, 256 is the feature dimension, Input feature maps of each scale The spatial dimensions, Indicates the length of the feature sequence after splicing;

[0024] The feature sequence Input the triangle position encoding layer for position encoding to obtain a feature sequence containing position information ,

[0025] The feature sequence Input 6 cascaded deformable self-attention encoding layers to obtain the feature sequence .

[0026] Furthermore, the semantic edge signal transmission-based decoder module includes a cross-layer interaction unit, two parallel CNN decoder units, and two parallel deformable cross-attention decoder units;

[0027] The deformable crisscross attention decoder unit includes 6 cascaded deformable crisscross attention decoding layers;

[0028] The operations of the decoder module include:

[0029] The feature sequence Input a deformable cross attention decoder unit to obtain the edge geometry information of the building ; Input the initial feature map of the aerial image into another deformable cross attention decoder unit to obtain high-level semantic information of the building ;

[0030] The high-level semantic information and edge geometry information Input the cross-layer interaction unit to obtain the semantic features that integrate the building edge information and edge features that incorporate building semantic information ;

[0031] The semantic features Input a CNN decoder unit to obtain the semantic segmentation result of the building; the edge features Input another CNN decoder unit to obtain the edge extraction result of the building.

[0032] Furthermore, the cross-layer interaction unit utilizes a channel attention mechanism (ECA) to adaptively embed building edge geometry information into building semantic information, and embed building semantic information into building edge geometry information;

[0033] The high-level semantic information and edge geometry information Input the cross-layer interaction unit to obtain the semantic features that integrate the building edge information and edge features that incorporate building semantic information ,include:

[0034] ,

[0035] ,

[0036] ,

[0037] ,

[0038] in, represents global average pooling; Represents the convolution operation; represents normalization;

[0039] The semantic features Input a CNN decoder unit to obtain the semantic segmentation results of the building, including:

[0040] ,

[0041] ,

[0042] ,

[0043] ,

[0044] in, represents upsampling; Indicates that the aerial image is generated and Feature maps of the same scale; Indicates superposition along the channel direction; Indicates that the aerial image is generated and Feature maps of the same scale; Represents the convolution operation on the aerial image obtained by Feature maps of the same scale; express and Feature map obtained by convolution and upsampling; express and Feature map obtained by convolution and upsampling; express and Feature map obtained by convolution; Represents the semantic segmentation result of the building.

[0045] Furthermore, the sub-image is input into a pre-trained roof structure vectorization perception model to obtain a vectorized topological map of the roof structure of a single building, including:

[0046] Performing multi-scale feature extraction on the sub-image by the feature extractor module to obtain a multi-scale feature sequence;

[0047] The multi-scale feature sequence is position-encoded and attention-encoded through a triangular position encoding layer and six cascaded deformable self-attention encoding layers to obtain a feature sequence ;

[0048] According to the feature sequence , through the corner point generation module, obtain the roof corner point prediction map of the single building ;

[0049] Based on the prediction diagram of the roof corner points of a single building ,Through the topology structure reasoning module, a vectorized topology map of the roof structure of a single building is obtained.

[0050] Furthermore, the feature extractor module includes a ResNet50 model embedded with a frequency domain learning strategy;

[0051] Performing multi-scale feature extraction on the sub-image by the feature extractor module includes:

[0052] Sub-image Input ResNet50 model and extract the feature map through a 7×7 convolution layer ,in, Represents a sub-image of high, Represents a sub-image width;

[0053] The spatial domain feature map is transformed into Convert to the frequency domain and get the real part of the frequency domain feature and the imaginary part ;

[0054] The real part of the frequency domain feature and the imaginary part After inputting the three convolutional layers for global learning, the real and imaginary feature maps after convolution are transformed back to the spatial domain using inverse discrete Fourier transform to obtain the feature map ;

[0055] The feature map and feature maps By stacking along the channel dimension, the input convolution layer obtains enhanced spatial domain features. ;

[0056] The enhanced spatial domain features The rest of the input is the ResNet50 model to extract high-level image features.

[0057] Furthermore, the corner point generation module includes a deformable cross attention decoder unit and a CNN decoder unit;

[0058] According to the feature sequence , through the corner point generation module, obtain the roof corner point prediction map of the single building ,include:

[0059] Divide the sub-image into multiple The tiles are of different sizes and mapped into tile feature sequences;

[0060] The position encoding of the block feature sequence is performed through the triangular position encoding layer to obtain the feature sequence ;

[0061] The feature sequence With characteristic sequence Input together into the deformable cross attention decoder unit to obtain a feature sequence that captures the high-level semantic information of the corner points ;

[0062] The feature sequence The spatial shape is transformed into a two-dimensional feature map and then input into the CNN decoder unit to obtain the predicted corner point map of the roof of the single building .

[0063] Furthermore, the topology inference module includes two weight-sharing deformable cross-attention decoder units;

[0064] Based on the prediction diagram of the roof corner points of a single building , through the topology structure reasoning module, a vectorized topology map of the roof structure of a single building is obtained, including:

[0065] Prediction map of roof corner points of the single building on Connect any two points among the corner points and we get candidate structural lines;

[0066] The midpoint coordinates of each candidate structure line are position-encoded through the triangular position encoding layer to initialize the features of each candidate structure line and obtain the initial feature sequence of the candidate structure line. ;

[0067] The initial feature sequence of the candidate structure line With characteristic sequence The candidate structure lines are input together into the deformable cross attention decoding layer to perform preliminary screening and retain those with high confidence. candidate structural lines, The feature sequence of candidate structural lines is recorded as ;

[0068] The feature sequence Input a deformable cross attention decoder unit to perform geometric decoding, and transform the feature sequence With characteristic sequence Input together into another deformable cross attention decoder unit to perform image decoding to obtain a confidence score for each candidate structure line;

[0069] A threshold is set to filter candidate structure lines with high confidence scores as predicted roof structure lines, and a vectorized topological map of the roof structure is constructed through the connectivity relationship between the endpoints and segments of the filtered structure lines.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] This paper proposes a hierarchical vector inference method for building roof structures in aerial imagery that "first extracts semantics, then perceives structure." In the building semantic extraction stage, addressing the issues of ambiguous building edge location due to the diverse shapes and sizes of buildings in aerial imagery, large intra-class variability, and low inter-class separability due to spectral similarity between buildings and surrounding features (such as parking lots and roads), this method proposes a building semantic segmentation model based on soft mask filtering and edge signal transfer. This model utilizes soft mask filtering to expand the receptive field of spatial domain convolution, enhancing the recognition and localization of buildings of varying sizes and shapes. Furthermore, a semantic edge signal transfer decoder is designed to aid in the location and extraction of building outlines by mining edge information, improving the accuracy of building extraction. In the vectorized perception stage of individual building roof structures, addressing the issues that existing technologies are often limited by the limited receptive field of spatial domain convolution, making it difficult to perceive overall geometric features and providing accurate topological information for global structural inference, this paper develops a frequency domain strategy that leverages low-level geometric fragments to holistically learn structure-related features. This strategy holistically fuses scattered geometric fragments at shallow layers of the network to generate geometric features rich in structural information, enabling topologically complete structural inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Schematic diagram of building roof structure vectorization reasoning and building structure reconstruction;

[0073] Figure 2 A schematic diagram of a flow chart of a vectorized inference method for a building roof structure provided by an embodiment of the present invention;

[0074] Figure 3 A schematic diagram of a process of performing building roof structure vector inference on an aerial image in accordance with a building roof structure vector inference method according to an embodiment of the present invention;

[0075] Figure 4 Schematic diagram of the structure of a building semantic segmentation model in an embodiment of the present invention;

[0076] Figure 5 Schematic diagram of the structure of the adaptive feature extractor unit in an embodiment of the present invention;

[0077] Figure 6 Schematic diagram of the structure of a decoder module based on semantic edge signal transmission in an embodiment of the present invention;

[0078] Figure 7 Schematic diagram of the structure of the roof structure vector perception model in an embodiment of the present invention;

[0079] Figure 8 Schematic diagram of the structure of the feature extractor module in an embodiment of the present invention;

[0080] Figure 9 A schematic diagram of a vectorized inference device for a building roof structure provided by an embodiment of the present invention;

[0081] Figure 10 This is a diagram of the internal structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. The same reference numerals in the drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations of the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0083] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0084] Example 1:

[0085] like Figures 2 to 8 As shown, an embodiment of the present invention provides a vectorized reasoning method for a building roof structure. Figure 2 This is a flow chart of the vectorized reasoning method for the roof structure of a building. This flow chart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different methods may be used without conflict. Figure 2 The steps shown or described are accomplished in the order shown. Figure 3 The figure is a flow chart of the building roof structure vector reasoning method of the present invention for performing building roof structure vector reasoning on an aerial image.

[0086] The building roof structure vectorized inference method provided in this embodiment can be applied to a terminal and can be executed by a building roof structure vectorized inference device, which can be implemented by software and / or hardware and can be integrated into a terminal.

[0087] See also Figure 2 and Figure 3 The method of the embodiment of the present invention specifically includes the following steps:

[0088] Step 1: Extract buildings from aerial images using a pre-trained building semantic segmentation model to obtain semantic segmentation results of the buildings. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission.

[0089] like Figure 4 As shown in the figure, the building semantic segmentation model of the present invention designs an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission. The former uses soft mask filtering to expand the receptive field of spatial domain convolution and enhance the recognition and positioning capabilities of buildings of different sizes and shapes. The latter assists in the positioning and extraction of building contours by mining edge information, thereby improving the accuracy of building extraction.

[0090] The soft mask filtering based encoder module includes a soft mask filter unit and an adaptive feature extractor unit.

[0091] The operation of the encoder module specifically includes steps 11 and 12, wherein:

[0092] Step 11: Input the aerial image into the soft mask filter unit to obtain a multi-level feature map ;

[0093] Step 11 specifically includes the following steps 1101 to 1104, wherein:

[0094] Step 1101: The present invention uses the ResNet50 model as a feature extractor to map the aerial image into spatial dimensions of 、 、 and Multi-level feature maps ;in, is the spatial size of the aerial image;

[0095] Step 1102: The feature map The frequency characteristics are obtained by transferring the original space to Fourier space through two-dimensional discrete cosine transform , expressed as:

[0096] ,

[0097] in, is the discrete cosine transform.

[0098] Step 1103: Convolution with a kernel size of 3×3 from the frequency feature Learning to obtain adaptive soft masks , expressed as:

[0099] ,

[0100] in, It is a 3×3 convolution;

[0101] Step 1104: Use the learned adaptive soft mask and frequency characteristics Perform point multiplication to perform adaptive frequency filtering, and transfer the filtered features back to the original space through inverse discrete cosine transform to obtain a multi-level feature map , expressed as:

[0102] ,

[0103] in, is the inverse discrete cosine transform.

[0104] The convolution theorem states that convolution in the spatial domain is mathematically equivalent to the corresponding point product in the Fourier domain. Therefore, the above steps are equivalent to using the tensor of the inverse discrete cosine transform of the adaptive soft mask learned in the frequency domain as the convolution kernel, and convolving it with the feature map in the original domain. This can effectively expand the receptive field without increasing the amount of additional computation and network depth, preventing the network from focusing too much on local feature information. It can perceive the characteristics of buildings as a whole, while enhancing the ability to recognize and locate buildings of different sizes.

[0105] Step 12: The feature map Input to the adaptive feature extractor unit to obtain the feature map .

[0106] The adaptive feature extractor unit of the present invention includes a triangular position encoding layer and 6 cascaded deformable self-attention encoding layers.

[0107] like Figure 5 As shown, in an embodiment of the present invention, the adaptive feature extractor unit is composed of a triangular position encoding layer and six deformable self-attention encoding layers with the same structure connected in series. Each deformable self-attention encoding layer consists of two sub-layer connection structures. The first sub-layer includes a deformable multi-head self-attention layer, a residual connection and a normalization layer, and the second sub-layer includes a feedforward fully connected layer, a residual connection and a normalization layer.

[0108] The present invention uses the triangle position coding layer and the deformable self-attention coding layer in many places. Take the structures of the two as examples to explain.

[0109] The triangle position encoding layer is used to encode the input feature map Performing position coding specifically includes steps 1201 and 1203, wherein:

[0110] Step 1201: Feature maps of different scales The spatial dimension of is flattened into a one-dimensional vector, and the channels are spliced ​​to obtain the feature sequence ; Among them, 256 is the feature dimension, Input feature maps of each scale The spatial dimensions, Indicates the length of the feature sequence after splicing;

[0111] Step 1202: The feature sequence Perform triangulated position encoding, including:

[0112] ,

[0113] ,

[0114] ,

[0115] in, is the feature sequence Middle The position encoding of the features, Position codes form a feature position code sequence ; For the The plane position coordinates of the features; The function encodes the order prior information and implicitly encodes the relative distance between coordinates;

[0116] Step 1203: Set the feature sequence Position coding sequence After adding, we get a feature sequence containing position information :

[0117] .

[0118] On the other hand, the characteristic sequence As the input of the deformable self-attention encoding layer, the deformable self-attention encoding layer is calculated as:

[0119] ,

[0120] ,

[0121] ,

[0122] in, Represents the feature sequence of the input deformable self-attention encoding layer; represents the fully connected operation; M is the number of attention heads; Indicates the The attention weight of each sampling point; Indicates that M* indivual Normalized to In the interval, and the sum is 1; Indicates the The position offset of each sampling point relative to the reference point; Indicates that the reference point is in the feature sequence Pixel coordinates on ; and represents the learnable parameter matrix.

[0123] like Figure 6 As shown, the decoder module based on semantic edge signal transmission includes a cross-layer interaction unit, two parallel CNN decoder units and two parallel deformable cross attention decoder units.

[0124] The operation of the decoder module includes the following steps 13 to 15, wherein:

[0125] Step 13: Sequence the features Input a deformable cross attention decoder unit to obtain the edge geometry information of the building ; Input the initial feature map of the aerial image into another deformable cross attention decoder unit to obtain high-level semantic information of the building .

[0126] Among them, the two deformable cross-attention decoder units of the present invention use multi-layer cascaded deformable cross-attention encoding layers to learn the edge geometry information and high-level semantic information of the building from the input features respectively.

[0127] In an embodiment of the present invention, the deformable cross-attention decoder unit is composed of a triangular position encoding layer and six deformable cross-attention decoding layers with the same structure connected in series. Each deformable cross-attention decoding layer consists of two sub-layer connection structures. The first sub-layer includes a deformable multi-head cross-attention layer, a residual connection and a normalization layer, and the second sub-layer includes a feed-forward fully connected layer, a residual connection and a normalization layer.

[0128] The present invention uses a deformable cross attention coding layer in many places to generate multi-scale features of the input image. and the query features initialized with relative position coordinates Take the deformable cross attention encoding layer as an example to explain the structure of the deformable cross attention encoding layer.

[0129] The calculation formula for each layer of attention in the deformable cross attention layer includes:

[0130] ,

[0131] ,

[0132] ,

[0133] ,

[0134] in, is the query feature, the length is , initialize it with relative position coordinates, represents a full connection operation, Obtain query feature sequence through linear mapping ; M is the number of attention heads; Indicates the The attention weight of each sampling point; Indicates that M* indivual Normalized to In the interval, and the sum is 1; Indicates that the reference point is in the feature sequence Pixel coordinates on ; Indicates the sampling points relative to the reference point Position offset; Represents the feature sequence input to the attention encoding layer; and represents the learnable parameter matrix.

[0135] Step 14: Translate the high-level semantic information and edge geometry information Input the cross-layer interaction unit to obtain the semantic features that integrate the building edge information and edge features that incorporate building semantic information .

[0136] The cross-layer interaction unit utilizes the channel attention mechanism ECA to adaptively embed the building edge geometry information into the building semantic information, and embed the building semantic information into the building edge geometry information.

[0137] The high-level semantic information and edge geometry information Input the cross-layer interaction unit to obtain the semantic features that integrate the building edge information and edge features that incorporate building semantic information ,include:

[0138] ,

[0139] ,

[0140] ,

[0141] ,

[0142] in, represents global average pooling; Represents the convolution operation; Indicates normalization.

[0143] The channel attention mechanism ECA inputs the semantic feature map and edge feature maps The shapes of After that, add and perform global average pooling, and the feature map size obtained is Subsequently, an adaptive one-dimensional convolution kernel of size 5 is calculated based on the number of channels in the feature map, and this convolution kernel is used to perform a convolution operation on the channel features of the feature map. Finally, the normalized weights are multiplied by the original input feature map channel by channel to obtain a weighted feature map, completing the mutual transmission of edge signals and semantic information.

[0144] Step 15: Integrate the semantic features of building edge information Input a CNN decoder unit to obtain the semantic segmentation result of the building; the edge features that integrate the semantic information are Input another CNN decoder unit to obtain the edge extraction result of the building.

[0145] The semantic features of the building edge information are integrated Input a CNN decoder unit to obtain the semantic segmentation results of the building, including:

[0146] ,

[0147] ,

[0148] ,

[0149] ,

[0150] in, represents upsampling; Indicates that the aerial image is generated and Feature maps of the same scale; Indicates superposition along the channel direction; Indicates that the aerial image is generated and Feature maps of the same scale; Represents the convolution operation on the aerial image obtained by Feature maps of the same scale; express and Feature map obtained by convolution and upsampling; express and Feature map obtained by convolution and upsampling; express and Feature map obtained by convolution; Represents the semantic segmentation result of the building.

[0151] The edge features that integrate semantic information The process of inputting another CNN decoder unit to obtain the edge extraction result of the building is exactly the same as the above-mentioned process of obtaining the semantic segmentation result of the building, which will not be repeated here.

[0152] Step 2: The aerial image is cropped and filled according to the semantic segmentation results of the building to obtain multiple sub-images containing individual buildings.

[0153] Step 2 specifically includes:

[0154] Step 21: Mark the connected domains of the building semantic segmentation binary map generated in step 1, generate the minimum bounding rectangle for each connected domain, and record the coordinates of the diagonal corners of these bounding rectangles in the map.

[0155] Step 22: Use the bounding rectangle to crop the original aerial image one by one. To ensure that the cropped image contains complete buildings and reduce the impact of insufficient semantic segmentation accuracy, it is necessary to expand the boundaries of the bounding rectangle before cropping (for example, expand the length and width by 12 pixels). The coordinate value of the center point of the rectangle remains unchanged after expansion.

[0156] Step 23: The sub-images obtained by cropping are of different sizes. In order to facilitate the subsequent vectorized reasoning of the roof structure of a single building, the cropped images need to be filled to a uniform size. Specifically, first scale the sub-images proportionally. times, where the height and width of the sub-image are and , the height and width of the target image are and ; Then add white pixels of equal width above and below or left and right of the scaled image to make the image size reach , and the sub-image is located at the center of the image.

[0157] Step 3: Input the sub-image into a pre-trained roof structure vectorization perception model to obtain a vectorization topology map of the roof structure of a single building. The roof structure vectorization perception model includes a feature extractor module, a corner point generation module and a topology structure reasoning module.

[0158] like Figure 7 As shown, the specific operations of step 3 include steps 31 to 34, wherein:

[0159] Step 31: Perform multi-scale feature extraction on the sub-image through the feature extractor module to obtain a multi-scale feature sequence.

[0160] like Figure 8 As shown, the main structure of the feature extractor module of the present invention is the ResNet50 model, and the frequency domain learning strategy is embedded in stage 0 of the ResNet50 model.

[0161] Performing multi-scale feature extraction on the sub-image by the feature extractor module includes:

[0162] Step 3101: Sub-image containing a single building Input ResNet50 model and extract the feature map through a 7×7 convolution layer ,in, Represents a sub-image of high, Represents a sub-image width;

[0163] Step 3102: Transform the spatial domain feature map by discrete Fourier transform Convert to the frequency domain and get the real part of the frequency domain feature and the imaginary part , the real part of the frequency domain feature and the imaginary part Expressed as:

[0164] ,

[0165] in, represents discrete Fourier transform;

[0166] Step 3103: The real part of the frequency domain feature and the imaginary part After inputting the three convolutional layers for global learning, the real and imaginary feature maps after convolution are transformed back to the spatial domain using inverse discrete Fourier transform to obtain the feature map ;

[0167] Step 3104: The feature map and feature maps By stacking along the channel dimension, the input convolution layer obtains enhanced spatial domain features. ;

[0168] Step 3105: Enhance the spatial domain features The rest of the input is the ResNet50 model to extract high-level image features.

[0169] Step 32: Perform position encoding and attention encoding on the multi-scale feature sequence through the triangular position encoding layer and 6 cascaded deformable self-attention encoding layers to obtain the feature sequence .

[0170] The network structures of the triangular position encoding layer and the deformable self-attention encoding layer have been explained in the above-mentioned building semantic segmentation model and will not be repeated here.

[0171] Specifically, the spatial dimension of the features output by the feature extractor module is flattened into a one-dimensional vector, and channel splicing is performed. The spliced ​​feature sequence is sequentially position-encoded and attention-encoded to obtain a multi-scale feature sequence. .

[0172] Step 33: According to the feature sequence , through the corner point generation module, obtain the roof corner point prediction map of the single building .

[0173] The corner point generation module of the present invention includes a deformable cross-attention decoder unit and a CNN decoder unit. The network structure of the deformable cross-attention decoder unit and the CNN decoder unit is the same as that in the building semantic segmentation model and will not be repeated here.

[0174] According to the feature sequence , through the corner point generation module, obtain the roof corner point prediction map of the single building , specifically including:

[0175] Step 3301: Divide the sub-image into multiple The tiles are sized and mapped into a sequence of tile features, which are the initial query features in the deformable cross attention calculation;

[0176] Step 3302: Position-encode the tile feature sequence through the triangle position coding layer to obtain a feature sequence ;

[0177] Step 3303: The feature sequence With characteristic sequence Input together into the deformable cross attention decoder unit to obtain a feature sequence that captures the high-level semantic information of the corner points ;

[0178] Step 3304: Sequence the feature The spatial shape is transformed into a two-dimensional feature map and then input into the CNN decoder unit to obtain the predicted corner point map of the roof of the single building .

[0179] Step 34: Predict the roof corner points of individual buildings ,Through the topology structure reasoning module, a vectorized topology map of the roof structure of a single building is obtained.

[0180] Among them, the topology structure reasoning module of the present invention includes two weight-sharing deformable cross-attention decoder units.

[0181] Step 34 specifically includes:

[0182] Step 3401: Predicting the roof corner points of the single building on Connect any two points among the corner points and we get candidate structural lines;

[0183] Step 3402: Position-encode the midpoint coordinates of each candidate structure line through the triangulated position encoding layer to initialize the features of each candidate structure line and obtain the initial feature sequence of the candidate structure line. ;

[0184] Step 3403: Initial feature sequence of candidate structural lines With characteristic sequence The candidate structure lines are input together into the deformable cross attention decoding layer to perform preliminary screening and retain those with high confidence. candidate structural lines, The feature sequence of candidate structural lines is recorded as ;

[0185] Step 3404: The feature sequence Input a deformable cross attention decoder unit to perform geometric decoding, and transform the feature sequence With characteristic sequence Input together into another deformable cross attention decoder unit to perform image decoding to obtain a confidence score for each candidate structure line. The two deformable cross attention decoder units in this step share weights;

[0186] Step 3405: ​​Set a threshold to filter candidate structural lines with high confidence scores as predicted roof structural lines, and construct a roof structure vector topology map based on the connectivity relationship between the endpoints and line segments of the filtered structural lines.

[0187] Step 4: Merge the vectorized topological maps of the roof structures of multiple individual buildings to generate a vectorized topological map of the roof structures of the buildings in the aerial image.

[0188] The white edges of the remote sensing sub-image containing the building units are cropped and scaled back to their corresponding size in the original image, and translated to the position in the original image. At the same time, according to the coordinate values ​​of the endpoints in this series of transformed vector topology maps, the vectorized topology maps of the roof structures of each individual building are merged to generate the final roof structure vectorized inference result.

[0189] Example 2:

[0190] Based on the same inventive concept as Example 1, this embodiment of the present invention also provides a vectorized inference device for implementing the aforementioned method for vectorized inference of building roof structures. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of the following embodiment of the vectorized inference device for building roof structures can be found in the aforementioned definition of the method for vectorized inference of building roof structures and will not be further elaborated here.

[0191] like Figure 9 As shown, an embodiment of the present invention provides a vectorized inference device for a building roof structure, comprising:

[0192] A building semantic segmentation module is used to extract buildings from aerial images using a pre-trained building semantic segmentation model to obtain building semantic segmentation results. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission;

[0193] A sub-image acquisition module is used to crop and fill the aerial image according to the semantic segmentation result of the building to obtain multiple sub-images containing individual buildings;

[0194] A single-building roof structure vectorized topology map acquisition module is used to input the sub-image into a pre-trained roof structure vectorized perception model to obtain a single-building roof structure vectorized topology map. The roof structure vectorized perception model includes a feature extractor module, a corner point generation module, and a topology structure reasoning module.

[0195] The building roof structure vectorized topology map generating module is used to merge the multiple single building roof structure vectorized topology maps to generate the building roof structure vectorized topology map in the aerial image.

[0196] Example 3:

[0197] The embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of the building roof structure vectorization inference method in the aforementioned embodiment are implemented.

[0198] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0199] Example 4:

[0200] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the following method:

[0201] Extracting buildings from aerial images using a pre-trained building semantic segmentation model to obtain building semantic segmentation results, wherein the building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission;

[0202] The aerial image is cropped and filled according to the semantic segmentation result of the building to obtain multiple sub-images containing individual buildings;

[0203] Inputting the sub-image into a pre-trained roof structure vectorization perception model to obtain a vectorized topological map of the roof structure of a single building, wherein the roof structure vectorization perception model includes a feature extractor module, a corner point generation module, and a topological structure reasoning module;

[0204] The vectorized topological maps of the roof structures of the plurality of individual buildings are merged to generate a vectorized topological map of the roof structures of the buildings in the aerial image.

[0205] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0206] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0207] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0209] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. A vectorized reasoning method for building roof structure, characterized by: include: Extracting buildings from aerial images using a pre-trained building semantic segmentation model to obtain building semantic segmentation results, wherein the building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission; The aerial image is cropped and filled according to the semantic segmentation result of the building to obtain multiple sub-images containing individual buildings; Inputting the sub-image into a pre-trained roof structure vectorization perception model to obtain a vectorized topological map of the roof structure of a single building, wherein the roof structure vectorization perception model includes a feature extractor module, a corner point generation module, and a topological structure reasoning module; The vectorized topological maps of the roof structures of the plurality of individual buildings are merged to generate a vectorized topological map of the roof structures of the buildings in the aerial image.

2. The building roof structure vectorization inference method according to claim 1, characterized in that: The encoder module based on soft mask filtering includes a soft mask filter unit and an adaptive feature extractor unit; The operations of the encoder module include: The aerial image is input into the soft mask filter unit to obtain a multi-level feature map ; The feature map Input to the adaptive feature extractor unit to obtain the feature map .

3. The building roof structure vectorization inference method according to claim 2, characterized in that: The aerial image is input into the soft mask filter unit to obtain a multi-level feature map ,include: Using the ResNet50 model, the aerial images are mapped into spatial sizes of 、 、 and Multi-level feature maps ;in, is the spatial size of the aerial image; The feature map The frequency characteristics are obtained by transferring the original space to Fourier space through two-dimensional discrete cosine transform ; Through a convolution with a convolution kernel size of 3×3, the frequency features Learning to obtain adaptive soft masks ; The adaptive soft mask and frequency characteristics Perform dot multiplication for adaptive frequency filtering; The filtered features are transferred back to the original space through inverse discrete cosine transform to obtain multi-level feature maps .

4. The building roof structure vectorization inference method according to claim 2, characterized in that: The adaptive feature extractor unit includes a triangular position encoding layer and 6 cascaded deformable self-attention encoding layers; The feature map Input to the adaptive feature extractor unit to obtain the feature map ,include: Feature maps of different scales The spatial dimension of is flattened into a one-dimensional vector, and the channels are spliced ​​to obtain the feature sequence ; Among them, 256 is the feature dimension, Input feature maps of each scale The spatial dimensions, Indicates the length of the feature sequence after splicing; The feature sequence Input the triangle position encoding layer for position encoding to obtain a feature sequence containing position information , The feature sequence Input 6 cascaded deformable self-attention encoding layers to obtain the feature sequence .

5. The building roof structure vectorization inference method according to claim 4, characterized in that: The decoder module based on semantic edge signal transmission includes a cross-layer interaction unit, two parallel CNN decoder units and two parallel deformable cross attention decoder units; The deformable crisscross attention decoder unit includes 6 cascaded deformable crisscross attention decoding layers; The operations of the decoder module include: The feature sequence Input a deformable cross attention decoder unit to obtain the edge geometry information of the building ; Input the initial feature map of the aerial image into another deformable cross attention decoder unit to obtain high-level semantic information of the building ; The high-level semantic information and edge geometry information Input the cross-layer interaction unit to obtain the semantic features that integrate the building edge information and edge features that incorporate building semantic information ; The semantic features Input a CNN decoder unit to obtain the semantic segmentation result of the building; the edge features Input another CNN decoder unit to obtain the edge extraction result of the building.

6. The building roof structure vectorization inference method according to claim 5, characterized in that: The cross-layer interaction unit utilizes the channel attention mechanism ECA to adaptively embed the building edge geometry information into the building semantic information, and embed the building semantic information into the building edge geometry information; The high-level semantic information and edge geometry information Input the cross-layer interaction unit to obtain the semantic features that integrate the building edge information and edge features that incorporate building semantic information ,include: ， ， ， ， in, represents global average pooling; Represents the convolution operation; represents normalization; The semantic features Input a CNN decoder unit to obtain the semantic segmentation results of the building, including: ， ， ， ， in, represents upsampling; Indicates that the aerial image is generated and Feature maps of the same scale, Indicates superposition along the channel direction; Indicates that the aerial image is generated and Feature maps of the same scale; Represents the convolution operation on the aerial image obtained by Feature maps of the same scale; express and Feature map obtained by convolution and upsampling; express and Feature map obtained by convolution and upsampling; express and Feature map obtained by convolution; Represents the semantic segmentation result of the building.

7. The building roof structure vectorization inference method according to claim 6, characterized in that: Inputting the sub-image into a pre-trained roof structure vectorization perception model to obtain a vectorized topological map of the roof structure of a single building, including: Performing multi-scale feature extraction on the sub-image by the feature extractor module to obtain a multi-scale feature sequence; The multi-scale feature sequence is position-encoded and attention-encoded through a triangular position encoding layer and six cascaded deformable self-attention encoding layers to obtain a feature sequence ; According to the feature sequence , through the corner point generation module, obtain the roof corner point prediction map of the single building ; Based on the prediction diagram of the roof corner points of a single building ,Through the topology structure reasoning module, a vectorized topology map of the roof structure of a single building is obtained.

8. The building roof structure vectorization inference method according to claim 7, characterized in that: The feature extractor module includes a ResNet50 model embedded with a frequency domain learning strategy; Performing multi-scale feature extraction on the sub-image by the feature extractor module includes: Sub-image Input ResNet50 model and extract the feature map through a 7×7 convolution layer ,in, Represents a sub-image of high, Represents a sub-image width; The spatial domain feature map is transformed into Convert to the frequency domain and get the real part of the frequency domain feature and the imaginary part ; The real part of the frequency domain feature and the imaginary part After inputting the three convolutional layers for global learning, the real and imaginary feature maps after convolution are transformed back to the spatial domain using inverse discrete Fourier transform to obtain the feature map ; The feature map and feature maps By stacking along the channel dimension, the input convolution layer obtains enhanced spatial domain features. ; The enhanced spatial domain features The rest of the input is the ResNet50 model to extract high-level image features.

9. The building roof structure vectorization inference method according to claim 7, characterized in that: The corner point generation module includes a deformable cross attention decoder unit and a CNN decoder unit; According to the feature sequence , through the corner point generation module, obtain the roof corner point prediction map of the single building ,include: Divide the sub-image into multiple The tiles are of different sizes and mapped into tile feature sequences; The position encoding of the block feature sequence is performed through the triangular position encoding layer to obtain the feature sequence ; The feature sequence With characteristic sequence Input together into the deformable cross attention decoder unit to obtain a feature sequence that captures the high-level semantic information of the corner points ; The feature sequence The spatial shape is transformed into a two-dimensional feature map and then input into the CNN decoder unit to obtain the predicted corner point map of the roof of the single building .

10. The building roof structure vectorization inference method according to claim 7, characterized in that: The topology inference module includes two weight-sharing deformable cross-attention decoder units; Based on the prediction diagram of the roof corner points of a single building , through the topology structure reasoning module, a vectorized topology map of the roof structure of a single building is obtained, including: Prediction map of roof corner points of the single building on Connect any two points among the corner points and we get candidate structural lines; The midpoint coordinates of each candidate structure line are position-encoded through the triangular position encoding layer to initialize the features of each candidate structure line and obtain the initial feature sequence of the candidate structure line. ; The initial feature sequence of the candidate structure line With characteristic sequence The candidate structure lines are input together into the deformable cross attention decoding layer to perform preliminary screening and retain those with high confidence. candidate structural lines, The feature sequence of candidate structural lines is recorded as ; The feature sequence Input a deformable cross attention decoder unit to perform geometric decoding, and transform the feature sequence With characteristic sequence Input together into another deformable cross attention decoder unit to perform image decoding to obtain a confidence score for each candidate structure line; A threshold is set to filter candidate structure lines with high confidence scores as predicted roof structure lines, and a vectorized topological map of the roof structure is constructed through the connectivity relationship between the endpoints and segments of the filtered structure lines.

Images