A Vectorized Reasoning Method for Building Roof Structures

Abstract

This invention discloses a method for vectorized inference of building roof structures, belonging to the field of computer vision technology. The method includes: extracting buildings from aerial images using a pre-trained building semantic segmentation model to obtain semantic segmentation results; cropping and filling the aerial images based on the semantic segmentation results to obtain multiple sub-images containing individual buildings; inputting the sub-images into a pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the individual building's roof structure and merging them. In the semantic extraction stage, this invention utilizes soft masking filtering to expand the receptive field of spatial domain convolution, enhancing the ability to recognize and locate buildings of different sizes and shapes. Furthermore, it designs a decoder module for semantic edge signal transmission, using edge information mining to assist in building contour localization and extraction, thereby improving the accuracy of building extraction.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a vectorized reasoning method for building roof structure. Background Technology

[0002] Buildings are primary urban features, occupying large areas, undergoing frequent changes, and closely related to human activities. Structured 3D models of buildings offer advantages such as rich topological information, lightweight design, and ease of computation and analysis, making them well-suited for a wide range of applications, including urban land planning, photovoltaic design, and landscape visualization. They are a crucial component of digital twin cities and the construction of a realistic 3D China. Vectorized reasoning of building roof structures is a key step in structured building modeling, such as... Figure 1 As shown, the vectorized reasoning of building roof structure focuses on the extraction of internal geometric and topological information of the roof, which can better reflect the actual structural situation of the building and has important application value.

[0003] With the continuous development of deep learning, it has become possible to perform vector perception on building roofs in high-resolution aerial imagery while considering topological consistency, thereby restoring the roof structure over a large area. Existing deep learning-based methods typically extract all key corner points of the roof from the image and then infer the actual roof structure from several candidate line segments formed by these corner points. Considering the rich variety of ground features in aerial imagery and the complex geometric features and neighborhood relationships between buildings and other ground features, such technical solutions generally suffer from low corner point extraction efficiency, high computational load for structural inference, and low accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for vectorized reasoning of building roof structures, which can improve the accuracy of vector extraction of building roof structures from aerial images in complex scenes.

[0005] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0006] This invention provides a method for vectorized reasoning of building roof structures, comprising:

[0007] Buildings in aerial images are extracted using a pre-trained building semantic segmentation model to obtain semantic segmentation results. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission.

[0008] The aerial image is cropped and filled based on the semantic segmentation results of the building to obtain multiple sub-images containing individual buildings;

[0009] The sub-image is input into a pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the roof structure of a single building. The roof structure vectorized perception model includes a feature extractor module, a corner point generation module, and a topological structure inference module.

[0010] The vectorized topology maps of the roof structures of multiple individual buildings are merged to generate the vectorized topology map of the building roof structure in the aerial image.

[0011] Furthermore, the encoder module based on soft mask filtering includes a soft mask filter unit and an adaptive feature extractor unit;

[0012] The operation of the encoder module includes:

[0013] The aerial imagery is input into a soft mask filter unit to obtain multi-level feature maps. ;

[0014] The feature map The input is fed into the adaptive feature extractor unit to obtain the feature map. .

[0015] Furthermore, the aerial image is input into a soft mask filter unit to obtain multi-level feature maps. ,include:

[0016] Using the ResNet50 model, the aerial imagery is mapped to spatial dimensions of... , , and Multi-level feature maps ;in, The spatial dimensions of the aerial image;

[0017] feature map Frequency features are obtained by transferring the original space to the Fourier space through two-dimensional discrete cosine transform. ;

[0018] Through a convolution with a kernel size of 3×3, frequency features are obtained. Learning to obtain adaptive soft masks ;

[0019] The adaptive soft mask With frequency characteristics Perform dot product to enable adaptive frequency filtering;

[0020] The filtered features are then transferred back to the original space using the inverse discrete cosine transform to obtain multi-level feature maps. .

[0021] Furthermore, the adaptive feature extractor unit includes a triangular position encoding layer and six cascaded deformable self-attention encoding layers;

[0022] The feature map The input is fed into the adaptive feature extractor unit to obtain the feature map. ,include:

[0023] Feature maps of different scales The spatial dimension is flattened into a one-dimensional vector, and then channel concatenation is performed to obtain the feature sequence. Where 256 is the feature dimension, Input feature maps at various scales Space dimensions, Indicates the length of the concatenated feature sequence;

[0024] The feature sequence The input is processed by a triangular positional encoding layer to obtain a feature sequence containing positional information. ,

[0025] The feature sequence Inputting six cascaded deformable self-attention encoding layers yields the feature sequence. .

[0026] Furthermore, the decoder module based on semantic edge signal transmission includes a cross-layer interaction unit, two parallel CNN decoder units, and two parallel deformable cross-attention decoder units.

[0027] The deformable cross-attention decoder unit includes six cascaded deformable cross-attention decoding layers;

[0028] The operation of the decoder module includes:

[0029] The feature sequence Input a deformable cross-attention decoder unit to obtain the edge geometry information of the building. The initial feature map of the aerial image is input into another deformable cross-attention decoder unit to obtain high-level semantic information of the buildings. ;

[0030] The high-level semantic information and edge geometry information Inputting cross-layer interaction units yields semantic features that incorporate building edge information. and edge features that incorporate semantic information of buildings ;

[0031] The semantic features Input a CNN decoder unit to obtain the semantic segmentation result of the building; then process the edge features. Input another CNN decoder unit to obtain the edge extraction results of the building.

[0032] Furthermore, the cross-layer interaction unit adaptively embeds building edge geometric information into building semantic information and embeds building semantic information into building edge geometric information using the channel attention mechanism ECA;

[0033] The high-level semantic information and edge geometry information Inputting cross-layer interaction units yields semantic features that incorporate building edge information. and edge features that incorporate semantic information of buildings ,include:

[0034] ,

[0035] ,

[0036] ,

[0037] ,

[0038] in, Indicates global average pooling; Indicates the convolution operation; Indicates normalization;

[0039] The semantic features Input a CNN decoder unit to obtain the semantic segmentation results of the building, including:

[0040] ,

[0041] ,

[0042] ,

[0043] ,

[0044] in, Indicates upsampling; This indicates that the aerial imagery is generated using a feature extractor. Feature maps of the same scale; This indicates superposition along the channel direction; This indicates that the aerial imagery is generated using a feature extractor. Feature maps of the same scale; This indicates that the aerial image obtained by performing a convolution operation is... Feature maps of the same scale; express and Feature maps obtained through convolution and upsampling; express and Feature maps obtained through convolution and upsampling; express and Feature maps obtained through convolution; This represents the semantic segmentation result of the building.

[0045] Further, the sub-image is input into a pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the roof structure of a single building, including:

[0046] The feature extractor module performs multi-scale feature extraction on the sub-image to obtain a multi-scale feature sequence.

[0047] The multi-scale feature sequence is subjected to positional encoding and attention encoding through a triangular positional encoding layer and six cascaded deformable self-attention encoding layers to obtain the feature sequence. ;

[0048] Based on the feature sequence The corner generation module is used to obtain a predicted map of the roof corners of a single building. ;

[0049] Based on the prediction map of the roof corner of a single building The topology reasoning module obtains a vectorized topology map of the roof structure of a single building.

[0050] Furthermore, the feature extractor module includes a ResNet50 model with an embedded frequency domain learning strategy;

[0051] The feature extractor module performs multi-scale feature extraction on the sub-image, including:

[0052] Sub-image Inputting a ResNet50 model, the feature map is extracted through a 7×7 convolutional layer. ,in, Sub-image of high, Sub-image The width;

[0053] The spatial domain feature map is obtained through Discrete Fourier Transform. Transform to the frequency domain to obtain the real part of the frequency domain feature. and the virtual part ;

[0054] The real part of the frequency domain feature and the virtual part After global learning is performed by inputting the data into three convolutional layers, the real and imaginary feature maps after convolution are transformed back into the spatial domain using the inverse discrete Fourier transform to obtain the feature maps. ;

[0055] The feature map and feature map Stacked along the channel dimension, the input convolutional layer yields enhanced spatial domain features. ;

[0056] Enhanced spatial domain features The remaining part of the input ResNet50 model is used to extract high-level image features.

[0057] Furthermore, the corner generation module includes a deformable cross-attention decoder unit and a CNN decoder unit;

[0058] Based on the feature sequence The corner generation module is used to obtain a predicted map of the roof corners of a single building. ,include:

[0059] The sub-image is divided into multiple The size of the patch is mapped to a sequence of patch features;

[0060] The feature sequence is obtained by position encoding the feature sequence of the image patch through a triangular position encoding layer. ;

[0061] The feature sequence With characteristic sequence The features are input together into the deformable cross-attention decoder unit to obtain a feature sequence that captures high-level semantic information of corner points. ;

[0062] feature sequence After the spatial shape is transformed into a two-dimensional feature map, it is input into the CNN decoder unit to obtain the predicted map of the roof corner of a single building. .

[0063] Furthermore, the topology inference module includes two weight-shared deformable cross-attention decoder units;

[0064] Based on the prediction map of the roof corner of a single building The topology reasoning module obtains a vectorized topology map of the roof structure of a single building, including:

[0065] Prediction map of the roof corner of the single building On Connect any two points among the corner points to obtain Candidate structure lines;

[0066] The midpoint coordinates of each candidate structure line are positionally encoded using a triangular positional encoding layer to initialize the features of each candidate structure line, thus obtaining the initial feature sequence of the candidate structure lines. ;

[0067] The initial feature sequence of candidate structure lines With characteristic sequence A deformable cross-attention decoding layer is input together to perform initial screening of candidate structure lines and retain those with high confidence. Candidate structure lines, The feature sequence of the candidate structure lines is denoted as ;

[0068] The feature sequence Input one of the deformable cross-attention decoder units to perform geometric decoding, and input the feature sequence. With characteristic sequence They are then input together into another deformable cross-attention decoder unit to perform image decoding and obtain the confidence score for each candidate structure line;

[0069] A threshold is set to filter candidate structural lines with high confidence scores as predicted roof structural lines. A vectorized topology map of the roof structure is constructed by the endpoints and line segments of the selected structural lines.

[0070] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0071] This invention proposes a hierarchical vector reasoning method for building roof structures in aerial imagery, prioritizing semantic extraction followed by structural perception. In the semantic extraction stage, addressing the challenges of ambiguous building edge localization due to the diversity of shapes and sizes, significant intra-class differences, and low inter-class separability caused by spectral similarity between buildings and surrounding features (such as parking lots and roads), this invention proposes a building semantic segmentation model based on soft masking filtering and edge signal propagation. The model utilizes soft masking filtering to expand the receptive field of spatial domain convolution, enhancing the recognition and localization capabilities of buildings of different sizes and shapes. Simultaneously, a semantic edge signal propagation decoder is designed to assist in building contour localization and extraction by mining edge information, improving the accuracy of building extraction. In the vector perception stage of individual building roof structures, addressing the limitations of existing technologies, such as the limited receptive field of spatial domain convolution, which hinders the perception of overall geometric features and fails to provide accurate topological information for global structural reasoning, this invention constructs a frequency domain strategy that utilizes low-level geometric fragments to learn structurally relevant features holistically. This strategy fuses scattered geometric fragments at a shallow network layer, generating geometric features rich in structural information, thereby enabling topologically complete structural reasoning. Attached Figure Description

[0072] Figure 1 A schematic diagram illustrating the vectorization reasoning of a building's roof structure and the structural reconstruction of the building;

[0073] Figure 2 A flowchart illustrating a method for vectorized reasoning of building roof structures provided in an embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram illustrating the process of performing vector inference of building roof structure on an aerial image according to the vector inference method of building roof structure in an embodiment of the present invention.

[0075] Figure 4 This is a schematic diagram of the building semantic segmentation model in an embodiment of the present invention;

[0076] Figure 5 This is a schematic diagram of the adaptive feature extractor unit in an embodiment of the present invention;

[0077] Figure 6 This is a schematic diagram of the decoder module based on semantic edge signal transmission in an embodiment of the present invention;

[0078] Figure 7 This is a schematic diagram of the vectorized perception model of the roof structure in an embodiment of the present invention;

[0079] Figure 8 This is a schematic diagram of the feature extractor module in an embodiment of the present invention;

[0080] Figure 9 This is a schematic diagram of a vectorization inference device for a building roof structure provided in an embodiment of the present invention;

[0081] Figure 10 An internal structural diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0082] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The same reference numerals in the drawings indicate the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. The embodiments and specific features within the embodiments of this application are detailed descriptions of the technical solution of this application, and not limitations thereof. Where there is no conflict, the embodiments and technical features within the embodiments of this application can be combined with each other.

[0083] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0084] Example 1:

[0085] like Figures 2 to 8 As shown, this embodiment of the invention provides a vectorized reasoning method for building roof structures. Figure 2 This is a flowchart illustrating the vectorized reasoning method for the building's roof structure. This flowchart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the invention, different methods may be used, provided they do not conflict with each other. Figure 2 Complete the steps shown or described in the order indicated. Figure 3 This is a schematic diagram illustrating the process of performing vector inference of building roof structure on an aerial image using the building roof structure vector inference method of the present invention.

[0086] The building roof structure vectorization reasoning method provided in this embodiment can be applied to a terminal and can be executed by a building roof structure vectorization reasoning device. This device can be implemented by software and / or hardware and can be integrated into the terminal.

[0087] See Figure 2 and Figure 3 The method of this invention specifically includes the following steps:

[0088] Step 1: Extract buildings from aerial images using a pre-trained building semantic segmentation model to obtain the semantic segmentation results of the buildings. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission.

[0089] like Figure 4 As shown, the semantic segmentation model of the building in this invention designs an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission. The former uses soft mask filtering to expand the receptive field of spatial domain convolution and enhances the ability to identify and locate buildings of different sizes and shapes. The latter assists in the localization and extraction of building contours by mining edge information, thereby improving the accuracy of building extraction.

[0090] The encoder module based on soft mask filtering includes a soft mask filter unit and an adaptive feature extractor unit.

[0091] The operation of the encoder module specifically includes steps 11 and 12, wherein:

[0092] Step 11: Input the aerial image into a soft mask filter unit to obtain multi-level feature maps. ;

[0093] Step 11 specifically includes steps 1101 to 1104, wherein:

[0094] Step 1101: This invention utilizes the ResNet50 model as a feature extractor to map the aerial imagery into spatially sized regions. , , and Multi-level feature maps ;in, The spatial dimensions of the aerial image;

[0095] Step 1102: Transfer the feature map Frequency features are obtained by transferring the original space to the Fourier space through two-dimensional discrete cosine transform. , is represented as:

[0096] ,

[0097] in, It is the discrete cosine transform.

[0098] Step 1103: Extract frequency features from a convolutional kernel of size 3×3. Learning to obtain adaptive soft masks , is represented as:

[0099] ,

[0100] in, It is a 3×3 convolution;

[0101] Step 1104: Apply the learned adaptive soft mask With frequency characteristics Dot multiplication is performed for adaptive frequency filtering. The filtered features are then transferred back to the original space using inverse discrete cosine transform to obtain multi-level feature maps. , is represented as:

[0102] ,

[0103] in, This is the inverse discrete cosine transform.

[0104] The convolution theorem states that convolution in the spatial domain is mathematically equal to the dot product in the corresponding Fourier domain. Therefore, the above steps are equivalent to using the tensor of the adaptive soft mask learned in the frequency domain and performing a discrete cosine inverse transform as the convolution kernel, and then convolving it with the feature map in the original domain. This can effectively expand the receptive field without increasing the amount of additional computation or network depth, avoid the network focusing too much on local feature information, and enable the network to perceive building features as a whole, while enhancing the ability to identify and locate buildings of different sizes.

[0105] Step 12: Transfer the feature map The input is fed into the adaptive feature extractor unit to obtain the feature map. .

[0106] The adaptive feature extractor unit of the present invention includes a triangular position encoding layer and six cascaded deformable self-attention encoding layers.

[0107] like Figure 5 As shown, in this embodiment of the invention, the adaptive feature extractor unit is composed of a triangular position coding layer and six structurally identical deformable self-attention coding layers connected in series. Each deformable self-attention coding layer consists of two sub-layer connection structures. The first sub-layer includes a deformable multi-head self-attention layer, a residual connection and a normalization layer. The second sub-layer includes a feedforward fully connected layer, a residual connection and a normalization layer.

[0108] This invention utilizes triangular position coding layers and deformable self-attention coding layers in several places, in order to We will use examples to explain the structure of both.

[0109] The triangular position encoding layer is used to process the input feature map. Position encoding is performed, specifically including steps 1201 and 1203, wherein:

[0110] Step 1201: Convert feature maps of different scales The spatial dimension is flattened into a one-dimensional vector, and then channel concatenation is performed to obtain the feature sequence. Where 256 is the feature dimension, Input feature maps at various scales Space dimensions, Indicates the length of the concatenated feature sequence;

[0111] Step 1202: Process the feature sequence Perform triangular position encoding, including:

[0112] ,

[0113] ,

[0114] ,

[0115] in, For characteristic sequences The Middle Position encoding of each feature The positional codes make up the feature positional code sequence. ; For the first The planar position coordinates of each feature; The function encodes prior information about the order and implicitly encodes the relative distance between coordinates;

[0116] Step 1203: Transfer the feature sequence With positional encoding sequence The summation yields a feature sequence containing location information. :

[0117] .

[0118] On the other hand, with feature sequences As input to the deformable self-attention coding layer, the deformable self-attention coding layer is calculated as follows:

[0119] ,

[0120] ,

[0121] ,

[0122] in, This represents the feature sequence that is input to a deformable self-attention encoding layer; This indicates a fully connected operation; M represents the number of attention heads. Indicates the first Attention weights for each sampling point; This indicates that M* indivual Normalization to Within the interval, and the sum is 1; Indicates the first The positional offset of each sampling point relative to the reference point; Indicates the reference point in the feature sequence pixel coordinates on; and This represents the learnable parameter matrix.

[0123] like Figure 6 As shown, the decoder module based on semantic edge signal transmission includes a cross-layer interaction unit, two parallel CNN decoder units, and two parallel deformable cross-attention decoder units.

[0124] The operation of the decoder module includes steps 13 to 15, wherein:

[0125] Step 13: Transfer the feature sequence Input a deformable cross-attention decoder unit to obtain the edge geometry information of the building. The initial feature map of the aerial image is input into another deformable cross-attention decoder unit to obtain high-level semantic information of the buildings. .

[0126] In this invention, the two deformable cross-attention decoder units utilize multi-layered cascaded deformable cross-attention coding layers to learn the edge geometric information and high-level semantic information of buildings from the input features, respectively.

[0127] In this embodiment of the invention, the deformable cross-attention decoder unit is composed of a triangular position coding layer and six deformable cross-attention decoding layers with the same structure connected in series. Each deformable cross-attention decoding layer consists of two sub-layer connection structures. The first sub-layer includes a deformable multi-head cross-attention layer, a residual connection and a normalization layer. The second sub-layer includes a feedforward fully connected layer, a residual connection and a normalization layer.

[0128] This invention utilizes deformable cross-attention coding layers in several places to generate multi-scale features from input images. and query features initialized with relative position coordinates The structure of the deformable cross-attention coding layer will be explained using an example.

[0129] The calculation formula for the attention of each layer in the deformable cross-attention layer includes:

[0130] ,

[0131] ,

[0132] ,

[0133] ,

[0134] in, For query features, length is Initialize it using relative position coordinates. Indicates a fully connected operation. The query feature sequence is obtained through linear mapping. M represents the number of heads that receive attention. Indicates the first Attention weights for each sampling point; This indicates that M* indivual Normalization to Within the interval, and the sum is 1; Indicates the reference point in the feature sequence pixel coordinates on; Indicates the first Each sampling point relative to the reference point Position offset; This represents the feature sequence input to the attention encoding layer; and This represents the learnable parameter matrix.

[0135] Step 14: Transfer the high-level semantic information and edge geometry information Inputting cross-layer interaction units yields semantic features that incorporate building edge information. and edge features that incorporate semantic information of buildings .

[0136] The cross-layer interaction unit adaptively embeds building edge geometric information into building semantic information and embeds building semantic information into building edge geometric information using the channel attention mechanism ECA.

[0137] The high-level semantic information and edge geometry information Inputting cross-layer interaction units yields semantic features that incorporate building edge information. and edge features that incorporate semantic information of buildings ,include:

[0138] ,

[0139] ,

[0140] ,

[0141] ,

[0142] in, Indicates global average pooling; Indicates the convolution operation; This indicates normalization.

[0143] Channel attention mechanism ECA will input semantic feature maps and edge feature map The shapes are respectively transformed into Then, the sums are combined and global average pooling is performed to obtain a feature map of size . Subsequently, the size of the adaptive one-dimensional convolution kernel, set to 5, is calculated based on the number of channels in the feature map. This kernel is then used to perform convolution operations on the features of each channel in the feature map. Finally, the normalized weights are multiplied channel-by-channel by the original input feature map to obtain the weighted feature map, thus completing the mutual transfer of edge signals and semantic information.

[0144] Step 15: Integrate semantic features that incorporate building edge information Input a CNN decoder unit to obtain the semantic segmentation result of the building; then fuse the edge features with semantic information. Input another CNN decoder unit to obtain the edge extraction results of the building.

[0145] semantic features that incorporate building edge information Input a CNN decoder unit to obtain the semantic segmentation results of the building, including:

[0146] ,

[0147] ,

[0148] ,

[0149] ,

[0150] in, Indicates upsampling; This indicates that the aerial imagery is generated using a feature extractor. Feature maps of the same scale; This indicates superposition along the channel direction; This indicates that the aerial imagery is generated using a feature extractor. Feature maps of the same scale; This indicates that the aerial image obtained by performing a convolution operation is... Feature maps of the same scale; express and Feature maps obtained through convolution and upsampling; express and Feature maps obtained through convolution and upsampling; express and Feature maps obtained through convolution; This represents the semantic segmentation result of the building.

[0151] Edge features that incorporate semantic information The process of inputting another CNN decoder unit to obtain the edge extraction results of the building is exactly the same as the process of obtaining the semantic segmentation results of the building described above, and will not be repeated here.

[0152] Step 2: Based on the semantic segmentation results of the buildings, the aerial image is cropped and filled to obtain multiple sub-images containing individual buildings.

[0153] Step 2 specifically includes:

[0154] Step 21: Label the connected components of the binary graph of building semantic segmentation generated in Step 1, generate the minimum bounding rectangle for each connected component, and record the coordinates of the diagonals of these bounding rectangles in the graph.

[0155] Step 22: Use the bounding rectangle to crop the original aerial images one by one. In order to ensure that the cropped image contains complete buildings and reduce the impact of insufficient semantic segmentation accuracy, the boundary of the bounding rectangle needs to be expanded before cropping (e.g., the length and width are both increased by 12 pixels). The coordinate value of the center point of the rectangle remains unchanged after expansion.

[0156] Step 23: The cropped sub-images are of varying sizes. To facilitate subsequent vectorization reasoning of the roof structure of individual buildings, the cropped images need to be filled to a uniform size. Specifically, the sub-images are first scaled proportionally. The height and width of the sub-image are times, respectively. and The height and width of the target image are respectively and Then, add white pixels of equal width to the top, bottom, left, and right sides of the scaled image to make the image size reach [the desired size]. Meanwhile, the sub-image is located at the center of the image.

[0157] Step 3: Input the sub-image into the pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the roof structure of a single building. The roof structure vectorized perception model includes a feature extractor module, a corner generation module, and a topological inference module.

[0158] like Figure 7 As shown, the specific operations of step 3 include steps 31 to 34, wherein:

[0159] Step 31: Perform multi-scale feature extraction on the sub-image using the feature extractor module to obtain a multi-scale feature sequence.

[0160] like Figure 8 As shown, the main structure of the feature extractor module of the present invention is a ResNet50 model, and a frequency domain learning strategy is embedded in stage 0 of the ResNet50 model.

[0161] The feature extractor module performs multi-scale feature extraction on the sub-image, including:

[0162] Step 3101: Extract sub-images containing individual buildings Inputting a ResNet50 model, the feature map is extracted through a 7×7 convolutional layer. ,in, Sub-image of high, Sub-image The width;

[0163] Step 3102: Transform the spatial domain feature map using Discrete Fourier Transform. Transform to the frequency domain to obtain the real part of the frequency domain feature. and the virtual part The real part of the frequency domain feature and the virtual part Represented as:

[0164] ,

[0165] in, Represents the Discrete Fourier Transform;

[0166] Step 3103: Convert the real part of the frequency domain feature and the virtual part After global learning is performed by inputting the data into three convolutional layers, the real and imaginary feature maps after convolution are transformed back into the spatial domain using the inverse discrete Fourier transform to obtain the feature maps. ;

[0167] Step 3104: Transfer the feature map and feature map Stacked along the channel dimension, the input convolutional layer yields enhanced spatial domain features. ;

[0168] Step 3105: Enhance the spatial domain features The remaining part of the input ResNet50 model is used to extract high-level image features.

[0169] Step 32: Perform positional encoding and attention encoding on the multi-scale feature sequence through a triangular positional encoding layer and six cascaded deformable self-attention encoding layers to obtain the feature sequence. .

[0170] The network structure of the triangular position encoding layer and the deformable self-attention encoding layer has been explained in the above building semantic segmentation model, and will not be repeated here.

[0171] Specifically, the spatial dimension of the features output by the feature extractor module is flattened into a one-dimensional vector, and channel concatenation is performed. The concatenated feature sequence is then sequentially encoded using positional encoding and attention encoding to obtain a multi-scale feature sequence. .

[0172] Step 33: Based on the feature sequence The corner generation module is used to obtain a predicted map of the roof corners of a single building. .

[0173] The corner generation module of the present invention includes a deformable cross-attention decoder unit and a CNN decoder unit. The network structure of the deformable cross-attention decoder unit and the CNN decoder unit is the same as that in the building semantic segmentation model, and will not be described again here.

[0174] Based on the feature sequence The corner generation module is used to obtain a predicted map of the roof corners of a single building. Specifically, it includes:

[0175] Step 3301: Divide the sub-image into multiple The size of the map is plotted and mapped to a sequence of map features, which are the initial query features in deformable cross attention computation;

[0176] Step 3302: Perform positional encoding on the feature sequence of the image patch using a triangular positional encoding layer to obtain the feature sequence. ;

[0177] Step 3303: Transfer the feature sequence With characteristic sequence The features are input together into the deformable cross-attention decoder unit to obtain a feature sequence that captures high-level semantic information of corner points. ;

[0178] Step 3304: Transfer the feature sequence After the spatial shape is transformed into a two-dimensional feature map, it is input into the CNN decoder unit to obtain the predicted map of the roof corner of a single building. .

[0179] Step 34: Predict the roof corner points of individual buildings The topology reasoning module obtains a vectorized topology map of the roof structure of a single building.

[0180] The topology reasoning module of the present invention includes two deformable cross-attention decoder units with shared weights.

[0181] Step 34 specifically includes:

[0182] Step 3401: Predict the roof corner points of the individual building. On Connect any two points among the corner points to obtain Candidate structure lines;

[0183] Step 3402: The midpoint coordinates of each candidate structure line are positionally encoded using a triangular positional encoding layer to initialize the features of each candidate structure line, thus obtaining the initial feature sequence of the candidate structure lines. ;

[0184] Step 3403: Initialize the feature sequence of the candidate structure line. With characteristic sequence A deformable cross-attention decoding layer is input together to perform initial screening of candidate structure lines and retain those with high confidence. Candidate structure lines, The feature sequence of the candidate structure lines is denoted as ;

[0185] Step 3404: Transfer the feature sequence Input one of the deformable cross-attention decoder units to perform geometric decoding, and input the feature sequence. With characteristic sequence The two deformable cross-attention decoder units are input together to perform image decoding and obtain the confidence score of each candidate structure line. The two deformable cross-attention decoder units share weights in this step.

[0186] Step 3405: ​​Set a threshold to filter candidate structural lines with high confidence scores as predicted roof structural lines, and construct a vectorized topology map of the roof structure by connecting the endpoints and segments of the filtered structural lines.

[0187] Step 4: Merge the vectorized topology maps of the roof structures of multiple individual buildings to generate the vectorized topology map of the roof structures of the buildings in the aerial image.

[0188] The white borders of the remote sensing sub-images containing individual buildings are cropped, scaled back to their corresponding size in the original image, and translated to their positions in the original image. At the same time, based on the coordinate values ​​of the endpoints in this series of transformed vector topology maps, the vectorized topology maps of the roof structures of each individual building are merged to generate the final vectorized inference result of the roof structure.

[0189] Example 2:

[0190] Based on the same inventive concept as Embodiment 1, this embodiment of the invention also provides a building roof structure vectorization inference device for implementing the above-described building roof structure vectorization inference method. The solution provided by this device is similar to the solution described in the above-described method; therefore, the specific limitations in the embodiments of the building roof structure vectorization inference device provided below can be found in the limitations of the building roof structure vectorization inference method described above, and will not be repeated here.

[0191] like Figure 9 As shown, an embodiment of the present invention provides a vectorized reasoning device for building roof structures, comprising:

[0192] The building semantic segmentation module is used to extract buildings from aerial images using a pre-trained building semantic segmentation model to obtain the semantic segmentation results of the buildings. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission.

[0193] The sub-image acquisition module is used to crop and fill the aerial image based on the semantic segmentation results of the building to acquire multiple sub-images containing individual buildings;

[0194] A module for obtaining a vectorized topology map of a single building's roof structure is used to input the sub-image into a pre-trained vectorized roof structure perception model to obtain a vectorized topology map of the single building's roof structure. The vectorized roof structure perception model includes a feature extractor module, a corner point generation module, and a topology inference module.

[0195] The building roof structure vectorized topology map generation module is used to merge multiple individual building roof structure vectorized topology maps to generate the building roof structure vectorized topology map in the aerial image.

[0196] Example 3:

[0197] This invention also provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps of the building roof structure vectorization inference method described in the foregoing embodiments.

[0198] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0199] Example 4:

[0200] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the following method:

[0201] Buildings in aerial images are extracted using a pre-trained building semantic segmentation model to obtain semantic segmentation results. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission.

[0202] The aerial image is cropped and filled based on the semantic segmentation results of the building to obtain multiple sub-images containing individual buildings;

[0203] The sub-image is input into a pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the roof structure of a single building. The roof structure vectorized perception model includes a feature extractor module, a corner point generation module, and a topological structure inference module.

[0204] The vectorized topology maps of the roof structures of multiple individual buildings are merged to generate the vectorized topology map of the building roof structure in the aerial image.

[0205] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0206] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0207] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0208] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0209] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for vectorized reasoning of building roof structures, characterized in that, include: Buildings in aerial images are extracted using a pre-trained building semantic segmentation model to obtain semantic segmentation results. The building semantic segmentation model includes an encoder module based on soft mask filtering and a decoder module based on semantic edge signal transmission. The aerial image is cropped and filled based on the semantic segmentation results of the building to obtain multiple sub-images containing individual buildings; The sub-image is input into a pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the roof structure of a single building. The roof structure vectorized perception model includes a feature extractor module, a corner point generation module, and a topological structure inference module. The vectorized topology maps of the roof structures of multiple individual buildings are merged to generate the vectorized topology map of the roof structures of the buildings in the aerial image; The encoder module based on soft masking filtering includes a soft masking filter unit and an adaptive feature extractor unit to acquire feature maps. ; The decoder module based on semantic edge signal transmission includes a cross-layer interaction unit, two parallel CNN decoder units, and two parallel deformable cross-attention decoder units; the operation of the decoder module includes: The feature map Input a deformable cross-attention decoder unit to obtain the edge geometry information of the building. The initial feature map of the aerial image is input into another deformable cross-attention decoder unit to obtain high-level semantic information of the buildings. ; The high-level semantic information and edge geometry information Inputting cross-layer interaction units yields semantic features that incorporate building edge information. and edge features that incorporate semantic information of buildings ; The semantic features Input a CNN decoder unit to obtain the semantic segmentation result of the building; then process the edge features. Input another CNN decoder unit to obtain the edge extraction results of the building.

2. The vectorized reasoning method for building roof structures according to claim 1, characterized in that, The operation of the encoder module includes: The aerial imagery is input into a soft mask filter unit to obtain multi-level feature maps. ; The feature map The input is fed into the adaptive feature extractor unit to obtain the feature map. .

3. The vectorized reasoning method for building roof structures according to claim 2, characterized in that, The aerial image is input into a soft mask filter unit to obtain multi-level feature maps. ,include: Using the ResNet50 model, the aerial imagery is mapped to spatial dimensions of... , , and Multi-level feature maps ;in, The spatial dimensions of the aerial image; feature map Frequency features are obtained by transferring the original space to the Fourier space through two-dimensional discrete cosine transform. ; Through a convolution with a kernel size of 3×3, frequency features are obtained. Learning to obtain adaptive soft masks ; The adaptive soft mask With frequency characteristics Perform dot product to enable adaptive frequency filtering; The filtered features are then transferred back to the original space using the inverse discrete cosine transform to obtain multi-level feature maps. .

4. The vectorized reasoning method for building roof structures according to claim 2, characterized in that, The adaptive feature extractor unit includes a triangular position coding layer and six cascaded deformable self-attention coding layers; The feature map The input is fed into the adaptive feature extractor unit to obtain the feature map. ,include: Feature maps of different scales The spatial dimension is flattened into a one-dimensional vector, and then channel concatenation is performed to obtain the feature sequence. Where 256 is the feature dimension, Input feature maps at various scales Space dimensions, Indicates the length of the concatenated feature sequence; The feature sequence The input is processed by a triangular positional encoding layer to obtain a feature sequence containing positional information. , The feature sequence Inputting six cascaded deformable self-attention encoding layers yields the feature sequence. .

5. The vectorized reasoning method for building roof structures according to claim 4, characterized in that, The deformable cross-attention decoder unit includes six cascaded deformable cross-attention decoding layers.

6. The vectorized reasoning method for building roof structures according to claim 5, characterized in that, The cross-layer interaction unit adaptively embeds building edge geometric information into building semantic information and embeds building semantic information into building edge geometric information using the channel attention mechanism ECA. The high-level semantic information and edge geometry information Inputting cross-layer interaction units yields semantic features that incorporate building edge information. and edge features that incorporate semantic information of buildings ,include: ， ， ， ， in, Indicates global average pooling; Indicates the convolution operation; Indicates normalization; The semantic features Input a CNN decoder unit to obtain the semantic segmentation results of the building, including: ， ， ， ， in, Indicates upsampling; This indicates that the aerial imagery is generated using a feature extractor. Feature maps of the same scale This indicates superposition along the channel direction; This indicates that the aerial imagery is generated using a feature extractor. Feature maps of the same scale; This indicates that the aerial image obtained by performing a convolution operation is... Feature maps of the same scale; express and Feature maps obtained through convolution and upsampling; express and Feature maps obtained through convolution and upsampling; express and Feature maps obtained through convolution; This represents the semantic segmentation result of the building.

7. The vectorized reasoning method for building roof structures according to claim 6, characterized in that, The sub-images are input into a pre-trained roof structure vectorized perception model to obtain a vectorized topological map of the roof structure of a single building, including: The feature extractor module performs multi-scale feature extraction on the sub-image to obtain a multi-scale feature sequence. The multi-scale feature sequence is subjected to positional encoding and attention encoding through a triangular positional encoding layer and six cascaded deformable self-attention encoding layers to obtain the feature sequence. ; Based on the feature sequence The corner point generation module is used to obtain a predicted map of the roof corners of a single building. ; Based on the prediction map of the roof corner of a single building The topology reasoning module obtains a vectorized topology map of the roof structure of a single building.

8. The method for vectorized reasoning of building roof structures according to claim 7, characterized in that, The feature extractor module includes a ResNet50 model with an embedded frequency domain learning strategy; The feature extractor module performs multi-scale feature extraction on the sub-image, including: Sub-image Inputting a ResNet50 model, the feature map is extracted through a 7×7 convolutional layer. ,in, Sub-image of high, Sub-image The width; The spatial domain feature map is obtained through Discrete Fourier Transform. Transform to the frequency domain to obtain the real part of the frequency domain feature. and the virtual part ; The real part of the frequency domain feature and the virtual part After global learning is performed by inputting the data into three convolutional layers, the real and imaginary feature maps after convolution are transformed back into the spatial domain using the inverse discrete Fourier transform to obtain the feature maps. ; The feature map and feature map Stacked along the channel dimension, the input convolutional layer yields enhanced spatial domain features. ; Enhanced spatial domain features The remaining part of the input ResNet50 model is used to extract high-level image features.

9. The method for vectorized reasoning of building roof structures according to claim 7, characterized in that, The corner generation module includes a deformable cross-attention decoder unit and a CNN decoder unit; Based on the feature sequence The corner point generation module is used to obtain a predicted map of the roof corners of a single building. ,include: The sub-image is divided into multiple The size of the patch is mapped to a sequence of patch features; The feature sequence is obtained by position encoding the feature sequence of the image patch through a triangular position encoding layer. ; The feature sequence With characteristic sequence The features are input together into the deformable cross-attention decoder unit to obtain a feature sequence that captures high-level semantic information of corner points. ; feature sequence After the spatial shape is transformed into a two-dimensional feature map, it is input into the CNN decoder unit to obtain the predicted map of the roof corner of a single building. .

10. The vectorized reasoning method for building roof structures according to claim 7, characterized in that, The topology inference module includes two weight-shared deformable cross-attention decoder units; Based on the prediction map of the roof corner of a single building The topology reasoning module obtains a vectorized topology map of the roof structure of a single building, including: Prediction map of the roof corner of the single building On Connect any two points among the corner points to obtain Candidate structure lines; The midpoint coordinates of each candidate structure line are positionally encoded using a triangular positional encoding layer to initialize the features of each candidate structure line, thus obtaining the initial feature sequence of the candidate structure lines. ; The initial feature sequence of candidate structure lines With characteristic sequence A deformable cross-attention decoding layer is input together to perform initial screening of candidate structure lines and retain those with high confidence. Candidate structure lines, The feature sequence of the candidate structure lines is denoted as ; The feature sequence Input one of the deformable cross-attention decoder units to perform geometric decoding, and input the feature sequence. With characteristic sequence They are then input together into another deformable cross-attention decoder unit to perform image decoding and obtain the confidence score for each candidate structure line; A threshold is set to filter candidate structural lines with high confidence scores as predicted roof structural lines. A vectorized topology map of the roof structure is constructed by the endpoints and line segments of the selected structural lines.

Images