A graph structure extraction method

Abstract

This invention discloses a graph structure extraction method, comprising the following steps: extracting corner points and edge primitives of building structures from an image using a multi-scale context-aware graph primitive generation network to generate a structural key point feature response map; performing triangulation processing based on the structural key point feature response map to construct an initial graph structure with triangular units as nodes; optimizing the initial graph structure using a graph shrinkage optimization method oriented towards topological integrity, merging redundant nodes and edges to obtain the final vector representation of the building structure; this invention transforms a planar graph into a graph structure for optimization, achieving better results in terms of structural compactness and consistency.

Description

Technical Field

[0001] This invention relates to the field of image architecture technology, and more specifically to a method for extracting the structure of an image. Background Technology

[0002] Image-based building structure extraction is the process of automatically or semi-automatically identifying and extracting the outlines, boundaries, and structural information of buildings in images using computer vision and machine learning methods. The process begins with acquiring high-resolution image data, typically containing various buildings from urban and rural areas. Image processing techniques are then used to preprocess the images, enhancing building features and boundaries. Next, image segmentation, feature extraction, and machine learning classification methods are employed to identify and classify building regions within the image. Simultaneously, morphological processing and multi-scale analysis can be combined to optimize and refine the initial extraction results, yielding more accurate and complete extraction outcomes. In practical applications, building structure extraction often requires consideration of numerous factors, such as image lighting, shadows, occlusion, and the diversity and complexity of buildings. Therefore, it is crucial to select appropriate methods and techniques based on specific task requirements and data characteristics, and to perform parameter tuning and model training to achieve satisfactory building extraction results. Currently, mainstream building structure extraction methods include: image segmentation-based methods, and methods based on vector representation for contour extraction and topological prediction.

[0003] Image segmentation-based building structure extraction methods: When images contain complex scenes, numerous occlusions, or dense building clusters, image segmentation methods often struggle to accurately separate buildings from the background, leading to inaccurate and incomplete extraction results. Pixel-level occlusion can easily result in incomplete information; for example, when buildings are partially obscured by trees, vehicles, or other objects, image segmentation algorithms often fail to effectively distinguish between buildings and occluders, thus affecting the extraction outcome. Vector-based contour extraction methods: Vector data uses geometric elements such as points, lines, and surfaces to describe spatial objects like maps, buildings, and administrative divisions. Common algorithms include boundary tracing algorithms, which extract closed contour lines from vector data, and polygon fitting algorithms, which fit lines in vector data into closed polygon contours. These methods aim to extract building structures; however, contour extraction methods can only obtain the outermost polygonal loops of the building structure and cannot extract the complete structure. Topology prediction methods based on vector representation: This method predicts and reconstructs building structures by using the geometric shapes and topological relationships of features in vector data such as building outlines. These geometric shapes and topological relationships are typically represented as sets of elements such as points, lines, and surfaces. Through analysis and processing of vector data, the extracted geometric and topological relationship information is used to construct a prediction model. Commonly used models include machine learning models and deep learning models. These models can learn the spatial relationships between features in existing vector data to predict new relationships between features, thus completing the building extraction task. Topology prediction can extract structural edges for more complex scenes. However, existing extraction methods based on topology prediction suffer from weak structural integrity due to a lack of global regional information integration. Summary of the Invention

[0004] Purpose of the Invention: The purpose of this invention is to provide a graph structure extraction method that solves the problem of incomplete results caused by image segmentation techniques in reconstructing building structures (pixel-level masking easily leads to fragmented and incomplete results). It addresses the issue that contour extraction methods based on vector representations can only obtain the outermost polygonal loops of the building structure. It also solves the problem of weak structural integrity caused by the lack of global region information integration in graph structure prediction methods based on vector representations.

[0005] Technical Solution: The present invention provides a graph structure extraction method, comprising the following steps: extracting corner points and edge primitives of building structures from an input image using a multi-scale context-aware graph primitive generation network to generate a structural key point feature response map; performing triangulation processing based on the structural key point feature response map to construct an initial graph structure with triangular units as nodes; optimizing the initial graph structure using a graph shrinkage optimization method oriented towards topological integrity, merging redundant nodes and edges to obtain the final vector representation of the building structure.

[0006] Furthermore, the multi-scale context-aware graph primitive generation network includes: a structure extraction module for extracting region feature response maps, corner feature response maps, and edge feature response maps from the input image; and a rendering synthesis module for multi-scale feature fusion of the region feature response maps, corner feature response maps, and edge feature response maps based on a channel attention mechanism to generate structural key point feature response maps.

[0007] Furthermore, the structure extraction module includes: a region reconstruction network for extracting the feature response map of the main building area; a corner reconstruction network for extracting the feature response map of the building corners; and an edge reconstruction network for extracting the feature response map of the building edges.

[0008] Furthermore, the rendering and compositing module extracts features at different scales through multi-scale convolutional kernels and uses a channel attention mechanism to weight and fuse features at different scales, ultimately outputting a feature response map of structural key points.

[0009] Furthermore, triangulation connects the key points of the structure into a triangular mesh, with each triangular cell serving as a node in the graph structure, and adjacent triangular cells connected by edges.

[0010] Furthermore, the graph shrinkage optimization method includes: initial graph cohesion based on acute angle suppression and shrinkage based on relational edges; wherein, initial graph cohesion based on acute angle suppression is used to merge triangular nodes into polygonal nodes; and shrinkage based on relational edges is used to merge adjacent nodes according to the edge confidence value, thereby gradually optimizing the graph structure.

[0011] Furthermore, the edge confidence value is obtained by calculating the difference in heat values ​​between the region feature response map and the edge feature response map at the edge location.

[0012] Furthermore, a building contour closure constraint mechanism is introduced during the graph shrinkage process to impose a penalty term on non-closed contours, ensuring the integrity of the final structure.

[0013] Furthermore, the final building structure vector representation is a topological graph structure with polygonal regions as nodes and adjacent relationships as edges.

[0014] An electronic device according to the present invention includes a memory and a processor. The memory stores a computer program, and the processor executes the program to implement the steps of the method.

[0015] The present invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method.

[0016] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: This invention proposes a building structure extraction method based on region graph construction and graph shrinkage, RegionGraph. RegionGraph constructs a topological graph structure with room primitives as nodes from the vector reconstruction results, where edges correspond to the adjacency relationships between different room nodes. RegionGraph consists of two stages. In the first stage, contour structure key points are first obtained through primitive extraction. After triangulation based on the set of structural key points, triangular units are used as nodes to construct an initial graph structure. In the second stage, an acute angle suppression algorithm is used to shrink the graph based on the initial graph structure to obtain the final reconstruction result. The region-based topological graph construction approach effectively improves the regional completeness and accuracy of image building structure extraction. By constructing a graph shrinkage problem, structural optimization is effectively performed, and the final structural representation is obtained based on the initial graph aggregation based on acute angle suppression and the shrinkage strategy based on relational edges. Attached Figure Description

[0017] Figure 1 This is an overall framework diagram of the present invention; Figure 2 The diagram shown in this invention is a structural diagram of the ConPNet multi-scale context-aware graph primitive generation network. Figure 3 This invention relates to the Squeeze-Excitation module based on the channel attention mechanism. Figure 4 This is a graphical representation of the reconstruction results of the present invention; Figure 5 This is the graph shrinking process based on node merging in this invention. Detailed Implementation

[0018] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0019] like Figure 1 As shown, this embodiment of the invention provides a method for extracting the structure of a graph, including the following steps: (1) Multi-scale context-aware graph primitive generation network: In graph construction, key point information is first obtained through a key point feature response map regression task for graph initialization. To achieve the transformation from image to building structure primitive feature response map, a new neural network ConPNet is proposed. Given an image containing the main building... This necessarily includes inherent structural information that remains unchanged under different environmental conditions, denoted by S: (1); Where V is the set of building corner points, and E is the set of building structural outline edges. A combination of a set of V and a set of E determines unique structural information. Since corner points and edges belong to different set categories, to more uniformly represent structural information, the concept of structural key points is used to fuse these two geometric primitives, approximating S as... The representation of: (2); Where P is a set of structural key points uniformly and discretely distributed along the edge of the building outline, and Therefore, for a given input image X, the task of ConPNet can be represented as: (3); in This is represented as the operation of extracting feature response maps. Indicates based on Kernelized feature response maps of point sets. Based on the above analysis and by fully utilizing the structural characteristics of different feature response maps, ConPNet will extract structural key point sets. Viewed as two steps—structure extraction and rendering compositing—the process can be represented as follows: (4); (5); in This represents the structure extraction operation, which aims to remove stylistic effects from the input image and restore its vector structural information. The effect of this step on the image is achieved through edge reconstruction operations. and corner reconstruction operation Remove environmental noise and facade textures from the input image and restore the building outline edges and corners. This step represents the rendering and compositing operation, which aims to discretely sample boundary structure information based on corner points. The effect on the image is that, based on the boundary feature response map, structural key points of the corresponding structure are generated by sampling according to the positions of the corner points. Since the reconstruction problem focuses more on the main building area, and directly extracting corner points and contour structures from the input is challenging, ConPNet incorporates prior geometric information of the region in the structure extraction operation. Therefore, the transformation process is represented as follows: (6); in Represents the region feature response map reconstruction operation.

[0020] Based on the above ideas, ConPNe is designed to consist of two parts: a structural feature response map reconstruction network and a rendering synthesis network. Its overall framework is as follows: Figure 1 As shown in the diagram. According to the process represented by formula (6), the overall network framework consists of two modules: structure extraction and rendering synthesis. In the structure extraction module, the input image first generates a region feature response map (R) through a region reconstruction network (RRN), and the input, output and extracted features are added together based on the residual idea as the input of the subsequent reconstruction network. Then, the edge feature response map network (B) and the corner feature response map (CP) are obtained through the edge reconstruction network (BRN) and the corner reconstruction network (PRN), respectively. Finally, in the rendering synthesis module, the various features extracted by the reconstruction network are connected through channels as the input of the rendering synthesis network to obtain the final result.

[0021] The structural details of the network module are as follows Figure 2 As shown. For a given image input The network first performs preliminary feature extraction through the backbone network to obtain feature blocks. The backbone network consists of a 3 × 3 convolutional layer, a residual network structure ConvBlock, and another 3 × 3 convolutional layer. The data is then fed into a region reconstruction network for further feature extraction. And obtain the region feature response map through the output head. R is then sampled through an upsampling block. Size of the region sensing feature block, and with and Adding them together gives As input to the subsequent structure extraction module. The corner sensing feature blocks are obtained by inputting them into the point reconstruction network and the edge reconstruction network, respectively. and edge sensing feature blocks . and Then, point feature response maps are obtained through the output head. and edge feature response map B In the rendering and compositing steps and Input is obtained through channel connection. The input is fed into the rendering synthesis network to obtain the structural key point feature response map. The following section will detail the specific structure of each network module involved in the structure extraction and rendering generation steps.

[0022] Structure Extraction: The main task of this step is to generate various primitive feature response maps representing architectural structural information from the input image. Since the [Hourglass] module has proven effective in previous multi-scale feature extraction and location localization tasks, the network modules in the structure extraction part mainly adopt the Hourglass-style network structure for feature extraction. The structural design of each network in structure extraction is described below: S1. Region Reconstruction Network (RNN): The purpose of the RRN is to remove the complex environmental information of the original input image from the initial feature blocks, extract the region-aware feature blocks corresponding to the main building, and generate a region feature response map. The RRN consists of a three-layer Hourglass module and an output head. The input is first processed by the Hourglass module to extract the region-aware feature blocks. The region feature response map R is obtained based on the output header (RegionHead). R is added to the network as prior geometric information of the region to constrain the generation of subsequent primitive feature response maps.

[0023] S2. Point Reconstruction Network (PRN): The purpose of the Point Reconstruction Network is to infer the distribution area of ​​corner points in the building outline from the fused feature blocks. Similar to RRN, PRN also consists of a three-layer Hourglass module and an output head. The fused feature blocks, incorporating prior information on region geometry, are first processed by the Hourglass module in the PRN to obtain corner-aware feature blocks. Meanwhile, the corner feature response map R is output through the output head (PointHead).

[0024] S3. Boundary Reconstruction Network (BRN): The purpose of the BRN is to infer the region corresponding to the building outline from the fused feature blocks. The structure of BRN is the same as that of PPN, which uses a three-layer Hourglass module to extract edge-aware feature blocks, and then outputs the edge feature response map B through the output head BoundaryHead.

[0025] Rendering and Synthesis: The main task of this step is to discretely sample the edge feature response map based on the corner feature response map to obtain the structural keypoint feature response map. ConPNet uses a Rendering and Synthesis Network (Fusion Network) to fit the rendering and synthesis operation. Its input is the corner-aware feature blocks and edge-aware feature blocks obtained from the structure extraction network, and its output is the structural keypoint feature response map. To better utilize the associated feature space information of the corner feature response map and the edge feature response map, a multi-scale perceptual structure based on an attention mechanism is designed in FN. Next, we will first introduce the channel attention method used in FN.

[0026] Channel attention is an attention mechanism that focuses on the importance of different channels in the input data. By weighting the features of different channels, the model can focus on the most important channels, thereby improving the accuracy and efficiency of the task. The idea behind channel attention is as follows: Figure 3 As shown, the overall process consists of three steps: Squeeze, Excitation, and Transform. For a given input, the channel attention module first extracts the global receptive field during the Squeeze phase using a global average pooling operation, abstracting all feature channels into a single point. Figure 3 In The process then proceeds to the Excitation step, where a two-layer fully connected layer performs a non-linear feature transformation to learn the correlation between different feature channels. Figure 3 In The operation then proceeds. Finally, in the Transform stage, the feature weights are standardized using the Sigmoid activation function. That is, through... This approach emphasizes important feature channels while de-emphasizing less important ones. Channel attention mechanisms have wide applications in various computer vision tasks, such as image classification, object detection, and image segmentation.

[0027] The rendering synthesis network uses a channel attention mechanism to fuse feature information at different scales. For example... Figure 2As shown, for a given feature block, the rendering synthesis network first inputs it into convolutional layers (12 layers in total) with kernel sizes of 3 × 3, 5 × 5, 7 × 7, ..., 25 × 25 to extract feature information at different scales. Larger feature scales help to obtain global structural information of the building, while sufficiently small feature scales help to obtain local structural details. Then, feature information at different scales is concatenated through channels to obtain overall structural awareness feature blocks. These blocks undergo average pooling for squeezing, and in the excitation step, fully connected layers learn the correlation between different feature channels. Finally, the feature weights are normalized using an activation function. The weights obtained above are then multiplied and added element-wise with the fused feature information, and processed by convolutional layers to output the structural keypoint feature response map (WP). The FN uses [AWL] and... Come to train Its loss function is shown in the formula: (7); (8); (9); in It is the ground truth of the feature response map of the structural key points. For the predicted structural keypoint feature response map, Indicates continuous loss. Represents the predicted boundary point set. The region segmentation results, A and C, are represented as follows: (10); (11); (2) Graph contraction optimization for topological integrity: Based on the idea of ​​region representation, the results of the multi-scale context-aware graph primitive generation network ConPNet are transformed into an initial graph structure representing vector structure information. In this graph, nodes represent polygonal regions, composed of a set of edges connecting the contours of the corresponding regions. An edge indicates whether the polygonal regions corresponding to two nodes are adjacent, consisting of two adjacent regions sharing an edge. Based on this vector representation, the reconstructed target result can be transformed into the corresponding region vector map structure, such as... Figure 4 As shown. Therefore, graph optimization is to simplify the initial graph structure to the target graph structure. For the initial graph structure G = {E, V}, the optimization objective is to minimize the energy term. Its expression is as follows: (12); Node item ( ): The node item measures the difference between a node and a complete, individual building roof component. It combines the region feature response map R and the edge feature response map B generated in the previous stage to comprehensively reflect the completeness of the vector polygon representation of a node as a component. Its expression is shown in formula (13): (13); (14); (15); in The penalty term for the feature response map is calculated as shown in formula (15), where Indicates the connection edge Length, This represents the number of edges in node v. Indicates the connection edge The heat value of position u on the regional characteristic response map R. Indicates the connection edge The position u is the heat value on the edge characteristic response map B. and They represent the edges respectively. The two endpoints on. Measuring the difference in the architectural outline corresponding to the connecting edges, connecting edges The smaller the value, the higher the probability that it is a structural line. When the polygonal region is closer to the complete building structure, the average value of its connected edges increases. The smaller the value; and when all nodes in the vector diagram structure are close to representing complete building components, the entire diagram structure... Minimum. Therefore, reduce graph structure. The process is one of continuously moving closer to the target reconstruction result.

[0028] Connecting terms ( The term "edge connection" measures the degree of region merging based on adjacency relationships. When two adjacent regions share an edge, the average... A smaller value indicates a higher probability that adjacent areas sharing a common edge correspond to the actual building outline structure line, meaning a higher probability that the two areas are adjacent. Conversely, a larger value indicates a higher probability that the two areas belong to the same component. When the overall vector diagram structure... The smaller the value, the greater the independence of each node in the structure, and the smaller the region merging degree. Therefore, reducing... The process is to continuously reduce the merging degree of the vector regions corresponding to the graph structure. The expression is shown in formula (16): (16); in This represents the relational edges in E. express The triangles contained within. Indicates the relationship between edges The number of sides of the triangle in the diagram.

[0029] The process of minimizing the penalty term described above is constructed as a graph contraction problem based on node merging. That is, nodes representing different regions of the same component are merged so that each individual node tends to represent the complete building component, and the edges connecting the nodes tend to represent the overall structural lines. The graph contraction process consists of two steps: initial graph condensation with acute angle suppression and a contraction strategy based on relational edges. These will be described in detail below.

[0030] A1: Initial graph condensation with acute angle suppression: Based on the concept of acute angle suppression, it's easy to see that a triangular node must belong to the same component region as one of its adjacent nodes. Therefore, to reduce the search space, in the first step of graph shrinking, the initial graph convergence method using acute angle suppression is used to merge triangular nodes into polygonal nodes. The idea is to merge nodes whose adjacent edges form the longest side of a triangle. For the initial graph... Where E is the edge set and V is the node set, the result after merging is The specific process is shown in the following formula: , (17; , (18); , (19); and The merged set of nodes and the set of edges are given by the following: Connect the adjacency relationships between nodes u and v. Let represent the longest connected edge in triangle t, T be the set of triangles after triangulation, and w be the new node formed by merging u and v.

[0031] A2: Shrinking strategy based on relational edges: Given an initial graph with acute angle suppression, the graph structure after agglomeration processing. The nodes have been transformed into polygons composed of multiple triangles. To further ensure that the region corresponding to each node represents a complete building roof component, the second stage of merging starts from the relational edges in the graph and merges nodes by removing relational edges that meet specific conditions. The specific process consists of three steps: Calculate the edge weights. Given a set of edges. For each of these relational edges Calculate the corresponding As the confidence value for connecting edges. The specific calculation formula is shown in formula (15).

[0032] Edge set update. Set a threshold. Edges with a confidence value greater than the threshold in the edge set are removed. The process can be described as follows: (20); Node update. For relational edges with weights greater than a threshold, merge the two connected nodes. This process can be described as follows: , (twenty one); in , express Adjacent nodes in the array An edge represents a connection between two nodes. Indicates merger , New nodes. In addition, a building outline closure constraint mechanism is introduced: (twenty two); in, Closed energy term / constraint value of building outline Represents the set of vertices. This represents a function for determining the state of a fixed-point closure. The penalty coefficient represents the non-closed contour. Figure 5 The overall contraction process is illustrated. Given an initial graph structure, the triangular regions are first merged based on an initial graph agglomeration strategy using acute angle suppression. Then, adjacent node regions are contracted based on the confidence values ​​of the edges connected by relationships. Since null background nodes were added during graph structure construction, these background nodes are finally removed to obtain the final vector structure representation.

Claims

1. A method for extracting the structure of a graph, characterized in that, Includes the following steps: A multi-scale context-aware graph primitive generation network extracts region, corner, and edge primitives of building structures from the input image to generate a structural key point feature response map. Triangulation is performed on the structural key point feature response map to construct an initial graph structure with triangular units as nodes; The initial graph structure is optimized using a graph shrinkage optimization method oriented towards topological integrity. Node merging is performed iteratively to output the final building structure topology graph with polygonal regions as nodes and adjacency relationships as edges.

2. The graph structure extraction method according to claim 1, characterized in that, The multi-scale context-aware graph primitive generation network includes: a structure extraction module for extracting region feature response maps, corner feature response maps, and edge feature response maps from the input image; and a rendering synthesis module for multi-scale feature fusion of the region feature response maps, corner feature response maps, and edge feature response maps based on a channel attention mechanism to generate structural key point feature response maps.

3. The graph structure extraction method according to claim 2, characterized in that, The structure extraction module includes: a region reconstruction network for extracting the feature response map of the main building area; a corner reconstruction network for extracting the feature response map of the building corners; and an edge reconstruction network for extracting the feature response map of the building edges.

4. The method for extracting the structure of a graph according to claim 2, characterized in that, The rendering and compositing module extracts features at different scales through multi-scale convolutional kernels and uses a channel attention mechanism to weight and fuse features at different scales, ultimately outputting a feature response map of structural key points.

5. The method for extracting the structure of a graph according to claim 1, characterized in that, Triangulation is the process of connecting key points of a structure into a triangular mesh, where each triangular cell serves as a node in the graph structure, and adjacent triangular cells are connected by edges.

6. The graph structure extraction method according to claim 1, characterized in that, The graph shrinkage optimization method is as follows: In the first stage, the initial triangular mesh graph is agglomerated based on the acute angle suppression strategy, and the triangular nodes are merged into polygonal nodes; in the second stage, the nodes connected by edges with confidence scores below a preset threshold are merged based on the edge confidence assessment.

7. The method for extracting the structure of a graph according to claim 6, characterized in that, The edge confidence value is obtained by calculating the difference in heat values ​​between the region feature response map and the edge feature response map at the edge location.

8. The method for extracting the structure of a graph according to claim 6, characterized in that, During the diagram shrinkage process, a building outline closure constraint mechanism is introduced to impose a penalty term on non-closed outlines, ensuring the integrity of the final structure.

9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the program to implement the steps of the method according to claims 1-8.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the method described in claims 1-8.

Images