Roof facet generation method for remote sensing image based on ridge-slope intermediate representation

Abstract

The application discloses a roof facet generation method based on ridge-slope intermediate representation of remote sensing images, relates to the field of real scene three-dimensional reconstruction, and solves the problems of high data acquisition cost and slow updating speed of the existing three-dimensional reconstruction, and comprises the following steps: S1, acquiring a standardized roof image block; S2, fusing a roof binary mask, a cornice boundary graph and a shielding confidence graph to form a roof semantic boundary representation; S3, generating a ridge-valley structure line confidence; predicting a pixel-level slope field, combining the roof semantic boundary representation, obtaining an initial roof partition, and constituting a ridge-slope intermediate representation; S4, generating an initial roof facet to form a structural expression of a candidate roof facet; analyzing the adjacency relationship between the roof facets; S5, performing consistency analysis on the candidate roof facets; and S6, generating a lightweight roof structure result of a target building; and the application can effectively improve the three-dimensional reconstruction updating speed and reduce the data cost.

Description

Technical Field

[0001] This invention belongs to the field of real-scene 3D reconstruction, specifically involving 3D generation technology for building roofs, and more specifically, a method for generating roof patches based on remote sensing images with a ridge-slope orientation representation. Background Technology

[0002] Current 3D reconstruction of building roofs mainly relies on lidar point clouds and multi-view oblique photography point clouds. Point cloud methods can recover roof height and roof plan well under high-density data conditions, but they are highly dependent on flight costs, point cloud density, and occlusion conditions. Specifically, the existing technology has the following main shortcomings: 1. Existing mesh deformation methods typically start from regular planar meshes and form the roof surface through vertex displacement. This method struggles to automatically adjust the number of ridges, valleys, and roof connections based on different buildings. When encountering L-shaped, T-shaped, combined pitched roofs, or partially attached roofs, it is prone to problems where the mesh surface is continuous but the ridge topology is incorrect.

[0003] 2. The roof texture, shadow and material color in remote sensing images cannot uniquely determine the roof height. If the model directly regresses the grid vertex height, it is easy to generate a roof that is visually smooth but geometrically does not conform to the real slope. In areas with roof shadows, tree occlusion, roof equipment occlusion or repeated roof textures, errors will propagate along the grid to adjacent roofs.

[0004] 3. Existing methods rely more on overall image features and local grid attention, and do not treat the ridge, valley line, eaves and aspect field as independent structural elements for prediction and constraint, resulting in problems such as broken roof intersection lines, discontinuous ridges, inconsistent aspects and unclosed eaves boundaries.

[0005] To address this, a method for generating roof patches from remote sensing images based on the ridge-slope aspect representation is proposed. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation. This invention aims to reduce the cost of 3D reconstruction and ensure modeling accuracy.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for generating roof patches based on remote sensing images of ridge-slope aspect representation, the specific working process of each step of which is as follows: Step S1: Acquire remote sensing images of the target building, extract the building outline from the remote sensing images, and crop the remote sensing images according to the building outline to obtain standardized roof image blocks. Step S2: Based on the standardized roof image blocks, extract multi-scale image features, segment the roof area, and obtain a roof binary mask; record the eaves boundary using the multi-scale image features to generate an eaves boundary map; simultaneously identify the occlusion status of the area, set an occlusion confidence level for the image, and generate an occlusion confidence map; fuse the roof binary mask, the eaves boundary map, and the occlusion confidence map to form a roof semantic boundary representation; Step S3: Based on the semantic boundary representation of the roof, combined with multi-scale image features, generate the confidence of the ridge-valley structure line; process the confidence of the ridge-valley structure line, extract candidate lines, record the line parameters of the candidate lines, merge the candidate lines, and generate a continuous structure line. The pixel-level aspect field is predicted using a binary roof mask. The roof is divided into initial roof partitions by combining the confidence of the ridge-valley structure line with the semantic boundary representation of the roof. The scale anchor points of the initial roof partitions are recorded to form the intermediate representation of the ridge-aspect. Step S4: Generate initial roof patches based on the initial roof zoning, continuous structural lines, and aspect field. From the initial roof patches, form the structural representation of candidate roof patches. Collect the two-dimensional boundary vertex set and geometric parameters of the candidate roof patches, analyze the adjacency relationship between the patches, and filter the patches. Step S5: Statistically analyze the candidate roof patches and construct a roof patch map; perform planar consistency and topological consistency analysis based on the roof patch map, and optimize the roof patch map; Step S6: Map the low-confidence regions to the roof patch map from the occlusion confidence map to generate candidate supplementary patches. Combine the roof patch map to generate the lightweight roof structure result of the target building.

[0008] Furthermore, the specific steps of step S1 are as follows: Step S11: Based on the remote sensing image of the target building, the building outline is acquired, the image boundary of the target building is determined, the boundary buffer is acquired based on the image boundary of the target image, and the remote sensing image is cropped by the image boundary of the target image and the boundary buffer to obtain the building image block; Step S12: Acquire image data from remote sensing images, map the remote sensing images to the world coordinate system based on the image data, extract the mapping information corresponding to the building image blocks, and obtain standardized roof image blocks.

[0009] Furthermore, the specific steps of step S2 are as follows: Step S21: Transmit the standardized roof image blocks to the shared visual encoder to capture roof edges, eaves lines, shadow boundaries, and texture abrupt changes, record feature regions and feature semantics to obtain low-level image features, identify the main roof area, roof material type, tree occlusion, roof equipment, and background area, collect pixel distributions of different areas, record corresponding feature semantics, and obtain high-level image features; perform statistical analysis on the low-level and high-level image features to obtain multi-scale image features; Step S22: Based on multi-scale image features, extract the main area of ​​the roof and generate a roof binary mask by combining it with the remote sensing image of the target building; based on multi-scale image features, identify the roof eaves line and generate an eaves boundary map; based on multi-scale image features, identify tree occlusion, shadow coverage, roof equipment, texture blurring, and image missing areas to obtain occlusion areas, assign occlusion confidence values ​​based on the occlusion area being lower than the image, and generate an occlusion confidence map; statistically analyze the roof binary mask, eaves boundary map, and occlusion confidence map to obtain the roof semantic boundary representation.

[0010] Furthermore, the specific steps of step S3 are as follows: Step S31: Based on the roof semantic boundary representation, obtain the eaves boundary map, obtain the roof semantic features from the multi-scale image features, traverse the roof eaves line from the eaves boundary map, extract the roof semantic features around the roof eaves line, and generate the confidence score of the ridge-valley structure line. Non-maximum suppression is applied to the confidence of the ridge-valley structure line to obtain candidate structure lines. The parameters of the candidate structure lines are collected, and the candidate structure lines are merged based on the line parameters to generate continuous structure lines. Step S32: Based on the roof binary mask, extract the roof range. Using a shared visual encoder, predict the pixel-level aspect field of the roof range to generate a two-dimensional unit vector, recording the cosine and sine values ​​of the orientation angles respectively. Combine continuous structural lines with the roof semantic boundary representation to divide the roof. Extract the roof outline using the roof semantic boundary representation. Divide the roof space into local roof spaces using continuous structural lines. Cluster the two-dimensional unit vectors of the corresponding pixels in each local roof space to classify the roof aspect and obtain the initial roof partition. Collect the roof height based on the initial roof partition to obtain the scale anchor points of the initial roof partition. The confidence level of the ridge-valley structural line, continuous structural line, aspect field, initial roof zoning, and scale anchor points are statistically analyzed to form an intermediate representation of the ridge-aspect.

[0011] Furthermore, the specific steps of step S31 are as follows: Step S311: Based on the confidence of the ridge-valley structure line, collect the confidence of the structure line at each pixel position. From the eaves boundary map, estimate the normal direction of the ridge-valley structure line. Search the left and right neighborhoods of the pixel along the normal direction. If the confidence of the current pixel is not the maximum value in the normal neighborhood, suppress and clear it to zero. Retain the peak value of the confidence in the normal direction to generate candidate structure lines. Step S312: Count the number of candidate structural lines as as; collect the line segment direction angles fxj1 to fxj for each candidate structural line. as Collect the endpoint positions ddw1 to ddw of each candidate structural line. as Angle threshold jdy and distance threshold jly are set, and consistency calculation is performed on candidate lines with different structures to obtain consistency evaluation value pgz(a1, a2). If the consistency evaluation value pgz(a1, a2) = 1, it indicates that the candidate structural lines are consistent, and the candidate structural lines will be merged. If the consistency evaluation value pgz(a1, a2) = 0, it indicates that the candidate structural lines are inconsistent, and the structural lines are traversed again.

[0012] Furthermore, the specific steps of step S4 are as follows: Step S41: Extract the roof area from the initial roof area partition, combine it with the continuous structure line, generate adjacent roof patches on both sides of the continuous structure line, count the generated adjacent roof patches to obtain the initial roof patch; input the initial roof patch into the Transformer decoder, aggregate the texture information, structure line information and orientation information of the roof area corresponding to the initial roof patch to form the structural representation of the candidate roof patch. Step S42: Based on the structural representation of the candidate roof panels, fit the boundaries of the roof panels to the continuous structural lines to generate closed candidate roof panels, and record the non-closed panels. Collect the two-dimensional boundary vertices of each candidate roof panel to obtain a set of two-dimensional boundary vertices. Collect the normal parameters, slope parameters, and elevation offset of the roof panel to obtain the geometric parameters. Step S43: Analyze the adjacency relationships between different candidate roof tiles using the two-dimensional boundary vertex set and geometric parameters of the candidate roof tiles, and store the analysis results; based on the adjacency relationships between different candidate roof tiles, combined with the slope aspect and shading confidence of the corresponding positions of the candidate roof tiles, filter the candidate roof tiles to obtain a set of candidate roof tiles.

[0013] Furthermore, the specific steps of step S43 are as follows: Step S431: Traverse the candidate roof tiles, collect the 2D boundary vertex set of the candidate roof tiles, generate the candidate roof tile boundaries, and determine the overlap of the candidate roof tile boundaries of different candidate roof tiles. If there is an overlap in the candidate roof tile boundaries, it indicates that the candidate roof tiles are generating a conflict. Collect the center points of the candidate roof tiles, divide the overlapping parts, and record the adjacency relationship of the corresponding candidate roof tiles as a conflict relationship. If there is an edge overlap in the candidate roof tile boundaries, record the adjacency relationship of the candidate roof tiles based on the overlapping edges; store all adjacency relationships. Step S432: Collect the slope direction and shading confidence of the corresponding location of the candidate roof pieces. Combine the adjacency relationship between different candidate roof pieces to make a consistency judgment on the candidate roof pieces that share the same edge. If the candidate roof piece has a shading condition and the slope direction of the candidate roof piece is inconsistent with that of the candidate roof piece that shares the same edge with the candidate roof piece, mark the candidate roof piece as an invalid candidate for screening.

[0014] Furthermore, the specific steps of step S5 are as follows: Step S51: Extract candidate roof patches from the candidate roof patch set, traverse the candidate roof patches as nodes, record the connecting edges between nodes based on the adjacency relationship between different candidate roof patches, and construct a roof patch graph. Step S52: Based on the roof patch map, perform planar consistency and topological consistency checks on the candidate roof patches; collect the geometric parameters and pixel-level slope field of the candidate roof patches, analyze the roof planar consistency, and correct the roof; judge the boundary overlap between adjacent patches based on the connection edges between nodes, and adjust the patches accordingly; optimize the boundary closure of the patches by combining the eaves boundary map; generate a roof patch map optimized for geometric consistency and topological consistency.

[0015] Furthermore, the specific steps of step S52 are as follows: Step S521: Obtain the geometric parameters and pixel-level slope field of each candidate roof piece. Extract the normal parameters of the candidate roof piece from the geometric parameters. Compare the normal parameters with the pixel-level slope field. If the angle direction shown by the normal parameters and the pixel-level slope field is consistent, it indicates that the candidate roof piece satisfies planar consistency. If the angle direction shown by the normal parameters and the pixel-level slope field is inconsistent, it indicates that there is a local anomaly in the candidate roof piece. Correct the roof by applying consistency constraints to the normal parameters and the pixel-level slope field at different locations of the candidate roof piece. Step S522: Collect adjacent patches from the connecting edges between nodes, record the adjacency relationship based on the connecting edges, use the adjacency relationship as a patch constraint, extract the topological relationship between adjacent patches, and optimize abnormal topology based on the adjacency relationship.

[0016] Furthermore, the specific steps of step S6 are as follows: Step S61: Based on the occlusion confidence map, uncertain areas caused by occlusion, shadows, or texture anomalies are collected to obtain low-confidence regions. The low-confidence regions are mapped to the roof patch map, and candidate roof patches involved in the low-confidence regions are extracted to obtain low-confidence candidate roof patches. The adjacent patches of the low-confidence candidate roof patches are collected, and the roof structure of the low-confidence candidate roof patches is inferred by combining the adjacency relationship between the low-confidence candidate roof patches and the adjacent patches. The inference parameters of the low-confidence candidate roof patches are recorded, and candidate supplementary patches are generated. Step S62: Based on the roof panel diagram, perform planar consistency and topological consistency checks on the candidate supplementary panels, verify the candidate supplementary panels, and statistically analyze the verified candidate supplementary panels and candidate roof panels to generate the lightweight roof structure result of the target building.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention divides the roof by using the confidence of the ridge-valley structure line and the semantic boundary of the roof, which can flexibly express the roof structure and is not limited by the fixed initial grid topology. It can better adapt to combined roofs, multi-slope roofs, L-shaped roofs and partial attached roofs. At the same time, it performs detailed analysis of the roof structure line based on the confidence of the structure line to ensure the accuracy of the roof topology.

[0018] 2. This invention records the semantic features of images through multi-scale image features, identifies regional occlusion conditions, generates an occlusion confidence map, and performs differentiated processing on low-confidence areas such as trees, shadows, and roof equipment to avoid erroneous texture features directly affecting the generation of roof structure.

[0019] 3. In the process of roof generation, this invention uses multiple constraints through roof zoning, continuous structural lines and slope field to ensure the accuracy and effectiveness of roof patch generation. Multiple verifications are performed during the generation process, and through planar consistency and topological consistency, the local accuracy and overall stability of the generated building are ensured, effectively reducing ridge breakage, slope confusion and roof boundary errors. Attached Figure Description

[0020] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the method of the present invention; Figure 2 This is a schematic diagram of the consistency analysis of the present invention; Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] This application provides a method for generating roof patches from remote sensing images based on the ridge-slope intermediate representation. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method for generating roof patches from remote sensing images based on the ridge-slope intermediate representation can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0024] Reference Figure 1 The diagram shown is a flowchart illustrating a method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation, according to an embodiment of the present invention. In this embodiment, the method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation includes: Step S1: Acquire remote sensing images of the target building, extract the building outline from the remote sensing images, and crop the remote sensing images according to the building outline to obtain standardized roof image blocks. Step S11: Based on the remote sensing image of the target building, the building outline is acquired, the image boundary of the target building is determined, the boundary buffer is acquired based on the image boundary of the target image, and the remote sensing image is cropped by the image boundary of the target image and the boundary buffer to obtain the building image block; It should be noted that the boundary buffer refers to the recording of the image outside the image boundary with a preset width.

[0025] Step S12: Acquire image data of remote sensing images (including pixel resolution, geographical location, and camera parameters), map the remote sensing images to the world coordinate system based on the image data, extract the mapping information corresponding to the building image blocks, and obtain standardized roof image blocks.

[0026] Step S2: Based on the standardized roof image blocks, extract multi-scale image features, segment the roof area, and obtain a roof binary mask; record the eaves boundary using the multi-scale image features to generate an eaves boundary map; simultaneously identify the occlusion status of the area, set an occlusion confidence level for the image, and generate an occlusion confidence map; fuse the roof binary mask, the eaves boundary map, and the occlusion confidence map to form a roof semantic boundary representation; Step S21: Transmit the standardized roof image blocks to the shared visual encoder to capture roof edges, eaves lines, shadow boundaries, and texture abrupt changes, record feature regions and feature semantics to obtain low-level image features, identify the main roof area, roof material type, tree occlusion, roof equipment, and background area, collect pixel distributions of different areas, record corresponding feature semantics, and obtain high-level image features; perform statistical analysis on the low-level and high-level image features to obtain multi-scale image features; It should be noted that the shared visual encoder is a convolutional neural network, a visual Transformer, or a combination of convolution and Transformer.

[0027] Step S22: Based on multi-scale image features, extract the main area of ​​the roof and generate a roof binary mask by combining it with the remote sensing image of the target building; based on multi-scale image features, identify the roof eaves line and generate an eaves boundary map; based on multi-scale image features, identify tree occlusion, shadow coverage, roof equipment, texture blurring, and image missing areas to obtain occlusion areas, assign occlusion confidence values ​​based on the occlusion area being lower than the image, and generate an occlusion confidence map; statistically analyze the roof binary mask, eaves boundary map, and occlusion confidence map to obtain the roof semantic boundary representation.

[0028] Step S3: Based on the semantic boundary representation of the roof, combined with multi-scale image features, generate the confidence of the ridge-valley structure line; process the confidence of the ridge-valley structure line, extract candidate lines, record the line parameters of the candidate lines, merge the candidate lines, and generate a continuous structure line. The pixel-level aspect field is predicted using a binary roof mask. The roof is divided into initial roof partitions by combining the confidence of the ridge-valley structure line with the semantic boundary representation of the roof. The scale anchor points of the initial roof partitions are recorded to form the intermediate representation of the ridge-aspect. Step S31: Based on the roof semantic boundary representation, obtain the eaves boundary map, obtain the roof semantic features from the multi-scale image features, traverse the roof eaves line from the eaves boundary map, extract the roof semantic features around the roof eaves line, and generate the confidence score of the ridge-valley structure line. It should be noted that: ridge lines usually correspond to the elevation intersection lines between roofs with different slopes, and valley lines usually correspond to the low-lying intersection lines between roofs with different slopes; by collecting multi-scale image features, collecting color changes, shadow directions, and texture direction differences to comprehensively determine the location of structural lines, the accuracy of structural line identification is improved.

[0029] Non-maximum suppression is applied to the confidence level of the ridge-valley structure line to obtain candidate structure lines. The parameters of the candidate structure lines are collected, and the candidate structure lines are merged based on the line parameters to generate continuous structure lines.

[0030] Step S311: Based on the confidence of the ridge-valley structure line, collect the confidence of the structure line at each pixel position. From the eaves boundary map, estimate the normal direction of the ridge-valley structure line. Search the left and right neighborhoods of the pixel along the normal direction. If the confidence of the current pixel is not the maximum value in the normal neighborhood, suppress and clear it to zero. Retain the peak value of the confidence in the normal direction to generate candidate structure lines. Step S312: Count the number of candidate structural lines as as; collect the line segment direction angles fxj1 to fxj for each candidate structural line. as Collect the endpoint positions ddw1 to ddw of each candidate structural line. as Angle threshold jdy and distance threshold jly are set, and consistency calculation is performed on candidate lines with different structures to obtain consistency evaluation value pgz(a1, a2). ; Among them, ||ddw a1 -ddw a2 || represents the Euclidean distance between the endpoints of the a1th and a2th structural candidate lines.

[0031] Where a1 represents the a1th structural candidate line, a2 represents the a2th structural candidate line, and pgz(a1, a2) is the consistency evaluation between the a1th and a2th structural candidate lines. If the consistency evaluation value pgz(a1, a2) = 1, it indicates that the candidate structural lines are consistent, and the candidate structural lines will be merged. If the consistency evaluation value pgz(a1, a2) = 0, it indicates that the candidate structural lines are inconsistent, and the structural lines are traversed again.

[0032] It should be noted that the angle threshold jdy and the distance threshold jly are set based on the building structure and image occlusion. The angle threshold merges line segments with the same direction, and its value is usually between 0 and 5°. The distance threshold merges line segments that are close to each other, and its value is between 0 and 10 centimeters.

[0033] Step S32: Based on the roof binary mask, extract the roof range. Using a shared visual encoder, predict the pixel-level aspect field of the roof range to generate a two-dimensional unit vector, recording the cosine and sine values ​​of the orientation angles respectively. Combine continuous structural lines with the roof semantic boundary representation to divide the roof. Extract the roof outline using the roof semantic boundary representation. Divide the roof space into local roof spaces using continuous structural lines. Cluster the two-dimensional unit vectors of the corresponding pixels in each local roof space to classify the roof aspect and obtain the initial roof partition. Collect the roof height based on the initial roof partition to obtain the scale anchor points of the initial roof partition. The confidence level of the ridge-valley structural line, continuous structural line, aspect field, initial roof zoning, and scale anchor points are statistically analyzed to form an intermediate representation of the ridge-aspect.

[0034] Step S4: Generate initial roof patches based on the initial roof zoning and continuous structural lines. From the initial roof patches, form the structural representation of candidate roof patches. Collect the two-dimensional boundary vertex set and geometric parameters of the candidate roof patches, analyze the adjacency relationship between the patches, and filter the patches to obtain the candidate roof patch set. Step S41: Extract the roof area from the initial roof area partition, combine it with the continuous structure line, generate adjacent roof patches on both sides of the continuous structure line, count the generated adjacent roof patches to obtain the initial roof patch; input the initial roof patch into the Transformer decoder, aggregate the texture information, structure line information and orientation information of the roof area corresponding to the initial roof patch to form the structural representation of the candidate roof patch. Step S42: Based on the structural representation of the candidate roof panels, fit the boundaries of the roof panels to the continuous structural lines to generate closed candidate roof panels, and record the non-closed panels. Collect the two-dimensional boundary vertices of each candidate roof panel to obtain a set of two-dimensional boundary vertices. Collect the normal parameters, slope parameters, and elevation offset of the roof panel to obtain the geometric parameters. Step S43: Analyze the adjacency relationships between different candidate roof tiles using the two-dimensional boundary vertex set and geometric parameters of the candidate roof tiles, and store the analysis results; based on the adjacency relationships between different candidate roof tiles, combined with the slope aspect and shading confidence of the corresponding positions of the candidate roof tiles, filter the candidate roof tiles to obtain a set of candidate roof tiles.

[0035] Step S431: Traverse the candidate roof tiles, collect the 2D boundary vertex set of the candidate roof tiles, generate the candidate roof tile boundaries, and determine the overlap of the candidate roof tile boundaries of different candidate roof tiles. If there is an overlap in the candidate roof tile boundaries, it indicates that the candidate roof tiles are generating a conflict. Collect the center points of the candidate roof tiles, divide the overlapping parts, and record the adjacency relationship of the corresponding candidate roof tiles as a conflict relationship. If there is an edge overlap in the candidate roof tile boundaries, record the adjacency relationship of the candidate roof tiles based on the overlapping edges; store all adjacency relationships. It should be noted that the adjacency relationship includes types such as shared edge adjacency, ridge adjacency, valley line adjacency, eaves adjacency, and spatial separation.

[0036] Step S432: Collect the slope direction and shading confidence of the corresponding location of the candidate roof pieces. Combine the adjacency relationship between different candidate roof pieces to make a consistency judgment on the candidate roof pieces that share the same edge. If the candidate roof piece has a shading condition and the slope direction of the candidate roof piece is inconsistent with that of the candidate roof piece that shares the same edge with the candidate roof piece, mark the candidate roof piece as an invalid candidate for screening.

[0037] Step S5: Statistically analyze the candidate roof patches and construct a roof patch map; perform planar consistency and topological consistency analysis based on the roof patch map, and optimize the roof patch map; Step S51: Extract candidate roof patches from the candidate roof patch set, traverse the candidate roof patches as nodes, record the connecting edges between nodes based on the adjacency relationship between different candidate roof patches, and construct a roof patch graph. Step S52: Based on the roof patch map, perform planar consistency and topological consistency checks on the candidate roof patches; collect the geometric parameters and pixel-level slope field of the candidate roof patches, analyze the roof planar consistency, and correct the roof; judge the boundary overlap between adjacent patches based on the connection edges between nodes, and adjust the patches accordingly; optimize the boundary closure of the patches by combining the eaves boundary map; generate a roof patch map optimized for geometric consistency and topological consistency.

[0038] Please see Figure 2 Step S521: Obtain the geometric parameters and pixel-level slope field of each candidate roof piece. Extract the normal parameters of the candidate roof piece from the geometric parameters. Compare the normal parameters with the pixel-level slope field. If the angle direction shown by the normal parameters and the pixel-level slope field is consistent, it indicates that the candidate roof piece satisfies planar consistency. If the angle direction shown by the normal parameters and the pixel-level slope field is inconsistent, it indicates that there is a local anomaly in the candidate roof piece. Correct the roof by applying consistency constraints to the normal parameters and the pixel-level slope field at different locations of the candidate roof piece. Step S522: Collect adjacent patches from the connecting edges between nodes, record the adjacency relationship based on the connecting edges, use the adjacency relationship as a patch constraint, extract the topological relationship between adjacent patches, and optimize abnormal topology based on the adjacency relationship.

[0039] It should be noted that: when adjacency is used as a face constraint, such as when the adjacency relationship is adjacent to the roof ridge, check whether the boundaries of adjacent faces coincide on the roof ridge line and adjust the positions of the shared edges.

[0040] Step S6: Map the low-confidence areas to the roof patch map from the occlusion confidence map to generate candidate supplementary patches. Combine the roof patch map to generate the lightweight roof structure result of the target building. Step S61: Based on the occlusion confidence map, uncertain areas caused by occlusion, shadows, or texture anomalies are collected to obtain low-confidence regions. The low-confidence regions are mapped to the roof patch map, and candidate roof patches involved in the low-confidence regions are extracted to obtain low-confidence candidate roof patches. The adjacent patches of the low-confidence candidate roof patches are collected, and the roof structure of the low-confidence candidate roof patches is inferred by combining the adjacency relationship between the low-confidence candidate roof patches and the adjacent patches. The inference parameters of the low-confidence candidate roof patches are recorded, and candidate supplementary patches are generated. Step S62: Based on the roof panel diagram, perform planar consistency and topological consistency checks on the candidate supplementary panels, verify the candidate supplementary panels, and statistically analyze the verified candidate supplementary panels and candidate roof panels to generate the lightweight roof structure result of the target building.

[0041] Compared to the problems described in the background art, this invention divides the roof by using the confidence level of the ridge-valley structural line and the semantic boundary representation of the roof, flexibly expressing the roof structure without being limited by a fixed initial grid topology. This allows for better adaptation to combined roofs, multi-slope roofs, L-shaped roofs, and partial ancillary roofs. Furthermore, it performs detailed analysis of the roof structural lines based on the structural line confidence level, ensuring the accuracy of the roof topology. Further, this invention records the semantic features of the image using multi-scale image features, while simultaneously identifying regional occlusion and generating an occlusion confidence map. This allows for differential analysis of low-confidence areas such as trees, shadows, and roof equipment. Differentiation processing avoids erroneous texture features directly affecting roof structure generation. Finally, in the roof generation process, this invention uses multiple constraints through roof zoning, continuous structural lines, and aspect fields to ensure the accuracy and effectiveness of roof patch generation. Multiple verifications are performed during the generation process, and planar consistency and topological consistency are used to ensure the local accuracy and overall stability of the generated building, effectively reducing ridge breaks, aspect confusion, and roof boundary errors. Therefore, the roof patch generation method based on the ridge-aspect intermediate representation of remote sensing images provided by this invention can improve the accuracy of 3D reconstruction and reduce data costs.

[0042] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0043] Finally, it should be noted that deleting any one of the above embodiments does not affect the technical solutions of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation, characterized in that, include: Step S1: Acquire remote sensing images of the target building, crop the remote sensing images to obtain standardized roof image blocks; Step S2: Based on the standardized roof image blocks, extract multi-scale image features, segment the roof area, and obtain a roof binary mask; generate an eaves boundary map and an occlusion confidence map from the multi-scale image features; fuse the roof binary mask, the eaves boundary map, and the occlusion confidence map to form a roof semantic boundary representation; Step S3: Based on the semantic boundary representation of the roof, combined with multi-scale image features, generate the confidence of the ridge-valley structure line, extract candidate lines, merge the candidate lines, and generate continuous structure lines; The pixel-level aspect field is predicted using a binary roof mask. The roof is divided into initial roof partitions by combining the confidence of the ridge-valley structure line with the semantic boundary representation of the roof. The scale anchor points of the initial roof partitions are recorded to form the intermediate representation of the ridge-aspect. Step S4: Generate initial roof patches based on the initial roof zoning, continuous structural lines, and aspect field to form the structural representation of candidate roof patches; collect the two-dimensional boundary vertex set and geometric parameters of the candidate roof patches, analyze the adjacency relationship between patches, and filter the patches; Step S5: Statistically analyze the candidate roof patches and construct a roof patch map; perform planar consistency and topological consistency analysis based on the roof patch map, and optimize the roof patch map; Step S6: Map the low-confidence regions to the roof patch map from the occlusion confidence map to generate candidate supplementary patches. Combine the roof patch map to generate the lightweight roof structure result of the target building.

2. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S11: Based on the remote sensing image of the target building, the building outline is acquired, the image boundary of the target building is determined, the boundary buffer is acquired based on the image boundary of the target image, and the remote sensing image is cropped by the image boundary of the target image and the boundary buffer to obtain the building image block; Step S12: Acquire image data from remote sensing images, map the remote sensing images to the world coordinate system based on the image data, extract the mapping information corresponding to the building image blocks, and obtain standardized roof image blocks.

3. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S21: Transmit the standardized roof image blocks to the shared visual encoder to capture roof edges, eaves lines, shadow boundaries, and texture abrupt changes, record feature regions and feature semantics to obtain low-level image features, identify the main roof area, roof material type, tree occlusion, roof equipment, and background area, collect pixel distributions of different areas, record corresponding feature semantics, and obtain high-level image features; perform statistical analysis on the low-level and high-level image features to obtain multi-scale image features; Step S22: Based on multi-scale image features, extract the main area of ​​the roof and generate a roof binary mask by combining the remote sensing image of the target building. Based on multi-scale image features, the roof eaves line is identified, and an eaves boundary map is generated. Based on multi-scale image features, tree occlusion, shadow coverage, roof equipment, texture blurring, and image missing areas are identified to obtain occlusion areas. Occlusion confidence is assigned based on the occlusion area being lower than the image, and an occlusion confidence map is generated. The roof binary mask, eaves boundary map, and occlusion confidence map are statistically analyzed to obtain the roof semantic boundary representation.

4. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S31: Based on the roof semantic boundary representation, obtain the eaves boundary map, obtain the roof semantic features from the multi-scale image features, traverse the roof eaves line from the eaves boundary map, extract the roof semantic features around the roof eaves line, and generate the confidence score of the ridge-valley structure line. Non-maximum suppression is applied to the confidence of the ridge-valley structure line to obtain candidate structure lines. The parameters of the candidate structure lines are collected, and the candidate structure lines are merged based on the line parameters to generate continuous structure lines. Step S32: Based on the roof binary mask, extract the roof range. Using a shared visual encoder, predict the pixel-level aspect field of the roof range to generate a two-dimensional unit vector, recording the cosine and sine values ​​of the orientation angles respectively. Combine continuous structural lines with the roof semantic boundary representation to divide the roof. Extract the roof outline using the roof semantic boundary representation. Divide the roof space into local roof spaces using continuous structural lines. Cluster the two-dimensional unit vectors of the corresponding pixels in each local roof space to classify the roof aspect and obtain the initial roof partition. Collect the roof height based on the initial roof partition to obtain the scale anchor points of the initial roof partition. The confidence level of the ridge-valley structural line, continuous structural line, aspect field, initial roof zoning, and scale anchor points are statistically analyzed to form an intermediate representation of the ridge-aspect.

5. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 4, characterized in that, The specific steps of step S31 are as follows: Step S311: Based on the confidence of the ridge-valley structure line, collect the confidence of the structure line at each pixel position. From the eaves boundary map, estimate the normal direction of the ridge-valley structure line. Search the left and right neighborhoods of the pixel along the normal direction. If the confidence of the current pixel is not the maximum value in the normal neighborhood, suppress and clear it to zero. Retain the peak value of the confidence in the normal direction to generate candidate structure lines. Step S312: Count the number of candidate structural lines as as; collect the line segment direction angles fxj1 to fxj for each candidate structural line. as Collect the endpoint positions ddw1 to ddw of each candidate structural line. as Angle threshold jdy and distance threshold jly are set, and consistency calculation is performed on candidate lines with different structures to obtain consistency evaluation value pgz(a1, a2). Where a1 represents the a1th structural candidate line, a2 represents the a2th structural candidate line, and pgz(a1, a2) is the consistency evaluation between the a1th and a2th structural candidate lines. If the consistency evaluation value pgz(a1, a2) = 1, it indicates that the candidate structural lines are consistent, and the candidate structural lines will be merged. If the consistency evaluation value pgz(a1, a2) = 0, it indicates that the candidate structural lines are inconsistent, and the structural lines are traversed again.

6. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S41: Extract the roof area from the initial roof area partition, combine it with the continuous structure line, generate adjacent roof patches on both sides of the continuous structure line, count the generated adjacent roof patches to obtain the initial roof patch; input the initial roof patch into the Transformer decoder, aggregate the texture information, structure line information and orientation information of the roof area corresponding to the initial roof patch to form the structural representation of the candidate roof patch. Step S42: Based on the structural representation of the candidate roof panels, fit the boundaries of the roof panels to the continuous structural lines to generate closed candidate roof panels, and record the non-closed panels. Collect the two-dimensional boundary vertices of each candidate roof panel to obtain a set of two-dimensional boundary vertices. Collect the normal parameters, slope parameters, and elevation offset of the roof panel to obtain the geometric parameters. Step S43: Analyze the adjacency relationships between different candidate roof tiles using the two-dimensional boundary vertex set and geometric parameters of the candidate roof tiles, and store the analysis results; based on the adjacency relationships between different candidate roof tiles, combined with the slope aspect and shading confidence of the corresponding positions of the candidate roof tiles, filter the candidate roof tiles to obtain a set of candidate roof tiles.

7. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 6, characterized in that, The specific steps of step S43 are as follows: Step S431: Traverse the candidate roof tiles, collect the 2D boundary vertex set of the candidate roof tiles, generate the candidate roof tile boundaries, and determine the overlap of the candidate roof tile boundaries of different candidate roof tiles. If there is an overlap in the candidate roof tile boundaries, it indicates that the candidate roof tiles are generating a conflict. Collect the center points of the candidate roof tiles, divide the overlapping parts, and record the adjacency relationship of the corresponding candidate roof tiles as a conflict relationship. If there is an edge overlap in the candidate roof tile boundaries, record the adjacency relationship of the candidate roof tiles based on the overlapping edges; store all adjacency relationships. Step S432: Collect the slope direction and shading confidence of the corresponding location of the candidate roof pieces. Combine the adjacency relationship between different candidate roof pieces to make a consistency judgment on the candidate roof pieces that share the same edge. If the candidate roof piece has a shading condition and the slope direction of the candidate roof piece is inconsistent with that of the candidate roof piece that shares the same edge with the candidate roof piece, mark the candidate roof piece as an invalid candidate for screening.

8. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S51: Extract candidate roof patches from the candidate roof patch set, traverse the candidate roof patches as nodes, record the connecting edges between nodes based on the adjacency relationship between different candidate roof patches, and construct a roof patch graph. Step S52: Based on the roof patch map, perform planar consistency and topological consistency checks on the candidate roof patches; collect the geometric parameters and pixel-level slope field of the candidate roof patches, analyze the roof planar consistency, and correct the roof; judge the boundary overlap between adjacent patches based on the connection edges between nodes, and adjust the patches accordingly; optimize the boundary closure of the patches by combining the eaves boundary map; generate a roof patch map optimized for geometric consistency and topological consistency.

9. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 8, characterized in that, The specific steps of step S52 are as follows: Step S521: Obtain the geometric parameters and pixel-level slope field of each candidate roof piece. Extract the normal parameters of the candidate roof piece from the geometric parameters. Compare the normal parameters with the pixel-level slope field. If the angle direction shown by the normal parameters and the pixel-level slope field is consistent, it indicates that the candidate roof piece satisfies planar consistency. If the angle direction shown by the normal parameters and the pixel-level slope field is inconsistent, it indicates that there is a local anomaly in the candidate roof piece. Correct the roof by applying consistency constraints to the normal parameters and the pixel-level slope field at different locations of the candidate roof piece. Step S522: Collect adjacent patches from the connecting edges between nodes, record the adjacency relationship based on the connecting edges, use the adjacency relationship as a patch constraint, extract the topological relationship between adjacent patches, and optimize abnormal topology based on the adjacency relationship.

10. The method for generating roof patches from remote sensing images based on the ridge-slope aspect intermediate representation according to claim 1, characterized in that, The specific steps of step S6 are as follows: Step S61: Based on the occlusion confidence map, uncertain areas caused by occlusion, shadows, or texture anomalies are collected to obtain low-confidence regions. The low-confidence regions are mapped to the roof patch map, and candidate roof patches involved in the low-confidence regions are extracted to obtain low-confidence candidate roof patches. The adjacent patches of the low-confidence candidate roof patches are collected, and the roof structure of the low-confidence candidate roof patches is inferred by combining the adjacency relationship between the low-confidence candidate roof patches and the adjacent patches. The inference parameters of the low-confidence candidate roof patches are recorded, and candidate supplementary patches are generated. Step S62: Based on the roof panel diagram, perform planar consistency and topological consistency checks on the candidate supplementary panels, verify the candidate supplementary panels, and statistically analyze the verified candidate supplementary panels and candidate roof panels to generate the lightweight roof structure result of the target building.

Images