A method and system for semantic decomposition of three-dimensional components of ancient buildings

Abstract

This invention discloses a method and system for semantic decomposition of three-dimensional components of ancient buildings. The method includes: generating projection images from multiple perspectives based on an original three-dimensional model; projecting triangular facets of the original three-dimensional model onto the projection images from each perspective; inputting the projection images from multiple perspectives into a trained GAS segmentation network to obtain corresponding semantic masks; identifying pixels falling inside two-dimensional triangles, dividing these pixels into different polygonal regions based on the semantic masks, segmenting these polygonal regions to generate multiple new triangles, enhancing the original three-dimensional model based on these new triangles to obtain an enhanced three-dimensional model; reprojecting the triangular facets of the enhanced three-dimensional building, and applying a majority voting method to the semantic masks of the projection images from all perspectives to obtain the final semantic label for each triangular facet. This invention achieves accurate and complete extraction of slender components from buildings.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, system, storage medium, and electronic device for semantic decomposition of three-dimensional components of ancient buildings. Background Technology

[0002] Semantic decomposition of architecture is crucial in fields such as digital archiving of cultural relics, structural analysis of ancient buildings, urban knowledge modeling, and cultural heritage protection. Significant progress has been made in the decomposition of coarse-grained components such as roofs, facades, and windows using 3D architectural models based on polyhedral models or point cloud representations. However, research on the decomposition of finer components, such as roof ridges, beams, columns in historical buildings, or curtain wall frames and trusses in modern buildings, remains insufficient. These components are not only structurally and visually vital but also carry important architectural and cultural information. The inability to accurately extract these slender components will limit the effectiveness of 3D architectural models in high-level analysis and knowledge-driven applications.

[0003] The segmentation challenge of slender components stems from a combination of geometric and semantic issues. Geometrically, slender, curved components are tightly connected to adjacent surfaces, making them difficult to extract in a topologically consistent manner. Semantically, these components resemble the texture of nearby roofs or facades, easily resulting in weak or blurred feature responses.

[0004] In recent years, image-based semantic decomposition methods have demonstrated that high-resolution texture information can, to some extent, compensate for geometric ambiguity, making oblique photogrammetric architectural models containing fine geometry and texture an important data form for architectural semantic decomposition. Therefore, increasing research is focusing on image-geometry fusion methods for 3D architectural analysis. However, existing convolutional neural network architectures typically rely on fixed receptive fields, failing to effectively align with the geometric flow of slender or curved components. This results in frequent breaks and discontinuities in these components during 2D prediction, thus affecting the topological consistency of the 3D mesh.

[0005] To address these issues, existing research has primarily focused on two technical directions: one approach uses deformable convolutions to learn flexible sampling offsets for better alignment of slender or curved structures; the other employs encoder-decoder architectures, feature pyramids, and attention mechanisms for cross-scale feature aggregation, aiming to improve the extraction of fine structures. While these strategies can alleviate the feature loss problem of fine components to some extent, they do not fundamentally solve the problem of accurate feature extraction of fine components. Furthermore, the resolution mismatch between high-resolution images and relatively coarse 3D meshes further exacerbates this problem. Slender structures clearly visible in 2D images often degenerate into sparse triangulation or overly smoothed surfaces during mesh reconstruction, making it difficult to obtain topologically consistent and complete fine-grained building component extraction results even after the fine 2D semantic information is projected onto the 3D photogrammetric model. Summary of the Invention

[0006] This invention provides a method, system, storage medium, and electronic device for semantic decomposition of three-dimensional components of ancient buildings, which can solve the problems of semantic continuity and topological consistency in the feature extraction process of slender components, and realize the accurate and complete extraction of slender components in buildings.

[0007] This invention provides a method for semantic decomposition of three-dimensional components of ancient buildings, including: Obtain the original 3D model of the ancient building, and generate projection images from multiple perspectives based on the original 3D model; The triangular facets of the original 3D model are projected onto the projected images from various viewpoints to obtain two-dimensional triangles on the projected images. The projected images from multiple perspectives are input into the trained GAS segmentation network to obtain the corresponding semantic masks. Find the pixels that fall inside the two-dimensional triangle. Based on the semantic mask, divide the pixels that fall inside the two-dimensional triangle into different polygonal regions. Segment the different polygonal regions to generate multiple new triangles. Enhance the original three-dimensional model based on the multiple new triangles to obtain the enhanced three-dimensional model. Reproject the triangular facets of the enhanced three-dimensional building and apply the majority voting method to the semantic mask of the projected image from all views to obtain the final semantic label of each triangular facet.

[0008] Furthermore, according to the above-mentioned semantic decomposition method for three-dimensional components of ancient buildings, nine perspectives are defined for each original three-dimensional model, and the projected images of the nine perspectives include a top view and eight tilted views with a first preset angle.

[0009] Furthermore, according to the above-mentioned semantic decomposition method for three-dimensional components of ancient buildings, projecting the triangular facets of the original three-dimensional model onto the projected images from various viewpoints includes: Define the triangular facets of the original 3D model Normal vector of a triangular facet and the normal vector of the projected image ; The triangular facets are arranged according to The projection rules for angles smaller than the second preset angle are projected onto the projected images at various viewpoints.

[0010] Furthermore, according to the above-mentioned semantic decomposition method for three-dimensional components of ancient buildings, the GAS segmentation network includes an input feature extraction module, an integrated encoder, and a progressive decoder; the processing procedure of the GAS segmentation network includes: The projected image is input into the feature extraction module, and the original features are obtained through convolution, max pooling and downsampling operations. The original features are input into the integrated encoder to perform multi-scale feature extraction and fusion to obtain multi-scale context fusion features; The multi-scale context fusion features are input into the progressive decoder to obtain the corresponding semantic mask.

[0011] Furthermore, according to the aforementioned semantic decomposition method for three-dimensional components of ancient buildings, the integrated encoder includes multiple deformable SE bottleneck structures and an ASPP module. The processing procedure of the integrated encoder includes: Multi-scale features are obtained by extracting multiple deformable SE bottleneck structures from the original features. The multi-scale features are fused using the ASPP module to obtain multi-scale context fusion features; The deformable SE bottleneck structure includes multiple convolutional layers and an SE attention module. The processing procedure of the deformable SE bottleneck structure includes: The original features are sequentially passed through a CBR convolutional layer, a dCBR convolutional layer, a CB convolutional layer, and an SE attention mechanism to obtain the first feature. The original features are then passed through a 2D convolutional layer to obtain the second feature. The first feature and the second feature are fused and then passed through an activation function to obtain the feature at the corresponding scale.

[0012] Furthermore, according to the above-mentioned semantic decomposition method for three-dimensional components of ancient buildings, the CBR convolutional layer includes a 1×1 convolution, batch normalization and ReLU activation function connected in sequence, the dCBR convolutional layer includes a 3×3 deformable convolution, batch normalization and ReLU activation function, and the CB convolutional layer includes a 1×1 convolution and batch normalization.

[0013] Furthermore, according to the above-mentioned semantic decomposition method for three-dimensional components of ancient buildings, the step of segmenting different polygonal regions to generate multiple new triangles, and enhancing the original three-dimensional model based on these multiple new triangles, includes: The Alpha Shape algorithm is used to extract boundary points for each polygon region, and all boundary points are sorted clockwise to obtain a set. ,in Indicates boundary points; When set When the value is greater than 3, the Bowyer-Watson algorithm is used to triangulate the polygonal region to obtain multiple new triangles.

[0014] This invention also provides a semantic decomposition system for three-dimensional components of ancient buildings, comprising: The model is obtained to acquire the original three-dimensional model of the ancient building, and projection images from multiple perspectives are generated based on the original three-dimensional model. The projection module is used to project the triangular facets of the original three-dimensional model onto the projection images from various viewpoints to obtain two-dimensional triangles on the projection images. The semantic segmentation module is used to input the projected images from multiple viewpoints into the trained GAS segmentation network to obtain the corresponding semantic masks. The enhancement and update module is used to find pixels that fall inside two-dimensional triangles. Based on semantic masks, the pixels falling inside two-dimensional triangles are divided into different polygonal regions. The different polygonal regions are segmented to generate multiple new triangles. The original 3D model is enhanced based on multiple new triangles to obtain the enhanced 3D model. The triangular facets of the enhanced 3D building are reprojected, and the semantic masks of the projected images from all views are applied by majority voting to obtain the final semantic label of each triangular facet.

[0015] The present invention also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the above-described methods for semantic decomposition of three-dimensional components of ancient buildings.

[0016] The present invention also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in the steps of the semantic decomposition method for three-dimensional components of ancient buildings described in any of the preceding claims.

[0017] This invention provides a method, system, storage medium, and electronic device for semantic decomposition of 3D components of ancient buildings. By utilizing a GAS segmentation network, this invention addresses the issues of semantic continuity and topological consistency in feature extraction of slender components, achieving accurate and complete extraction of these components from buildings. This invention combines multi-view semantic parsing with local mesh optimization, effectively improving the performance of 3D models in detail components while ensuring the accurate extraction of slender components from 3D architectural models. Attached Figure Description

[0018] The technical solution and other beneficial effects of the present invention will become apparent from the following detailed description of specific embodiments of the invention, in conjunction with the accompanying drawings.

[0019] Figure 1 A flowchart of a method for semantic decomposition of three-dimensional components of ancient buildings provided in an embodiment of the present invention.

[0020] Figure 2 The three-dimensional model of an ancient building obtained by photogrammetry for semantic decomposition is provided in an embodiment of the present invention.

[0021] Figure 3 This is a schematic projection of a triangular facet provided in an embodiment of the present invention.

[0022] Figure 4 The projection image generated based on the three-dimensional model is provided for the embodiments of the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the GAS segmentation network provided in an embodiment of the present invention.

[0024] Figure 6 This is a flowchart for enhancing the original three-dimensional model, provided as an embodiment of the present invention.

[0025] Figure 7 The diagram shows the effect of the local triangulation strategy provided in the embodiment of the present invention.

[0026] Figure 8 The diagram shows the segmentation effect of the entire frame for segmenting and refining structural building components with a high aspect ratio, as provided in the embodiments of the present invention.

[0027] Figure 9 A schematic diagram of the structure of the semantic decomposition system for three-dimensional components of ancient buildings provided in an embodiment of the present invention.

[0028] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This invention provides a method, system, storage medium, and electronic device for semantic decomposition of three-dimensional components of ancient buildings. The semantic decomposition system for three-dimensional components of ancient buildings provided by this invention can be integrated into an electronic device, such as a terminal or server. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0031] Please see Figure 1 , Figure 1 A flowchart of a semantic decomposition method for three-dimensional components of ancient buildings provided in an embodiment of the present invention, which is applied in electronic devices, includes the following steps: S1: Obtain the original 3D model of the ancient building and generate projection images from multiple perspectives based on the original 3D model.

[0032] Specifically, nine viewpoints are defined for each original 3D model. The projected images of the nine viewpoints include one top view (BEV, Bird's Eye View) and eight tilted views at a first preset angle. In one specific embodiment, the first preset angle is 45°. Figure 2 The three-dimensional model of ancient buildings obtained by photogrammetry for semantic decomposition is provided in the embodiment of the present invention, wherein (a)-(d) are all classic buildings located in a certain province.

[0033] S2 projects the triangular facets of the original 3D model onto the projected images from various viewpoints, resulting in two-dimensional triangles on the projected images.

[0034] In one embodiment, step S2 includes the following steps: S21 defines the triangular facets of the original 3D model. Normal vector of a triangular facet and projected images normal vector ; S22, arrange the triangular facets according to The projection rules for angles smaller than the second preset angle are projected onto the projected images at various viewpoints.

[0035] In one specific embodiment, the second preset angle is 90°.

[0036] Specifically, the plane of each projection viewpoint is defined by the normal vector of that plane. Given the three-dimensional origin O, the equation of the plane is: XYZ is a Cartesian coordinate system, while UVW is a coordinate system used to define texture coordinates. It is the angle between the X-axis and the U-axis. It is the angle between the Y-axis and the V-axis. It is the angle between the Z-axis and the W-axis.

[0037] Figure 3 The projection diagram of the triangular facet provided in the embodiment of the present invention includes two sub-images, (a) and (b). Figure 3 (a) shows a triangular facet. Projected onto the projection image The process of obtaining its corresponding projected coordinates. By using a rotation matrix R to flip the coordinate system: transforming from a spatial coordinate system to a planar coordinate system, the rotation matrix R and the projected coordinates P are... j It can be represented as:

[0038]

[0039] When multiple triangles overlap in image space, minor errors may occur due to occlusion. For example... Figure 3 As shown in (b), the triangular facet and Overlapping projection onto the projected image When, in order to obtain the correct projection, when and When the IoU exceeds 50%, depth filtering is applied to preserve the triangular facets closer to the view plane. Finally, each projected image is generated using affine texture mapping with a resolution of 0.02 meters. Figure 4 The projection image generated based on a three-dimensional model is provided in the embodiments of the present invention, wherein Figure 4 Includes four columns of images, from left to right: Figure 2 The images (a), (b), (c), and (d) are projected from the image, where the colors in the projected image do not represent categories; they simply distinguish different components.

[0040] S3. The projected images from multiple perspectives are input into the trained GAS (Geometry-adaptivesemantic) segmentation network to obtain the corresponding semantic masks.

[0041] This network borrows from the DeepLabV3+ backbone network and integrates deformable convolutions to adapt to the slender shapes of components, achieving more flexible and accurate feature extraction. Meanwhile, to preserve fine-scale structural details, we adopt a progressive upsampling strategy with skip connections, similar to U Net, in the decoder. To further enhance the recovery of fine details, we introduce an efficient channel attention (ECA) module, which selectively enhances effective features and suppresses irrelevant responses during feature fusion. Figure 5 This is a schematic diagram of the structure of the GAS segmentation network provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the GAS segmentation network includes an input feature extraction module, an integrated encoder, and a progressive decoder; the processing steps of the GAS segmentation network include: S31, the projected image is input into the feature extraction module, and the original features are obtained through convolution, max pooling and downsampling operations.

[0042] S32 inputs the original features into the integrated encoder to perform multi-scale feature extraction and fusion to obtain multi-scale context fusion features.

[0043] In one embodiment, the integrated encoder includes multiple deformable SE bottleneck structures and an ASPP module. The processing of the integrated encoder includes: S321, multi-scale features are obtained by extracting the original features through multiple deformable SE bottleneck structures; S322 uses the ASPP (Spatial Pyramid Pooling) module to fuse multi-scale features, resulting in multi-scale contextual fusion features.

[0044] In one specific embodiment, there can be 16 deformable SE bottleneck structures.

[0045] In one embodiment, the deformable SE bottleneck structure includes multiple convolutional layers and an SE attention module. The processing of the deformable SE bottleneck structure includes: The original features are sequentially passed through a CBR convolutional layer, a dCBR convolutional layer, a CB convolutional layer, and an SE attention mechanism to obtain the first feature. The original features are then passed through a 2D convolutional layer to obtain the second feature. The first and second features are then fused and passed through an activation function to obtain the feature at the corresponding scale.

[0046] The CBR convolutional layer includes a 1×1 convolution, batch normalization, and ReLU activation function connected in sequence; the dCBR convolutional layer includes a 3×3 deformable convolution, batch normalization, and ReLU activation function; and the CB convolutional layer includes a 1×1 convolution and batch normalization.

[0047] In one embodiment, after obtaining the multi-scale context fusion features, the multi-scale context fusion features are input into the ECA module. The ECA module includes a CBR convolutional layer, an average pooling layer, a 2D convolutional layer, and an activation function connected in sequence. After the multi-scale context fusion features are input into the CBR convolutional layer, a first fusion feature is obtained. The first fusion feature is then passed through an average pooling layer, a 2D convolutional layer, and a sigmoid activation function in sequence to obtain a second fusion feature. The first fusion feature and the second fusion feature are fused to obtain the final output of the integrated encoder.

[0048] S33, input the multi-scale context fusion features into the progressive decoder to obtain the corresponding semantic mask.

[0049] This embodiment begins with deformable convolutions on the first deformable SE bottleneck structure to ensure adaptation to the geometry of curved and slender components, enabling effective feature extraction from complex architectural structures. Subsequently, the ASPP module captures multi-scale contextual information. Finally, the decoder progressively upsamples the extracted multi-scale contextual fusion features and fuses them with features from the corresponding level in the encoder via skip connections. To ensure accurate recovery of fine details in slender components, we introduce an ECA module in the feature fusion stage for more efficient feature aggregation.

[0050] S4. Find the pixels that fall inside the two-dimensional triangle. Based on the semantic mask, divide the pixels that fall inside the two-dimensional triangle into different polygonal regions. Segment the different polygonal regions to generate multiple new triangles. Enhance the original three-dimensional model based on the multiple new triangles to obtain the enhanced three-dimensional model. Reproject the triangular facets of the enhanced three-dimensional building and apply the majority voting method to the semantic mask of the projected image from all views to obtain the final semantic label of each triangular facet.

[0051] In 3D models, slender roof components are typically represented by large, sparse triangles. Existing methods often assign a single semantic label to each triangle using a "majority voting" approach with multi-view projection. However, this method leads to structural discontinuities when a triangle spans multiple components. To address this issue, this invention proposes a 2D semantic-driven local triangulation strategy. Using a top-view projection, each 3D triangular facet is examined to determine if its projection onto the 2D image spans multiple semantic regions. If so, the triangular facet is subdivided based on a predicted semantic mask. Figure 6 The flowchart for enhancing the original 3D model provided in the embodiments of the present invention is as follows: Figure 6 As shown.

[0052] First, based on the projected 2D triangle, the pixels falling inside the triangle are identified. Then, combined with the semantic mask predicted by the GAS segmentation network, these pixels are divided into different polygonal regions. Different regions belong to different categories.

[0053] Next, for each region Ri, the Alpha Shape algorithm is used to extract its boundary points and arrange them in a clockwise direction to obtain a set. ,in Represents boundary points. When the set When the value is greater than 3, the Bowyer-Watson algorithm is used for triangulation to obtain a series of smaller new triangles. Otherwise, new triangles are directly formed from the boundary points.

[0054] Subsequently, using a ray-triangle intersection strategy based on the Möller-Trumbore formula, the newly obtained small triangles were back-projected into 3D space and integrated into the original 3D building model. The integrated 3D model exhibited more refined detail in its components. Finally, the enhanced 3D building triangles were reprojected onto all nine views, and a majority voting method was applied to the semantic masks corresponding to these nine views to obtain the final semantic label for each triangle. Figure 7 The following are the effect diagrams of the local triangulation strategy provided in the embodiments of the present invention. Column (a) shows the segmentation results in the image, column (b) shows the segmentation results in the three-dimensional building model without applying the local triangulation strategy, column (c) shows the segmentation results in the three-dimensional building model with the local triangulation strategy applied, and column (d) shows the superimposed comparison of the segmentation effects of the ridge part in the three-dimensional building before and after applying local triangulation. Figure 8 The following are segmentation effect diagrams of the building components provided by the entire framework for segmentation and refinement of high aspect ratio structural building components in the embodiments of the present invention. (A)-(D) are multi-view projection images, and (a)-(d) are three-dimensional models and corresponding semantic segmentation results, respectively.

[0055] Based on the method described in the above embodiments, this embodiment will further describe it from the perspective of the semantic decomposition system of three-dimensional components of ancient buildings. The semantic decomposition system of three-dimensional components of ancient buildings can be implemented as an independent entity or integrated into an electronic device. The electronic device can be a terminal, server, or other device. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessor box, or other devices.

[0056] Please see Figure 9 , Figure 9This invention specifically describes a semantic decomposition system for three-dimensional components of ancient buildings, which is applied in electronic devices. This system may include: The model is obtained to acquire the original three-dimensional model of the ancient building, and projection images from multiple perspectives are generated based on the original three-dimensional model. The projection module is used to project the triangular facets of the original three-dimensional model onto the projection images from various viewpoints to obtain two-dimensional triangles on the projection images. The semantic segmentation module is used to input the projected images from multiple viewpoints into the trained GAS segmentation network to obtain the corresponding semantic masks. The enhancement and update module is used to find pixels that fall inside two-dimensional triangles. Based on semantic masks, the pixels falling inside two-dimensional triangles are divided into different polygonal regions. The different polygonal regions are segmented to generate multiple new triangles. The original 3D model is enhanced based on multiple new triangles to obtain the enhanced 3D model. The triangular facets of the enhanced 3D building are reprojected, and the semantic masks of the projected images from all views are applied by majority voting to obtain the final semantic label of each triangular facet.

[0057] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0058] In addition, this embodiment of the invention also provides an electronic device, which may be a computer, tablet computer, or other similar device. This electronic device can implement the steps in any embodiment of the semantic decomposition method for three-dimensional components of ancient buildings provided by this embodiment of the invention. Therefore, it can achieve the beneficial effects that any of the semantic decomposition methods for three-dimensional components of ancient buildings provided by this embodiment of the invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0059] Figure 10 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the semantic decomposition method for three-dimensional components of ancient buildings provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0060] The memory 520 can be used to store software programs and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520. The memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 520 may further include memory remotely located relative to the processor 580, and these remote memories can be connected to the electronic device 500 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0061] The input unit 530 can be used to receive input numeric or character information, and to generate a keyboard and mouse related to user settings and function control. Display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. Display unit 540 may include display panel 541, which may optionally be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms.

[0062] Electronic device 500, through transmission module 570 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 570 is shown in the figure, it is understood that it is not an essential component of electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0063] The processor 580 is the control center of the electronic device 500. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, thereby providing overall monitoring of the electronic device. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0064] Electronic device 500 also includes a power supply 590 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 590 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0065] Although not shown, the electronic device 500 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations: Obtain the original 3D model of the ancient building, and generate projection images from multiple perspectives based on the original 3D model; The triangular facets of the original 3D model are projected onto the projected images from various viewpoints to obtain two-dimensional triangles on the projected images. The projected images from multiple perspectives are input into the trained GAS segmentation network to obtain the corresponding semantic masks. Find the pixels that fall inside the two-dimensional triangle. Based on the semantic mask, divide the pixels that fall inside the two-dimensional triangle into different polygonal regions. Segment the different polygonal regions to generate multiple new triangles. Enhance the original three-dimensional model based on the multiple new triangles to obtain the enhanced three-dimensional model. Reproject the triangular facets of the enhanced three-dimensional building and apply the majority voting method to the semantic mask of the projected image from all views to obtain the final semantic label of each triangular facet.

[0066] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0067] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any embodiment of the semantic decomposition method for three-dimensional components of ancient buildings provided by the present invention.

[0068] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0069] Since the instructions stored in the storage medium can execute the steps in any embodiment of the semantic decomposition method for three-dimensional components of ancient buildings provided in the embodiments of the present invention, the beneficial effects that any semantic decomposition method for three-dimensional components of ancient buildings provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0070] The foregoing has provided a detailed description of a method, system, storage medium, and electronic device for semantic decomposition of three-dimensional components of ancient buildings provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for semantic decomposition of three-dimensional components of ancient buildings, characterized in that, The method includes: Obtain the original 3D model of the ancient building, and generate projection images from multiple perspectives based on the original 3D model; The triangular facets of the original 3D model are projected onto the projected images from various viewpoints to obtain two-dimensional triangles on the projected images. The projected images from multiple perspectives are input into the trained GAS segmentation network to obtain the corresponding semantic masks. Find the pixels that fall inside the two-dimensional triangle. Based on the semantic mask, divide the pixels that fall inside the two-dimensional triangle into different polygonal regions. Segment the different polygonal regions to generate multiple new triangles. Enhance the original three-dimensional model based on the multiple new triangles to obtain the enhanced three-dimensional model. Reproject the triangular facets of the enhanced three-dimensional building and apply the majority voting method to the semantic mask of the projected image from all views to obtain the final semantic label of each triangular facet. This involves segmenting different polygonal regions to generate multiple new triangles, and then enhancing the original 3D model based on these new triangles, including: The Alpha Shape algorithm is used to extract boundary points for each polygon region, and all boundary points are sorted clockwise to obtain a set. ,in Indicates boundary points; When set When the value is greater than 3, the Bowyer-Watson algorithm is used to triangulate the polygonal region to obtain multiple new triangles.

2. The semantic decomposition method for three-dimensional components of ancient buildings according to claim 1, characterized in that, Nine viewpoints are defined for each of the original 3D models. The projected images of the nine viewpoints include one top view and eight tilted views with a first preset angle.

3. The semantic decomposition method for three-dimensional components of ancient buildings according to claim 2, characterized in that, Projecting the triangular facets of the original 3D model onto projected images from various viewpoints includes: Define the triangular facets of the original 3D model Normal vector of a triangular facet and the normal vector of the projected image ; The triangular facets are arranged according to The projection rules for angles smaller than the second preset angle are projected onto the projected images at various viewpoints.

4. The semantic decomposition method for three-dimensional components of ancient buildings according to claim 1, characterized in that, The GAS segmentation network includes an input feature extraction module, an integrated encoder, and a progressive decoder; the processing procedure of the GAS segmentation network includes: The projected image is input into the feature extraction module, and the original features are obtained through convolution, max pooling and downsampling operations; The original features are input into the integrated encoder to perform multi-scale feature extraction and fusion to obtain multi-scale context fusion features; The multi-scale context fusion features are input into the progressive decoder to obtain the corresponding semantic mask.

5. The semantic decomposition method for three-dimensional components of ancient buildings according to claim 4, characterized in that, The integrated encoder includes multiple deformable SE bottleneck structures and an ASPP module. The processing procedure of the integrated encoder includes: Multi-scale features are obtained by extracting multiple deformable SE bottleneck structures from the original features. The multi-scale features are fused using the ASPP module to obtain multi-scale context fusion features; The deformable SE bottleneck structure includes multiple convolutional layers and an SE attention module. The processing procedure of the deformable SE bottleneck structure includes: The original features are sequentially passed through a CBR convolutional layer, a dCBR convolutional layer, a CB convolutional layer, and an SE attention mechanism to obtain the first feature. The original features are then passed through a 2D convolutional layer to obtain the second feature. The first feature and the second feature are fused and then passed through an activation function to obtain the feature at the corresponding scale.

6. The semantic decomposition method for three-dimensional components of ancient buildings according to claim 5, characterized in that, The CBR convolutional layer includes a 1×1 convolution, batch normalization, and ReLU activation function connected in sequence; the dCBR convolutional layer includes a 3×3 deformable convolution, batch normalization, and ReLU activation function; and the CB convolutional layer includes a 1×1 convolution and batch normalization.

7. A semantic decomposition system for three-dimensional components of ancient buildings, characterized in that, include: The model is obtained to acquire the original three-dimensional model of the ancient building, and projection images from multiple perspectives are generated based on the original three-dimensional model. The projection module is used to project the triangular facets of the original three-dimensional model onto the projection images from various viewpoints to obtain two-dimensional triangles on the projection images. The semantic segmentation module is used to input the projected images from multiple viewpoints into the trained GAS segmentation network to obtain the corresponding semantic masks. The enhancement and update module is used to find pixels that fall inside a two-dimensional triangle. Based on semantic masking, the pixels that fall inside the two-dimensional triangle are divided into different polygonal regions. The different polygonal regions are segmented to generate multiple new triangles. The original three-dimensional model is enhanced based on the multiple new triangles to obtain the enhanced three-dimensional model. The triangular facets of the enhanced 3D building are reprojected, and a majority voting method is applied to the semantic masks of the projected images from all viewpoints to obtain the final semantic label for each triangular facet. This involves segmenting different polygonal regions to generate multiple new triangles, and then enhancing the original 3D model based on these new triangles, including: The Alpha Shape algorithm is used to extract boundary points for each polygon region, and all boundary points are sorted clockwise to obtain a set. ,in Indicates boundary points; When set When the value is greater than 3, the Bowyer-Watson algorithm is used to triangulate the polygonal region to obtain multiple new triangles.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted to be loaded by a processor to execute the semantic decomposition method for three-dimensional components of ancient buildings according to any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to execute the steps in the semantic decomposition method for three-dimensional components of ancient buildings according to any one of claims 1 to 6.

Images