Knowledge graph construction method for corbel primitive point cloud labeling

Abstract

The invention discloses a knowledge graph construction method for bucket arch primitive point cloud labeling, which comprises the following steps: S1, construction of a bucket arch theoretical primitive surface point cloud labeling knowledge graph: performing graph concept layer design on a bucket arch, performing knowledge extraction from a bucket arch component layer and a surface primitive layer, and constructing the bucket arch theoretical primitive surface point cloud labeling knowledge graph; s2, labeling a bracket system primitive component based on a knowledge graph: carrying out super voxel clustering on the bracket system point cloud to obtain a bracket system clustering point cloud surface set, matching a point cloud surface retrieved from the bracket system clustering point cloud surface set with a bracket system theoretical primitive surface point cloud labeling knowledge graph to determine whether the point cloud surface is valid, and when the point cloud surface is valid, carrying out the labeling of the bracket system primitive component; the method comprises the following steps: determining components to which point cloud surfaces belong and point cloud surfaces adjacent to the components on the basis of a bracket theory primitive surface point cloud labeling knowledge graph, and labeling the point cloud surfaces; the method has the advantages that the constructed mapping knowledge domain contains the detail features of the form of the real bracket set point cloud, and can be more effectively used for the marking work of the actual bracket set point cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph construction, and in particular to a knowledge graph construction method for bracket arch primitive point cloud annotation. Background Art

[0002] Component-level 3D solid models of ancient building brackets play an important role in ancient building simulation, disease monitoring, and display and utilization. As the third type of spatial data after maps and images, 3D point clouds can fully and accurately depict the three-dimensional real world and have become the main data source for 3D modeling of ancient buildings. Point cloud annotation is an important step in point cloud modeling of ancient buildings. Acquiring semantic primitive components from the overall point cloud is a key task in component-level 3D solid modeling of brackets. Currently, point cloud semantic annotation algorithms mainly include point cloud semantic annotation algorithms based on intelligent learning algorithms and traditional semantic annotation algorithms.

[0003] Research on point cloud segmentation using intelligent learning algorithms (including machine learning and deep learning) has garnered significant attention from academic circles both domestically and internationally, and has yielded numerous research results. Scholars in the architectural heritage field have incorporated intelligent algorithms into point cloud annotation of ancient buildings. For example, some researchers have used random forests to semantically label facades, roofs, windows, and columns. Other researchers have introduced DGCNN into point cloud annotation of Western and Chinese ancient buildings, achieving annotation of major architectural structures such as roofs and hulls. Furthermore, to address the difficulty of labeling the small differences between ancient building components, researchers have introduced a self-attention segmentation network model, SQN-DLA, which effectively distinguishes similar components. The results of semantic labeling between ancient building components using mainstream algorithms, such as Point Net and DGCNN, were compared, demonstrating the superiority of machine intelligence in point cloud annotation of ancient building components. Datasets are a key factor influencing the effectiveness of machine learning point cloud annotation. In addition to publicly available point cloud annotation datasets, Wuhan University and Beijing University of Civil Engineering have each proposed datasets for the WHU-TLS ancient building data set.

[0004] Intelligent learning-based point cloud annotation algorithms achieve good segmentation results, but these algorithms are only suitable for annotating the main structures of ancient buildings, such as facades and roof brackets. However, when annotating bracket primitive components, these algorithms still face numerous challenges. Deep learning is essentially about learning feature representations, and the shape features of bracket primitive components in different locations are similar or even identical. Furthermore, bracket components intersect and combine, splitting the same physical component in the point cloud morphology, and coplanar faces of different components. Therefore, the actual annotation process requires the use of relevant reasoning knowledge.

[0005] Conventional point cloud semantic annotation methods mainly rely on traditional point cloud segmentation algorithms (usually following relatively strict mathematical models, such as region growing, model fitting, etc.) to cluster ancient building point clouds into point clouds with the same geometric properties, and then attribute these point cloud clusters to corresponding objects based on the corresponding prior knowledge. In order to solve the problem of excessive time-consuming semantic annotation of the Milan Cathedral, one method proposed a multi-level and multi-resolution point cloud semantic annotation algorithm, which achieved the segmentation of the roof, spire, and house body. Another method proposed a multi-scale roof segmentation algorithm through the analysis of ancient building roof knowledge, which achieved fine-grained segmentation of ancient Chinese building roofs. Compared with intelligent machine algorithms, this type of method does not require particularly large training samples, but its accuracy depends on the accuracy of point cloud clustering and the accuracy of prior knowledge.

[0006] Fortunately, the construction of ancient Chinese architecture follows specific architectural rules that define component types, assembly patterns, and geometric dimensions. In the aforementioned studies, many scholars have applied these rules to construct ancient building models. At the component level, some researchers, based on construction knowledge, manually create models of various ancient building components using software such as Revit for 3D reconstruction. Regarding component assembly patterns, some researchers have used the hierarchical topology of Song Dynasty architecture's unique modular systems and structural patterns as a foundation, parameterizing wooden elements based on XML language rules and assembling components into Song Dynasty architecture for modeling ancient Chinese architecture from the Song Dynasty. Furthermore, some researchers have generated a Bayesian algorithm to represent the relationships between components and structures of ancient buildings, implementing a data-driven modeling approach. Through an interactive interface, simply selecting a small number of component prerequisites allows the generation of ancient building models. Ancient building knowledge bases are used to guide the automated construction of more accurate ancient building models. These studies demonstrate the importance of incorporating relevant ancient architectural knowledge into the modeling of diverse styles and types of ancient buildings to ensure effective modeling.

[0007] The knowledge base for ancient building construction rules is primarily based on the knowledge contained in documents such as the "Yingzao Fashi" and the "Qing Gongbu Gongcheng Fazuo Zeli". These rules serve as standard guidance for the construction of ancient buildings, but there is still a fundamental problem in how to apply these rules to the semantic annotation of bracket arch primitive components. However, there are still two problems in bracket arch construction:

[0008] (1) The knowledge of bracket arch primitive components is mainly oriented towards idealized complete physical components. The provided bracket arch primitive knowledge (including primitive component dimensions and connection relationships) is idealized dimensions and combination relationships. However, due to the influence of manufacturing process and long-term deformation, there are differences between the actual bracket arch and theoretical knowledge. When annotating bracket arch primitives, this knowledge lacks consideration of the bracket arch point cloud data form.

[0009] (2) Point cloud annotation is an operation on point clouds. However, point cloud acquisition equipment can only obtain surface point cloud information of ancient building components and cannot reveal the hidden structure between component combinations. In terms of point cloud geometry, the combination of components causes different components to be coplanar and the same component to be broken and separated. The knowledge of the general rules of bracket arches records the dimensions and characteristics of the components, providing clues for the annotation of bracket arch primitives. It is necessary to convert the component knowledge into component annotation knowledge. There is a difference in the data format between the two, and it is impossible to directly apply the knowledge of ancient buildings to the process of bracket arch component annotation. Summary of the Invention

[0010] The purpose of the present invention is to provide a knowledge graph construction method for bracket arch primitive point cloud annotation, so as to solve the above-mentioned problems existing in the prior art.

[0011] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0012] A knowledge graph construction method for bracket arch primitive point cloud annotation includes the following steps:

[0013] S1. Construction of knowledge graph for face point cloud annotation of bracket arch theory primitives: By designing the graph concept layer of bracket arch and extracting knowledge from the bracket arch component level and face primitive level, a knowledge graph for face point cloud annotation of bracket arch theory primitives is constructed;

[0014] S2. Labeling of bracket arch primitive components based on knowledge graph: Super voxel clustering is performed on the bracket arch point cloud to obtain the bracket arch cluster point cloud face set, and the point cloud face retrieved from the bracket arch cluster point cloud face set is matched with the bracket arch theoretical primitive face point cloud labeling knowledge graph to determine whether it is valid. When the point cloud face is valid, the component to which the point cloud face belongs and the point cloud faces adjacent to the component are determined based on the bracket arch theoretical primitive face point cloud labeling knowledge graph, and these point cloud faces are labeled.

[0015] Preferably, step S1 specifically includes the following contents:

[0016] S11. Based on the mutual mapping logical relationship between the bracket point cloud features and the primitive point cloud annotations, the overall knowledge graph is constructed;

[0017] S12. Starting with bracket arch structure knowledge and bracket arch point cloud data, design the graph concept layer and design nodes, relationships, and attributes.

[0018] S13, extracting bracket primitive surface knowledge from component entity related knowledge and point cloud morphological primitive surface knowledge, and processing the messy knowledge into a structured data table;

[0019] S14. Use the knowledge of bracket arch entities as a framework for knowledge processing to complete the framework of the bracket arch primitive point cloud annotation knowledge graph. Integrate the bracket arch point cloud surface knowledge and use Neo4j primitive software for knowledge management to form a complete bracket arch primitive point cloud annotation knowledge graph.

[0020] Preferably, in step S12, at the knowledge level of bracket set structure, the bracket set spatial structure form, primitive components and their connection relationship are analyzed, and the bracket set spatial position, shape and component entity are used as the basis for classification. The bracket set is classified according to its spatial position and type, and the component entity and specific primitive components are further divided.

[0021] Preferably, the knowledge extraction in step S13 is specifically to further derive the primitive surface normal, position and shape knowledge through the name, size, topological relationship, position relationship and primitive surface of the component entity, so as to realize the extraction of the bracket primitive surface knowledge.

[0022] Preferably, the knowledge extraction in step S13 includes the extraction of bracket knowledge at the component level, specifically,

[0023] Given a component assembly A=(p1,p2…p n ), where the primitive component p i and p j is the primitive component of the component assembly A, and the combination constraint matrix of the primitive component is,

[0024]

[0025] Among them, if α i,j =0 or α j,i =0, and i≠j, then p i and p j There is no combination relationship between them; if α i,j =1 or α j,i =1, and i≠j, then p i and p j There is a combination relationship between them;

[0026] For the component class of ancient buildings, the knowledge expression is P = {A,F P ,N}, A is the physical set of all components that make up the ancient building, F P is the set of factors that affect the primitive component category, and N is the number of influencing factors;

[0027] For the connection class that provides component connection relationships, the knowledge expression is S={O,C R ,R C ,L}, O is the spatial area of ​​the primitive component; C R R is the component associated with the primitive component in the spatial region; Cis the relationship between the primitive component and the associated components; L is the position constraint of the primitive component;

[0028] For the space class that provides component spatial positioning, the knowledge expression is C0={C,R L ,F C}, C is the mortise and tenon set of the component; P L is the topological relationship of the components; F C It is the modeling feature attribute.

[0029] Preferably, the knowledge extraction in step S13 includes bracket knowledge extraction at the face primitive level, specifically,

[0030] Based on the extracted surface primitives’ normal, position, size, and primitive face type, the relationship between geometric primitives and primitive components is established through data collation. The knowledge expression of primitive components is:

[0031] P S ={S C ,O S ,C S ,R S ,L S}

[0032] Among them, P S is a primitive component; S C is the set of surface primitives that constitute the primitive component, S C ={S i |i=1,2,…,N}, N is the number of primitives that make up the component surface; S C is the spatial area where the primitive component is a composite component; S is the associated primitive component; R S is the topological relationship between surface primitives; L S is the position constraint;

[0033] Considering the shape, normal, position, topological relationship of the bracket component surface primitives and whether the surface primitives are hidden and shared, the knowledge expression of a single surface primitive of the primitive component point cloud is:

[0034] S i ={type,normal,loc,rel,iSvalid1,iSvalid2,P i}

[0035] Among them, type is the type of surface primitive, normal is the direction of the surface primitive, loc is the position of the surface primitive, rel is the topological relationship of the surface primitive, iSvalid1 indicates whether the surface primitive is shared by different components, and iSvalid2 indicates whether the surface primitive is hidden; P i is the surface primitive S i The primitive component to which it belongs;

[0036] Finally, the knowledge of bracket components and point cloud is organized, semantic, spatial and attribute relationships are established, and connections between nodes are realized.

[0037] Preferably, the principles that need to be followed when semantically annotating real bracket sets are: external relations are superior to internal relations, topological structures are superior to dimensional relations, and the geometric dimensions of components need to include a certain amount of adjustment.

[0038] Preferably, step S2 specifically includes the following contents:

[0039] S21, performing supervoxel clustering on the bracket point cloud to form a bracket cluster point cloud surface set;

[0040] S22, searching the point cloud faces in the bracket arch cluster point cloud face set, matching the retrieved point cloud faces with the bracket arch theoretical primitive face point cloud annotation knowledge graph according to the relative normal relationship and position, and determining whether the point cloud faces are valid;

[0041] S23. When the point cloud surface is an invalid surface, it means that it is shared by multiple components. The point cloud constituting the point cloud surface cannot be used for component annotation. It is necessary to return to the bracket cluster point cloud surface set and search again.

[0042] S24. When the point cloud surface is a valid surface, it indicates that it is a primitive surface unique to a single component. Return to the bracket arch theoretical primitive surface point cloud annotation knowledge graph. Based on the position, topological relationship, and shape of the theoretical primitive surface and the point cloud surface, determine the component to which the point cloud surface belongs. Then, obtain the set of point cloud surfaces adjacent to the point cloud surface based on the component.

[0043] S25. Annotate the valid point cloud surfaces and adjacent point cloud surfaces according to the components to form a component point cloud collection, and search the point cloud surfaces in the bracket arch cluster point cloud surface collection one by one until all point cloud surfaces are traversed to complete the correct annotation of the bracket arch point cloud.

[0044] The beneficial effects of the present invention are: on the basis of existing knowledge, starting from the two aspects of bracket set knowledge and point cloud, it fully integrates the geometric primitive surface position, normal and other attributes of the bracket set point cloud and the concepts of component type, position and so on in the bracket set knowledge, and constructs a knowledge graph for bracket set primitive point cloud annotation. Different from the existing ancient building knowledge base that uses ideal component size templates as knowledge units, this knowledge graph includes the consideration of the detailed features of the real bracket set point cloud morphology, and is more effective for actual bracket set point cloud annotation work. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of bracket types in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the logical structure of bracket sets in an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of a bracket member in an embodiment of the present invention;

[0048] Figure 4 Schematic diagram of a single-arch bracket at the doukou in an embodiment of the present invention;

[0049] Figure 5 is a schematic diagram of a knowledge graph constructed in an embodiment of the present invention;

[0050] Figure 6 This is a logic flow chart of bracket point cloud annotation in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of model point cloud data in an embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of a bracket set image in an embodiment of the present invention;

[0053] Figure 9 Schematic diagram of collecting point cloud data by photogrammetry in an embodiment of the present invention;

[0054] Figure 10 Schematic diagram of point cloud data collected by a scanner in an embodiment of the present invention;

[0055] Figure 11 is a schematic diagram of geometric primitive extraction in an embodiment of the present invention;

[0056] Figure 12 This is a schematic diagram of a primitive annotation merging structure in an embodiment of the present invention;

[0057] Figure 13 2 is a schematic diagram of a knowledge-guided labeling error matrix according to an embodiment of the present invention; DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] Example 1

[0060] In order to solve the problems caused by the occlusion of combined components, the breakage of the same component, and the overlap of different components in the semantic annotation of bracket arch primitive components, the present invention constructs a knowledge graph for bracket arch primitive point cloud annotation based on the analysis and summary of existing knowledge. Starting from the two aspects of bracket arch knowledge and point cloud, the geometric primitive surface position, normal and other attributes of the bracket arch point cloud and the concepts of component type, position and so on in the bracket arch knowledge are fully integrated. On this basis, a logical framework for the mutual mapping of bracket arch point cloud features and knowledge in primitive point cloud annotation is proposed, and the construction of the knowledge graph for bracket arch primitive point cloud annotation is completed. Different from the existing ancient building knowledge base that uses the ideal component size template as the knowledge unit, this knowledge graph includes the consideration of the detailed features of the real bracket arch point cloud morphology, and is more effective for actual bracket arch point cloud annotation work. The method of the present invention specifically includes the following two parts:

[0061] 1. Construction of knowledge graph for point cloud annotation of bracket theory primitives

[0062] By designing the concept layer of the bracket arch and extracting knowledge from the bracket arch component level and the surface primitive level, a knowledge graph of bracket arch theoretical primitive surface point cloud annotation is constructed. The specific implementation process is as follows:

[0063] S11. First, the construction of the overall knowledge graph is guided by the mutual mapping logical relationship between the bracket point cloud features and the primitive point cloud annotations.

[0064] S12. Then, starting with the bracket structure knowledge and bracket point cloud data, the conceptual layer of the graph is designed, and the knowledge of nodes, relationships, attributes, etc. is designed.

[0065] S13. Then, bracket primitive surface knowledge is extracted from the component entity related knowledge and the point cloud morphological primitive surface knowledge respectively, and the messy knowledge is processed into a structured data table.

[0066] S14. Use the knowledge of bracket arch entities as a framework to perform knowledge processing and complete the framework of the bracket arch primitive point cloud annotation knowledge graph. Finally, integrate the bracket arch point cloud surface knowledge and use Neo4j primitive software for knowledge management to form a complete bracket arch primitive point cloud annotation knowledge graph.

[0067] 1.1. Knowledge of Dougong

[0068] The arch of wooden architecture, also known as DouGong, DouKe, PuZuo, etc., is a unique structure of Chinese architecture. Based on the position of the arch in ancient buildings, the space where the arch is located can be divided into capitals, inter-column spaces, corners, etc. Each space contains different types of arches (such as Figure 1 As shown in the figure, there are capital family, flat body family, corner family, etc.), the type of bracket determines its spatial position.

[0069] Dougong is composed of components. Different types of dougong contain different component entities and are classified according to the component's shape (dou, sheng, gong, etc.). Components of different shapes occupy different positions within the dougong, and the position of the components in turn constrains the component's shape. Dougong's shape is extremely complex, and the components follow specific construction rules, making it a complex structure with mutually constrained components.

[0070] At the knowledge level of bracket arch structure, the spatial structural form of bracket arch, primitive components and their connection relationship are analyzed, and the classification is based on the spatial position, shape and component entity of bracket arch. According to the spatial position and type of bracket arch, it is classified and further divided into component entities, such as dou, sheng, ang, qiao, gong and other categories, as well as specific primitive components (such as da dou, san cai sheng, etc.). In essence, bracket arch is a combination. Its combination logic is as follows: Figure 2 shown.

[0071] 1.2. Extraction of Dougong Knowledge

[0072] The shape of a dougong is extremely complex, and its components adhere to specific construction rules, making it a complex structure with mutually constrained components. Due to the unique nature of dougong, a unified classification standard currently lacks. Instead, it can be divided into multiple categories based on different characteristic attributes. First, we review knowledge about dougong construction. Based on their position in ancient architecture, the space occupied by dougongs is divided into capitals, inter-column spaces, and corners. Each spatial location contains different types of dougong (capital section, flat section, corner section, etc.), and the type of dougong determines its spatial location. Dougongs are composed of components, and different types of dougongs contain different component entities. These components are classified into different categories based on their form (dou, sheng, gong, etc., as shown in Table 1). Components of different shapes occupy different positions within a dougong, and component positions in turn constrain their form, as shown in Table 2. The spatial position, type, and component entities of dougongs constrain and interact with each other, forming a comprehensive knowledge system for dougong construction. When annotating dougong point cloud primitives, breaks and overlaps in dougong components occur on geometric primitive surfaces. Therefore, when constructing the knowledge base, it is important to focus on the representation of primitive surface knowledge. Through the properties of the component entity such as name, size, topological relationship, position and primitive face, we can further deduce the knowledge of the primitive face normal, position and shape, and complete the construction of the face knowledge part.

[0073] Table 1 Component name

[0074]

[0075]

[0076] Table 2 Component Description

[0077]

[0078] (1) Extraction of bracket knowledge at the component level

[0079] Essentially, the combination relationship of the primitive components in the bracket combination is expressed as follows: a given component combination A = (p1, p2…p n ), where the primitive component p i and p j is the primitive component of the component assembly A, and the combination constraint matrix of the primitive component is,

[0080]

[0081] Among them, α i,j Represents component p i and p j The combination relationship between i,j =0 or α j,i =0, and i≠j, then p i and p j There is no combination relationship between them; if α i,j =1 or α j,i =1, and i≠j, then p j and p j There is a combination relationship between them.

[0082] The component class provides the component set that makes up the ancient building, the connection class provides the connection relationship between the components, and the space class provides the spatial positioning of the components. The three concepts are defined as follows:

[0083] ① Component type: Components can be classified according to their characteristic factors. P ,N}, A is the physical set of all components that make up the ancient building, F P is the set of factors that affect the category of primitive components, and N is the number of influencing factors. Spatial attributes, geometric attributes and structural attributes.

[0084] ② Space class: The space class mainly performs spatial positioning on components. The space class of a component S={O,C R ,R C ,L}, O is the spatial area of ​​the primitive component; C R R is the component associated with the primitive component in the spatial region; C is the relationship between the primitive component and the associated components; L is the position constraint of the primitive component.

[0085] like Figure 3 As shown, for a single-arch bracket, O represents the spatial area of ​​the bracket, and the spatial area of ​​the primitive component is defined as:

[0086] O i ={x imin ≤l xi ≤x imax ,y imin ≤l yi ≤y imax ,z imin ≤h zi ≤z imax |i∈N}

[0087] l xi 、l yi Indicates that components are generally arranged in the direction of depth (or width), h zi Indicates the height of the component in the bracket. R Represents the associated component, C R =2 means there are two components associated with it, R C Indicates the relationship with the associated components. The L position constraint indicates that the components are arranged along the depth (or width). The spatial position of the bucket can be expressed as follows:

[0088] P dadou ={O i ∈O∩C R ∩L i}

[0089] ③ Connection class: The connection class is an abstract expression of the component connection object. The connected components in the assembly structure are connected by mortise and tenon. The connection class C0 = {C, R L ,F C}, C is the mortise and tenon set of the component; R L is the topological relationship of the components; F C It is a modeling feature attribute. In point cloud semantic annotation, the connection class cannot be obtained by the point cloud, so it can only exist as a logical connection.

[0090] (2) Knowledge extraction at the face element level

[0091] The collected point cloud is attached to the surface of the bracket arch. The bracket arch primitive annotation is essentially the annotation of each face primitive of the bracket arch. On the other hand, compared with the point features and line features of the point cloud, the surface feature extraction is easier and more stable. Therefore, in order to achieve this, it is necessary to further establish the face primitive knowledge of the bracket arch primitive component. The primitive component is an entity that can be composed of a set of closed faces, and each face is represented by the definition of the surface it is on plus its boundary. The boundary of the face is the union of the edges, and the edges are represented by points. The face primitives of the primitive component can be represented by the boundary representation (Boundary Representation is also called B-Rep representation).

[0092] The extracted surface primitive attributes are mainly normal, position, size, type of primitive face, etc. By sorting out the data, the relationship between geometric primitives and primitive components is established. The ideal primitive component in the semantic annotation process is expressed as follows:

[0093] P S ={S C ,O S ,C S ,R S ,L S}

[0094] Among them, P S is a primitive component; S C is the set of surface primitives that constitute the primitive component, S C ={S i |i=1,2,…,N}, N is the number of primitives that make up the component surface; S C is the spatial area where the primitive component is a composite component; S is the associated primitive component; R S is the topological relationship between surface primitives; L S Position constraints. Ideal primitive components are constructed strictly in accordance with historical architectural knowledge. From a single surface primitive, not only can the spatial distribution and geometric dimensions of all the surface primitives of the entire component be determined, but also hidden structural information can be obtained. This can be further used for semantic annotation of point clouds.

[0095] However, due to the influence of construction and deformation, the actual bracket sets are not exactly the same as the geometric dimensions and relationships of the components specified by the construction rules. When performing semantic annotation, the following principles should be followed:

[0096] ① External relations are better than internal relations. Figure 4 As shown in (a), s i It is the real geometric primitive surface of the grasshopper head with hemp leaf head. Based on the construction rules of the ideal component, the positioning information of the grasshopper head with hemp leaf head can be derived (such as Figure 4 However, due to the influence of processing, the actual center position of Moukou is not consistent with the derivation result. Therefore, the two primitive surfaces s of Wangong are used to calculate the center position of Moukou. m and s n The thickness between the two can determine the slot width s' of the Mou mouth m s′ n And the center line of Moukou, etc.

[0097] ②Topological structure is superior to size relationship. The combination of brackets results in the extraction of exposed surface s j It is composed of three components (grasshopper head with hemp leaf head, single ang and supporting wood). Similarly, according to the construction knowledge and the extracted primitive surface s iIt can be deduced that the surface s of the component grasshopper head with hemp leaf head j2 , however, through s a and s b Locate the primitive face s j2 The range is more accurate and reliable.

[0098] ③ The geometric dimensions of the components need to be adjusted to a certain extent. Taking the complex primitive surfaces such as the grasshopper head with hemp leaf head and the plane, curved surface, and inclined surface in the middle of the single-angle bracket as an example, the extracted geometric primitives are s i ,s j ,s k and s l A local coordinate system is established with the center axis of the bracket as the origin, the front of the depth as the positive direction of the X axis, and the right of the width as the positive direction of the Y axis to represent the semantic information of the component's surface primitives.

[0099] Considering the shape, normal, position, topological relationship of the bracket component surface primitives and whether the surface primitives are hidden and shared, the knowledge expression of a single surface primitive in the primitive component point cloud is as follows:

[0100] S i ={type,normal,loc,rel,iSvalid1,iSvalid2,P i}

[0101] S i ={type,normal,loc,rel,iSvalid1,iSvalid2,P i}

[0102] Among them, type is the type of surface primitive, normal is the direction of the surface primitive, loc is the position of the surface primitive, rel is the topological relationship of the surface primitive, iSvalid1 indicates whether the surface primitive is shared by different components, and iSvalid2 indicates whether the surface primitive is hidden; P i is the surface primitive S i Finally, organize the knowledge of bracket structure and point cloud, establish semantic, spatial, and attribute relationships, and connect nodes. When annotating, you can use the method shown in Table 3.

[0103] Table 3 Relationship between the semantics and geometric primitives of Qing-style doukou single-angle flat body primitive components

[0104]

[0105] (3) Knowledge storage management

[0106] A bottom-up manual construction method is used to construct a knowledge graph of bracket arch primitive point cloud annotation. The present invention uses the Neo4j graph source database management system to store data. Neo4j uses graphs to represent and store data. The data consists of nodes and relationships. This model is very suitable for representing entities such as bracket arches, which have many types of components and complex connection relationships. The graph source data can be used to more intuitively and effectively visualize and call the knowledge of bracket arch components. The data after knowledge processing is further processed into a structured data CSV file, as shown in Table 4 below.

[0107] Table 4 CSV file input format for bracket component entities (excerpt)

[0108]

[0109] Import these data into Neo4j software. According to the model layer structure, the domain bracket annotation knowledge graph is constructed according to the rules defined above, forming a component annotation knowledge graph for bracket point cloud. Figure 5 shown.

[0110] 2. Knowledge Graph-Based Labeling of Bracket Primitive Components

[0111] In order to more effectively annotate the real bracket point cloud and solve the problem of component semantic annotation and feature extraction caused by the complex shapes and mutual combination of components, the bracket component knowledge graph is applied to the entire process of point cloud annotation: the bracket point cloud is super voxel clustered to obtain the bracket cluster point cloud face set, and the point cloud face retrieved from the bracket cluster point cloud face set is matched with the bracket theory primitive face point cloud annotation knowledge graph to determine whether it is valid. When the point cloud face is valid, the component to which the point cloud face belongs and the point cloud faces adjacent to the component are determined based on the bracket theory primitive face point cloud annotation knowledge graph, and these point cloud faces are annotated. The logical framework is as follows Figure 6 The specific process is as follows:

[0112] S21, performing supervoxel clustering on the bracket point cloud to form a bracket cluster point cloud surface set;

[0113] S22, searching the point cloud faces in the bracket arch cluster point cloud face set, matching the retrieved point cloud faces with the bracket arch theoretical primitive face point cloud annotation knowledge graph according to the relative normal relationship and position, and determining whether the point cloud faces are valid;

[0114] S23. When the point cloud surface is an invalid surface, it means that it is shared by multiple components. The point cloud constituting the point cloud surface cannot be used for component annotation. It is necessary to return to the bracket cluster point cloud surface set and search again.

[0115] S24. When the point cloud surface is a valid surface, it indicates that it is a primitive surface unique to a single component. Return to the bracket arch theoretical primitive surface point cloud annotation knowledge graph. Based on the position, topological relationship, and shape of the theoretical primitive surface and the point cloud surface, determine the component to which the point cloud surface belongs. Then, obtain the set of point cloud surfaces adjacent to the point cloud surface based on the component.

[0116] S25. Annotate the valid point cloud surfaces and adjacent point cloud surfaces according to the components to form a component point cloud collection, and search the point cloud surfaces in the bracket arch cluster point cloud surface collection one by one until all point cloud surfaces are traversed to complete the correct annotation of the bracket arch point cloud.

[0117] Example 2

[0118] To verify the effectiveness of the knowledge graph proposed in this paper, this knowledge graph was applied to a supervoxel region-growing bracket segmentation experiment and the accuracy of bracket segmentation experiments using PointNet and PointNet++ neural networks was compared. The experimental results show that introducing bracket knowledge into the primitive annotation process can significantly improve the accuracy of the annotation results. This method provides new ideas and methods for complex entity point cloud segmentation. The experiment specifically includes the following content:

[0119] 1. Experimental environment and data preparation

[0120] This experiment is divided into two parts: supervoxel region growing and knowledge-guided primitive annotation. The region growing experimental system environment is Visual Studio 2019 + PCL1.11.1 version, which imports the required libraries and additional dependencies. The geometric primitive and merging experimental system environment is PyCharm Community Edition 2022.2.4 + Python 3.11. To verify the universality and applicability of the point cloud annotation logic framework proposed in this invention, experiments were conducted using two types of model brackets, two photogrammetry point clouds, and one scanner point cloud data.

[0121] (1) Model data: Model data is obtained by forward modeling, drawing CAD drawings with reference to the knowledge of ancient books and documents, modeling through 3DSMAX software and extracting mesh point cloud data, such as Figure 7 shown.

[0122] (2) Image point cloud data: By collecting pictures of the bracket arch dataset and using the picture data to build a model, a bracket arch point cloud is generated. Since the bracket arch model is small in size and complex in shape, a circular route is used to shoot in three layers: upper, middle, and lower. The overlap of each photo is greater than 80%, and all-round and full-angle data is collected. A total of more than 300 pictures of each type of bracket arch are collected, as shown below. Figure 8 As shown in the figure. The final point cloud data is as follows Figure 9 shown.

[0123] (3) Data collected by scanner: The data of the xx Hall of xx Temple was collected on site and processed. The 3D laser scanning equipment was a Faros 350 3D scanner, and the parameters are shown in Table 5. The process of obtaining the point cloud data of the dougong by 3D laser scanning was based on the Beijing local standard DB11 / T 1796-2020 Technical Specification for 3D Information Collection of Cultural Relics Buildings. The data collection process includes control measurement, scanning station layout, station scanning, field data inspection, data export and backup. The scanning point cloud results are shown in Table 5. Figure 10 shown.

[0124] Table 5 Faros 350 3D scanner parameters

[0125]

[0126] 2. Experimental results

[0127] (1) Supervoxel region growth experiment

[0128] All dougong point cloud data are converted into supervoxel data through the supervoxel algorithm. For the model data, its point cloud quality is high, while the real dougong point cloud quality is low and the data volume is relatively sparse. Therefore, when conducting experiments on different point clouds, the parameters are set separately. After converting the dougong point cloud data into supervoxel data, the geometric primitives of the dougong components are directly extracted through the supervoxel region growing algorithm. The geometric primitive extraction results are shown in the figure below. Figure 11 As shown in Table 6, the extracted geometric primitive faces are statistically analyzed.

[0129] Table 6 Number of geometric primitives

[0130]

[0131] (2) Knowledge-guided primitive annotation experiment

[0132] The point cloud results obtained from the region growing experiment and the relative normal relationship file of the point cloud face set are used as data input to conduct a knowledge-guided geometric primitive merging experiment in Pycharm. The experiment uses position and normal dual constraints to match the point cloud face and the knowledge base to increase the accuracy of the primitive face attribution judgment. At the same time, in order to improve the compatibility between the knowledge base and the point cloud face, when processing the point cloud of invalid faces, the judgment range parameter is increased to improve the matching tolerance, and the merging result is as follows: Figure 12 Finally, the error matrix is ​​as follows Figure 13 As shown in Table 7, the error matrix analysis is used to obtain the labeling results of each bracket knowledge-guided primitive, as shown in Table 7.

[0133] Table 7 Accuracy analysis

[0134]

[0135] 3. Experimental analysis

[0136] In order to better verify the effectiveness of this method, the present invention uses PointNet and PointNet++ neural networks to conduct comparative experiments on the segmentation of bracket sets with single-arm flat body and single-arm column head models.

[0137] The combined accuracy of the annotation of the four dougong data was analyzed, and the number of correct annotations, the number of incorrect annotations, and the accuracy were used as indicators to measure the quality of the annotation. The measurement indicators are shown in Table 8.

[0138] Table 8 Labeling accuracy

[0139]

[0140]

[0141] As can be seen from the table, the accuracy of geometric primitive annotation for most components is above 90%, and the accuracy for a very small number of components with more complex modeling features is between 85% and 90%. The annotation effect is better for components with regular shapes and simple modeling features, such as the head beam, the center beam, and the cantilever beam, and can achieve complete annotation accuracy. For components with more complex shapes, such as the center melon bracket, the center Wan bracket, and the groove riser, which are fixed with mortise and tenon structures, there are many incorrectly labeled geometric primitive components. This may be due to the overlap of components such as the center melon bracket and the center Wan bracket with geometric primitive faces in spatial position, resulting in the incorrect annotation of the component primitive faces during the annotation process, resulting in a decrease in annotation accuracy. Overall, whether it is model data or real data, this method can effectively annotate and merge component primitive faces, thereby completing the segmentation of bracket components.

[0142] 4. Conclusion

[0143] Existing knowledge bases lack consideration of the true detailed features of bracket sets. Existing algorithms cannot directly meet the labeling requirements for the same component fracture in the bracket set component point cloud morphology. To address the bracket set component labeling problem, the present invention takes bracket set structural knowledge and point clouds into consideration, fully integrating the position, normal, and other attributes of the bracket set point cloud geometric primitives with concepts such as component type and position in bracket set knowledge. This paper proposes an expression framework for mapping bracket set point clouds and entity knowledge, and completes a knowledge graph for bracket set primitive point cloud labeling. Comprehensive experiments have shown that compared to other algorithms, the present method is more accurate in bracket set component labeling and achieves excellent performance in various types of bracket set point clouds.

[0144] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0145] The present invention discloses a knowledge graph construction method for bracket arch primitive point cloud annotation. Based on existing knowledge, starting from bracket arch knowledge and point cloud, it fully integrates the geometric primitive surface position, normal and other attributes of bracket arch point cloud as well as concepts such as component type and position in bracket arch knowledge, and constructs a knowledge graph for bracket arch primitive point cloud annotation. Different from the existing ancient building knowledge base that uses ideal component size templates as knowledge units, this knowledge graph takes into account the detailed features of the actual bracket arch point cloud morphology, and is more effective for actual bracket arch point cloud annotation work.

[0146] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A knowledge graph construction method for bracket arch primitive point cloud annotation, characterized by: The following steps are included: S1. Construction of knowledge graph for face point cloud annotation of bracket arch theory primitives: By designing the graph concept layer of bracket arch and extracting knowledge from the bracket arch component level and face primitive level, a knowledge graph for face point cloud annotation of bracket arch theory primitives is constructed; S2. Labeling of bracket arch primitive components based on knowledge graph: Super voxel clustering is performed on the bracket arch point cloud to obtain the bracket arch cluster point cloud face set, and the point cloud face retrieved from the bracket arch cluster point cloud face set is matched with the bracket arch theoretical primitive face point cloud labeling knowledge graph to determine whether it is valid. When the point cloud face is valid, the component to which the point cloud face belongs and the point cloud faces adjacent to the component are determined based on the bracket arch theoretical primitive face point cloud labeling knowledge graph, and these point cloud faces are labeled.

2. The knowledge graph construction method for bracket arch primitive point cloud annotation according to claim 1 is characterized by: Step S1 specifically includes the following contents: S11. Based on the mutual mapping logical relationship between the bracket point cloud features and the primitive point cloud annotations, the overall knowledge graph is constructed; S12. Starting with bracket arch structure knowledge and bracket arch point cloud data, design the graph concept layer and design nodes, relationships, and attributes. S13, extracting bracket primitive surface knowledge from component entity related knowledge and point cloud morphological primitive surface knowledge, and processing the messy knowledge into a structured data table; S14. Use the knowledge of bracket arch entities as a framework for knowledge processing to complete the framework of the bracket arch primitive point cloud annotation knowledge graph. Integrate the bracket arch point cloud surface knowledge and use Neo4j primitive software for knowledge management to form a complete bracket arch primitive point cloud annotation knowledge graph.

3. The knowledge graph construction method for bracket arch primitive point cloud annotation according to claim 2 is characterized by: In step S12, at the knowledge level of bracket set structure, the bracket set spatial structure form, primitive components and their connection relationship are analyzed, and the bracket set spatial position, shape and component entity are used as the basis for classification. The bracket set is classified according to its spatial position and type, and the component entity and specific primitive components are further divided.

4. The knowledge graph construction method for bracket arch primitive point cloud annotation according to claim 2 is characterized in that: The knowledge extraction in step S13 is specifically to further deduce the primitive surface normal, position and shape knowledge through the name, size, topological relationship, position relationship and primitive surface of the component entity, so as to realize the extraction of the bracket primitive surface knowledge.

5. The knowledge graph construction method for bracket arch primitive point cloud annotation according to claim 4 is characterized in that: The knowledge extraction in step S13 includes the extraction of bracket knowledge at the component level, specifically, Given a component assembly A=(p1,p2…p n ), where the primitive component p i and p j is the primitive component of the component assembly A, and the combination constraint matrix of the primitive component is, Among them, if α i,j =0 or α j,i =0, and i≠j, then p i and p j There is no combination relationship between them; if α i,j =1 or α j,i =1, and i≠j, then p i and p j There is a combination relationship between them; For the component class of ancient buildings, the knowledge expression is P = {A,F p ,N}, A is the physical set of all components that make up the ancient building, F P is the set of factors that affect the primitive component category, and N is the number of influencing factors; For the connection class that provides component connection relationships, the knowledge expression is S={O,C R ,R C ,L}, O is the spatial area of ​​the primitive component; C R R is the component associated with the primitive component in the spatial region; C is the relationship between the primitive component and the associated components; L is the position constraint of the primitive component; For the space class that provides component spatial positioning, the knowledge expression is C0={C,R L ,F C }, C is the mortise and tenon set of the component; R L is the topological relationship of the components; F C It is the modeling feature attribute.

6. The knowledge graph construction method for bracket arch primitive point cloud annotation according to claim 4 is characterized in that: The knowledge extraction in step S13 includes the bracket knowledge extraction at the face primitive level, specifically, Based on the extracted surface primitives’ normal, position, size, and primitive face type, the relationship between geometric primitives and primitive components is established through data collation. The knowledge expression of primitive components is: P S ={S C ,O S ,C S ,R S ,L S } Among them, P S is a primitive component; S C is the set of surface primitives that constitute the primitive component, S C ={S i |i=1,2,…,N}, N is the number of primitives that make up the component surface; S C is the spatial area where the primitive component is a composite component; S is the associated primitive component; R S is the topological relationship between surface primitives; L S is the position constraint; Considering the shape, normal, position, topological relationship of the bracket component surface primitives and whether the surface primitives are hidden and shared, the knowledge expression of a single surface primitive of the primitive component point cloud is: S i ={type,normal,loc,rel,iSvalid1,iSvalid2,P i } Among them, type is the type of surface primitive, normal is the direction of the surface primitive, loc is the position of the surface primitive, rel is the topological relationship of the surface primitive, iSvalid1 indicates whether the surface primitive is shared by different components, and iSvalid2 indicates whether the surface primitive is hidden; P i is the surface primitive S i The primitive component to which it belongs; Finally, the knowledge of bracket components and point cloud is organized, semantic, spatial and attribute relationships are established, and connections between nodes are realized.

7. The knowledge graph construction method for bracket arch primitive point cloud annotation according to claim 6, characterized in that: The principles that need to be followed when semantically annotating real bracket sets are: external relationships are superior to internal relationships, topological structures are superior to dimensional relationships, and the geometric dimensions of components need to include a certain amount of adjustment.

8. The knowledge graph construction method for bracket arch primitive point cloud annotation according to any one of claims 1 to 7, characterized in that: Step S2 specifically includes the following contents: S21, performing supervoxel clustering on the bracket point cloud to form a bracket cluster point cloud surface set; S22, searching the point cloud faces in the bracket arch cluster point cloud face set, matching the retrieved point cloud faces with the bracket arch theoretical primitive face point cloud annotation knowledge graph according to the relative normal relationship and position, and determining whether the point cloud faces are valid; S23. When the point cloud surface is an invalid surface, it means that it is shared by multiple components. The point cloud constituting the point cloud surface cannot be used for component annotation. It is necessary to return to the bracket cluster point cloud surface set and search again. S24. When the point cloud surface is a valid surface, it indicates that it is a primitive surface unique to a single component. Return to the bracket arch theoretical primitive surface point cloud annotation knowledge graph. Based on the position, topological relationship, and shape of the theoretical primitive surface and the point cloud surface, determine the component to which the point cloud surface belongs. Then, obtain the set of point cloud surfaces adjacent to the point cloud surface based on the component. S25. Annotate the valid point cloud surfaces and adjacent point cloud surfaces according to the components to form a component point cloud collection, and search the point cloud surfaces in the bracket arch cluster point cloud surface collection one by one until all point cloud surfaces are traversed to complete the correct annotation of the bracket arch point cloud.

Images