Automatic building modeling method based on multimodal information and knowledge graph

Abstract

This application relates to an automated building modeling method based on multimodal information and knowledge graphs. The method includes three key steps: first, a feature extraction and preprocessing process for modeling tasks is constructed for heterogeneous data such as drawings, text, and tables to ensure the accurate acquisition of key modeling parameters; second, a unified semantic framework is constructed based on the IFC standard, and the multimodal information knowledge graph is constructed with the help of the graph database Neo4j, realizing structured data management and semantic association modeling; finally, the semantic retrieval capability of the graph is combined with the IfcOpenShell programming interface to complete the automatic mapping of parameters and components and model generation, output IFC files that comply with industry standards, and realize the visualization of three-dimensional models. This method effectively improves the extraction integrity of multimodal data and the accuracy of model construction, providing new ideas for the automated construction and intelligent management of building information models.

Description

Technical Field

[0001] The present application relates to the technical field of building modeling, and in particular to an automated building modeling method based on multimodal information and knowledge graphs. Background Art

[0002] The building industry is a major source of global energy consumption and carbon emissions, accounting for approximately 40% of total global energy consumption, according to a United Nations Environment Report. The operations and maintenance (O&M) phase, due to its lengthy and complex nature, has gradually become the most energy-intensive phase of a building's lifecycle. In recent years, numerous studies have focused on controlling carbon emissions and improving energy efficiency in buildings, particularly proposing various approaches in areas such as spatial planning, facility management, fault detection, and energy optimization. However, the effective application of these approaches relies heavily on high-quality, structured building data. Currently, the data required for building O&M primarily consists of as-built data generated during the design and construction phases (such as construction drawings and as-built drawings) and O&M information accumulated during the building's lifetime (such as energy consumption records and equipment maintenance). This data is generally fragmented, heterogeneous, and frequently updated, making manual processing inefficient and unable to meet the current demand for intelligent and refined management.

[0003] To address the challenges posed by data fragmentation, Building Information Modeling (BIM) is widely used in full-lifecycle management. It not only supports the digital representation of a building's geometric and semantic information, but also serves as an information integration and collaboration platform. Research has shown that BIM-based operations and maintenance management can significantly improve management efficiency, reduce energy consumption, and facilitate the effective transfer of information across different stages and disciplines. However, due to their age, most existing buildings do not have BIM models and lack standardized information systems. Information on these buildings is primarily retained in unstructured materials such as two-dimensional drawings, specification texts, and explanatory documents. The lack of complete three-dimensional data suitable for modeling limits their digital reconstruction and intelligent management. Therefore, extracting key modeling parameters from existing multimodal data and automatically generating three-dimensional BIM models has become a core requirement for building digital transformation.

[0004] Current 3D modeling methods face numerous challenges when processing existing architectural data. First, design standards and drafting specifications vary across historical periods, resulting in complex and diverse architectural forms depicted in 2D drawings. This makes traditional approaches, which rely on pre-defined rules, less adaptable and generalizable when dealing with different representations of the same architectural object. Furthermore, current 3D architectural modeling faces widespread challenges such as reliance on a single data source for parameter extraction, fragmented multimodal information, a lack of a unified management mechanism, and low modeling efficiency. Summary of the Invention

[0005] Based on this, it is necessary to provide an automated building modeling method based on multimodal information and knowledge graphs to address the technical problems that are prevalent in the current three-dimensional building modeling process, such as parameter extraction relying on a single data source, multimodal information fragmentation, lack of a unified organizational mechanism, and low modeling efficiency.

[0006] An automated building modeling method based on multimodal information and knowledge graph, the method comprising:

[0007] Use vector graphics analysis and layer semantic recognition to extract component geometry information from architectural drawings;

[0008] Combining the BERT-BiLSTM-CRF model and dictionary enhancement strategy, named entity recognition is performed on architectural text to obtain component semantic parameters.

[0009] Component attribute information is extracted from building tables through table structure recognition and table logic reconstruction.

[0010] JSON files are used to store component geometry information, component semantic parameters, and component attribute information in a standardized manner, and a unique identifier ID mechanism is introduced to complete the association between multi-source data.

[0011] Build a node-relationship model based on the IFC standard, use the Neo4j knowledge graph to express the semantic, spatial and attribute relationships of components, and realize the unified storage and association modeling of multimodal data.

[0012] Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, GlobalID is used as the unique identifier of parameter data to achieve accurate docking between the knowledge graph and the IFC model.

[0013] By calling the IfcOpenShell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, realizing the automated construction of the building information model.

[0014] The above-mentioned automated building modeling method based on multimodal information and knowledge graphs includes three key steps: First, a modeling-oriented feature extraction and preprocessing process is constructed for heterogeneous data such as drawings, text, and tables to ensure the accurate acquisition of key modeling parameters; second, a unified semantic framework is constructed based on the IFC standard, and the multimodal information knowledge graph is constructed with the help of the graph database Neo4j, realizing structured data management and semantic association modeling; finally, the semantic retrieval capabilities of the graph are combined with the IfcOpenShell programming interface to complete the automatic mapping of parameters and components and model generation, outputting an IFC file that complies with industry standards and realizing the visualization of the 3D model. This method effectively improves the extraction integrity of multimodal data and the accuracy of model construction, providing new ideas for the automated construction and intelligent management of building information models. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 1 is a flow chart of an automated building modeling method based on multimodal information and knowledge graphs in one embodiment;

[0016] Figure 2 A flowchart of a method for automated building modeling based on multimodal information and knowledge graphs in one embodiment is provided;

[0017] Figure 3 A schematic diagram of a Neo4j visualization display of hierarchical relationship mapping in another embodiment;

[0018] Figure 4 A schematic diagram of Neo4j visualization of spatial ownership relationship mapping in another embodiment;

[0019] Figure 5 A schematic diagram of topological relationship mapping in another embodiment;

[0020] Figure 6 This is a flowchart of knowledge graph to IFC level coordinate conversion in another embodiment;

[0021] Figure 7 A complete flowchart for generating an IFC model from knowledge graph data retrieval in another embodiment;

[0022] Figure 8 This is an overview diagram of the knowledge graph in another embodiment;

[0023] Figure 9 A flowchart for building an IFC file based on Neo4j and IfcOpenShell in another embodiment;

[0024] Figure 10 A schematic diagram of a three-dimensional view analysis of an IFC model in another embodiment;

[0025] Figure 11 It is a schematic diagram of the cross-sectional view analysis of an IFC model in another embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0027] In one embodiment, Figure 1 、 Figure 2 As shown, a building automation modeling method based on multimodal information and knowledge graph is provided, which includes the following steps:

[0028] Step 100: Use vector graphics primitive analysis and layer semantic recognition to extract component geometry information from architectural drawings.

[0029] Specifically, in drawing processing, drawing division and alignment are completed through frame recognition and feature point registration, and invalid graphics elements are cleaned up and layer naming is standardized to achieve geometric parameter extraction under the joint analysis of vector graphics elements and layer semantics.

[0030] Architectural drawings can be, but are not limited to, 2D CAD drawings.

[0031] Step 102: Combine the BERT-BiLSTM-CRF model and the dictionary enhancement strategy to perform named entity recognition on the architectural text to obtain component semantic parameters.

[0032] Specifically, in text processing, OCR, layout structure analysis and semantic correction mechanism are combined to extract component attributes, and the BERT-BiLSTM-CRF model and dictionary rule fusion strategy are introduced to improve the accuracy of entity recognition.

[0033] Step 104: Extract component attribute information from the building table through table structure recognition and table logic reconstruction.

[0034] Specifically, in table processing, a method based on multi-scale windows and structure reconstruction is designed to effectively analyze the table structure and logical relationships and achieve complete extraction of attribute parameters.

[0035] Step 106: Use JSON files to standardize the storage of component geometry information, component semantic parameters, and component attribute information, and introduce a unique identifier ID mechanism to complete the association between multi-source data.

[0036] Step 108: Construct a node-relationship model based on the IFC standard, use the Neo4j knowledge graph to express the semantic, spatial and attribute relationships of components, and realize unified storage and association modeling of multimodal data.

[0037] Specifically, in order to solve the heterogeneity problem between drawings, texts and table data, a node-relationship model based on the IFC standard is constructed, and the Neo4j knowledge graph is used to express the semantic, spatial and attribute relationships of components to achieve unified storage and association modeling of multimodal data.

[0038] Step 110: Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, GlobalID is used as the unique identifier of the parameter data to achieve accurate docking between the knowledge graph and the IFC model.

[0039] Step 112: By calling the IfcOpenShell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, thereby realizing the automated construction of the building information model.

[0040] Specifically, by calling the Ifcopenshell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, realizing the automated construction of the building information model.

[0041] The aforementioned automated building modeling method based on multimodal information and knowledge graphs includes three key steps: First, a modeling-oriented feature extraction and preprocessing process is constructed for heterogeneous data such as drawings, text, and tables to ensure the accurate acquisition of key modeling parameters. Second, a unified semantic framework is constructed based on the IFC standard, and the multimodal information knowledge graph is constructed with the help of the graph database Neo4j, enabling structured data management and semantic association modeling. Finally, the semantic retrieval capabilities of the graph are combined with the IfcOpenShell programming interface to complete the automatic mapping of parameters and components and model generation, outputting an IFC file that complies with industry standards and enabling the visualization of 3D models. This method effectively improves the extraction integrity of multimodal data and the accuracy of model construction, providing new ideas for the automated construction and intelligent management of building information models.

[0042] In one embodiment, step 100 includes: in terms of drawing parameter extraction, establishing a complete extraction process from drawing preprocessing, component identification, topological relationship reconstruction to parameter structured output; through layer standardization and primitive geometric feature analysis, accurate extraction of basic component geometric position information is achieved; basic components include walls and doors and windows.

[0043] Specifically, the completeness and accuracy of building information extraction are key to achieving accurate 3D building modeling. Architectural data primarily comes from drawings, text, and tables, each carrying geometric information, attribute parameters, and component relationships, providing data support for building design, construction, and management. However, existing research has largely focused on extracting geometric information from drawings, neglecting the attribute information and component relationships contained in text and tables. This results in incomplete model semantics and missing component parameters, hindering the construction of accurate building information models.

[0044] Architectural data is diverse and complementary. Different types of data, each with its own focus on information expression, collectively constitute a complete building information system. Drawing data primarily records the geometric form, dimensional parameters, and spatial relationships of components, serving as the core source of spatial information for the model. Text data focuses on material properties, construction requirements, and functional attributes, supplementing non-geometric information missing from drawings. Tabular data, storing door and window information, material structure, and construction methods in a standardized format, possesses a strong structured nature and further complements the physical properties and parameters of components. These three data elements complement each other, supporting the completeness and accuracy of the building model.

[0045] For drawing parameter extraction, a complete extraction process is established, spanning drawing preprocessing, component identification, topological relationship reconstruction, and structured parameter output. Through layer normalization and primitive geometric feature analysis, accurate geometric positional information of basic components, such as walls, doors, and windows, is extracted. For architectural text parsing, a multi-strategy fusion parameter extraction model, PENet, is proposed to address the challenges of diverse parameter expressions, unit confusion, and character misrecognition. This model integrates term normalization, semantic encoding based on a pretrained language model (BERT), contextual modeling using a Bidirectional Long Short-Term Memory (BiLSTM) network, and a label sequence optimization module using a Conditional Random Field (CRF). This model effectively improves the recognition accuracy and boundary integrity of component parameter entities. For table parameter extraction, a three-stage process, "region screening - structure reconstruction - content identification," is constructed. Layer features and a multi-scale search strategy are utilized to effectively extract parameters from various types of tables. Furthermore, this paper utilizes JSON files to standardize the storage of parameter information from different sources and introduces a unique identifier (ID) mechanism to link data from multiple sources.

[0046] The pseudo code for storing JSON files is as follows:

[0047] geometry.json / / Main content: geometric information;

[0048] {

[0049] "id": "W01": / / Define the wall number;

[0050] "geometry":{ / / Define geometric dimensions;

[0051] "start": [-7335.51, -10447.55, 0.0], / / starting point coordinates (x, y, z);

[0052] "length":: 4500.0, / / wall length;

[0053] "thickness": 240.0, / / wall thickness;

[0054] "height": 3799.99, / / wall height;

[0055] "direction" :: [l, 0, 0] , / / Wall direction (vector);

[0056] };

[0057] "polygon": [ / / Define the polygon vertex coordinates of the wall in space (usually used for spatial positioning);

[0058] [-7335.51,-10447.55,0.0], [-2835.51,-10447.55.0.0], [-2835.51.-10207.55.0.0],

[0059] [-7335.51.-10207.55.0.0]]};

[0060] table.json / / Main content: physical performance parameters;

[0061] {

[0062] "id": "W01", / / Wall number;

[0063] "thermal_transmittance": 0.7941, / / heat transfer coefficient;

[0064] "weight": 12000.0, / / weight;

[0065] "U_value": 1.2, / / U value (thermal performance index);

[0066] };

[0067] text.json / / Main content: description information (mostly text attributes);

[0068] {

[0069] "id": "W01": / / Wall number

[0070] "name": "Wall 1" / / Wall name (such as "Wall 1")

[0071] "category": "Standard Wall!", / / The category is standard wall;

[0072] "floor": 1F, / / Floor (such as "1F")

[0073] "material": "concrete", / / material type is concrete

[0074] "function": Load-bearing, / wall function (such as load-bearing);

[0075] "fire_resistance": 2h, / / define the fire resistance limit (such as "2 hours");

[0076] }.

[0077] In one embodiment, step 108 includes: constructing a node-relationship model based on the IFC standard, and the node-relationship model based on the IFC standard includes: hierarchical structure relationship mapping, spatial ownership relationship mapping and topological connection relationship mapping; the hierarchical structure relationship mapping is used to standardize the affiliation between projects, buildings, floors and components; in the knowledge graph, according to the aggregation relationship model, HAS_BUILDING, HAS_FLOOR and CONTAINS_ELEMENT relationships are defined to build a hierarchical organizational structure of projects, buildings, floors and components; among them, HAS_BUILDING reflects the ownership relationship between projects and buildings; HAS_FLOOR indicates the affiliation between buildings and floors; CONTAINS_ELEMENT indicates the relationship between components contained in floors; by using the IFC relationship standard to construct a knowledge graph of multimodal data, building information data can be orderly and regularized in a multi-level structure, thereby realizing the layer-by-layer refinement from projects to components; the spatial ownership relationship mapping is used to clarify the structure The specific location, functional zoning and spatial dependency of components in the building environment; nodes in the knowledge graph are mapped to IfcBuilding (building layer), IfcBuildingStorey (floor) and IfcSpace (space layer), and the "Contains" and "FillsElement" relationship concepts are introduced to express the spatial affiliation and nesting status between components, ensuring data integrity and consistency; topological connection relationship mapping is used to describe the geometric connection, physical support and functional nesting between components, and is a key way to reflect the "physical contact" and "dependency" of components in the modeling process; in the knowledge graph, topological relationships are defined through the "AdjacentTo", "SupportedBy" and "HasOpening" relationship types; each node is given a GlobalID as a unique identifier, and combined with the entity name and description, it improves the completeness and queryability of model parameters.

[0078] Specifically, within a building information model (BIM), spatial relationships among components can be categorized into three types from the perspective of the IFC relational model: ① hierarchical affiliation; ② spatial ownership; and ③ topological connectivity. Based on these three types of relational models, this paper proposes a multidimensional mapping relational model encompassing three core modules: node mapping, relationship mapping, and attribute mapping. To ensure the orderly organization and efficient querying of the integrated multimodal data, this paper utilizes a graph database to perform relational modeling on the extracted multimodal data, achieving semantic association within the model.

[0079] (1) Hierarchical structure relationship mapping

[0080] The IFC standard standardizes the affiliation between projects, buildings, floors, and components using the IfcRelAggregates (IFC's aggregation model, used to describe spatial affiliation). This model utilizes a "whole-part" relationship to ensure the integrity and traceability of building data organization. In the knowledge graph, this article defines relationships such as HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT based on the aggregation model to establish a hierarchical organizational structure for projects, buildings, floors, and components. HAS_BUILDING reflects the affiliation between projects and buildings; HAS_FLOOR indicates the affiliation between buildings and floors; and CONTAINS_ELEMENT represents the relationship between components within a floor. By utilizing the IFC relationship standard to construct a knowledge graph for multimodal data, building information data can be organized and structured within a multi-level structure, enabling layer-by-layer refinement from projects to components.

[0081] (2) Spatial ownership relationship mapping

[0082] The IFC standard introduces the IfcRelContainedInSpatialStructure (component-spatial affiliation) relationship model to describe the affiliation between a component and the spatial unit in which it resides. This relationship is primarily used to clarify the specific location, functional zoning, and spatial dependencies of a component within a building environment.

[0083] In the model constructed in this example, nodes in the knowledge graph are mapped to spatial units such as IfcBuilding (building layer), IfcBuildingStorey (floor), and IfcSpace (space layer). To further express the spatial affiliation and nesting relationships between components, the graph introduces relationship concepts such as "Contains" and "FillsElement" to express the spatial affiliation and nesting between components, ensuring data integrity and consistency.

[0084] (3) Topological connection relationship mapping

[0085] Topological connection relationships are used to describe the geometric connections, physical support, and functional nesting between components. They are a key way to reflect the "physical contact" and "dependency" of components during the modeling process. They are essentially different from the "affiliation positioning" in spatial affiliation relationships.

[0086] The IFC standard introduces IfcRelConnectsElements (component connection relationships) and its subclass IfcRelFillsElement (component fill relationship), which standardizes the connection, nesting, and support relationships between components in building information models. In the knowledge graph constructed in this article, topological relationships are defined through relationship types such as "AdjacentTo," "SupportedBy," and "HasOpening," aiming to enhance the topological integrity and semantic expressiveness of the model.

[0087] After modeling the hierarchical relationships, spatial attribution, and topological connections of the BIM data, the knowledge graph successfully integrated the extracted 2D-to-3D modeling parameters and established relationships between components, forming a complete BIM data framework. Each node is assigned a GlobalID as a unique identifier, combined with attributes such as entity name and description, to further enhance the completeness and queryability of model parameters.

[0088] In one embodiment, the entity mapping mechanism based on the IFC standard includes an IFC component mapping strategy, an IFC parameter mapping strategy, and an identifier mapping strategy; the IFC component mapping strategy is a mapping rule proposed for typical building components based on the information description model of the IFC standard; the IFC parameter mapping strategy includes: geometric parameter mapping rules, physical parameter mapping rules, and functional parameter mapping rules; the IfcPropertySet (property set) and IfcMaterial (material) mechanisms are introduced to realize hierarchical mapping of parameters; the identifier mapping strategy is used to map the unique identification field of the component in the knowledge graph to the globally unique identifier GlobalID attribute of the IFC model. When the model is incrementally updated, the globally unique identifier exists as a unique identifier to maintain data consistency when adding, deleting or modifying component parameters. The globally unique identifier also serves as the basis for matching components from different data sources; at the same time, when querying data, the globally unique identifier is also used as an index field for retrieving models and query parameters for rapid positioning of components and their associated properties.

[0089] In one embodiment, the relationship mapping mechanism based on the IFC standard includes a mapping rule for a hierarchical relationship among components, a mapping rule for a spatial relationship among components, and a mapping rule for a topological connection relationship among components.

[0090] Specifically, within a building information model (BIM), spatial relationships among components can be categorized into three types from the perspective of the IFC relational model: ① hierarchical affiliation; ② spatial ownership; and ③ topological connectivity. Based on these three types of relational models, this paper proposes a multidimensional mapping relational model encompassing three core modules: node mapping, relationship mapping, and attribute mapping. To ensure the orderly organization and efficient querying of the integrated multimodal data, this paper utilizes a graph database to perform relational modeling on the extracted multimodal data, achieving semantic association within the model.

[0091] (1) Hierarchical structure relationship mapping

[0092] The IFC standard uses the IfcRelAggregates aggregation relationship model to standardize the affiliation between projects, buildings, floors, and components. This model uses the "whole-part" relationship to ensure the integrity and traceability of building data organization. In the knowledge graph, this article defines relationships such as HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT based on the aggregation relationship model to build a hierarchical organizational structure for projects, buildings, floors, and components. Among them, HAS_BUILDING reflects the affiliation between projects and buildings; HAS_FLOOR indicates the affiliation between buildings and floors; CONTAINS_ELEMENT represents the relationship between the components contained in a floor. By using the IFC relationship standard to construct a knowledge graph of multimodal data, building information data can be organized in an orderly manner in a multi-level structure, thereby achieving layer-by-layer refinement from projects to components.

[0093] The pseudo code for hierarchical relationship mapping is as follows:

[0094] Cypher is the query language for the Neo4j graph database, similar to SQL for relational databases. It is divided into two parts:

[0095] (1) Create a building hierarchy (node)

[0096] CREATE (:Project {id: "P001", name: "Industrial Building Project"}); / / Create a project node;

[0097] CREATE (:Building {id: "B001", name: "Office Building"}); / / Create a building;

[0098] CREATE (:Floor {id: "F001", name: "1F"}); / / Create a floor node;

[0099] CREATE (:Wall {id: "W001", height: 3.0, material: "Concrete"}); / / Create a wall node;

[0100] (2) Establishing hierarchical relationships (edges)

[0101] MATCH (p:Project {id: "P001"}), (b:Building {id: "B001"}) CREATE (p)-[:CONTAINS]->(b); / / Query when the project node P001 contains the building node B001, create the relationship between the nodes (buildings contain floors);

[0102] MATCH (b:Building {id: "B001"}), (f:Floor {id: "F001"}) CREATE (b)-[:CONTAINS]->(f); / / Query when building node B001 contains floor node F001, create the relationship between nodes (building contains floors);

[0103] MATCH (f:Floor {id: "F001"}), (w:Wall {id: "W001"}) CREATE (f)-[:HAS_COMPONENT]->(w); / / Query when floor node B001 contains wall node F001, create the relationship between nodes (floor has wall components).

[0104] The Neo4j visualization diagram of hierarchical relationship mapping is as follows Figure 3 shown.

[0105] (2) Spatial ownership relationship mapping

[0106] The IFC standard introduces the IfcRelContainedInSpatialStructure (component-spatial affiliation) relationship model to describe the affiliation between a component and the spatial unit in which it resides. This relationship is primarily used to clarify the specific location, functional zoning, and spatial dependencies of a component within a building environment.

[0107] In the model constructed in this example, nodes in the knowledge graph are mapped to spatial units such as IfcBuilding (building layer), IfcBuildingStorey (floor), and IfcSpace (space layer). To further express the spatial affiliation and nesting relationships between components, the graph introduces relationship concepts such as "Contains," "Belongs to," and "FillsElement." These concepts are used to express the spatial affiliation and nesting between components, ensuring data integrity and consistency.

[0108] The pseudo code for spatial belonging relationship mapping is:

[0109] Cypher is Neo4j's query language, and the MERGE statement is used to "find or create" nodes and relationships, ensuring there are no duplications;

[0110] (1) Create / find a node (entity)

[0111] MERGE (b:Building {id: "B001", name: "Office Building"}) / / Create or find the office building node;

[0112] MERGE (f:Floor {id: "F001", name: "1F"}) / / Create or find the 1F floor node;

[0113] MERGE (s:Space {id: "S001", name: "Office"}) / / Create or find office space node;

[0114] MERGE (w:Wall {id: "W001", height: 3.0, material: "Concrete"}) / / Create or find "concrete wall node;

[0115] MERGE (d:Door {id: "D001", height: 2.1, material: "wood"}) / / Create or find a wooden door node;

[0116] (2) Create / find relationships (edges)

[0117] MERGE (b)-[:HAS_FLOOR]->(f) / / Create or find the relationship between office buildings and floors;

[0118] MERGE (f)-[:CONTAINS]->(s) / / Create or find the relationship where the floor contains the space (office);

[0119] MERGE (w)-[:BELONGS_TO]->(s) / / Create or find the relationship that a wall belongs to a certain space;

[0120] MERGE (d)-[:FILL_ELEMENT]->(w) / / Create or find the relationship in which a door is filled in a wall (i.e., the door is a "filling component" on the wall).

[0121] Neo4j visualization diagram of spatial ownership relationship mapping is as follows Figure 4 shown.

[0122] (3) Topological connection relationship mapping

[0123] Topological connection relationships are used to describe the geometric connections, physical support, and functional nesting between components. They are a key way to reflect the "physical contact" and "dependency" of components during the modeling process. They are essentially different from the "affiliation positioning" in spatial affiliation relationships.

[0124] The pseudo code for topological relationship mapping is:

[0125] (1) Create a node (entity)

[0126] CREATE (w1:Wall {id: "W001", name: "Wall 1", height: 3.0, material: "Concrete"}); / / Created wall node W001;

[0127] CREATE (w2:Wall {id: "W002", name: "Wall 2", height: 3.0, material: "Brick Wall"}); / / Created wall node W002;

[0128] CREATE (d:Door {id: "D001", height: 2.1, material: "wood"}); / / Create a wooden door node;

[0129] CREATE (o:Opening {id: "O001", name: "Opening 1"}); / / Create an opening node;

[0130] (2) Create a relationship (edge)

[0131] CREATE (w1)-[:ADJACENT_TO {ConnectionType: "Edge-to-Edge", Offset:0.1}]->(w2); / / Create walls 1 and 2 adjacent to each other, with the connection type set to "Edge-to-Edge" and an offset of 0.1 meters;

[0132] CREATE (d)-[:FILLS_ELEMENT {JointType: "Sealed", Offset: 0.05}]->(w1); / / Fill door D001 on wall 1 with "sealed" mode and 0.05m offset;

[0133] CREATE (w1)-[:HAS_OPENING {Location: "Center"}]->(o); / / Wall 1 has an opening (such as a window or door) at the "center" position.

[0134] The Neo4j visualization diagram of topological relationship mapping is as follows Figure 5 shown.

[0135] The IFC standard introduces IfcRelConnectsElements (component connection relationships) and its subclass IfcRelFillsElement (component fill relationship), which standardizes the connection, nesting, and support relationships between components in building information models. In the knowledge graph constructed in this paper, topological relationships are defined through relationship types such as "AdjacentTo," "SupportedBy," and "HasOpening," aiming to enhance the topological integrity and semantic expression capabilities of the model.

[0136] After modeling the hierarchical relationships, spatial attribution, and topological connections of the BIM data, the knowledge graph successfully integrated the extracted 2D-to-3D modeling parameters and established relationships between components, forming a complete BIM data framework. Each node is assigned a GlobalID as a unique identifier, combined with attributes such as entity name and description, to further enhance the completeness and queryability of model parameters.

[0137] An example of verifying spatial ownership consistency using Cypher statements is as follows:

[0138] Two nodes of the Neo4j visualization interface:

[0139] Floor node: (:Floor {name: "1F", id: "F001"}) / / represents a floor (Floor), with the attribute "1F" and the number "F001";

[0140] Room node: (:Space {name: "Office", id: "S001"}) / / represents a room (space) named "Office" and numbered "S001";

[0141] There is a relationship between these two nodes: the "1F" floor contains the "Office" space.

[0142] The pseudo code of the Cypher query statement is:

[0143] MATCH (f:Floor)-[:CONTAINS]->(s:Space) / / Find the space / room (s:Space) contained in the floor (f:Floor);

[0144] MATCH (w:Wall)-[:BELONGS_TO]->(s) / / Find the wall (w:Wall) belonging to the space;

[0145] MATCH (d:Door)-[:FILL_ELEMENT]->(w) / / Find the door (d:Door) filled in the wall;

[0146] RETURN f, s, w, d / / Finally return the information of all these nodes.

[0147] Two other nodes of the Neo4j visualization interface:

[0148] Wall node: (:Wall {material: "Concrete", id: "W001", height: 3.0}) / / represents a wall with the material "Concrete", the ID "W001", and the height 3.0 meters;

[0149] Door node: (:Door {material: "Wood", id: "D001", height: 2.1}) / / represents a door, with the material being "Wood", the ID being "D001", and the height being 2.1 meters;

[0150] The relationships between these nodes in the Neo4j visualization interface are also indicated by arrows, reflecting the structure that doors belong to walls, walls belong to spaces, and spaces belong to floors.

[0151] In one embodiment, step 110 includes: combining the knowledge graph structure to construct an automatic conversion process for IFC spatial coordinates, extracting the spatial positions of components from the bottom up and mapping them to global coordinates in sequence to realize the coordinate expression of components in the IFC model; using IfcExtrudedAreaSolid as the core method, automatically modeling walls and panels; and automatically generating from the semantic layer to the model layer by extracting the semantic parameters of components in the knowledge graph and mapping them to IFC fields.

[0152] Specifically, (1) Component coordinate representation mechanism

[0153] To achieve spatial positioning of components in 3D models, the IFC standard introduces the Local Coordinate System (LCS) mechanism. This mechanism uses Object Placement and Representation to collaboratively define the position and shape of components in 3D space. Object Placement can be described using global coordinates, relative coordinates, or grid-based constraint coordinates. A coordinate hierarchy is established using PlacementRelTo (relative position reference datum). RelativePlacement (relative placement) combined with the IfcAxis2Placement3D (3D spatial positioning) entity defines the origin and orientation of the local coordinate system, enabling spatial mapping from higher-level coordinate systems. This structure supports nested spatial positioning, effectively simplifying model parameter maintenance.

[0154] IfcAxis2Placement3D defines the local coordinate system through Location (origin), Axis (Z-axis direction) and RefDirection (X-axis direction). The Y-axis direction can be automatically derived according to the right-hand rule. For two-dimensional coordinates, IfcAxis2Placement2D (a two-dimensional plane in three-dimensional space) is used. Only the X-axis direction needs to be defined, and the Y-axis is automatically derived vertically. This paper further combines the knowledge graph structure to construct an automatic conversion process for IFC space coordinates, extracting the spatial position of components from the bottom up and mapping them to global coordinates in sequence, realizing the coordinate expression of components in the IFC model ( Figure 6 ).

[0155] (2) Methods for expressing component geometric information

[0156] IFC models support a variety of geometric representation methods, including Constructive Solid Graph (CSG), SweptSolid (SweptSolid), and Boundary Representation (Brep). CSG uses Boolean operations to construct complex solids, offering concise modeling but limited representation accuracy. SweptSolid generates 3D volumes by extruding or rotating 2D contours. This method is suitable for regular components but has limited boundary representation. Brep focuses on modeling boundary information, supporting complex topological representation but with high data redundancy. A comparison of the three methods is shown in Table 1.

[0157] To balance modeling efficiency and representational integrity, this method, using IfcExtrudedAreaSolid as its core method, prioritizes the automated modeling of conventional components such as walls and slabs. By extracting component semantic parameters (such as thickness, height, and width) from the knowledge graph and mapping them to IFC fields, it achieves automated generation from the semantic layer to the model layer.

[0158] Table 1 Comparison of common geometric expressions in IFC data models

[0159]

[0160] In one embodiment, the automatic conversion process of IFC spatial coordinates specifically includes: extracting component relative coordinate information from Neo4j to determine whether the global coordinates of the upper-level object exist; if so, directly using the existing coordinate standard; if not, establishing a coordinate hierarchy through PlacementRelTo, defining the origin and direction of the local coordinate system through RelativePlacement combined with IfcAxis2Placement3D entities to achieve spatial mapping from the upper-level coordinate system; using IfcAxis2Placement3D to store coordinates and convert them to IfcLocalPlacement (local position placement).

[0161] In one embodiment, step 112 includes: extracting geometric parameters, semantic information, and spatial positioning-related data of building components from the knowledge graph using the Cypher language; automatically filling the extracted data parameters into the corresponding IFC entities using the IFCOpenShellAPI interface in data mapping and model building; mapping wall data into standard wall classes; treating doors and windows as two independent components, mapping them into IfcDoor (door) and IfcWindow (window), respectively, and nesting them into corresponding wall openings according to IfcRelFillsElement (door and window-opening filling relationship); and using the IFCOpenShell (IFC open source geometry processing library) open source tool to download the IfcOpenShell in the model verification and export stage. The ell.validation (model rule validation engine) module checks data integrity, geometric relationships, and attribute parameters. The resulting IFC file can be viewed in a text editor to determine its standard data structure and can also be input into visualization tools that support the IFC format for 3D visualization. During the automatic construction of slabs, the slab data is mapped to IfcSlab (slab class), and the slab geometry model is constructed using IfcExtrudedAreaSolid (extruded geometry). The slab geometry profile is established using IfcProfileDef (profile definition). Material parameters are standardized using IfcMaterialLayerSetUsage (material layer setting) to ensure the completeness of the slab attribute parameters in the IFC model.

[0162] Specifically, this method is based on the component geometric parameters and attribute information retrieved from the knowledge graph, and completes component semantic analysis, IFC semantic mapping, model generation and result verification in sequence, realizing the automatic construction of the building 3D model. The specific implementation steps are as follows: Figure 7 shown.

[0163] In the data extraction phase, the Cypher language is first used to extract the geometric parameters, semantic information and spatial positioning related data of building components from the knowledge graph. In order to ensure the consistency and integrity of the data, the ID number is introduced as the unique identifier of the component and mapped to the GlobalID field in the IFC standard.

[0164] During data mapping and model building, the IFC OpenShell API was used to automatically populate the extracted data parameters onto the corresponding IFC entities. Wall data was mapped to the standard wall class (IfcWallStandardCase). Furthermore, doors and windows were treated as separate components, mapped as IfcDoor (door) and IfcWindow (window), respectively. These were then nested within the corresponding wall openings using the IfcRelFillsElement (door / window-opening fill relationship). This design resulted in a complete integration of walls, openings, doors, and windows.

[0165] During the model validation and export phase, this method uses the ifcopenshell.validation module under the IFCOpenShell open-source tool to check data integrity, geometric relationships, and attribute parameters. This ensures that the model complies with the IFC standard. The resulting IFC file can be viewed in a text editor to determine its standard data structure and can also be input into a visualization tool that supports the IFC format for 3D visualization.

[0166] During the automated slab construction process, slab data is mapped to the IfcSlab class, and the slab geometry is constructed using the IfcExtrudedAreaSolid object. The slab geometry profile is established using the IfcProfileDef object to ensure accurate slab boundaries. Furthermore, this method standardizes material parameters using the IfcMaterialLayerSetUsage object to ensure the completeness of slab attribute parameters in the IFC model.

[0167] In a validation example, experimental data was derived from architectural design materials for a complex building in a certain region. This data primarily includes multimodal information such as architectural drawings, design specifications, and component lists. These data form the foundation for the transition from 2D design to 3D modeling. Architectural drawings, a typical example of multimodal data in the dataset, are the core of the experimental data, encompassing floor plans, general layouts, and door and window layouts. The design specifications, recorded in text format, detail the building's functional divisions, material selection, construction methods, fire ratings, and construction requirements. Furthermore, component lists and bills of quantities tabulate information such as door and window numbers, construction dimensions, material properties, and component quantities. This data helps supplement component attribute details and supports building performance analysis and model semantic enhancement.

[0168] (1) Preprocessing and parameter extraction of building multimodal data

[0169] 1) Data preprocessing

[0170] To address the issues of chaotic drawing segmentation, inconsistent coordinates, and non-standard layer naming in DXF architectural drawings, this paper addresses these issues through three aspects: drawing registration, coordinate unification, and layer standardization. Drawing segmentation is achieved by identifying drawing frames and floor numbers. Translation and rotation registration of multi-layer drawings is achieved using central axis and column grid feature points, and a unified local coordinate system (LCS) is constructed to ensure spatial consistency. Furthermore, a layer mapping table is constructed based on experimental layer features to achieve semantic consistency for different design standards. To improve the accuracy of PDF image text recognition, pytesseract (a Python library based on the Google Tesseract OCR engine that can extract text from images, supports multiple languages, is easy to use, and has strong compatibility) is used in conjunction with pdf2image (a Python library for converting PDF files to image formats) for OCR on scanned images. Spelling correction and regularization filtering strategies are introduced to optimize recognition results. In terms of table recognition, this paper uses layer features to screen potential table areas, introduces a multi-scale window search mechanism, integrates line segment density and text distribution features, effectively extracts structured component attribute information, and enhances the accuracy of table area positioning and parsing.

[0171] (2) Data parameter extraction and experimental analysis

[0172] 1) Drawing component parameter extraction and identification evaluation

[0173] This method utilizes vector primitive parsing and layer semantic analysis to identify and extract the geometric parameters of components such as wall lines, walls, doors, windows, columns, and roofs in architectural drawings. Evaluation metrics include recall, precision, and F1 value. Results show that wall lines, walls, and columns achieve good recognition, while complex components such as doors, windows, and roofs have lower accuracy, primarily due to nesting and structural complexity. To improve accuracy, this method integrates architectural text and tabular data to semantically supplement and verify component attributes. Statistics on the accuracy of architectural drawing component recognition are shown in Table 2.

[0174] Table 2 Statistics on the accuracy of component recognition in architectural drawings

[0175]

[0176] 2) Architectural text parameter identification and model comparison analysis

[0177] (a) Corpus Source and Data Annotation

[0178] The experimental corpus was selected from design specifications, general construction instructions, and construction practice texts from the complex building project. It covers common descriptions of components such as walls, doors, windows, and floor slabs, including dimensional parameters, material information, and construction methods. A total of 1,550 architectural semantic text fragments were compiled to serve as training material for the named entity recognition model. These fragments were divided into training, validation, and test sets in an 8:1:1 ratio. Text annotation was performed using the open-source tool YEDDA, ​​manually and incorporating architectural terminology. Irregularities and mixed symbol usage were corrected item by item to ensure clear entity boundaries. The annotations use the BIO encoding system and are uniformly converted to the CoNLL format to accommodate model input. Each line corresponds to a character and its label, and segments are separated by blank lines.

[0179] (b) Model calling and parameter setting

[0180] This paper completes the construction of a parametric entity recognition model for architectural semantic text based on the PyTorch framework, using BERT as a pre-trained language model, and combining BiLSTM and CRF for sequence modeling and label decoding. Considering that component descriptions in architectural texts are usually long and have concentrated terms, in order to balance training efficiency and semantic coverage, the maximum input sequence length is set to 128. In terms of hyperparameter settings, this paper conducted multiple rounds of parameter adjustment experiments on the validation set, tried different batch sizes, training rounds, and learning rate combinations, and finally selected the configuration of Batch Size=32 and Epoch=18 as the experimental scheme. This setting is relatively stable in terms of accuracy and recall. The Adam optimizer is used during training, and the initial learning rate is set to 5. Dropout of 0.5 was added before and after the BiLSTM layer to mitigate the slight overfitting observed in the experiment. Regarding the model architecture, the contextual semantic vectors output by BERT are input as sequence features into the BiLSTM network to model the contextual dependencies between parameter items. A CRF model is then used to perform global decoding of the label sequence. This task defines nine named entity categories. The architectural text entity categories and BIO rules are shown in Table 3. Therefore, the output dimension of the CRF layer is set to 9. Finally, after training, the model performance is evaluated on the test set. Table 4 shows the model training hyperparameter settings.

[0181] Table 3 Architectural text entity categories and BIO rules

[0182]

[0183] Table 4 Model training hyperparameter settings

[0184]

[0185] (c) Model comparison and performance evaluation

[0186] Based on a unified model architecture, hyperparameter settings, and data annotation system, this paper designs four sets of comparative experiments to analyze the impact of different strategy combinations on the performance of parametric entity recognition in architectural semantic text.

[0187] The experiment uses precision and recall as the main evaluation indicators. The comparison results of text parameter extraction performance are shown in Table 5.

[0188] Table 5 Comparison of text parameter extraction performance

[0189]

[0190] 3) Table parameter extraction and recognition effect analysis

[0191] This method extracts text content within cells by referencing the coordinates of identified cell vertices. Specifically, text data is extracted within coordinate intervals and stored in a JSON file. During the experiment, we extracted and evaluated relevant attribute information for different table types. Table 6 shows the differences in recognition accuracy and extraction performance for different table types.

[0192] Table 6 Differences in recognition accuracy and extraction effect of different table types during the extraction process

[0193]

[0194] By using the community version of Neo4j as the basis for importing and managing multimodal data, we first configure the graph database. After setting the environment variables, run the neo4j.bat console command in the console to start the graph database. The console will display the default port (such as 7687) and HTTP access address (such as http: / / localhost:7474) containing the Bolt protocol. Then enter this address into the browser to enter the Neo4j Browser, the graphical interface of the graph database, where you can view the data model.

[0195] This paper uses the Neo4j graph database to perform structured relational modeling of multimodal data, visually demonstrating how building information stored in a knowledge graph is mapped to the IFC standard semantic structure. During the experiment, an architectural knowledge graph containing 4,690 entity nodes and 5,956 relationships was automatically generated, covering key components such as walls, floor slabs, doors, windows, and columns, as well as their associated parameters. Figure 8A global overview of the knowledge graph is shown, in which nodes of different colors represent different types of building components, such as green nodes for walls and blue nodes for doors and windows, clearly reflecting the hierarchical relationship and spatial topological structure between components.

[0196] To verify the correctness of the constructed knowledge graph, this method performs query verification on the entities and their relationships in the graph database. The specific steps are to use Cypher language query statements to retrieve key parameters such as the wall's geometric information, material properties, and fire rating, and compare the query results with the original extracted JSON format data to verify the accuracy of the data extraction.

[0197] (4) IFC file generation

[0198] This paper designs a knowledge graph-based IFC model construction method to achieve standardized conversion of semantic information into 3D modeling parameters. This method leverages component entities and their attribute information stored in the Neo4j graph database to establish a data mapping mechanism from the semantic layer to the modeling layer. Furthermore, it integrates the IFC standard to achieve structured conversion of component parameters and model generation.

[0199] like Figure 9 As shown in the figure, the IFC file construction process based on Neo4j and IfcOpenShell includes:

[0200] Step 1: Initialize the IFC project.

[0201] Specifically, use the ifcopenshell library to create an IFC file and initialize the project structure. The pseudo-initialization IFC project code is as follows:

[0202] import ifcopenshell / /

[0203] model = ifcopenshell.file() / / Create IFC file object;

[0204] / / Create IFC project structure

[0205] project=model.create_entity("IfcProject", Name="Architecture Project") / / Create project entity;

[0206] site= model.create_entity("IfcSite", Name="Building Site") / / Create site entity;

[0207] building=model.create_entity("IfcBuilding",Name="Comprehensive Building") / / Create entities such as buildings;

[0208] model.create_entity("IfcRelAggregates",RelatingObject=project,RelatedObjects=[site]) / / Establish a hierarchical relationship between projects and sites;

[0209] model.create_entity("IfcRelAggregates",RelatingObject=site,RelatedObjects=[building]) / / Establish a hierarchical relationship between sites and buildings.

[0210] Step 2: Extract data from Neo4j.

[0211] Specifically, the py2neo library is used to connect to the Neo4j database to query the wall node and its geometric properties.

[0212] The pseudo code for extracting data from Neo4j is as follows:

[0213] from py2neo import Graph / / Extract data from Neo4j

[0214] graph = Graph("bolt: / / localhost:7687", auth=("neo4j", "password")) / / Connect to the local neo4j database;

[0215] walls = graph.run("""

[0216] MATCH (w:Wall)-[:HAS_GEOMETRY]->(g:Geometry) / / Query data

[0217] RETURN w.GlobalId, w.name, g.start, g.length, g.thickness,g.height

[0218] """).data() / / The query result is a list, each element contains the wall's ID, name, starting point, length, thickness, height and other information.

[0219] Step 3: IFC data conversion.

[0220] Specifically, the wall data obtained from Neo4j is traversed and converted into IFC entities.

[0221] The pseudo code for IFC data conversion is as follows:

[0222] for wall in walls:

[0223] ifc_wall=model.create_entity("IfcWallStandardCase", Name=wall['name']) / / Create wall node;

[0224] geom= model.create_entity("IfcExtrudedAreaSolid", Depth=wall['length']) / / Create wall geometry;

[0225] ifc_wall.Representation = geom / / Each wall is converted into an IFC standard wall object and given geometric information.

[0226] Step 4: Export IFC file.

[0227] Specifically, the constructed IFC model is written into a local file.

[0228] The pseudo code for exporting IFC files is:

[0229] model.write("output.ifc") / / Export IFC file;

[0230] print("IFC file has been generated successfully!") / / Print IFC file has been generated successfully.

[0231] (5) Visual verification and integrity analysis of IFC models

[0232] After exporting the IFC file, this article uses the IfcViewer tool to visually verify the generated model to ensure its correctness and integrity. The verification process includes four aspects: model integrity, attribute parameters, geometric parameters, and spatial relationships:

[0233] 1) Model integrity: FC model 3D view such as Figure 10 As shown, Figure 10 The overall structure of the building and various components, such as walls, doors and windows, are clearly displayed, and the components are correctly allocated to the corresponding floors, meeting the IFC aggregation relationship standards.

[0234] 2) Attribute parameters: The material properties, thermal conductivity, and fire rating of walls, doors, and windows were checked to ensure that the attribute information was correctly mapped to the IFC attribute set and complied with the IFC standard.

[0235] 3) Geometric parameters: IFC model section view such as Figure 11 As stated, Figure 11 The dimensional deviation of the displayed components is controlled within ±2mm, the spatial position, size ratio and nested structure are consistent with the design drawings, and the floor boundary is accurately defined.

[0236] 4) Spatial Relationships: Verified the ownership, nesting, and topological connections in the model to ensure that the spatial hierarchy is clear, without misalignment, and in compliance with IFC semantic standards.

[0237] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0238] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0239] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present application, and such modifications and improvements are all within the scope of protection of the present application.

Claims

1. A building automation modeling method based on multimodal information and knowledge graph, characterized in that: The method comprises: Use vector graphics analysis and layer semantic recognition to extract component geometry information from architectural drawings; Combining the BERT-BiLSTM-CRF model and dictionary enhancement strategy, named entity recognition is performed on architectural text to obtain component semantic parameters; Extract component attribute information from building tables through table structure recognition and table logic reconstruction; Utilize JSON files to standardize the storage of component geometry information, component semantic parameters, and component attribute information, and introduce a unique identifier ID mechanism to complete the association between multi-source data; A node-relationship model based on the IFC standard is constructed. Based on the node-relationship model based on the IFC standard, the graph database Neo4j is used to construct a building multimodal information knowledge graph to express the semantic, spatial and attribute relationships of components, thereby realizing structured unified storage management and semantic association modeling of heterogeneous multimodal data between drawings, text and tabular data. The node-relationship model based on the IFC standard includes: hierarchical structure relationship mapping, spatial affiliation relationship mapping, and topological connection relationship mapping. Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, GlobalID is used as the unique identifier of parameter data to achieve accurate docking between the knowledge graph and the IFC model. By calling the IfcOpenShell programming interface, the object geometry, semantics, and spatial positioning parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities. Component semantic analysis, IFC semantic mapping, model generation, and result verification are completed in sequence to achieve the automated construction of the building information model.

2. The method for automated building modeling based on multimodal information and knowledge graph according to claim 1, characterized in that: Using vector graphics primitive analysis and layer semantic recognition, we extract component geometry information from architectural drawings, including: In terms of drawing parameter extraction, a complete extraction process is established from drawing preprocessing, component identification, topological relationship reconstruction to parameter structured output; Through layer standardization and primitive geometric feature analysis, accurate extraction of the geometric position information of basic components is achieved; basic components include walls, doors and windows.

3. The method for automated building modeling based on multimodal information and knowledge graph according to claim 1, characterized in that: Build a node-relationship model based on the IFC standard, use the Neo4j knowledge graph to express the semantic, spatial, and attribute relationships of components, and achieve unified storage and associative modeling of multimodal data, including: Construct a node-relationship model based on the IFC standard; Hierarchical structure relationship mapping is used to standardize the affiliation between projects, buildings, floors, and components. In the knowledge graph, based on the aggregation relationship model, the HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT relationships are defined to establish a hierarchical organizational structure for projects, buildings, floors, and components. Among them, HAS_BUILDING reflects the affiliation between projects and buildings; HAS_FLOOR indicates the affiliation between buildings and floors; CONTAINS_ELEMENT represents the relationship between the components contained in a floor. By using the IFC relationship standard to construct a knowledge graph of multimodal data, building information data can be organized in a multi-level structure, thereby achieving layer-by-layer refinement from projects to components. Spatial ownership relationship mapping is used to clarify the specific location, functional zoning, and spatial dependencies of components in the building environment. Nodes in the knowledge graph are mapped to IfcBuilding, IfcBuildingStorey, and IfcSpace. The "Contains" and "FillsElement" relationship concepts are also introduced to express the spatial ownership and nesting of components, ensuring data integrity and consistency. Topological connection relationship mapping is used to describe the geometric connection, physical support, and functional nesting between components. It is a key method for reflecting the "physical contact" and "dependency" of components during the modeling process. In the knowledge graph, topological relationships are defined through the "AdjacentTo", "SupportedBy", and "HasOpening" relationship types. Each node is assigned a GlobalID as a unique identifier, which is combined with the entity name and description to improve the completeness and queryability of model parameters.

4. The method for automated building modeling based on multimodal information and knowledge graph according to claim 1, characterized in that: The entity mapping mechanism based on the IFC standard includes an IFC component mapping strategy, an IFC parameter mapping strategy, and an identifier mapping strategy; The IFC component mapping strategy is a mapping rule proposed for typical building components based on the information description model of the IFC standard; The IFC parameter mapping strategy includes: geometric parameter mapping rules, physical parameter mapping rules and functional parameter mapping rules; introducing IfcPropertySet and IfcMaterial mechanisms to achieve hierarchical mapping of parameters; The identifier mapping strategy is used to map the unique identification field of the component in the knowledge graph to the globally unique identifier GlobalID attribute of the IFC model. When the model is incrementally updated, the globally unique identifier exists as a unique identifier to maintain data consistency when adding, deleting or modifying component parameters. The globally unique identifier also serves as the basis for matching components from different data sources. At the same time, when querying data, the globally unique identifier is also used as an index field for retrieving models and query parameters to quickly locate components and their associated attributes.

5. The method for automated building modeling based on multimodal information and knowledge graph according to claim 1, characterized in that: The relationship mapping mechanism based on the IFC standard includes a mapping rule for a hierarchical belonging relationship between components, a mapping rule for a spatial belonging relationship between components, and a mapping rule for a topological connection relationship between components.

6. The method for automated building modeling based on multimodal information and knowledge graph according to claim 1, characterized in that: Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, GlobalID is used as the unique identifier of parameter data to achieve accurate docking between the knowledge graph and the IFC model, including: Combined with the knowledge graph structure, an automatic conversion process for IFC spatial coordinates was constructed. The spatial positions of components were extracted from the bottom up and mapped to global coordinates in sequence, thus realizing the coordinate expression of components in the IFC model. Using IfcExtrudedAreaSolid as the core method, walls and plates are automatically modeled. By extracting the semantic parameters of components in the knowledge graph and mapping them to IFC fields, automatic generation from the semantic layer to the model layer is achieved.

7. The method for automated building modeling based on multimodal information and knowledge graph according to claim 6, characterized in that: The automatic conversion process of IFC space coordinates specifically includes: Extract component relative coordinate information from Neo4j to determine whether the global coordinates of the upper-level object exist; If it exists, the existing coordinate standard is used directly; If it does not exist, a coordinate hierarchy is established through PlacementRelTo, and the origin and direction of the local coordinate system are defined through RelativePlacement combined with IfcAxis2Placement3D entities to achieve spatial mapping from the upper coordinate system; Use IfcAxis2Placement3D for coordinate storage and convert it to IfcLocalPlacement.

8. The method for automated building modeling based on multimodal information and knowledge graph according to claim 1, characterized in that: By calling the IfcOpenShell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, realizing the automated construction of the building information model, including: Use Cypher language to extract geometric parameters, semantic information and spatial positioning related data of building components from the knowledge graph; In data mapping and model building, the IFCOpenShell API interface is used to automatically fill the extracted data parameters into the corresponding IFC entities; wall data is mapped to the standard wall class; doors and windows are treated as two independent components, mapped to IfcDoor and IfcWindow respectively, and nested into the corresponding wall opening according to IfcRelFillsElement; During the model validation and export phase, the ifcopenshell.validation module of the IFCOpenShell open-source tool is used to check data integrity, geometric relationships, and attribute parameters. The resulting IFC file can be viewed in a text editor to determine its standard data structure and can also be input into visualization tools that support the IFC format for 3D visualization. During the automatic construction of floor slabs, the floor slab data is mapped to IfcSlab, and the geometric model of the floor slab is constructed using IfcExtrudedAreaSolid. The floor slab geometric profile is established using IfcProfileDef. The material parameters are standardized using IfcMaterialLayerSetUsage to ensure the completeness of the floor slab attribute parameters in the IFC model.

Images