Automatic building modeling method based on multi-modal information and knowledge graph

Abstract

The invention relates to an automatic building modeling method based on multi-modal information and a knowledge graph, and the method comprises three key links: firstly, for heterogeneous data such as drawings, texts and tables, constructing a modeling task-oriented feature extraction and preprocessing process, and guaranteeing the accurate acquisition of key modeling parameters; secondly, a unified semantic framework is constructed based on an IFC standard, construction of a multi-modal information knowledge graph is completed with the help of a graph database Neo4j, and structured management and semantic association modeling of data are achieved; and finally, in combination with the semantic retrieval capability of the atlas and an Ifopen Shell programming interface, completing automatic mapping of parameters and components and model generation, outputting an IFC file conforming to industry specifications, and realizing visual display of a three-dimensional model. According to the method, the extraction integrity of the multi-modal data and the model construction accuracy are effectively improved, and a new thought is provided for automatic construction and intelligent management of the building information model.

Description

Technical Field

[0001] This application relates to the technical field of building modeling, and particularly to a building automated modeling method based on multi-modal information and knowledge graph. Background Art

[0002] The construction industry is one of the main sources of global energy consumption and carbon emissions. The United Nations Environment Report points out that it accounts for about 40% of the total global energy consumption. Among them, the operation and maintenance (O&M) stage has gradually become the stage with the highest energy consumption in the whole life cycle of buildings due to its long cycle and complex affairs. In recent years, numerous studies have focused on building carbon emission control and energy efficiency improvement, and various methods have been proposed especially in aspects such as space planning, facility management, fault detection, and energy optimization. However, the effective application of these methods highly depends on high-quality and structured building data. Currently, the data required for building operation and maintenance mainly includes the as-built documents (such as construction drawings and as-built drawings) formed in the design and construction stages and the operation and maintenance information (such as energy consumption records, equipment maintenance, etc.) accumulated during the building use process. Such data generally has characteristics such as scattered sources, strong heterogeneity, and frequent updates, making manual processing inefficient and difficult to meet the current requirements for intelligent and refined management.

[0003] To address the challenges brought by data fragmentation, Building Information Modeling (BIM) has been widely used in the whole life cycle management. It not only supports the digital expression of building geometric and semantic information but also can serve as an information integration and collaboration platform. Research shows that operation and maintenance management based on BIM can significantly improve management efficiency, reduce energy consumption, and promote the effective transfer of information between different stages and specialties. However, currently, most existing buildings were built a long time ago, without BIM models established and lacking a standardized information system. The information of these buildings is mainly retained in unstructured materials such as two-dimensional drawings, specification texts, and instruction documents, lacking complete three-dimensional data available for modeling, which restricts the in-depth development of their digital reconstruction and intelligent management. Therefore, extracting key modeling parameters from existing multi-modal materials and automatically generating a three-dimensional BIM model has become the core requirement for building digital transformation.

[0004] Current three-dimensional modeling methods face many challenges when dealing with existing building materials. Firstly, there are differences in design standards and drawing specifications in different historical periods, resulting in complex and diverse building forms shown in two-dimensional drawings. As a result, the traditional way relying on pre-set rules shows poor adaptability and generalization when dealing with different expression forms of the same building object. Additionally, for the problems commonly existing in current building three-dimensional modeling, such as parameter extraction relying on a single data source, fragmentation of multi-modal information, lack of a unified management mechanism, and low modeling efficiency. Summary of the Invention

[0005] Based on this, it is necessary to provide a building automation modeling method based on multi-modal information and knowledge graph to address the technical problems commonly existing in the current building three-dimensional modeling process, such as the extraction of parameters relying on a single data source, the fragmentation of multi-modal information, the lack of a unified organization mechanism, and the low modeling efficiency.

[0006] A building automation modeling method based on multi-modal information and knowledge graph, the method includes: Adopt vector graphic element parsing and layer semantic recognition to extract component geometric information from architectural drawings; Combine the BERT-BiLSTM-CRF model and dictionary enhancement strategy to perform named entity recognition on building texts to obtain component semantic parameters.

[0007] Extract component attribute information from architectural tables through table structure recognition and table logic reconstruction.

[0008] Use JSON files to standardize the storage of component geometric information, component semantic parameters, and component attribute information, and introduce a unique identifier ID mechanism to complete the association between multi-source data.

[0009] Construct a node-relationship model based on the IFC standard, and use the Neo4j knowledge graph to express the semantic, spatial, and attribute relationships of components, so as to realize the unified storage and associated modeling of multi-modal data.

[0010] Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, use GlobalID as the unique identifier of parameter data to achieve the accurate docking of the knowledge graph and the IFC model.

[0011] By calling the IfcOpenShell programming interface, automatically map the parameters retrieved from the knowledge graph to generate IFC component entities, and realize the automatic construction of the building information model.

[0012] The above building automation modeling method based on multi-modal information and knowledge graph includes three key steps: First, for heterogeneous data such as drawings, texts, and tables, a feature extraction and preprocessing process for the modeling task is constructed to ensure the accurate acquisition of key modeling parameters; Second, relying on the IFC standard to construct a unified semantic framework, and using the graph database Neo4j to complete the construction of the multi-modal information knowledge graph, realizing the structured management of data and semantic association modeling; Finally, combining the semantic retrieval ability of the graph and the IfcOpenShell programming interface, complete the automatic mapping of parameters and components and model generation, output IFC files that meet industry specifications and realize the visual display of 3D models. This method effectively improves the extraction integrity of multi-modal data and the model construction accuracy, providing new ideas for the automatic construction and intelligent management of building information models. Brief Description of the Drawings

[0013] Figure 1 It is a schematic flowchart of the building automation modeling method based on multi-modal information and knowledge graph in an embodiment; Figure 2 It is a flowchart framework of the building automation modeling method based on multi-modal information and knowledge graph in an embodiment; Figure 3 It is a schematic diagram of the Neo4j visual display of the hierarchical relationship mapping in another embodiment; Figure 4 It is a schematic diagram of the Neo4j visual display of the spatial attribution relationship mapping in another embodiment; Figure 5 It is a schematic diagram of the topological relationship mapping in another embodiment; Figure 6 It is a flowchart of the coordinate conversion from the knowledge graph to the IFC hierarchy in another embodiment; Figure 7 It is a complete flowchart from the knowledge graph data retrieval to the IFC model generation in another embodiment; Figure 8 It is an overview diagram of the knowledge graph in another embodiment; Figure 9 It is a flowchart of the IFC file construction based on Neo4j and IfcOpenShell in another embodiment; Figure 10 It is a schematic diagram of the 3D view analysis of the IFC model in another embodiment; Figure 11 It is a schematic diagram of the sectional view analysis of the IFC model in another embodiment. Detailed Description of the Invention

[0014] To make the objectives, technical solutions, and advantages of this application more clear and understandable, the following further details this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application.

[0015] In one embodiment, as Figure 1 , Figure 2 shown, a building automation modeling method based on multi-modal information and knowledge graph is provided. The method includes the following steps: Step 100: Extract component geometric information from architectural drawings by using vector graphic element parsing and layer semantic recognition.

[0016] Specifically, in drawing processing, map sheet division and alignment are completed through drawing frame recognition and feature point registration, and invalid graphic elements are cleared and layer naming is standardized to achieve geometric parameter extraction under the joint parsing of vector graphic elements and layer semantics.

[0017] Architectural drawings can be, but are not limited to, two-dimensional CAD drawings.

[0018] Step 102: Combine the BERT-BiLSTM-CRF model and dictionary enhancement strategy to perform named entity recognition on architectural text to obtain component semantic parameters.

[0019] Specifically, in text processing, component attributes are extracted by combining OCR, layout structure analysis, and semantic correction mechanisms, and a fusion strategy of the BERT-BiLSTM-CRF model and dictionary rules is introduced to improve the accuracy of entity recognition.

[0020] Step 104: Extract component attribute information from architectural tables through table structure recognition and table logic reconstruction.

[0021] Specifically, in table processing, a method based on multi-scale windows and structure reconstruction is designed to effectively parse the table structure and logical relationship and achieve the complete extraction of attribute parameters.

[0022] Step 106: Standardize and store the component geometric information, component semantic parameters, and component attribute information using a JSON file, and introduce a unique identifier ID mechanism to complete the association between multi-source data.

[0023] Step 108: Construct a node-relationship model based on the IFC standard, and use the Neo4j knowledge graph to express the semantic, spatial, and attribute relationships of components to achieve unified storage and associated modeling of multi-modal data.

[0024] Specifically, to solve the problem of heterogeneity among drawings, texts, and tabular data, a node-relationship model based on the IFC standard is constructed, and the Neo4j knowledge graph is used to express the semantic, spatial, and attribute relationships of components, realizing the unified storage and associated modeling of multimodal data.

[0025] Step 110: Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, using the GlobalID as the unique identifier of parameter data, the accurate docking between the knowledge graph and the IFC model is achieved.

[0026] Step 112: By calling the IfcOpenShell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, realizing the automated construction of the building information model.

[0027] Specifically, by calling the Ifcopenshell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, realizing the automated construction of the building information model.

[0028] In the above method for automated building modeling based on multimodal information and knowledge graph, the method includes three key links: First, for heterogeneous data such as drawings, texts, and tables, a feature extraction and preprocessing process for modeling tasks is constructed to ensure the accurate acquisition of key modeling parameters; Second, relying on the IFC standard to build a unified semantic framework, and with the help of the graph database Neo4j, the construction of the multimodal information knowledge graph is completed, realizing the structured management of data and semantic associated modeling; Finally, combining the semantic retrieval ability of the graph and the IfcOpenShell programming interface, the automatic mapping of parameters and components and model generation are completed, outputting IFC files that meet industry specifications and realizing the visual display of 3D models. This method effectively improves the extraction integrity of multimodal data and the model construction accuracy, providing new ideas for the automated construction and intelligent management of building information models.

[0029] In one of the embodiments, step 100 includes: In terms of drawing parameter extraction, a complete extraction process from drawing preprocessing, component recognition, topological relationship reconstruction to parameter structured output is established; Through layer standardization and primitive geometric feature analysis, the accurate extraction of the geometric position information of basic components is realized; The basic components include walls and doors and windows.

[0030] Specifically, the integrity and accuracy of building information extraction are the keys to achieving precise 3D building modeling. Building data mainly comes from drawings, texts, and tables, which respectively carry geometric information, attribute parameters, and component relationships, providing data support for building design, construction, and management. However, existing research has mostly focused on the extraction of geometric information from drawings, neglecting the attribute information and component associations contained in texts and tables, resulting in incomplete model semantics and missing component parameters, which affect the construction of a precise building information model.

[0031] Building data is diverse and complementary. Different types of data have their own focuses in information expression and jointly constitute a complete building information system. Drawing data mainly records the geometric shapes, dimensional parameters, and spatial relationships of components and is the core source of model spatial information; text data focuses on material properties, construction requirements, and functional attributes, which can supplement the non-geometric information missing in drawings; table data stores door and window information, material structures, and construction methods in a standardized form, with strong structural characteristics, further supplementing the physical attributes and parameters of components. The three complement each other and jointly support the information integrity and accuracy of the building model.

[0032] In terms of drawing parameter extraction, a complete extraction process from drawing preprocessing, component recognition, topological relationship reconstruction to parameter structured output has been established. Through layer standardization and analysis of primitive geometric features, precise extraction of the geometric position information of basic components such as walls and doors and windows has been achieved; basic components include walls and doors and windows. In building text parsing, a parameter extraction model PENet that integrates multiple strategies has been proposed, focusing on solving problems such as diverse parameter expressions, confusion of numerical units, and character misrecognition. This model integrates modules such as term normalization, semantic encoding based on a pre-trained language model (BERT), context modeling of bidirectional long short-term memory networks (BiLSTM), and label sequence optimization of conditional random fields (CRF), effectively improving the recognition accuracy and boundary integrity of component parameter entities. In terms of table parameter extraction, a three-stage process of "region screening - structure reconstruction - content recognition" has been constructed, and effective extraction of different types of table parameters has been achieved using layer features and multi-scale search strategies. In addition, this paper uses JSON files to standardize the storage of parameter information from different sources and introduces a unique identifier ID mechanism to complete the association between multi-source data.

[0033] The pseudo-code for JSON file storage is as follows: geometry.json / / Main content: Geometric information; { "id": "W01": / / Define the wall number; "geometry": { / / Define geometric dimensions; "start": [-7335.51, -10447.55, 0.0], / / Starting point coordinates (x, y, z); "length": 4500.0, / / Wall length; "thickness": 240.0, / / Wall thickness; "height": 3799.99, / / Wall height; "direction": [l, 0, 0], / / Wall orientation (vector); }; "polygon": [ / / Define the polygon vertex coordinates of the wall in space (usually used for spatial positioning); [-7335.51, -10447.55, 0.0], [-2835.51, -10447.55.0.0], [-2835.51.-10207.55.0.0], [-7335.51.-10207.55.0.0]]}; table.json / / Main content: Physical property parameters; { "id": "W01", / / Wall number; "thermal_transmittance": 0.7941, / / Heat transfer coefficient; "weight": 12000.0, / / Weight; "U_value": 1.2, / / U-value (thermal performance index); }; text.json / / Main content: Description information (mostly text attributes); { "id": "W01": / / Wall number "name": “Wall 1” / / Wall name (e.g., “Wall 1”) "category": "Standard Wal!", / / Category is standard wall; "floor": 1F, / / Floor where it is located (e.g., “1F”) "material": “Concrete”, / / Material type is concrete "function": Load-bearing, / Wall function (e.g., load-bearing); "fire_resistance": 2h, / / Define the fire resistance limit (e.g., “2 hours”); }。

[0034] In one of the embodiments, step 108 includes: constructing a node-relationship model based on the IFC standard. The node-relationship model based on the IFC standard includes: hierarchical structure relationship mapping, spatial attribution relationship mapping, and topological connection relationship mapping. The hierarchical structure relationship mapping is used to standardize the subordination relationships among projects, buildings, floors, and components. In the knowledge graph, according to the aggregation relationship model, the relationships of HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT are defined to build the hierarchical organizational structure of projects, buildings, floors, and components. Among them, HAS_BUILDING reflects the attribution association between the project and the building; HAS_FLOOR indicates the subordination relationship between the building and the floor; CONTAINS_ELEMENT represents the relationship between the components contained in the floor. By using the IFC relationship standard to construct the knowledge graph of multimodal data, the building information data can be orderly organized in a multi-level structure, thereby realizing the gradual refinement from the project to the components. The spatial attribution relationship mapping is used to clarify the specific location, functional partition, and spatial dependence of the components in the building environment. The nodes in the knowledge graph are mapped to IfcBuilding (building layer), IfcBuildingStorey (floor), and IfcSpace (space layer), and the relationship concepts of "Contains" and "FillsElement" are also introduced to express the spatial subordination and nesting of components, ensuring the integrity and consistency of the data. The topological connection relationship mapping is used to describe the geometric connection, physical support, and functional nesting between components, which is the key way to reflect the "physical contact" and "attachment relationship" of components in the modeling process. In the knowledge graph, the topological relationship is defined by relationship types such as "AdjacentTo", "SupportedBy", and "HasOpening". Each node is assigned a GlobalID as the unique identifier, and combined with the entity name and description, the integrity and queryability of the model parameters are improved.

[0035] Specifically, in the building information model, the spatial relationships of components can be divided into three categories from the perspective of the IFC relationship model: ① the subordination relationship based on the hierarchical structure; ② the spatial attribution relationship; ③ the topological connection relationship. Based on these three relationship models, this paper proposes a multi-dimensional mapping relationship model, which covers three core modules: node mapping, relationship mapping, and attribute mapping. To ensure the orderly organization and efficient query of the integrated multimodal data, this paper uses a graph database to perform relationship modeling on the extracted multimodal data, realizing the semantic association of the model.

[0036] (1) Hierarchical structure relationship mapping With the help of the IfcRelAggregates (the aggregation relationship model in IFC, used to describe the spatial subordination relationship), the IFC standard standardizes the subordination relationships among projects, buildings, floors, and components. This model uses the "whole - part" relationship to ensure the integrity and traceability of building data organization. In the knowledge graph, according to the aggregation relationship model, this paper defines relationships such as HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT to build the hierarchical organizational structure of projects, buildings, floors, and components. Among them, HAS_BUILDING reflects the attribution relationship between the project and the building; HAS_FLOOR indicates the subordination relationship between the building and the floor; CONTAINS_ELEMENT represents the relationship between the components contained in the floor. By using the IFC relationship standard to construct the knowledge graph of multimodal data, building information data can be orderly organized in a multi - level structure, thus realizing the layer - by - layer refinement from the project to the components.

[0037] (2)Spatial attribution relationship mapping In the IFC standard, the relationship model of IfcRelContainedInSpatialStructure (component - spatial subordination relationship) is introduced to describe the subordinate relationship between components and the spatial units they are located in. This kind of association is mainly used to clarify the specific location, functional zoning, and spatial dependence of components in the building environment.

[0038] In the model constructed in this embodiment, the nodes in the knowledge graph are mapped to spatial units such as IfcBuilding (building layer), IfcBuildingStorey (floor), and IfcSpace (space layer). To further express the spatial subordination and nesting relationships between components, relationship concepts such as "Contains" and "FillsElement" are introduced in the graph to express the spatial subordination and nesting status between components, ensuring the integrity and consistency of the data.

[0039] (3)Topological connection relationship mapping The topological connection relationship is used to describe the geometric connection, physical support, and functional nesting between components, which is a key way to reflect the "physical contact" and "attachment relationship" of components in the modeling process, and is essentially different from the "subordinate positioning" in the spatial attribution relationship.

[0040] In the IFC standard, by introducing IfcRelConnectsElements (element connection relationship) and its subclass IfcRelFillsElement (filled element relationship), the connection, nesting, and support relationships between elements in the building information model are defined in a standardized manner. In the knowledge graph constructed in this paper, topological relationships are defined through relationship types such as "AdjacentTo", "SupportedBy", and "HasOpening", aiming to enhance the topological integrity and semantic expression ability of the model.

[0041] After completing the modeling of the hierarchical relationship, spatial attribution relationship, and topological connection relationship of the building information model data, the knowledge graph has successfully integrated the extracted two-dimensional to three-dimensional modeling parameters, established the association relationship between each component, and formed a complete building information model data framework. Each node is assigned a GlobalID as the unique identifier, and combined with attributes such as entity name and description, it further improves the integrity and queryability of the model parameters.

[0042] In one embodiment, the entity mapping mechanism based on the IFC standard includes an IFC component mapping strategy, an IFC parameter mapping strategy, and an identifier mapping strategy; the IFC component mapping strategy is a mapping rule proposed for typical building components according to the information description mode of the IFC standard; the IFC parameter mapping strategy includes: geometric parameter mapping rules, physical parameter mapping rules, and functional parameter mapping rules; by introducing the IfcPropertySet (property set) and IfcMaterial (material) mechanisms, hierarchical mapping of parameters is achieved; the identifier mapping strategy is used to map the unique identifier field of components in the knowledge graph to the GlobalID attribute of the global unique identifier in the IFC model. When the model is incrementally updated, the global unique identifier exists as the unique identifier. When adding, deleting, or modifying component parameters, data consistency is maintained. The global unique identifier is also used as the basis for matching components from different data sources; at the same time, when querying data, the global unique identifier is also used as the index field for retrieving the model and query parameters for quick positioning of components and their associated attributes.

[0043] In one embodiment, the relationship mapping mechanism based on the IFC standard includes hierarchical attribution relationship mapping rules between components, spatial attribution relationship mapping rules between components, and topological connection relationship mapping rules between components.

[0044] Specifically, in the Building Information Model, the spatial relationships of components can be classified into three categories from the perspective of the IFC relationship model: ① the subordination relationship based on the hierarchical structure; ② the spatial attribution relationship; ③ the topological connection relationship. Based on these three relationship models, this paper proposes a multi-dimensional mapping relationship model, which covers three core modules: node mapping, relationship mapping, and attribute mapping. To ensure the orderly organization and efficient query of the integrated multi-modal data, this paper uses a graph database to perform relationship modeling on the extracted multi-modal data, realizing the semantic association of the model.

[0045] (1) Hierarchical Structure Relationship Mapping The IFC standard uses the aggregation relationship model of IfcRelAggregates to standardize the subordination relationships among projects, buildings, floors, and components. This model uses the "whole-part" relationship to ensure the integrity and traceability of the organization of building data. In the knowledge graph, this paper defines relationships such as HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT according to the aggregation relationship model to build the hierarchical organizational structure of projects, buildings, floors, and components. Among them, HAS_BUILDING reflects the attribution relationship between the project and the building; HAS_FLOOR indicates the subordination relationship between the building and the floor; CONTAINS_ELEMENT represents the relationship between the components contained in the floor. By using the IFC relationship standard to construct the knowledge graph of multi-modal data, the building information data can be orderly organized in a multi-level structure, thus realizing the gradual refinement from the project to the components.

[0046] The pseudo-code for hierarchical relationship mapping is as follows: Cypher is the query language of the Neo4j graph database, similar to SQL in relational databases. Here it is divided into two parts: (1) Create the building hierarchical structure (nodes) CREATE (:Project {id: "P001", name: "Industrial Building Project"}); / / Create a project node; CREATE (:Building {id: "B001", name: "Office Building"}); / / Create a building; CREATE (:Floor {id: "F001", name: "1F"}); / / Create a floor node; CREATE (:Wall {id: "W001", height: 3.0, material: "Concrete"}); / / Create a wall node; (2) Establish the hierarchical relationship (edges) MATCH (p:Project {id: "P001"}), (b:Building {id: "B001"}) CREATE (p)-[:CONTAINS]->(b); / / Query to create the relationship between nodes when the project node P001 contains the building node B001 (building contains floors); MATCH (b:Building {id: "B001"}), (f:Floor {id: "F001"}) CREATE (b)-[:CONTAINS]->(f); / / Query to create the relationship between nodes when the building node B001 contains the floor node F001 (building contains floors); MATCH (f:Floor {id: "F001"}), (w:Wall {id: "W001"}) CREATE (f)-[:HAS_COMPONENT]->(w); / / Query to create the relationship between nodes when the floor node B001 contains the wall node F001 (floor has wall components).

[0047] The Neo4j visualization diagram of the hierarchical relationship mapping is as Figure 3 shown.

[0048] (2)Spatial attribution relationship mapping In the IFC standard, the relationship model of IfcRelContainedInSpatialStructure (component - spatial attribution relationship) is introduced to describe the subordinate relationship between components and the spatial units they are located in. This association is mainly used to clarify the specific location, functional zoning, and spatial dependence of components in the building environment.

[0049] In the model constructed in this embodiment, the nodes in the knowledge graph are mapped to spatial units such as IfcBuilding (building layer), IfcBuildingStorey (floor), and IfcSpace (space layer). To further express the spatial attribution and nesting relationships between components, relationship concepts such as "Contains", "Belongs_to", and "FillsElement" are introduced in the graph to express the spatial attribution and nesting status between components, ensuring the integrity and consistency of the data.

[0050] The pseudocode for the spatial attribution relationship mapping is: Cypher is the query language of Neo4j, and the MERGE statement is used to "find or create" nodes and relationships to ensure no duplicates; (1)Create / Find Nodes (Entities) MERGE (b:Building {id: "B001", name: "Office Building"}) / / Create or find the office building node; MERGE (f:Floor {id: "F001", name: "1F"}) / / Create or find the 1F floor node; MERGE (s:Space {id: "S001", name: "Office"}) / / Create or find the office space node; MERGE (w:Wall {id: "W001", height: 3.0, material: "Concrete"}) / / Create or find the "concrete wall node; MERGE (d:Door {id: "D001", height: 2.1, material: "Wood"}) / / Create or find the wooden door node; (2)Create / Find Relationships (Edges) MERGE (b)-[:HAS_FLOOR]->(f) / / Create or find the relationship that the office building has a floor; MERGE (f)-[:CONTAINS]->(s) / / Create or find the relationship that the floor contains a space (office); MERGE (w)-[:BELONGS_TO]->(s) / / Create or find the relationship that the wall belongs to a certain space; MERGE (d)-[:FILL_ELEMENT]->(w) / / Create or find the relationship that the door fills in a certain wall (i.e., the door is a "filling component" on the wall).

[0051] The Neo4j visualization diagram of the space attribution relationship mapping is as Figure 4 shown.

[0052] (3)Topological Connection Relationship Mapping Topological connection relationships are used to describe the geometric connections, physical supports, and functional nestings between components. They are the key ways to reflect the "physical contact" and "attachment relationships" between components during the modeling process, and are essentially different from the "subordination positioning" in the space attribution relationship.

[0053] The pseudocode for topological relationship mapping is as follows: (1)Create Nodes (Entities) CREATE (w1:Wall {id: "W001", name: "Wall 1", height: 3.0, material: "Concrete"}); / / Created the wall node W001; CREATE (w2:Wall {id: "W002", name: "Wall 2", height: 3.0, material: "Brick wall"}); / / Created the wall node W002; CREATE (d:Door {id: "D001", height: 2.1, material: "Wooden"}); / / Created a wooden door node; CREATE (o:Opening {id: "O001", name: "Opening 1"}); / / Created an opening node; (2)Create relationships (edges) CREATE (w1)-[:ADJACENT_TO {ConnectionType: "Edge-to-Edge", Offset:0.1}]->(w2); / / Created wall 1 adjacent to wall 2, with connection type "Edge-to-Edge" and offset of 0.1 meters; CREATE (d)-[:FILLS_ELEMENT {JointType: "Sealed", Offset: 0.05}]->(w1); / / The door D001 fills wall 1 in a "sealed" manner with an offset of 0.05 meters; CREATE (w1)-[:HAS_OPENING {Location: "Center"}]->(o); / / Wall 1 has an opening (such as a window or door opening) at the "center" position.

[0054] The Neo4j visualization diagram of the topological relationship mapping is as Figure 5 shown.

[0055] In the IFC standard, by introducing IfcRelConnectsElements (element connection relationship) and its subclass IfcRelFillsElement (filled element relationship), the connection, nesting, and support relationships between components in the building information model are standardized. In the knowledge graph constructed in this paper, the topological relationship is defined through relationship types such as "AdjacentTo", "SupportedBy", and "HasOpening", aiming to improve the topological integrity and semantic expression ability of the model.

[0056] After completing the modeling of the hierarchical relationship, spatial attribution relationship, and topological connection relationship of building information model data, the knowledge graph has successfully integrated the extracted 2D-to-3D modeling parameters, established the association relationship between each component, and formed a complete building information model data framework. Each node is assigned a GlobalID as the unique identifier, and combined with attributes such as entity name and description, further improving the integrity and queryability of the model parameters.

[0057] The verification example of querying the spatial attribution consistency through Cypher statements is as follows: Two nodes in the Neo4j visualization interface: Floor node: (:Floor {name: "1F", id: "F001"}) / / Represents a floor, with the attribute "1F" and the number "F001"; Room node: (:Space {name: "Office", id: "S001"}) / / Represents a room (space), with the name "Office" and the number "S001"; There is a relationship between these two nodes: It is clear that the "1F" floor contains the "Office" space.

[0058] The pseudo-code of the Cypher query statement is: MATCH (f:Floor)-[:CONTAINS]->(s:Space) / / Find the spaces / rooms (s:Space) contained in the floor (f:Floor); MATCH (w:Wall)-[:BELONGS_TO]->(s) / / Find the walls (w:Wall) belonging to this space; MATCH (d:Door)-[:FILL_ELEMENT]->(w) / / Find the doors (d:Door) filled on this wall; RETURN f, s, w, d / / Finally, return the information of all these nodes.

[0059] Another two nodes in the Neo4j visualization interface: Wall node: (:Wall {material: "Concrete", id: "W001", height: 3.0}) / / Represents a wall, with the material "Concrete", the number "W001", and the height of 3.0 meters; Door node: (:Door {material: "wood", id: "D001", height: 2.1}) / / Represents a door, with material "wood", number "D001", and height 2.1 meters; In the Neo4j visualization interface, the relationships between these nodes are also identified by arrows, reflecting the structure where doors belong to walls, walls belong to spaces, and spaces belong to floors.

[0060] In one of the embodiments, step 110 includes: Combining the knowledge graph structure, constructing an automatic conversion process for IFC space coordinates, extracting the spatial positions of components from bottom to top and sequentially mapping them to global coordinates to achieve the coordinate representation of components in the IFC model; Using IfcExtrudedAreaSolid as the core method to automatically model walls and slabs; By extracting the semantic parameters of components in the knowledge graph and mapping them to IFC fields, realizing the automatic generation from the semantic layer to the model layer.

[0061] Specifically, (1) Component coordinate representation mechanism To achieve the spatial positioning of components in a 3D model, the IFC standard introduces a local coordinate system (LocalCoordinate System, LCS) mechanism, which collaboratively defines the position and shape of components in 3D space through ObjectPlacement and Representation. Among them, ObjectPlacement can be described using global coordinates, relative coordinates, or axis network-based constraint coordinates, and a coordinate hierarchy is established through PlacementRelTo (relative position reference benchmark), and the origin and direction of the local coordinate system are defined by combining RelativePlacement (relative placement position) with IfcAxis2Placement3D (3D space positioning) entity to achieve the spatial mapping from the upper coordinate system. This structure supports nested spatial positioning and effectively simplifies model parameter maintenance.

[0062] IfcAxis2Placement3D defines the local coordinate system through the three elements of Location (origin), Axis (Z-axis direction), and RefDirection (X-axis direction), and the Y-axis direction can be automatically deduced according to the right-hand rule. Two-dimensional coordinates use IfcAxis2Placement2D (a 2D plane in 3D space), and only the X-axis direction needs to be defined, and the Y-axis is automatically deduced perpendicular. This article further combines the knowledge graph structure to construct an automatic conversion process for IFC space coordinates, extracts the spatial positions of components from bottom to top and sequentially maps them to global coordinates to achieve the coordinate representation of components in the IFC model ( Figure 6 )

[0063] (2) Geometric Information Expression Method of Components The IFC model supports multiple geometric expression methods, including Constructive Solid Geometry (CSG), Swept Solid, and Boundary Representation (Brep). CSG constructs complex entities through Boolean operations, with concise modeling but limited expression accuracy; Swept Solid generates three-dimensional bodies by stretching or rotating two-dimensional profiles, suitable for regular components but with limited boundary expression; Brep focuses on boundary information modeling, supporting complex topology expression but with high data redundancy. The comparison among the three is shown in Table 1.

[0064] To balance modeling efficiency and expression integrity, this method takes IfcExtrudedAreaSolid (extruded solid definition) as the core method, which is preferentially used for automatic modeling of conventional components such as walls and slabs. By extracting semantic parameters (such as thickness, height, width, etc.) of components from the knowledge graph and mapping them to IFC fields, automatic generation from the semantic layer to the model layer is achieved.

[0065] Table 1 Comparison of Common Geometric Expression Methods in the IFC Data Model

[0066] In one of the embodiments, the automatic conversion process of IFC spatial coordinates specifically includes: extracting the relative coordinate information of components from Neo4j and determining whether there is a global coordinate of the upper-level object; if it exists, directly use the existing coordinate standard; if it does not exist, establish a coordinate hierarchy through PlacementRelTo, and combine RelativePlacement with the IfcAxis2Placement3D entity to define the origin and direction of the local coordinate system, realizing the spatial mapping from the upper-level coordinate system; use IfcAxis2Placement3D for coordinate storage and convert it to IfcLocalPlacement (local position placement).

[0067] In one of the embodiments, step 112 includes: extracting geometric parameters, semantic information, and spatial positioning-related data of building components from the knowledge graph using the Cypher language; in data mapping and model construction, adopting the IFCOpenShell API interface to automatically fill the extracted data parameters into the corresponding IFC entities; mapping wall data to the standard wall class; regarding doors and windows as two independent components, mapping doors and windows to IfcDoor (doors) and IfcWindow (windows) respectively, and nesting them into the corresponding wall openings according to IfcRelFillsElement (door / window-opening filling relationship); during the model verification and export phase, using the ifcopenshell.validation (model rule verification engine) module under the IFCOpenShell (IFC open-source geometry processing library) open-source tool to check data integrity, geometric relationships, and attribute parameters. The finally generated IFC file can view its standard data structure in a text editor and can also be input into a visualization tool that supports the IFC format for three-dimensional visualization display; during the process of automatically constructing the floor slab, the data of the floor slab is mapped to IfcSlab (floor slab class), and the geometric model of the floor slab is constructed using IfcExtrudedAreaSolid (extruded geometry); the geometric profile of the floor slab is established by IfcProfileDef (profile definition); the material parameters are standardized using IfcMaterialLayerSetUsage (material layer setting) to ensure the completeness of the attribute parameters of the floor slab in the IFC model.

[0068] Specifically, based on the component geometric parameters and attribute information retrieved from the knowledge graph, this method sequentially completes component semantic parsing, IFC semantic mapping, model generation, and result verification, realizing the automatic construction of the building three-dimensional model. The specific implementation steps are as Figure 7 shown.

[0069] In the data extraction phase, first, geometric parameters, semantic information, and spatial positioning-related data of building components are extracted from the knowledge graph using the Cypher language. To ensure data consistency and integrity, an ID number is introduced as the unique identifier of the component and mapped to the GlobalID field in the IFC standard. [[ID=...]] [[ID=...]]

[0070] During data mapping and model building, the IFC OpenShell API was used to automatically populate the extracted data parameters onto the corresponding IFC entities. Wall data was mapped to the standard wall class (IfcWallStandardCase). Furthermore, doors and windows were treated as separate components, mapped as IfcDoor (door) and IfcWindow (window), respectively. These were then nested within the corresponding wall openings using the IfcRelFillsElement (door / window-opening fill relationship). This design resulted in a complete integration of walls, openings, doors, and windows.

[0071] During the model validation and export phase, this method uses the ifcopenshell.validation module under the IFCOpenShell open-source tool to check data integrity, geometric relationships, and attribute parameters. This ensures that the model complies with the IFC standard. The resulting IFC file can be viewed in a text editor to determine its standard data structure and can also be input into a visualization tool that supports the IFC format for 3D visualization.

[0072] During the automated slab construction process, slab data is mapped to the IfcSlab class, and the slab geometry is constructed using the IfcExtrudedAreaSolid object. The slab geometry profile is established using the IfcProfileDef object to ensure accurate slab boundaries. Furthermore, this method standardizes material parameters using the IfcMaterialLayerSetUsage object to ensure the completeness of slab attribute parameters in the IFC model.

[0073] In a validation example, experimental data was derived from architectural design materials for a complex building in a certain region. This data primarily includes multimodal information such as architectural drawings, design specifications, and component lists. These data form the foundation for the transition from 2D design to 3D modeling. Architectural drawings, a typical example of multimodal data in the dataset, are the core of the experimental data, encompassing floor plans, general layouts, and door and window layouts. The design specifications, recorded in text format, detail the building's functional divisions, material selection, construction methods, fire ratings, and construction requirements. Furthermore, component lists and bills of quantities tabulate information such as door and window numbers, construction dimensions, material properties, and component quantities. This data helps supplement component attribute details and supports building performance analysis and model semantic enhancement.

[0074] (1) Preprocessing and parameter extraction of building multimodal data 1) Data preprocessing Aiming at the problems of chaotic drawing sheet segmentation, inconsistent coordinates, and non-standard layer naming in DXF format architectural drawings, this paper conducts standardized processing from three aspects: drawing sheet registration, coordinate unification, and layer standardization. The drawing sheets are segmented by identifying the drawing frame and floor number; the translation and rotation registration of multi-layer drawings are completed using the central axis and grid feature points, and a unified local coordinate system (LCS) is constructed to ensure spatial consistency. At the same time, a layer mapping table is constructed based on the experimental layer features to achieve the semantic unification of layers under different design standards. To improve the accuracy of PDF image text recognition, pytesseract (a Python library based on the Google Tesseract-OCR engine that can extract text from images, supports multiple languages, is easy to use and has strong compatibility) combined with pdf2image (a Python library for converting PDF files to image formats) is used for OCR recognition of scanned images, and spelling correction and regular filtering strategies are introduced to optimize the recognition results. In terms of table recognition, this paper uses layer features to screen potential table areas, and introduces a multi-scale window search mechanism, which combines line segment density and text distribution features to effectively extract the attribute information of structured components and enhance the positioning and parsing accuracy of table areas.

[0075] (2) Parameter Extraction and Experimental Analysis of Data 1) Parameter Extraction and Recognition Evaluation of Drawing Components This method uses vector graphic element parsing and layer semantic analysis methods to identify and extract the geometric parameters of components such as wall lines, walls, doors, windows, columns, and roofs in architectural drawings. The evaluation indicators include recall rate, precision rate, and F1 value. The results show that the recognition effects of wall lines, walls, and columns are better, while the recognition accuracy of complex components such as doors, windows, and roofs is relatively low, mainly affected by the nesting form and the complexity of the structural form. To improve the accuracy, this paper combines architectural text and table data to semantically supplement and verify the component attributes. The statistical results of the recognition accuracy of architectural drawing components are shown in Table 2.

[0076] Table 2 Statistical Results of the Recognition Accuracy of Architectural Drawing Components

[0077] 2) Parameter Recognition of Architectural Text and Model Comparative Analysis (a) Corpus Source and Data Annotation The experimental corpus was selected from the design specifications, general construction specifications, and construction method texts in the comprehensive building project, covering common descriptive statements such as the dimensional parameters, material information, and construction methods of components such as walls, doors and windows, and floors. Finally, a total of 1550 architectural semantic text fragments were sorted out as the training corpus for the named entity recognition model, and were divided into a training set, a validation set, and a test set according to the ratio of 8:1:1. Text annotation was carried out using the open-source tool YEDDA, and all were manually completed by combining architectural professional terms. During the process, non-standard expressions, symbol mixing, etc. were revised item by item to ensure clear entity boundaries. The BIO coding system was used for annotation, and it was uniformly converted into the CoNLL format to adapt to the model input. Each line corresponded to a character and its label, and the sentence segments were separated by blank lines.

[0078] (b) Model invocation and parameter setting In this paper, a parameter entity recognition model for architectural semantic text was constructed based on the PyTorch framework. BERT was used as the pre-trained language model, and BiLSTM and CRF were combined for sequence modeling and label decoding. Considering that the component descriptions in architectural texts are usually long sentences and the terms are concentrated, in order to balance the training efficiency and semantic coverage ability, the maximum input sequence length was set to 128. In terms of hyperparameter settings, this paper conducted multiple rounds of parameter tuning experiments on the validation set, tried different combinations of batch sizes, training epochs, and learning rates, and finally selected the configuration of Batch Size = 32 and Epoch = 18 as the experimental plan, and this setting showed relatively stable performance in terms of accuracy and recall. The Adam optimizer was used during the training process, and the initial learning rate was set to 5 , and a random inactivation (Dropout) of 0.5 was added before and after the BiLSTM layer to alleviate the slight overfitting phenomenon observed in the experiment. In terms of the model architecture, the context semantic vectors output by BERT were used as sequence features and input into the BiLSTM network to model the context dependencies between parameter items, and the CRF model was used for global decoding of the label sequence. This task defined 9 types of named entities, and the architectural text entity categories and BIO rules are shown in Table 3. Therefore, the output dimension of the CRF layer was set to 9. Finally, after the training was completed, the model performance was evaluated on the test set. The hyperparameter settings for model training are shown in Table 4.

[0079] Table 3 Architectural text entity categories and BIO rules

[0080] Table 4 Hyperparameter settings for model training

[0081] (c) Model comparison and performance evaluation Based on the unified model architecture, hyperparameter settings, and data annotation system, this paper designed four groups of comparative experiments to analyze the impact of different strategy combinations on the performance of parameter entity recognition in building semantic texts.

[0082] The experiments used accuracy (Precision) and recall (Recall) as the main evaluation indicators, and the comparison results of text parameter extraction performance are shown in Table 5.

[0083] Table 5 Comparison of Text Parameter Extraction Performance

[0084] 3) Analysis of Table Parameter Extraction and Recognition Effect This method extracts the text content within the cell by referring to the identified vertex coordinates of the cell, specifically extracts text data through coordinate intervals, and then saves the data in a JSON format file. During the experiment, for different types of tables, relevant attribute information extraction and experimental evaluation were carried out. The differences in recognition accuracy and extraction effect of different table types during the extraction process are shown in Table 6.

[0085] Table 6 Differences in Recognition Accuracy and Extraction Effect of Different Table Types during Extraction

[0086] Based on the community version of Neo4j, the import and management of multimodal data were realized. First, the graph database was configured. After setting the environment variables, the neo4j.bat console command was run in the console to start the graph database. The console will display the default port containing the Bolt protocol (such as: 7687) and the HTTP access address (such as: http: / / localhost:7474). Then, this address is entered into the browser, and the graphical interface Neo4j Browser of the graph database can be entered, where the data model can be viewed in the visualization interface.

[0087] This paper used the graph database Neo4j to conduct a structured relationship modeling of multimodal data, intuitively showing how the building information stored in the knowledge graph is mapped to the IFC standard semantic structure. During the experiment, a building knowledge graph containing 4690 entity nodes and 5956 relationships was automatically generated, covering main components such as walls, floors, doors and windows, columns, and their related parameters. Figure 8 A global overview diagram of the knowledge graph is shown, where nodes of different colors represent different types of building components. For example, green nodes represent walls and blue nodes represent doors and windows, clearly reflecting the hierarchical relationship and spatial topological structure between the components.

[0088] To verify the correctness of the constructed knowledge graph, this method conducts query verification on the entities and their relationships in the graph database. The specific steps are to use Cypher language query statements to retrieve key parameters such as the geometric information, material properties, and fire resistance ratings of the walls, and compare the query results with the originally extracted JSON-formatted data to verify the accuracy of data extraction.

[0089] (4)IFC File Generation This paper designs an IFC model construction method based on a knowledge graph to achieve the standardized conversion of semantic information into three-dimensional modeling parameters. This method relies on the component entities and their attribute information stored in the graph database Neo4j to establish a data mapping mechanism from the semantic layer to the modeling layer, and combines the IFC standard to complete the structured conversion and model generation of component parameters.

[0090] As Figure 9 shown, the IFC file construction process based on Neo4j and IfcOpenShell includes: Step 1: Initialize the IFC project.

[0091] Specifically, use the ifcopenshell library to create an IFC file and initialize the project structure. The pseudo-code for initializing the IFC project is as follows: import ifcopenshell / / model = ifcopenshell.file() / / Create an IFC file object; / / Create the IFC project structure project=model.create_entity("IfcProject", Name="Building Project") / / Create a project entity; site= model.create_entity("IfcSite", Name="Building Site") / / Create a site entity; building=model.create_entity("IfcBuilding",Name="Comprehensive Building") / / Create building and other entities; model.create_entity("IfcRelAggregates",RelatingObject=project,RelatedObjects=[site]) / / Establish the hierarchical relationship between the project and the site; model.create_entity("IfcRelAggregates", RelatingObject=site, RelatedObjects=[building]) / / Establish the hierarchical relationship between the site and the building.

[0092] Step 2: Extract data from Neo4j.

[0093] Specifically, use the py2neo library to connect to the Neo4j database and query the Wall nodes and their geometric attributes.

[0094] The pseudocode for extracting data from Neo4j is as follows: from py2neo import Graph / / Extract data from Neo4j graph = Graph("bolt: / / localhost:7687", auth=("neo4j", "password")) / / Connect to the local neo4j database; walls = graph.run(""" MATCH (w:Wall)-[:HAS_GEOMETRY]->(g:Geometry) / / Query data RETURN w.GlobalId, w.name, g.start, g.length, g.thickness, g.height """).data() / / The query result is a list, and each element contains information such as the ID, name, start point, length, thickness, and height of the wall.

[0095] Step 3: IFC data conversion.

[0096] Specifically, traverse the wall data obtained from Neo4j and convert it into IFC entities.

[0097] The pseudocode for IFC data conversion is as follows: for wall in walls: ifc_wall = model.create_entity("IfcWallStandardCase", Name=wall['name']) / / Create a wall node; geom = model.create_entity("IfcExtrudedAreaSolid", Depth=wall['length']) / / Create the wall geometry; ifc_wall.Representation = geom / / Each wall is converted into a wall object in the IFC standard and geometric information is assigned.

[0098] Step 4: Export the IFC file.

[0099] Specifically, write the constructed IFC model to a local file.

[0100] The pseudo-code for exporting the IFC file is as follows: model.write("output.ifc") / / Export the IFC file; print("IFC file has been successfully generated!") / / Print that the IFC file has been successfully generated.

[0101] (5) Visual verification and integrity analysis of the IFC model After completing the export of the IFC file, this paper uses the IfcViewer tool to perform visual verification on the generated model to ensure the correctness and integrity of the model. The verification process includes four aspects: the integrity of the model, property parameters, geometric parameters, and spatial relationships: 1) Model integrity: The 3D view of the IFC model is as Figure 10 shown, Figure 10 clearly showing the overall structure of the building and various components, such as walls, doors, and windows, and the components are correctly assigned to the corresponding floors, meeting the IFC aggregation relationship standard.

[0102] 2) Property parameters: Checked the material properties, thermal conductivity, fire resistance rating, etc. of walls, doors, and windows to ensure that the property information is correctly mapped to the IFC property set and complies with the IFC standard.

[0103] 3) Geometric parameters: The sectional view of the IFC model is as Figure 11 described, Figure 11 showing that the dimensional deviation of the components is controlled within ±2 mm, the spatial position, dimensional ratio, and nested structure are consistent with the design drawings, and the floor boundary is accurately defined.

[0104] 4) Spatial relationships: Verified the attribution relationships, nested relationships, and topological connections in the model to ensure that the spatial hierarchical structure is clear and there is no misalignment phenomenon, meeting the IFC semantic standard.

[0105] It should be understood that although Figure 1 the steps in the flowchart of Figure 1At least some of the steps may include multiple sub-steps or multiple stages, and these sub-steps or stages do not necessarily need to be completed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages does not necessarily need to be sequential, but can be executed alternately or in turn with at least some of the other steps or the sub-steps or stages of the other steps.

[0106] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0107] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application.

Claims

1. A building automated modeling method based on multimodal information and knowledge graph, characterized in that The method includes: Adopting vector graphic element parsing and layer semantic recognition to extract component geometric information from architectural drawings; Combining the BERT-BiLSTM-CRF model and dictionary enhancement strategy to perform named entity recognition on architectural texts to obtain component semantic parameters; Extracting component attribute information from architectural tables through table structure recognition and table logic reconstruction; Using a JSON file to standardize and store the component geometric information, the component semantic parameters, and the component attribute information, and introducing a unique identifier ID mechanism to complete the association between multi-source data; Constructing a node-relationship model based on the IFC standard, and using the Neo4j knowledge graph to express the semantic, spatial, and attribute relationships of components, so as to realize the unified storage and association modeling of multi-modal data; Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, using GlobalID as the unique identifier of parameter data, to achieve the accurate docking of the knowledge graph and the IFC model; By calling the IfcOpenShell programming interface, automatically mapping the parameters retrieved from the knowledge graph to generate IFC component entities, so as to realize the automatic construction of the building information model.

2. The building automation modeling method based on multimodal information and knowledge graph according to claim 1, characterized in that, Adopting vector graphic element parsing and layer semantic recognition to extract component geometric information from architectural drawings, including: In terms of drawing parameter extraction, establishing a complete extraction process from drawing preprocessing, component recognition, topological relationship reconstruction to parameter structured output; Through layer standardization and graphic element geometric feature analysis, realizing the accurate extraction of the geometric position information of basic components; The basic components include walls and doors and windows.

3. The building automation modeling method based on multimodal information and knowledge graph according to claim 1, characterized in that Constructing a node-relationship model based on the IFC standard, and using the Neo4j knowledge graph to express the semantic, spatial, and attribute relationships of components, so as to realize the unified storage and association modeling of multi-modal data, including: Constructing a node-relationship model based on the IFC standard, and the node-relationship model based on the IFC standard includes: hierarchical structure relationship mapping, spatial attribution relationship mapping, and topological connection relationship mapping; The hierarchical structure relationship mapping is used to standardize the subordination relationship between projects, buildings, floors, and components; In the knowledge graph, according to the aggregation relationship model, the HAS_BUILDING, HAS_FLOOR, and CONTAINS_ELEMENT relationships are defined to build the hierarchical organizational structure of projects, buildings, floors, and components; Among them, HAS_BUILDING reflects the attribution association between the project and the building; HAS_FLOOR indicates the subordination relationship between the building and the floor; CONTAINS_ELEMENT represents the relationship between the components contained in the floor; By using the IFC relationship standard to construct the knowledge graph of multi-modal data, the building information data can be orderly regulated in the multi-level structure, so as to realize the gradual refinement from the project to the components; Spatial attribution relationship mapping is used to clarify the specific location, functional zoning, and spatial dependence of components in the building environment; the nodes in the knowledge graph are mapped to IfcBuilding, IfcBuildingStorey, and IfcSpace, and the relationship concepts of "Contains" and "FillsElement" are also introduced to express the spatial subordination and nesting of components, ensuring the integrity and consistency of data; Topological connection relationship mapping is used to describe the geometric connection, physical support, and functional nesting between components, which is a key way to reflect the "physical contact" and "attachment relationship" of components in the modeling process; in the knowledge graph, the topological relationship is defined through relationship types such as "AdjacentTo", "SupportedBy", and "HasOpening"; Each node is assigned a GlobalID as the unique identifier, combined with the entity name and description, to improve the integrity and queryability of model parameters.

4. The building automation modeling method based on multimodal information and knowledge graph according to claim 1, characterized in that, The entity mapping mechanism based on the IFC standard includes the IFC component mapping strategy, the IFC parameter mapping strategy, and the identifier mapping strategy; The IFC component mapping strategy is the mapping rule proposed for typical building components according to the information description mode of the IFC standard; The IFC parameter mapping strategy includes: geometric parameter mapping rules, physical parameter mapping rules, and functional parameter mapping rules; the IfcPropertySet and IfcMaterial mechanisms are introduced to achieve hierarchical mapping of parameters; The identifier mapping strategy is used to map the unique identifier field of components in the knowledge graph to the global unique identifier GlobalID attribute of the IFC model. When the model is incrementally updated, the global unique identifier exists as the unique identifier. When adding, deleting, or modifying component parameters, data consistency is maintained. The global unique identifier is also used as the basis for matching components from different data sources; at the same time, when querying data, the global unique identifier is also used as the index field for retrieving the model and query parameters for rapid positioning of components and their associated attributes.

5. The building automation modeling method based on multimodal information and knowledge graph according to claim 1, wherein The relationship mapping mechanism based on the IFC standard includes the hierarchical attribution relationship mapping rule between components, the spatial attribution relationship mapping rule between components, and the topological connection relationship mapping rule between components.

6. The building automation modeling method based on multimodal information and knowledge graph according to claim 1, characterized in that Based on the entity mapping mechanism and relationship mapping mechanism of the IFC standard, with GlobalID as the unique identifier of parameter data, the accurate docking between the knowledge graph and the IFC model is realized, including: Combined with the knowledge graph structure, an automatic conversion process of IFC spatial coordinates is constructed, extracting the spatial positions of components from bottom to top and mapping them to global coordinates in sequence to realize the coordinate expression of components in the IFC model; Taking IfcExtrudedAreaSolid as the core method, automatic modeling of walls and slabs is carried out; by extracting the semantic parameters of components in the knowledge graph and mapping them to IFC fields, automatic generation from the semantic layer to the model layer is realized.

7. The building automation modeling method based on multi-modal information and knowledge graph according to claim 6, characterized in that The specific steps of the automatic conversion process of IFC spatial coordinates include: Extract the relative coordinate information of components from Neo4j and determine whether there is the global coordinate of the upper-level object; If it exists, directly use the existing coordinate standard; If it does not exist, establish the coordinate hierarchy through PlacementRelTo, and combine RelativePlacement with the IfcAxis2Placement3D entity to define the origin and direction of the local coordinate system, so as to realize the spatial mapping from the upper-level coordinate system; Use IfcAxis2Placement3D for coordinate storage and convert it to IfcLocalPlacement.

8. The building automation modeling method based on multimodal information and knowledge graph according to claim 1, characterized in that By calling the IfcOpenShell programming interface, the parameters retrieved from the knowledge graph are automatically mapped to generate IFC component entities, realizing the automatic construction of the building information model, including: Use the Cypher language to extract the geometric parameters, semantic information and spatial location-related data of building components from the knowledge graph; In data mapping and model construction, use the IFCOpenShell API interface to automatically fill the extracted data parameters into the corresponding IFC entities; map the wall data to the standard wall class; regard doors and windows as two independent components, map doors and windows to IfcDoor and IfcWindow respectively, and nest them into the corresponding wall openings according to IfcRelFillsElement; In the process of model verification and export, use the ifcopenshell.validation module under the IFCOpenShell open-source tool to check the data integrity, geometric relationship and attribute parameters. The finally generated IFC file can view its standard data structure in a text editor, and can also be input into a visualization tool that supports the IFC format for three-dimensional visualization display; In the process of automatically constructing the floor slab, the floor slab data is mapped to IfcSlab, and IfcExtrudedAreaSolid is used to complete the construction of the floor slab geometric model; the floor slab geometric profile is established by IfcProfileDef; the IfcMaterialLayerSetUsage is used to standardize the definition of material parameters to ensure the completeness of the attribute parameters of the floor slab in the IFC model.

Images