Construction safety intelligent inspection method based on BIM+AR and knowledge base

Abstract

The application discloses a construction safety intelligent inspection method based on BIM+AR and a knowledge base, and comprises the following steps: constructing a construction safety knowledge graph and a knowledge base, constructing a BIM information base, constructing an augmented reality inspection scene based on BIM, searching safety knowledge based on BIM information, and constructing a construction safety BIM+AR intelligent inspection system. BIM, AR, natural language processing and deep learning technology are utilized to provide real-time knowledge assistance for managers in the construction safety inspection process, guide real-time feedback of the inspection, overcome the difference between the experience of managers and the cognition level of specifications, realize intelligent knowledge guidance of the safety inspection, avoid inspection omissions caused by insufficient experience and unfamiliarity with specifications and regulations of managers in a multi-specialty cross construction scene, and help to comprehensively and quickly identify safety hazards and reduce the probability of accidents.

Description

Technical Field

[0001] The present invention belongs to the technical field of construction safety and relates to an intelligent construction safety inspection method based on BIM+AR and a knowledge base. Background Art

[0002] Safety hazards can easily lead to accidents, hindering construction efficiency and threatening construction safety. Promptly identifying and addressing safety hazards has become a key focus of construction site safety management. Construction safety inspections are crucial for identifying and preventing safety hazards. Large-scale construction projects, characterized by complex on-site environments and the interplay of diverse disciplines, place higher demands on managers' understanding of relevant safety management regulations and practices. Conventional construction site safety inspections are limited by managers' misunderstanding of safety regulations and practices, resulting in inadequate implementation and non-automated management methods, which inevitably lead to oversights in on-site safety inspections. Augmented reality (AR) technology can overlay images, text, and other information onto real-world scenes, enabling viewers to interact with virtual information in real-world scenarios. Combined with engineering design and construction information and the corresponding safety management knowledge base within Building Information Modeling (BIM), AR provides an effective tool for addressing these issues.

[0003] Currently, there are relevant researches on engineering management and control based on BIM+AR technology. Liu Dandan constructed a mechanical and electrical field database based on WBS (Work Breakdown Structure), established the relationship between BIM components and WBS codes through the connection rule set, and developed a corresponding BIM+AR mechanical and electrical engineering inspection application; Cao Yang et al. designed a construction safety management system combining BIM and AR technology, and Peng Lei proposed a subway mechanical and electrical equipment maintenance application framework based on augmented reality, which relies on BIM model for multi-source information integration to realize the safety management and control or equipment maintenance of three-dimensional model and information visualization in virtual and real fusion scene; Zhang Tian used BIM+AR technology to collect and integrate real-time information such as construction site safety, quality, and progress, and constructed a mechanical and electrical construction information cloud platform to realize efficient mechanical and electrical construction site information management; Cao Wei et al. invented an AR-based building construction design system, which divides building construction into a number of modular units and elements for AR simulation of the entire building construction design; Chen Y J et al. proposed a BIM+AR-based fire-fighting equipment inspection method to overcome the influence of tedious paper file retrieval on efficiency in the inspection process; Alirezaei S et al. proposed a construction risk management and control system combining BIM and AR; Ratajczak J et al. developed a location-based augmented reality construction application system to provide construction site personnel with information such as building materials and construction task lists, and realize AR visualization of construction progress and performance data calculation.

[0004] The above researches show that BIM+AR technology can realize the virtual and real fusion display and interaction of various information on the construction site, providing tools for guiding work and on-site inspection. However, the existing technology focuses on manually building BIM information library and visualizing augmented reality display of geometry and design drawings, and does not propose a method for associating massive and unstructured text construction safety knowledge with BIM models and providing safety knowledge prompts in on-site inspection to realize virtual and real augmented construction safety inspection guidance. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a construction safety intelligent inspection method based on BIM+AR and knowledge base, which uses BIM, AR, natural language processing, and deep learning technology to provide knowledge assistance for managers in real time during construction safety inspection, guide real-time feedback, and overcome the differences in experience and specification recognition level of managers to limit the effect of construction site safety inspection.

[0006] To solve the above technical problems, the technical solution adopted by the present application is: a construction safety intelligent inspection method based on BIM+AR and knowledge base, comprising the following steps:

[0007] Step 1: Build a construction safety knowledge graph and knowledge base. Based on the different types of unit projects, sub-items, and sub-projects involved, use natural language processing and deep learning methods to extract entities from relevant construction safety specifications, construction organization designs, and safety management document texts to build a construction safety knowledge graph and knowledge base. The construction safety knowledge graph consists of edges and nodes, where nodes are divided into three categories: the first type of node defines the project type. Pro ; The second type of node definition corresponds to a certain entity part in the project Com ; The third type of node defines the associated security knowledge Kno ;

[0008] Step 2: Build a BIM information library, including BIM model component code, component name, component size, project type, and component physical location information. The component code is the unique identifier of the BIM component and is used to search the building information library.

[0009] Step 3: Build an augmented reality inspection scene based on BIM. Export the BIM model to build the AR scene. Bind the component code, which serves as unique identification information, to the BIM model component one by one through the component name or script form. Simultaneously, set registration markers, design the UI interface, and write scripts for the knowledge base interactive query function.

[0010] Step 4: Safety knowledge retrieval based on BIM information: Retrieve relevant information from the BIM information database based on BIM component codes, perform knowledge graph retrieval based on project type and physical location, and intelligently query relevant knowledge. The project type and construction location in BIM information are manually entered, and their language expression may not be completely consistent with the expression in the standard safety management documents. Therefore, a method based on semantic similarity comparison is used to perform knowledge graph retrieval.

[0011] Based on the pre-trained language model, obtain the semantic vector of the selected BIM component corresponding to the engineering type A 1. Semantic vector corresponding to entity parts A 2;

[0012] Get all first-class node engineering types in the knowledge graph ,in I is the number of nodes in the first category, and its semantic vector is obtained using the same pre-trained language model B i , calculate them one by one according to the cosine similarity formula (1) A Semantic similarity of 1 , and sort them from small to large, and get the node with the largest similarity value greater than the set threshold ;

[0013] ;(1)

[0014] Traverse all nodes The second type of associated nodes ,in J is the number of nodes in the second category, and its semantic vector is obtained using the same pre-trained language model C j , calculate them one by one according to formula (1) A Semantic similarity of 2 , and sort them from small to large, and get the node with the largest similarity value greater than the set threshold ;

[0015] Step 5: Based on the construction safety BIM+AR intelligent inspection system, on the AR mobile device, the BIM model is aligned with the actual application scenario to enter the AR scene that integrates virtual and real elements. The BIM model components in the AR scene are named with component codes. By selecting the corresponding model component and clicking the knowledge base query button, the component code is automatically transferred to the database, and the relevant safety knowledge is returned and rendered in the AR interface to realize safety knowledge-guided inspections.

[0016] In step 1, the specific steps for building the construction safety knowledge graph and knowledge base are as follows:

[0017] In step 1-1, relevant construction safety specifications and safety management documents were collected and divided into paragraphs. To ensure effective entity and relationship extraction, each paragraph was limited to 512 characters. The entities and relationships were then annotated using the doccano annotation platform. Entities were categorized into project type and construction location, and relationships into containment and location. The annotated dataset was divided into a training set and a test set with an 8:2 ratio.

[0018] In step 1-2, select the ERNIE-UIE general information extraction model for entity and relationship extraction, and fine-tune the model using the divided training set; the information extraction template schema is defined as:

[0019] schema=['project type', 'construction location', {'project type': ['includes']}, {'physical location': ['includes', 'located at']}]; (2)

[0020] Steps 1-3, knowledge graph and knowledge base construction, are constructed based on relevant safety specifications and management documents. First, the specifications are encoded according to chapters and items as the third type of entity in the knowledge graph; then, the model fine-tuned in steps 1-2 is used to extract entities and relationships for the project types and construction locations involved in each specification content, and organize them into triples.

[0021] In step 2, the BIM information database is constructed based on the analysis of IFC (Industry Fundamental Classes) files. The specific steps are as follows:

[0022] Step 2-1: enrich the semantics of the IFC architecture and add the attribute set Pset_projectinformation, which contains two attributes: "project type" and "construction location", both in text format;

[0023] Step 2-2: Supplement the engineering type and construction location parameters of each component in the BIM modeling software, and export the BIM information into an IFC format file based on the semantically rich IFC data structure;

[0024] In steps 2-3, a Python program is written using the IfcOpenshell plug-in to extract the GUID, name, family, project type, and construction location information of each physical component in the BIM model to build a BIM information database. The GUID is the unique code of each BIM physical component in the IFC architecture and serves as the primary key of the BIM information database.

[0025] In step 4, the BIM information library, knowledge graph, and knowledge base are used to implement security knowledge retrieval based on the BIM component GUID in the AR scene based on the semantic similarity between the BIM component's project type, construction location, and knowledge graph. The specific steps are as follows:

[0026] Step 4-1: Select the pre-trained language model Chinese-RoBERTa-wwm-ext that has been fine-tuned based on text data in the construction field as the encoder, with a vector dimension of 768, and obtain the semantic vector corresponding to the project type returned by the selected BIM component information library. , semantic vector corresponding to the construction location ;

[0027] Step 4-2: Select all the first-class node engineering types in the knowledge graph constructed in step 2 , use the same pre-trained language model to obtain its semantic vector According to the cosine similarity calculation formula in formula (3), the semantic similarity cos_sim( A 1, B i ), the closer the similarity is to 1, the higher the similarity is; set the similarity threshold to 0.8, filter all nodes with similarity greater than 0.8 and sort them, and obtain the node with the highest similarity value greater than the set threshold ; ;(3)

[0028] Step 4-3, extract all nodes with Directly related , or indirectly related to The second type of node , use the same pre-trained language model in step 2 to obtain its semantic vector , calculate them one by one with A 2 and sort them to get the node with the highest similarity value greater than the set threshold. ;

[0029] Step 4-4: If there is no second-class node with a similarity value greater than the set threshold, select the first-class node with the second highest similarity value in step 2, and repeat steps 4-3 until the node is obtained. ;

[0030] Step 4-5, traverse all nodes Directly associated third type of node , obtain the associated security knowledge item code;

[0031] Step 4-6: In the construction safety knowledge base, the corresponding safety knowledge and knowledge source documents and chapter names are obtained based on the knowledge entry codes in step 4-5, thereby realizing the safety knowledge retrieval associated with BIM information.

[0032] In step 5, knowledge-enhanced intelligent inspection of construction safety is implemented. The AR inspection device is Hololens2, and the knowledge graph and knowledge base constructed in step 2 are used. The specific steps are as follows:

[0033] Step 5-1: Publish and upload safety inspection tasks. The administrator logs in to the backend management system, selects the project to be inspected, creates a new inspection task, and uploads the inspection task list. The list should include the safety inspection location and inspection content, and is synchronized to the remote database.

[0034] Step 5-2, task reception: Project personnel receive the task on the mobile terminal, retrieve the corresponding BIM model, scan the landmarks to achieve registration and 3D registration between the model and the real scene, and enter the virtual-reality fusion inspection scene;

[0035] Step 5-3: Knowledge-guided inspection. During the inspection process, when the inspector selects a component and clicks on the safety knowledge query, the mobile system will automatically obtain the name of the selected component, that is, the GUID, and send it to the BIM information database. The BIM information database uses the GUID to retrieve the project type and construction location corresponding to the component. The application server searches for relevant safety knowledge according to step 3 and returns the safety knowledge to the AR inspection mobile terminal, rendering it on the AR interface, thereby guiding managers to conduct inspections as required. At the same time, managers can view the corresponding inspection task list in the AR scene and check them one by one.

[0036] Step 5-4, inspection result feedback; During the inspection process, the manager records the safety issues and safety hazard images through the AR mobile terminal. After the inspection, the manager exits the AR mobile terminal, and the inspection task status is automatically updated to completed. It is automatically uploaded to the inspection problem database and distributed to the relevant person in charge. The relevant person in charge can view and rectify the records through the background management system to achieve closed-loop safety management.

[0037] The main beneficial effects of the present invention are:

[0038] By utilizing natural language processing, deep learning and other technologies, a knowledge graph for construction safety inspections is constructed based on texts such as safety specifications and safety management documents, and associated queries based on BIM information are implemented. This avoids the time-consuming and labor-intensive manual organization of knowledge and associated design, and provides knowledge support for knowledge-guided construction site safety inspections.

[0039] A method and system for construction site augmented reality safety inspections coupled with a knowledge base are proposed. This method implements intelligent knowledge guidance for safety inspections and overcomes inspection omissions caused by managers' lack of experience and unfamiliarity with regulations in multi-disciplinary construction scenarios. This helps ensure comprehensive and rapid identification of safety hazards and reduce the probability of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention will be further described below with reference to the accompanying drawings and examples.

[0041] Figure 1 Flowchart of the present invention.

[0042] Figure 2 This is the knowledge graph structure diagram of the present invention.

[0043] Figure 3 This is the knowledge base diagram of the present invention. DETAILED DESCRIPTION

[0044] like Figures 1 to 3 In the present invention, a construction safety intelligent inspection method based on BIM+AR and knowledge base includes the following steps:

[0045] Step 1: Build a construction safety knowledge graph and knowledge base. Based on the different types of unit projects, sub-items, and sub-projects involved, use natural language processing and deep learning methods to extract entities from relevant construction safety specifications, construction organization designs, and safety management document texts to build a construction safety knowledge graph and knowledge base. The construction safety knowledge graph consists of edges and nodes, where nodes are divided into three categories: the first type of node defines the project type. Pro ; The second type of node definition corresponds to a certain entity part in the project Com ; The third type of node defines the associated security knowledge Kno ;

[0046] Preferably, the construction safety knowledge graph is used to integrate construction safety knowledge texts from multiple sources based on different project types and building entity parts to achieve rapid retrieval of safety knowledge.

[0047] Preferably, since the knowledge graph is not convenient for representing long texts, the third type of nodes Kno It mainly consists of security knowledge item codes, which are unique and correspond to each item of security knowledge. The edges between nodes represent the relationship between different entities, and the nodes of the same type are subordinate relationships. Pro 1, contain, Pro 2}, { Com 1, contain, Com 2},{ Kno 1, contain, Kno 2}. The relationship between the first and second type nodes is located at locateat, { Com 1, locateat, Pro 1}; The relationship between the first or second type of node and the third type of node is satisfying, { Pro 1,satisfy, Kno 1},{ Com 1,satisfy, Kno 2}.

[0048] Preferably, the construction safety knowledge base is used to store safety knowledge content, which should include four fields: knowledge item code, safety file name, chapter name, and safety knowledge, to assist in obtaining safety knowledge based on knowledge graph retrieval results; wherein the knowledge item code is a unique primary key, which is consistent with the knowledge item code in the knowledge graph.

[0049] Step 2: Build a BIM information library, including BIM model component code, component name, component size, project type, and component physical location information. The component code is the unique identifier of the BIM component and is used to search the building information library.

[0050] Step 3: Build an augmented reality inspection scene based on BIM. Export the BIM model to build the AR scene. Bind the component code, which serves as unique identification information, to the BIM model component one by one through the component name or script form. Simultaneously, set registration markers, design the UI interface, and write scripts for the knowledge base interactive query function.

[0051] Step 4: Safety knowledge retrieval based on BIM information: Retrieve relevant information from the BIM information database based on BIM component codes, perform knowledge graph retrieval based on project type and physical location, and intelligently query relevant knowledge. The project type and construction location in BIM information are manually entered, and their language expression may not be completely consistent with the expression in the standard safety management documents. Therefore, a method based on semantic similarity comparison is used to perform knowledge graph retrieval.

[0052] Based on the pre-trained language model, obtain the semantic vector of the selected BIM component corresponding to the engineering type A 1. Semantic vector corresponding to entity parts A 2;

[0053] Get all first-class node engineering types in the knowledge graph ,in I is the number of nodes in the first category, and its semantic vector is obtained using the same pre-trained language model B i , calculate them one by one according to the cosine similarity formula (1) A Semantic similarity of 1 , and sort them from small to large, and get the node with the largest similarity value greater than the set threshold ;

[0054] ;(1)

[0055] Traverse all nodes The second type of associated nodes ,in J is the number of nodes in the second category, and its semantic vector is obtained using the same pre-trained language model C j , calculate them one by one according to formula (1) A Semantic similarity of 2 , and sort them from small to large, and get the node with the largest similarity value greater than the set threshold ;

[0056] Step 5: Based on the construction safety BIM+AR intelligent inspection system, on the AR mobile device, the BIM model is aligned with the actual application scenario to enter the AR scene that integrates virtual and real elements. The BIM model components in the AR scene are named with component codes. By selecting the corresponding model component and clicking the knowledge base query button, the component code is automatically transferred to the database, and the relevant safety knowledge is returned and rendered in the AR interface to realize safety knowledge-guided inspections.

[0057] In the preferred solution, in step 1, the specific steps of building the construction safety knowledge graph and knowledge base are as follows:

[0058] In step 1-1, relevant construction safety specifications and safety management documents were collected and divided into paragraphs. To ensure effective entity and relationship extraction, each paragraph was limited to 512 characters. The entities and relationships were then annotated using the doccano annotation platform. Entities were categorized into project type and construction location, and relationships into containment and location. The annotated dataset was divided into a training set and a test set with an 8:2 ratio.

[0059] In step 1-2, select the ERNIE-UIE general information extraction model for entity and relationship extraction, and fine-tune the model using the divided training set; the information extraction template schema is defined as:

[0060] schema=['project type', 'construction location', {'project type': ['includes']}, {'physical location': ['includes', 'located at']}]; (2)

[0061] Steps 1-3, knowledge graph and knowledge base construction, are constructed based on the "Municipal Engineering Construction Safety Inspection Standard". First, the specifications are encoded according to chapters and items as the third type of entity in the knowledge graph; then, the fine-tuned model in (2) is used to extract entities and relationships for the project types and construction locations involved in each specification content, and organize them into triples.

[0062] Preferably, if { Com _Steel sheet pile, locateat, Pro _Foundation pit support engineering}, { Pro _ Foundation engineering,contain, Pro _ foundation pit support engineering}, and set the relationship between the engineering part entity and the code of the specification item in the corresponding specification item to be satisfied, such as { Com _Steel sheet pile,satisfy, Kno _Safety Knowledge Item 1.1.1}; If the corresponding item does not identify the engineering part entity, the relationship between the engineering type entity and the specification item code is set to be satisfied, such as { Pro _Foundation pit support engineering, satisfaction, Kno _Safety Knowledge Item 1.1.1}, to build the construction safety knowledge graph. The extracted entities and relationships between entities are imported into Neo4j to obtain the structure of the knowledge graph as follows Figure 2 As shown in Figure 3 At the same time, the four fields of specification name, chapter name, corresponding knowledge item code, and knowledge content are stored in the relational database for subsequent knowledge query.

[0063] In the preferred solution, in step 2, a BIM information database is constructed based on the analysis of IFC, i.e., Industry Fundamental Classes files. The specific steps are as follows:

[0064] Step 2-1: enrich the semantics of the IFC architecture and add the attribute set Pset_projectinformation, which contains two attributes: "project type" and "construction location", both in text format;

[0065] Step 2-2: Supplement the engineering type and construction location parameters of each component in the BIM modeling software, and export the BIM information into an IFC format file based on the semantically rich IFC data structure;

[0066] In steps 2-3, a Python program is written using the IfcOpenshell plug-in to extract the GUID, name, family, project type, and construction location information of each physical component in the BIM model to build a BIM information database. The GUID is the unique code of each BIM physical component in the IFC architecture and serves as the primary key of the BIM information database.

[0067] In the preferred solution, in step 4, the BIM information library, knowledge graph, and knowledge base are used to implement security knowledge retrieval based on the BIM component GUID in the AR scene based on the semantic similarity between the BIM component's project type, construction location, and knowledge graph. The specific steps are as follows:

[0068] Step 4-1: Select the pre-trained language model Chinese-RoBERTa-wwm-ext that has been fine-tuned based on text data in the construction field as the encoder, with a vector dimension of 768, and obtain the semantic vector corresponding to the project type returned by the selected BIM component information library. , semantic vector corresponding to the construction location ;

[0069] Step 4-2: Select all the first-class node engineering types in the knowledge graph constructed in step 2 , using the same pre-trained language model to obtain its semantic vector , the semantic similarity cos_sim between the engineering type corresponding to the BIM component and all first-type nodes is calculated according to the cosine similarity calculation formula in formula (3) A 1, B i The closer the similarity is to 1, the higher the similarity is. The similarity threshold is set to 0.8, all nodes with a similarity greater than 0.8 are filtered and sorted to obtain the node with the highest similarity value greater than the set threshold ; ; (3)

[0070] Step 4-3, extract all second-type nodes directly associated with node , i.e. , or indirectly associated, i.e. , using the same pre-trained language model in step 2 to obtain its semantic vector , calculate the semantic similarity of each node with 2 and sort them to obtain the node with the highest similarity value greater than the set threshold A ;

[0071] Step 4-4, if there is no second-type node with a similarity value greater than the set threshold, select the second-highest first-type node in step 2, repeat step 3 until ;

[0072] Step 4-5, traverse all third-type nodes directly associated with node , obtain the associated safety knowledge item code ;

[0073] Step 4-6, in the construction safety knowledge base, based on the knowledge item code in step 4-5, obtain the corresponding safety knowledge and knowledge source document, chapter name, thereby realizing the safety knowledge retrieval associated with BIM information.

[0074] In the preferred scheme, in step 5, the construction safety intelligent inspection with knowledge enhancement is realized, the AR inspection device selected is Hololens2, and the knowledge graph and knowledge base constructed in step 2 are adopted, the specific steps are as follows:

[0075] Step 5-1, safety inspection task publishing and uploading; the management personnel logs in the background management system, selects the project to be inspected, creates a new inspection task and uploads the inspection task list, the list should include the safety inspection part and the inspection content, and is synchronized to the remote database;

[0076] ​Step 5-2, task reception: Project personnel receive the task on the mobile terminal, retrieve the corresponding BIM model, scan the landmarks to achieve registration and 3D registration between the model and the real scene, and enter the virtual-reality fusion inspection scene;

[0077] Step 5-3: Knowledge-guided inspection. During the inspection process, when the inspector selects a component and clicks on the safety knowledge query, the mobile system will automatically obtain the name of the selected component, that is, the GUID, and send it to the BIM information database. The BIM information database uses the GUID to retrieve the project type and construction location corresponding to the component. The application server searches for relevant safety knowledge according to step 3 and returns the safety knowledge to the AR inspection mobile terminal, rendering it on the AR interface, thereby guiding managers to conduct inspections as required. At the same time, managers can view the corresponding inspection task list in the AR scene and check them one by one.

[0078] Step 5-4, inspection result feedback; During the inspection process, the manager records the safety issues and safety hazard images through the AR mobile terminal. After the inspection, the manager exits the AR mobile terminal, and the inspection task status is automatically updated to completed. It is automatically uploaded to the inspection problem database and distributed to the relevant person in charge. The relevant person in charge can view and rectify the records through the background management system to achieve closed-loop safety management.

[0079] The implementation of the above method relies on a construction safety BIM+AR intelligent inspection system, which consists of three parts: a database and application server, a backend management terminal, and an AR mobile terminal. The database and application server are used to store and analyze inspection tasks, the safety knowledge base and knowledge graph, inspection feedback data, and the BIM model information library. The AR mobile terminal can be a smartphone, tablet, or augmented reality headset. It should be an intelligent device with network connectivity, display, data acquisition, positioning, and sufficient computing power. It is used to receive safety inspection tasks and realize the virtual and real overlay of BIM models and intelligent inspection guidance. The backend management terminal includes inspection task management, inspection result feedback and rectification, and safety knowledge base management modules.

[0080] Before conducting an inspection, managers can upload safety knowledge documents based on different project types in the backend management system and transfer them to the application server for parsing. Using information extraction technology, the application server extracts project part entities, project type entities, and inter-entity relationships from the different knowledge items in the documents, expanding the existing construction safety inspection knowledge graph and safety knowledge base. Then, managers publish inspection tasks in the backend management system's task management module, upload corresponding task forms, and synchronize them to a remote database. Inspectors receive the tasks on their mobile devices, select the corresponding BIM model for 3D registration, and enter the AR scene to conduct safety inspections. During the safety inspection, inspectors can conduct item-by-item inspections based on the uploaded task forms, query the associated knowledge of corresponding BIM components for targeted safety checks, and provide real-time feedback and record safety issues. The associated knowledge query, after sending the BIM component code to the remote database and server via the AR mobile device, proceeds according to step 4. After the inspection, responsible personnel can review safety issues and record their rectifications in the backend management system, achieving closed-loop safety management.

[0081] The above embodiments are merely preferred technical solutions of the present invention and should not be construed as limiting the present invention. The embodiments and features in the embodiments of this application may be arbitrarily combined with each other unless they conflict. The scope of protection of the present invention shall be the technical solutions described in the claims, including equivalent alternatives to the technical features of the technical solutions described in the claims. Equivalent alternatives and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A construction safety intelligent inspection method based on BIM+AR and knowledge base, characterized by: The steps include: Step 1: Build a construction safety knowledge graph and knowledge base. Based on the different types of unit projects, sub-items, and sub-projects involved, use natural language processing and deep learning methods to extract entities from the text of relevant construction safety specifications, construction organization designs, and safety management documents to build a construction safety knowledge graph and knowledge base. Step 2: Build a BIM information library, including BIM model component code, component name, component size, project type, and component physical location information. The component code is the unique identifier of the BIM component and is used to search the building information library. Step 3: Build an augmented reality inspection scene based on BIM; Export the BIM model for AR scene construction, and bind the component code as the unique identification information to the BIM model component one by one through the component name or script form; At the same time, registration landmark setting, UI design, and knowledge base interactive query function script writing are carried out; Step 4: Retrieve safety knowledge based on BIM information. Retrieve relevant information from the BIM information database based on BIM component coding, perform knowledge graph retrieval based on project type and physical location, and perform intelligent query on relevant knowledge. Based on the pre-trained language model, obtain the semantic vector of the selected BIM component corresponding to the engineering type A 1. Semantic vector corresponding to entity parts A 2; Get all first-class node engineering types in the knowledge graph ,in I is the number of nodes in the first category, and its semantic vector is obtained using the same pre-trained language model B i , calculate them one by one according to the cosine similarity formula (1) A Semantic similarity of 1 , and sort them from small to large, and get the node with the largest similarity value greater than the set threshold ; ； (1) Traverse all nodes The second type of associated nodes ,in J is the number of nodes in the second category, and its semantic vector is obtained using the same pre-trained language model C j , calculate them one by one according to formula (1) A Semantic similarity of 2 , and sort them from small to large, and get the node with the largest similarity value greater than the set threshold ; Step 5: Based on the construction safety BIM+AR intelligent inspection system; On AR mobile devices, the BIM model is aligned with the actual application scenario to enter an AR scene that integrates virtual and real elements.

2. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized in that: In step 1, the construction safety knowledge graph consists of edges and nodes, where nodes are divided into three categories: the first type of node defines the project type Pro ; The second type of node definition corresponds to a certain entity part in the project Com ; The third type of node defines the associated security knowledge Kno .

3. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized by: In step 1, the specific steps for building the construction safety knowledge graph and knowledge base are as follows: In step 1-1, relevant construction safety specifications and safety management documents were collected and divided into paragraphs. To ensure effective entity and relationship extraction, each paragraph was limited to 512 characters. The entities and relationships were then annotated using the doccano annotation platform. Entities were categorized into project type and construction location, and relationships into containment and location. The annotated dataset was divided into a training set and a test set with an 8:2 ratio. In step 1-2, select the ERNIE-UIE general information extraction model for entity and relationship extraction, and fine-tune the model using the divided training set; the information extraction template schema is defined as: schema=['project type', 'construction location', {'project type': ['includes']}, {'physical location': ['includes', 'located at']}]; (2) Steps 1-3, knowledge graph and knowledge base construction, are constructed based on relevant safety specifications and management documents. First, the specifications are encoded according to chapters and items as the third type of entity in the knowledge graph; then, the model fine-tuned in steps 1-2 is used to extract entities and relationships for the project types and construction locations involved in each specification content, and organize them into triples.

4. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized by: In step 2, the BIM information database is constructed based on the analysis of IFC (Industry Fundamental Classes) files. The specific steps are as follows: Step 2-1: enrich the semantics of the IFC architecture and add the property set Pset_projectinformation. This property set contains two attributes: "Project Type" and "Construction Location", both in text format. Step 2-2: Supplement the engineering type and construction location parameters of each component in the BIM modeling software, and export the BIM information into an IFC format file based on the semantically rich IFC data structure; In steps 2-3, a Python program is written using the IfcOpenshell plug-in to extract the GUID, name, family, project type, and construction location information of each physical component in the BIM model to build a BIM information database. The GUID is the unique code of each BIM physical component in the IFC architecture and serves as the primary key of the BIM information database.

5. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized by: In step 4, the project type and construction location in the BIM information are filled in manually, and their language expressions may not be completely consistent with those in the standard safety management documents. A method based on semantic similarity comparison is used to perform knowledge graph retrieval.

6. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized by: In step 4, the BIM information library, knowledge graph, and knowledge base are used to implement security knowledge retrieval based on the BIM component GUID in the AR scene based on the semantic similarity between the BIM component's project type, construction location, and knowledge graph. The specific steps are as follows: Step 4-1: Select the pre-trained language model Chinese-RoBERTa-wwm-ext that has been fine-tuned based on text data in the construction field as the encoder, with a vector dimension of 768, and obtain the semantic vector corresponding to the project type returned by the selected BIM component information library. , semantic vector corresponding to the construction location ; Step 4-2: Select all the first-class node engineering types in the knowledge graph constructed in step 2 , use the same pre-trained language model to obtain its semantic vector According to the cosine similarity calculation formula in formula (3), the semantic similarity cos_sim( A 1, B i ), the closer the similarity is to 1, the higher the similarity is; Set the similarity threshold to 0.8, filter all nodes with similarity greater than 0.8 and sort them, and get the node with the highest similarity value greater than the set threshold ; ；(3) Step 4-3, extract all nodes with Directly related , or indirectly related to The second type of node , use the same pre-trained language model in step 2 to obtain its semantic vector , calculate them one by one with A 2 and sort them to get the node with the highest similarity value greater than the set threshold. ; Step 4-4: If there is no second-class node with a similarity value greater than the set threshold, select the first-class node with the second highest similarity value in step 2, and repeat steps 4-3 until the node is obtained. ; Step 4-5, traverse all nodes Directly associated third type of node , obtain the associated security knowledge item code; Step 4-6: In the construction safety knowledge base, the corresponding safety knowledge and knowledge source documents and chapter names are obtained based on the knowledge entry codes in step 4-5, thereby realizing the safety knowledge retrieval associated with BIM information.

7. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized by: In step 5, the BIM model components in the AR scene are named with component codes. By selecting the corresponding model component and clicking the knowledge base query button, the component code is automatically transferred to the database, and the relevant safety knowledge is returned and rendered in the AR interface, realizing safety knowledge-guided inspections.

8. The construction safety intelligent inspection method based on BIM+AR and knowledge base according to claim 1 is characterized by: In step 5, knowledge-enhanced intelligent inspection of construction safety is implemented. The AR inspection device selected is Hololens2, and the knowledge graph and knowledge base constructed in step 2 are used. The specific steps are as follows: Step 5-1: Publish and upload safety inspection tasks. The administrator logs in to the backend management system, selects the project to be inspected, creates a new inspection task, and uploads the inspection task list. The list should include the safety inspection location and inspection content, and is synchronized to the remote database. Step 5-2, task reception: Project personnel receive the task on the mobile terminal, retrieve the corresponding BIM model, scan the landmarks to achieve registration and 3D registration between the model and the real scene, and enter the virtual-reality fusion inspection scene; Step 5-3, knowledge-guided inspection; During the inspection process, when the inspector selects a component and clicks on the safety knowledge query, the mobile system will automatically obtain the name of the selected component, that is, the GUID, and send it to the BIM information database. The BIM information database uses the GUID to retrieve the project type and construction location corresponding to the component. The application server searches for the relevant safety knowledge according to step 3 and returns the safety knowledge to the AR inspection mobile terminal, rendering it on the AR interface, thereby guiding the management personnel to conduct inspections according to the requirements. At the same time, the management personnel can view the corresponding inspection task list in the AR scene and check them one by one. Step 5-4, inspection result feedback; During the inspection process, managers use the AR mobile terminal to record safety issues and safety hazard images. After the inspection, managers exit the AR mobile terminal, and the inspection task status is automatically updated to completed. It will be automatically uploaded to the inspection problem database and distributed to the relevant person in charge. The relevant person in charge will view and rectify the records through the background management system to achieve closed-loop safety management.

Images