Interaction system and interaction method of BIM model

Abstract

The invention discloses an interaction system and an interaction method of a BIM (Building Information Modeling), the system architecture of the interaction system comprises a user layer, a business layer, a capability layer and an infrastructure layer, the user layer comprises a Web end, a mobile APP and a PC (Personal Computer) client, and is used for providing an interaction interface for a user; the business layer comprises a model uploading module, a model conversion module, a model display module and a model operation module; the capability layer comprises a BIM lightweight engine and an AI intelligent agent; the infrastructure layer comprises server hardware, a server operating system and a database. The interaction method is operated in an interaction system of the BIM model. According to the invention, the problems of tedious BIM model interaction operation and low efficiency are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of interactive systems, and in particular to an interactive system and an interactive method of a BIM model. Background Art

[0002] Building information modeling (BIM) is a three-dimensional digital expression of the 3D geometry and functional attributes of a building, which can support full life cycle decision-making of engineering construction projects.

[0003] The application of building information modeling in the engineering construction industry involves two links: model creation and model application. Since the scale of engineering projects is generally large, a large number of interactive and editing operations are required for BIM model construction during the modeling and application process, such as selection, moving, copying, deleting, adding attributes, etc.

[0004] The existing interactive operation is to operate on the computer screen with a mouse or on the touch screen with a finger. It is necessary to directly select the BIM model component, or click a series of software menus, create a filter, filter the BIM model by a certain attribute, and then perform the desired operation. However, this method is cumbersome and inefficient. The BIM model of modern engineering construction projects often contains millions of graphics elements to operate. This method is very inefficient and greatly restricts the application value of the BIM model, especially for situations where some graphics elements of the model need to be finely interacted. The defects of this method are more prominent. Summary of the invention

[0005] 1. Technical issues to be resolved Based on the above problems, the present invention provides an interactive system and an interactive method for a BIM model to solve the problem that the interactive operation of the BIM model is cumbersome and inefficient.

[0006] (II) Technical solution Based on the above technical problems, the present invention provides an interactive system of a BIM model, wherein the system architecture includes a user layer, a business layer, a capability layer and an infrastructure layer; The user layer includes a web client, a mobile APP and a PC client. The user layer is used to provide an interactive interface for users, allowing users to upload BIM models, convert models, display models, input model operation instructions and display functions through web browsers, mobile APPs and PC software; The business layer includes a model upload module, a model conversion module, a model display module and a model operation module; the model upload module supports users to upload local BIM model files from the device used and transmit them to the server through the network; the model conversion module is used to convert the original format BIM files uploaded by users into lightweight format files; the model display module is used to parse and render the BIM lightweight data converted by the server BIM lightweight engine; the model operation module is used to input BIM model interaction instructions to users; The capability layer includes a BIM lightweight engine and an AI intelligent agent; the BIM lightweight engine is used to perform lightweight conversion on the BIM model uploaded by the user to the server, and transmit the converted data to the user layer, and render it on a web page or APP for display to the user; the AI ​​intelligent agent is used to process the interactive operation instructions input by the user, and the functions of the AI ​​intelligent agent include voice-to-text, text instruction splitting and text instruction-to-code conversion. The text instruction splitting is used to split a complex instruction containing multiple instructions into multiple simple instructions, and a simple instruction only contains an operation name and an operation object; The infrastructure layer includes server hardware, server operating system and database, and is used to provide computing and storage resources for the operation of the capability layer.

[0007] Furthermore, the BIM lightweight engine is developed based on WebGL and deployed on a server. The model conversion module and the model display module of the business layer need to call the BIM lightweight engine through the network at the user layer.

[0008] Furthermore, the AI ​​intelligent agent is deployed on a server. The AI ​​intelligent agent takes a large language model as its core, has the capabilities of task decomposition, multi-solution selection, external module-assisted planning, reflection and optimization, and memory-enhanced planning, and has the functions of language understanding and generation, context memory, tool calling, and task execution; the large language model is a deep learning model trained using text data, which can understand the meaning of language text or generate natural language text.

[0009] Furthermore, the AI ​​intelligent agent can build a BIM interactive intelligent agent based on the BIM model, including: The general open source multimodal large language model GLM-4-9B was selected. After the model training was completed, the post-training quantization technology ZeroQuant-FP was used to quantize the model to reduce the scale of model parameters and obtain the quantized basic model. Parse the BIM lightweight engine API document, create a command fine-tuning dataset, use the open source fine-tuning framework LLaMA-Factory, and use the AdaloRA method to fine-tune the basic model with the command fine-tuning dataset to create a BIM large model; The instruction fine-tuning dataset is saved to the Chroma vector database as a retrieval enhancement knowledge plug-in for knowledge update of the BIM large model; Use the langchain framework to call the API of the big language model through Open API to build a BIM interactive intelligent agent based on the BIM big model.

[0010] Furthermore, the infrastructure layer includes physical hardware and cloud-based virtual hardware.

[0011] The present invention also discloses a BIM model interaction method, which runs on the BIM model interaction system and includes: S1. Enable manual command input function; S2, issue manual instructions; S3. For instructions that only contain one task, directly proceed to the next step S4; for instructions that contain multiple tasks, use the AI ​​intelligent agent to perform the first instruction splitting, split the instructions containing multiple tasks into multiple tasks, and perform task planning; S4. Perform instruction conversion: First, the AI ​​intelligent agent performs a second instruction splitting on the manual instruction, into interactive operations and operation objects that need to be performed by the BIM lightweight engine; then the AI ​​intelligent agent retrieves the relevant object and event codes from the BIM lightweight engine API library to generate a BIM lightweight engine API instruction; S5. Interaction operation: The AI ​​intelligent agent sends the converted BIM lightweight engine API instruction to the BIM lightweight engine, and drives the BIM lightweight engine to perform the interaction operation required in the prompt on the BIM elements involved in the manual instruction through the BIM lightweight engine API interface; S6. If the system operation meets the requirements, go to the next step, otherwise adjust the prompt and return to S2; S7. Turn off the manual command input function.

[0012] Furthermore, the manual instructions include voice instructions or text instructions. If they are voice instructions, the BIM interactive intelligent agent is used to convert them into text prompts.

[0013] Furthermore, in S3 and S4, the instruction splitting method of the AI ​​intelligent agent is: first use the text embedding algorithm to vectorize the text so that semantically similar text fragments have close vector representations; then divide the vectorized document into small blocks, store them in the vector database, and then segment them according to Token and combined with chunk_size and chunk_overlap.

[0014] Furthermore, in S4, the AI ​​intelligent agent retrieves relevant object and event codes from the BIM lightweight engine API library to generate BIM lightweight engine API instructions, including: the AI ​​intelligent agent processes through retrieval enhancement generation technology, stores manual instructions, BIM interactive operation instructions, and BIM lightweight engine API instructions into a vector database, performs similarity retrieval, generates corresponding code instructions, and realizes the conversion of manual instructions into BIM lightweight engine API instructions.

[0015] (III) Beneficial effects The above technical solution of the present invention has the following advantages: (1) The present invention uses prompts such as text and voice to enable quick interaction with the BIM model without manual intervention, thereby improving the efficiency of BIM model interaction; (2) The present invention can include multiple tasks in one manual prompt, which are automatically split, converted and operated by the system without manual intervention. The operator can do other work during this period, realizing multiple tasks in parallel, improving the degree of automation, and greatly improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings: Figure 1 A schematic diagram of the structure of an interactive system of a BIM model according to an embodiment of the present invention; Figure 2 A flowchart of constructing a BIM interactive intelligent agent according to an embodiment of the present invention; Figure 3 A core schematic diagram of an interactive system of a BIM model according to an embodiment of the present invention; Figure 4 A flowchart of a BIM model interaction method according to an embodiment of the present invention; Figure 5 Schematic diagram of instruction splitting and conversion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0018] The embodiment of the present invention is an interactive system of a BIM model, and the system architecture includes a user layer, a business layer, a capability layer and an infrastructure layer, such as Figure 1 shown.

[0019] The user layer includes the Web client, mobile APP and PC client. The user layer is used to provide users with an interactive interface, allowing users to upload BIM models, convert models, display models, input model operation instructions and display functions through web browsers, mobile APPs and PC software. Users only need to install the corresponding software on their mobile phones or PCs to use the above business functions. The Web client is developed using web development languages ​​such as HTML, CSS and JavaScript. The mobile APP and PC clients are developed using the uni-app framework.

[0020] The business layer includes functional modules such as model upload module, model conversion module, model display module and model operation module. The model upload module supports users to upload local BIM model files from the device they are using and transmit them to the server through the network. The network used can be a wired network or a wireless network. The model conversion module is used to convert the original format BIM files uploaded by users, such as RVT format files, into lightweight format files, such as B3D format, so that users can browse them at the user layer. The model display module is used to parse and render the BIM lightweight data converted by the server BIM lightweight engine. The model operation module is used to input BIM model interaction instructions to users. The instructions issued by users can be voice instructions or text instructions.

[0021] The capability layer includes BIM lightweight engine and AI intelligent agent. BIM lightweight engine, such as China Construction BIM, is used to perform lightweight conversion on BIM models uploaded by users to the server, and transmit the converted data to the user layer, and render it on the web page or APP for display to users. Model conversion and model display at the business layer require calling BIM lightweight engine through the network at the user layer. BIM lightweight engine is developed based on WebGL and deployed on the server. AI intelligent agent is used to process interactive operation instructions input by users and is also deployed on the server. The functions of AI intelligent agent include voice to text, text instruction splitting and text instruction to code conversion. Text instruction splitting is used to split complex instructions containing multiple instructions into multiple simple instructions. Simple instructions only contain operation names and operation objects. AI intelligent agent takes large language model as the core, and has basic capabilities such as task decomposition, multi-scheme selection, external module assisted planning, reflection and optimization, memory enhancement planning, etc. It has functions such as language understanding and generation, context memory, tool calling and task execution.

[0022] The AI ​​intelligent agent can build a BIM interactive intelligent agent based on the BIM model. Specifically, (1) a general open source multimodal large language model, such as GLM-4-9B, is selected as the basic model. Since the basic model has too many parameters and the model is large, it consumes a lot of computer resources. After the model training is completed, the post-training quantization technology ZeroQuant-FP is used to quantize the model to reduce the scale of model parameters and obtain the quantized basic model; (2) the BIM lightweight engine API document is then parsed to create an instruction fine-tuning dataset; the open source fine-tuning framework LLaMA-Factory is used to fine-tune the model through Adal oRA method, use the instruction fine-tuning dataset to fine-tune the basic model, create a BIM big model, and solve the basic model's ability to understand the BIM model and lightweight engine code; the big language model refers to a deep learning model trained with a large amount of text data, which can understand the meaning of language text or generate natural language text. The big language model can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc.; (3) Save the instruction fine-tuning dataset to the Chroma vector database as a retrieval enhancement knowledge plug-in for knowledge update of the BIM big model; (4) Use the langchain framework to call the API of the GLM-4-9B model through the Open API to build a BIM interactive intelligent agent based on the BIM big model. The process from the open source big model to the BIM interactive intelligent agent is shown in the attached Figure 2 .

[0023] The infrastructure layer includes server hardware, server operating systems, databases and other components. The infrastructure layer is used to provide computing and storage resources for the operation of the capability layer. It can be in the form of physical hardware or cloud-based virtual hardware.

[0024] The core components of the building information model interactive system include users, intelligent agents and BIM lightweight engine. The core process includes uploading BIM models to the BIM lightweight engine and interacting with the model through intelligent agents. Figure 3 , the user sends interactive instructions to the AI ​​intelligent agent, the AI ​​intelligent agent sends the converted instruction code to the BIM lightweight engine, and the BIM lightweight engine returns the operation result to the user based on the BIM model and instruction code provided by the user.

[0025] Therefore, the BIM model interaction method of the embodiment of the present invention is as follows Figure 4 As shown, including: S1. Enable manual command input function: Click the system user interface with the mouse or speak a keyword to trigger the system to enter the manual input state to avoid accidental system operation. Keywords can be customized by the user.

[0026] S2. Issue manual instructions: The system operator issues voice or text instructions through the input device in the system. If it is a voice instruction, the BIM interactive intelligent agent is used to convert it into a text prompt. The instruction can contain one or more interactive tasks.

[0027] S3. For instructions that only contain one task, go directly to the next step S4; for instructions that contain multiple tasks, use AI intelligent agents to perform the first instruction splitting (instruction splitting 1), split the instructions into multiple tasks, and perform task planning. The splitting method is to first use the text embedding algorithm to vectorize the text so that semantically similar text fragments have similar vector representations; secondly, the vectorized document is divided into small blocks, stored in the vector database, and then split according to Token and combined with chunk_size (block size) and chunk_overlap (the overlap size between blocks).

[0028] S4, perform command conversion; first, the AI ​​intelligent agent splits the manual command for the second time (command splitting 2), into interactive operations and operation objects that need to be performed by the BIM lightweight engine, and the splitting method is similar to S3; then the AI ​​intelligent agent retrieves the relevant object and event code from the BIM lightweight engine API library, and generates the BIM lightweight engine API command, see attached Figure 5 ,Split the manual instructions (e.g. delete the 1st floor structural column) into interactive operations (e.g. delete) and operation objects (e.g. ,) and filter out the corresponding API instructions in the BIM elements.

[0029] Specifically, the AI ​​intelligent agent is processed through RAG (Retrieval-Augmented Generation) technology. By storing manual instructions, BIM interactive operation instructions, and BIM lightweight engine API instructions in a vector database, similarity retrieval is performed, and corresponding code instructions are generated to realize the conversion of manual instructions into BIM lightweight engine API instructions (code).

[0030] S5. Interactive operation: The AI ​​intelligent agent sends the converted BIM lightweight engine API (Application Programming Interface) instructions to the BIM lightweight engine, and drives the BIM lightweight engine through the BIM lightweight engine API interface to perform the interactive operations required in the prompts on the BIM elements involved in the manual instructions.

[0031] S6. If the system operation meets the requirements, go to the next step, otherwise adjust the prompt and return to S2.

[0032] S7. Turn off the manual command input function: After the operation is completed, click the system button with the mouse or say a keyword, such as "end task", to trigger the system to turn off the manual input state to avoid accidental system operation.

[0033] In summary, the above-mentioned BIM model interaction system and interaction method have the following beneficial effects: (1) The present invention uses prompts such as text and voice to enable quick interaction with the BIM model without manual intervention, thereby improving the efficiency of BIM model interaction; (2) The present invention can include multiple tasks in one manual prompt, which are automatically split, converted and operated by the system without manual intervention. The operator can do other work during this period, realizing multiple tasks in parallel, improving the degree of automation, and greatly improving work efficiency.

[0034] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall fall within the scope defined by the appended claims.

Claims

1. An interactive system for a BIM model, characterized in that: The system architecture includes user layer, business layer, capability layer and infrastructure layer; The user layer includes a web client, a mobile APP and a PC client. The user layer is used to provide an interactive interface for users, allowing users to upload BIM models, convert models, display models, input model operation instructions and display functions through web browsers, mobile APPs and PC software; The business layer includes a model upload module, a model conversion module, a model display module and a model operation module; the model upload module supports users to upload local BIM model files from the device used and transmit them to the server through the network; the model conversion module is used to convert the original format BIM files uploaded by users into lightweight format files; the model display module is used to parse and render the BIM lightweight data converted by the server BIM lightweight engine; the model operation module is used to input BIM model interaction instructions to users; The capability layer includes a BIM lightweight engine and an AI intelligent agent; the BIM lightweight engine is used to perform lightweight conversion on the BIM model uploaded by the user to the server, and transmit the converted data to the user layer, and render it on a web page or APP for display to the user; the AI ​​intelligent agent is used to process the interactive operation instructions input by the user, and the functions of the AI ​​intelligent agent include voice-to-text, text instruction splitting and text instruction-to-code conversion. The text instruction splitting is used to split a complex instruction containing multiple instructions into multiple simple instructions, and a simple instruction only contains an operation name and an operation object; The infrastructure layer includes server hardware, server operating system and database, and is used to provide computing and storage resources for the operation of the capability layer.

2. The BIM model interactive system according to claim 1, characterized in that: The BIM lightweight engine is developed based on WebGL and deployed on a server. The model conversion module and the model display module of the business layer need to call the BIM lightweight engine through the network at the user layer.

3. The interactive system of BIM model according to claim 1, characterized in that: The AI ​​intelligent agent is deployed on a server. The AI ​​intelligent agent takes a large language model as its core, has the capabilities of task decomposition, multi-solution selection, external module-assisted planning, reflection and optimization, and memory-enhanced planning, and has the functions of language understanding and generation, context memory, tool calling, and task execution; the large language model is a deep learning model trained using text data, which can understand the meaning of language text or generate natural language text.

4. The BIM model interactive system according to claim 1, characterized in that: The AI ​​intelligent agent can build a BIM interactive intelligent agent based on the BIM model, including: The general open source multimodal large language model GLM-4-9B was selected. After the model training was completed, the post-training quantization technology ZeroQuant-FP was used to quantize the model to reduce the scale of model parameters and obtain the quantized basic model. Parse the BIM lightweight engine API document, create a command fine-tuning dataset, use the open source fine-tuning framework LLaMA-Factory, and use the AdaloRA method to fine-tune the basic model with the command fine-tuning dataset to create a BIM large model; The instruction fine-tuning dataset is saved to the Chroma vector database as a retrieval enhancement knowledge plug-in for knowledge update of the BIM large model; Use the langchain framework to call the API of the big language model through Open API to build a BIM interactive intelligent agent based on the BIM big model.

5. The BIM model interactive system according to claim 1, characterized in that: The infrastructure layer includes physical hardware and cloud-based virtual hardware.

6. A BIM model interaction method, characterized in that: An interactive system running on the BIM model according to any one of claims 1 to 5, comprising: S1. Enable manual command input function; S2, issue manual instructions; S3. For instructions that only contain one task, directly proceed to the next step S4; for instructions that contain multiple tasks, use the AI ​​intelligent agent to perform the first instruction splitting, split the instructions containing multiple tasks into multiple tasks, and perform task planning; S4. Perform instruction conversion: First, the AI ​​intelligent agent performs a second instruction splitting on the manual instruction, into interactive operations and operation objects that need to be performed by the BIM lightweight engine; then the AI ​​intelligent agent retrieves the relevant object and event codes from the BIM lightweight engine API library to generate a BIM lightweight engine API instruction; S5. Interaction operation: The AI ​​intelligent agent sends the converted BIM lightweight engine API instruction to the BIM lightweight engine, and drives the BIM lightweight engine to perform the interaction operation required in the prompt on the BIM elements involved in the manual instruction through the BIM lightweight engine API interface; S6. If the system operation meets the requirements, go to the next step, otherwise adjust the prompt and return to S2; S7. Turn off the manual command input function.

7. The BIM model interaction method according to claim 6, characterized in that: In S2, the manual instructions include voice instructions or text instructions. If they are voice instructions, the BIM interactive intelligent agent is used to convert them into text prompts.

8. The BIM model interaction method according to claim 6, characterized in that: In S3 and S4, the instruction splitting method of the AI ​​intelligent agent is: first use the text embedding algorithm to vectorize the text so that semantically similar text fragments have close vector representations; then divide the vectorized document into small blocks, store them in the vector database, and then segment them according to Token and combined with chunk_size and chunk_overlap.

9. The BIM model interaction method according to claim 6, characterized in that: In S4, the AI ​​intelligent agent retrieves relevant object and event codes from the BIM lightweight engine API library to generate BIM lightweight engine API instructions, including: the AI ​​intelligent agent processes through retrieval enhancement generation technology, stores manual instructions, BIM interactive operation instructions, and BIM lightweight engine API instructions into a vector database, performs similarity retrieval, generates corresponding code instructions, and realizes the conversion of manual instructions into BIM lightweight engine API instructions.

Images