Rail transit BIM model data question-answering query method, device and system based on large language model

Abstract

The present invention discloses a rail transit BIM model data question-answering query method based on a large language model, comprising: configuring and installing the large language model in a user network; extracting text data from the rail transit BIM model data and slicing text documents, slicing to generate text blocks, vectorizing the generated text blocks and storing them in a vector database; assigning a unique ID number to each text block generated by the slicing; binding resources associated with the semantic information of the text block to the unique ID number of the corresponding text block; vectorizing the unique ID number and synchronously storing it in the vector database; storing the resources bound to each unique ID number in a relational database; a user inputs a query question; the large model organizes the resources and returns them to the user; the query efficiency is improved, the operational difficulty is greatly reduced, the complex training of the multimodal large language model is avoided, and the huge investment in computing power and manpower is reduced.

Description

Technical Field

[0001] The present invention belongs to the fields of computer science and artificial intelligence, and specifically relates to a data query method and an implementation system thereof that combines a large language model with a three-dimensional building information model (BIM) for rail transit. Background Art

[0002] With the rapid development of urban rail transit, the complexity of its design, construction and operation management has also increased. BIM (Building Information Modeling) technology, as an emerging building information modeling technology, can provide rich three-dimensional geometric information and associated engineering data. Through the BIM model, various facility information of rail transit can be effectively managed and queried, such as construction and installation information during the construction period, asset information and operation and maintenance information of civil facilities such as stations and sections during the operation and maintenance period, and equipment systems. However, existing BIM applications usually require users to have professional operating skills, and the query process is often cumbersome, which is not conducive to use by non-professionals. These problems often affect the user experience and the efficiency of information retrieval.

[0003] 1) Inaccurate information retrieval

[0004] Traditional question-and-answer scenarios often rely on keyword matching to retrieve information, but this approach is ineffective when handling complex queries or semantic contexts. When a user's question involves complex semantics or requires a deep understanding of the document, keyword matching may fail to accurately understand the actual intent of the question, resulting in answers that do not meet the user's expectations. Furthermore, the presence of synonyms and polysemous words can lead to inaccurate retrieval results.

[0005] 2) Information update lags

[0006] Traditional question-and-answer scenarios often lack timely updates, especially in rapidly changing fields such as those involving frequent version updates between construction drawing design and construction detailing models. When users seek the latest information or data, traditional question-and-answer systems may only provide outdated answers, which not only reduces the value of the information but can also mislead users into making incorrect decisions. Furthermore, manually updating document repositories is a time-consuming and inefficient process, increasing maintenance costs.

[0007] 3) Poor user interaction experience

[0008] Traditional question-answering systems often lack effective interaction design, making it difficult for users to express complex queries and understand the system's responses. These systems often employ a static question-answering model, lacking the ability to understand conversation context and providing a coherent conversational experience. Furthermore, the system interfaces are often unfriendly, making it difficult for users to navigate and find the information they need. The lack of personalized answers and solutions is also a significant contributor to poor user experience, limiting the system's ability to meet the specific needs of individual users.

[0009] Secondly, traditional data management systems are often user-friendly, with complex user interfaces that make them difficult for non-experts to quickly master. Effective use of these systems requires specialized training, which undoubtedly increases both labor and time costs. Furthermore, due to the lack of intuitive interactive design, even trained users often face operational complexity and inefficiencies. For example, users may need to manually enter complex query commands or navigate through multi-level menus to find the desired function, which not only increases operational difficulty but also reduces work efficiency.

[0010] Due to these limitations, the application of traditional data management and question-answering retrieval systems in the modern rail transit industry is severely restricted. They are unable to fully mine and utilize collected data, nor can they provide timely and accurate data support to decision makers, thus hindering the intelligent development and efficient operation of the entire rail transit system. Therefore, a new solution is urgently needed to address these technical issues and meet the development needs of modern urban rail transit.

[0011] On the other hand, large language models have made significant progress in natural language processing in recent years and have become a new paradigm for research in this field. In vertical fields such as finance and law, intelligent question-answering systems based on open source large language models combined with local knowledge bases have been developed, promoting academic research and application of large language model technology in related fields. These models can be trained on large amounts of text data, understand complex query intent, and generate fluent and natural text responses. However, the lack of high-quality, relevant data in the rail transit sector makes the development of similar large language model question-answering systems face significant challenges. Summary of the Invention

[0012] The present invention aims to solve the problems of inaccurate information retrieval, delayed information update and poor user interaction experience in the prior art of BIM model data query.

[0013] The present invention adopts the following scheme: a rail transit BIM model data question-answering query method based on a large language model, comprising:

[0014] Step 1: Basic large language model deployment: Configure and install the large language model in the user network through interface calls or local deployment, allowing users to access and query the vector database configured by the large language model through query mode;

[0015] Step 2: Text data extraction and vectorization: Extract the text data from the rail transit BIM model data and slice the text document to generate text blocks. The generated text blocks are vectorized and stored in the vector database configured in step 1.

[0016] Step 3: Unique ID number association: Assign a unique ID number to each text block generated by slicing in step 2; based on the semantic information of each text block, the resources in the rail transit BIM model data that are associated with the semantic information of the text block, including pictures, documents, model links, and videos, are also bound to the unique ID number of the corresponding text block to establish an associated knowledge base.

[0017] Step 4: Vectorize the unique ID number: vectorize the unique ID number of each text block obtained in step 3 and store it synchronously in the vector database configured in step 1;

[0018] Step 5: Relational database storage: Store the resources, including images, documents, model links, and videos, associated with each unique ID number in step 3 in a relational database.

[0019] Step 6: User Interactive Query: The user enters a query. After the large language model matches the relevant text block, the relevant text block and the original query are constructed into a prompt word. This prompt word is then input into the large language model, which generates a response to the user's query. Simultaneously, the vector search engine retrieves the unique ID number corresponding to the text block. Based on the unique ID number, the relational database is searched to obtain the resources associated with the slice, which are then sorted and returned to the user.

[0020] Furthermore, the rail transit BIM model data in step 2 includes the original BIM data information, the mounted structured engineering information, and the mounted unstructured documents and drawing information.

[0021] Furthermore, the method for extracting text data from the rail transit BIM model data in step 2 includes: processing the unstructured document and drawing information together with the original BIM data information and the mounted structured engineering information into text through layout parsing (Parser) and OCR (text recognition) components, and processing them into text blocks through a word segmenter (Text Splitter).

[0022] Furthermore, the method for organizing the resources and returning them to the user in step 6 is: through the front-end interface, organize the sliced question and answer pairs and the retrieved resources, realize viewing the text answers, viewing the models, viewing the pictures, viewing the videos, and viewing the electronic document content through human-computer interaction operations, integrate each reply knowledge vector into an answer in combination with the current context, and return each knowledge vector linked to each reply knowledge vector and the answer to the user.

[0023] Furthermore, step 6 also includes synchronously storing historical conversations. Historical conversations contain records of interactions between the user and the system, which can be used to associate specific contextual information during embedding vector retrieval. By utilizing this historical conversation database, the system can better understand the user's query intent and provide more accurate and relevant feedback.

[0024] Furthermore, the large language model in step one is pre-fine-tuned to ensure that it can adapt to specific query and retrieval document input requirements, and is particularly optimized for BIM data question and answer scenarios; this fine-tuning process enables the large language model to better understand and process professional information related to rail transit.

[0025] Furthermore, the user inputs the query question in step six including text, voice and other input methods that can be converted into text.

[0026] As another aspect of the present invention, it also relates to a rail transit BIM model data question-answering query device based on a large language model, comprising:

[0027] The basic large language model deployment module is used to configure and install the large language model in the user network through interface calls or local deployment, allowing users to access and query the vector database configured by the large language model through queries;

[0028] Text data extraction and vectorization module: used to extract text data from rail transit BIM model data and slice text documents, generate text blocks through slicing, vectorize the generated text blocks and store them in the configured vector database;

[0029] Unique ID number association module: used to assign a unique ID number to each text block generated by slicing; based on the semantic information of each text block, the images, documents, model links, and video resources associated with the semantic information of the text block in the rail transit BIM model data are also bound to the unique ID number of the corresponding text block, thereby establishing an associated knowledge base;

[0030] Unique ID number vectorization module: used to vectorize the unique ID number of each text block and synchronously store it in the vector database configured in step 1;

[0031] Relational database storage module: used to store resources including pictures, documents, model links, and videos bound to each unique ID number in a relational database;

[0032] User interactive query module: This module is used to process user-entered queries. After the large language model matches the relevant text block, the vector search engine is used to retrieve the unique ID number corresponding to the text block. Based on the unique ID number, the module searches the relational database to obtain the resources associated with the slice, and then organizes the resources and returns them to the user.

[0033] As another aspect of the present invention, it also relates to a rail transit BIM model data question and answer query system based on a large language model, which includes the above-mentioned rail transit BIM model data question and answer query device based on a large language model.

[0034] As another aspect of the present invention, it also relates to a computer storage medium. When the computer program is executed by a processor, it implements the above-mentioned rail transit BIM model data question and answer query method based on a large language model.

[0035] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0036] (1) The large language model-based rail transit BIM model data question-answering query method of the present invention combines document vectorization and binds resources including pictures, documents, model links, and videos associated with the semantic information of text blocks in the rail transit BIM model data to the unique ID number of the sliced text block and stores them in a relational database, thereby avoiding the complex training of the multimodal large language model. Through the basic large language model, no training and fine-tuning are required (or only a small amount of training and fine-tuning is required), thereby reducing the huge investment in computing power and manpower.

[0037] (2) The large language model-based rail transit BIM model data question-answering query method of the present invention integrates a large language model with the rail transit BIM model and utilizes the powerful understanding capabilities of the large language model to more accurately capture the semantics and context of user queries, providing more intelligent and personalized search result sorting and display. This realizes an intuitive and efficient data query system that not only improves query efficiency but also greatly reduces operational difficulty, allowing non-professionals to easily obtain the required information. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of a preferred embodiment of the invention;

[0039] Figure 2 A schematic diagram of a device according to a preferred embodiment of the invention;

[0040] Figure 3 Schematic diagram of the principle of large language model vector retrieval in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0042] Please refer to Figure 1-Figure 2 The present invention relates to a rail transit BIM model data question-answering query method based on a large language model, comprising:

[0043] Step 1: Basic large language model deployment: Configure and install the large language model in the user network through interface calls or local deployment, allowing users to access and query the vector database configured by the large language model through query mode;

[0044] Step 2: Text data extraction and vectorization: Extract the text data from the rail transit BIM model data and slice the text documents. Slice to generate text blocks, vectorize the generated text blocks and store them in the vector database configured in step 1 (based on the text blocks obtained by the data processing module, vectorize (Embedding), create indexes and generate embedded vectors, create corresponding indexes, and vector data can be retrieved through the search engine. The search engine connects the obtained text blocks so that each text block can be quickly found and accessed through its unique vector identifier); the rail transit BIM model data includes the original BIM data information, the mounted structured engineering information, and the mounted unstructured documents and drawing information. For example, a model of electromechanical equipment, in addition to the model itself, also includes the corresponding certificate (picture), operation and maintenance manual (PDF electronic file), and on-site physical photo (picture) information. The method for extracting text data from rail transit BIM model data includes: through layout parsing (Parser) and OCR (text recognition) components, the unstructured document and drawing information together with the original BIM data information and the mounted structured engineering information are processed into text, and then processed into text blocks through the word segmenter (Text Splitter). Layout parsing (Parser), OCR (text recognition), and word segmenter (Text Splitter) are existing technologies related to large models, so they are not explained in detail.

[0045] Step 3: Unique ID number association: Assign a unique ID number to each text block generated by slicing in step 2; based on the semantic information of each text block, bind the resources in the rail transit BIM model data that are associated with the semantic information of the text block, including pictures, documents, model links, and videos, to the unique ID number of the corresponding text block to establish an associated knowledge base;

[0046] Step 4: Vectorize the unique ID number: vectorize the unique ID number of each text block obtained in step 3 and store it synchronously in the vector database configured in step 1;

[0047] Step 5: Relational database storage: Store the resources, including images, documents, model links, and videos, associated with each unique ID number in step 3 in a relational database.

[0048] Step 6: User Interaction Query: Please refer to Figure 3 , the user inputs a query question (including text, voice and other input methods that can be converted into text), and after the large language model hits the relevant text block, the relevant text block together with the original query question is constructed into a prompt word; this prompt word is then input into the large language model, and the large language model generates a reply to the user's query, and simultaneously retrieves the unique ID number corresponding to the text block according to the vector retrieval engine, and searches the relational database based on the unique ID number to obtain the slice-related resources, and then sorts the resources and returns them to the user. Among them, the method of returning the sorted resources to the user is: through the front-end interface, organize the slice question and answer pairs and the retrieved resources, and realize viewing the text answer, viewing the model, viewing the picture, viewing the video, and viewing the electronic document content through human-computer interaction operations, integrating each reply knowledge vector into an answer in combination with the current context, and returning each knowledge vector linked to each reply knowledge vector to the answer. In some embodiments, the generated reply includes not only the direct answer text, but also covers the knowledge vector links related to the answer. These knowledge vector links are associated with the detailed content in the system database, which can provide users with more in-depth information reference. Ultimately, all this information - including the answer

[0049] Text and knowledge vector links - are integrated and presented to users to meet their specific needs.

[0050] In order to effectively process this information, the system uses layout parsing (Parser) and optical character recognition (OCR) technology to convert unstructured document and drawing information into text format. The processed text data is further divided into multiple text blocks by a word segmenter (Text Splitter). In the data storage link, the present invention generates a unique identifier (ID) for each text block and associates these IDs with the storage links of pictures, drawings and the storage links of BIM models, thereby forming a complete information network in the knowledge base. This processing and storage method not only improves the queryability and operability of the data, but also enhances the depth and breadth of the knowledge base's coverage of information in the field of rail transit. In this way, the present invention can provide a highly integrated and interconnected information query platform, greatly improving the accuracy and efficiency of data retrieval, and providing powerful data support for professionals in the field of rail transit.

[0051] As a preferred solution, step 6 also includes synchronously storing historical conversations. Historical conversations contain records of interactions between the user and the system, which can be used to associate specific contextual information during embedding vector retrieval. By utilizing this historical conversation database, the system can better understand the user's query intent and provide more accurate and relevant feedback.

[0052] In some embodiments, the large language model in step one is pre-fine-tuned to ensure that it can adapt to specific query and retrieval document input requirements, and is particularly optimized for BIM data question and answer scenarios; this fine-tuning process enables the large language model to better understand and process professional information related to rail transit.

[0053] Please refer to Figure 2 As another aspect of the present invention, it also relates to a rail transit BIM model data question-answering query device based on a large language model, comprising:

[0054] The basic large language model deployment module is used to configure and install the large language model in the user network through interface calls or local deployment, allowing users to access and query the vector database configured by the large language model through queries;

[0055] Text data extraction and vectorization module: used to extract text data from rail transit BIM model data and slice text documents, generate text blocks through slicing, vectorize the generated text blocks and store them in the configured vector database;

[0056] Unique ID number association module: used to assign a unique ID number to each text block generated by slicing; based on the semantic information of each text block, the images, documents, model links, and video resources associated with the semantic information of the text block in the rail transit BIM model data are also bound to the unique ID number of the corresponding text block, thereby establishing an associated knowledge base;

[0057] Unique ID number vectorization module: used to vectorize the unique ID number of each text block and synchronously store it in the vector database configured in step 1;

[0058] Relational database storage module: used to store resources including pictures, documents, model links, and videos bound to each unique ID number in a relational database;

[0059] User interactive query module: This module is used to process user-entered queries. After the large language model matches the relevant text block, the vector search engine is used to retrieve the unique ID number corresponding to the text block. Based on the unique ID number, the module searches the relational database to obtain the resources associated with the slice, and then organizes the resources and returns them to the user.

[0060] As another aspect of the present invention, it also relates to a rail transit BIM model data question and answer query system based on a large language model, which includes the above-mentioned rail transit BIM model data question and answer query device based on a large language model.

[0061] As another aspect of the present invention, it also relates to a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the rail transit BIM model data question and answer query method based on a large language model as described in any one of claims 1 to 7 is implemented.

[0062] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A rail transit BIM model data question-answering query method based on a large language model, characterized in that: include: Step 1: Basic large language model deployment: Configure and install the large language model in the user network through interface calls or local deployment, allowing users to access and query the vector database configured by the large language model through queries; Step 2: Text data extraction and vectorization: Extract the text data from the rail transit BIM model data and slice the text document to generate text blocks. The generated text blocks are vectorized and stored in the vector database configured in step 1. Step 3: Unique ID number association: Assign a unique ID number to each text block generated by slicing in step 2; based on the semantic information of each text block, bind the resources in the rail transit BIM model data that are associated with the semantic information of the text block, including pictures, documents, model links, and videos, to the unique ID number of the corresponding text block to establish an associated knowledge base; Step 4: Vectorize the unique ID number: vectorize the unique ID number of each text block obtained in step 3 and store it synchronously in the vector database configured in step 1; Step 5: Relational database storage: Store the resources, including images, documents, model links, and videos, associated with each unique ID number in step 3 in a relational database. Step 6: User interactive query: The user enters a query question. After the large language model matches the relevant text block, the relevant text block and the original query are constructed into a prompt word. This prompt is then fed into a large language model, which generates a response to the user's query. Simultaneously, the vector search engine retrieves the unique ID number corresponding to the text block. Based on the unique ID number, the relational database is searched to obtain the resources associated with the slice, which are then collated and returned to the user. The generated response includes not only the direct answer text but also links to knowledge vectors related to the answer. The rail transit BIM model data in step 2 includes the original BIM data information, the mounted structured engineering information, and the mounted unstructured documents and drawing information.

2. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized by: The method for extracting text data from the rail transit BIM model data in step 2 includes: processing the unstructured document and drawing information together with the original BIM data information and the mounted structured engineering information into text through layout parsing and OCR components, and processing them into text blocks through a word segmenter.

3. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized by: The method for organizing the resources and returning them to the user in step six is: through the front-end interface, organize and slice the question and answer pairs and the retrieved resources, realize viewing text answers, viewing models, viewing pictures, viewing videos, and viewing electronic document content through human-computer interaction operations, integrate each reply knowledge vector into an answer in combination with the current context, and return each knowledge vector linked to each reply knowledge vector and the answer to the user.

4. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized by: The step 6 also includes synchronously storing historical conversations; historical conversations include interaction records between users and the system, described Records can be used to associate specific contextual information during embedding vector retrieval.

5. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized by: In step 6, the user inputs the query question in the form of text, voice, or other input methods that can be converted into text.

6. A rail transit BIM model data question-answering query device based on a large language model, characterized in that: include: The basic large language model deployment module configures and installs the large language model in the user network through interface calls or local deployment, allowing users to access and query the vector database configured by the large language model through queries; Text data extraction and vectorization module: extracts text data from rail transit BIM model data and slices text documents, generates text blocks through slicing, vectorizes the generated text blocks and stores them in the vector database configured in step 1; Rail transit BIM model data includes original BIM data information, mounted structured engineering information, and mounted unstructured documents and drawing information; Unique ID number association module: assigns a unique ID number to each text block generated by slicing; based on the semantic information of each text block, resources in the rail transit BIM model data that are associated with the semantic information of the text block, including pictures, documents, model links, and videos, are also bound to the unique ID number of the corresponding text block to establish an associated knowledge base; Unique ID number vectorization module: used to vectorize the unique ID number of each text block and store it synchronously in the configured vector database; Relational database storage module: used to store resources including pictures, documents, model links, and videos bound to each unique ID number in a relational database; User Interaction Query Module: The user enters a query question. After the large language model matches the relevant text block, the relevant text block and the original query question are constructed into a prompt word. This prompt word is then input into the large language model, which generates a response to the user's query. Simultaneously, the vector search engine retrieves the unique ID number corresponding to the text block. Based on the unique ID number, the relational database is searched to obtain the resources associated with the slice, and the resources are sorted and returned to the user. The generated response includes not only the direct answer text, but also covers the knowledge vector links related to the answer.

7. A rail transit BIM model data question-answering query system based on a large language model, characterized by: The system includes the rail transit BIM model data question-answering query device based on a large language model as described in claim 6.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the rail transit BIM model data question-answering query method based on a large language model as described in any one of claims 1 to 5 is implemented.

Images