Rail transit BIM model data question and answer query method, device and system based on large language model

Through the combination of large language model and vector database, the problem of inaccurate information retrieval and poor user interaction experience in BIM model data query is solved, efficient and accurate data query is achieved, and complex needs in the field of rail transit is adapted to improve the The system is intelligent and user experience.

CN118916462AActive Publication Date: 2024-11-08BEIJING URBAN CONSTRUCTION DESIGN & DEVELOPMENT GROUP CO LIMITED

Patent Information

Application Number
CN202411000055.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-08
Estimated Expiration
2044-07-24

AI Technical Summary

Technical Problem

The existing BIM model data query system has problems such as inaccurate information retrieval, lagging information updates and poor user interaction experience. Especially in the fields of complex semantic queries and rapid changes, it is difficult to provide accurate and timely data support, which affects the rail transit system. Intelligent construction and efficient operation.

Method used

Using a large language model to combine vector databases and relational databases, establish an association knowledge base through text data extraction, vectorization and unique ID number association, realize user interaction query and resource retrieval, use a large language model to generate replies and organize answers, and combine history The dialogue library improves the accuracy and personalization of queries.

Benefits of technology

It improves query efficiency and accuracy, reduces operational difficulty, and allows non-professionals to easily obtain the required information, providing an intuitive and efficient data query system to meet the needs of modern rail transit fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916462A_ABST
    Figure CN118916462A_ABST
Patent Text Reader

Abstract

The invention discloses a rail transit BIM model data question and answer query method based on a large language model. The method comprises the steps that the large language model is configured and installed in a user network; text data in the rail transit BIM model data are extracted, text document slicing is carried out, text blocks are generated through slicing, and the generated text blocks are vectorized and then stored in a vector database; endowing each text block generated by slicing with a unique ID number; binding the resource associated with the semantic information of the text block to the unique ID number of the corresponding text block; vectorizing the unique ID number and then synchronously storing the unique ID number in a vector database; storing the bound resource corresponding to each unique ID number in a relational database; a user inputs a query question; the large model organizes the resources and returns the organized resources to the user; according to the method, the query efficiency is improved, the operation difficulty is greatly reduced, complex training of a multi-modal large language model is avoided, and huge investment of computing power and manpower is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer science and artificial intelligence, and specifically relates to a data query method and an implementation system thereof that combines a large language model with a three-dimensional building information model (Building Information Modeling, BIM) of rail transit. Background Art

[0002] With the rapid development of urban rail transit, the complexity of its design, construction and operation management has also increased. As an emerging building information modeling technology, BIM (Building Information Modeling) technology can provide rich three-dimensional geometric information and related engineering data. Through the BIM model, various facility information of rail transit can be effectively managed and queried, such as construction and installation information during the construction period, civil facilities such as stations and sections during the operation and maintenance period, asset information and operation and maintenance information of equipment systems, etc. However, existing BIM applications usually require users to have professional operating skills, and the query process is often cumbersome, which is not conducive to the use of non-professionals. These problems often affect the user experience and the efficiency of information retrieval.

[0003] 1) Information retrieval is not accurate

[0004] Traditional question-answering scenarios often rely on keyword matching technology to retrieve information, which does not work well when dealing with complex queries or semantic contexts. When a user's question involves complex semantics or requires a deep understanding of the document, keyword matching may not accurately understand the actual intent of the question, resulting in the returned answer not matching the user's expectations. In addition, the presence of synonyms and polysemous words may also lead to inaccurate retrieval results.

[0005] 2) Information update lags

[0006] Traditional question-and-answer scenarios are often not updated in a timely manner, especially in rapidly changing fields, such as frequent updates to construction drawing design stage models and construction deepening models. When users seek the latest information or data, traditional question-and-answer scenario systems may only provide outdated answers, which not only reduces the value of the information, but may also mislead users to make wrong decisions. In addition, manually updating the document library is a time-consuming and inefficient process, which increases maintenance costs.

[0007] 3) Poor user interaction experience

[0008] Traditional question-answering systems often lack effective interaction design, making it difficult for users to express complex queries or understand the system's answers. These systems often adopt a static question-answering model, lack the ability to understand the conversation context, and cannot provide a coherent conversation experience. In addition, the system interface is often not user-friendly, making it difficult for users to navigate or find the information they need. The lack of personalized answers and solutions is also an important reason for poor user experience, which limits the system's ability to meet the specific needs of individual users.

[0009] Secondly, traditional data management systems are usually not user-friendly, and the operation interface is often complicated, making it difficult for non-professionals to quickly get started. Users must undergo special training to use these systems effectively, which undoubtedly increases labor costs and time costs. At the same time, due to the lack of intuitive interactive design, even trained users often face problems of complex operations and low efficiency during use. For example, users may need to manually enter complex query instructions or search for the required functions through multi-level menus, which not only increases the difficulty of operation but also reduces work efficiency.

[0010] Due to these limitations, the application of traditional data management and question-answering retrieval systems in the modern rail transit industry is severely restricted. They cannot fully mine and utilize the collected data, and it is difficult to provide timely and accurate data support to decision makers, thus affecting the intelligent construction and efficient operation of the entire rail transit system. Therefore, a new solution is urgently needed to solve these technical problems in order to meet the development needs of modern urban rail transit.

[0011] On the other hand, large language models have made significant progress in the field of natural language processing in recent years and have become a new paradigm for research in this field. In vertical fields such as finance and law, intelligent question-answering systems based on open source large model bases combined with local knowledge bases have been implemented for large models in vertical fields, promoting academic research and application of large model technology in related fields. Such models can learn large amounts of text data through training, understand complex query intent, and generate fluent and natural text responses. However, due to the lack of high-quality relevant data in the rail transit field, the development of similar large model question-answering systems faces significant challenges. Summary of the invention

[0012] The present invention aims to solve the problems of inaccurate information retrieval, delayed information update and poor user interaction experience in BIM model data query in the prior art.

[0013] The present invention adopts the following scheme: a rail transit BIM model data question-answering query method based on a large language model, comprising:

[0014] Step 1: Basic large language model deployment: Configure and install the large language model in the user network through interface calls or local deployment, so that users can access and query the vector database configured by the large language model through query mode;

[0015] Step 2: Text data extraction and vectorization: Extract the text data in the rail transit BIM model data and slice the text document, slice to generate text blocks, vectorize the generated text blocks and store them in the vector database configured in step 1;

[0016] Step 3: Unique ID number association: Assign a unique ID number to each text block generated by slicing in step 2; based on the semantic information of each text block, the resources in the rail transit BIM model data that are associated with the semantic information of the text block, including pictures, documents, model links, and videos, are also bound to the unique ID number of the corresponding text block to establish an associated knowledge base.

[0017] Step 4: Vectorize the unique ID number: vectorize the unique ID number of each text block obtained in step 3 and store it synchronously in the vector database configured in step 1;

[0018] Step 5: Relational database storage: Store the resources including pictures, documents, model links, and videos bound to each unique ID number in step 3 in a relational database;

[0019] Step 6: User interactive query: The user inputs a query question. After the large language model hits the relevant text block, the relevant text block together with the original query question is constructed into a prompt word. This prompt word is then input into the large language model, which generates a response to the user's query. At the same time, the vector search engine retrieves the unique ID number corresponding to the text block, searches the relational database based on the unique ID number, obtains the resources associated with the slice, and returns the resources to the user after sorting.

[0020] Furthermore, the rail transit BIM model data in step 2 includes the original BIM data information, the mounted structured engineering information, and the mounted unstructured documents and drawing information.

[0021] Furthermore, the method for extracting text data from the rail transit BIM model data in the step 2 includes: processing the unstructured document and drawing information together with the original BIM data information and the mounted structured engineering information into text through layout parsing (Parser) and OCR (text recognition) components, and processing them into text blocks through a word segmenter (Text Splitter).

[0022] Furthermore, the method for organizing the resources and returning them to the user in step 6 is: through the front-end interface, organize the sliced ​​question and answer pairs and the retrieved resources, realize viewing the text answers, viewing the models, viewing the pictures, viewing the videos, and viewing the electronic document content through human-computer interaction operations, integrate each reply knowledge vector into an answer in combination with the current context, and return each knowledge vector linked to each reply knowledge vector and the answer to the user.

[0023] Furthermore, step 6 also includes synchronously storing historical conversations. Historical conversations contain interaction records between the user and the system, which can be used to associate specific context information during the embedding vector retrieval process. By utilizing the historical conversation library, the system can better understand the user's query intent and provide more accurate and relevant information feedback.

[0024] Furthermore, the large language model in step one is pre-fine-tuned to ensure that it can adapt to specific query and retrieval document input requirements, and is particularly optimized for BIM data question and answer scenarios; this fine-tuning process enables the large language model to better understand and process professional information related to rail transit.

[0025] Furthermore, in step six, the user inputs the query question in a manner including text, voice, and other input methods that can be converted into text.

[0026] As another aspect of the present invention, it also relates to a rail transit BIM model data question-answering query device based on a large language model, comprising:

[0027] The basic large language model deployment module is used to configure and install the large language model in the user network through interface calls or local deployment, so that users can access and query the vector database configured by the large language model through query mode;

[0028] Text data extraction and vectorization module: used to extract text data from rail transit BIM model data and slice text documents, generate text blocks by slicing, and vectorize the generated text blocks and store them in the configured vector database;

[0029] Unique ID number association module: used to assign a unique ID number to each text block generated by slicing; based on the semantic information of each text block, the pictures, documents, model links, and video resources associated with the semantic information of the text block in the rail transit BIM model data are also bound to the unique ID number of the corresponding text block, so as to establish an associated knowledge base;

[0030] Unique ID number vectorization module: used to vectorize the unique ID number of each text block obtained and store it synchronously in the vector database configured in step 1;

[0031] Relational database storage module: used to store resources including pictures, documents, model links, and videos bound to each unique ID number in a relational database;

[0032] User interactive query module: used for query questions (Query) input by users. After hitting the relevant text block through the large language model, the unique ID number corresponding to the text block is retrieved according to the vector search engine. Based on the unique ID number, the relational database is searched to obtain the resources associated with the slice, and the resources are sorted and returned to the user.

[0033] As another aspect of the present invention, it also relates to a rail transit BIM model data question and answer query system based on a large language model, which includes the above-mentioned rail transit BIM model data question and answer query device based on a large language model.

[0034] As another aspect of the present invention, it also relates to a computer storage medium. When the computer program is executed by a processor, the above-mentioned rail transit BIM model data question and answer query method based on a large language model is implemented.

[0035] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0036] (1) The large language model-based rail transit BIM model data question-answering query method of the present invention combines document vectorization, and binds resources including pictures, documents, model links, and videos associated with the semantic information of text blocks in the rail transit BIM model data to the unique ID number of the sliced ​​text block and stores them in a relational database, thereby avoiding the complex training of the multimodal large language model. Through the basic large language model, there is no need for training and fine-tuning (or only a small amount of training and fine-tuning is required), which reduces the huge investment in computing power and manpower.

[0037] (2) The large language model rail transit BIM model data question-answering query method of the present invention can more accurately capture the semantics and context of user queries and provide more intelligent and personalized search result sorting and display by integrating the large language model with the rail transit BIM model and utilizing the powerful understanding ability of the large language model. An intuitive and efficient data query system is realized, which not only improves the query efficiency, but also greatly reduces the difficulty of operation, so that non-professionals can easily obtain the required information. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of a preferred embodiment of the invention;

[0039] Figure 2 A schematic diagram of a preferred embodiment of the invention;

[0040] Figure 3 Schematic diagram of the principle of large language model vector retrieval in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0042] Please refer to Figure 1-Figure 2 The present invention relates to a rail transit BIM model data question-answering query method based on a large language model, comprising:

[0043] Step 1: Basic large language model deployment: Configure and install the large language model in the user network through interface calls or local deployment, so that users can access and query the vector database configured by the large language model through query mode;

[0044] Step 2: Text data extraction and vectorization: Extract the text data in the rail transit BIM model data and slice the text document, slice to generate text blocks, vectorize the generated text blocks and store them in the vector database configured in step 1 (based on the text blocks obtained by the data processing module, vectorize (Embedding), create indexes and generate embedded vectors, create corresponding indexes, and vector data can be retrieved through the search engine. The search engine connects the obtained text blocks so that each text block can be quickly found and accessed through its unique vector identifier); the rail transit BIM model data includes the original BIM data information, the mounted structured engineering information, and the mounted unstructured documents and drawing information. For example, a model of electromechanical equipment, in addition to the model itself, also includes the corresponding certificate (picture), operation and maintenance manual (PDF electronic file), and on-site physical photo (picture) information. The method for extracting text data from rail transit BIM model data includes: through layout parsing (Parser) and OCR (text recognition) components, the unstructured document and drawing information together with the original BIM data information and the mounted structured engineering information are processed into text, and then processed into text blocks through a word segmenter (Text Splitter). Layout parsing (Parser), OCR (text recognition), and word segmenter (Text Splitter) are existing technologies related to large models, so they are not described in detail.

[0045] Step 3: Unique ID number association: assign a unique ID number to each text block generated by slicing in step 2; based on the semantic information of each text block, the resources including pictures, documents, model links, and videos in the rail transit BIM model data that are associated with the semantic information of the text block are also bound to the unique ID number of the corresponding text block, so as to establish an associated knowledge base;

[0046] Step 4: Vectorize the unique ID number: vectorize the unique ID number of each text block obtained in step 3 and store it synchronously in the vector database configured in step 1;

[0047] Step 5: Relational database storage: Store the resources including pictures, documents, model links, and videos bound to each unique ID number in step 3 in a relational database;

[0048] Step 6: User interactive query: Please refer to Figure 3 , the user inputs a query question (including text, voice and other input methods that can be converted into text). After the large language model hits the relevant text block, the relevant text block together with the original query question is constructed into a prompt word; this prompt word is then input into the large language model, and the large language model generates a reply to the user's query, and simultaneously retrieves the unique ID number corresponding to the text block according to the vector search engine. According to the unique ID number, the relational database is retrieved to obtain the resources associated with the slice, and the resources are sorted and returned to the user. Among them, the method of returning the resources to the user after sorting is: through the front-end interface, organize the slice question and answer pairs and the retrieved resources, realize the viewing of text answers, models, pictures, videos, and electronic document contents through human-computer interaction operations, integrate each reply knowledge vector into an answer in combination with the current context, and return each knowledge vector linked to each reply knowledge vector and the answer to the user. In some embodiments, the generated reply includes not only the direct answer text, but also covers the knowledge vector links related to the answer. These knowledge vector links are associated with the detailed content in the system database, which can provide users with more in-depth information references. In the end, all this information - including the answer

[0049] Text and knowledge vector links - are integrated and presented to users to meet their specific needs.

[0050] In order to effectively process this information, the system uses layout parsing (Parser) and optical character recognition (OCR) technology to convert unstructured document and drawing information into text format. The processed text data is further divided into multiple text blocks by a word segmenter (Text Splitter). In the data storage link, the present invention generates a unique identifier (ID) for each text block, and associates these IDs with the storage links of pictures and drawings and the storage links of BIM models, thereby forming a complete information network in the knowledge base. This processing and storage method not only improves the queryability and operability of the data, but also enhances the depth and breadth of the knowledge base's coverage of information in the field of rail transit. In this way, the present invention can provide a highly integrated and interconnected information query platform, greatly improving the accuracy and efficiency of data retrieval, and providing powerful data support for professionals in the field of rail transit.

[0051] As a preferred solution, step 6 also includes synchronously storing historical conversations. Historical conversations contain interaction records between the user and the system, which can be used to associate specific context information during the embedding vector retrieval process. By using the historical conversation library, the system can better understand the user's query intent and provide more accurate and relevant information feedback.

[0052] In some embodiments, the large language model in step one is pre-fine-tuned to ensure that it can adapt to specific query and retrieval document input requirements, and is particularly optimized for BIM data question and answer scenarios; this fine-tuning process enables the large language model to better understand and process professional information related to rail transit.

[0053] Please refer to Figure 2 As another aspect of the present invention, it also relates to a rail transit BIM model data question-answering query device based on a large language model, comprising:

[0054] The basic large language model deployment module is used to configure and install the large language model in the user network through interface calls or local deployment, so that users can access and query the vector database configured by the large language model through query;

[0055] Text data extraction and vectorization module: used to extract text data from rail transit BIM model data and slice text documents, generate text blocks by slicing, and vectorize the generated text blocks and store them in the configured vector database;

[0056] Unique ID number association module: used to assign a unique ID number to each text block generated by slicing; based on the semantic information of each text block, the pictures, documents, model links, and video resources associated with the semantic information of the text block in the rail transit BIM model data are also bound to the unique ID number of the corresponding text block, so as to establish an associated knowledge base;

[0057] Unique ID number vectorization module: used to vectorize the unique ID number of each text block obtained and store it synchronously in the vector database configured in step 1;

[0058] Relational database storage module: used to store resources including pictures, documents, model links, and videos bound to each unique ID number in a relational database;

[0059] User interactive query module: used for query questions (Query) input by users. After hitting the relevant text block through the large language model, the unique ID number corresponding to the text block is retrieved according to the vector search engine. Based on the unique ID number, the relational database is searched to obtain the resources associated with the slice, and the resources are sorted and returned to the user.

[0060] As another aspect of the present invention, it also relates to a rail transit BIM model data question and answer query system based on a large language model, which includes the above-mentioned rail transit BIM model data question and answer query device based on a large language model.

[0061] As another aspect of the present invention, it also relates to a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the rail transit BIM model data question and answer query method based on a large language model as described in any one of claims 1 to 7 is implemented.

[0062] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A rail transit BIM model data question-answering query method based on a large language model, characterized in that: include: Step 1: Basic large language model deployment: Configure and install the large language model in the user network through interface calls or local deployment, so that users can access and query the vector database configured by the large language model through query mode; Step 2: Text data extraction and vectorization: Extract the text data in the rail transit BIM model data and slice the text document, slice to generate text blocks, vectorize the generated text blocks and store them in the vector database configured in step 1; Step 3: Unique ID number association: assign a unique ID number to each text block generated by slicing in step 2; based on the semantic information of each text block, the resources including pictures, documents, model links, and videos in the rail transit BIM model data that are associated with the semantic information of the text block are also bound to the unique ID number of the corresponding text block, so as to establish an associated knowledge base; Step 4: Vectorize the unique ID number: vectorize the unique ID number of each text block obtained in step 3 and store it synchronously in the vector database configured in step 1; Step 5: Relational database storage: Store the resources including pictures, documents, model links, and videos bound to each unique ID number in step 3 in a relational database; Step 6: User interactive query: The user enters a query question. After the large language model matches the relevant text block, the relevant text block and the original query question are constructed into a prompt word. This prompt word is then input into the large language model, which generates a response to the user's query. The vector search engine simultaneously retrieves the unique ID number corresponding to the text block, searches the relational database based on the unique ID number, obtains the resources associated with the slice, and returns the resources to the user after sorting them out.

2. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized in that: The rail transit BIM model data in step 2 includes the original BIM data information, the mounted structured engineering information, and the mounted unstructured documents and drawing information.

3. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized in that: The method for extracting text data from the rail transit BIM model data in step 2 includes: processing the unstructured document and drawing information together with the original BIM data information and the mounted structured engineering information into text through layout parsing and OCR components, and processing them into text blocks through a word segmenter.

4. The rail transit BIM model data question-answering query method based on a large language model according to claim 1, characterized in that: The method for collating the resources and returning them to the user in step 6 is as follows: through the front-end interface, the sliced ​​question-answer pairs and the retrieved resources are organized to enable viewing of text answers, models, images, videos, and electronic document contents through human-computer interaction, integrating each reply knowledge vector into an answer in combination with the current context, and returning each knowledge vector linked to each reply knowledge vector and the answer to the user.

5. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized in that: The step 6 also includes synchronously storing historical conversations; the historical conversations contain interaction records between the user and the system, which can be used to associate specific context information during the embedding vector retrieval process; by utilizing the historical conversation library, the system can better understand the user's query intent and provide more accurate and relevant information feedback.

6. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized in that: The large language model in step one is pre-fine-tuned to ensure that it can adapt to specific query and retrieval document input requirements, and is particularly optimized for BIM data question and answer scenarios; this fine-tuning process enables the large language model to better understand and process professional information related to rail transit.

7. The rail transit BIM model data question-answering query method based on a large language model according to claim 1 is characterized in that: In step six, the user inputs the query question in the form of text, voice, or other input methods that can be converted into text.

8. A rail transit BIM model data question-answering query device based on a large language model, characterized in that: include: The basic large language model deployment module is used to configure and install the large language model in the user network through interface calls or local deployment, so that users can access and query the vector database configured by the large language model through query; Text data extraction and vectorization module: used to extract text data from rail transit BIM model data and slice text documents, generate text blocks by slicing, and vectorize the generated text blocks and store them in the configured vector database; Unique ID number association module: used to assign a unique ID number to each text block generated by slicing; based on the semantic information of each text block, the pictures, documents, model links, and video resources associated with the semantic information of the text block in the rail transit BIM model data are also bound to the unique ID number of the corresponding text block, so as to establish an associated knowledge base; Unique ID number vectorization module: used to vectorize the unique ID number of each text block obtained and store it synchronously in the vector database configured in step 1; Relational database storage module: used to store resources including pictures, documents, model links, and videos bound to each unique ID number in a relational database; User interactive query module: used for query questions (Query) input by users. After hitting the relevant text block through the large language model, the unique ID number corresponding to the text block is retrieved according to the vector search engine. Based on the unique ID number, the relational database is searched to obtain the resources associated with the slice, and the resources are sorted and returned to the user.

9. A rail transit BIM model data question-answering query system based on a large language model, characterized in that: The system includes the rail transit BIM model data question and answer query device based on a large language model as described in claim 8.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the rail transit BIM model data question-and-answer query method based on a large language model as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Data fragmentation and database division method and device, computer equipment and storage medium

    CN116719884A

  • Knowledge base construction method and question and answer dialogue method and system based on generative large language model

    CN117056471A

  • Question and answer system based on large language model

    CN117951269A

  • Knowledge base construction method, video automatic production method and software product

    CN117952203A

  • Streaming question and answer illustration method and system

    CN118035416A

Cited By

  • System and method for developing electronic manual of optical image measurement equipment

    CN119292731A

  • IFC data analysis method and device, equipment, storage medium and program product

    CN119962521A

  • IFC data analysis method, device, equipment, storage medium and program product

    CN119962521B

  • Rail transit intelligent customer service expert mixed big language model

    CN120525045A

  • A Method and System for Demand Coverage Analysis of Rail Transit Products Based on Large Language Models

    CN123114957A