Multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance
The multi-stage retrieval method enhances industrial process maintenance by using BERT for query routing, BM25 for initial filtering, TF-IDF for further refinement, and dense vector retrieval to optimize document ranking, addressing inefficiencies and accuracy issues in existing systems.
Patent Information
- Application Number
- CN202510393591.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-15
AI Technical Summary
The existing general RAG system is inefficient in industrial process operation and maintenance, consumes high computing resources, and is difficult to meet the query needs of professional fields, affecting the accuracy of the search results.
A multi-stage progressive enhanced knowledge retrieval method is adopted, including query routing, keyword recall, lightweight filtering and vector fine placing. The search process is optimized through BERT model classification, BM25 algorithm screening, TF-IDF word frequency-inverse document frequency and cosine similarity, and precise sorting based on dense vectors.
It improves the search efficiency, reduces the consumption of computing resources, ensures the accuracy and response speed of query, and is suitable for the efficient and intelligent retrieval needs of industrial process operations and maintenance.
Smart Images

Figure CN120316241A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of natural language processing and information retrieval, and in particular to a multi-stage progressively enhanced knowledge retrieval method for industrial process operation and maintenance. Background Art
[0002] Industrial process operation and maintenance involves monitoring, optimizing and maintaining production equipment, process flow and operating parameters to ensure production efficiency, safety and compliance. With the acceleration of industrialization, the scale of production systems is expanding, and the complexity of equipment and processes is increasing. The traditional operation and maintenance model that relies on manual experience can no longer meet the needs of modern industry for efficient and intelligent operation and maintenance. Specifically, it is manifested as follows: (1) Low troubleshooting efficiency. When equipment fails or process flow fluctuates, operation and maintenance personnel usually need to read a large number of technical documents, which is time-consuming and labor-intensive. (2) Lag in knowledge transfer. The training cycle for new employees is long, and it is difficult for them to quickly master complex operating procedures and equipment management points. (3) Fast knowledge update. With the development of industrial automation and intelligent manufacturing, various new equipment, new processes and new standards are constantly emerging, requiring practitioners to continuously learn and adapt to new specifications and technologies. In order to meet the above challenges, industrial process operation and maintenance urgently needs efficient intelligent retrieval technology. An efficient intelligent retrieval system can quickly locate key information from massive documents, significantly improve knowledge acquisition efficiency, shorten fault handling time, and provide real-time and accurate technical support for operation and maintenance decisions. This not only helps reduce equipment downtime and improve production stability, but also optimizes process flows, reduces energy consumption, ensures the company's safe production and compliant operations, and helps industrial operations and maintenance develop in an intelligent and standardized direction.
[0003] In recent years, with the advent of large language models (LLM), retrieval-augmented generation (RAG) technology has become an effective tool for handling complex query tasks. By combining retrieval and generation, RAG systems can quickly find relevant information in large-scale document repositories and generate accurate answers. However, general RAG systems still have some problems when facing highly specialized knowledge such as industrial operation and maintenance.
[0004] First, general RAG systems usually rely on full-text retrieval, which means they need to compare and calculate the query with all documents in the knowledge base. This is inefficient for large-scale knowledge bases, especially in real-time query scenarios where the response time is long, affecting the user experience. Second, general RAG systems often rely on a single retrieval algorithm, such as BM25 (Best Matching 25) or vector retrieval. However, a single algorithm is difficult to meet the needs of all types of queries. Especially when facing complex professional field problems, there are often a large number of professional terms, which affect the accuracy of retrieval results. Finally, since the retrieval and generation processes of RAG systems require a large amount of computing resources, especially in the generation stage, it may lead to excessive computing overhead, increase hardware costs, and limit its practical effect in efficient dynamic applications. To address these problems, how to optimize the retrieval strategy, improve retrieval efficiency, and reduce resource consumption has become the key to enhancing the application of RAG technology in fields such as industrial operation and maintenance. Summary of the Invention
[0005] The object of the present invention is to provide a multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance in order to solve the problems of a large knowledge base and low query efficiency in industrial process operation and maintenance.
[0006] The above object of the present application is achieved through the following technical solutions:
[0007] S1: Construct a knowledge base for the field of industrial process operation and maintenance; obtain the user query and classify it;
[0008] S2: Screen the documents in the knowledge base through the classified user query and the BM25 algorithm to determine the first candidate documents;
[0009] S3: Screen the first candidate documents through a lightweight filtering method combining TF-IDF (term frequency-inverse document frequency) and cosine similarity to obtain the second candidate documents;
[0010] S4: Perform precise sorting on the second candidate documents through dense vector-based retrieval to obtain the answer that best matches the user query.
[0011] Optionally, step S1 includes:
[0012] Obtain a knowledge base for the field of industrial process operation and maintenance; divide the knowledge base for the field of industrial process operation and maintenance into four major categories: process control, equipment maintenance, safety management, and regulations and standards to complete the construction of the knowledge base for the field of industrial process operation and maintenance;
[0013] The specific steps of classifying the user query by the BERT model include:
[0014] Obtain industrial equipment data, and combined with the prompt engineering of the BERT model, classify and label the industrial equipment data; the industrial equipment data includes: operation manuals, operation and maintenance logs, and expert Q&A;
[0015] Train the BERT model with the labeled industrial equipment data;
[0016] Use the trained BERT model, combined with the knowledge base, to classify user queries.
[0017] Optionally, step S2 includes:
[0018] Calculate the relevance between the user query and the documents in the knowledge base through the BM25 algorithm, screen the documents in the knowledge base, and determine the candidate texts;
[0019] For each document d in the knowledge base and user query q, the relevance score S BM25 (q, d) is represented by the following formula:
[0020]
[0021] where, t i represents the i-th word in the user query, f(t i , d) represents the word frequency of the word t i in the document d, |d| is the length of the document d, |d| avg is the average length of all documents in the document collection, and k1 and b are adjustment parameters; IDF(t i ) is the inverse document frequency.
[0022] Optionally, step S2 further includes:
[0023] The formula for calculating the inverse document frequency is:
[0024]
[0025] where, N is the total number of documents, and n(t i ) is the number of documents containing the word t i .
[0026] Optionally, step S3 includes:
[0027] Adopt a lightweight filtering method that combines TF-IDF (term frequency-inverse document frequency) and cosine similarity to screen the first candidate documents to obtain the second candidate documents;
[0028] The formula for calculating the term frequency TF is as follows:
[0029]
[0030] Among them, TF(t, d) represents the term frequency; f(t, d) is the number of times the word t appears in the document d; ∑ t′ f(t ′ , d) represents the total number of occurrences of all words in the document d;
[0031] The TF-IDF term frequency-inverse document frequency formula is as follows:
[0032] TF–IDF(t, d) = TF(t, d) × IDF(t)
[0033] Among them, IDF(t) represents the inverse document frequency.
[0034] Optionally, step S4 includes:
[0035] Converting the second candidate document and the user query into dense vectors through the open-source embedding model bge-base-en-v1.5;
[0036] Calculating the distance between the dense vectors to obtain the similarity between the second candidate document and the user query; determining the second candidate document with the highest similarity as the answer that best matches the user query.
[0037] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes a multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance.
[0038] A computer-readable storage medium stores instructions that, when executed, execute a multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance.
[0039] The beneficial effects brought by the technical solution provided in this application are:
[0040] 1. Multi-stage progressive retrieval method: The present invention optimizes the retrieval process layer by layer through the retrieval strategies of query routing, keyword recall, lightweight filtering, and vector fine-ranking, thereby improving the retrieval efficiency.
[0041] 2. Query routing: Using the fine-tuned BERT model to accurately classify user questions, ensuring that the query can be quickly routed to the relevant knowledge base, and improving the accuracy and response speed of question retrieval.
[0042] 3. Lightweight filtering stage: By combining the TF-IDF term frequency-inverse document frequency and cosine similarity methods, rapid document filtering is performed to remove documents irrelevant to the query, greatly improving the retrieval efficiency. Description of the Drawings
[0043] The present application will be further described below in conjunction with the accompanying drawings and embodiments. In the accompanying drawings:
[0044] Figure 1 is the step diagram in the embodiment of the present application;
[0045] Figure 2 is the comparison diagram of the average response time in the embodiment of the present application;
[0046] Figure 3 is the schematic diagram of the electronic device structure in the embodiment of the present application. Specific Embodiments
[0047] For a clearer understanding of the technical features, objectives, and effects of the present application, the specific embodiments of the present application will now be described in detail with reference to the accompanying drawings.
[0048] The embodiment of the present application provides a multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance.
[0049] Please refer to Figure 1 , Figure 1 is the step diagram of a multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance in the embodiment of the present application, including:
[0050] S1: Construct a knowledge base in the field of industrial process operation and maintenance; obtain and classify user queries;
[0051] S2: Screen the documents in the knowledge base through the classified user queries and the BM25 algorithm to determine the first candidate documents;
[0052] S3: Screen the first candidate documents through a lightweight filtering method combining TF-IDF (term frequency-inverse document frequency) and cosine similarity to obtain the second candidate documents;
[0053] S4: Perform precise ranking on the second candidate documents through dense vector-based retrieval to obtain the answer that best matches the user query.
[0054] As an embodiment, the multi-stage screening process of the application not only ensures the retrieval accuracy but also greatly reduces the consumption of computing resources and improves the response speed of the system, thus achieving a balance between efficiency and accuracy.
[0055] As an embodiment, the overall structure of the proposed multi-stage progressive retrieval strategy is as Figure 1 shown.
[0056] Step S1 includes:
[0057] Obtain the knowledge base in the field of industrial process operation and maintenance; divide the knowledge base in the field of industrial process operation and maintenance into four major categories: process control, equipment maintenance, safety management, and regulations and standards, and complete the construction of the knowledge base in the field of industrial process operation and maintenance;
[0058] As an embodiment, collect various text materials such as technical documents, enterprise standards, industry specifications, and failure cases related to industrial operation and maintenance. The content should cover different links of the industrial production process, including but not limited to aspects such as process control, equipment maintenance, operation monitoring, and safety management. In addition, it should also include fields such as equipment fault diagnosis and operation procedure optimization to meet the knowledge needs at different levels in industrial operation and maintenance. The knowledge base is divided into several first-level classifications, such as "Process Control", "Equipment Maintenance", "Safety Management", and "Regulations and Standards". Each first-level classification is further divided into multiple second-level subcategories. For example, "Pump System Maintenance" and "Heat Exchanger Cleaning" under "Equipment Maintenance". Each second-level subcategory stores relevant documents and metadata to ensure that the documents can be efficiently retrieved in a specific field.
[0059] The specific steps for the BERT model to classify user queries include:
[0060] As an embodiment, since the BERT model is pre-trained with a large amount of general corpus, when facing tasks in professional fields, using the general BERT model may not fully understand the professional terms and knowledge in the field of industrial operation and maintenance. Therefore, it needs to be fine-tuned.
[0061] Obtain industrial equipment data, and combine the prompt engineering of the BERT model to classify and label the industrial equipment data; the industrial equipment data includes: operation manuals, operation and maintenance logs, and expert Q&A;
[0062] Train the BERT model with the labeled industrial equipment data;
[0063] As an embodiment, the labels include first-level classifications and second-level subcategories to ensure that each piece of text can be attributed to an accurate field. Finally, through manual review, the accuracy of the labels is ensured. For example, the question "How to deal with serious fouling of the heat exchanger tube bundle?" will be automatically classified as "Equipment Maintenance / Heat Exchanger Cleaning", and this label will be saved in the training dataset.
[0064] Use the trained BERT model, combined with the knowledge base, to classify user queries.
[0065] As an embodiment, construct accurate classification of problems through query routing to ensure that the retrieval is only carried out in the knowledge base of the relevant field. Through the BERT model, the system can accurately identify the first-level classification and second-level subcategory to which the problem belongs, and then only retrieve in the corresponding field knowledge base, reducing the search scope, thereby improving the efficiency and accuracy of the retrieval.
[0066] Step S2 includes:
[0067] As an embodiment, in the keyword recall stage: after query routing, the system enters the keyword recall stage, aiming to quickly find documents relevant to the user query from the knowledge base of the target domain. In this stage, the BM25 algorithm is adopted for preliminary retrieval to quickly screen out the documents most relevant to the query.
[0068] Through the BM25 algorithm, the relevance between the user query and the documents in the knowledge base is calculated, and the documents in the knowledge base are screened to determine the candidate text;
[0069] For each document d in the knowledge base and the user query q, the relevance score S BM25 (q, d) of the BM25 algorithm is represented by the following formula:
[0070]
[0071] where t i represents the i-th word in the user query, f(t i , d) represents the term frequency of the word t i in the document d, |d| is the length of the document d, |d| avg is the average length of all documents in the document set, and k1 and b are adjustment parameters; IDF(t i ) is the inverse document frequency.
[0072] Step S2 also includes:
[0073] The formula for calculating the inverse document frequency is:
[0074]
[0075] where N is the total number of documents, and n(t i ) is the number of documents containing the word t i .
[0076] As an embodiment, through the formula of the BM25 algorithm, BM25 can measure the matching degree of each keyword in the document with the query and generate a relevance score for the document. According to this score, the system can quickly recall the documents with higher relevance and use them as candidate results for subsequent processing. The goal of this stage is to minimize the computational amount as much as possible, quickly screen out the documents highly relevant to the query, and provide a basis for the subsequent lightweight filtering and fine ranking stages.
[0077] Step S3 includes:
[0078] As an example, in the keyword recall stage, although the BM25 algorithm can quickly screen out some relevant documents, it may also recall some inaccurate documents. If dense vector similarity matching is directly performed at this time, it may lead to slow calculation speed and low calculation efficiency. Therefore, the present invention adds a lightweight filtering stage (a method combining TF-IDF term frequency-inverse document frequency and cosine similarity) to further improve the relevance of documents. The goal of this stage is to remove irrelevant documents by quickly calculating semantic similarity, thereby improving the efficiency and accuracy of subsequent vector fine ranking.
[0079] A lightweight filtering method combining TF-IDF term frequency-inverse document frequency and cosine similarity is adopted to screen the first candidate documents to obtain second candidate documents;
[0080] As an example, TF-IDF can measure the importance of each word in a document, while cosine similarity is used to calculate the similarity between two texts. The formula is as follows:
[0081]
[0082] Among them, A and B are the vector representations of two texts, A·B is their dot product, and ‖A‖ and ‖B‖ represent the norms. By calculating the cosine similarity, the similarity between two texts can be judged, and then the most relevant documents can be screened out.
[0083] As an example, the core idea of TF-IDF is to generate a sparse vector representation of a document by calculating the term frequency (TF) and inverse document frequency (IDF) of each word. The term frequency represents the frequency of a certain word appearing in a document.
[0084] The calculation formula of the term frequency TF is as follows:
[0085]
[0086] Among them, TF(t, d) represents the term frequency; f(t, d) is the number of times the word t appears in the document d; ∑ t′ f(t ′ , d) represents the total number of times all words appear in the document d.
[0087] As an example, the term frequency TF reflects the importance of a word in a document. The inverse document frequency represents the importance of a certain word in the entire document collection. It reduces the weight of common words that appear frequently in all documents by measuring the universality of word occurrences. TF-IDF is the product of TF and IDF, representing the importance of a certain word in a document.
[0088] The TF-IDF term frequency-inverse document frequency formula is as follows:
[0089] TF–IDF(t, d) = TF(t, d) × IDF(t)
[0090] Among them, IDF(t) represents the inverse document frequency.
[0091] As an example, by combining the term frequency and the inverse document frequency IDF, TF-IDF can effectively reflect the importance of a certain word in a specific document. The vectors generated by TF-IDF are usually sparse because most words do not appear in any specific document. A sparse vector means that most elements are zero, and only a few eigenvalues are non-zero. This representation method greatly reduces the computational amount. Compared with the dense vector retrieval method, TF-IDF and cosine similarity have higher computational efficiency because they rely on the representation of sparse vectors and only need to focus on non-zero elements during calculation, without the need for complex high-dimensional vector calculations. Dense vector retrieval needs to convert the document into a dense vector through a deep learning model and then calculate the similarity between vectors, which involves high-dimensional vector operations and has a large computational overhead, especially when dealing with a large-scale document library, with slower speed and higher computational cost. Therefore, the lightweight filtering stage can significantly improve the retrieval efficiency while ensuring accuracy, providing more accurate and efficient candidate documents for the subsequent fine-ranking stage.
[0092] Convert the second candidate document and the user query into dense vectors through the open-source embedding model bge-base-en-v1.5;
[0093] Calculate the distance between the dense vectors to obtain the similarity between the second candidate document and the user query; determine the second candidate document with the highest similarity as the answer that best matches the user query.
[0094] As an example, the goal of the vector fine-ranking stage is to further optimize the document ranking to ensure that the most relevant answer is finally returned to the user. In this stage, the second candidate document and the user query are converted into dense vector representations through the open-source embedding model bge-base-en-v1.5. Different from the previous two stages that rely on keyword and sparse vector calculations, the distance calculation between dense vectors can more accurately evaluate the similarity between the document and the query, thereby capturing more semantic information and improving the accuracy of the retrieval results. Finally, through vector fine-ranking, the system can select the most relevant document from the filtered documents as the prompt word for the generation stage to input into the large model.
[0095] In the present invention, three different retrieval methods are involved: keyword recall (BM25 algorithm), lightweight filtering (TF-IDF and cosine similarity), and vector fine-ranking (retrieval based on dense vectors). Each retrieval method focuses on different document retrieval requirements and computational efficiency. Therefore, it is necessary to construct indexes and store them separately for each retrieval method.
[0096] BM25 Inverted Index Construction: In the present invention, the retrieval engine Elasticsearch is adopted to construct the inverted index. The construction process of the inverted index mainly includes document tokenization, index creation, and storage. First, the system tokenizes the field text in each document, breaking the text into tokens. Then, for each token, the system records all the document IDs and related information where it appears, constructing the inverted index. The basic structure of the inverted index consists of a dictionary and a posting list. The dictionary stores all the tokens, while the posting list records information such as the positions and frequencies of each token in the documents. This structure enables the system to quickly find the documents containing a query token for any query, improving the retrieval efficiency.
[0097] Sparse Vector Index Construction: In the present invention, TfidfVectorizer of Scikit-learn is used to construct the sparse vector index. TfidfVectorizer can transform text data into a sparse matrix, facilitating efficient storage and processing. First, TfidfVectorizer preprocesses the text, such as tokenization and stop word removal, and then calculates the TF and IDF of each word in the document, thereby assigning a weight to each word. In this way, the TF-IDF model can effectively distinguish common words and keywords specific to certain documents and generate a high-dimensional sparse matrix, thus achieving more efficient text analysis and search.
[0098] Dense Vector Index Construction: The present invention uses the open-source embedding model bge-base-en-v1.5 to represent each document or query as a fixed-dimensional real-valued dense vector. These vectors are usually high-dimensional and can better capture the meaning of the document in the semantic space. Then, these vectors are stored in the Chroma database. Chroma supports efficient vector insertion, storage, and index construction, and can automatically handle vector similarity queries to achieve fast and accurate similarity retrieval in the last step.
[0099] Multi-stage Retrieval Process:
[0100] The core objective of the multi-stage retrieval process is to improve the accuracy and relevance of the retrieval results through gradual screening and ranking. The specific steps are as follows:
[0101] 1) Query Routing: After the user enters a question, first use the fine-tuned BERT model to classify the question. This step accurately maps the query to the corresponding knowledge base by identifying the intent of the question, ensuring that the subsequent retrieval process focuses on relevant content, avoiding interference from irrelevant content, and thus improving the retrieval efficiency.
[0102] 2) Fast Recall: After classifying the question and determining the relevant knowledge base during the query routing phase, the next second phase is to quickly recall relevant paragraphs using the BM25 algorithm. In this phase, the system quickly searches the documents in the knowledge base based on the keywords of the user's question and finds the most relevant paragraphs. The BM25 algorithm calculates the relevance score of each document to the query and returns the document paragraphs with the highest scores. The advantage of this step is its speed, which can quickly narrow down the retrieval scope and ensure that the recalled content has a high lexical relevance to the query. However, since BM25 only relies on lexical matching, it may miss some semantically related but not fully matched content. Therefore, the subsequent sparse vector retrieval and re-ranking phases will further improve the quality of the results.
[0103] 3) Lightweight Filtering: After the initial recall by the BM25 algorithm, the third phase uses TF-IDF and vector similarity to further screen the recalled candidate paragraphs. In this phase, the system calculates the similarity between the query and the candidate document paragraphs based on the TF-IDF vectors. Since the vectors generated by TF-IDF are usually sparse, the computational efficiency is high. In this phase, sparse vector retrieval not only improves the accuracy but also accelerates the screening of relevant paragraphs, thus further improving the relevance of the retrieval results while ensuring high efficiency.
[0104] 4) Vector Re-ranking: After sparse vector retrieval, the fourth phase uses dense vector re-ranking to optimize the retrieval results. The bge-base-en-v1.5 model converts the user's question into a high-dimensional dense vector and calculates the cosine similarity between the query vector and the paragraph vectors through the Chroma vector database to evaluate the semantic matching degree. Chroma supports efficient vector storage and similarity search and can provide fast retrieval performance on large-scale datasets through GPU acceleration. Through this mechanism, the vector re-ranking process ensures that the most semantically relevant paragraphs are returned to the user first, thus significantly improving the accuracy and relevance of the retrieval and avoiding the lexical matching limitations that may occur in traditional retrieval methods.
[0105] Answer Generation:
[0106] After the above multi-stage progressive retrieval, the system has screened out the most relevant document paragraphs, which contain the information that best matches the user's query. However, the simple retrieval results are not sufficient to directly answer the question, and further understanding and generation are needed through a large model.
[0107] During the answer generation process, the system takes the user's query and the retrieved document content as prompt words and inputs them into a pre-trained large language model. The model will understand the user's intention based on the input context information, extract relevant facts from the retrieved candidate documents, and generate an accurate and concise answer by combining the information in the documents. To ensure the accuracy of the answer, the system will also use special prompt word design techniques, such as guiding the model to focus on the key information in the documents, avoiding generating irrelevant content or incorrect answers. Finally, this answer will be returned to the user to complete the entire retrieval and generation process.
[0108] The retrieval strategy proposed in the present invention can be evaluated according to the average response time, the mean reciprocal rank (MRR), and the precision. Precision@n refers to the proportion of relevant texts among the top n retrieval results. In this experiment, n is set to 5.
[0109] This experiment takes the sewage treatment process as the application scenario to evaluate the actual performance of the proposed multi-stage progressive retrieval method. Sewage treatment removes pollutants in sewage through physical, chemical, and biological methods to achieve the purpose of environmental protection discharge or resource recovery. Its operation involves multiple technical links, such as pretreatment, biochemical treatment, advanced treatment, and sludge disposal, and relies on a large number of technical documents and standards and specifications. Therefore, efficient knowledge retrieval is crucial for improving the operation and maintenance efficiency.
[0110] The dataset used in the experiment is knowledge documents in the sewage treatment field collected from public channels, including various types such as process specifications, equipment maintenance manuals, technical literature, policies and regulations, etc. There are more than 200 documents in total, and the total vocabulary is about 2 million. These document contents are comprehensive, covering various sewage treatment technologies such as the activated sludge process, SBR process, AAO process, MBR membrane technology, etc., as well as the operation specifications and fault handling solutions of related equipment. To verify the actual retrieval effect of the system, 100 test questions are designed for these documents. These questions include various types such as process parameter queries, equipment fault troubleshooting, and policy and regulation interpretation, simulating the user's query needs in real scenarios. The questions cover the main knowledge points in the document set to ensure the comprehensiveness and representativeness of the evaluation results.
[0111] To evaluate the system performance more meticulously, these 100 test questions are divided into two categories: The first category is simple questions, which usually have a single source and the answers can be obtained from a single document; the second category is complex questions, and the answers to these questions involve multiple document sources. Each category contains 50 questions. The experiment will test simple questions and complex questions separately to comprehensively evaluate the system's performance in different types of queries. The experimental control group uses dense vector indexing combined with global retrieval for queries. The experiment evaluates the advantages of the multi-stage progressive retrieval strategy in improving query efficiency and maintaining high accuracy by recording the average response time, MRR, and Precision@5 of the two methods. The experimental result metrics are shown in Table 1:
[0112] Table 1 Performance Metrics of the Proposed Retrieval Strategy and the Control Group
[0113]
[0114] Table 1 shows that the multi-stage retrieval strategy proposed in the present invention performs excellently in improving the system response efficiency, reducing the average query response time by approximately 72.1%. The comparison of the average response times of the proposed strategy and global vector retrieval is as Figure 2 shown. In the test of simple questions, the Precision@5 and MRR metrics of the proposed strategy show a slight decline compared with global vector retrieval because global vector retrieval has no intermediate filtering interference and can retain more relevant results at the cost of more time. In the test of complex questions, the Precision@5 and MRR metrics of the proposed strategy also show a slight decline. This is because complex questions rely more on the semantic understanding ability of dense vector retrieval and may wrongly exclude relevant texts in the filtering stage; at the same time, complex questions are also more likely to lead to classification errors in the query routing stage, resulting in a deviation in the retrieval scope. In this study, four groups of ablation experiments were also designed to explore the impact of each stage in the multi-stage retrieval process on the system response time. Specifically, the following three stages were removed respectively: (1) the BERT classification model (used for query routing); (2) BM25 fast recall; (3) TF-IDF sparse vector retrieval. By comparing the retrieval efficiency of the complete system and the system after removing a certain filtering stage, the role of each stage in improving the response speed can be intuitively seen. All results are based on the test of the same batch of 100 test questions. The experimental results are shown in Table 2:
[0115] Experimental setup Average response time (milliseconds) The proposed complete retrieval strategy 879.6 Remove the BERT classification model 2321.2 Remove the BM25 fast recall 1785.7 Remove the TF-IDF sparse retrieval 1593.4
[0116] As can be seen from the above results, each stage has an important impact on the system retrieval efficiency: after removing the query routing, the system cannot classify according to the query category, resulting in a significant expansion of the retrieval scope and a significant increase in the average response time; after removing the BM25 fast recall and TF-IDF sparse retrieval, the vector refinement stage needs to calculate more irrelevant documents, resulting in an increase in the response time. Therefore, each stage of the proposed retrieval strategy has an important impact on the system retrieval efficiency.
[0117] In summary, the overall experimental results show that the proposed multi-stage progressive retrieval strategy has significant advantages in response time, has high retrieval efficiency, and does not significantly sacrifice the result quality, making it suitable for scenarios with high real-time requirements. In the future, it can be considered to improve the filtering accuracy of the filtering stage to further optimize the retrieval quality and efficiency of the proposed strategy.
[0118] This application also discloses an electronic device. Referring to Figure 3 , Figure 3 is a schematic structural diagram of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.
[0119] Among them, the communication bus 502 is used to realize the connection and communication between these components.
[0120] Among them, the user interface 503 may include a display screen. Optionally, the user interface 503 may further include a standard wired interface and a wireless interface.
[0121] Among them, the network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).
[0122] This application also discloses a computer-readable storage medium, which stores multiple instructions suitable for a processor to load and execute the above-mentioned multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance.
[0123] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereby. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure.
[0124] This application aims to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The description and embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.
Claims
1. A multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance, characterized in that The method includes the following steps: S1: Construct a knowledge base in the field of industrial process operation and maintenance; obtain user queries and classify them; S2: Screen the documents in the knowledge base through the classified user queries and the BM25 algorithm to determine the first candidate documents; S3: Screen the first candidate documents through a lightweight filtering method that combines TF-IDF (term frequency-inverse document frequency) and cosine similarity to obtain the second candidate documents; S4: Through retrieval based on dense vectors, accurately sort the second candidate documents to obtain the answer that best matches the user query.
2. The multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance as claimed in claim 1, wherein Step S1 includes: Obtain a knowledge base in the field of industrial process operation and maintenance; divide the knowledge base in the field of industrial process operation and maintenance into four major categories: process control, equipment maintenance, safety management, and regulations and standards to complete the construction of the knowledge base in the field of industrial process operation and maintenance; The specific steps for classifying user queries using the BERT model include: Obtain industrial equipment data, and in combination with the prompt engineering of the BERT model, classify and label the industrial equipment data; the industrial equipment data includes: operation manuals, operation and maintenance logs, and expert Q&A; Train the BERT model with the labeled industrial equipment data; Use the trained BERT model, in combination with the knowledge base, to classify user queries.
3. The multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance as described in claim 1, characterized in that, Step S2 includes: Calculate the relevance between the user query and the documents in the knowledge base through the BM25 algorithm, and screen the documents in the knowledge base to determine the candidate texts; For each document d in the knowledge base and user query q, the relevance score S BM25 (q, d) is expressed by the following formula: Among them, t i represents the i-th word in the user query, f(t i , d) represents the word frequency of the word t i in the document d, |d| is the length of the document d, |d| avg is the average length of all documents in the document collection, k1 and b are adjustment parameters; IDF(t i ) is the inverse document frequency.
4. The multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance as claimed in claim 3, wherein Step S2 also includes: The inverse document frequency calculation formula is: Among them, N is the total number of documents, and n(t i ) is the number of documents containing the word t i .
5. The multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance as claimed in claim 1, wherein Step S3 includes: Adopt a lightweight filtering method that combines TF-IDF and cosine similarity to screen the first candidate documents to obtain the second candidate documents; The calculation formula for the term frequency TF is as follows: Among them, TF(t, d) represents the term frequency; f(t, d) is the number of times the word t appears in the document d; ∑ t′ f(t ′ , d) represents the total number of occurrences of all words in the document d; The TF-IDF (term frequency-inverse document frequency) formula is as follows: TF–IDF(t,d) = TF(t,d) × IDF(t) where IDF(t) represents the inverse document frequency.
6. The multi-stage progressive enhanced knowledge retrieval method for industrial process operation and maintenance according to claim 1, characterized in that Step S4 includes: Convert the second candidate documents and the user query into dense vectors through the open-source embedding model bge-base-en-v1.5; Calculate the distance between the dense vectors to obtain the similarity between the second candidate documents and the user query; determine the second candidate document with the highest similarity as the answer that best matches the user query.
7. An electronic device, characterized in that, It includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1-6.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions. When the instructions are executed by a computer, the method according to any one of claims 1-6 is executed.
Citation Information
Cited By
Knowledge reasoning method and device for integrated circuit wafer process and medium
CN120744140A
Intelligent insurance question and answer method based on large language model and insurance knowledge base
CN121255989A
Semantic core weighted query method and device oriented to retrieval enhancement generation and medium
CN121745079A
A semantic core weighted query method, device and medium for retrieval enhancement generation
CN121745079B
