Bill-of-material-oriented hybrid retrieval enhancement generation method

Abstract

The invention discloses a mixed retrieval enhancement generation method oriented to a bill of materials, and belongs to the technical field of computers, and the method comprises the following steps: receiving a user query, and dividing the query into a generalization class, a precision class or a mixed class through an AI intention classification model with a dynamic adaptive training mechanism; executing branch retrieval according to classification, and constructing an enhancement prompt according to priority fusion context; after an initial answer is generated through the large language model, a target answer is output through illusion verification. According to the method, intelligent routing of a retrieval path of generalization, accuracy and mixed query is accurately realized through AI intention classification, a structured fact is obtained by multi-hop reasoning of a BOM graph database, an accurate hallucination pain point caused by standard RAG semantic retrieval is cracked from the source in combination with a hallucination closed-loop verification mechanism, the accuracy of BOM related query is effectively improved, and the search efficiency is improved. And high-reliability and high-precision technical support is provided for core decisions such as engineering design, change evaluation and maintenance traceability in safety-critical fields such as aviation and ships.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and more specifically, relates to a hybrid retrieval enhancement generation method for bills of materials. Background Technology

[0002] In fields such as aerospace engineering, shipbuilding, automobile manufacturing, and complex equipment manufacturing, the Bill of Materials (BOM) serves as the core data for Product Lifecycle Management (PLM) and is a crucial link connecting design, production, maintenance, and the supply chain. BOM data essentially presents a deeply nested and complex hierarchical structure, including multiple nodes such as parts, assemblies, and suppliers, as well as multiple relationships such as hierarchical dependencies, assembly relationships, and supply chain dependencies. Its query requirements cover various types, including generalized, precise, and hybrid queries, which places certain demands on the accuracy, efficiency, and flexibility of natural language interaction.

[0003] Large Language Models (LLMs) are widely used in information retrieval due to their powerful natural language understanding and generation capabilities. Retrieval Augmented Generation (RAG) architectures, by introducing external retrieval context, have become a mainstream solution to alleviate the LLM illusion problem. However, existing technologies still have certain shortcomings when handling BOM queries: While the standard RAG architecture relies on semantic similarity retrieval from vector databases and can respond well to generalized queries, it cannot accurately identify the structured hierarchical relationships within BOM data. For exact queries, semantic retrieval is prone to recalling incorrect parts based on the semantic similarity of the query text, potentially leading to a false sense of precision and causing disastrous consequences in safety-critical fields such as aerospace engineering. Furthermore, the standard RAG uses fixed retrieval thresholds, which cannot accommodate the different precision requirements of generalized and exact queries, potentially exacerbating retrieval bias. Traditional BOM data management often uses relational or graph databases. Relational databases require complex and expensive JOIN operations when handling multi-hop dependency queries in BOMs, resulting in extremely low query efficiency and potentially failing to meet the rapid tracing needs of complex BOMs. While graph databases can achieve efficient multi-hop inference through node and edge modeling, fitting the natural structure of BOMs, they lack natural language interaction capabilities. Engineers need to master specialized graph query languages ​​to operate them, greatly limiting their flexibility. Moreover, their multi-hop inference depth is often fixed, easily leading to issues such as missed deep node queries or redundant shallow node queries.

[0004] Existing technologies have failed to achieve a harmonious balance between the flexibility of semantic understanding and the accuracy of structured queries. While the standard RAG architecture provides a natural language interface, its reliability is insufficient in precise BOM queries. Although graph databases ensure the accuracy of structured queries, they cannot respond to generalized query requirements. The two remain independent and separate, unable to handle high-value mixed queries, and lack a closed-loop verification mechanism for LLM-generated results, which may make it difficult to effectively eliminate illusions. Summary of the Invention

[0005] To address the aforementioned problems and technical deficiencies, this application adopts the following technical solution: a hybrid retrieval enhancement generation method for bills of materials, comprising the following steps: Step S1: Receive user queries; Step S2: Using an artificial intelligence intent classification model, classify the user query into one of the following: a generalized problem, a precise problem, or a hybrid problem. The artificial intelligence intent classification model adopts a dynamic adaptive training mechanism, which regularly updates the training set based on historical query logs, user query feedback data, and new query samples in a specific domain to achieve continuous optimization of classification accuracy. Step S3: Based on the classification results of step S2, execute the branch retrieval strategy: S301. If the query is a generalized problem, then a first context is retrieved from a vector database based on semantic similarity; the vector database adopts a dynamic threshold mechanism for semantic retrieval relevance, and the retrieval threshold corresponding to the generalized problem is lower than the retrieval threshold corresponding to the precise problem. S302. If the query is the exact class question, then multi-hop reasoning is performed from a bill of materials database to retrieve the second context; the multi-hop reasoning adopts a reasoning depth adaptive adjustment mechanism, which dynamically adjusts the depth threshold of graph traversal according to the hierarchical depth of the bill of materials, the complexity of the query intent and the historical retrieval results. S303. If the query is a mixed-type question, then execute steps S301 and S302 to retrieve the first context and the second context; Step S4: Combine the context retrieved in step S3 with the user query to construct an enhanced prompt; the fusion process adopts a context priority sorting rule, the second context corresponding to the structured facts of the bill of materials has a higher priority than the first context corresponding to the generalized document, and in mixed-type problems, the context weight related to the exact query part has a higher weight than the context weight related to the generalized query part. Step S5: Input the enhanced prompts into a large language model to generate an initial answer in response to the user query; Step S6: Perform illusion verification on the initial answer: Extract the structured information related to the bill of materials from the initial answer, compare it with the original data in the bill of materials database, and if there is any inconsistency, correct the initial answer and finally output the target answer to suppress illusion.

[0006] Preferably, the dynamic adaptive training mechanism of the artificial intelligence intent classification model in step S2 specifically includes: Real-time collection of user feedback on classification results; Every 10 days, new query samples and query samples corresponding to feedback information are added to the training set to incrementally fine-tune the artificial intelligence intent classification model. The preset period can be dynamically adjusted according to the query volume in a specific field. When the query volume is greater than the preset threshold, the period is shortened; when the query volume is less than the preset threshold, the period is extended.

[0007] Furthermore, the adaptive adjustment mechanism for the inference depth of multi-hop inference in step S302 specifically includes: Pre-acquire the hierarchical depth data of each part node in the bill of materials database, and establish an initial mapping relationship between hierarchical depth and inference depth; For precise queries, analyze key information such as part numbers and assembly names involved in the query to locate the corresponding bill of materials nodes; If the level depth of the bill of materials node is greater than the first preset depth threshold, the inference depth is adjusted to the first depth value; if the level depth of the bill of materials node is less than the second preset depth threshold, and the second preset depth threshold is less than the first preset depth threshold, the inference depth is adjusted to the second depth value. If there are omissions or redundancies in the multi-hop reasoning results of this type of query in historical retrieval, the initial mapping relationship is dynamically corrected based on feedback.

[0008] Preferably, the rule for setting the dynamic threshold for semantic retrieval relevance in step S301 is as follows: The retrieval threshold range for generalized questions is 0.6-0.75, and the retrieval threshold range for exact questions is 0.8-0.95. In mixed-type problems, a generalized threshold is used for retrieval of the generalized part, while a precise threshold is used for semantic retrieval of the precise part.

[0009] Furthermore, the context priority sorting rule in step S4 specifically includes: The second context is sorted in descending order of its relevance to the exact part of the query, and the first N serialized entries are taken as the core context. The first context is sorted in descending order of semantic similarity, and the top M are taken as auxiliary contexts.

[0010] Preferably, the hallucination verification in step S6 specifically includes the following sub-steps: S601. Use a named entity recognition model to extract bill of materials related entity information from the initial answer. The entity information includes part number, assembly number, subordinate relationship between parts and assemblies, and supplier information. S602. Convert the extracted entity information into a graph query language and execute the query in the bill of materials database; If the query results match the extracted entity information, that part of the content is retained; if the query results do not contain corresponding entity information or there is a conflict, that part of the content is deleted and marked as not supported by relevant bill of materials structured data. S603. Reorganize the revised content to generate the target answer.

[0011] Preferably, the bill of materials database is a graph database, and the graph database adopts any one of Neo4j, ArangoDB or NebulaGraph.

[0012] Furthermore, the multi-hop reasoning in step S302 is a graph traversal query performed on the graph database, used to trace the hierarchical dependencies, assembly relationships, or supply chain dependencies represented by nodes and edges in the bill of materials.

[0013] Furthermore, the artificial intelligence intent classification model in step S2 is a classification model based on the Transformer architecture or a lightweight decision tree model, and the training data includes domain-specific labeled query logs, dynamically fed-back query samples, and manually corrected classification error samples.

[0014] Furthermore, the vector database in step S301 contains text embeddings extracted from unstructured or semi-structured documents, including technical manuals, design specifications, maintenance logs, fault analysis reports, or industry standard documents; the vector database uses any one of Milvus, FAISS, Chroma, or Weaviate.

[0015] Compared to existing technologies, the beneficial effects of this application are as follows: (1) This application uses an artificial intelligence intent classification model to accurately classify user queries into generalized, precise and hybrid categories, thereby achieving intelligent routing of retrieval paths. Generalized queries ensure response flexibility through semantic retrieval of vector databases, precise queries obtain 100% structured facts through multi-hop reasoning of BOM graph databases, and hybrid queries integrate the results of the two types of retrieval, fundamentally solving the problem of precise illusion caused by standard RAG semantic retrieval. Combined with the newly added illusion closed-loop verification step, the BOM structured information in the generated answer is extracted through named entity recognition and compared with the original data of the graph database to realize error information correction and annotation, ensuring that the accuracy of BOM-related answers is effectively improved and guaranteed, and providing reliable support for engineering decisions in security-critical fields. (2) In this application, the vector database adopts a semantic retrieval relevance dynamic threshold mechanism. The generalized query uses a lower threshold to ensure the recall rate, and the precise query uses a higher threshold to ensure the accuracy rate, thus avoiding retrieval bias caused by a single threshold. The multi-hop inference of the BOM graph database adopts a depth adaptive adjustment mechanism, which dynamically adjusts the inference depth based on the depth of the BOM node hierarchy, query complexity and historical feedback. This not only solves the omission or redundancy problem caused by fixed depth, but also effectively improves query efficiency compared with the JOIN operation of traditional relational databases. At the same time, the model supports the selection of mainstream databases such as Neo4j and Milvus, adapts to the existing technical architecture of different enterprises, and can be flexibly applied to complex BOM query scenarios in multiple fields such as aviation, shipbuilding and automobiles.

[0016] (3) The AI ​​intent classification model in this application adopts a dynamic adaptive training mechanism. By collecting user feedback in real time, periodically incremental fine-tuning and dynamically adjusting the training cycle, the classification accuracy is continuously optimized with the usage time, which is 7%-10% higher than that of the static model. The context fusion adopts a priority sorting rule, setting the BOM structured facts as the core context and the generalized documents as the auxiliary context, clarifying the attention allocation logic of LLM, avoiding answer bias caused by information interference, and thus improving the robustness of the system in long-term operation.

[0017] (4) This application supports direct querying of BOM-related issues using natural language. Engineers do not need to master professional drawing query language or database operation skills, which lowers the threshold for use. Through comprehensive coverage of mixed-type queries, it can simultaneously respond to complex needs such as technical specification consultation, parts traceability, and change impact analysis, avoiding multiple query operations for users. This effectively improves the efficiency of engineering design change assessment, maintenance and repair traceability, supply chain management, and other work, and reduces production costs and safety risks caused by query errors for enterprises. Attached Figure Description

[0018] In the attached diagram: Figure 1 This is a flowchart of an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments. Generally, the components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0020] Example 1 like Figure 1 As shown, a hybrid retrieval enhancement generation method for bills of materials includes the following steps: Step S1: Receive user queries; The system receives the query text input by the engineer through a web-based chat interface, converts it into a standardized string format, and uses it as the raw input for subsequent processing. Step S2: Using an AI intent classification model, classify user queries into one of the following: generalized questions, precise questions, or mixed questions. The AI ​​intent classification model adopts a dynamic adaptive training mechanism, which regularly updates the training set based on historical query logs, user query feedback data, and new query samples in a specific domain to achieve continuous optimization of classification accuracy. Step S3: Based on the classification results of step S2, execute the branch retrieval strategy: S301. If the query is a generalized question, the first context is retrieved from a vector database based on semantic similarity. The vector database adopts a dynamic threshold mechanism for semantic retrieval relevance, and the retrieval threshold for generalized questions is lower than the retrieval threshold for precise questions. S302. If the query is an exact question, multi-hop reasoning is performed from a bill of materials database to retrieve the second context. Multi-hop reasoning adopts an adaptive adjustment mechanism for reasoning depth, which dynamically adjusts the depth threshold of graph traversal according to the hierarchical depth of the bill of materials, the complexity of the query intent, and the historical retrieval results. S303. If the query is a mixed-type question, then execute steps S301 and S302 to retrieve the first context and the second context; Step S4: Combine the context retrieved in step S3 with the user query to construct enhanced hints; the fusion process adopts a context priority sorting rule, the second context corresponding to the structured facts of the bill of materials has a higher priority than the first context corresponding to the generalized document, and in mixed-type questions, the context weight related to the exact query part has a higher weight than the context weight related to the generalized query part. Step S5: Input the enhanced prompts into the large language model to generate an initial answer in response to the user's query; Step S6: Perform illusion verification on the initial answer: Extract the structured information related to the bill of materials from the initial answer, compare it with the original data in the bill of materials database, and if there is any inconsistency, correct the initial answer and finally output the target answer to suppress the illusion.

[0021] The dynamic adaptive training mechanism of the AI ​​intent classification model in step S2 specifically includes: Real-time collection of user feedback on classification results (including correct classification markers and incorrect classification correction markers); Every 10 days, new query samples and query samples corresponding to feedback information are added to the training set to incrementally fine-tune the artificial intelligence intent classification model. The preset period can be dynamically adjusted according to the query volume in a specific field. When the query volume is greater than the preset threshold, the period is shortened; when the query volume is less than the preset threshold, the period is extended.

[0022] The adaptive adjustment mechanism for inference depth in multi-hop inference in step S302 specifically includes: Pre-acquire the hierarchical depth data of each part node in the bill of materials database, and establish an initial mapping relationship between hierarchical depth and inference depth; For precise queries, analyze key information such as part numbers and assembly names in the query to locate the corresponding bill of materials nodes; If the level depth of the bill of materials node is greater than the first preset depth threshold, the inference depth is adjusted to the first depth value; if the level depth of the bill of materials node is less than the second preset depth threshold, and the second preset depth threshold is less than the first preset depth threshold, the inference depth is adjusted to the second depth value. If there are omissions or redundancies in the multi-hop reasoning results of this type of query in historical retrieval, the initial mapping relationship is dynamically corrected based on feedback.

[0023] The rule for setting the dynamic threshold for semantic retrieval relevance in step S301 is as follows: The retrieval threshold range for generalized questions is 0.6-0.75, and the retrieval threshold range for exact questions is 0.8-0.95. In mixed-type problems, a generalized threshold is used for retrieval of the generalized part, while a precise threshold is used for semantic retrieval of the precise part.

[0024] The context priority sorting rules in step S4 specifically include: The second context is sorted in descending order of its relevance to the exact part of the query, and the first N serialized entries are taken as the core context. The first context is sorted in descending order of semantic similarity, and the top M are taken as auxiliary contexts; The structure of enhanced suggestions is as follows: user query, core context, auxiliary context, and priority instructions (the instructions prioritize answering precise questions based on the core context, answer generalized questions based on the auxiliary context, and prohibit referencing information outside the context).

[0025] The hallucination verification in step S6 specifically includes the following sub-steps: S601. Use a named entity recognition model to extract bill of materials related entity information from the initial answer. The entity information includes part number, assembly number, subordinate relationship between parts and assemblies, and supplier information. S602. Convert the extracted entity information into a graph query language and execute the query in the bill of materials database; If the query results match the extracted entity information, that part of the content is retained; if the query results do not contain corresponding entity information or there is a conflict, that part of the content is deleted and marked as not supported by relevant bill of materials structured data. S603. Reorganize the revised content to generate the target answer.

[0026] The bill of materials database is a graph database, which can be any one of Neo4j, ArangoDB, or NebulaGraph.

[0027] The multi-hop reasoning in step S302 is a graph traversal query performed on the graph database to trace the hierarchical dependencies, assembly relationships, or supply chain dependencies represented by nodes and edges in the bill of materials.

[0028] The AI ​​intent classification model in step S2 is a classification model based on the Transformer architecture or a lightweight decision tree model. The training data includes domain-specific labeled query logs, dynamically fed-back query samples, and manually corrected classification error samples.

[0029] The vector database in step S301 contains text embeddings extracted from unstructured or semi-structured documents, including technical manuals, design specifications, maintenance logs, fault analysis reports, or industry standard documents; the vector database uses any one of Milvus, FAISS, Chroma, or Weaviate.

[0030] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications, improvements, and substitutions without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A hybrid retrieval-enhanced generation method for bills of materials, characterized in that, Includes the following steps: Step S1: Receive user query; Step S2: Using an artificial intelligence intent classification model, classify the user query into one of the following: a generalized problem, a precise problem, or a hybrid problem. The artificial intelligence intent classification model adopts a dynamic adaptive training mechanism, which regularly updates the training set based on historical query logs, user query feedback data, and new query samples in a specific domain to achieve continuous optimization of classification accuracy. Step S3: Based on the classification results of step S2, execute the branch retrieval strategy: S301. If the query is a generalized problem, then a first context is retrieved from a vector database based on semantic similarity; the vector database adopts a dynamic threshold mechanism for semantic retrieval relevance, and the retrieval threshold corresponding to the generalized problem is lower than the retrieval threshold corresponding to the precise problem. S302. If the query is a precise type question, then multi-hop reasoning is performed from a bill of materials database to retrieve a second context; the multi-hop reasoning adopts a reasoning depth adaptive adjustment mechanism, which dynamically adjusts the depth threshold of graph traversal according to the hierarchical depth of the bill of materials, the complexity of the query intent, and the historical retrieval results. S303. If the query is a mixed-type question, then execute steps S301 and S302 to retrieve the first context and the second context; Step S4: Combine the context retrieved in step S3 with the user query to construct an enhanced suggestion; The fusion process adopts a context priority sorting rule, in which the second context corresponding to the structured facts of the bill of materials has a higher priority than the first context corresponding to the generalized document, and in mixed-type problems, the context weight related to the exact query part has a higher weight than the context weight related to the generalized query part. Step S5: Input the enhanced prompts into a large language model to generate an initial answer in response to the user query; Step S6: Perform illusion verification on the initial answer: Extract the structured information related to the bill of materials from the initial answer, compare it with the original data in the bill of materials database, and if there is any inconsistency, correct the initial answer and finally output the target answer to suppress illusion.

2. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The dynamic adaptive training mechanism of the artificial intelligence intent classification model in step S2 specifically includes: Real-time collection of user feedback on classification results; Every 10 days, new query samples and query samples corresponding to feedback information are added to the training set to incrementally fine-tune the artificial intelligence intent classification model. The preset period can be dynamically adjusted according to the query volume in a specific field. When the query volume is greater than the preset threshold, the period is shortened; when the query volume is less than the preset threshold, the period is extended.

3. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The adaptive adjustment mechanism for inference depth in multi-hop inference in step S302 specifically includes: Pre-acquire the hierarchical depth data of each part node in the bill of materials database, and establish an initial mapping relationship between hierarchical depth and inference depth; For precise queries, analyze key information such as part numbers and assembly names in the query to locate the corresponding bill of materials nodes; If the level depth of the bill of materials node is greater than the first preset depth threshold, the inference depth is adjusted to the first depth value; if the level depth of the bill of materials node is less than the second preset depth threshold, and the second preset depth threshold is less than the first preset depth threshold, the inference depth is adjusted to the second depth value. If there are omissions or redundancies in the multi-hop reasoning results of this type of query in historical retrieval, the initial mapping relationship is dynamically corrected based on feedback.

4. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The rule for setting the dynamic threshold for semantic retrieval relevance in step S301 is as follows: The retrieval threshold range for generalized questions is 0.6-0.75, and the retrieval threshold range for exact questions is 0.8-0.

95. In mixed-type problems, a generalized threshold is used for retrieval of the generalized part, while a precise threshold is used for semantic retrieval of the precise part.

5. The hybrid retrieval enhancement generation method for bill of materials according to claim 4, characterized in that, The context priority sorting rules in step S4 specifically include: The second context is sorted in descending order of its relevance to the exact part of the query, and the first N serialized entries are taken as the core context. The first context is sorted in descending order of semantic similarity, and the top M are taken as auxiliary contexts.

6. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The hallucination verification in step S6 specifically includes the following sub-steps: S601. Use a named entity recognition model to extract bill of materials related entity information from the initial answer. The entity information includes part number, assembly number, subordinate relationship between parts and assemblies, and supplier information. S602. Convert the extracted entity information into a graph query language and execute the query in the bill of materials database; If the query results match the extracted entity information, then that part of the content is retained; If the query results do not contain corresponding entity information or contain conflicts, delete that part of the content and mark it as not supported by relevant bill of materials structured data; S603. Reorganize the revised content to generate the target answer.

7. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The bill of materials database is a graph database, which can be any one of Neo4j, ArangoDB, or NebulaGraph.

8. The hybrid retrieval enhancement generation method for bill of materials according to claim 7, characterized in that, The multi-hop reasoning in step S302 is a graph traversal query performed on the graph database, used to trace the hierarchical dependencies, assembly relationships, or supply chain dependencies represented by nodes and edges in the bill of materials.

9. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The artificial intelligence intent classification model in step S2 is a classification model based on the Transformer architecture or a lightweight decision tree model. The training data includes domain-specific labeled query logs, dynamically fed-back query samples, and manually corrected classification error samples.

10. The hybrid retrieval enhancement generation method for bill of materials according to claim 1, characterized in that, The vector database in step S301 contains text embeddings extracted from unstructured or semi-structured documents, including technical manuals, design specifications, maintenance logs, fault analysis reports, or industry standard documents; the vector database uses any one of Milvus, FAISS, Chroma, or Weaviate.

Images