Complex system decision support method and device based on hybrid retrieval enhancement generation

Abstract

The invention belongs to the field of artificial intelligence aided decision making, and particularly relates to a complex system decision supporting method and device based on hybrid retrieval enhancement generation. According to the method, a knowledge graph is constructed for documents in the complex system field, a semantic embedding vector of each text block is generated, double retrieval channels of a vector retrieval channel and a graph retrieval channel are designed, and parallel retrieval is performed on user query; wherein the vector retrieval channel calculates semantic similarity between user query and all text blocks according to semantic embedding vectors of all the text blocks, and the graph retrieval channel performs graph traversal on the knowledge graph and calculates matching degree scores of all the text blocks based on traversal results; sorting results of the retrieval results in the two retrieval channels are obtained according to the similarity or scores, fusion scores of all text blocks in the retrieval results are calculated through a reciprocal rank fusion algorithm, and a fusion result is obtained; and decision suggestions are generated based on the fusion result, so that the accuracy and reliability of emergency decision support of the complex system are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence-assisted decision-making, specifically relating to a method and apparatus for decision support of complex systems based on hybrid retrieval enhancement generation. Background Technology

[0002] Emergency decision-making scenarios in complex systems are characterized by sparse events, unpredictability, and severe consequences, making it extremely difficult to accumulate decision-making experience through real-world cases. Complex systems typically involve the interaction of multiple subsystems, nonlinear causal relationships, and dynamic evolutionary processes, requiring decision-making systems to perform complex causal reasoning and multivariate trade-offs based on specialized knowledge within a very short timeframe. In recent years, Large Language Models (LLMs) have demonstrated powerful capabilities in text understanding, logical reasoning, and knowledge generation, providing new approaches to solving knowledge-intensive emergency decision-making problems. However, general-purpose LLMs often face the following problems when dealing with highly specialized domains: 1) Knowledge illusion: The model may generate seemingly reasonable but actually erroneous specialized content; 2) Outdated knowledge: The timeliness limitations of pre-training data prevent the acquisition of the latest domain knowledge; 3) Lack of domain depth: The proportion of specialized domain knowledge in general training data is limited, making it difficult to support deep reasoning.

[0003] Retrieval-Augmented Generation (RAG) technology effectively alleviates the aforementioned problems by retrieving relevant external knowledge before decision generation. Existing RAG methods mainly fall into two categories: The first category is vector-based retrieval methods, which encode queries and documents as points in a high-dimensional vector space and use metrics such as cosine similarity for nearest neighbor search. This method has achieved success in tasks such as open-domain question answering and fact-checking, but it has the following limitations: it relies solely on semantic similarity, making it difficult to capture causal relationships between knowledge; it cannot utilize the structured connections of knowledge for multi-hop reasoning; and it easily misses knowledge that is semantically distant but logically strongly related. The second category is knowledge graph-based methods. For example, Chinese patent application CN120929785A discloses a building elevator detection and diagnosis decision-making method based on graph retrieval-augmented intelligent agents. This method uses graph retrieval-augmented generation for multi-dimensional retrieval and generates answers with reasoning paths based on the retrieved subgraph structure. Knowledge graph-based methods explicitly represent entities and relationships as nodes and edges, which can characterize the structured features of knowledge and support multi-hop reasoning based on relationship paths. However, a single graph retrieval strategy also has its shortcomings: graph retrieval may miss semantically similar knowledge that lacks explicit relational links; the quality of graph construction directly affects retrieval results, and errors in entity disambiguation and relation extraction will propagate downstream; and it is difficult to handle the flexibility and diversity of natural language expressions.

[0004] In summary, single-search enhancement strategies lack comprehensive knowledge coverage, impacting decision accuracy. Directly weighting and fusing the results of vector and graph search enhancements is prone to bias due to inconsistent scoring dimensions across different search channels, leading to poor fusion performance. Therefore, existing decision-making methods all result in low decision accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for decision support of complex systems based on hybrid retrieval enhancement, in order to solve the problem that the knowledge coverage of a single retrieval enhancement strategy is incomplete or the multi-source fusion effect is poor, resulting in low decision accuracy of complex systems.

[0006] To address the aforementioned technical problems, this invention provides a decision support method for complex systems based on hybrid retrieval enhancement, comprising: dividing a document in the domain of complex systems into text blocks, generating semantic embedding vectors for each text block, and constructing a knowledge graph based on the entities and relationships in each text block; inputting user queries into a vector retrieval channel and a graph retrieval channel in parallel to obtain retrieval results; the vector retrieval channel is used to calculate the semantic similarity between the user query and all text blocks based on the semantic embedding vectors of each text block, and selecting the top K results after sorting them in descending order of semantic similarity. v The text blocks are used as vector retrieval results; the graph retrieval channel is used to perform graph traversal of the knowledge graph using entities determined from the user query, and calculates the matching score of each text block based on the traversal results. The top K text blocks are then selected in descending order of matching score. g Each text block is used as a graph retrieval result; based on the ranking of each text block in the two retrieval channels, the reciprocal rank fusion algorithm is used to calculate the fusion score of each text block in the retrieval result, and the fusion result is obtained; decision suggestions are generated based on the fusion result.

[0007] Furthermore, the traversal result includes a set of seed entities identified in the knowledge graph using entities determined from user queries, and a set of extended entities obtained by expanding from the seed entities within the same community and through multi-hop neighbor expansion.

[0008] Furthermore, the matching score of each text block is determined based on the matching degree between the text block and the seed entities in the seed entity set, the matching degree between the text block and the extended entities in the extended entity set, and the semantic similarity between the text block and the user query.

[0009] Furthermore, when generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion results are first deduplicated. The deduplicated text blocks are then used as structured evidence, and decision recommendations are generated based on the structured evidence.

[0010] Furthermore, when generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to the first preset threshold in the fusion ranking list are first deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion score are selected for filtering. The deduplicated and filtered text blocks are used as structured evidence, and decision recommendations are generated based on the structured evidence.

[0011] Furthermore, when generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion ranking list are first deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion scores are selected for filtering. Meta-information is added to each text block after deduplication and filtering as structured evidence, and decision recommendations are generated based on the structured evidence. The meta-information includes the source chapter, page number, ranking result in the vector retrieval channel, and ranking result in the graph retrieval channel.

[0012] Furthermore, the community expansion from the seed entity to the community is achieved by modularizing the knowledge graph using a community detection algorithm and generating knowledge modules with community summary descriptions using a large language model.

[0013] Furthermore, the entities constructed in the knowledge graph are obtained after entity disambiguation using a strategy that combines rule matching and semantic similarity. The strategy is as follows: if the entity names are the same, they are directly determined to be the same entity; if the entity names are not completely the same, the cosine similarity of the description vectors corresponding to the two entities is calculated, and when the cosine similarity exceeds a second preset threshold, they are merged into the same entity.

[0014] Furthermore, generating decision recommendations based on structured evidence involves inputting structured evidence and user queries into a large language model, and then using the large language model to generate decision recommendations.

[0015] The beneficial effects of the above technical solution are as follows: This invention is an improved invention. By constructing a knowledge graph of a complex system's professional domain, it designs a dual-channel parallel retrieval mechanism of vector retrieval and graph retrieval. It adopts a reciprocal rank fusion algorithm to achieve parameter-free integration of the retrieval results of the two channels. Decisions are made based on the fusion results, thereby integrating the semantic capture capability of vector retrieval and the causal reasoning capability of graph retrieval. It can retrieve semantically similar operational norms and discover cross-domain causal dependencies, significantly improving the comprehensiveness of evidence recall. Fusion is based solely on text block ranking rather than original scores, avoiding the problem of inconsistent scoring dimensions in different retrieval channels. Effective fusion can be achieved without training data, reducing the method's dependence on labeled data. It provides high-quality and interpretable domain knowledge context for large language models, thereby improving the accuracy and reliability of emergency decision support for complex systems, and further improving the reliability and responsiveness of emergency decision support.

[0016] To address the aforementioned technical problems, this invention also provides a decision support device for complex systems based on hybrid retrieval enhancement generation, comprising a processor. The processor implements a decision support method for complex systems based on hybrid retrieval enhancement generation. This method includes: dividing a document in the complex system domain into text blocks, generating semantic embedding vectors for each text block, and constructing a knowledge graph based on the entities and relationships in each text block; inputting user queries into a parallel vector retrieval channel and a graph retrieval channel to obtain retrieval results; the vector retrieval channel is used to calculate the semantic similarity between the user query and all text blocks based on the semantic embedding vectors of each text block, and then selecting the top K results after sorting them in descending order of semantic similarity. v The text blocks are used as vector retrieval results; the graph retrieval channel is used to perform graph traversal of the knowledge graph using entities determined from the user query, and calculates the matching score of each text block based on the traversal results. The top K text blocks are then selected in descending order of matching score. g Each text block is used as a graph retrieval result; based on the ranking of each text block in the two retrieval channels, the reciprocal rank fusion algorithm is used to calculate the fusion score of each text block in the retrieval result, and the fusion result is obtained; decision suggestions are generated based on the fusion result.

[0017] Furthermore, the traversal result includes a set of seed entities identified in the knowledge graph using entities determined from user queries, and a set of extended entities obtained by expanding from the seed entities within the same community and through multi-hop neighbor expansion.

[0018] Furthermore, the matching score of each text block is determined based on the matching degree between the text block and the seed entities in the seed entity set, the matching degree between the text block and the extended entities in the extended entity set, and the semantic similarity between the text block and the user query.

[0019] Furthermore, when generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion results are first deduplicated. The deduplicated text blocks are then used as structured evidence, and decision recommendations are generated based on the structured evidence.

[0020] Furthermore, when generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to the first preset threshold in the fusion ranking list are first deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion score are selected for filtering. The deduplicated and filtered text blocks are used as structured evidence, and decision recommendations are generated based on the structured evidence.

[0021] Furthermore, when generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion ranking list are first deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion scores are selected for filtering. Meta-information is added to each text block after deduplication and filtering as structured evidence, and decision recommendations are generated based on the structured evidence. The meta-information includes the source chapter, page number, ranking result in the vector retrieval channel, and ranking result in the graph retrieval channel.

[0022] Furthermore, the community expansion from the seed entity to the community is achieved by modularizing the knowledge graph using a community detection algorithm and generating knowledge modules with community summary descriptions using a large language model.

[0023] Furthermore, the entities constructed in the knowledge graph are obtained after entity disambiguation using a strategy that combines rule matching and semantic similarity. The strategy is as follows: if the entity names are the same, they are directly determined to be the same entity; if the entity names are not completely the same, the cosine similarity of the description vectors corresponding to the two entities is calculated, and when the cosine similarity exceeds a second preset threshold, they are merged into the same entity.

[0024] Furthermore, generating decision recommendations based on structured evidence involves inputting structured evidence and user queries into a large language model, and then using the large language model to generate decision recommendations.

[0025] The beneficial effects of the above technical solution are as follows: This invention is an improved invention, providing a device that constructs a knowledge graph of a complex system's professional domain, designs a dual-channel parallel retrieval mechanism of vector retrieval and graph retrieval, and uses a reciprocal rank fusion algorithm to achieve parameter-free integration of the retrieval results of the two channels. Decisions are made based on the fusion results, thereby integrating the semantic capture capability of vector retrieval with the causal reasoning capability of graph retrieval. This not only retrieves semantically similar operational norms but also discovers cross-domain causal dependencies, significantly improving the comprehensiveness of evidence recall. Fusion is based solely on text block ranking rather than original scores, avoiding the problem of inconsistent scoring dimensions across different retrieval channels. Effective fusion can be achieved without training data, reducing the method's dependence on labeled data. This provides high-quality and interpretable domain knowledge context for large language models, thereby improving the accuracy and reliability of emergency decision support for complex systems, and ultimately enhancing the reliability and responsiveness of emergency decision support. Attached Figure Description

[0026] Figure 1 This is a flowchart of a complex system decision-making process based on hybrid retrieval enhancement, as presented in this invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0028] This invention designs a dual-channel parallel retrieval mechanism that combines vector retrieval and graph retrieval. It employs a reciprocal rank fusion algorithm to achieve parameter-free integration of the retrieval results from the two channels, and makes decisions based on the fusion results, thereby improving the accuracy of decision-making in complex systems.

[0029] Method Implementation

[0030] This invention provides a decision support method for complex systems based on hybrid retrieval and enhanced generation, applicable to decision support for complex systems in various professional fields, such as knowledge-intensive complex systems like medical diagnosis, legal consulting, engineering design, and emergency management. Figure 1 As shown, the method includes the following steps:

[0031] S1. Divide the documents in the domain of complex systems into text blocks, generate semantic embedding vectors for each text block, and construct a knowledge graph based on the entities and relationships in each text block.

[0032] 1. Document preprocessing and text block division.

[0033] Documents in the complex systems domain are preprocessed, including layout parsing, text cleaning, and structure extraction. Then, a sliding window strategy is used to divide the document into fixed-length text blocks, with overlapping areas between adjacent text blocks to maintain semantic continuity. Metadata such as source chapter, original text location information, and adjacent block indexes are added to each text block.

[0034] The text block size and overlapping area can be customized according to the specific document domain. For example, the text block length can be 1000-1500 characters, and the overlapping area between adjacent text blocks can be 150-250 characters. Preferably, the text block length is set to 1200 characters, and the overlapping area between adjacent text blocks is set to 200 characters. This configuration has been experimentally verified to balance contextual integrity and retrieval granularity. For technical documents, the text block size can be appropriately increased to retain more context; for highly structured documents, the text block size can be decreased to provide finer positioning. The overlapping area setting should ensure that entities and relationships across text blocks are not truncated.

[0035] 2. Construct a knowledge graph.

[0036] 1) Domain Entity and Relationship Pattern Design: Define the set of entity types and the set of relation types according to the domain characteristics of complex systems.

[0037] The entity type set includes categories such as equipment / systems, operations / procedures, personnel / roles, parameters / indicators, faults / anomalies, and environments / conditions. The equipment / system category refers to entities that include both equipment and systems; the others are similar. The relationship type set includes categories such as composition, operation, influence, dependency, cause, and processing.

[0038] 2) Entity and relation extraction based on large language models.

[0039] Construct a prompt template, which includes role settings, task scope, extraction principles, entity type and relation type definitions, output format constraints, and a few-sample example.

[0040] Guided by the prompt template, the large language model performs entity recognition and relation extraction on each text block to obtain structured output. The structured output obtained by the large language model includes a set of entities and a set of relations, with each relation represented as a (subject, relation type, object, description) quadruple.

[0041] 3) A strategy combining rule matching and semantic similarity is adopted to disambiguate entities and construct a unified knowledge graph.

[0042] In a knowledge graph, nodes represent entities, and edges represent relationships. The strategy for entity disambiguation is as follows: for two candidate entities, if their names are strictly identical, they are directly identified as the same entity (node); if their names are not completely identical, the cosine similarity of their corresponding description vectors is calculated. If the cosine similarity exceeds a second preset threshold... Entities are merged into the same entity (node). The knowledge graph after entity disambiguation is represented as follows: , where V is the set of nodes and E is the set of edges.

[0043] Furthermore, this invention organizes the knowledge graph into topic-specific modules through community detection to support topic-focused retrieval, thereby further improving retrieval relevance and efficiency.

[0044] A community detection algorithm is used to modularize the knowledge graph, identifying knowledge modules with clearly defined themes. Preferably, the Louvain algorithm can be used for community detection. By employing the Louvain algorithm to perform community detection on the knowledge graph, and iteratively optimizing the community affiliation of nodes based on the principle of maximizing modularity, a community set is obtained. A large language model is used to generate a summary description for each community, summarizing the community's themes, core entities, and main relationship patterns. In other words, communities are knowledge modules that are modularized using community detection algorithms and whose summary descriptions are generated using a large language model.

[0045] 3. Generate semantic embedding vectors for each text block.

[0046] A pre-trained language model is used to generate semantic embedding vectors for each text block. The semantic embedding vectors output by the pre-trained language model have a dimension of 2048-8192. Preferably, the pre-trained language model uses a high-dimensional embedding model of 4096 dimensions to obtain the semantic embedding vectors for each text block. semantic embedding vector set The generated high-dimensional semantic embedding vector is used for subsequent vector retrieval.

[0047] S2: Dual-channel parallel retrieval.

[0048] User queries are input in parallel into the vector retrieval channel and the graph retrieval channel to obtain retrieval results, which include both vector retrieval results and graph retrieval results.

[0049] 1. Vector retrieval channel.

[0050] This is used to calculate the semantic similarity between the user query and all text blocks based on the semantic embedding vectors of each text block, and then select the top K text blocks in descending order of semantic similarity. v Each text block is used as a vector retrieval result.

[0051] For the vector retrieval channel, given a user query q, the user query q is encoded into a query vector e using the same encoder (i.e., a pre-trained language model) as the semantic embedding vector of the text block. q Calculate the cosine similarity between the query vector and the semantic embedding vectors of all text blocks. Select the K with the highest similarity v The text blocks are used as the vector retrieval result set D vec .

[0052] K v The value range of K is set according to the size of the knowledge base and the complexity of the query. When the knowledge base is large, K can be appropriately increased. v The value K should be increased to improve recall; when query complexity is high, K should also be increased. v To obtain more sufficient evidence, we need to consider the value of K. v A value that is too large will increase computational overhead and the difficulty of subsequent fusion; a value between 15 and 25 is recommended. Preferably, K... v Set it to 20.

[0053] 2. Image retrieval channel.

[0054] This tool is used to perform graph traversal on a knowledge graph using entities identified from user queries, calculate the matching score for each text block based on the traversal results, and select the top K blocks after sorting them in descending order of matching scores. gEach text block serves as a graph retrieval result. The traversal results include a set of seed entities identified in the knowledge graph using entities determined from the user query, and a set of extended entities obtained by expanding from the seed entities through community expansion and multi-hop neighbor expansion.

[0055] First, seed entities existing in the knowledge graph are identified from the user query, serving as the starting point for graph traversal. Specifically, the names of graph entities appearing in the user query are searched, and the semantic similarity between the user query and the entity names and descriptions in the knowledge graph is calculated, resulting in a seed entity set V. seed .

[0056] Secondly, starting from the seed entities, graph expansion and knowledge aggregation are performed, including two methods: community expansion and multi-hop neighbor expansion, to obtain the expanded entity set V. expanded Regarding the neighbor expansion hop count, a larger hop count can uncover more indirect connections, but it may also introduce noise. For complex systems with long causal chains, the hop count can be appropriately increased; for knowledge-intensive but simple connections, a smaller hop count is recommended. A neighbor expansion hop count between 1 and 3 is suggested, for example, set to 2.

[0057] The matching score for each text block is determined based on its matching degree with the seed entities in the seed entity set, its matching degree with the extended entities in the extended entity set, and its semantic similarity to the user query. In one implementation, a comprehensive scoring function is designed to comprehensively consider the seed entity matching degree, extended entity matching degree, and semantic similarity of the text blocks to calculate the matching score for each text block, and the K with the highest comprehensive score is selected. g The text blocks are used as the graph retrieval result set D. graph The comprehensive scoring function is as follows:

[0058]

[0059] in, For text block c i Overall score For text block c i The collection of entities contained therein For user queries and text blocks c i The semantic similarity is calculated using α, β, and γ as weight parameters.

[0060] The weight parameters α, β, and γ reflect the relative importance of seed entity matching, extended entity matching, and semantic similarity, respectively. For applications emphasizing precise matching, α should be increased; for applications requiring the discovery of indirect associations, β should be increased; and for applications with high semantic understanding requirements, γ should be increased. It is recommended to maintain the order α > β > γ. The suggested values ​​for weight parameters α are 2.5-3.5, β is 0.8-1.2, and γ is 0.3-0.7. Preferably, α = 3.0, β = 1.0, and γ = 0.5.

[0061] Select the highest-rated K g The text blocks are used as the graph retrieval result set D. graph K g The value range of K is set according to the size of the knowledge base and the complexity of the query. When the knowledge base is large, K can be appropriately increased. g The value K should be increased to improve recall; when query complexity is high, K should also be increased. g To obtain more sufficient evidence, we need to consider the value of K. g A value that is too large will increase computational overhead and the difficulty of subsequent fusion; a value between 15 and 25 is recommended. Preferably, K... g Set it to 20.

[0062] S3: Evidence integration based on reciprocal rank fusion.

[0063] Based on the ranking of each text block in the two search channels in the search results, the reciprocal rank fusion algorithm is used to calculate the fusion score of each text block in the search results, and the dual-channel search results are fused to obtain the fusion result.

[0064] Specifically, collect the vector retrieval result set D vec And graph retrieval result set D graph All unique text blocks corresponding to the given text constitute the candidate set D. For each text block d in the candidate set, its rank (i.e., the sorting result) in both retrieval channels is recorded. vec (d) and rank graph (d) The reciprocal rank fusion algorithm is used to calculate the fusion score of each text block. The formula for calculating the fusion score using the reciprocal rank fusion algorithm is as follows:

[0065]

[0066] In the formula, rank r (d) represents the rank of text block d in retrieval channel r, and k is a smoothing parameter with a value ranging from 50 to 70. When a text block does not appear in a certain retrieval channel, the contribution of that channel to its fusion score is 0.

[0067] The RRF smoothing parameter k controls the degree to which ranking differences affect the fusion score. A larger k value compresses the score differences between top-ranked and bottom-ranked documents, resulting in a smoother fusion result; a smaller k value gives top-ranked documents a greater advantage, leading to a more concentrated fusion result. It is recommended to choose a value between 50 and 70 based on application needs. Preferably, k is set to 60.

[0068] The text blocks in the candidate set are sorted in descending order of their fusion scores to obtain the fusion sort list D. fused .

[0069] S4: Generate decision recommendations based on the fusion results.

[0070] 1. Evidence reconstruction and quality control.

[0071] 1) Semantic deduplication.

[0072] Text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion results are deduplicated. The deduplicated text blocks are used as structured evidence, and decision suggestions are generated based on the structured evidence.

[0073] In one implementation, the selected document set `selected` is initialized to an empty set, and the fusion sorting list `D` is traversed in descending order of fusion score. fused Each text block d in i Check d i The semantic similarity with each text block in the selected document set is used to determine if there is a similarity greater than or equal to a first preset threshold. If the maximum similarity is less than the first preset threshold, then skip the text block. , then d i Added to selected.

[0074] For the first preset threshold (i.e., the deduplication threshold), a high threshold indicates a conservative deduplication strategy, which would rather retain some redundancy than mistakenly delete valuable evidence; a low threshold indicates an aggressive deduplication strategy, which compresses the evidence size to the greatest extent possible. It is recommended to select a threshold between 0.85 and 0.95 based on the evidence quality requirements. Preferably, the first preset threshold is set to 0.9.

[0075] 2) Quantity filtering.

[0076] Text blocks in the fusion ranking list with semantic similarity greater than or equal to the first preset threshold are deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion score are selected for filtering. The deduplicated and filtered text blocks are used as structured evidence, and decision suggestions are generated based on the structured evidence.

[0077] For example, first, deduplication is performed on text blocks in the fusion ranking list whose semantic similarity is greater than or equal to a first preset threshold, resulting in a selected document set `selected`. If the number of text blocks in the selected document set `selected` after deduplication is greater than the maximum number N, then... max Then, select the top N documents with the highest fusion scores from the selected document set. max The text blocks are filtered to form the final evidence set D. filtered Decision suggestions are generated based on text blocks in the final evidence set after deduplication and filtering.

[0078] N max The final number of evidence pieces needs to strike a balance between evidence coverage and contextual efficiency. Too few pieces of evidence may miss key information, while too many pieces of evidence will increase the processing burden on large language models. It is recommended to choose between 6 and 10, depending on the complexity of the specific task. Preferably, N... max Set it to 8.

[0079] 3) Meta-information annotation.

[0080] Meta-information is added to each text block obtained in step 1) or step 2 to form a structured evidence representation D. structured Decision recommendations are generated based on structured evidence. Meta-information includes the source chapter, page number, ranking result in the vector retrieval channel, and ranking result in the graph retrieval channel.

[0081] This invention performs semantic deduplication, data filtering, and meta-information annotation on the obtained fusion results to reconstruct evidence. While ensuring evidence coverage, it controls the input scale and compresses the average retrieval results to reduce the processing burden of large language models, improve the reliability and interpretability of the generated results, avoid wasting context space with redundant information, and improve the quality of evidence.

[0082] 2. Decision generation.

[0083] Structured evidence and user queries are input into a large language model, which is then used to generate decision recommendations. This includes:

[0084] 1) Construct prompt template P final .

[0085] Design a prompt template P for generating professional decision-making advice. final Prompt template P final Including system prompt P system Evidence Context P evidence User query P query and task instruction P instruction Among them, the system prompts P system Used to define model roles and generation constraints, evidence context Pevidence Used to embed the structured evidence obtained after the above evidence reconstruction and quality control, user query P query Task instruction P is used to describe the specific problems or decision-making needs raised by the user. instruction Used to define the generation target and output format requirements.

[0086] 2) Generate decisions.

[0087] Please use the prompt template P final Input into a large language model to generate professional decision-making suggestions.

[0088] The generation parameters of the large language model can be set as follows: temperature parameter is 0.5-0.9, top_p parameter is 0.85-0.95, and the maximum number of generated tokens max_tokens is 1500-2500. For example, temperature = 0.7, top_p = 0.9, max_tokens = 2000.

[0089] Device Implementation

[0090] The present invention provides a decision support device for complex systems based on hybrid retrieval enhancement generation, comprising a processor, the processor being used to implement a decision support method for complex systems based on hybrid retrieval enhancement generation.

[0091] Specifically, the device includes a knowledge graph construction module, a dual-channel retrieval module, a fusion and reconstruction module, and a decision generation module. The knowledge graph construction module is used for preprocessing, entity relation extraction, entity disambiguation, community detection, and text block embedding of professional domain documents. The dual-channel retrieval module includes a vector retrieval submodule and a graph retrieval submodule, which perform evidence retrieval based on semantic similarity and knowledge graph structure, respectively. The fusion and reconstruction module is used for reciprocal rank fusion, semantic deduplication, quantity filtering, and meta-information annotation of the dual-channel retrieval results. The decision generation module is used to construct prompt templates and call a large language model to generate professional decision suggestions. The device also includes a storage module for storing the knowledge graph, text block embedding vectors, and retrieval index. The storage module uses a graph database to store the knowledge graph and a vector database to store the text block embedding vectors, supporting efficient graph traversal queries and vector similarity retrieval. It supports applications in knowledge-intensive decision support in fields such as medical diagnosis, legal consultation, engineering design, emergency management, financial analysis, or technical support. The method includes the following steps:

[0092] S1. The knowledge graph construction module divides the documents of complex system domains into text blocks, generates semantic embedding vectors for each text block, and constructs a knowledge graph based on the entities and relationships in each text block.

[0093] 1. Document preprocessing and text block division.

[0094] Documents in the complex systems domain are preprocessed, including layout parsing, text cleaning, and structure extraction. Then, a sliding window strategy is used to divide the document into fixed-length text blocks, with overlapping areas between adjacent text blocks to maintain semantic continuity. Metadata such as source chapter, original text location information, and adjacent block indexes are added to each text block.

[0095] Text block size and overlapping areas can be customized according to the specific document requirements. Overlapping area settings should ensure that entities and relationships across text blocks are not truncated.

[0096] 2. Construct a knowledge graph.

[0097] 1) Domain Entity and Relationship Pattern Design: Define the set of entity types and the set of relation types according to the domain characteristics of complex systems.

[0098] The entity type set includes categories such as equipment / systems, operations / procedures, personnel / roles, parameters / indicators, faults / anomalies, and environments / conditions. The equipment / system category refers to entities that include both equipment and systems; the others are similar. The relationship type set includes categories such as composition, operation, influence, dependency, cause, and processing.

[0099] 2) Entity and relation extraction based on large language models.

[0100] Construct a prompt template, which includes role settings, task scope, extraction principles, entity type and relation type definitions, output format constraints, and a few-sample example.

[0101] Guided by the prompt template, the large language model performs entity recognition and relation extraction on each text block to obtain structured output. The structured output obtained by the large language model includes a set of entities and a set of relations, with each relation represented as a (subject, relation type, object, description) quadruple.

[0102] 3) A strategy combining rule matching and semantic similarity is adopted to disambiguate entities and construct a unified knowledge graph.

[0103] In a knowledge graph, nodes represent entities, and edges represent relationships. The strategy for entity disambiguation is as follows: for two candidate entities, if their names are strictly identical, they are directly identified as the same entity (node); if their names are not completely identical, the cosine similarity of their corresponding description vectors is calculated. If the cosine similarity exceeds a second preset threshold... Entities are merged into the same entity (node). The knowledge graph after entity disambiguation is represented as follows: , where V is the set of nodes and E is the set of edges.

[0104] Furthermore, this invention organizes the knowledge graph into topic-specific modules through community detection to support topic-focused retrieval, thereby further improving retrieval relevance and efficiency.

[0105] A community detection algorithm is used to modularize the knowledge graph, identifying knowledge modules with clearly defined themes. Preferably, the Louvain algorithm can be used for community detection. By employing the Louvain algorithm to perform community detection on the knowledge graph, and iteratively optimizing the community affiliation of nodes based on the principle of maximizing modularity, a community set is obtained. A large language model is used to generate a summary description for each community, summarizing the community's themes, core entities, and main relationship patterns. In other words, communities are knowledge modules that are modularized using community detection algorithms and whose summary descriptions are generated using a large language model.

[0106] 3. Generate semantic embedding vectors for each text block.

[0107] Semantic embedding vectors are generated for each text block using a pre-trained language model, resulting in the text block... semantic embedding vector set The generated high-dimensional semantic embedding vector is used for subsequent vector retrieval.

[0108] S2: Input the user query into the dual-channel retrieval module in parallel. The dual-channel retrieval module performs a dual-channel parallel retrieval of the input user query using both vector retrieval and graph retrieval channels to obtain retrieval results, which include vector retrieval results and graph retrieval results.

[0109] 1. Vector retrieval channel.

[0110] This is used to calculate the semantic similarity between the user query and all text blocks based on the semantic embedding vectors of each text block, and then select the top K text blocks in descending order of semantic similarity. v Each text block is used as a vector retrieval result.

[0111] For the vector retrieval channel, given a user query q, the user query q is encoded into a query vector e using the same encoder (i.e., a pre-trained language model) as the semantic embedding vector of the text block. q Calculate the cosine similarity between the query vector and the semantic embedding vectors of all text blocks. Select the K with the highest similarity v The text blocks are used as the vector retrieval result set D vec .

[0112] 2. Image retrieval channel.

[0113] The knowledge graph is traversed using entities identified from user queries. A matching score is calculated for each text block based on the traversal results, and the top K blocks are selected in descending order of matching score.g Each text block serves as a graph retrieval result. The traversal results include a set of seed entities identified in the knowledge graph using entities determined from the user query, and a set of extended entities obtained by expanding from the seed entities through community expansion and multi-hop neighbor expansion.

[0114] First, seed entities existing in the knowledge graph are identified from the user query, serving as the starting point for graph traversal. Specifically, the names of graph entities appearing in the user query are searched, and the semantic similarity between the user query and the entity names and descriptions in the knowledge graph is calculated, resulting in a seed entity set V. seed .

[0115] Secondly, starting from the seed entities, graph expansion and knowledge aggregation are performed, including two methods: community expansion and multi-hop neighbor expansion, to obtain the expanded entity set V. expanded .

[0116] The matching score for each text block is determined based on its matching degree with the seed entities in the seed entity set, its matching degree with the extended entities in the extended entity set, and its semantic similarity to the user query. In one implementation, a comprehensive scoring function is designed to calculate the matching score for each text block. The text blocks are then ranked according to the scoring results. Taking into account the seed entity matching degree, extended entity matching degree, and semantic similarity, the K text block with the highest comprehensive score is selected. g The text blocks are used as the graph retrieval result set D. graph The comprehensive scoring function is as follows:

[0117]

[0118] in, For text block c i Overall score For text block c i The collection of entities contained therein For user queries and text blocks c i The semantic similarity is calculated using α, β, and γ as weight parameters.

[0119] Select the highest-rated K g The text blocks are used as the graph retrieval result set D. graph .

[0120] S3: Evidence integration based on reciprocal rank fusion.

[0121] Based on the ranking of each text block in the two search channels in the search results, the reciprocal rank fusion algorithm is used to calculate the fusion score of each text block in the search results, and the dual-channel search results are fused to obtain the fusion result.

[0122] Specifically, collect the vector retrieval result set D vecAnd graph retrieval result set D graph All unique text blocks corresponding to the given text constitute the candidate set D. For each text block d in the candidate set, its rank (i.e., the sorting result) in both retrieval channels is recorded. vec (d) and rank graph (d) The reciprocal rank fusion algorithm is used to calculate the fusion score of each text block. The formula for calculating the fusion score using the reciprocal rank fusion algorithm is as follows:

[0123]

[0124] In the formula, rank r (d) represents the rank of text block d in retrieval channel r, and k is a smoothing parameter with a value ranging from 50 to 70. When a text block does not appear in a certain retrieval channel, the contribution of that channel to its fusion score is 0.

[0125] The text blocks in the candidate set are sorted in descending order of their fusion scores to obtain the fusion sort list D. fused .

[0126] S4: Generate decision recommendations based on the fusion results.

[0127] 1. Evidence reconstruction and quality control.

[0128] 1) Semantic deduplication.

[0129] Text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion results are deduplicated. The deduplicated text blocks are used as structured evidence, and decision suggestions are generated based on the structured evidence.

[0130] In one implementation, the selected document set `selected` is initialized to an empty set, and the fusion sorting list `D` is traversed in descending order of fusion score. fused Each text block d in i Check d i The semantic similarity with each text block in the selected document set is used to determine if there is a similarity greater than or equal to a first preset threshold. If the maximum similarity is less than the first preset threshold, then skip the text block. , then d i Added to selected.

[0131] 2) Quantity filtering.

[0132] Text blocks in the fusion ranking list with semantic similarity greater than or equal to the first preset threshold are deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion score are selected for filtering. The deduplicated and filtered text blocks are used as structured evidence, and decision suggestions are generated based on the structured evidence.

[0133] First, deduplication is performed on text blocks in the fusion ranking list whose semantic similarity is greater than or equal to a first preset threshold, resulting in a selected document set. If the number of text blocks in the selected document set after deduplication is greater than the maximum number N, then... max Then, select the top N documents with the highest fusion scores from the selected document set. max The text blocks are filtered to form the final evidence set D. filtered Decision suggestions are generated based on text blocks in the final evidence set after deduplication and filtering.

[0134] 3) Meta-information annotation.

[0135] Meta-information is added to each text block obtained in step 1) or step 2 to form a structured evidence representation D. structured Decision recommendations are generated based on structured evidence. Meta-information includes the source chapter, page number, ranking result in the vector retrieval channel, and ranking result in the graph retrieval channel.

[0136] This invention performs semantic deduplication, data filtering, and meta-information annotation on the obtained fusion results to reconstruct evidence. While ensuring evidence coverage, it controls the input scale and compresses the average retrieval results to reduce the processing burden of large language models, improve the reliability and interpretability of the generated results, avoid wasting context space with redundant information, and improve the quality of evidence.

[0137] 2. Decision generation.

[0138] Structured evidence and user queries are input into a large language model, which is then used to generate decision recommendations. This includes:

[0139] 1) Construct prompt template P final .

[0140] Design a prompt template P for generating professional decision-making advice. final Prompt template P final Including system prompt P system Evidence Context P evidence User query P query and task instruction P instruction Among them, the system prompts P system Used to define model roles and generation constraints, evidence context P evidence Used to embed the structured evidence obtained after the above evidence reconstruction and quality control, user query P query Task instruction P is used to characterize the specific questions or decision-making needs raised by the user. instruction Used to define the generation target and output format requirements.

[0141] 2) Generate decisions.

[0142] Please use the prompt template P final Input into a large language model to generate professional decision-making suggestions.

[0143] The generation parameters of the large language model can be set as follows: temperature parameter is 0.5-0.9, top_p parameter is 0.85-0.95, and the maximum number of generated tokens max_tokens is 1500-2500.

[0144] The effectiveness of this invention will be illustrated below by taking an application scenario of providing emergency decision support for abnormal situations involving multiple coupled subsystems in a complex system as an example.

[0145] The knowledge base utilizes authoritative books in the field of complex systems, with a total text volume of approximately 200,000 words. After knowledge graph construction, a knowledge graph containing 545 entities, 699 relationships, and 33 topic communities was obtained.

[0146] Given a user query describing a specific abnormal scenario and parameter conditions, the task is to generate an emergency response plan. The specific process for executing the method of this invention is as follows:

[0147] In the vector retrieval stage, the query is encoded into a 4096-dimensional vector, and its cosine similarity with all text blocks is calculated. The 20 text blocks with the highest semantic similarity are retrieved, which mainly contain directly related operation specifications and parameter standards.

[0148] During the graph retrieval phase, five seed entities were identified, including devices, parameters, and anomaly types. Community expansion revealed 12 related entities on the same topic, and two-hop neighbor expansion revealed 18 associated entities on the causal chain. Twenty text blocks were retrieved, containing indirectly related systematic knowledge and causal dependencies.

[0149] During the RRF fusion phase, a total of 28 unique text blocks were retrieved across both channels, with 12 of these blocks appearing in both channels. Calculating the RRF score for each text block revealed that the key evidence ranked 8th in vector retrieval, 3rd in graph retrieval, and 2nd after fusion, successfully improving the ranking of important evidence.

[0150] During the evidence reconstruction stage, the semantic deduplication operation removes 5 highly similar documents, and selects the 8 documents with the highest RRF scores from the remaining 23 documents, adding meta-information annotations.

[0151] During the decision generation phase, a prompt containing eight pieces of structured evidence is constructed, input into a large language model to generate an emergency decision-making plan, and outputs a four-stage handling process, with each stage supported by professional evidence.

[0152] Verification has shown that this invention outperforms existing methods in four evaluation dimensions: operational compliance, logical rigor, dynamic evolution authenticity, and terminology accuracy. It improves performance by 13.7% compared to a baseline without RAG and by 4.8%-7.8% compared to a single retrieval strategy, fully validating the effectiveness of the dual-channel hybrid retrieval and reciprocal rank fusion mechanism.

[0153] This invention constructs a domain knowledge graph and designs a dual-channel parallel retrieval mechanism, integrating the semantic capture capability of vector retrieval with the causal reasoning capability of graph retrieval. This not only retrieves semantically similar operational norms but also discovers cross-domain causal dependencies, significantly improving the comprehensiveness of evidence recall. Experimental data shows that evidence coverage increased from 65% in single-channel retrieval to 85%, an improvement of 30.8%. During graph retrieval, community detection organizes the knowledge graph into topic-specific modules, supporting topic-focused retrieval and improving retrieval relevance and efficiency. A reciprocal rank fusion algorithm is employed to achieve parameter-free integration. Fusion is based solely on text block ranking rather than raw scores, avoiding inconsistencies in scoring dimensions across different retrieval channels. Effective fusion can be achieved without training data, reducing the method's dependence on labeled data. Through an evidence reconstruction process involving semantic deduplication, quantity filtering, and meta-information annotation, the system controls the input scale while ensuring evidence coverage. For example, the average number of search results is compressed from 40 to 8, achieving a compression rate of 80%. This effectively reduces the processing burden on large language models, improves the reliability and interpretability of generated results, and provides high-quality and interpretable domain knowledge context for large language models. This enhances the accuracy and reliability of emergency decision support for complex systems, achieving a balance between accuracy and interpretability in complex systems facing data-sparse and knowledge-intensive decision-making scenarios, thereby improving the reliability and responsiveness of emergency decision support. The methodology is universal, does not rely on domain-specific labeled data, and can be quickly adapted to different professional fields. It has application value in decision support for knowledge-intensive complex systems such as medical diagnosis, legal consultation, engineering design, and emergency management.

Claims

1. A decision support method for complex systems based on hybrid retrieval enhancement, characterized in that, include: Documents in the domain of complex systems are divided into text blocks, semantic embedding vectors of each text block are generated, and a knowledge graph is constructed based on the entities and relationships in each text block. The user query is input into both the vector retrieval channel and the graph retrieval channel in parallel to obtain the retrieval results. The vector retrieval channel is used to calculate the semantic similarity between the user query and all text blocks based on the semantic embedding vectors of each text block, and then selects the results after sorting them in descending order of semantic similarity. K v Each text block is used as a vector retrieval result; The graph retrieval channel is used to perform graph traversal on the knowledge graph using entities identified from user queries, and calculates the matching score for each text block based on the traversal results. The top-ranked text blocks are then selected in descending order of their matching scores. K g Each text block is used as a graph retrieval result; Based on the ranking of each text block in the two search channels in the search results, the reciprocal rank fusion algorithm is used to calculate the fusion score of each text block in the search results to obtain the fusion result. Decision recommendations are generated based on the fusion results.

2. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 1, characterized in that, The traversal results include a set of seed entities identified in the knowledge graph using entities determined from user queries, and a set of extended entities obtained by expanding from the seed entities within the same community and through multi-hop neighbor expansion.

3. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 2, characterized in that, The matching score of each text block is determined based on the matching degree between the text block and the seed entities in the seed entity set, the matching degree between the text block and the extended entities in the extended entity set, and the semantic similarity between the text block and the user query.

4. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 1, characterized in that, When generating decision recommendations based on the fusion results, the text blocks with semantic similarity greater than or equal to the first preset threshold in the fusion results are first deduplicated. The deduplicated text blocks are then used as structured evidence, and decision recommendations are generated based on the structured evidence.

5. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 1, characterized in that, When generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to the first preset threshold in the fusion ranking list are first deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion score are selected for filtering. The deduplicated and filtered text blocks are used as structured evidence, and decision recommendations are generated based on the structured evidence.

6. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 5, characterized in that, When generating decision recommendations based on the fusion results, text blocks with semantic similarity greater than or equal to a first preset threshold in the fusion ranking list are first deduplicated. When the number of deduplicated text blocks is greater than the maximum number, the text blocks with the highest fusion scores are selected for filtering. Meta-information is added to each text block after deduplication and filtering as structured evidence, and decision recommendations are generated based on the structured evidence. The meta-information includes the source chapter, page number, ranking result in the vector retrieval channel, and ranking result in the graph retrieval channel.

7. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 2, characterized in that, Starting from seed entities, the community expansion within the same community involves modularizing the knowledge graph using community detection algorithms and generating knowledge modules that provide a summary description of the community using a large language model.

8. The decision support method for complex systems based on hybrid retrieval enhancement generation according to claim 1, characterized in that, The entities constructed in the knowledge graph are obtained by disambiguation using a strategy that combines rule matching and semantic similarity. The strategy is as follows: if the names of the entities are the same, they are directly determined to be the same entity; if the names of the entities are not completely the same, the cosine similarity of the description vectors corresponding to the two entities is calculated, and when the cosine similarity exceeds a second preset threshold, they are merged into the same entity.

9. The decision support method for complex systems based on hybrid retrieval enhancement generation according to any one of claims 4-6, characterized in that, Decision suggestion generation based on structured evidence involves inputting structured evidence and user queries into a large language model, and then using the large language model to generate decision suggestions.

10. A decision support device for complex systems based on hybrid retrieval enhancement generation, comprising a processor, characterized in that, The processor is used to implement the complex system decision support method based on hybrid retrieval enhancement generation as described in any one of claims 1-9.

Images