Large model interpretable question and answer method and system based on knowledge graph fusion

Abstract

The invention provides a large model interpretable question answering method and system based on knowledge graph fusion, and the method comprises the steps: obtaining related knowledge of a question text in a knowledge graph, and taking the embedding of the related knowledge as static embedding; performing dynamic interactive exploration in the knowledge graph by utilizing the initial large model based on the problem text and static embedding to obtain a knowledge sub-graph, and fusing the knowledge sub-graph into the initial large model to obtain a large model in which external knowledge is introduced; inputting the question text into the large model for question-answer decision and outputting a question-answer result; determining a reasoning process based on attention distribution and an intermediate hidden state in a large model question and answer decision process; according to the method, related knowledge in the knowledge graph is obtained through static embedding and dynamic interactive retrieval, and the decision-making efficiency and accuracy of a large model are improved; reasoning is carried out through an attention mechanism, input data with high attention can be compared with entities and relations in the knowledge sub-graph, and the interpretability of the question-answering system and the credibility of the question-answering system on a large model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of large-model question answering, and in particular to a large-model explainable question answering method and system based on knowledge graph fusion. Background Art

[0002] As an important branch in the field of natural language processing, question-answering technology has evolved from a dedicated question-answering system based on a structured database, to a general question-answering system based on a large document set, and then to a question-answering system based on a large language model. The question-answering technology of large models has gradually demonstrated powerful semantic representation and complex reasoning capabilities. With the in-depth study of large model question-answering technology, researchers have realized that relying solely on the knowledge base of the large model for reasoning is limited, so they began to explore how to introduce external knowledge such as knowledge graphs to enhance the reasoning ability of large models. The research at this stage mainly focuses on knowledge representation, knowledge graphs, knowledge fusion, etc., aiming to introduce external knowledge into large models in an appropriate way to improve the reasoning performance and accuracy of the model. With the continuous development of artificial intelligence technology, large models have made significant progress in the field of natural language processing.

[0003] However, the internal working mechanism of these large models is still a "black box", and it is difficult to explain their question-answering decision-making process and reasoning logic. In order to improve the transparency and credibility of large models, researchers have begun to focus on the interpretability verification technology of large models.

[0004] In the early days of large models, researchers began to explore how to explain the decision-making process of the model. Early attempts focused on visualizing the model's attention weights to show the parts of the text that the model focuses on when processing a specific task. As research deepened, a series of tools and methods for large model interpretability verification were developed. These tools and methods include saliency methods, attention attribution, and counterfactual analysis.

[0005] Saliency methods and attention attribution reveal the basis of model decisions by analyzing the model's attention to different parts of the input data; counterfactual analysis infers the model's decision logic by changing certain parts of the input data and observing changes in the model output. However, these existing interpretability methods can often only provide surface explanations of model decisions, focusing more on the processing of input and output data, and cannot go deep into the calculation process and logical structure inside the model; therefore, it is difficult to understand the model's decision basis, which affects the trust in the model in the question-answering system. Summary of the invention

[0006] In order to overcome the above-mentioned deficiencies of the prior art, the present invention proposes a large-model explainable question answering method based on knowledge graph fusion, comprising:

[0007] Obtain relevant knowledge of the question text in the knowledge graph, and use the embedding of the relevant knowledge as static embedding;

[0008] Using the initial large model, based on the question text and static embedding, dynamic interactive exploration is performed in the knowledge graph to obtain a knowledge subgraph, and the knowledge subgraph is integrated into the initial large model to obtain a large model that introduces external knowledge;

[0009] Inputting the question text into the large model for question-answering decision, and outputting the question-answering result of the question text;

[0010] Based on the attention distribution and intermediate hidden states of the large model during the question-answering decision process, the reasoning process of the large model during question-answering decision-making is determined.

[0011] Optionally, the initial large model is used to perform dynamic interactive exploration in the knowledge graph based on the question text and static embedding to obtain a knowledge subgraph, including:

[0012] The question text and the static embedding are spliced ​​and aligned and then input into the initial large model;

[0013] Using the question text and static embedding at the input end of the initial large model, determine the exploration starting point of the initial large model in the knowledge graph, recorded as an initial node;

[0014] Dynamically generate initial exploration directions using the initial large model to obtain new nodes, and construct an initial subgraph in combination with the initial nodes; continue to explore based on the text questions, static embedding and the initial subgraph, and expand the initial subgraph to obtain a new subgraph;

[0015] Based on the semantic similarity between the question text and the new subgraph, pruning the new subgraph to obtain a pruned subgraph;

[0016] Continue exploring on the basis of the pruned subgraph until the preset exploration depth and width are reached, stop expanding, and obtain the final knowledge subgraph.

[0017] Optionally, integrating the knowledge subgraph into the initial large model to obtain a large model that introduces external knowledge includes:

[0018] Inputting the knowledge subgraph into the initial large model;

[0019] The knowledge subgraph and the implicit knowledge of the initial large model are concatenated or weightedly integrated to obtain a large model that introduces external knowledge;

[0020] The implicit knowledge of the initial large model itself is the knowledge learned based on the training corpus when the initial large model performs preliminary question-answering decision training.

[0021] Optionally, determining the reasoning process of the large model during question-answering decision-making based on the attention distribution and intermediate hidden states of the large model during the question-answering decision-making process includes:

[0022] Obtaining the attention weight and intermediate hidden state of each word in the input sequence corresponding to the question text during the large model question answering decision process;

[0023] Determine the attention degree of each word in the input sequence when making a decision based on the attention weight; determine the decision process based on the intermediate hidden state;

[0024] The reasoning process of the large model when making question-answering decisions is determined by combining the decision-making process, the attention paid to each word in the input sequence, the implicit knowledge of the large model itself, and the knowledge subgraph.

[0025] Optionally, the step of obtaining relevant knowledge of the question text in the knowledge graph includes:

[0026] extracting keywords from the question text;

[0027] Using keyword matching technology, by matching keywords in the question text with knowledge content in the knowledge graph, relevant knowledge of the question text in the knowledge graph is obtained;

[0028] The knowledge content of the knowledge graph includes entities, relationships and attributes.

[0029] Optionally, after inputting the question text into the large model for question-answering decision and outputting the question-answering result of the question text, the method further includes:

[0030] Compare the question-answer result with the knowledge in the knowledge graph to obtain a comparison result;

[0031] The comparison result is combined with preset judgment rules to determine whether the large model has hallucination problems or generates errors during the reasoning process, so as to verify the question-answering results.

[0032] Optionally, before obtaining the relevant knowledge of the question text in the knowledge graph, the method further includes:

[0033] Natural language processing technology is used to automatically update the knowledge in the knowledge graph based on a preset update frequency and update content range.

[0034] Optionally, before obtaining the relevant knowledge of the question text in the knowledge graph, the method further includes:

[0035] The knowledge in the knowledge graph is transferred across domains to obtain the knowledge graph after the knowledge transfer.

[0036] Optionally, before inputting the question text into the large model for question-answering decision, the method further includes:

[0037] The language model of the large model that has completed concept alignment is detected by LPAQA or LAMA to verify the accuracy of the large model's understanding of knowledge.

[0038] Based on the same inventive concept, the present invention proposes a large model explainable question answering system based on knowledge graph fusion, including:

[0039] The external knowledge introduction module is used to obtain the relevant knowledge of the question text in the knowledge graph, and embed the relevant knowledge as static embedding; using the initial large model based on the question text and the static embedding, a dynamic interactive exploration is performed in the knowledge graph to obtain a knowledge subgraph, and the knowledge subgraph is integrated into the initial large model to obtain a large model that introduces external knowledge;

[0040] A question-answering decision module, used to input the question text into the large model for question-answering decision and output the question-answering result of the question text;

[0041] The decision reasoning module is used to determine the reasoning process of the large model during question-answering decision-making based on the attention distribution and intermediate hidden states of the large model during the question-answering decision-making process.

[0042] Optionally, the external knowledge introduction module is specifically used to:

[0043] The question text and the static embedding are spliced ​​and aligned and then input into the initial large model;

[0044] Using the question text and static embedding at the input end of the initial large model, determine the exploration starting point of the initial large model in the knowledge graph, recorded as an initial node;

[0045] Dynamically generate initial exploration directions using the initial large model to obtain new nodes, and construct an initial subgraph in combination with the initial nodes; continue to explore based on the text questions, static embedding and the initial subgraph, and expand the initial subgraph to obtain a new subgraph;

[0046] Based on the semantic similarity between the question text and the new subgraph, pruning the new subgraph to obtain a pruned subgraph;

[0047] Continue exploring on the basis of the pruned subgraph until the preset exploration depth and width are reached, stop expanding, and obtain the final knowledge subgraph.

[0048] Optionally, the external knowledge introduction module is specifically used to:

[0049] Inputting the knowledge subgraph into the initial large model;

[0050] The knowledge subgraph and the implicit knowledge of the initial large model are concatenated or weightedly integrated to obtain a large model that introduces external knowledge;

[0051] The implicit knowledge of the initial large model itself is the knowledge learned based on the training corpus when the initial large model performs preliminary question-answering decision training.

[0052] Optionally, the decision reasoning module is specifically used to:

[0053] Obtaining the attention weight and intermediate hidden state of each word in the input sequence corresponding to the question text during the large model question answering decision process;

[0054] Determine the attention degree of each word in the input sequence when making a decision based on the attention weight; determine the decision process based on the intermediate hidden state;

[0055] The reasoning process of the large model when making question-answering decisions is determined by combining the decision-making process, the attention paid to each word in the input sequence, the implicit knowledge of the large model itself, and the knowledge subgraph.

[0056] Optionally, the external knowledge introduction module is specifically used to:

[0057] extracting keywords from the question text;

[0058] Using keyword matching technology, by matching keywords in the question text with knowledge content in the knowledge graph, relevant knowledge of the question text in the knowledge graph is obtained;

[0059] The knowledge content of the knowledge graph includes entities, relationships and attributes.

[0060] Optionally, the system further comprises a verification module, configured to:

[0061] After the question text is input into the big model for question-answering decision and the question-answering result of the question text is output, the question-answering result is compared with the knowledge in the knowledge graph to obtain a comparison result;

[0062] The comparison result is combined with preset judgment rules to determine whether the large model has hallucination problems or generates errors during the reasoning process, so as to verify the question-answering results.

[0063] Optionally, the system further includes a data preprocessing module, configured to:

[0064] Before obtaining the relevant knowledge of the question text in the knowledge graph, natural language processing technology is used to automatically update the knowledge in the knowledge graph based on a preset update frequency and update content range.

[0065] Optionally, the data preprocessing module is further used to:

[0066] Before obtaining the relevant knowledge of the question text in the knowledge graph, cross-domain knowledge migration is performed on the knowledge in the knowledge graph to obtain the knowledge graph after knowledge migration.

[0067] Optionally, the system further includes a model detection module, which is used to:

[0068] Before inputting the question text into the big model for question-answering decision, the big model that has completed concept alignment is subjected to language model detection using LPAQA or LAMA to verify the accuracy of the big model's understanding of knowledge.

[0069] In another aspect, the present application further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0070] The memory is used to store one or more programs;

[0071] When the one or more programs are executed by the at least one processor, a large-model explainable question-answering method based on knowledge graph fusion as described above is implemented.

[0072] On the other hand, the present application also provides a computer-readable storage medium having an execution program stored thereon. When the execution program is executed, it implements a large-model explainable question-answering method based on knowledge graph fusion as described above.

[0073] Compared with the closest prior art, the present invention has the following beneficial effects:

[0074] The present invention provides a large-model explainable question-answering method and system based on knowledge graph fusion, including: obtaining relevant knowledge of the question text in the knowledge graph, and embedding the relevant knowledge as static embedding; using the initial large model to perform dynamic interactive exploration in the knowledge graph based on the question text and static embedding to obtain a knowledge subgraph, integrating the knowledge subgraph into the initial large model, and obtaining a large model that introduces external knowledge; inputting the question text into the large model for question-answering decision-making, and outputting the question-answering result of the question text; determining the question text based on the attention distribution and intermediate hidden state of the large model in the question-answering decision-making process. The present invention describes the reasoning process of the large model when making question-answering decisions; static embedding is used to guide the large model to focus on task-related areas, and then more relevant knowledge in the knowledge graph is obtained through dynamic interactive retrieval, thereby avoiding information redundancy and improving the decision-making efficiency and accuracy of the question-answering system; the entities and relationships in the knowledge graph are clearly defined, so when the model based on knowledge fusion is inferred through the attention mechanism, the input data with high attention can be compared with the entities and relationships in the knowledge subgraph, and each step of the decision can correspond to clear and specific entities and relationships, thereby improving the interpretability of the question-answering system and the trust in the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A flowchart of a large-model explainable question-answering method based on knowledge graph fusion provided by the present invention;

[0076] Figure 2 A schematic diagram of the structure of a large-model explainable question-answering system based on knowledge graph fusion provided by the present invention;

[0077] Figure 3 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0078] The above-mentioned knowledge graph (KG) is a data structure that represents the relationship between entities and is used to solve various problems, such as recommendation systems, question-answering systems, and semantic search. With the vigorous development of artificial intelligence, key issues such as knowledge extraction, representation, fusion, reasoning, and question-answering involved in knowledge graphs have been solved and broken through to a certain extent, and knowledge graphs have become a new hot spot in the field of knowledge services. Knowledge graph technology originated from the 1950s to the 1970s: In 1955, Garfield proposed the idea of ​​applying citation indexes to retrieve documents; in 1965, citation networks (citation relationships between scientific documents) were regarded as similar to the "topographic map" of contemporary scientific development, and analyzing citation networks began to become a common method for studying the context of contemporary scientific development, thus forming the concept of knowledge graphs; in 1968, Quillian (JRQuillian) proposed semantic networks, which were originally proposed as an obvious axiomatic model of human associative memory, and were subsequently used in AI for natural language understanding to represent propositional information. Semantic networks are a form of expressing human knowledge structures in a network format, and are one of the representation methods used by artificial intelligence programs.

[0079] Developed from the 1970s to the early 21st century: In 1977, at the Fifth International Conference on Artificial Intelligence, American computer scientist B.A. Feigenbaum first proposed the concept of knowledge engineering. Knowledge engineering is a system that solves user questions by storing existing knowledge. Among them, the most typical and successful application of knowledge engineering is the rule-based expert system. Since then, knowledge base systems represented by expert systems have begun to be widely studied and applied. In 1991, American computer expert R. Niches and others proposed a new idea for building intelligent systems while completing a scientific research project on knowledge sharing for the Defense Advanced Research Projects Agency (DARPA) of the United States Department of Defense. The intelligent system consists of two parts: "ontologies" and "problem solving methods" (PSMs). Ontologies are the core of the knowledge base, involving knowledge structures shared in a specific field, and are static knowledge; the latter involves reasoning knowledge in the corresponding field, and is dynamic knowledge. In 2002, the concept of institutional knowledge base was proposed, and knowledge representation and knowledge organization began to be studied in depth and widely applied to the data collation work of various institutions.

[0080] Prosperity in the 2010s ~ till now: In 2012, Google proposed Google Knowledge Graph, and the knowledge graph was officially named. Google improved the performance of search engines through knowledge graph technology. With the vigorous development of artificial intelligence, key issues such as knowledge extraction, representation, fusion, reasoning and question-answering involved in knowledge graphs have been solved and broken through to a certain extent, and knowledge graphs have become a new hot spot in the field of knowledge services.

[0081] It is mentioned above that external knowledge such as knowledge graphs are introduced into large models to enhance the reasoning ability of large models. The origin of large models can be traced back to the early development of deep learning, especially the extensive research and application of artificial neural networks (ANN). With the development of computing hardware, especially the popularization of GPUs (Graphics Processing Units), and the acquisition of massive labeled data, deep learning has gradually become a key technology for realizing artificial intelligence. On this basis, large models, that is, large deep learning models based on massive data and computing resources, have been able to develop rapidly and have made significant breakthroughs in natural language processing, computer vision, speech recognition and other fields. With the deepening of research, it is realized that relying solely on the knowledge base of the large model for reasoning is limited, so we began to explore how to introduce external knowledge such as knowledge graphs to enhance the reasoning ability of large models.

[0082] At present, when verifying the interpretability of large models, in addition to the methods mentioned above, visualization tools and explanatory libraries are also included. Visualization tools (such as TransformerViz and LLM Visualization) display the internal working mechanism of the model in an intuitive way, which helps researchers understand the decision-making process of the model; explanatory libraries (such as TransformerLens and Inseq) provide a series of tools and methods for analyzing and explaining the internal working mechanism of the Transformer model. The research history of large model interpretability verification technology based on language model analysis shows that this field has gone through multiple stages from early exploration to technological progress and method development to theoretical research and practical application. In the future, with the continuous advancement of technology and the continuous expansion of application scenarios, research in this field will pay more attention to practicality and innovation to promote the further development of artificial intelligence technology.

[0083] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0084] Example 1

[0085] The present invention provides a large model explainable question answering method based on knowledge graph fusion, such as Figure 1 As shown, including:

[0086] S1. Obtain relevant knowledge of the question text in the knowledge graph, and use the embedding of the relevant knowledge as static embedding;

[0087] S2. Using the initial large model to perform dynamic interactive exploration in the knowledge graph based on the question text and static embedding to obtain a knowledge subgraph, and integrating the knowledge subgraph into the initial large model to obtain a large model that introduces external knowledge;

[0088] S3, inputting the question text into the large model for question-answering decision, and outputting the question-answering result of the question text;

[0089] S4. Based on the attention distribution and intermediate hidden states of the large model during the question-answering decision process, determine the reasoning process of the large model during the question-answering decision process.

[0090] In step S1, relevant knowledge of the question text in the knowledge graph is obtained, and the embedding of the relevant knowledge is used as static embedding.

[0091] External knowledge bases usually exist in a structured form, such as knowledge graphs, databases, or knowledge bases in professional fields. The entities, relationships, and attributes in these knowledge bases are represented and embedded so that they can be integrated with the representation space of the large model. This includes mapping the entities and relationships in the knowledge base into low-dimensional vector representations, or using techniques such as graph neural networks to embed the initial subgraph.

[0092] Specifically, the plug-in knowledge base in the present invention selects a knowledge graph (which may also be a database or a knowledge base in a professional field), and vectorizes and embeds entities, relationships, and attributes in the knowledge graph;

[0093] Extract keywords from the question text; use keyword matching technology to match the keywords in the question text with the knowledge content in the knowledge graph to obtain relevant knowledge of the question text in the knowledge graph, where the knowledge content of the knowledge graph includes entities, relationships and attributes; embed the relevant knowledge in the knowledge graph as static embedding.

[0094] The information in the knowledge graph and knowledge base may change over time, and it is necessary to deal with the update of the knowledge base and dynamic knowledge management, including timely acquisition of the latest knowledge, automatic update of the information in the knowledge base, and handling of conflicts or inconsistencies in the knowledge base. That is, before obtaining the relevant knowledge of the question text in the knowledge graph, natural language processing technology is used to automatically update the knowledge in the knowledge graph based on the pre-set update frequency and update content range.

[0095] Enable the knowledge base to keep pace with the big model and adapt to new knowledge changes.

[0096] Specifically, clarify the specific requirements and goals of knowledge base updates. This may include the frequency of updates, the scope of updated content, and the quality requirements of updates. These requirements and goals will guide subsequent automated update strategies and technology selection.

[0097] Utilize natural language processing technology to analyze, understand, and generate text that is close to human expression. By applying natural language processing technology, intelligent analysis and processing of knowledge base content can be achieved, including information extraction, sentiment analysis, language models, etc., to support automated content generation and updating. Set a clear content framework, including elements such as the type, structure, style, and tone of knowledge. This will provide a guideline for automated tools to ensure that the generated content meets predetermined standards and requirements. Develop specific update rules and strategies based on the characteristics and needs of the knowledge base. For example, the update frequency and priority of certain information can be set to ensure that the information in the knowledge base is always up to date. Finally, implement an automated update process, conduct in-depth analysis of the captured information through technologies such as NLP, extract useful knowledge and data, and integrate them into the knowledge base (knowledge graph).

[0098] The external knowledge base may cover knowledge from multiple fields, so cross-domain knowledge transfer is needed. Apply knowledge from one field to reasoning tasks in another field. Design adaptive methods to appropriately transform and apply cross-domain knowledge, thereby improving the reasoning ability of large models in new fields.

[0099] That is, before obtaining the relevant knowledge of the question text in the knowledge graph, it also includes: performing cross-domain knowledge migration on the knowledge in the knowledge graph to obtain the knowledge graph after knowledge migration.

[0100] In step S2, S2-1, using the initial large model based on the question text and static embedding, dynamic interactive exploration is performed in the knowledge graph to obtain a knowledge subgraph.

[0101] Based on the above question text, relevant subgraphs are extracted from the knowledge graph using entity-based keyword extraction, relationship-based upstream and downstream extraction, and field / topic-based extraction methods as knowledge subgraphs. That is, based on the initial static embedding, a dynamic interactive exploration mechanism between the large model and the knowledge graph is designed to obtain knowledge subgraphs in order to obtain knowledge with higher relevance.

[0102] Specifically, the question text and the static embedding are spliced ​​and aligned and then input into the initial large model, that is, at the input end of the initial large model, the question text and the static embedding are spliced ​​or weighted added to align the question text and the static embedding; the static embedding can constrain the attention distribution and generation process of the large model, and guide it to focus on the knowledge area related to the task;

[0103] Using the question text and static embedding at the input end of the initial large model, determine the exploration starting point (such as related entities) of the initial large model in the knowledge graph, which is recorded as an initial node;

[0104] Using the initial large model to dynamically generate initial exploration directions (new entities, relationships) to obtain new nodes, and combining the initial nodes to construct an initial subgraph; continuing to explore based on the text questions, static embeddings and initial subgraphs, and expanding the initial subgraph to obtain new subgraphs;

[0105] Based on the semantic similarity (or knowledge importance) between the question text and the new subgraph, pruning the new subgraph to obtain a pruned subgraph;

[0106] Continue exploring on the basis of the pruned subgraph until the preset exploration depth and width are reached, stop expanding, and obtain the final knowledge subgraph.

[0107] By expanding the nodes and relationships in the subgraph step by step, more comprehensive problem-related knowledge can be obtained; the above-mentioned refinement of the subgraph by pruning the subgraph can remove the parts with less relevance to the problem task, eliminate the deviation path, and only retain the high-value subgraph; by controlling the exploration depth and width, the scale of the knowledge subgraph can be controlled to avoid information redundancy.

[0108] S2-2. Integrate the knowledge subgraph into the initial large model to obtain a large model that introduces external knowledge.

[0109] Inputting the knowledge subgraph into the initial large model;

[0110] The knowledge subgraph and the implicit knowledge of the initial large model are concatenated or weightedly integrated to obtain a large model that introduces external knowledge;

[0111] The implicit knowledge of the initial large model itself is the knowledge learned based on the training corpus when the initial large model performs preliminary question-answering decision training.

[0112] The knowledge subgraph is used as explicit knowledge to merge with the implicit knowledge of the large model, and the explicit knowledge and implicit knowledge are combined to jointly derive decisions, thereby improving the depth and coverage of derived decisions and optimizing the knowledge representation of the large model. Static embedding provides global support, and dynamic exploration completes implicit knowledge, and the two work together to achieve knowledge enhancement.

[0113] In step S3, the question text is input into the large model for question-answering decision, and the question-answering result of the question text is output.

[0114] The question text is input into the large model for question-answering decision making. The large model generates and outputs the question-answering result of the question text based on the explicit knowledge subgraph and the supplement of implicit knowledge.

[0115] In step S4, based on the attention distribution and intermediate hidden states of the large model during the question-answering decision process, the reasoning process of the large model during the question-answering decision process is determined.

[0116] By combining the structural information of the knowledge graph and the reasoning process of the large model, the reasoning process of the large model in the question-answering task is parsed and visualized. This can help users understand how the large model performs reasoning and generates answers, and improve the interpretability of the question-answering process. Specifically, it includes reasoning path analysis and visualization of the reasoning process. In the dynamic exploration and reasoning process, the entities, relationships, and attention distribution of the large model are recorded. By observing the reasoning path of the large model in the question-answering task, that is, the steps and intermediate reasoning process that the model goes through when processing the problem, you can understand how the model gradually derives the answer.

[0117] Specifically, obtaining the attention weight and intermediate hidden state of each word in the input sequence corresponding to the question text in the large model question-answering decision process;

[0118] Determine the attention degree of each word in the input sequence when making a decision based on the attention weight; determine the decision process based on the intermediate hidden state;

[0119] The reasoning process of the large model when making question-answering decisions is determined by combining the decision-making process, the attention paid to each word in the input sequence, the implicit knowledge of the large model itself, and the knowledge subgraph.

[0120] That is, by combining knowledge subgraphs and implicit knowledge, the big model is guided to generate a reasoning path from the input question text to the question-answering result, including the logical deduction chain of explicit knowledge (based on the relationship of knowledge subgraphs) and the supplementary reasoning of implicit knowledge (based on the intrinsic knowledge of the big model). The sources of explicit and implicit knowledge are marked in the reasoning path to ensure interpretability.

[0121] The attention weight reflects the degree of attention paid by the model to each input element, and can be used to understand which words the model focuses on when generating answers; each layer of the model produces intermediate hidden states, which provide clues about how information flows and transforms within the model. The attention weight and intermediate hidden states together constitute a complete reasoning path.

[0122] Visualize the reasoning process so that users can intuitively understand the reasoning process and intermediate results of the large model. This is achieved by drawing reasoning path diagrams, attention heat maps, etc. to show the intermediate steps of model reasoning.

[0123] In addition to the determination of the reasoning process provided in this solution, it also includes the analysis of the pre-trained language model (i.e., the large model in this solution). The pre-trained language model is the process of in-depth research and understanding of the pre-trained language model. By adopting different methods, such as association based on knowledge graphs, causal analysis, neuron activation, etc., the internal mechanism and knowledge expression of the language model in the process of reasoning and generating results are revealed. This analysis method can understand the reasoning process, dependency patterns, and stored implicit knowledge of the language model, thereby improving the interpretation ability and understanding of the application scenarios of the language model. The determination of the reasoning process can locate the specific knowledge in the parameter range of the large model, and then perform targeted training, update the knowledge that has been trained, and optimize the large model.

[0124] S5. After inputting the question text into the large model for question-answering decision and outputting the question-answering result of the question text, the process further includes:

[0125] Compare the question-answer result with the knowledge in the knowledge graph to obtain a comparison result;

[0126] The comparison result is combined with preset judgment rules to determine whether the large model has hallucination problems or generates errors during the reasoning process, so as to verify the question-answering results.

[0127] Specifically, the factual knowledge in the knowledge graph is used to verify the reasoning process and output of the large model in the question-answering task. By comparing the question-answering results of the large model with the relevant information in the knowledge graph, the possible hallucination problems or the situations where the generated information is inconsistent with the facts in the reasoning process of the large model are detected. This can improve the reliability and explainability of the large model question-answering.

[0128] By injecting factual knowledge in the knowledge graph into the reasoning process of the big model, the knowledge learned by the big model can be supplemented and enriched. For example, in a fault diagnosis scenario, by introducing fault-related knowledge graph information into the reasoning process of the big model, the big model can be helped to diagnose the fault more accurately and generate answers that are consistent with the facts.

[0129] In this solution, before the question text is input into the big model for question-answering decision, it also includes: using LPAQA or LAMA to perform language model detection on the big model that has completed concept alignment to verify the accuracy of the big model's understanding of knowledge.

[0130] Through language model detection, hallucinations and contradictions can be reduced, and the accuracy and reliability of large-model question answering can be improved.

[0131] Specifically, language model detection is performed using LAMA (Language Model Analysis) or LPAQA (Language Models for Prompting, Analysis, and Question Answering). LAMA first converts facts in the knowledge graph into fill-in-the-blank statements through predefined prompt templates. Next, LAMA uses the language model to predict the entities in the fill-in-the-blank. It inputs the fill-in-the-blank statements into the language model and lets the model generate possible entities. By comparing the generated entities with the entities in the real knowledge graph, the language model's ability to understand and predict the entities can be evaluated. Finally, the prediction results are used to evaluate the knowledge stored in the language model. By comparing the predicted entities with the entities in the knowledge graph, conclusions can be drawn about the accuracy and completeness of the language model's knowledge in a specific field.

[0132] LPAQA mines large-scale text data to discover sentences or phrases that contain knowledge about a specific domain. These sentences or phrases can be used as prompts to test the language model's understanding of the knowledge in that domain. In order to generate diverse and high-quality prompts, LPAQA uses rewriting technology to rewrite the mined sentences or phrases. Rewriting can include operations such as synonym replacement and syntactic conversion to increase the diversity of prompts. The generated diverse prompts are used to evaluate the knowledge in the language model. The rewritten prompts are input into the language model, the results generated by the model are observed and compared with the real knowledge, so as to evaluate the model's mastery of knowledge in a specific domain.

[0133] The present invention is used to study the interpretable question-answering method and system of a large model based on knowledge graph fusion. Based on the introduction of knowledge graph, a means of explaining and reasoning knowledge can be provided for the question-answering system, making the output of the system easier to understand and accept. By combining structured information such as entities and relationships in the knowledge graph, the knowledge and reasoning path that the large model relies on when making decisions can be revealed, thereby improving the interpretability of the question-answering system and helping to promote the question-answering system to develop in a more intelligent and humanized direction.

[0134] The entities and relationships in the knowledge graph are clearly defined, reducing ambiguity and ambiguity, making the knowledge points on which the model decision is based clearer; it is easier to trace the source of information and the relationship chain, thereby simplifying the reasoning path; when reasoning based on the fused knowledge graph, each decision step can correspond to specific entities and relationships. This mapping relationship increases the transparency of large model decisions.

[0135] In addition, by studying the knowledge graph to understand the knowledge learned by the large model and analyze the large model question-answering reasoning process, the application in high-risk accurate question-answering scenarios such as fault diagnosis and defect judgment is promoted. This includes using the knowledge graph to detect and verify the knowledge stored in the large model, injecting factual knowledge during question-answering task reasoning, and facilitating the exploration of solving the illusion problem and the contradiction between the generated and the facts when the large model is reasoning.

[0136] Example 2

[0137] Based on the same inventive concept, the present invention also provides a large model explainable question answering system based on knowledge graph fusion, such as Figure 2 As shown, including:

[0138] The external knowledge introduction module is used to obtain the relevant knowledge of the question text in the knowledge graph, and embed the relevant knowledge as static embedding; using the initial large model based on the question text and the static embedding, a dynamic interactive exploration is performed in the knowledge graph to obtain a knowledge subgraph, and the knowledge subgraph is integrated into the initial large model to obtain a large model that introduces external knowledge;

[0139] A question-answering decision module, used to input the question text into the large model for question-answering decision and output the question-answering result of the question text;

[0140] The decision reasoning module is used to determine the reasoning process of the large model during question-answering decision-making based on the attention distribution and intermediate hidden states of the large model during the question-answering decision-making process.

[0141] In a possible implementation, the external knowledge introduction module is specifically used to:

[0142] The question text and the static embedding are spliced ​​and aligned and then input into the initial large model;

[0143] Using the question text and static embedding at the input end of the initial large model, determine the exploration starting point of the initial large model in the knowledge graph, recorded as an initial node;

[0144] Dynamically generate initial exploration directions using the initial large model to obtain new nodes, and construct an initial subgraph in combination with the initial nodes; continue to explore based on the text questions, static embedding and the initial subgraph, and expand the initial subgraph to obtain a new subgraph;

[0145] Based on the semantic similarity between the question text and the new subgraph, pruning the new subgraph to obtain a pruned subgraph;

[0146] Continue exploring on the basis of the pruned subgraph until the preset exploration depth and width are reached, stop expanding, and obtain the final knowledge subgraph.

[0147] In a possible implementation, the external knowledge introduction module is specifically used to:

[0148] Inputting the knowledge subgraph into the initial large model;

[0149] The knowledge subgraph and the implicit knowledge of the initial large model are concatenated or weightedly integrated to obtain a large model that introduces external knowledge;

[0150] The implicit knowledge of the initial large model itself is the knowledge learned based on the training corpus when the initial large model performs preliminary question-answering decision training.

[0151] In a possible implementation, the decision reasoning module is specifically used to:

[0152] Obtaining the attention weight and intermediate hidden state of each word in the input sequence corresponding to the question text during the large model question answering decision process;

[0153] Determine the attention degree of each word in the input sequence when making a decision based on the attention weight; determine the decision process based on the intermediate hidden state;

[0154] The reasoning process of the large model when making question-answering decisions is determined by combining the decision-making process, the attention paid to each word in the input sequence, the implicit knowledge of the large model itself, and the knowledge subgraph.

[0155] In a possible implementation, the external knowledge introduction module is specifically used to:

[0156] extracting keywords from the question text;

[0157] Using keyword matching technology, by matching keywords in the question text with knowledge content in the knowledge graph, relevant knowledge of the question text in the knowledge graph is obtained;

[0158] The knowledge content of the knowledge graph includes entities, relationships and attributes.

[0159] In a possible implementation, the system further includes a verification module, which is used to:

[0160] After the question text is input into the big model for question-answering decision and the question-answering result of the question text is output, the question-answering result is compared with the knowledge in the knowledge graph to obtain a comparison result;

[0161] The comparison result is combined with preset judgment rules to determine whether the large model has hallucination problems or generates errors during the reasoning process, so as to verify the question-answering results.

[0162] In a possible implementation, the system further includes a data preprocessing module, which is used to:

[0163] Before obtaining the relevant knowledge of the question text in the knowledge graph, natural language processing technology is used to automatically update the knowledge in the knowledge graph based on a preset update frequency and update content range.

[0164] In a possible implementation, the data preprocessing module is further used to:

[0165] Before obtaining the relevant knowledge of the question text in the knowledge graph, cross-domain knowledge migration is performed on the knowledge in the knowledge graph to obtain the knowledge graph after knowledge migration.

[0166] In a possible implementation, the system further includes a model detection module, which is used to:

[0167] Before inputting the question text into the big model for question-answering decision, the big model that has completed concept alignment is subjected to language model detection using LPAQA or LAMA to verify the accuracy of the big model's understanding of knowledge.

[0168] Example 3

[0169] like Figure 3 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0170] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of a large-model explainable question-answering method based on knowledge graph fusion in the above-mentioned embodiment.

[0171] Example 4

[0172] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of a large model explainable question-answering method based on knowledge graph fusion in the above embodiment.

[0173] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A large-model explainable question answering method based on knowledge graph fusion, characterized in that: include: Obtain relevant knowledge of the question text in the knowledge graph, and use the embedding of the relevant knowledge as static embedding; Using the initial large model, based on the question text and static embedding, dynamic interactive exploration is performed in the knowledge graph to obtain a knowledge subgraph, and the knowledge subgraph is integrated into the initial large model to obtain a large model that introduces external knowledge; Inputting the question text into the large model for question-answering decision, and outputting the question-answering result of the question text; Based on the attention distribution and intermediate hidden states of the large model during the question-answering decision process, the reasoning process of the large model during question-answering decision-making is determined.

2. The method according to claim 1, characterized in that The initial large model is used to dynamically interact and explore in the knowledge graph based on the question text and static embedding to obtain a knowledge subgraph, including: The question text and the static embedding are spliced ​​and aligned and then input into the initial large model; Using the question text and static embedding at the input end of the initial large model, determine the exploration starting point of the initial large model in the knowledge graph, recorded as an initial node; Using the initial large model to dynamically generate an initial exploration direction to obtain new nodes, and combining the initial nodes to construct an initial subgraph; continuing to explore based on the text question, static embedding and the initial subgraph, and expanding the initial subgraph to obtain a new subgraph; Based on the semantic similarity between the question text and the new subgraph, pruning the new subgraph to obtain a pruned subgraph; Continue exploring on the basis of the pruned subgraph until the preset exploration depth and width are reached, stop expanding, and obtain the final knowledge subgraph.

3. The method according to claim 1, characterized in that The step of integrating the knowledge subgraph into the initial large model to obtain a large model that introduces external knowledge includes: Inputting the knowledge subgraph into the initial large model; The knowledge subgraph and the implicit knowledge of the initial large model are concatenated or weightedly integrated to obtain a large model that introduces external knowledge; The implicit knowledge of the initial large model itself is the knowledge learned based on the training corpus when the initial large model performs preliminary question-answering decision training.

4. The method according to any one of claims 1 to 3, characterized in that: The determining of the reasoning process of the large model during question-answering decision-making based on the attention distribution and intermediate hidden states of the large model during question-answering decision-making includes: Obtaining the attention weight and intermediate hidden state of each word in the input sequence corresponding to the question text during the large model question answering decision process; Determine the attention degree of each word in the input sequence when making a decision based on the attention weight; determine the decision process based on the intermediate hidden state; The reasoning process of the large model when making question-answering decisions is determined by combining the decision-making process, the attention paid to each word in the input sequence, the implicit knowledge of the large model itself, and the knowledge subgraph.

5. The method according to claim 1, characterized in that The step of obtaining relevant knowledge of the question text in the knowledge graph includes: extracting keywords from the question text; Using keyword matching technology, by matching keywords in the question text with knowledge content in the knowledge graph, relevant knowledge of the question text in the knowledge graph is obtained; The knowledge content of the knowledge graph includes entities, relationships and attributes.

6. The method according to claim 1, characterized in that After inputting the question text into the large model for question-answering decision and outputting the question-answering result of the question text, the method further includes: Compare the question-answer result with the knowledge in the knowledge graph to obtain a comparison result; The comparison result is combined with preset judgment rules to determine whether the large model has hallucination problems or generates errors during the reasoning process, so as to verify the question-answering results.

7. The method according to claim 1, characterized in that Before obtaining the relevant knowledge of the question text in the knowledge graph, the method further includes: Natural language processing technology is used to automatically update the knowledge in the knowledge graph based on a preset update frequency and update content range.

8. The method according to claim 1, characterized in that Before obtaining the relevant knowledge of the question text in the knowledge graph, the method further includes: The knowledge in the knowledge graph is transferred across domains to obtain the knowledge graph after the knowledge transfer.

9. The system according to claim 1, characterized in that Before inputting the question text into the large model for question-answering decision, the method further includes: The language model of the large model that has completed concept alignment is detected by LPAQA or LAMA to verify the accuracy of the large model's understanding of knowledge.

10. A large-model explainable question-answering system based on knowledge graph fusion, characterized in that: include: The external knowledge introduction module is used to obtain the relevant knowledge of the question text in the knowledge graph and embed the relevant knowledge as static embedding; Using the initial large model, based on the question text and static embedding, dynamic interactive exploration is performed in the knowledge graph to obtain a knowledge subgraph, and the knowledge subgraph is integrated into the initial large model to obtain a large model that introduces external knowledge; A question-answering decision module, used for inputting the question text into the large model for question-answering decision and outputting the question-answering result of the question text; The decision reasoning module is used to determine the reasoning process of the large model during question-answering decision-making based on the attention distribution and intermediate hidden states of the large model during the question-answering decision-making process.

Images