Large language model heterogeneous knowledge reasoning method and device based on atomic knowledge operator

Abstract

The invention provides a large language model heterogeneous knowledge reasoning method and device based on an atomic knowledge operator, and the method comprises the steps: building an atomic reasoning tree based on an original to-be-answered question; wherein the atomic inference tree comprises father nodes and leaf nodes, root nodes in the father nodes are original to-be-answered questions, non-root nodes in the father nodes are different levels of sub-questions obtained by decomposing the original to-be-answered questions, the leaf nodes are atomic questions obtained by decomposing the original to-be-answered questions, and each leaf node corresponds to an atomic knowledge operator; the atomic knowledge operator comprises a search operator, a relation operator and a filtering operator; for leaf nodes, reasoning atomic problems according to retrieval-knowledge on the basis of a large language model and an atomic knowledge operator; for a father node, based on a large language model, according to child node reasoning answers, brother node reasoning answers or retrieval-knowledge, reasoning child questions; reasoning the atomic reasoning tree from a leaf node to a root node from bottom to top to obtain a reasoning result of the original to-be-answered question; wherein the retrieval-knowledge is obtained by executing dynamic knowledge retrieval from various heterogeneous knowledge sources. According to the method, an original to-be-answered question is decomposed into atomic-level fine granularity, a leaf node in an atomic reasoning tree corresponds to an atomic knowledge operator, a more accurate reasoning result is achieved, meanwhile, knowledge is allowed to be dynamically retrieved from various heterogeneous knowledge sources at each sub-question and atomic question node, and the reasoning efficiency is improved. Different types of queries can be flexibly coped with, richer, more accurate and more complementary information is provided, and the overall reasoning ability of the algorithm is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and knowledge reasoning, and in particular to a large language model heterogeneous knowledge reasoning method and device based on atomic knowledge operators. Background Art

[0002] Knowledge-intensive reasoning is a challenging task that requires the ability to reason about large amounts of knowledge using a variety of reasoning techniques, such as multi-hop reasoning, comparison, and computation. Although recent advances in large language models (LLMs) have enabled them to perform well in a variety of natural language processing tasks, it is difficult to rely solely on LLMs to perform reliable knowledge-intensive reasoning due to their inefficiency in reasoning planning and the hallucination problem, i.e., the phenomenon where LLMs confidently make up false facts.

[0003] In order to solve the above problems, a mainstream solution in recent years is to combine thought chain reasoning technology with retrieval enhancement generation technology. Specifically, many existing studies have proposed methods to first decompose complex problems into multiple simple problems, and then perform retrieval enhancement generation on each sub-problem, thereby reducing the illusion of large models.

[0004] However, these existing knowledge-intensive reasoning methods have the following main defects: (1) Insufficient problem decomposition. Most existing methods attempt to decompose complex problems into multiple sub-problems in a chain or tree structure. However, they only perform free problem decomposition based on large models, and the granularity of the decomposed sub-problems is often not fine enough, which will lead to errors in reasoning results. (2) Lack of support for multiple heterogeneous knowledge sources. Heterogeneous knowledge sources include online web pages, local text corpora, structured knowledge bases and other knowledge sources. These knowledge sources have different structures, but each contains rich knowledge that can complement each other. However, most existing methods only use one fixed knowledge source for retrieval.

[0005] Therefore, how to solve the problem that existing knowledge-intensive reasoning methods have low reasoning accuracy due to insufficient granularity of problem decomposition and inability to support multiple knowledge sources at the same time is an important issue that needs to be urgently addressed in the field of natural language processing and knowledge reasoning. Summary of the invention

[0006] The present invention provides a large language model heterogeneous knowledge reasoning method and device based on atomic knowledge operators, which is used to overcome the defects of existing knowledge-intensive reasoning methods that the problem decomposition granularity is not fine enough and multiple knowledge sources cannot be supported simultaneously, resulting in low reasoning accuracy, and achieve more accurate reasoning results.

[0007] On the one hand, the present invention provides a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators, including: constructing an atomic reasoning tree based on an original question to be answered; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original question to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original question to be answered, the leaf nodes are atomic problems decomposed from the original question to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filtering operators; for leaf nodes, based on a large language model and atomic knowledge operators, reasoning is performed on the atomic problem according to retrieval-knowledge; for parent nodes, based on a large language model, reasoning is performed on the sub-problem according to the child node reasoning answer, the sibling node reasoning answer or retrieval-knowledge; reasoning is performed on the atomic reasoning tree from the leaf node to the root node from the bottom up to obtain the reasoning result of the original question to be answered; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0008] Furthermore, the method of constructing an atomic reasoning tree based on the original question to be answered includes: inputting the original question to be answered, the problem answering requirements, the definition of the atomic reasoning tree, the atomic knowledge operator information, and examples into the large language model to obtain an initial atomic reasoning tree; indexing the nodes in the initial atomic reasoning tree according to a preset search order, and using reference placeholders to represent the actual answers to sub-questions and atomic questions to construct the atomic reasoning tree.

[0009] Furthermore, the method of reasoning about atomic problems based on retrieval-knowledge based on the large language model and the atomic knowledge operator includes: inputting the atomic problem, multiple heterogeneous knowledge sources to be selected, and their corresponding applicable scenarios and examples into the large language model to obtain the target knowledge source; searching in the target knowledge source according to the parameters of the atomic problem and its corresponding atomic knowledge operator to obtain the target retrieval-knowledge; using the target retrieval-knowledge and the atomic problem as inputs of the atomic knowledge operator to output the reasoning result of the atomic problem; wherein the heterogeneous knowledge sources include at least online web pages, text corpora, and structured knowledge bases.

[0010] Furthermore, the method of reasoning the sub-problem based on the large language model according to the child node reasoning answer, the sibling node reasoning answer or the retrieval-knowledge includes: when the child node reasoning answer meets the reasoning condition, the child node reasoning answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; when the child node reasoning answer does not meet the reasoning condition, but the sibling node reasoning answer meets the reasoning condition, the sibling node reasoning answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; when both the child node reasoning answer and the sibling node reasoning answer do not meet the reasoning condition, the retrieval-knowledge and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem.

[0011] Furthermore, the search operator takes an entity name and an optional entity descriptor as input, and outputs a target entity; the relationship operator takes a given head entity and a relationship as input, and outputs a tail entity; or, the relationship operator takes a given head entity and an attribute as input, and outputs an attribute value; or, the relationship operator takes a given head entity and a tail entity as input, and outputs a relationship between the head entity and the tail entity; the filter operator takes an entity set and attribute constraints as input, and outputs entities that satisfy the attribute constraints.

[0012] Furthermore, it also includes: constructing an evaluation data set based on question samples and multiple heterogeneous knowledge sources; using the F1 score as an evaluation indicator, and using the evaluation data set to verify the large language model heterogeneous knowledge reasoning method; wherein the F1 score is the harmonic mean of precision and recall.

[0013] In a second aspect, the present invention also provides a large language model heterogeneous knowledge reasoning device based on atomic knowledge operators, including: an atomic reasoning tree construction module, used to construct an atomic reasoning tree based on the original problem to be answered; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, the leaf nodes are atomic problems decomposed from the original problem to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filtering operators; a heterogeneous knowledge reasoning module, used to, for leaf nodes, based on a large language model and atomic knowledge operators, reason about atomic problems according to retrieval-knowledge; for parent nodes, based on a large language model, reason about answers according to child nodes, brother nodes or retrieval-knowledge to reason about sub-problems; reasoning the atomic reasoning tree from bottom to top from leaf nodes to root nodes to obtain the reasoning result of the original problem to be answered; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0014] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators as described above is implemented.

[0015] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators as described in any one of the above.

[0016] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described heterogeneous knowledge reasoning methods for large language models based on atomic knowledge operators.

[0017] The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided by the present invention constructs an atomic reasoning tree based on the original problem to be answered; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, the leaf node is the atomic problem decomposed from the original problem to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator includes a search operator, a relational operator and a filter operator; then, for the leaf node, based on the large language model and the atomic knowledge operator, the atomic problem is reasoned according to retrieval-knowledge; for the parent node, based on the large language model, the answer is inferred from the child node, the answer is inferred from the sibling node, or the sub-problem is reasoned according to retrieval-knowledge, so as to complete the process of reasoning the atomic reasoning tree from the leaf node to the root node from the bottom up, and obtain the reasoning result of the original problem to be answered; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources. This method achieves more accurate reasoning results by decomposing the original problem to be solved into atomic-level fine-grained pieces and making the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, it can flexibly respond to different types of queries, provide richer, more accurate and complementary information, and enhance the overall reasoning ability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided in an embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of the reasoning process of the atomic reasoning tree provided by an embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the overall process of a large language model heterogeneous knowledge reasoning method based on atomic knowledge provided by an embodiment of the present invention.

[0022] Figure 4 It is a structural diagram of a large language model heterogeneous knowledge reasoning device based on atomic knowledge operators provided in an embodiment of the present invention.

[0023] Figure 5 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be noted that most of the knowledge-intensive reasoning methods provided by existing technologies first decompose complex problems into multiple simple problems, and then perform retrieval enhancement generation on each sub-problem to reduce the illusion of a large model. However, this method has the defects of insufficient problem decomposition and lack of support for multiple heterogeneous knowledge sources, which leads to low accuracy of reasoning results.

[0026] In view of this, the present invention proposes a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators. Specifically, Figure 1 A schematic flow chart of a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided in an embodiment of the present invention is shown.

[0027] like Figure 1 As shown, the method includes steps S110-S120, and steps S110-S120 and related steps will be described in detail below.

[0028] S110, constructing an atomic reasoning tree based on the original problem to be solved; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be solved, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be solved, the leaf nodes are atomic problems decomposed from the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filtering operators.

[0029] It is easy to understand that after determining the original problem to be answered according to user needs, the large language model can be used to construct an atomic reasoning tree to decompose the original problem to be answered into multiple sub-problems at the finest atomic level. Specifically, the original problem to be answered and the corresponding prompt words are input into the large language model together to obtain the output initial atomic reasoning tree. Among them, the prompt words corresponding to the original problem to be answered include the problem answering requirements, the definition of the atomic reasoning tree, the atomic knowledge operator information and related examples.

[0030] Question answering requirements refer to requirement texts such as "Please decompose the original question to be answered into an atomic reasoning tree at the atomic level, and attach the corresponding atomic knowledge operators and corresponding parameters under the atomic questions represented by the leaf nodes", which can be adjusted according to actual conditions. The definition of the atomic reasoning tree can be obtained by dialoguing with a large language model, or it can be defined according to the user's own understanding, and no specific limitation is made here. Atomic knowledge operator information includes the type of atomic knowledge operator, as well as the input, output, and function of each type of atomic knowledge operator. The example is the atomic-level problem decomposition process edited by the user.

[0031] Next, the nodes in the initial atomic reasoning tree can be indexed according to the preset search order. At the same time, in order to reference the intermediate answer, this embodiment also uses The reference placeholders are used to represent the actual answers to the sub-questions and atomic questions. In the reasoning execution phase, the reference placeholders will be replaced with the actual reasoning answers to the sub-questions and atomic questions. The preset search order can be set according to actual needs, such as the breadth-first search order (Breadth-First Search, BFS), which is not specifically limited here.

[0032] In this way, an atomic reasoning tree can be constructed.

[0033] In this embodiment, the nodes in the constructed atomic reasoning tree are divided into two categories, namely parent nodes and leaf nodes. The parent nodes include root nodes and non-root nodes. The root nodes correspond to the original problem to be solved, the non-root nodes correspond to the sub-problems of different levels decomposed from the original problem to be solved, and the leaf nodes correspond to the atomic problems decomposed from the original problem to be solved.

[0034] It should be noted that this embodiment strictly distinguishes between sub-problems and atomic problems. Sub-problems can be further decomposed, while the nature of atomic problems is the same as that of leaf nodes, and they cannot be further divided.

[0035] It is worth mentioning that this embodiment not only uses a large language model to decompose the original question to be answered into an atomic reasoning tree at the atomic level, but also involves three common atomic knowledge operators for each atomic question, namely, a search operator, a relational operator, and a filtering operator (this will be elaborated in detail in the embodiments below). These three basic atomic operators can be flexibly combined and applied to various heterogeneous knowledge sources such as online web pages, local text corpora, and structured knowledge bases.

[0036] After building an atomic reasoning tree based on the original question to be answered in step S110, further, step S120 is performed.

[0037] S120, for leaf nodes, based on the large language model and atomic knowledge operators, reasoning is performed on the atomic question according to retrieval-knowledge; for parent nodes, based on the large language model, reasoning is performed on the sub-question according to the child node reasoning answer, the sibling node reasoning answer or retrieval-knowledge; the atomic reasoning tree is reasoned from the leaf node to the root node from the bottom up to obtain the reasoning result of the original question to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0038] It is easy to understand that for the constructed atomic reasoning tree, by reasoning from the leaf node to the root node from bottom to top, the reasoning result of the original problem to be solved can be obtained.

[0039] It is worth mentioning that this embodiment adopts different reasoning schemes for leaf nodes and parent nodes in the atomic reasoning tree.

[0040] Specifically, for leaf nodes, the pre-designed atomic knowledge operator is used to reason about the atomic question. During the reasoning process, the large language model is used as a dynamic knowledge source selector through context learning. After selecting the appropriate knowledge source for each atomic question, knowledge retrieval is initiated in the selected knowledge source to obtain the corresponding retrieval-knowledge. Subsequently, the retrieval-knowledge is used as auxiliary information to participate in the reasoning process of the atomic knowledge operator on the atomic question, thereby obtaining the local answer to the atomic question, that is, the reasoning result of the atomic question.

[0041] It should be noted that in this embodiment, there is a one-to-one correspondence between atomic problems and atomic knowledge operators. What kind of atomic knowledge operator (search operator, relational operator or filter operator) each atomic problem should correspond to should be determined based on the content of the atomic problem and is not specifically limited here.

[0042] It should also be noted that in addition to being inferred through predefined atomic knowledge operators, atomic questions can also be answered by analyzing the answer to the previous sibling question (also an atomic question) in some special cases. In this case, although the atomic question corresponds to a leaf node, it is not necessary to set an atomic knowledge operator for it. For example, the answer to the previous sibling question is "Alice sold books a, b, c", and the current atomic question is "How many books did Alice sell?" Obviously, this atomic question can be directly answered based on the previous sibling question, so there is no need to call any operators or perform retrieval.

[0043] For the parent node, the child node reasoning answer, the sibling node reasoning answer and one of the retrieval-knowledge are used as auxiliary information to participate in the reasoning process of the large language model for the sub-problem, so as to obtain the local answer to the sub-problem, that is, the reasoning result of the sub-problem.

[0044] It should be noted that the use of one of the child node reasoning answer, sibling node reasoning answer and retrieval-knowledge as auxiliary information is not randomly determined, and there is a preset priority relationship between the three. Specifically, the child node reasoning answer and the sibling node reasoning answer are given priority, and the two are in a parallel relationship. The existence of the child node reasoning answer and the sibling node reasoning answer is determined when the atomic reasoning tree is constructed. Therefore, when reasoning, if there is a child node reasoning answer, the child node reasoning answer is applicable, and if there is no child node reasoning answer, the sibling node reasoning answer is applicable. If there is no child node reasoning answer and the sibling node reasoning answer, the retrieval-knowledge is finally applied. In other words, only when the accurate result of the current sub-problem cannot be inferred based on the child node reasoning answer and the sibling node reasoning answer, will the retrieval-knowledge be used to assist in reasoning the current sub-problem.

[0045] It should also be noted that the retrieval knowledge in the atomic problem reasoning process and the retrieval knowledge in the sub-problem reasoning process are both obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources. The acquisition sources of the two are the same, but the essential contents of the two are determined by the sub-problems and atomic problems, which are different.

[0046] In this embodiment, an atomic reasoning tree is constructed based on the original problem to be answered; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, the leaf nodes are atomic problems decomposed from the original problem to be answered, and each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators, and filter operators; then, for the leaf nodes, based on the large language model and the atomic knowledge operators, the atomic problems are reasoned according to retrieval-knowledge; for the parent nodes, based on the large language model, the answers are reasoned according to the child nodes, the answers are reasoned according to the sibling nodes, or the retrieval-knowledge is reasoned for the sub-problems, so as to complete the process of reasoning the atomic reasoning tree from the leaf nodes to the root nodes from the bottom up, and obtain the reasoning result of the original problem to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources. This method achieves more accurate reasoning results by decomposing the original problem to be solved into atomic-level fine-grained pieces and making the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, it can flexibly respond to different types of queries, provide richer, more accurate and complementary information, and enhance the overall reasoning ability of the algorithm.

[0047] On the basis of the above embodiments, the types and functions of atomic knowledge operators will be described in detail below.

[0048] It is easy to understand that this embodiment designs three general atomic knowledge operators for leaf nodes (atomic problems), namely: search operator, relation operator and filter operator. These three basic atomic operators can be flexibly combined and applied to various heterogeneous knowledge sources.

[0049] In a specific embodiment, a search operator is described in detail.

[0050] Search operators are designed for entity disambiguation, which is to accurately retrieve the desired entity from a large pool of entities, especially when multiple entities share similar names. In the knowledge graph, the search operator can be analogized to accurately locating the initial entity node in the graph.

[0051] In this embodiment, the search operator is defined as the following formula (1).

[0052] (1).

[0053] According to formula (1), the search operator contains two inputs, namely the entity name and the optional entity descriptor to assist in disambiguation, and the output of the search operator is the target entity. If the optional entity descriptor is not provided, the search operator will return the most popular entity.

[0054] For example, suppose you want to find a person named "John", but there may be many people named "John", so you need some additional information to determine which "John" it is. To do this, you can call the search operator , it will return those people who live in New York and whose name is John. John is the entity name and the optional entity descriptor is " ”.

[0055] In another specific embodiment, a relational operator is described in detail.

[0056] Relational operators are designed for single-hop reasoning. In a knowledge graph, relational operators can be analogized to operations that process related knowledge from a head entity to a tail entity. There are three possibilities for single-hop reasoning: given a head entity and a relationship, retrieving a tail entity; given an entity and an attribute, retrieving an attribute value; and retrieving the relationship between a head entity and a tail entity. These three possibilities can be described as follows: (2)-(4).

[0057] (2).

[0058] (3).

[0059] (4).

[0060] In equations (2)-(4), Represents the head entity, Represents the tail entity, Represents the relationship between the head entity and the tail entity, Represents an attribute.

[0061] According to equations (2)-(4), the relation operator can take a given head entity and a relation as input and output a tail entity, or take a given head entity and an attribute as input and output an attribute value, or take a given head entity and a tail entity as input and output the relationship between the head entity and the tail entity.

[0062] In yet another specific embodiment, a filtering operator is described in detail.

[0063] The filter operator is designed for entity filtering. In the knowledge graph, the filter operator is similar to filtering out a subset of entities that meet the attribute conditions from the initial entity set. In this embodiment, the filter operator is defined as the following formula (5).

[0064] (5).

[0065] In formula (5), Represents an attribute constraint.

[0066] According to formula (5), the filter operator takes the entity set and attribute constraints as input and outputs the entities that satisfy the attribute constraints.

[0067] In this embodiment, by making the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator, which is one of a search operator, a relational operator and a filter operator, a more accurate reasoning result is achieved.

[0068] On the basis of the above-mentioned embodiment, the reasoning process of the leaf node (atomic problem) will be described in detail below.

[0069] Based on a large language model and an atomic knowledge operator, reasoning is performed on atomic questions according to retrieval-knowledge, including: inputting the atomic question, multiple heterogeneous knowledge sources to be selected, and their corresponding applicable scenarios and examples into the large language model to obtain a target knowledge source; searching in the target knowledge source according to the atomic question and the parameters of its corresponding atomic knowledge operator to obtain target retrieval-knowledge; using the target retrieval-knowledge and the atomic question as inputs of the atomic knowledge operator, and outputting the reasoning result of the atomic question; wherein the heterogeneous knowledge sources include at least online web pages, text corpora, and structured knowledge bases.

[0070] It is easy to understand that the process of reasoning about atomic problems using predefined atomic knowledge operators is mainly achieved through the following three steps.

[0071] First, dynamic knowledge source selection.

[0072] Specifically, previous works all retrieved knowledge from static knowledge sources, while this embodiment adopts dynamic knowledge source selection for each atomic question, which enables this embodiment to flexibly identify the most appropriate knowledge source to answer each atomic question.

[0073] In detail, this embodiment uses a large model as a dynamic knowledge source selector through context learning, inputs atomic questions, multiple heterogeneous knowledge sources to be selected and their corresponding applicable scenarios and examples into the large language model, and obtains the output target knowledge source, which is used for subsequent multi-source knowledge retrieval steps.

[0074] It should be noted that the number of target knowledge sources can be one or more, and is not specifically limited here.

[0075] Then, multi-source knowledge retrieval.

[0076] After selecting a suitable knowledge source for each atomic question, this embodiment searches the target knowledge source according to the parameters of the atomic question and its corresponding atomic knowledge operator (i.e., the inputs in the definitions of the search operator, relational operator, and filter operator described above can be extracted from the atomic question) to obtain the target retrieval-knowledge. The retrieval method varies depending on the type of knowledge source. For example, this embodiment queries Web knowledge sources through a search engine API, queries text knowledge sources through a dense searcher, and queries knowledge graphs through a structured graph query language.

[0077] For Web knowledge sources and text knowledge sources, this embodiment searches for the previous k paragraphs and articles as target retrieval-knowledge, among which k is a predefined hyperparameter. For the knowledge graph, this embodiment returns a complete list of structured answers.

[0078] Finally, atomic operator reasoning.

[0079] With the atomic question and target retrieval-knowledge, the atomic knowledge operator can start to execute. In order to ensure the flexibility and robustness of the method, this embodiment uses a large model as an adaptive executor instead of a static symbolic code to execute the atomic knowledge operator, takes the target retrieval-knowledge and the atomic question as the input of the atomic knowledge operator, and outputs the reasoning result of the atomic question. This process is the atomic operator reasoning. Finally, the output of the adaptive executor is used as the local answer to the relevant atomic question.

[0080] In this way, accurate reasoning about atomic problems can be achieved.

[0081] In this embodiment, the target knowledge source is obtained by inputting the atomic problem, multiple heterogeneous knowledge sources to be selected, and their corresponding applicable scenarios and examples into the large language model, and the target retrieval-knowledge is obtained by searching in the target knowledge source according to the parameters of the atomic problem and its corresponding atomic knowledge operator, and then the target retrieval-knowledge and the atomic problem are used as the input of the atomic knowledge operator to output the reasoning result of the atomic problem, thereby achieving more accurate reasoning results. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources, it can flexibly respond to different types of queries, provide richer, accurate and complementary information, and enhance the overall reasoning ability of the algorithm.

[0082] On the basis of the above embodiment, the reasoning process of the parent node (sub-problem) will be described in detail below.

[0083] Based on the large language model, reasoning is performed on the sub-problem according to the child-node reasoning answer, the sibling node reasoning answer or the retrieval-knowledge, including: when the child-node reasoning answer meets the reasoning conditions, the child-node reasoning answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; when the child-node reasoning answer does not meet the reasoning conditions, but the sibling node reasoning answer meets the reasoning conditions, the sibling node reasoning answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; when both the child-node reasoning answer and the sibling node reasoning answer do not meet the reasoning conditions, the retrieval-knowledge and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem.

[0084] It is easy to understand that in an atomic reasoning tree, all nodes that do not call atomic knowledge operators are regarded as parent nodes, and parent nodes may undergo child node reasoning answer reasoning, sibling node reasoning answer reasoning, or direct retrieval enhancement reasoning.

[0085] First, the sub-question reasoning is performed based on the inference answers of the child nodes, that is, the process of inferring the answer of the parent node by integrating the inference answers of the child nodes, which can be defined as the following formula (6).

[0086] (6).

[0087] In formula (6), is the sub-problem of the current parent node, A list of question-answer pairs that are child nodes of the current parent node.

[0088] The process of reasoning about sub-problems based on the inference answers of child nodes is achieved through context learning of the large language model. Specifically, the inference answer of the child node and the sub-problem of the current parent node are input into the large language model together to obtain the reasoning result of the sub-problem.

[0089] It should be noted that in this process, the possibility of not being able to obtain a suitable reasoning result needs to be considered, that is, it is necessary to consider whether the child node reasoning answer meets the reasoning conditions. The specific judgment process is to add instruction prompts to the input of the large language model. The instruction prompts here are such as "whether the current sub-problem can be accurately reasoned based on the child node reasoning answer". If the answer of the large language model is affirmative, the child node reasoning answer meets the reasoning conditions. Otherwise, the child node reasoning answer does not meet the reasoning conditions. At this time, consider reasoning the sub-problem based on the sibling node reasoning answer.

[0090] The process of reasoning about sub-problems based on the answers to sibling nodes is similar to the process of reasoning about sub-problems based on the answers to child nodes. The process of inferring the answer to the parent node by integrating the reasoning answers of the sibling nodes can be defined as follows: Equation (7).

[0091] (7).

[0092] In formula (7), is the sub-problem of the current parent node, A list of question-answer pairs that are siblings of the current parent node.

[0093] The process of reasoning about sub-problems based on the inference answers of sibling nodes is also achieved through contextual learning of the large language model. Specifically, the inference answers of the sibling nodes and the sub-problems of the current parent node are input into the large language model together to obtain the reasoning results of the sub-problems.

[0094] It is also necessary to note that in this process, the possibility of not being able to obtain a suitable reasoning result needs to be considered, that is, it is necessary to consider whether the sibling node reasoning answer meets the reasoning conditions. The specific judgment process is to add instruction prompts to the input of the large language model. The instruction prompts here are such as "whether the current sub-problem can be accurately reasoned based on the sibling node reasoning answer". If the answer of the large language model is affirmative, the sibling node reasoning answer meets the reasoning conditions. Otherwise, the sibling node reasoning answer does not meet the reasoning conditions. At this time, consider reasoning the sub-problem based on retrieval-knowledge.

[0095] Each parent node is mainly implemented by reasoning about sub-problems through child node reasoning answers or sibling node reasoning answers. However, if the parent node cannot obtain the answer based on child node reasoning answers and sibling node reasoning answers, this embodiment performs sub-problem reasoning based on retrieval-knowledge, that is, direct retrieval enhanced reasoning is adopted.

[0096] Specifically, first, the current sub-problem Perform dynamic knowledge source selection and then Start multi-source knowledge retrieval for the query statement. Finally, given With the retrieved knowledge, the large model is used to perform standard knowledge retrieval through context learning to generate reasoning results. This design ensures that although this embodiment focuses on knowledge retrieval on leaf nodes, external knowledge can be flexibly retrieved at parent nodes when necessary.

[0097] In this embodiment, when the child node inference answer meets the reasoning conditions, the child node inference answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; when the child node inference answer does not meet the reasoning conditions, but the sibling node inference answer meets the reasoning conditions, the sibling node inference answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; when both the child node inference answer and the sibling node inference answer do not meet the reasoning conditions, the retrieval-knowledge and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem, thereby achieving more accurate reasoning results. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources, it is possible to flexibly respond to different types of queries, provide richer, accurate and complementary information, and enhance the overall reasoning ability of the algorithm.

[0098] in addition, Figure 2 A schematic diagram of the reasoning process of the atomic reasoning tree provided in an embodiment of the present invention is shown, especially showing the reasoning process of leaf nodes and parent nodes.

[0099] like Figure 2 As shown, first determine whether the node is a leaf node. If it is a leaf node, prepare the atomic problem and the parameters of the atomic knowledge operator , and use the large language model as a dynamic knowledge source selector to determine the target knowledge source , further, knowledge retrieval is performed in the target knowledge source based on the parameters of the atomic knowledge operator to obtain the target retrieval-knowledge Finally, the atomic knowledge operator is called to reason about the atomic problem, and its input is the atomic problem , parameters of atomic knowledge operators and target retrieval-knowledge , and get the reasoning answer to the atomic problem .

[0100] If it is not a leaf node, it is a parent node. In this case, prepare the sub-question , as well as sibling node reasoning answers and child node reasoning answers Prioritize the use of sub-questions and sub-nodes to reason about answers. If no answer is obtained, use the sibling nodes to reason about answers. If no answer is obtained, use the sub-questions and retrieval-knowledge Perform inference and obtain inference results . Retrieval-Knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0101] Thus, the atomic reasoning tree is reasoned from the leaf node to the root node from bottom to top, and finally the reasoning result of the original problem to be solved is obtained.

[0102] Figure 3 The overall process diagram of the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided by an embodiment of the present invention is shown.

[0103] like Figure 3 As shown, the left side shows three atomic knowledge operators involved in the atomic problem in the embodiment of the present invention, namely, the search operator Search, the relation operator Relate, and the filter operator Filter. Figure 3 The right side shows the reasoning process of leaf nodes and parent nodes in the atomic reasoning tree provided by the embodiment of the present invention, which has been elaborated in detail in the above embodiment and will not be repeated here.

[0104] In some other embodiments, a dataset for evaluating heterogeneous knowledge reasoning is also constructed.

[0105] An evaluation dataset is constructed based on question samples and multiple heterogeneous knowledge sources. The F1 score is used as the evaluation indicator to verify the heterogeneous knowledge reasoning method of the large language model using the evaluation dataset. Among them, the F1 score is the harmonic mean of precision and recall.

[0106] It is easy to understand that for a question sample, it is retrieved through multi-hop knowledge sources to construct a multi-hop sample pair, that is, question sample-knowledge source 1-knowledge source 2-knowledge source 3, and so on. Thus, an evaluation data set can be constructed. Then, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided by the prior art and the embodiment of the present invention is experimentally verified on the evaluation data set.

[0107] Specifically, this embodiment is evaluated on three traditional single-knowledge-source complex reasoning datasets, HotpotQA, 2WikiMultiHop, and Musique, and a multi-knowledge-source complex reasoning dataset, CRAG, and an evaluation dataset, BlendQA, constructed by this embodiment. Experiments show that the effect of the present invention is far superior to existing technologies, achieving significant F1 score improvements of 5.4%, 9.4%, and 1.2% on three single-knowledge-source datasets, and 9.5% and 6.6% on two multi-knowledge-source datasets. Compared with existing methods, the present invention significantly reduces the illusion of knowledge-intensive reasoning with large models, and is more efficient and cost-effective than existing methods, and can achieve better results with fewer API calls and computing resources than existing methods.

[0108] Corresponding to the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators described in the above embodiments, the present invention also proposes a large language model heterogeneous knowledge reasoning device based on atomic knowledge operators.

[0109] Specifically, Figure 4 A schematic diagram of the structure of a large language model heterogeneous knowledge reasoning device based on atomic knowledge operators provided by an embodiment of the present invention is shown.

[0110] like Figure 4 As shown, the device includes: an atomic reasoning tree construction module 410, which is used to construct an atomic reasoning tree based on the original problem to be answered; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, and the leaf nodes are atomic problems decomposed from the original problem to be answered, and each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filtering operators; a heterogeneous knowledge reasoning module 420, which is used to, for leaf nodes, reason about atomic problems according to retrieval-knowledge based on a large language model and atomic knowledge operators; for parent nodes, reason about answers according to child nodes, sibling nodes or retrieval-knowledge based on a large language model; the atomic reasoning tree is reasoned from bottom to top from leaf nodes to root nodes to obtain the reasoning result of the original problem to be answered; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0111] In this embodiment, an atomic reasoning tree is constructed based on the original problem to be answered through the atomic reasoning tree construction module 410; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, the leaf nodes are atomic problems decomposed from the original problem to be answered, and each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filter operators; then, for the leaf nodes, the atomic problems are reasoned according to retrieval-knowledge based on the large language model and the atomic knowledge operators through the heterogeneous knowledge reasoning module 420; for the parent node, the answer is inferred from the child node, the answer is inferred from the sibling node, or the sub-problem is reasoned according to retrieval-knowledge based on the large language model through the heterogeneous knowledge reasoning module 420, so as to complete the process of reasoning the atomic reasoning tree from the leaf node to the root node from the bottom up, and obtain the reasoning result of the original problem to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources. The device achieves more accurate reasoning results by decomposing the original problem to be solved into atomic-level fine-grained pieces and making the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, it can flexibly respond to different types of queries, provide richer, more accurate and complementary information, and enhance the overall reasoning ability of the algorithm.

[0112] It should be noted that the large language model heterogeneous knowledge reasoning device based on atomic knowledge operators provided in the embodiment of the present invention can correspond to the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators described in the above embodiments, and will not be repeated here.

[0113] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the large language model heterogeneous knowledge reasoning method based on the atomic knowledge operator, the method comprising: constructing an atomic reasoning tree based on the original question to be answered; wherein the atomic reasoning tree comprises a parent node and a leaf node, the root node in the parent node is the original question to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original question to be answered, the leaf node is the atomic problem decomposed from the original question to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relational operator and a filtering operator; for the leaf node, based on the large language model and the atomic knowledge operator, the atomic problem is reasoned according to retrieval-knowledge; for the parent node, based on the large language model, the answer is inferred according to the child node, the answer is inferred according to the sibling node, or the retrieval-knowledge is inferred to the sub-problem; the atomic reasoning tree is reasoned from the leaf node to the root node from the bottom up to obtain the reasoning result of the original question to be answered; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0114] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0115] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided by the above methods, and the method includes: based on the original problem to be answered, constructing an atomic reasoning tree; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, and the leaf nodes The invention relates to an atomic problem decomposed from the original problem to be solved, wherein each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filter operators; for the leaf nodes, based on the large language model and the atomic knowledge operators, the atomic problem is reasoned according to the retrieval-knowledge; for the parent node, based on the large language model, the answer is inferred according to the child node, the answer is inferred according to the sibling node or the retrieval-knowledge is reasoned for the sub-problem; the atomic reasoning tree is reasoned from the leaf node to the root node from the bottom up to obtain the reasoning result of the original problem to be solved; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0116] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators provided by the above methods, the method comprising: constructing an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises a parent node and a leaf node, the root node in the parent node is the original problem to be solved, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be solved, and the leaf nodes are original problems decomposed from the original problem to be solved. Sub-problems, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filter operators; for leaf nodes, based on the large language model and atomic knowledge operators, the atomic problem is reasoned according to retrieval-knowledge; for parent nodes, based on the large language model, the sub-problems are reasoned according to the child node reasoning answer, the sibling node reasoning answer or retrieval-knowledge; the atomic reasoning tree is reasoned from the leaf node to the root node from the bottom up to obtain the reasoning result of the original problem to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0117] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large language model heterogeneous knowledge reasoning method based on atomic knowledge operators, characterized in that: include: Based on the original problem to be answered, an atomic reasoning tree is constructed; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be answered, the leaf nodes are atomic problems decomposed from the original problem to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include a search operator, a relational operator, and a filtering operator; For leaf nodes, based on the large language model and atomic knowledge operators, the atomic questions are reasoned according to retrieval-knowledge; for parent nodes, based on the large language model, the answers are reasoned according to the child nodes, the answers are reasoned according to the sibling nodes, or the retrieval-knowledge is reasoned according to the sub-questions; the atomic reasoning tree is reasoned from the leaf nodes to the root nodes from the bottom up to obtain the reasoning result of the original question to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

2. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to claim 1 is characterized in that: The step of constructing an atomic reasoning tree based on the original question to be answered includes: Input the original question to be answered, the question answering requirements, the definition of the atomic reasoning tree, the atomic knowledge operator information, and examples into the large language model to obtain the initial atomic reasoning tree; The nodes in the initial atomic reasoning tree are indexed according to a preset search order, and reference placeholders are used to represent the actual answers to sub-questions and atomic questions to construct an atomic reasoning tree.

3. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to claim 1 is characterized in that: The method of reasoning about atomic questions based on retrieval-knowledge based on a large language model and atomic knowledge operators includes: Input the atomic question, multiple heterogeneous knowledge sources to be selected, and their corresponding applicable scenarios and examples into the large language model to obtain the target knowledge source; According to the atomic question and the parameters of its corresponding atomic knowledge operator, search in the target knowledge source to obtain the target search-knowledge; Take the target retrieval-knowledge and atomic questions as inputs of the atomic knowledge operator, and output the reasoning result of the atomic question; Among them, heterogeneous knowledge sources at least include online web pages, text corpora and structured knowledge bases.

4. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to claim 1 is characterized in that: The method of reasoning about sub-questions based on the large language model and inferring answers from child nodes, inferring answers from sibling nodes, or retrieving knowledge includes: When the inference answer of the sub-node satisfies the inference condition, the inference answer of the sub-node and the sub-question are input into the large language model to obtain the inference result of the sub-question; When the child node reasoning answer does not satisfy the reasoning condition, but the sibling node reasoning answer satisfies the reasoning condition, the sibling node reasoning answer and the sub-question are input into the large language model to obtain the reasoning result of the sub-question; When the child node reasoning answer and the sibling node reasoning answer do not meet the reasoning conditions, the retrieval-knowledge and the sub-question are input into the large language model to obtain the reasoning result of the sub-question.

5. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to claim 1 is characterized in that: The search operator takes an entity name and an optional entity descriptor as input and outputs a target entity; The relation operator takes a given head entity and a relation as input and outputs a tail entity; Alternatively, the relational operator takes a given head entity and an attribute as input and outputs an attribute value; Alternatively, the relationship operator takes a given head entity and a tail entity as input, and outputs a relationship between the head entity and the tail entity; The filtering operator takes an entity set and attribute constraints as input, and outputs entities that satisfy the attribute constraints.

6. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to any one of claims 1 to 5, characterized in that: Also includes: Construct an evaluation dataset based on problem samples and multiple heterogeneous knowledge sources; Taking the F1 score as the evaluation indicator, the large language model heterogeneous knowledge reasoning method is verified using the evaluation data set; The F1 score is the harmonic mean of precision and recall.

7. A large language model heterogeneous knowledge reasoning device based on atomic knowledge operators, characterized in that: include: An atomic reasoning tree construction module is used to construct an atomic reasoning tree based on the original problem to be solved; wherein the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original problem to be solved, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original problem to be solved, the leaf nodes are atomic problems decomposed from the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include a search operator, a relational operator, and a filtering operator; A heterogeneous knowledge reasoning module is used to reason about atomic questions according to retrieval-knowledge for leaf nodes based on a large language model and atomic knowledge operators; for parent nodes, reason about answers according to child nodes, sibling nodes or retrieval-knowledge for sub-questions based on a large language model; reason about the atomic reasoning tree from leaf nodes to root nodes from bottom to top to obtain reasoning results of the original question to be answered; wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators as described in any one of claims 1 to 6 is implemented.

Images