Heterogeneous knowledge reasoning method and device for large language model based on atomic knowledge operator

Abstract

The application provides a large language model heterogeneous knowledge reasoning method and device based on atomic knowledge operators, and the method comprises the following steps: constructing an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, the root node in the parent nodes is the original problem to be solved, the non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved; each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filter operator; for the leaf nodes, the atomic problems are reasoned based on a large language model and the atomic knowledge operator according to search-knowledge; for the parent nodes, the sub-problems are reasoned based on the large language model according to the reasoning answers of the child nodes, the reasoning answers of the sibling nodes or search-knowledge; the atomic reasoning tree is reasoned from the leaf nodes to the root node from bottom to top, and the reasoning result of the original problem to be solved is obtained; wherein the search-knowledge is obtained by executing dynamic knowledge search from multiple heterogeneous knowledge sources. The method realizes more accurate reasoning results by decomposing the original problem to be solved to the fine granularity of the atomic level and making the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator; meanwhile, by allowing dynamic search of knowledge from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, different types of queries can be flexibly responded to, more rich, accurate and complementary information is provided, and the overall reasoning ability of the algorithm is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and knowledge reasoning, and particularly relates to a large language model heterogeneous knowledge reasoning method and device based on atomic knowledge operators. BACKGROUND

[0002] Knowledge-intensive reasoning is a challenging task that requires the ability to reason about a large amount of knowledge using various reasoning skills, such as multi-hop reasoning, comparison, and calculation. Although recent advances in large language models (LLMs) have enabled them to perform well in a variety of natural language processing tasks, it is difficult to rely solely on LLMs themselves to complete reliable knowledge-intensive reasoning due to the inefficiency of LLMs in reasoning planning and the hallucination problem, i.e., the phenomenon of LLMs confidently fabricating false facts.

[0003] To solve the above problems, one of the mainstream solutions in recent years is to combine the thinking chain reasoning technology and the retrieval-augmented generation technology. Specifically, the methods proposed by many existing studies first decompose complex problems into multiple simple problems, and then perform retrieval-augmented generation for each sub-problem, thereby reducing the hallucination of large models.

[0004] However, the existing knowledge-intensive reasoning methods mainly have the following defects: (1) insufficient problem decomposition. Most existing methods attempt to decompose complex problems into multiple sub-problems in a chain or tree structure, but they are only based on large models for free problem decomposition, and the granularity of the decomposed sub-problems is often not fine enough, which can lead to errors in reasoning results. (2) Lack of support for multiple heterogeneous knowledge sources. Heterogeneous knowledge sources include online web pages, local text corpora, and structured knowledge bases, among others. These knowledge sources have different structures, but each contains rich knowledge that can complement each other. However, most existing methods only use a fixed knowledge source for retrieval.

[0005] Therefore, how to solve the problem of insufficient problem decomposition granularity and the inability to support multiple knowledge sources simultaneously, which leads to low reasoning accuracy, is an important issue in the field of natural language processing and knowledge reasoning that needs to be addressed. SUMMARY

[0006] The present application provides a large language model heterogeneous knowledge reasoning method and device based on atomic knowledge operators to overcome the defects of insufficient problem decomposition granularity and the inability to support multiple knowledge sources simultaneously, which leads to low reasoning accuracy, and to achieve more accurate reasoning results.

[0007] In one aspect, the present application provides a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators, comprising: constructing an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, the root node in the parent nodes is the original problem to be solved, the non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filtering operator; for the leaf nodes, based on a large language model and an atomic knowledge operator, an atomic problem is reasoned according to search-knowledge; for the parent nodes, based on a large language model, a sub-problem is reasoned according to a child node reasoning answer, a sibling node reasoning answer or search-knowledge; the atomic reasoning tree is reasoned from the leaf nodes to the root node from bottom to top, and a reasoning result of the original problem to be solved is obtained; wherein the search-knowledge is obtained by performing dynamic knowledge search from multiple heterogeneous knowledge sources.

[0008] Further, the atomic reasoning tree is constructed based on the original problem to be solved, comprising: inputting the original problem to be solved, problem solving requirements, definitions of the atomic reasoning tree, atomic knowledge operator information and examples into a large language model to obtain an initial atomic reasoning tree; indexing the nodes in the initial atomic reasoning tree according to a preset search order, and using a reference placeholder to represent actual answers of sub-problems and atomic problems, to obtain the atomic reasoning tree.

[0009] Further, the atomic problem is reasoned according to search-knowledge based on a large language model and an atomic knowledge operator, comprising: inputting the atomic problem, multiple heterogeneous knowledge sources to be selected and corresponding applicable scenarios and examples into a large language model to obtain a target knowledge source; searching in the target knowledge source according to the atomic problem and parameters of the corresponding atomic knowledge operator to obtain target search-knowledge; inputting the target search-knowledge and the atomic problem as inputs of the atomic knowledge operator, and outputting a reasoning result of the atomic problem; wherein the heterogeneous knowledge sources at least include online web pages, text corpus and structured knowledge base.

[0010] Further, the large language model is used to infer the sub-problems according to the child node inference answer, the sibling node inference answer or the retrieval-knowledge, including: in the case that the child node inference answer meets the inference condition, inputting the child node inference answer and the sub-problems into the large language model to obtain an inference result of the sub-problems; in the case that the child node inference answer does not meet the inference condition, but the sibling node inference answer meets the inference condition, inputting the sibling node inference answer and the sub-problems into the large language model to obtain the inference result of the sub-problems; in the case that neither the child node inference answer nor the sibling node inference answer meets the inference condition, inputting the retrieval-knowledge and the sub-problems into the large language model to obtain the inference result of the sub-problems.

[0011] Further, the search operator takes an entity name and an optional entity descriptor as input and outputs a target entity; the relation operator takes a given head entity and a relation as input and outputs a tail entity; or the relation operator takes a given head entity and a property as input and outputs a property value; or the relation operator takes a given head entity and a tail entity as input and outputs a relation between the head entity and the tail entity; and the filter operator takes a set of entities and a property constraint as input and outputs entities that satisfy the property constraint.

[0012] Further, the method further comprises: constructing an evaluation dataset according to the question sample and the plurality of heterogeneous knowledge sources; and verifying the large language model heterogeneous knowledge inference method by using the evaluation dataset as an evaluation index, wherein the evaluation index is an F1 score, and the F1 score is a harmonic mean of a precision rate and a recall rate.

[0013] In a second aspect, the present application further provides a large language model heterogeneous knowledge inference device based on atomic knowledge operators, comprising: an atomic inference tree construction module, configured to construct an atomic inference tree based on an original question to be answered, wherein the atomic inference tree comprises parent nodes and leaf nodes, a root node in the parent nodes is the original question to be answered, non-root nodes in the parent nodes are different level sub-questions obtained by decomposing the original question to be answered, and the leaf nodes are atomic questions obtained by decomposing the original question to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relation operator and a filter operator; a heterogeneous knowledge inference module, configured to, for the leaf nodes, infer the atomic questions based on a large language model and the atomic knowledge operators, and for the parent nodes, infer sub-questions based on the large language model according to child node inference answers, sibling node inference answers or retrieval-knowledge; and infer from bottom to top from the leaf nodes to the root node of the atomic inference tree to obtain an inference result of the original question to be answered, wherein the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from a plurality of heterogeneous knowledge sources.

[0014] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for heterogeneous knowledge reasoning of large language model based on atomic knowledge operator according to any one of the above aspects.

[0015] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the method for heterogeneous knowledge reasoning of large language model based on atomic knowledge operator according to any one of the above aspects.

[0016] In a fifth aspect, the present application also provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the method for heterogeneous knowledge reasoning of large language model based on atomic knowledge operator according to any one of the above aspects.

[0017] The method for heterogeneous knowledge reasoning of large language model based on atomic knowledge operator provided by the present application comprises the following steps: constructing an atomic reasoning tree based on an original problem to be solved, wherein the atomic reasoning tree comprises parent nodes and leaf nodes, a root node in the parent nodes is the original problem to be solved, non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filter operator; then, for the leaf nodes, the atomic problems are reasoned based on a large language model and the atomic knowledge operator according to search-knowledge; for the parent nodes, the sub-problems are reasoned based on the large language model according to a reasoning answer of a child node, a reasoning answer of a sibling node or search-knowledge, so as to complete a reasoning process from the leaf nodes to the root node of the atomic reasoning tree from bottom to top, and obtain a reasoning result of the original problem to be solved; and the search-knowledge is obtained by performing dynamic knowledge search from multiple heterogeneous knowledge sources. The method decomposes the original problem to be solved to a fine granularity of atomic level, and makes the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator, so that a more accurate reasoning result is achieved. At the same time, by allowing dynamic search of knowledge from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, different types of queries can be flexibly handled, more rich, accurate and complementary information is provided, and the overall reasoning ability of the algorithm is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is a flowchart of a large language model heterogeneous knowledge reasoning method based on an atomic knowledge operator provided by an embodiment of the present application.

[0020] Figure 2 is a schematic diagram of a reasoning process of an atomic reasoning tree provided by an embodiment of the present application.

[0021] Figure 3 is a schematic diagram of the overall flow of a large language model heterogeneous knowledge reasoning method based on atomic knowledge provided by an embodiment of the present application.

[0022] Figure 4 is a structural schematic diagram of a large language model heterogeneous knowledge reasoning device based on an atomic knowledge operator provided by an embodiment of the present application.

[0023] Figure 5 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0025] It should be noted that the knowledge-intensive reasoning methods provided by the prior art mostly decompose complex problems into multiple simple problems, and then retrieve and enhance generation for each sub-problem, thereby reducing the hallucination of large models. However, this method has the defects of insufficient problem decomposition and lack of support for multiple heterogeneous knowledge sources, thereby resulting in low accuracy of reasoning results.

[0026] In view of this, the present application provides a large language model heterogeneous knowledge reasoning method based on an atomic knowledge operator, specifically, Figure 1 shows a flowchart of a large language model heterogeneous knowledge reasoning method based on an atomic knowledge operator provided by an embodiment of the present application.

[0027] As Figure 1 shown, the method comprises steps S110-S120, which will be described in detail below.

[0028] S110, constructing an atomic reasoning tree based on the original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, a root node in the parent nodes is the original problem to be solved, non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filter operator.

[0029] It is easy to understand that after determining the original problem to be solved according to the user demand, the atomic reasoning tree can be constructed by using a large language model to decompose the original problem to be solved into a plurality of sub-problems at the atomic level with the finest granularity. Specifically, the original problem to be solved and the corresponding prompt word are input into the large language model, and an initial atomic reasoning tree output is obtained. The prompt word corresponding to the original problem to be solved includes problem solving requirements, definitions of atomic reasoning trees, atomic knowledge operator information and related examples.

[0030] The problem solving requirement refers to a requirement text such as "please decompose the original problem to be solved into an atomic reasoning tree at the atomic level, and attach the corresponding atomic knowledge operator and corresponding parameters under the atomic problem represented by the leaf node", which can be adjusted according to actual conditions. The definition of the atomic reasoning tree can be obtained by dialogue with the large language model, or can be defined according to the user's own understanding, which is not limited here. The atomic knowledge operator information includes the type of the atomic knowledge operator, and the input, output and function of each type of atomic knowledge operator. The example is the problem decomposition process at the atomic level edited by the user.

[0031] Then, the nodes in the initial atomic reasoning tree can be indexed according to a preset search order, and at the same time, in order to reference the intermediate answer, the embodiment also uses a reference placeholder represented as to represent the actual answers of the sub-problems and the atomic problems, and in the reasoning execution stage, the reference placeholder will be replaced by the actual reasoning answers of the sub-problems and the atomic problems. The preset search order can be set according to actual requirements, for example, a breadth-first search order (Breadth-First Search, BFS), which is not limited here.

[0032] Thus, the atomic reasoning tree can be constructed.

[0033] In this embodiment, the nodes in the constructed atomic reasoning tree are divided into two categories, namely parent nodes and leaf nodes, the parent nodes include root nodes and non-root nodes, the root nodes correspond to the original problem to be solved, the non-root nodes correspond to different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes correspond to atomic problems obtained by decomposing the original problem to be solved.

[0034] It should be noted that this embodiment strictly distinguishes between subproblems and atomic problems. Subproblems can be further decomposed, while the nature of atomic problems is the same as that of leaf nodes, and they cannot be further divided.

[0035] It is worth mentioning that this embodiment not only uses a large language model to decompose the original question to be answered into an atomic reasoning tree at the atomic level, but also involves three common atomic knowledge operators for each atomic question, namely the search operator, the relation operator, and the filter operator (this will be elaborated in detail in the embodiments below). These three basic atomic operators can be flexibly combined and applied to various heterogeneous knowledge sources such as online web pages, local text corpora, and structured knowledge bases.

[0036] After constructing an atomic reasoning tree based on the original question to be answered in step S110, further, step S120 is performed.

[0037] S120, for leaf nodes, based on the large language model and atomic knowledge operators, reasoning is performed on the atomic question according to retrieval-knowledge; for parent nodes, based on the large language model, reasoning is performed on the sub-question according to the child node reasoning answer, the sibling node reasoning answer, or retrieval-knowledge; the atomic reasoning tree is reasoned from the leaf node to the root node from the bottom up to obtain the reasoning result of the original question to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0038] It is easy to understand that for the constructed atomic reasoning tree, by reasoning from the leaf node to the root node from bottom to top, the reasoning result of the original problem to be solved can be obtained.

[0039] It is worth mentioning that this embodiment adopts different reasoning schemes for leaf nodes and parent nodes in the atomic reasoning tree.

[0040] Specifically, for leaf nodes, a pre-designed atomic knowledge operator is used to reason about the atomic question. During the reasoning process, a large language model is used as a dynamic knowledge source selector through contextual learning. After selecting an appropriate knowledge source for each atomic question, knowledge retrieval is initiated within the selected knowledge source to obtain the corresponding retrieval knowledge. Subsequently, the retrieval knowledge is used as auxiliary information to participate in the atomic knowledge operator's reasoning process on the atomic question, thereby obtaining the local answer to the atomic question, that is, the reasoning result of the atomic question.

[0041] It should be noted that in this embodiment, there is a one-to-one correspondence between atomic problems and atomic knowledge operators. What kind of atomic knowledge operator (search operator, relational operator or filtering operator) each atomic problem should correspond to should be determined based on the content of the atomic problem and is not specifically limited here.

[0042] It should also be noted that in addition to being able to reason through predefined atomic knowledge operators, atomic problems can also be answered by analyzing the answer to a previous sibling problem (also an atomic problem) in some special cases, even though the atomic problem corresponds to a leaf node, it does not need to be set with an atomic knowledge operator. For example, if the previous sibling problem answer is "Alice sold books a, b, and c", and the current atomic problem is "How many books did Alice sell?", obviously, the atomic problem can be directly answered according to the previous sibling problem answer, so it does not need to call any operator or perform retrieval.

[0043] For the parent node, one of the child node reasoning answer, sibling node reasoning answer and retrieval-knowledge is taken as auxiliary information to participate in the reasoning process of the large language model for the sub-problem, so as to obtain the local answer of the sub-problem, that is, the reasoning result of the sub-problem.

[0044] It should be noted that taking one of the child node reasoning answer, sibling node reasoning answer and retrieval-knowledge as auxiliary information is not randomly determined, and there is a preset priority relationship between the three. Specifically, the child node reasoning answer and the sibling node reasoning answer are given priority, and the two are in parallel relationship. Whether the child node reasoning answer and the sibling node reasoning answer exist or not is determined when the atomic reasoning tree is constructed, so when reasoning, if there is a child node reasoning answer, the child node reasoning answer is used, if there is no child node reasoning answer, the sibling node reasoning answer is used, and if there is no child node reasoning answer and sibling node reasoning answer, the retrieval-knowledge is finally used. That is to say, only in the case where the child node reasoning answer and the sibling node reasoning answer cannot be used to reason the accurate result of the current sub-problem, the retrieval-knowledge is used to assist in reasoning the current sub-problem.

[0045] It should also be noted that the retrieval-knowledge in the atomic problem reasoning process and the retrieval-knowledge in the sub-problem reasoning process are both obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources, and the sources of the two are the same, but the substantive content of the two is determined by the sub-problem and the atomic problem, and is different.

[0046] In this embodiment, an atomic reasoning tree is constructed based on the original problem to be solved; wherein the atomic reasoning tree includes parent nodes and leaf nodes, the root node in the parent nodes is the original problem to be solved, the non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator includes a search operator, a relationship operator and a filter operator; then, for the leaf nodes, the atomic problems are reasoned based on the large language model and the atomic knowledge operator; for the parent nodes, the sub-problems are reasoned based on the large language model, the answers of the child nodes, the answers of the sibling nodes or the search-knowledge, to complete the reasoning process of the atomic reasoning tree from the leaf nodes to the root node from bottom to top, and obtain the reasoning result of the original problem to be solved; wherein the search-knowledge is obtained by performing dynamic knowledge search from multiple heterogeneous knowledge sources. This method realizes more accurate reasoning results by decomposing the original problem to be solved to the fine granularity of the atomic level and making the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator. At the same time, by allowing dynamic knowledge search from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, different types of queries can be flexibly handled, more rich, accurate and complementary information can be provided, and the overall reasoning ability of the algorithm is enhanced.

[0047] On the basis of the above-mentioned embodiments, further, the types and functions of the atomic knowledge operators will be described in detail below.

[0048] It is easy to understand that this embodiment designs three general atomic knowledge operators for the leaf nodes (atomic problems), namely: search operator, relationship operator and filter operator. These three basic atomic operators can be flexibly combined and applied to multiple heterogeneous knowledge sources.

[0049] In a specific embodiment, the search operator is described in detail.

[0050] The search operator is designed specifically for entity disambiguation, that is, to accurately retrieve the required entity from a large pool of entities, especially when multiple entities share similar names. In a knowledge graph, the search operator can be compared to accurately positioning the initial entity node in the graph.

[0051] This embodiment defines the search operator as follows formula (1).

[0052] (1).

[0053] According to formula (1), the search operator contains two inputs, namely the entity name and the optional entity descriptor to assist in disambiguation, and the output of the search operator is the target entity. If no optional entity descriptor is provided, the search operator will return the most popular entity.

[0054] For example, suppose we want to find a person named "John", but there can be many people named "John", so some additional information is needed to determine which "John" is intended. To do this, we can invoke the search operator , which returns those people named John who live in New York. Here, John is an entity name, and "lives in New York" is an optional entity descriptor.

[0055] In another specific embodiment, the relationship operator is described in detail.

[0056] The relationship operator is designed for single-hop reasoning. In a knowledge graph, the relationship operator can be compared to an operation that processes the knowledge from a head entity to a tail entity. There are three possibilities for single-hop reasoning: given a head entity and a relationship, retrieve a tail entity, given an entity and a property, retrieve a property value, and retrieve a relationship between a head entity and a tail entity. These three possibilities can be described as equations (2)-(4) as follows.

[0057] (2).

[0058] (3).

[0059] (4).

[0060] In equations (2)-(4), denotes a head entity, denotes a tail entity, denotes a relationship between a head entity and a tail entity, denotes a property.

[0061] According to equations (2)-(4), the relationship operator can take a head entity and a relationship as input and output a tail entity, or take a head entity and a property as input and output a property value, or take a head entity and a tail entity as input and output a relationship between the head entity and the tail entity.

[0062] In yet another specific embodiment, the filter operator is described in detail.

[0063] The filter operator is designed for entity filtering. In a knowledge graph, the filter operator is similar to filtering an initial entity set to obtain a subset of entities that satisfy the property conditions. This embodiment defines the filter operator as equation (5) as follows.

[0064] (5).

[0065] In equation (5), denotes a property constraint. ​

[0066] According to formula (5), the filtering operator takes the entity set and the attribute constraint condition as input, and outputs the entity satisfying the attribute constraint condition.

[0067] In the embodiment, by making the leaf node in the atomic reasoning tree correspond to an atomic knowledge operator, the atomic knowledge operator is one of the search operator, the relationship operator and the filtering operator, a more accurate reasoning result is realized.

[0068] On the basis of the above embodiment, further, the reasoning process of the leaf node (atomic problem) will be described in detail.

[0069] Based on the large language model and the atomic knowledge operator, the atomic problem is reasoned according to the search-knowledge, including: inputting the atomic problem, a plurality of heterogeneous knowledge sources to be selected and corresponding applicable scenarios and examples into the large language model to obtain a target knowledge source; according to the atomic problem and the parameters of the corresponding atomic knowledge operator, searching in the target knowledge source to obtain a target search-knowledge; taking the target search-knowledge and the atomic problem as inputs of the atomic knowledge operator, and outputting a reasoning result of the atomic problem; wherein the heterogeneous knowledge sources at least include online web pages, text corpus and structured knowledge base.

[0070] It is easy to understand that the process of reasoning the atomic problem by using the predefined atomic knowledge operator is mainly realized through the following three steps.

[0071] First, dynamic knowledge source selection.

[0072] Specifically, the previous work is to retrieve knowledge from static knowledge sources, while the embodiment adopts dynamic knowledge source selection on each atomic problem, which enables the embodiment to flexibly identify the most suitable knowledge source to answer each atomic problem.

[0073] In detail, the embodiment uses the large model as a dynamic knowledge source selector through context learning, inputs the atomic problem, a plurality of heterogeneous knowledge sources to be selected and corresponding applicable scenarios and examples into the large language model to obtain the output target knowledge source, and the target knowledge source is used for the subsequent multi-source knowledge search step.

[0074] It should be noted that the number of target knowledge sources can be one or multiple, which is not limited here.

[0075] Then, multi-source knowledge search.

[0076] After selecting the appropriate knowledge source for each atomic question, the embodiment will search in the target knowledge source according to the atomic question and the parameters of the corresponding atomic knowledge operator (i.e. the inputs in the definition of the search operator, the relationship operator and the filter operator described above, which can be extracted from the atomic question) to obtain the target retrieval-knowledge. The retrieval method varies depending on the type of knowledge source, for example, the embodiment queries the Web knowledge source through the search engine API, queries the text knowledge source through the dense retriever, and queries the knowledge graph through the structured graph query language.

[0077] For Web knowledge sources and text knowledge sources, the embodiment respectively retrieves the top k paragraphs and articles as target retrieval-knowledge, where k is a pre-defined hyperparameter. For knowledge graphs, the embodiment returns a complete list of structured answers.

[0078] Finally, atomic operator reasoning.

[0079] With the atomic question and the target retrieval-knowledge, the atomic knowledge operator can start execution. To ensure the flexibility and robustness of the method, the embodiment uses a large model as an adaptive executor instead of static symbolic code to execute the atomic knowledge operator, takes the target retrieval-knowledge and the atomic question as the input of the atomic knowledge operator, and outputs the reasoning result of the atomic question, which is the process of atomic operator reasoning. Finally, the output of the adaptive executor is taken as the local answer of the relevant atomic question.

[0080] Thus, accurate reasoning of the atomic question can be achieved.

[0081] In the embodiment, by inputting the atomic question, the multiple heterogeneous knowledge sources to be selected and their corresponding applicable scenarios and example inputs into the large language model, the target knowledge source is obtained, and according to the atomic question and the parameters of the corresponding atomic knowledge operator, the target knowledge source is searched to obtain the target retrieval-knowledge, and then the target retrieval-knowledge and the atomic question are taken as the input of the atomic knowledge operator, and the reasoning result of the atomic question is output, which realizes more accurate reasoning result. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources, different types of queries can be flexibly handled, more rich, accurate and complementary information can be provided, and the overall reasoning ability of the algorithm is enhanced.

[0082] On the basis of the above embodiment, further, the reasoning process of the parent node (sub-question) will be described in detail.

[0083] Based on the large language model, the sub-problem is reasoned according to the child node reasoning answer, the sibling node reasoning answer or the retrieval-knowledge, including: in the case that the child node reasoning answer satisfies the reasoning condition, the child node reasoning answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; in the case that the child node reasoning answer does not satisfy the reasoning condition, but the sibling node reasoning answer satisfies the reasoning condition, the sibling node reasoning answer and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem; in the case that the child node reasoning answer and the sibling node reasoning answer do not satisfy the reasoning condition, the retrieval-knowledge and the sub-problem are input into the large language model to obtain the reasoning result of the sub-problem.

[0084] It is easy to understand that in an atomic reasoning tree, all nodes that do not call atomic knowledge operators are regarded as parent nodes, and the parent nodes may undergo child node reasoning answer reasoning, sibling node reasoning answer reasoning or direct retrieval-enhanced reasoning.

[0085] First, the sub-problem reasoning according to the child node reasoning answer, that is, the process of inferring the parent node answer by synthesizing the reasoning answers of the child nodes, can be defined as formula (6) as follows.

[0086] (6).

[0087] In formula (6), is the sub-problem of the current parent node, is the question-answer pair list of the child nodes of the current parent node.

[0088] The process of sub-problem reasoning according to the child node reasoning answer is realized by large language model context learning. Specifically, the child node reasoning answer and the sub-problem of the current parent node are input into the large language model together, and the reasoning result of the sub-problem can be obtained.

[0089] It should be noted that the possibility of not being able to obtain a suitable reasoning result needs to be considered in this process, that is, whether the child node reasoning answer satisfies the reasoning condition needs to be considered. The specific judgment process is to add an instruction prompt in the input of the large language model, and the instruction prompt here is, for example, “whether the child node reasoning answer can accurately reason the current sub-problem”. If the answer of the large language model is affirmative, the child node reasoning answer satisfies the reasoning condition, otherwise the child node reasoning answer does not satisfy the reasoning condition, at which time the sub-problem reasoning according to the sibling node reasoning answer is considered.

[0090] The process of sub-problem reasoning according to the sibling node reasoning answer is similar to the process of sub-problem reasoning according to the child node reasoning answer. The process of inferring the parent node answer by synthesizing the reasoning answers of the sibling nodes can be defined as formula (7) as follows.

[0091] (7).

[0092] In formula (7), is the sub-problem of the current parent node, is the problem-answer pair list of the sibling node of the current parent node.

[0093] The process of sub-problem reasoning according to the sibling node reasoning answer is also implemented through large language model context learning. Specifically, the sibling node reasoning answer and the sub-problem of the current parent node are input into the large language model, and the reasoning result of the sub-problem can be obtained.

[0094] It also needs to be explained that the possibility of not being able to obtain a suitable reasoning result needs to be considered in this process, that is, whether the sibling node reasoning answer meets the reasoning condition needs to be considered. The specific judgment process is to add an instruction prompt in the input of the large language model, such as "whether the sibling node reasoning answer can accurately reason the current sub-problem". If the answer of the large language model is affirmative, the sibling node reasoning answer meets the reasoning condition, otherwise, the sibling node reasoning answer does not meet the reasoning condition, at this time, sub-problem reasoning according to retrieval-knowledge is considered.

[0095] Each parent node mainly implements sub-problem reasoning through sub-node reasoning answer or sibling node reasoning answer, but how does the parent node obtain the answer according to the sub-node reasoning answer and the sibling node reasoning answer. This embodiment carries out sub-problem reasoning according to retrieval-knowledge, that is, direct retrieval enhanced reasoning is adopted.

[0096] Specifically, first, the current sub-problem is selected dynamically, then the is used as a query statement to start multi-source knowledge retrieval, finally, given and the retrieved knowledge, the large model is used to generate reasoning results through standard knowledge retrieval based on context learning. This design ensures that although this embodiment mainly focuses on knowledge retrieval on the leaf node, external knowledge is flexibly retrieved at the parent node when necessary.

[0097] In the embodiment, the reasoning result of the sub-problem is obtained by inputting the sub-node reasoning answer and the sub-problem to the large language model when the sub-node reasoning answer meets the reasoning condition, the reasoning result of the sub-problem is obtained by inputting the sibling node reasoning answer and the sub-problem to the large language model when the sub-node reasoning answer does not meet the reasoning condition but the sibling node reasoning answer meets the reasoning condition, and the reasoning result of the sub-problem is obtained by inputting the retrieval-knowledge and the sub-problem to the large language model when neither the sub-node reasoning answer nor the sibling node reasoning answer meets the reasoning condition, so that more accurate reasoning results are realized. At the same time, by allowing dynamic retrieval of knowledge from multiple heterogeneous knowledge sources, different types of queries can be flexibly responded to, more rich, accurate and complementary information is provided, and the overall reasoning capability of the algorithm is enhanced.

[0098] In addition, Figure 2 The reasoning process schematic diagram of the atomic reasoning tree provided by the embodiment of the application is shown, and the reasoning processes of the leaf nodes and the parent nodes are particularly shown.

[0099] As Figure 2 shown, it is first determined whether the node is a leaf node.

[0100] If the node is a leaf node, the atomic problem and the parameters of the atomic knowledge operator are prepared, the large language model is used as a dynamic knowledge source selector to determine the target knowledge source , further, knowledge retrieval is performed in the target knowledge source based on the parameters of the atomic knowledge operator to obtain the target retrieval-knowledge , finally, the atomic knowledge operator is called to perform reasoning of the atomic problem, the inputs of which are the atomic problem , the parameters of the atomic knowledge operator and the target retrieval-knowledge , and the reasoning answer of the atomic problem is obtained.

[0101] If the node is not a leaf node, it is a parent node, in which case, the sub-problem and the sibling node reasoning answer and the sub-node reasoning answer are prepared. Preferentially, reasoning is performed by using the sub-problem and the sub-node reasoning answer, if the answer cannot be obtained, reasoning is performed by using the sibling node reasoning answer, if the answer still cannot be obtained, reasoning is performed by using the sub-problem and the retrieval-knowledge , and the reasoning result is obtained. The retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0102] Thus, reasoning is performed from the leaf node to the root node of the atomic reasoning tree from bottom to top, and finally the reasoning result of the original problem to be solved is obtained.

[0103] Figure 3 The overall flowchart of the method for heterogeneous knowledge reasoning of the large language model based on the atomic knowledge operator provided by the embodiments of the present application is shown.

[0104] As shown in Figure 3 The left side shows three atomic knowledge operators related to the atomic problem, namely the search operator Search, the relationship operator Relate and the filter operator Filter. Figure 3 The right side shows the reasoning process of the leaf nodes and parent nodes in the atomic reasoning tree provided by the embodiments of the present application, which has been detailed in the above embodiments and will not be repeated here.

[0105] In some other embodiments, a data set for evaluating the heterogeneous knowledge reasoning is also constructed.

[0106] According to the problem sample and the multiple heterogeneous knowledge sources, the evaluation data set is constructed; the F1 score is used as the evaluation index, and the evaluation data set is used to verify the method for heterogeneous knowledge reasoning of the large language model; wherein the F1 score is the harmonic mean of the precision and the recall.

[0107] It is easy to understand that for a problem sample, it is retrieved through multiple-hop knowledge sources to construct a multiple-hop sample pair, i.e. problem sample-knowledge source 1-knowledge source 2-knowledge source 3, and the like. Thus, the evaluation data set can be constructed. Then, the prior art and the method for heterogeneous knowledge reasoning of the large language model based on the atomic knowledge operator provided by the embodiments of the present application are tested on the evaluation data set.

[0108] Specifically, the embodiments are evaluated on three traditional single-knowledge-source complex reasoning data sets HotpotQA, 2WikiMultiHop, Musique and the multi-knowledge-source complex reasoning data sets CRAG and the evaluation data set BlendQA constructed by the embodiments. The experiments show that the effect of the present application is much better than the existing technology, and the F1 score is improved by 5.4%, 9.4% and 1.2% respectively on the three single-knowledge-source data sets, and the F1 score is improved by 9.5% and 6.6% respectively on the two multi-knowledge-source data sets. Compared with the existing method, the present application significantly reduces the illusion of knowledge-intensive reasoning of the large model, and is more efficient and economical than the existing method, and can achieve better results with less API calls and computing resources than the existing method.

[0109] Corresponding to the method for heterogeneous knowledge reasoning of the large language model based on the atomic knowledge operator described in the above embodiments, the present application also provides a device for heterogeneous knowledge reasoning of the large language model based on the atomic knowledge operator.

[0110] Specifically, Figure 4 A structural schematic diagram of the atomic-knowledge-operator-based large language model heterogeneous knowledge reasoning device provided by the embodiment of the present application is shown.

[0111] As Figure 4 shown, the device comprises: an atomic reasoning tree construction module 410, configured to construct an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, a root node in the parent nodes is the original problem to be solved, non-root nodes in the parent nodes are different level sub-problems decomposed from the original problem to be solved, and the leaf nodes are atomic problems decomposed from the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filtering operator; a heterogeneous knowledge reasoning module 420, configured to, for the leaf nodes, perform reasoning on the atomic problems based on a large language model and the atomic knowledge operator according to search-knowledge; for the parent nodes, perform reasoning on the sub-problems based on the large language model according to a child node reasoning answer, a sibling node reasoning answer or search-knowledge; perform reasoning on the atomic reasoning tree from the leaf nodes to the root node from bottom to top to obtain a reasoning result of the original problem to be solved; wherein the search-knowledge is obtained by performing dynamic knowledge search from multiple heterogeneous knowledge sources.

[0112] In the embodiment, the atomic reasoning tree construction module 410 constructs an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, a root node in the parent nodes is the original problem to be solved, non-root nodes in the parent nodes are different level sub-problems decomposed from the original problem to be solved, and the leaf nodes are atomic problems decomposed from the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filtering operator; then, for the leaf nodes, the heterogeneous knowledge reasoning module 420 performs reasoning on the atomic problems based on a large language model and the atomic knowledge operator according to search-knowledge; for the parent nodes, the heterogeneous knowledge reasoning module 420 performs reasoning on the sub-problems based on the large language model according to a child node reasoning answer, a sibling node reasoning answer or search-knowledge, to complete the process of reasoning on the atomic reasoning tree from the leaf nodes to the root node from bottom to top, and obtain a reasoning result of the original problem to be solved; wherein the search-knowledge is obtained by performing dynamic knowledge search from multiple heterogeneous knowledge sources. The device decomposes the original problem to be solved to the atomic level of fine granularity, and makes the leaf nodes in the atomic reasoning tree correspond to an atomic knowledge operator, to achieve a more accurate reasoning result. At the same time, by allowing dynamic search of knowledge from multiple heterogeneous knowledge sources at each sub-problem and atomic problem node, different types of queries can be flexibly coped with, more rich, accurate and complementary information is provided, and the overall reasoning ability of the algorithm is enhanced.

[0113] It should be noted that the large language model heterogeneous knowledge reasoning device based on atomic knowledge operators provided in the embodiment of the present invention can be referenced to each other with the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators described in the above embodiments, and will not be repeated here.

[0114] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute a large language model heterogeneous knowledge reasoning method based on atomic knowledge operators, the method including: constructing an atomic reasoning tree based on the original question to be answered; wherein, the atomic reasoning tree includes a parent node and a leaf node, the root node in the parent node is the original question to be answered, the non-root nodes in the parent node are sub-problems of different levels decomposed from the original question to be answered, the leaf nodes are atomic problems decomposed from the original question to be answered, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators and filtering operators; for leaf nodes, based on the large language model and the atomic knowledge operator, the atomic problem is reasoned according to retrieval-knowledge; for parent nodes, based on the large language model, the answer is inferred according to the child node, the answer is inferred according to the sibling node, or the retrieval-knowledge is inferred for the sub-problem; the atomic reasoning tree is reasoned from the leaf node to the root node from the bottom up to obtain the reasoning result of the original question to be answered; wherein, the retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources.

[0115] In addition, the logic instructions in the memory 530 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0116] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor, so that the computer can execute the atomic knowledge operator-based large language model heterogeneous knowledge reasoning method provided by the above-mentioned method, the method comprises: constructing an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, the root node in the parent nodes is the original problem to be solved, the non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filtering operator; for the leaf node, based on the large language model and the atomic knowledge operator, the atomic problem is reasoned according to search-knowledge; for the parent node, based on the large language model, the sub-problem is reasoned according to the reasoning answer of the child node, the reasoning answer of the brother node or search-knowledge; the atomic reasoning tree is reasoned from the leaf node to the root node from bottom to top, and the reasoning result of the original problem to be solved is obtained; wherein the search-knowledge is obtained by executing dynamic knowledge search from a plurality of heterogeneous knowledge sources.

[0117] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for heterogeneous knowledge reasoning of large language model based on atomic knowledge operator as described above, and the method comprises: constructing an atomic reasoning tree based on an original problem to be solved; wherein the atomic reasoning tree comprises parent nodes and leaf nodes, a root node in the parent nodes is the original problem to be solved, non-root nodes in the parent nodes are different level sub-problems obtained by decomposing the original problem to be solved, and the leaf nodes are atomic problems obtained by decomposing the original problem to be solved, each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operator comprises a search operator, a relationship operator and a filtering operator; for the leaf nodes, the atomic problems are reasoned based on a large language model and the atomic knowledge operator according to search-knowledge; for the parent nodes, the sub-problems are reasoned based on the large language model according to the reasoning answers of the child nodes, the reasoning answers of the sibling nodes or search-knowledge; the atomic reasoning tree is reasoned from the leaf nodes to the root node from bottom to top to obtain a reasoning result of the original problem to be solved; and the search-knowledge is obtained by performing dynamic knowledge search from multiple heterogeneous knowledge sources.

[0118] The device embodiments described above are only schematic, wherein the units illustrated as separate components may or may not be physically separate, and the components illustrated as a unit may or may not be physical units, i.e., may be located in one place or distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0119] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0120] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A large language model heterogeneous knowledge reasoning method based on atomic knowledge operators, characterized by: include: Based on the original question to be answered, an atomic reasoning tree is constructed; wherein the atomic reasoning tree includes a parent node and leaf nodes, the root node of the parent node is the original question to be answered, the non-root nodes of the parent node are sub-problems of different levels decomposed from the original question to be answered, and the leaf nodes are atomic problems decomposed from the original question to be answered. Each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators, and filtering operators; For leaf nodes, reasoning is performed on atomic questions based on retrieval-knowledge based on a large language model and atomic knowledge operators. For parent nodes, reasoning is performed on sub-questions based on child node inference answers, sibling node inference answers, or retrieval-knowledge based on a large language model. The atomic inference tree is reasoned from leaf nodes to the root node from bottom to top to obtain the inference result of the original question to be answered. The retrieval-knowledge is obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources. The atomic reasoning tree is constructed based on the original question to be answered, including: Input the original question to be answered, the question answering requirements, the definition of the atomic reasoning tree, the atomic knowledge operator information, and examples into the large language model to obtain the initial atomic reasoning tree; Index the nodes in the initial atomic reasoning tree according to a preset search order, and use reference placeholders to represent the actual answers to sub-questions and atomic questions to construct an atomic reasoning tree; The search operator takes an entity name and an optional entity descriptor as input and outputs a target entity; The relational operator takes a given head entity and a relationship as input and outputs a tail entity; or, the relational operator takes a given head entity and an attribute as input and outputs an attribute value; or, the relational operator takes a given head entity and a tail entity as input and outputs the relationship between the head entity and the tail entity; The filtering operator takes an entity set and attribute constraints as input, and outputs entities that meet the attribute constraints.

2. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to claim 1 is characterized in that: The method of reasoning about atomic questions based on retrieval-knowledge based on a large language model and atomic knowledge operators includes: Input the atomic question, multiple heterogeneous knowledge sources to be selected, and their corresponding applicable scenarios and examples into the large language model to obtain the target knowledge source; According to the atomic question and the parameters of its corresponding atomic knowledge operator, search in the target knowledge source to obtain the target retrieval-knowledge; Take the target retrieval-knowledge and atomic questions as inputs of the atomic knowledge operator and output the reasoning result of the atomic question; Among them, heterogeneous knowledge sources include at least online web pages, text corpora and structured knowledge bases.

3. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to claim 1 is characterized in that: The method of reasoning about sub-questions based on the large language model and the child node reasoning answers, the sibling node reasoning answers, or the retrieval-knowledge includes: If the inference answer of the sub-node meets the inference condition, the inference answer of the sub-node and the sub-question are input into the large language model to obtain the inference result of the sub-question; If the child node reasoning answer does not meet the reasoning condition, but the sibling node reasoning answer meets the reasoning condition, the sibling node reasoning answer and the sub-question are input into the large language model to obtain the reasoning result of the sub-question; When the inference answer of the child node and the inference answer of the sibling node do not meet the inference conditions, the retrieval-knowledge and the sub-question are input into the large language model to obtain the inference result of the sub-question.

4. The large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to any one of claims 1 to 3, characterized in that: Also includes: Construct an evaluation dataset based on problem samples and multiple heterogeneous knowledge sources; Using the F1 score as the evaluation metric, the large language model heterogeneous knowledge reasoning method is verified using the evaluation dataset; The F1 score is the harmonic mean of precision and recall.

5. A large language model heterogeneous knowledge reasoning device based on atomic knowledge operators, characterized by: include: An atomic reasoning tree construction module is used to construct an atomic reasoning tree based on the original question to be answered; wherein the atomic reasoning tree includes a parent node and leaf nodes, the root node of the parent node is the original question to be answered, the non-root nodes of the parent node are sub-problems of different levels decomposed from the original question to be answered, and the leaf nodes are atomic problems decomposed from the original question to be answered. Each leaf node corresponds to an atomic knowledge operator, and the atomic knowledge operators include search operators, relational operators, and filtering operators; A heterogeneous knowledge reasoning module is configured to reason about atomic questions based on retrieval-knowledge for leaf nodes based on a large language model and atomic knowledge operators; and to reason about sub-questions based on child nodes, sibling nodes, or retrieval-knowledge for parent nodes based on a large language model. The module also reasoned from the bottom up along the atomic reasoning tree from leaf nodes to the root node to obtain the inference result for the original question to be answered. The retrieval-knowledge was obtained by performing dynamic knowledge retrieval from multiple heterogeneous knowledge sources. The atomic reasoning tree is constructed based on the original question to be answered, including: Input the original question to be answered, the question answering requirements, the definition of the atomic reasoning tree, the atomic knowledge operator information, and examples into the large language model to obtain the initial atomic reasoning tree; Index the nodes in the initial atomic reasoning tree according to a preset search order, and use reference placeholders to represent the actual answers to sub-questions and atomic questions to construct an atomic reasoning tree; The search operator takes an entity name and an optional entity descriptor as input and outputs a target entity; The relational operator takes a given head entity and a relationship as input and outputs a tail entity; or, the relational operator takes a given head entity and an attribute as input and outputs an attribute value; or, the relational operator takes a given head entity and a tail entity as input and outputs the relationship between the head entity and the tail entity; The filtering operator takes an entity set and attribute constraints as input, and outputs entities that meet the attribute constraints.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the large language model heterogeneous knowledge reasoning method based on atomic knowledge operators according to any one of claims 1 to 4 is implemented.

Images