Structured Knowledge Query System Based on Large Language Model

Abstract

This invention discloses a structured knowledge query system based on a large language model, comprising a knowledge representation module, a function generation module, and a query module. The knowledge representation module converts structured knowledge into a condition graph, which includes a set of nodes and a set of conditions. The function generation module generates a query function based on the query prompts and query questions using the large language model. The query module performs knowledge queries based on the function in the condition graph to obtain query results. This system combines the high accuracy of structured query languages ​​with the good generalization ability of LLMs, achieving accurate querying of various types of structured knowledge without data leakage.

Description

Technical Field

[0001] This invention belongs to the field of structured knowledge storage and retrieval technology, specifically relating to a structured knowledge retrieval system based on a large language model. Background Technology

[0002] In recent years, with the continuous development of artificial intelligence technology, knowledge representation and reasoning have become one of the key themes in realizing artificial intelligence. Structured knowledge such as tables, triples, and temporal triples appear extensively in people's daily lives and have a profound impact on the development of human society. The representation and reasoning of structured knowledge has become an important part of the artificial intelligence era.

[0003] Many methods for representing and reasoning about structured knowledge have been proposed, such as knowledge graphs using the SPARQL query language, general graphs using graph query languages, and tables using the SQL query language. Mastering query languages ​​requires a significant amount of learning, and those who want to access structured data may be unfamiliar with them.

[0004] Existing structured knowledge querying using natural language can be divided into two categories: (1) translating NL questions into structured queries and obtaining answers by executing these queries, abbreviated as NL2SQ2A (natural language to structured query to answer), and (2) answering questions without relying on structured queries, abbreviated as NL2A (natural language to answer). Each of these methods has its advantages and disadvantages, and the specific application depends on the complexity of the question and the accuracy requirements.

[0005] The conversion of natural language into structured queries is known as NL2SQ, and NL2SQ methods encompass several different strategies. First, architecture-based methods construct SQL queries by analyzing the database structure and data indexes. Second, parsing-based methods rely on parsing grammatical structures to understand the meaning of natural language questions. Additionally, neural machine translation-based methods formalize the NL2SQ task as a language translation problem. Pre-trained language models (PLMs) possess excellent language understanding capabilities and are therefore also applied to the NL2SQL task. These methods typically fine-tune PLMs on labeled training sets to enable them to generate accurate SQL queries.

[0006] In the context of Knowledge Graph Question Answering (KGQA), the commonly used query language is SPARQL; therefore, the conversion from natural language to structured queries is called NL2SPARQL. NL2SPARQL methods typically include three steps: question understanding, linking, and filtering. In the question understanding stage, some methods use NL parsers, such as dependency parsing, constituent parsing, and AMR parsing, while others use sequence-to-sequence generation methods to obtain graph patterns. The linking step maps elements extracted from the question to entities and relations in the knowledge graph, typically employing techniques such as mapping dictionaries, indexing systems, and embedding generation systems. In the filtering step, type constraints are usually applied to filter answers. LLMs are used; some methods, such as KB-BINDER, utilize LLMs to generate a logical draft of the question and bind elements to entities and relations in the knowledge graph with a few hints. However, most NL2SPARQL methods are trained for specific query languages, and sometimes even specific datasets, therefore they are not a general solution for structured knowledge queries.

[0007] Unlike NL2SQ2A, NL2A directly generates the answer without requiring the steps of generating and executing a structured query. For example, some research views knowledge graph question answering tasks as multi-hop reasoning along relations until the answer is reached. Some methods train a text encoder to encode the question and use an inference module (such as a graph neural network) to perform multi-hop reasoning. Others employ a unified model that handles both text encoding and inference. These methods require training on specific datasets because their performance is highly dependent on the features of the dataset.

[0008] To propose a more general pre-training solution applicable across datasets, some studies have proposed TableGPT. TableGPT is a unified fine-tuning framework designed to enable LLMs to understand and manipulate tabular data consistent with natural language. It pre-trains a general table encoder and fine-tunes the LLM with prompts on a broad corpus, with table embeddings as input. However, these pre-training methods are often only applicable to specific types of structured knowledge. On the other hand, the pioneering work StructGPT is a solution applicable to different types of structured knowledge. It extracts relevant knowledge from structured data through interfaces to different types of structured data, linearizes it, and inputs it into the LLM to directly generate answers. However, StructGPT inputs a large amount of structured knowledge into the prompts of the LLM, posing a high potential risk of data leakage. Summary of the Invention

[0009] In view of the above, the purpose of this invention is to provide a structured knowledge query system based on a large language model, which combines the high accuracy of structured query languages ​​with the good generalization ability of LLMs, and achieves accurate querying of various types of structured knowledge without leaking data.

[0010] To achieve the above-mentioned objectives, embodiments of the present invention provide a structured knowledge query system based on a large-scale language model, including a knowledge representation module, a function generation module, and a query module;

[0011] The knowledge representation module is used to convert structured knowledge into a condition graph, which includes a set of nodes composed of knowledge elements and a set of conditions composed of condition triples.

[0012] The function generation module is used to generate functions for querying based on consultation prompts and query questions using a large language model;

[0013] The query module is used to perform knowledge queries based on functions in the condition graph to obtain query results.

[0014] Preferably, converting structured knowledge into a conditional graph includes:

[0015] When structured knowledge is represented as a table with m rows and n columns, the first row records the semantic meaning of each column, and each row from the second row onwards represents a knowledge record. When converting the table into a conditional graph, a number equal to the row order is added to each row, and two conditional triples are generated for each record value except for the first row, represented as follows:

[0016]

[0017] in, and These represent the conditional charts corresponding to the tables. The set of nodes and the set of conditions in the data. i represents the row index, j represents the column index, and v represents the column index. i,j v represents the record value in the i-th row and j-th column. 1,j This represents the semantic meaning of the first row and j-th column, i.e., the column name. (i, v) 1,j ,[]) indicates that a row index can form an empty triple with the column name without any conditions, (v 1,j ,v i,j [i]) indicates that v is made possible by the row information [i]. 1,j With v i,j Interrelated conditions form a triplet.

[0018] Preferably, converting structured knowledge into a conditional graph includes:

[0019] When structured knowledge is transformed into a knowledge graph When, where ε1, and These are the entity set, the relation set, and the triple set, respectively. It represents the set of triples representing the relationship r1 between entities (h1, t1), and is used in the knowledge graph. Converted to a conditional graph, it is represented as:

[0020]

[0021] in, and Representing knowledge graphs Corresponding condition diagram The set of nodes and the set of conditions in the data. (h1,r1,[]) means that h1 and r1 can be connected without any conditions to form a triple with no conditions. (r1,t1,[h1]) means that r1 and t1 are connected to form a condition triple through the head entity [h1].

[0022] Preferably, converting structured knowledge into a conditional graph includes:

[0023] When structured knowledge is transformed into a time-based knowledge graph Where ε2, and These are the entity set, the relation set, and the time set, respectively. It is a set of quintuples. Let the start and end times of the fact (h2, r2, t2) be represented. The time knowledge graph is then converted into a conditional graph, represented as follows:

[0024]

[0025] in, and Representing time knowledge graphs Corresponding condition diagram The set of nodes and the set of conditions, Will Each (h2, r2, t2, st, ed) in the equation is converted into at least 5 conditional triples to construct the equation. The five conditional triples are (h2,r2,[]), (r2,t2,h2), (start time,st,[h2,r2,t2]), (end time,et,[h2,r2,t2]), and (time,time,[h2,r2,t2]), where time ∈ time interval [st,et], start time represents the start time, and end time represents the end time. (h2,r2,[]) indicates that conditions h2 and r2 are not required to form an empty conditional triple. (r2,t2,h2) indicates that the conditional triple is formed by the conditional header entity h2, relation r2, and t2. (start time,st,[h2,r2,t2]) uses the conditional triple [h2,r2,t2], the start time, and the time st to form a conditional triple indicating the start time of the triple occurrence. (end time,st,[h2,r2,t2]) (time,et,[h2,r2,t2]) represents the termination time, and (time,time,[h2,r2,t2]) represents the occurrence time.

[0026] Preferably, the function generated by the large language model includes at least one of the following: a predefined language query function, an inference function, and an undefined random function;

[0027] A language query function refers to the step-by-step query steps to solve a consultation problem, including information about head entities, tail entities, relations, keys, and values;

[0028] Inference functions include set operations and statistical computations, used to process and infer knowledge in conditional graphs;

[0029] An undefined random function refers to a function randomly generated by a large language model without prompting or learning.

[0030] Preferably, the process of generating the inference function includes:

[0031] The consultation prompts and questions are input into the generative large language model. The large language model obtains a preliminary query inference function through autoregressive reasoning based on the prediction of the next token. Then, the parameter with the highest similarity among the optional parameters of the preliminary query inference function is selected as the final inference function to achieve parameter alignment. The consultation prompts include generation instructions and examples of generated inference functions.

[0032] Preferably, the step of obtaining query results by performing knowledge retrieval based on functions in a condition graph includes:

[0033] When the large language model generates query functions that only include knowledge query functions, knowledge queries are performed in the condition graph based on these knowledge query functions, including:

[0034] Element Mapping and Alignment: Align and map the knowledge elements in the knowledge query function with the knowledge elements in the consultation question, that is, replace the corresponding knowledge elements in the knowledge query function with the knowledge elements in the consultation question to obtain the aligned knowledge query function;

[0035] Function transformation: The aligned knowledge query function is transformed into an execution query function, which includes three types of query functions: query node functions, query condition functions, and comparison functions.

[0036] Knowledge Query: The query results are obtained by executing the query function to search the condition graph.

[0037] Preferably, when the large language model generates functions for querying that include knowledge query functions and inference functions, knowledge querying is performed on the condition graph based on the knowledge query functions and inference functions. This includes element mapping and alignment steps for the knowledge query functions, function transformation steps, and also includes:

[0038] In the knowledge query step, query results are obtained by executing query functions and inference functions to search the condition graph. This includes: first, executing query functions to search the condition graph to obtain candidate query results; then, using inference functions to infer from the candidate query results to obtain the final query result.

[0039] Alternatively, the query conditions can be obtained first through the inference function, and then the query function can be executed in combination with the query conditions to obtain the final query result.

[0040] Preferably, the query module further includes a predefined LLM function. When the large language model generates a query function that includes at least one of a predefined language query function, an inference function, and an undefined random function, and the predefined language query function and / or inference function cannot solve the query problem, or only an undefined random function appears, the LLM function is used to generate query results through the large language model.

[0041] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0042] By converting structured knowledge into conditional graphs, a unified knowledge representation is achieved, facilitating subsequent knowledge retrieval. Furthermore, to prevent data leakage, a function for querying the conditional graphs is generated based on a large language model. This function is then used to perform knowledge queries within the conditional graphs, enabling accurate retrieval of structured knowledge while protecting data privacy. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the structure of the structured knowledge query system based on a large language model provided in the embodiment;

[0045] Figure 2 This is a sample query process diagram provided in the embodiment, using the GPT model as an LLM and real data from WikiSQL;

[0046] Figure 3 This is a graph showing the accuracy of question-and-answer representations of fixed-pattern knowledge provided in the embodiment;

[0047] Figure 4 This is a diagram of the Hit@1 results for question-and-answering of factual knowledge provided in the embodiment;

[0048] Figure 5 This is a Hit@1 result diagram of the question-and-answer session on temporal knowledge provided in the embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0050] like Figure 1 As shown, the structured knowledge query system 100 based on a large language model provided in the embodiment includes a knowledge representation module 110, a function generation module 120, and a query module 130. The knowledge representation module is used to convert structured knowledge into a condition graph; the function generation module is used to generate a function for querying based on the consultation prompts and query questions using a large language model; and the query module is used to perform knowledge querying based on the function in the condition graph to obtain query results.

[0051] In this embodiment, to address the current problem of inconsistent knowledge representation, a unified symbolic knowledge representation method called Conditional Graph (CG) is proposed to apply the solution to different types of structured knowledge. The Conditional Graph CG consists of a node set N composed of knowledge elements and a condition set T composed of condition triples, i.e., CG = {N, T}. The knowledge elements in the node set N include entities, relations, attributes, and values, which serve as nodes. Each knowledge condition in the condition set T represents a connection relationship between nodes; that is, each connection relationship has a knowledge condition, which is represented by a condition triple.

[0052] In this embodiment, to achieve a unified representation of structured data on the condition graph, the knowledge representation module 110 uses three different types of CG transformation functions to map tables, knowledge graphs, and time-series knowledge graphs to the condition graph, respectively. Specifically, this includes:

[0053] When structured knowledge is represented as a table with m rows and n columns, the first row records the semantic meaning of each column, and each row from the second row onwards represents a knowledge record. When converting the table into a conditional graph, a number equal to the row order is added to each row, and two conditional triples are generated for each record value except for the first row, represented as follows:

[0054]

[0055] in, and These represent the conditional charts corresponding to the tables. The set of nodes and the set of conditions in the data. i represents the row index, j represents the column index, and v represents the column index. i,j v represents the record value in the i-th row and j-th column. 1,j This represents the semantic meaning of the first row and j-th column, i.e., the column name. (i, v) 1,j ,[]) indicates that a row index can form an empty triple with the column name without any conditions, (v 1,j ,v i,j [i]) indicates that v is made possible by the row information [i]. 1,j With v i,j Interrelated conditions form a triplet.

[0056] When structured knowledge is transformed into a knowledge graph When, where ε1, and These are the entity set, the relation set, and the triple set, respectively. It represents the set of triples representing the relationship r1 between entities (h1, t1), and is used in the knowledge graph. Converted to a conditional graph, it is represented as:

[0057]

[0058] in, and Representing knowledge graphs Corresponding condition diagram The set of nodes and the set of conditions in the data. (h1,r1,[]) means that h1 and r1 can be connected without any conditions to form a triple with no conditions. (r1,t1,[h1]) means that r1 and t1 are connected to form a condition triple through the head entity [h1].

[0059] When structured knowledge is transformed into a time-based knowledge graph Where ε2, and These are the entity set, the relation set, and the time set, respectively. It is a set of quintuples. {st,et}∈T2 represents the start and end times of fact (h2,r2,t2). The time knowledge graph is converted into a conditional graph, represented as follows:

[0060]

[0061] in, and Representing time knowledge graphs Corresponding condition diagram The set of nodes and the set of conditions, Will Each (h2, r2, t2, st, ed) in the equation is converted into at least 5 conditional triples to construct the equation. The five conditional triples are (h2,r2,[]), (r2,t2,h2), (start time,st,[h2,r2,t2]), (end time,et,[h2,r2,t2]), and (time,time,[h2,r2,t2]), where time ∈ time interval [st,et], start time represents the start time, and end time represents the end time. (h2,r2,[]) indicates that conditions h2 and r2 are not required to form an empty conditional triple. (r2,t2,h2) indicates that the conditional triple is formed by the conditional header entity h2, relation r2, and t2. (start time,st,[h2,r2,t2]) uses the conditional triple [h2,r2,t2], the start time, and the time st to form a conditional triple indicating the start time of the triple occurrence. (end time,st,[h2,r2,t2]) (time,et,[h2,r2,t2]) represents the termination time, and (time,time,[h2,r2,t2]) represents the occurrence time.

[0062] In this embodiment, three types of functions are defined to implement knowledge-based question answering on a condition graph: query functions, inference functions, and LLM functions.

[0063] To minimize data leakage, a user-friendly query function for the large language model was designed, allowing it to learn how to write query functions effectively with minimal hints. Specifically, the designed query functions include a unified Language Query Function (LQF) and an Execution Query Function (EQF). The large language model writes easy-to-learn language query functions based on structured data and few-shot (small sample) query hints. That is, with the help of the user-friendly query function, inputting some structured data and a small number of query hints is sufficient for the large language model to learn how to write knowledge query functions, without needing to input the structured data directly into the model. The language query function (LQF) is then translated into the execution query function (EQF) that is actually executed during knowledge retrieval, thus realizing the function of acquiring knowledge from CG. The unified language query function (LQF) is as follows:

[0064] get_information(head_entity,relation,tail_entity,key,value)

[0065] Language query functions refer to the step-by-step query steps for resolving consultation problems. They aim to make LLM (Local Language Query Model) easier to write and generate, and facilitate subsequent queries. They are designed using common concepts such as head entity, relation, tail entity, key, and value; that is, language query functions include information about head entity, tail entity, relation, key, and value. The purpose of Execute Query Functions (EQF) is to execute queries and conditional operations to obtain query knowledge from the condition graph. Specifically, Execute Query Functions include the following functions: search node function (search_node), search condition function (search_condition), and compare function (compare), as follows:

[0066] Query node function:

[0067] search_node(node1,condition,scope)

[0068] Search for and return node2 within the condition facts contained in the scope of node1 and condition. scope=None by default means that the scope is all condition facts in the condition triple, and condition=None means any condition.

[0069] Query condition functions:

[0070] search_condition(node1,node2_value,op)

[0071] Search for and return the condition in the comparison relationship between node1 and node2, i.e., compare(op, node2_value, node2);

[0072] Query comparison functions:

[0073] compare(op,val1,val2)

[0074] Return True if val1{op}val2 satisfies the condition, otherwise return False, where op is a comparison symbol {>, <, =, ≥, ≤}.

[0075] Inference functions include easily understood operations such as set operations and statistical calculations, used to process and infer knowledge in conditional graphs. Set operations include union, intersection, difference, and negation, while statistical calculations include mean, maximum, minimum, and count. Specifically:

[0076] Set operation functions: set_union(set1,set2) finds the union of two sets, set_intersection(set1,set2) finds the intersection of two sets, and set_difference(set1,set2) finds the difference between two sets;

[0077] Calculation functions: mean() calculates the average value, max() calculates the maximum value, min() calculates the minimum value, and count() counts the number of elements in a set.

[0078] LLM functions refer to functions that call a large language model. Specifically, when predefined query functions and inference functions cannot solve the problem, or when there are undefined arbitrary functions, LLM functions are used to generate query results through LLM.

[0079] Based on the functions defined above, the large language model generates language query functions and inference functions adapted to the actual query problem in the application. Specifically, in the function generation module 120, the consultation prompts and query questions are input into the large language model, which then generates predefined language query functions and / or inference functions. Of course, the large language model will also generate undefined random functions, which refer to functions randomly generated by the large language model without prompting or learning.

[0080] Consultation hints are predefined and used to instruct the large language model how to generate language query functions and inference functions that conform to the consultation question. The consultation hints for the language query functions consist of two parts:

[0081] The first part is the instructions, which introduce the condition graph and query syntax. This part first explains the definition of condition triples, then describes the meaning of Language Query Functions (LQF), and finally explains the inference functions that may be used in the inference process for large models.

[0082] The second part is to generate a sample query function: the sample format is "[sample question] + [answer]", where the sample question is from the training set or a manually constructed question, and the answer is the correct query inference syntax for that sample question (combining the query function with the inference function).

[0083] Similarly, the advisory prompts for inference functions also include generation instructions and examples of generating inference functions.

[0084] Specifically, the process of generating language query functions or inference functions based on consultation prompts and questions through a large language model includes two parts: generating the language query functions or inference functions and parameter alignment. First, by inputting the consultation prompts and questions into the generative large language model, the model obtains preliminary language query functions or inference functions through autoregressive inference using Next Token Prediction. However, the parameters in the preliminary language query functions or inference functions may be ambiguous with structured knowledge, necessitating the second part: parameter alignment. The parameters of the preliminary generated language query functions or inference functions with the highest similarity among the available parameters are selected as the final result, thus achieving parameter alignment.

[0085] The execution query function and corresponding translation method defined above are set in query module 130. In query module 130, when the large language model generates only knowledge query functions for querying, the query needs to use the execution query function, such as... Figure 2 As shown, knowledge retrieval is specifically performed in the condition graph based on the knowledge query function, including:

[0086] Element Mapping and Alignment: Align and map the knowledge elements in the knowledge query function with the knowledge elements in the consultation question, that is, replace the corresponding knowledge elements in the knowledge query function with the knowledge elements in the consultation question to obtain the aligned knowledge query function;

[0087] Function transformation: Translate the aligned knowledge query function into an execution query function;

[0088] Knowledge Query: The query results are obtained by executing the query function to search the condition graph.

[0089] The translation method from knowledge query function to execution query function is shown in Table 1:

[0090] Table 1. Translation from knowledge query function to execution query function

[0091]

[0092]

[0093] The translation from a language query function to an executed query function is as follows: `gi`, `h`, `r`, `t`, `k`, and `v` represent `get_information`, `head_entity`, `relation`, `tail_entity`, `key`, and `value`, respectively. `sn`, `sc`, `n1`, `n2`, `c`, and `s` represent `search_node`, `search_condition`, `node1`, `node2`, `condition`, and `scope`, respectively. The `op` in `get_information` for parameters `t` and `v` is one of the following: `>`, `<`, `=`, `≥`, `≤`. Explanation of the translation process:

[0094] In sequence 1, the language query function gi(h=H) represents querying the head entity of H, and the corresponding translation execution query function sn(n1=H) represents returning the relation and key of the head entity H;

[0095] In sequence 2, the language query function gi(r=R) represents querying the relation of R, and the corresponding translation execution query function sn(n1=R) represents returning the tail entity under relation R;

[0096] In number 3, the language query function gi(k=K) represents querying the key K, and the corresponding translation execution query function sn(n1=K) represents returning the value under key K;

[0097] In sequence 4, the language query function gi(h=H,r=R) represents querying the relationship between the head entity of H and R, and the corresponding translation execution query function sn(n1=R,c=H) represents returning the tail entity of the head entity H under the relation R;

[0098] In sequence 5, the language query function gi(h=H,k=K) represents querying the head entity of H and the key of K, and the corresponding translation execution query function gi(h=H,k=K) represents returning the value of the head entity H under the key K;

[0099] In sequence 6, the language query function gi(r=R,t″=″T) indicates that the tail entity T and relation R are known, and the corresponding translation execution query function sc(n1=R,n2=T,op=″=″) indicates that the head entity with tail entity T under relation R is returned.

[0100] In number 7, the language query function gi(k=K,v″=″V) represents a known key K and value V, and the corresponding translation execution query function sc(n1=K,n2=V,op=″=″) represents the return of the head entity with value V under key K;

[0101] In sequence 8, the language query function gi(r=R,t″=″T,k=K,v″=″V) means that given the key K and value V, the tail entity T and relation R can be obtained simultaneously. The corresponding translation execution query function is...

[0102] output_of_query1=sc(n1=R,n2=T,op="=")

[0103] output_of_query2=sc(n1=K,n2=V,op="=")

[0104] set_interaction(set1=output_of_query1,set2=

[0105] `output_of_query2)` means returning the head entity whose tail entity is T under relation R and whose value is V under key K;

[0106] In sequence 9, the language query function gi(r=R,t″=″T,k=K) represents the execution query function for translation given the tail entity T of relation R and the key K.

[0107] output_of_query1=sc(n1=R,n2=T,op="=")

[0108] sn(n1=K,s=output_of_query1) means returning the value of the head entity with tail entity T under relation R with key K;

[0109] In sequence 10, the language query function gi(r=R,k=K,v=V) represents the execution query function for translation given the key K, value V, and relation R.

[0110] output_of_query1=sc(n1=K,n2=V,op="=")

[0111] sn(n1=R,s=output_of_query1) represents the tail entity of the head entity relation R with value V under key K;

[0112] In query module 130, inference functions may also be used during the query process. Specifically, when the large language model generates functions for querying that include knowledge query functions and inference functions, the knowledge query process based on these functions in the condition graph includes element mapping and alignment steps for the knowledge query functions, function transformation steps, and also includes:

[0113] In the knowledge query step, query results are obtained by executing query functions and inference functions to search the condition graph. This includes: first, executing query functions to search the condition graph to obtain candidate query results; then, using inference functions to infer from the candidate query results to obtain the final query result. For example, for the question "[What is the number of gold medals won by the country with the most gold medals?]", the query function first obtains the number of gold medals won by all countries, and then the inference function obtains the maximum number of gold medals won.

[0114] Alternatively, you can first use an inference function and then execute a query function to obtain the final result. For example, for the question "[Which country has won the most gold medals?]", the previous example would yield the highest number of gold medals, and then the query function would be used to find the country with the most gold medals.

[0115] The aforementioned predefined LLM functions are set in the query module 130. In the query module 130, when the large language model generates functions for querying that include at least one of the predefined language query functions, inference functions, and undefined random functions, and the predefined language query functions and / or inference functions cannot solve the consultation problem, or only undefined random functions appear, the LLM functions are used to generate query results through the large language model.

[0116] It should be noted that LLM functions are called in a few cases, so the illusion problem of LLMs has little impact on the query method of this invention.

[0117] The embodiments also provide query experiments of the above-mentioned structured knowledge query system on tables, knowledge graphs, and time-series index graphs. Specifically, query experiments were conducted on fixed pattern knowledge corresponding to tables, factual knowledge corresponding to knowledge graphs, and temporal knowledge corresponding to time-series knowledge graphs. Four datasets were used: WikiSQL, MetaQA, WebQSP, and CronQuestion. The large language models used include GPT3.5 and GPT4. The usage rights to these large language models were purchased through legal channels. It should be noted that the solution of this invention is a training-free solution.

[0118] In question-answering research based on fixed-schema knowledge, the WikiSQL tabular question-answering dataset was selected, and Denotation Accuracy was used to evaluate the predicted query answers. For example... Figure 3As shown, the proposed solution (NeSyGPT(GPT-3.5)) achieved an accuracy of 84.5% in the GPT-3.5 context, surpassing methods such as StructGPT, MAPO, and TAPAS. In the GPT-4 context, the proposed solution (NeSyGPT(GPT4)) further improved to 89.0%, approaching the state-of-the-art level of TAPEX. This demonstrates the excellent performance of the proposed solution in deep semantic understanding.

[0119] In the research on question answering for factual knowledge, two knowledge graph datasets, MetaQA and WebQuestionsSP (WebQSP), were selected, and the Hit@1 metric was used for evaluation. For example... Figure 4 As shown, experiments were conducted in MetaQA for 1-hop, 2-hop, and 3-hop scenarios. Due to the duplication of names for movies and actors, and the inherent ambiguity of the questions, the predicted answers were still correct even when the standard answer was a subset of the predicted answers. The proposed solution (NeSyGPT(GPT-3.5)) achieved Hit@1 accuracies of 99.9%, 99.8%, and 99.9% using GPT-3.5 in these cases. These results demonstrate the excellent performance of the proposed solution in understanding simple semantic problems. In WebQSP, the dataset was divided into two categories based on whether the corresponding entities could understand the semantic relationship: those related to semantically explicit relationships and those related to semantically ambiguous relationships. Experimental results show that the proposed solution (NeSyGPT(GPT-3.5)) achieved a Hit@1 accuracy of 70.9% using GPT-3.5, surpassing most pre-trained and LLM-based methods. The proposed solution (NeSyGPT(GPT4)) achieved a state-of-the-art accuracy of 76.5% on GPT-4. The comparison shows that for questions with semantically explicit relations, GPT-3.5 and GPT-4 have accuracy rates that are 23% and 16% higher, respectively, than those with semantically ambiguous relations.

[0120] In research on question answering based on temporal knowledge, the CronQuestions dataset was selected, and Hits@1 was used for evaluation. For example... Figure 5 As shown, on CronQuestions, the solution of this invention has achieved state-of-the-art results in both simple and time-sensitive problems, with overall performance comparable to LGQA.

[0121] The structured knowledge query system provided in the above embodiments realizes the query process from natural language questions to structured knowledge, fully leverages the ability of large language models to generate queries, and ensures the accuracy of queries and their alignment with the query questions by aligning and executing queries.

[0122] In the structured knowledge query system provided in the above embodiments, based on conditional graphs, in order to obtain accurate answers and avoid the illusion problem of large language models, the large language model is used as a writer of conditional graph queries to generate functions for querying. These functions serve as the query language, thus ensuring the correctness of the query language and making it more flexible and efficient to obtain information from different types of structured knowledge.

[0123] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A structured knowledge query system based on a large language model, characterized in that, The knowledge representation module, the function generation module, and the query module are included. The knowledge representation module is configured to convert structured knowledge into a conditional graph, and specifically map tables, knowledge graphs, and time-series knowledge graphs into conditional graphs, respectively. When the structured knowledge is a table with m rows and n columns, the first row of the table records the semantic meaning of each column, and each row starting from the second row represents a knowledge record. wherein, and denote the set of nodes and the set of conditions in the table corresponding condition graph i denotes the row index, j denotes the column index, v i,j denotes the record value in the i-th row and j-th column, v 1,j denotes the semantic meaning in the 1st row and j-th column, i.e. the column name, (i, v 1,j , []) denotes that the row index does not need to be associated with the column name through any condition to form a condition triple with empty condition, (v 1,j , v i,j , [i]) denotes that v 1,j is associated with v i,j through the row information [i] to form a condition triple;​ When the structured knowledge is a knowledge graph where ε1, and are a set of entities, a set of relations and a set of triples, respectively, is a set of triples representing the relation r1 between entities (h1, t1), the knowledge graph is converted into a conditional graph, represented as: wherein, and respectively represent a knowledge graph corresponding to a condition graph in the node set and the condition set, (h1, r1, []) indicates that h1 and r1 can be connected to form a triple with empty conditions without any conditions, and (r1, t1, [h1]) indicates that the relationship r1 and t1 are associated with each other to form a conditional triple through the head entity [h1]. When structured knowledge is time knowledge graph where ε2, and are entity set, relation set and time set respectively, is a five tuple set, {h2, t2} ∈ ε2, represent the start time and end time of the fact (h2, r2, t2), and the time knowledge graph is converted into a conditional graph, represented as: wherein, and respectively represent a time knowledge graph corresponding condition graph the node set and the condition set in convert each (h2, r2, t2, st, ed) in to at least 5 conditional triples to build 5 conditional triples are (h2, r2, []), (r2, t2, h2), (start time, st, [h2, r2, t2]), (end time, et, [h2, r2, t2]) and (time, time, [h2, r2, t2]), time ∈ time interval [st, et], start time represents the start time, end time represents the end time, (h2, r2, []) represents that the conditional empty triple can be formed without conditions h2 and r2, (r2, t2, h2) represents that the conditional triple is formed through the conditional head entity h2, the relationship r2 and t2, (start time, st, [h2, r2, t2]) forms the conditional triple through the conditional triple [h2, r2, t2], the start time start time and the time st, which represents the start time of the occurrence of the triple, (end time, et, [h2, r2, t2]) represents the end time, (time, time, [h2, r2, t2]) represents the occurrence time; The function generation module is configured to generate a function for querying based on the consultation prompt and the query question using a large language model. The query module is configured to perform knowledge querying in the conditional graph based on the function to obtain a query result. 2.The large language model-based structured knowledge query system of claim 1, wherein, The function generated using the large language model includes at least one of a predefined language query function, an inference function, and an undefined random function. The language query function refers to a query step for gradually solving the consultation question, and includes a query node function, a query condition function, and a comparison function. The inference function includes set operations and statistical calculations, and is used for processing and reasoning knowledge in the conditional graph. The undefined random function refers to a function randomly generated by the large language model without learning through a prompt. 3.The large language model-based structured knowledge query system of claim 2, wherein, The query node function is as follows: search_node(node1, condition, scope) The function searches for and returns a node node2 in the conditional facts within the scope of the node node1 and the condition condition, and scope=None by default means that the scope is all conditional facts in the conditional triple, and condition=None indicates any condition. The query condition function is as follows: search_condition(node1, node2_value, op) The function searches for and returns a condition in the comparison relationship op between node1 and node2, i.e., compare(op, node2_value, node2). The query comparison function is as follows: compare(op, val1, val2) If val1{op}val2 is satisfied, True is returned, otherwise False is returned, where op is a comparison symbol {>, <, =, >=, <=}. 4.The large language model-based structured knowledge query system of claim 2, wherein, The generation process of the inference function includes: The consultation prompt and the consultation question are input into the generative large language model. 5.The large language model based structured knowledge query system of claim 2, wherein, The large language model obtains a preliminary query inference function through the self-recurrent inference mode of next Token prediction. Then, the parameters of the preliminary query inference function are selected as the most similar parameters in the optional parameters to realize the alignment of the parameters, and the final inference function is obtained. The consultation prompt includes a generation instruction and a sample of the generated inference function. When the function generated by the large language model for querying only includes the knowledge query function, the knowledge query function is used to perform knowledge querying in the conditional graph, which includes: Element mapping and alignment: aligning and mapping the knowledge elements in the knowledge query function with the knowledge elements in the consultation question, that is, replacing the corresponding knowledge elements in the knowledge query function with the knowledge elements in the consultation question to obtain an aligned knowledge query function; Function conversion: converting the aligned knowledge query function into an execution query function, wherein the execution query function includes three types of query node functions, query condition functions, and comparison functions; Knowledge query: searching in the conditional graph through the execution query function to obtain a query result.

6. The large language model based structured knowledge query system according to claim 5, wherein, When the function generated by the large language model for querying contains a knowledge query function and a reasoning function, knowledge querying is performed in the conditional graph based on the knowledge query function and the reasoning function, including the element mapping and alignment steps for processing the knowledge query function, the function conversion step, and further including: In the knowledge query step, searching in the conditional graph through the execution query function and the reasoning function to obtain a query result, including: first searching in the conditional graph through the execution query function to obtain a candidate query result, and then reasoning the candidate query result through the reasoning function to obtain a query result: Or, first obtaining a problem search condition through the reasoning function, and then combining the problem search condition to execute the query function to obtain a final query result.

7. The large language model based structured knowledge query system of claim 2, wherein, The query module further includes a pre-defined LLM function. When the function generated by the large language model for querying contains at least one of a pre-defined language query function, a reasoning function, and an undefined random function, and the pre-defined language query function and / or the reasoning function cannot solve the consultation question, or only the undefined random function appears, the LLM function is used to generate a query result through the large language model.

Images