Large model question and answer method and device based on logic expression and knowledge graph

Abstract

The invention belongs to the technical field of knowledge questions and answers, and discloses a large model question and answer method and device based on a logic expression and a knowledge graph. The method comprises the steps of extracting a to-be-queried question from a text input by a user by using a large language model; inputting a to-be-queried question into the trained logic extraction and conversion module to generate a true value expression; inputting the truth value expression into the knowledge graph for query to obtain an effective truth value expression and a corresponding query result; and inputting the effective true value expression, the query result and the to-be-queried question into the large language model to generate a final answer. Through a small-scale large language model, questions and answers with high credibility and good interpretability can be provided for users.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge question answering, and in particular to a large-model question answering method and device based on logical expressions and knowledge graphs. Background Art

[0002] In recent years, large language models have achieved impressive results in a variety of tasks such as natural language understanding and natural language generation, and have demonstrated strong generalization capabilities. However, there are still great limitations in their application in question-answering systems. First, large language models cannot completely avoid the problem of hallucinatory facts, because large language models are essentially always dealing with text completion tasks, and the answers they generate are essentially a natural extension of the questions in semantics. This determines that when large language models deal with questions beyond the scope of training data, they are prone to "fabrication of facts", which is a fatal problem in question-answering systems that focus on correctness. Secondly, in large language models, the information used to answer questions is implicitly included in their parameters, and the answers directly generated by the model are difficult to understand, which further reduces its credibility. In order to reduce the above limitations of large language models, a widely used method is to fine-tune them for specific downstream tasks. However, fine-tuning a large language model may lead to catastrophic forgetting, resulting in a degradation of its general conversational capabilities. Summary of the invention

[0003] To this end, the embodiments of the present application provide a large-model question-answering method and device based on logical expressions and knowledge graphs, which provides users with highly reliable and interpretable question-answering services through a relatively small-scale large language model.

[0004] On the first aspect, the present application provides a large-model question-answering method based on logical expressions and knowledge graphs.

[0005] This application is achieved through the following technical solutions:

[0006] A large-model question answering method based on logical expressions and knowledge graphs, comprising:

[0007] Use a large language model to extract questions from user input text;

[0008] Input the query question into the trained logic extraction and conversion module to generate a truth value expression;

[0009] Input the truth value expression into the knowledge graph for query, and obtain a valid truth value expression and a corresponding query result;

[0010] The valid truth value expression, query result and question to be queried are input into the large language model to generate a final answer.

[0011] In a preferred example of the present application, it can be further set as follows:

[0012] The query question is input into the trained logic extraction and conversion module to generate a truth value expression, including:

[0013] Based on the query question, a candidate expression set is generated from a knowledge base by enumerating candidates;

[0014] After the query question and the candidate expressions in the candidate expression set are concatenated, the concatenation is input into a trained scoring model to obtain a similarity score between the query question and the candidate expressions, wherein the scoring model is trained based on the BERT model;

[0015] The similarity scores are sorted, and several candidate expressions with high scores are selected. The several candidate expressions with high scores and the query question are input into the trained generation model to generate a true value expression. The generation model is obtained by training based on the T5 model.

[0016] In a preferred example of the present application, it can be further configured that, based on the query question, a candidate expression set is generated from the knowledge base by enumerating candidates, including:

[0017] Identify all entities in the query question, enumerate a preset range of candidate paths from the knowledge base for each entity, convert the candidate paths into S expressions, and add the S expressions to a candidate expression set, wherein the S expressions are a triple structure containing a subject, a relationship, and an object.

[0018] In a preferred example of the present application, it can be further configured that when a scoring model is obtained based on BERT model training, the cross entropy loss function is minimized so that the BERT model learns from the comparison between the true value expression and the candidate expression, and when the performance reaches the preset requirements, a trained scoring model is generated.

[0019] In a preferred example of the present application, it can be further configured that, before training the BERT model, the method further includes: pre-training the BERT model using the candidate expression to obtain a pre-trained BERT model;

[0020] Input the candidate expressions and true value expressions into the pre-trained BERT model to obtain the similarity score between each candidate expression and the true value expression;

[0021] The similarity scores are sorted, and a preset number of candidate expressions are selected in order of the top scores as candidate expressions of the training set.

[0022] In a preferred example of the present application, it can be further set that when the generative model is obtained based on T5 model training, the loss function is expressed as:

[0023]

[0024] in, represents the predicted conditional probability of the generative model, gt t represents the conditional probability of the tth token appearing when the first t-1 tokens are generated in a given sequence, gt<t represents all tokens before t in the given sequence, and input represents the input constructed by the query question and several candidate expressions with the highest scores.

[0025] On the second aspect, the present application provides a large-model question-answering device based on logical expressions and knowledge graphs.

[0026] This application is achieved through the following technical solutions:

[0027] A large model question-answering device based on logical expressions and knowledge graphs, used to execute the method described in the first aspect, comprising:

[0028] A question extraction module is used to extract the query questions from the user input text using a large language model;

[0029] A logic extraction and conversion module, used for receiving the query question and extracting the truth value expression from the query question;

[0030] The knowledge graph module is used to receive truth value expressions and output query results for valid truth value expressions;

[0031] The answer confirmation module is used to obtain a final answer based on a valid truth value expression, a query result and a question to be queried, and the answer confirmation module is based on a large language model.

[0032] In a preferred example of the present application, it can be further set as follows:

[0033] The logic extraction and transformation module also includes:

[0034] A candidate expression enumeration unit, used to generate candidate expressions from a knowledge base by enumerating candidates based on the query question;

[0035] A scoring unit, used to concatenate the query question and the candidate expressions in the candidate expression set, and input the concatenated concatenation into a trained scoring unit to obtain a similarity score between the query question and the candidate expressions;

[0036] The generation unit is used to sort the similarity scores, select several candidate expressions with high scores, input the several candidate expressions with high scores and the query question into the trained generation unit, and generate a truth value expression.

[0037] On the third aspect, the present application is realized through the following technical solutions:

[0038] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned large-model question-answering methods based on logical expressions and knowledge graphs are implemented.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium.

[0040] This application is achieved through the following technical solutions:

[0041] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the above-mentioned large-model question-answering methods based on logical expressions and knowledge graphs.

[0042] In summary, compared with the prior art, the technical solution provided in the embodiment of the present application has at least the following beneficial effects:

[0043] This application uses a large language model to extract questions to be queried from user input text; inputs the questions to be queried into a trained logic extraction and conversion module to generate a truth expression; inputs the truth expression into the knowledge graph for query, obtains a valid truth expression and a corresponding query result; inputs the valid truth expression, the query result and the question to be queried into the large language model to generate a final answer. The method provided by this application uses a large language model and a logic extraction and conversion module, so that the question-answering system can understand and answer the user's questions more accurately, improving the relevance of the answer; the large model can obtain the most comprehensive and relevant knowledge about the user's questions to be queried in the knowledge base, enhancing the knowledge coverage of the question-answering system, thereby effectively reducing the shortcomings of model knowledge dependence in the prompt engineering, and improving the independence and robustness of the question-answering method; the quality of the answer returned by the large language model combined with the knowledge graph is highly correlated with the input structured knowledge graph knowledge, and the large language model integrates the information in the knowledge graph into natural language, so that the answer finally provided to the user is easy for the user to understand. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a large-model question-answering method based on logical expressions and knowledge graphs provided in one embodiment of the present application;

[0045] Figure 2 A schematic diagram of a process for generating a truth value expression provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a large-model question-answering device based on logical expressions and knowledge graphs provided in one embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of a large-model question-answering device based on logical expressions and knowledge graphs provided in another embodiment of the present application; Description of the accompanying drawings:

[0048] Question extraction module 100, logic extraction and conversion module 200, knowledge graph module 300, answer confirmation module 400, candidate expression enumeration unit 201, scoring unit 202, generation unit 203. DETAILED DESCRIPTION

[0049] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application, unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.

[0052] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on quantity and execution order.

[0053] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0054] First, let’s introduce the big language model mentioned in this application: the big language model is a type of artificial intelligence model based on deep learning, which is designed to process and generate natural language text. By training the big language models with large-scale text data, these models can understand and generate text similar to human language, and perform a variety of natural language processing tasks including text generation, translation, and sentiment analysis. The vast majority of existing big language models use the Transformer architecture, on this basis, they can be further divided into pure encoder models, pure decoder models, and encoder-decoder models.

[0055] There are three main commonly used implementation technologies for existing large language model question-answering systems, namely prompt engineering, fine tuning, and retrieval-augmented generation (RAG). Prompt engineering refers to the design and optimization of instructions input to the artificial intelligence model to ensure that the model generates the expected output. By building accurate and detailed prompts, the model's understanding and responsiveness can be improved, thereby achieving efficient human-computer interaction. Fine tuning is defined as the process of further training the model using a new, task-specific dataset based on a pre-trained large language model. The main purpose of this fine-tuning technique is to enable the model to adapt to new and specific tasks or fields without having to train a completely new model from scratch. The key steps involved in large model fine-tuning include further training on a pre-trained large language model (generally referred to as a "base model") using a specific dataset to adapt the model to specific tasks or fields.

[0056] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0057] like Figure 1 As shown, a large model question answering method based on logical expressions and knowledge graphs is provided in the first exemplary embodiment of the present application, which includes:

[0058] S10: Use a large language model to extract query questions from user input text.

[0059] The user input text is input into the pre-trained large language model, and the large language model outputs the query questions in the user input text. In this application, the large language model adopts the Transformer architecture.

[0060] S20: Input the query question into the trained logic extraction and conversion module to generate a truth value expression.

[0061] The logic extraction and conversion module is the RNG-KBQA system designed based on the GAIR-KBQA concept, which includes a scoring module and a generation module, and can realize the efficient and accurate conversion of user natural language questions to KB query logical expressions.

[0062] S30: Input the truth value expression into the knowledge graph for query to obtain a valid truth value expression and corresponding query results.

[0063] It should be noted that in automated knowledge graph queries, if an expression can be converted into a query that can be executed in the knowledge graph, the expression is considered executable; if the executed query can return results, the expression is considered valid.

[0064] S40: Input the valid truth value expression, query results and questions to be queried into the large language model to generate the final answer.

[0065] For the candidate expressions output by the generation model in the logic extraction and conversion module and the highly scored candidate expressions output by the scoring model, the former may not be executable, but has a wider coverage and has the opportunity to generate logical expressions that cannot be directly obtained from the knowledge graph path search, while the latter is guaranteed to be feasible and effective. Combining the above two characteristics, the present application selects several valid truth expressions from the outputs of the generation model and the scoring model respectively, and returns the corresponding query results as input to the large language model. This strategy can ensure to the greatest extent that the most relevant and accurate knowledge base information to be queried is input into the large language model.

[0066] In one embodiment, Figure 2 As shown, the query question is input into the trained logic extraction and conversion module to generate a truth value expression, including:

[0067] S21: Based on the query question, a candidate expression set is generated from the knowledge base by enumerating candidates.

[0068] Specifically, it includes: identifying all entities in the query question, enumerating a preset range of candidate paths from the knowledge base for each entity, converting the candidate path into an S expression, and adding the S expression to the candidate expression set, where the S expression is a triple structure containing a subject, a relationship, and an object. In the actual implementation process, the ELQ model is used to identify each entity e in the query question x, and the identified entity e is mapped to the ID corresponding to the database. Then, with the ID as the center, the path with a length of less than 2 and 3 as the endpoint is enumerated from the knowledge base in no direction (that is, all edges are regarded as undirected edges during enumeration) as a candidate path, and the candidate path is converted into an S expression, and the S expression is added to the candidate expression set C.

[0069] It should be noted that the candidate expression set C does not cover all possible logical forms within two hops of the knowledge base (for example, logical forms containing comparison, sorting, constraints, etc.), so the candidate expression set C cannot be guaranteed to contain the real expression corresponding to the query question x. Although a more detailed enumeration may contain the real expression corresponding to the query question x, it may generate a huge amount of data. For example, for queries on certain entities, more than 2,000,000 candidate expressions may be generated. Because in this application, based on the consideration of reducing resource usage, only a limited number of logical expressions will be enumerated in this step.

[0070] S22: After concatenating the query question and the candidate expressions in the candidate expression set, the concatenation is input into the trained scoring model to obtain the similarity score between the query question and the candidate expressions. The scoring model is trained based on the BERT model.

[0071] In order to select the S expression that is most relevant to the user's query, this application fine-tunes and trains the BERT model as the scoring model. Specifically, this step uses a BERT-based model, which takes the concatenation of the original query x and the candidate expression y as input, and the BERT model outputs the similarity score Score between the query x and the candidate expression y. The expression of the similarity score Score is:

[0072] Score(x;y)=Linear(CLS(x;y)),

[0073] Where CLS() is given the input [CLS] vector, and Linear() is a fully connected layer used to project the [cls] vector into a similarity score.

[0074] In the training process of the BERT model, the BERT model should maximize the comparison between the true expression and the candidate expression of the non-true value. If a training input contains several candidate expressions and only one true value expression, the task can be equivalent to a multi-classification task. Therefore, in the training process of the BERT model, the cross entropy loss function is minimized to enable the BERT model to learn from the comparison between the true value expression and the candidate expression. When the performance reaches the preset requirements, a trained BERT model is generated. The expression of the cross entropy loss function is:

[0075]

[0076] Where C is the set of candidate expressions in the training input.

[0077] S23: Sort the similarity scores, select several candidate expressions with high scores, input the several candidate expressions with high scores and the query question into the trained generation model, generate a true value expression, and the generation model is trained based on the T5 model.

[0078] In the previous steps, all candidate expressions and their scores corresponding to the query question x have been obtained. However, the candidate expressions are not guaranteed to contain true value expressions. To solve this problem, a generative model trained based on the T5 model is introduced. The input of the generative model is the original query question x and several candidate expressions with the highest scores, and the generation target is the true value expression. Even if the several candidate expressions with the highest scores as input may not contain true value expressions, they are likely to contain the information required to generate true value expressions. Therefore, it can be expected that true value expressions can be effectively generated through this sequence-sequence model.

[0079] Considering the limitation of GPU resources and the huge number of non-true candidate expressions, it is unrealistic to include all candidates in the input data when training the BERT model. Therefore, in order to improve the training efficiency, non-true candidate expressions are sampled in advance during training. Obviously, the simplest method is to use random sampling. However, compared with the full set, the number of candidate expressions that can be included in the training input is very small, and the learning effect of this random sampling method may not be ideal. In view of this, the present application adopts a sampling method based on bootstrapping. Specifically, before training the BERT model: first use the candidate expressions to train the BERT model for several rounds through random sampling to obtain a pre-trained BERT model; input the candidate expressions and the true value expressions into the pre-trained BERT model to obtain the similarity score between each candidate expression and the true value expression; sort the similarity scores, and select a preset number of candidate expressions in the order of the top scores as the candidate expressions of the training set in the post-training process.

[0080] During the training process of the T5 generative model, the loss function is expressed as:

[0081]

[0082] in, represents the predicted conditional probability of the generative model, gt t represents the conditional probability of the tth token appearing when the first t-1 tokens are generated in a given sequence, gt<t represents all tokens before t in the given sequence, and input represents the input constructed by the query question and several candidate expressions with the highest scores.

[0083] Through this training method, the T5 generative model can quickly learn the knowledge graph structure information (contained in the highly scored candidate expressions) and the semantic information of the question at a low cost, and generate highly grammatically correct truth expressions based on this, without having to use constraints to ensure that the truth expression generation rules are followed and its correctness is ensured, thereby making more efficient use of the training data. Finally, the model decodes the sequence using the Beam Search method. Since the T5 generative model does not introduce hard grammatical constraints, the generated logical form cannot be guaranteed to be executable, so the highest probability sequences will be retained as the final output of the model during decoding.

[0084] Another embodiment of the present application also provides a large model question-answering device based on logical expressions and knowledge graphs, which is used to execute the above method, such as Figure 3 As shown, the device includes: a question extraction module 100, a logic extraction and conversion module 200, a knowledge graph module 300 and an answer confirmation module 400;

[0085] The question extraction module 100 is used to extract the query question from the user input text using a large language model;

[0086] The logic extraction and conversion module 200 is used to receive the query question and extract the truth value expression from the query question;

[0087] The knowledge graph module 300 is used to receive truth value expressions and output query results for valid truth value expressions;

[0088] The answer confirmation module 400 is used to obtain a final answer based on a valid truth value expression, a query result and a question to be queried. The answer confirmation module is based on a large language model.

[0089] Among them, Figure 4 As shown, the logic extraction and conversion module 200 also includes: a candidate expression enumeration unit 201, a scoring unit 202, and a generation unit 203;

[0090] A candidate expression enumeration unit 201 is used to generate candidate expressions from a knowledge base by enumerating candidates based on the query question;

[0091] A scoring unit 202 is used to concatenate the query question and the candidate expressions in the candidate expression set, and then input the concatenated concatenated concatenated concatenated expressions into a trained scoring unit to obtain a similarity score between the query question and the candidate expressions;

[0092] The generation unit 203 is used to sort the similarity scores, select several candidate expressions with high scores, input the several candidate expressions with high scores and the query question into the trained generation unit, and generate a truth value expression.

[0093] In order to combine the large language model with the logic extraction and conversion module, the above system is encapsulated into a standard LangChain tool, and then a specific system prompt word is used to make the large language model actively call the LangChain tool in the format specified in the system prompt word. Relying on the LangChain tool, it aims to use a relatively lightweight model to inject structured information related to the input question in the large knowledge graph into the large language model, so that it can generate answers based on the information of the knowledge graph. The return value of the LangChain tool will contain several valid logical expressions (i.e., true value expressions) and corresponding query results, which can be used in the subsequent reasoning process of the large language model, thereby realizing the injection of knowledge graph information into the large language model. The large language model can understand the returned logical expression and extract the required information from it to generate an answer. It should be noted that the large language model and the logic extraction and conversion module can be split into two different independent units according to actual needs to perform different natural language processing tasks. At the same time, the scoring module and the generation module in the logic extraction and conversion module can actually be combined as needed.

[0094] This application uses a lightweight version of a large language model to achieve high credibility and good interpretability of question and answer with low hardware resource usage. Specifically, we deployed the BERT, T5, and ChatGLM models in turn. The BERT model provides efficient performance with its less than 1B parameters; the T5 model has about 1B to 2B parameters, balancing performance and resource consumption; and the ChatGLM model, despite its 6B parameters, has powerful performance making it an ideal choice for high-end applications. These models can be deployed locally on consumer-grade graphics cards, ensuring the accessibility and practicality of the system while maintaining high credibility and good interpretability. In addition, this system has good versatility. It only needs to replace the knowledge base and the front-end module to use the system for question and answer in different fields.

[0095] For the specific definition of the large-model question-answering device based on logical expressions and knowledge graphs provided in this embodiment, please refer to the embodiment of the large-model question-answering method above, which will not be repeated here. Each module in the above-mentioned large-model question-answering device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0096] This application trains a question-answering device based on a large language model and a large knowledge graph on the WebQSP dataset, and verifies the effectiveness of the question-answering device through comparative experiments on logic extraction and conversion modules, ablation experiments on natural language question-answering modules based on a large language model, and ablation experiments on logic extraction and conversion modules.

[0097] In the comparative experiment of the logic extraction and conversion module, in order to evaluate the performance of the module on the WebQSP dataset, it should generate a single output for a single question. The strategy adopted by this application is to only output the first valid expression obtained by the generation model (T5 model). If it does not exist, the highest-scoring candidate of the scoring model (BERT model) is output. This application uses the F1 score as the evaluation criterion and compares it with the three most advanced KBQA methods ArcaneQA, ReTrack, and QGG. It is found that the F1 score of the RNG-KBQA method used in the logic extraction and conversion module is 0.3, 1.0, and 1.6 percentage points higher than the above three methods, reaching 75.6%.

[0098] In the ablation experiment of the natural language question-answering module based on the large language model, this application uses ChatGLM3-6B as the large language model of the question-answering system, and focuses on whether the aforementioned logic extraction and conversion modules can improve the question-answering performance of the large language model. In view of the fact that WebQSP has no indicators that can be used to evaluate the output quality of the large language model, inspired by TruthfulQA, this application proposes a set of scoring criteria for knowledge base enhanced question-answering based on the authenticity and information content of the answer: Label 0: contains any false information or is self-contradictory, with a score of 0.0; Label 1: true, completely contains valid information in the true value query and fully answers the question, with a score of 1.0; Label 2: true, but the answer is incomplete in any aspect, with a score of 0.7; Label 3: topic-related but not answering the question, with a score of 0.5; Label 4: only keyword-related, with a score of 0.1; Label 5: only meaningless repeated questions, with a score of 0.0; Label 6: indicates that it cannot be answered, with a score of 0.2; Label 7: completely irrelevant to the question, with a score of 0.0.

[0099] According to the above standards, this experiment stratified sampled 100 questions from the WebQSP.Test dataset at a ratio of simple questions: complex questions = 7:3. Complex questions usually contain a large number of entities, and the logical relationships abstracted from the questions are more complex. For the ChatGLM3-6B model without adding the logic extraction and conversion module, its average score on simple questions is 0.39, and its average score on complex questions is 0.31. After adding the RNG-KBQA system module, the average score of the large language model on simple questions increased to 0.83, and the average score on complex questions increased to 0.55.

[0100] Ablation experiment of the logic extraction and conversion module. This experiment mainly studies and verifies the impact of the scoring model and the generation model on the overall performance in the logic extraction and conversion module. The strategy adopted here is to directly output the candidate with the highest score as the final answer when only the scoring model is used; when only the generation model is used, 5 expressions are randomly selected from the candidate expressions and concatenated with the original question as input, and the first decoded output is output as the final answer.

[0101] Taking the F1 score as the scoring standard, the score of using only the generative model is 69.9, the score of using the scoring model is 72.7, the score of using only the scoring model without bootstrapping is 71.3, and the score of the complete model is 75.6. This experimental result shows that the generative model can use the information in the original problem to generate expressions that do not appear explicitly, improving the F1 of the scoring model by 2.9; the scoring model can provide more reference information input for the generative model, improving the F1 of the generative model by 5.7. At the same time, the bootstrap sampling strategy of the scoring model has also been shown to improve the performance of the scoring model to a certain extent.

[0102] An embodiment of the present application provides a computer device, which may include a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the processor executes the steps of the large model question-answering method as in any of the above embodiments.

[0103] The working process, working details and technical effects of the computer device provided in this embodiment can be found in the embodiment of the large model question and answer method above, and will not be repeated here.

[0104] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the large model question-answering method in any of the above embodiments are implemented. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive and / or a memory stick, etc. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0105] The working process, working details and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiment of the large model question and answer method above, and will not be repeated here.

[0106] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system described in the present application is divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A large-model question-answering method based on logical expressions and knowledge graphs, characterized in that: include: Use a large language model to extract questions from user input text; Input the query question into the trained logic extraction and conversion module to generate a truth value expression; Input the truth value expression into the knowledge graph for query, and obtain a valid truth value expression and a corresponding query result; The valid truth value expression, query result and question to be queried are input into the large language model to generate a final answer.

2. The large-model question-answering method based on logical expressions and knowledge graphs according to claim 1 is characterized in that: The query question is input into the trained logic extraction and conversion module to generate a truth value expression, including: Based on the query question, a candidate expression set is generated from a knowledge base by enumerating candidates; After the query question and the candidate expressions in the candidate expression set are concatenated, the concatenation is input into a trained scoring model to obtain a similarity score between the query question and the candidate expressions, wherein the scoring model is trained based on the BERT model; The similarity scores are sorted, and several candidate expressions with high scores are selected. The several candidate expressions with high scores and the query question are input into the trained generation model to generate a true value expression. The generation model is obtained by training based on the T5 model.

3. The large-model question-answering method based on logical expressions and knowledge graphs according to claim 2 is characterized in that: Based on the query question, a candidate expression set is generated from the knowledge base by enumerating candidates, including: Identify all entities in the query question, enumerate a preset range of candidate paths from the knowledge base for each entity, convert the candidate paths into S expressions, and add the S expressions to a candidate expression set, wherein the S expressions are a triple structure containing a subject, a relationship, and an object.

4. The large-model question-answering method based on logical expressions and knowledge graphs according to claim 2 is characterized in that: When the scoring model is obtained based on BERT model training, the cross entropy loss function is minimized so that the BERT model learns from the comparison between the true value expression and the candidate expression. When the performance reaches the preset requirements, a trained scoring model is generated.

5. The large-model question-answering method based on logical expressions and knowledge graphs according to claim 4 is characterized in that: Before training the BERT model, it also includes: First, use the candidate expressions to pre-train the BERT model to obtain a pre-trained BERT model; Input the candidate expressions and true value expressions into the pre-trained BERT model to obtain the similarity score between each candidate expression and the true value expression; The similarity scores are sorted, and a preset number of candidate expressions are selected in order of the top scores as candidate expressions of the training set.

6. The large-model question-answering method based on logical expressions and knowledge graphs according to claim 1, characterized in that: When the generative model is obtained based on T5 model training, the loss function is expressed as: in, represents the predicted conditional probability of the generative model, gt t represents the conditional probability of the tth token appearing when the first t-1 tokens are generated in a given sequence, gt<t represents all tokens before t in the given sequence, and input represents the input constructed by the query question and several candidate expressions with the highest scores.

7. A large-model question-answering device based on logical expressions and knowledge graphs, characterized in that: Used to perform the method according to any one of claims 1 to 6, comprising: A question extraction module is used to extract the query questions from the user input text using a large language model; A logic extraction and conversion module, used for receiving the query question and extracting the truth value expression from the query question; The knowledge graph module is used to receive truth value expressions and output query results for valid truth value expressions; The answer confirmation module is used to obtain a final answer based on a valid truth value expression, a query result and a question to be queried, and the answer confirmation module is based on a large language model.

8. The large-model question-answering device based on logical expressions and knowledge graphs according to claim 7, characterized in that: The logic extraction and transformation module also includes: A candidate expression enumeration unit, used to generate candidate expressions from a knowledge base by enumerating candidates based on the query question; A scoring unit, used to concatenate the query question and the candidate expressions in the candidate expression set, and input the concatenated concatenation into a trained scoring unit to obtain a similarity score between the query question and the candidate expressions; The generation unit is used to sort the similarity scores, select several candidate expressions with high scores, input the several candidate expressions with high scores and the query question into the trained generation unit, and generate a truth value expression.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images