Question answering method and device based on knowledge base and language model

Abstract

The application provides a question and answer method and device based on a knowledge base and a language model, which comprises the following steps: inputting a target question into a knowledge generation model to output at least one dimension of target knowledge information; inputting target input information obtained according to the target question and each dimension of the target knowledge information into a question and answer model to obtain answer prediction information corresponding to the target question; the knowledge generation model is based on a first candidate question generated from a first knowledge tuple in the knowledge base, a second candidate question generated from the first candidate question and a second knowledge tuple in the knowledge base, first sample knowledge information converted from the first knowledge tuple, and second sample knowledge information obtained by learning prompt information of the second candidate question through a large language model, and is used for training a small language model. The application fuses the knowledge base and the language model to train a knowledge generation model with small size and capable of generating high-quality knowledge information, so as to conveniently and efficiently predict a high-precision answer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a question and answer method and device based on a knowledge base and a language model. BACKGROUND

[0002] With the development of the Internet and intelligent technology, the requirements of users for intelligent question and answer machines are continuously increasing. Question and answer is a challenging technical field of question and answer, aiming to enable intelligent agents to correctly predict intuitive phenomena in the physical world and expected changes in human psychology like ordinary humans. In order to solve related problems, it is often necessary to increase background knowledge that is not directly stated. Therefore, in the process of problem system processing, compared with directly enabling a question and answer model to predict an answer, adding useful knowledge basis information (related knowledge or reasoning details) can bring better answer prediction performance and result interpretability.

[0003] However, it is not easy to obtain high-quality knowledge basis information for a question. In existing technical practices, knowledge is often retrieved from a knowledge base as knowledge basis information, but this approach is often limited by the coverage and retrieval availability of existing knowledge bases, making it difficult to obtain relevant knowledge basis information required by the question, and thus leading to poor question and answer accuracy. SUMMARY

[0004] The present application provides a question and answer method and device based on a knowledge base and a language model, to solve the defect that it is difficult to obtain relevant knowledge basis information required by a question in the prior art, leading to poor question and answer accuracy, and to realize generation of high-quality question and answer knowledge basis information, to improve question and answer accuracy.

[0005] The present application provides a question and answer method based on a knowledge base and a language model, comprising:

[0006] inputting a target question into a knowledge generation model to obtain at least one dimension of target knowledge information of the target question;

[0007] obtaining target input information according to the target question and each dimension of the target knowledge information;

[0008] inputting the target input information into a question and answer model to obtain answer prediction information corresponding to the target question;

[0009] The knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question; and the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and an answer label of the sample question.

[0010] The first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by prompting information learning of the second candidate question by a large language model; the first knowledge tuple includes a knowledge tuple associated with the sample question; and the second knowledge tuple includes a knowledge tuple having a same relation entity, different question entities and different answer entities as the first knowledge tuple.

[0011] According to the method, the first candidate question is obtained based on the following steps:

[0012] The first knowledge tuple is extracted from the knowledge base;

[0013] A target verbalization template is determined according to a relation type of a relation entity in the first knowledge tuple;

[0014] The question entity and the relation entity in the first knowledge tuple are converted into the first candidate question according to the target verbalization template.

[0015] According to the method, the second candidate question is obtained based on the following steps:

[0016] At least one second knowledge tuple is extracted from the knowledge base;

[0017] The second candidate question is synthesized according to the first candidate question, an answer entity in the first knowledge tuple and an answer entity in each second knowledge tuple.

[0018] According to the method, the second sample knowledge information is obtained based on the following steps:

[0019] The second candidate question is input into the large language model to obtain multi-dimensional candidate knowledge information corresponding to each second candidate question and answer prediction information corresponding to each dimensional candidate knowledge information;

[0020] Consistency calculation is performed according to the answer prediction information corresponding to each dimensional candidate knowledge information and an answer label of the sample question to obtain a consistency score corresponding to each dimensional candidate knowledge information;

[0021] Sample change information of the answer prediction information before and after each dimensional candidate knowledge information is predicted on the basis of the second candidate question;

[0022] According to the sample change information, a beneficial score corresponding to each dimension of the candidate knowledge information is obtained;

[0023] According to the consistency score and the beneficial score, the second sample knowledge information is obtained from the candidate knowledge information in multiple dimensions.

[0024] According to the present application, a question and answer method based on a knowledge base and a language model is provided, and the training step of the knowledge generation model comprises:

[0025] According to the first candidate question and the first sample knowledge information, the small language model is pre-trained to obtain a pre-training knowledge model;

[0026] According to the second candidate question and the second sample knowledge information, the pre-training knowledge model is trained to obtain the knowledge generation model.

[0027] According to the present application, a question and answer method based on a knowledge base and a language model is provided, and the method further comprises:

[0028] The target input information is input into the question and answer model, and target change information of answer prediction information before and after each dimension of the target knowledge information is added on the basis of the target question is predicted;

[0029] According to the target change information, a beneficial score corresponding to each dimension of the target knowledge information is obtained;

[0030] The beneficial score corresponding to each dimension of the target knowledge information is taken as a reward value, the knowledge generation model is taken as a strategy network, and the knowledge generation model is updated by reinforcement learning.

[0031] According to the present application, a question and answer method based on a knowledge base and a language model is provided, and the target input information is obtained according to the target question and each dimension of the target knowledge information, comprising:

[0032] The blank knowledge information corresponding to the target question is generated;

[0033] According to the blank knowledge information and at least one dimension of the target knowledge information, a knowledge set is generated;

[0034] The target question is spliced with each knowledge information in the knowledge set to obtain the target input information.

[0035] The present application also provides a question and answer device based on a knowledge base and a language model, comprising:

[0036] The knowledge generation unit is used for inputting the target question into the knowledge generation model to obtain at least one dimension of the target knowledge information of the target question;

[0037] an information acquisition unit configured to acquire target input information according to the target question and each dimension of the target knowledge information;

[0038] an answer prediction unit configured to input the target input information into a question and answer model to obtain answer prediction information corresponding to the target question;

[0039] The knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information, and second sample knowledge information of a sample question; and the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information, the second sample knowledge information, and an answer label of the sample question.

[0040] The first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple includes a knowledge tuple associated with the sample question; and the second knowledge tuple includes a knowledge tuple having the same relationship entity, different question entities, and different answer entities as the first knowledge tuple.

[0041] The application further provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned question and answer method based on a knowledge base and a language model when executing the program.

[0042] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executable on a processor to implement the above-mentioned question and answer method based on a knowledge base and a language model.

[0043] The application further provides a computer program product including a computer program, and the computer program is executable on a processor to implement the above-mentioned question and answer method based on a knowledge base and a language model.

[0044] The application provides a knowledge base and language model based question and answer method and device, which automatically synthesizes common sense question and answer data through knowledge tuples in the knowledge base, and supplements diverse knowledge information through prompt learning of a large language model to synthesize a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question, and trains a small size knowledge generation model capable of generating high quality knowledge information based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and easy to deploy and adjust, and jointly deploys the knowledge generation model and the question and answer model, and learns to generate high quality multi-dimensional knowledge information for the target question requiring common sense question and answer according to the knowledge generation model, and provides high quality and high explanatory knowledge basis information for the question and answer model, so that the question and answer model can conveniently and efficiently predict effective, reliable and high precision answers. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 is one of the flowcharts of the knowledge base and language model based question and answer method provided by the application;

[0047] Figure 2 is another flowchart of the knowledge base and language model based question and answer method provided by the application;

[0048] Figure 3 is a structural schematic diagram of the knowledge base and language model based question and answer device provided by the application;

[0049] Figure 4 is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0050] In order to make the objects, technical solutions and advantages of the application clearer, the technical solutions in the application will be described clearly and completely below with reference to the drawings in the application. Obviously, the described embodiments are some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0051] The method provided in the embodiment can be applied to various question and answer fields, such as common sense question and answer, and the scheme provided in the embodiment will be described below taking common sense question and answer as an example.

[0052] With the development of the Internet and intelligent technology, the requirements of users for intelligent question-answering machines are constantly increasing. Common sense question-answering is a challenging question-answering technology field, aiming to enable intelligent agents to correctly predict intuitive phenomena in the physical world and expected changes in human psychology like ordinary humans. In order to solve related problems, often unexplained background common sense knowledge is needed. Therefore, in the process of question system processing, compared with directly enabling a question-answering model to predict an answer, adding useful knowledge basis information (hereinafter also referred to as knowledge information), such as answer-related knowledge and / or reasoning details, can bring better answer prediction performance and result interpretability. For example, when answering the question "what can an owl do?", giving the knowledge basis information "an owl is a bird; birds can fly" is of great help to correctly predict the answer.

[0053] However, it is not easy to obtain high-quality basis for common sense questions. In existing technical practices, if knowledge is retrieved from a common sense knowledge base, it is often limited by the coverage and retrieval availability of the existing common sense knowledge base, and it is difficult to obtain relevant knowledge basis information required by the question, resulting in poor question-answering accuracy.

[0054] To solve the above problems, the embodiment provides a question-answering method based on a knowledge base and a language model. The method generates questions for knowledge basis in common sense question-answering, and comprehensively utilizes question-answering data with knowledge basis information synthesized by a knowledge base and a language model to train a small-size knowledge generation model that is easy to deploy and adjust. The model can learn to generate high-quality knowledge basis information only by automatically synthesized target questions, and provide high-quality and highly interpretable knowledge basis information for a question-answering model. Thus, the question-answering model can conveniently and efficiently predict effective, reliable and high-precision answers.

[0055] Figure 1 A flowchart of the question-answering method based on the knowledge base and the language model is provided. As shown in Figure 1 the method includes:

[0056] Step 110, inputting a target question into a knowledge generation model to obtain at least one-dimensional target knowledge information of the target question;

[0057] The knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information, and second sample knowledge information of a sample question; the first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted according to the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple includes a knowledge tuple associated with the sample question; and the second knowledge tuple includes a knowledge tuple having a same relationship entity, different question entities, and different answer entities as the first knowledge tuple.

[0058] The target question can be a question required to be answered by common sense, which can be directly input by a user, or can be obtained by scanning an image through an image acquisition device such as a scanner, a mobile phone, or a camera, and performing optical character recognition (OCR) on the image. The present embodiment does not make a specific limitation on this.

[0059] Optionally, in order to improve the accuracy of the common sense question answering, before step 110 is performed, a small language model (small language model) is trained by comprehensively using common sense knowledge base and common sense question answering data with knowledge information synthesized by a large language model (large language model), so as to train a small-size knowledge generation model which is easy to deploy and adjust, and to provide useful and human-readable knowledge basis for common sense question answering.

[0060] The large language model can be various pre-trained language models having a large parameter scale, such as a third-generation general pre-training transformer.

[0061] The training steps of the knowledge generation model are as follows:

[0062] First, a large number of sample questions are determined, knowledge tuples associated with the sample questions are extracted from a knowledge base to obtain first knowledge tuples, and a first candidate question is synthesized according to entity information of the first knowledge tuples. The first knowledge tuple is a knowledge tuple in a three-tuple form, and the specific format can be (h, r, t), where h is a question entity, r is a relationship entity, and t is an answer entity. For example, the first knowledge tuple can be (owl, ability, fly).

[0063] In addition, at least one knowledge tuple having a same relationship entity r but different head entities (question entities) h and tail entities (answer entities) t as the first knowledge tuple is extracted from the knowledge base as a second knowledge tuple (hi ′ r, t i ′ ), and combining the first candidate question and the second knowledge element to obtain a second candidate question.

[0064] Subsequently, the first knowledge element is directly converted into basic form knowledge information for the second candidate question to obtain first sample knowledge information k source ; and a large language model is called for prompt information learning to perform diversified and high-quality knowledge information prediction for the second candidate question to obtain second sample knowledge information k llm .

[0065] Subsequently, the small language model is trained according to the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information of the sample question to obtain a knowledge generation model of small size which is easy to deploy and adjust and can generate high-quality knowledge information, so as to provide useful and human-readable knowledge information for common sense question answering.

[0066] The training method of the knowledge generation model can be one-stage overall training or multi-stage staged training. For example, the first candidate question and the second candidate question are taken as input, the first sample knowledge information and the second sample knowledge information are taken as labels, an overall training data set is constructed, and the small language model is one-stage overall trained to obtain the knowledge generation model. For another example, the first candidate question is taken as input and the first sample knowledge information is taken as label to construct first-stage training data, and the small language model is first-stage trained. Subsequently, the second candidate question is taken as input and the second sample knowledge information is taken as label to construct second-stage training data, and the small language model is second-stage trained on the basis of the first-stage trained small language model to obtain the knowledge generation model.

[0067] After the knowledge generation model is obtained by training, the target question is input into the trained knowledge generation model, and the knowledge generation model generates knowledge information of the target question according to the rich knowledge learned in the knowledge base and the generalization knowledge generation ability learned on the basis of the large language model, so as to obtain at least one-dimensional target knowledge information.

[0068] In step 120, target input information is obtained according to the target question and each dimension of the target knowledge information.

[0069] Optionally, after obtaining the target question and the target knowledge information of the target question, the target question and the target knowledge information of each dimension can be fused, so that the target input information input into the question and answer model contains question information and knowledge information, so that the question and answer model can output correct answers more accurately and efficiently. The so-called fusion can be splicing, coding, etc., which is not limited in the embodiment.

[0070] Step 130, inputting the target input information into the question and answer model to obtain answer prediction information corresponding to the target question; wherein the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and the answer label of the sample question;

[0071] Optionally, before step 130, the general question and answer model can be trained according to the common sense question and answer data with knowledge information synthesized by the common sense knowledge base and the large language model, and the question and answer model capable of accurately outputting answers is trained. The specific steps include:

[0072] The first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information are input, and the answer label of the sample question is labeled, and the general question and answer model is trained to obtain a question and answer model capable of accurately outputting answers according to questions and knowledge. The so-called general question and answer model is a language model pre-trained based on general domain question and answer data.

[0073] After training and obtaining the question and answer model, the target input information containing question information and knowledge information can be input into the question and answer model to output answer prediction information corresponding to the target question by the question and answer model.

[0074] The so-called question and answer model answering mode can be that the question and answer model predicts the probability p(a|q,k) of each candidate answer a(option) belonging to the answer of the target question q, and finds the maximum prediction probability when splicing knowledge information k of different dimensions for each candidate answer as the score of each candidate answer, and finally takes the corresponding candidate answer with the maximum score as the final answer prediction information. The specific calculation formula is:

[0075]

[0076] Wherein, A and K are answer sets containing multiple candidate answers and knowledge sets containing multiple knowledge information. The so-called prediction probability can be calculated based on the log-likelihood probability function.

[0077] The method provided in the embodiment automatically synthesizes common sense question and answer data through knowledge tuples in a knowledge base, supplements diverse knowledge information through prompt learning of a large language model, synthesizes a first candidate question, a second candidate question, first sample knowledge information, and second sample knowledge information of a sample question, and trains a small-size knowledge generation model that can generate high-quality knowledge information and is easy to deploy and adjust based on the first candidate question, the second candidate question, the first sample knowledge information, and the second sample knowledge information. The knowledge generation model and the question and answer model are jointly deployed. For a target question that needs common sense question and answer, the knowledge generation model is used to learn and generate high-quality multi-dimensional knowledge information, which provides high-quality and high-explanation knowledge basis information for the question and answer model. Therefore, the question and answer model can conveniently and efficiently predict an effective, reliable, and high-precision answer.

[0078] In some embodiments, the first candidate question is obtained based on the following steps:

[0079] extracting the first knowledge tuple from the knowledge base;

[0080] determining a target verbalization template according to a relationship type of a relationship entity in the first knowledge tuple;

[0081] converting a question entity and a relationship entity in the first knowledge tuple into the first candidate question according to the target verbalization template.

[0082] Optionally, to synthesize common sense question and answer data, a knowledge tuple associated with a sample question can be extracted from a knowledge base as a first knowledge tuple (h, r, t). According to a relationship type of a relationship entity r of the first knowledge tuple, a corresponding target verbalization template is associated and obtained. Then, the head entity and the relationship entity (h, r) of the first knowledge tuple are converted into a stem of a question, that is, a first candidate question, according to the target verbalization template. And the tail entity t is taken as a correct answer of the sample question, that is, an answer label a * .

[0083] For example, the first knowledge tuple (h, r, t) is (owl, ability, fly), and the corresponding first candidate question can be "What can an owl do".

[0084] The method provided in the embodiment automatically converts ternary knowledge tuples in a knowledge base according to natural language templates, which can ensure the accuracy, consistency, and adaptability of the conversion of each question, thereby improving the accuracy, flexibility, and intelligence of subsequent common sense question and answer.

[0085] In some embodiments, the second candidate question is obtained based on the following steps: ​

[0086] extracting at least one second knowledge tuple from the knowledge base;

[0087] synthesizing the second candidate question according to the first candidate question, the answer entity in the first knowledge tuple, and the answer entity in each second knowledge tuple.

[0088] Optionally, in order to synthesize more complete and diversified common sense question and answer data, several other triples with the same relationship entity r but different head entity and tail entity as the first knowledge tuple (h, r, t) can be randomly sampled from the knowledge base as the second knowledge tuple (h i ′ ,r,t i ′ ).

[0089] Subsequently, the tail entity t i ′ of the second knowledge tuple is taken as the interference item of the question, the first candidate question (stem) is connected with the answer entity in the first knowledge tuple and at least one randomly arranged interference item to automatically synthesize at least one multiple-choice question with diversified samples, thereby obtaining the second candidate question q.

[0090] For example, the first candidate question is "What can an owl do?", the answer entity in the first knowledge tuple is flying, and the tail entity t i ′ of the second knowledge tuple is speaking, and the synthesized second candidate question q is "What can an owl do? (A) Flying (B) Speaking".

[0091] The method provided in the embodiment can obtain common sense question and answer data samples in the form of multiple choices through the first candidate question, the answer entity in the first knowledge tuple, and the interference item formed by the second knowledge tuple with the same relationship entity r but different head entity and tail entity as the first knowledge tuple (h, r, t), thereby automatically synthesizing more complete and diversified common sense question and answer data, so as to improve the robustness and accuracy of question and answer.

[0092] In some embodiments, the step of obtaining the first sample knowledge information comprises:

[0093] According to the target verbalization template corresponding to the relationship entity in the first knowledge tuple, the first knowledge tuple (h, r, t) is converted into basic form knowledge information, that is, the first sample knowledge information k source For example, "Owls can fly", thereby obtaining the direct statement form of the knowledge triple in the knowledge base as the basic knowledge information, so that the knowledge generation model trained therefrom can more directly learn the knowledge in the knowledge base. ​​

[0094] In some embodiments, the step of obtaining the second sample knowledge information comprises:

[0095] inputting the second candidate question into the large language model to obtain multi-dimensional candidate knowledge information corresponding to each of the second candidate questions, and answer prediction information corresponding to each of the candidate knowledge information;

[0096] performing consistency calculation according to the answer prediction information corresponding to each of the candidate knowledge information and the answer label of the sample question to obtain a consistency score corresponding to each of the candidate knowledge information;

[0097] predicting sample change information of the answer prediction information before and after adding each of the candidate knowledge information based on the second candidate question;

[0098] obtaining a beneficial score corresponding to each of the candidate knowledge information according to the sample change information;

[0099] obtaining the second sample knowledge information from the multi-dimensional candidate knowledge information according to the consistency score and the beneficial score.

[0100] Optionally, since the basic knowledge information obtained from the database is relatively monotonous, in order to supplement more diverse knowledge information for the sample question, the embodiment adopts a large language model to obtain diversified knowledge information after prompt learning, and in order to ensure the quality of the generated knowledge information, the generated knowledge information is filtered for consistency (i.e., the answer prediction result obtained by the large language model is consistent with the answer label) and beneficial (i.e., the supplemented knowledge makes the general question and answer model more inclined to predict the correct answer), so as to convert the finally screened and retained knowledge information into the training input form of the generative language model. The specific implementation steps are as follows:

[0101] obtain diverse candidate knowledge information, such as "owl is a bird", by calling a large language model (such as a third-generation general pre-training converter) to perform prompt learning on the second candidate question q.

[0102] In addition, in order to filter out incorrect and invalid synthesized data, the large language model is required to generate answer prediction information corresponding to each of the candidate knowledge information during the process of calling the large language model, and the consistency score corresponding to each of the candidate knowledge information is obtained by performing consistency calculation on the answer prediction information corresponding to each of the candidate knowledge information and the answer label of the sample question. In addition, the general question and answer model is used to predict sample change information of the answer prediction probability before and after adding each of the candidate knowledge information based on the second candidate question, and the beneficial score corresponding to each of the candidate knowledge information is calculated according to the sample change information.

[0103] Then, the consistency score and the usefulness score are fused to filter the multi-dimensional candidate knowledge information based on the fusion result to obtain high-quality second sample knowledge information; or, the consistency filtering is performed based on the consistency score and the usefulness filtering is performed based on the usefulness score, to select, from the multi-dimensional candidate knowledge information, knowledge information with a consistency score greater than a first threshold value and a usefulness score greater than a second threshold value as the second sample knowledge information.

[0104] The calculation formula of the usefulness score is as follows:

[0105]

[0106] wherein, a * is the answer label.

[0107] The method provided by the embodiment can supplement diverse knowledge information by using the prompt learning capability of the large language model, and form high-quality common sense question answering and knowledge information after filtering, so that the knowledge generation model trained accordingly can more efficiently learn diverse and high-quality generalization knowledge information.

[0108] In some embodiments, the training step of the knowledge generation model comprises:

[0109] Based on the first candidate question and the first sample knowledge information, the small language model is pre-trained to obtain a pre-trained knowledge model;

[0110] Based on the second candidate question and the second sample knowledge information, the pre-trained knowledge model is trained to obtain the knowledge generation model.

[0111] Optionally, a smaller size pre-trained language model with a model size smaller than a certain scale, i.e., a small language model, can be selected; then, in order to enable the trained knowledge generation model to learn both the rich knowledge in the knowledge base and the generalization knowledge generation capability of the large language model, the small language model is trained in two stages by using the first candidate question and the second candidate question based on the sample question synthesized based on the knowledge base and the large language model, and the first sample knowledge information and the second sample knowledge information, to obtain a knowledge generation model that can efficiently output high-quality knowledge information according to various questions, so that the knowledge generation model can accurately learn the mapping x→y for generating a specific output for a specific input. The specific training steps of the knowledge generation model are as follows:

[0112] In the first stage, the first candidate question and the first sample knowledge information k source are used to pre-train the small language model, so as to train the small language model to generate a specific output for a specific input. In the case of generating source knowledge, i.e., the first sample knowledge information k source , thereby obtaining the pre-training knowledge model; the pre-training knowledge model can fully remember the association between the knowledge in the knowledge base and the question, i.e. y = k source .

[0113] For example, given the question "What can an owl do?" as input, the training goal is to enable the trained pre-training knowledge model to effectively predict the knowledge information "Owls can fly."

[0114] In the second stage, the second candidate question q and the second sample knowledge information k llm are synthesized by using the knowledge base and the large language model, and the pre-training knowledge model is trained to train the model, so as to generate generalized high-quality knowledge, i.e., the second sample knowledge information k llm , in the case of the second candidate question q, thereby obtaining the knowledge generation model; the knowledge generation model can imitate the large language model to generate high-quality knowledge information, i.e., x = q and y = k llm .

[0115] For example, given the question "What can an owl do? (A) Fly (B) Talk" as input, the training goal is to enable the trained knowledge generation model to effectively predict the knowledge information "Owls are birds, and birds can fly."

[0116] Compared with some related technologies that use large language models to generate "thinking chain" type knowledge basis information through prompts, this capability only emerges spontaneously in models with a large number of parameters (usually more than 10 billion parameters), which has a high deployment cost and is difficult to further adjust. If a knowledge basis generation model is to be built on the basis of a small-size pre-training language model, costly question-answer data labeling and knowledge basis data labeling are often required to enable the model to have the ability to generate diverse knowledge basis according to the question; the method provided in the embodiment trains a small-size language model in multiple stages through the first candidate question, the second candidate question, the first sample knowledge information, and the second sample knowledge information synthesized based on the knowledge base and the large language model, thereby strengthening the ability of the knowledge generation model to learn the rich knowledge in the knowledge base and the general knowledge generation capability of the large language model, while training a small-size knowledge generation model that is more convenient to deploy and has controllable parameters at a lower cost, thereby realizing low-cost and high-precision question answering.

[0117] In some embodiments, the method further comprises:

[0118] input the target input information into the question and answer model, and predict target change information of answer prediction information before and after each dimension of the target knowledge information is added on the basis of the target question;

[0119] According to the target change information, obtain the beneficial score corresponding to each dimension of the target knowledge information;

[0120] The beneficial score corresponding to each dimension of the target knowledge information is used as a reward value, and the knowledge generation model is used as a strategy network to update the knowledge generation model through reinforcement learning.

[0121] Optionally, due to the timeliness of the question and answer data, the distribution of the question and answer data will drift, thereby affecting the performance of knowledge generation. Therefore, when additional specific common sense question and answer task training data is available, the knowledge generation model can be updated in real time according to the performance of the question and answer model on real data, to further update the knowledge generation model in real time, and to improve the generalization ability of the knowledge generation model in a scenario with high timeliness requirement. The specific updating steps are as follows:

[0122] The knowledge generation model is used as a strategy model, the running result of the question and answer model on real data is calculated, and the beneficial score is used as a reward function. A proximal policy optimization algorithm is used to further update the knowledge generation model, so as to further improve the knowledge generation performance of the knowledge generation model on specific common sense question and answer tasks.

[0123] Here, the step of calculating the beneficial score can be that the target input information is input into the question and answer model, and target change information of answer prediction information before and after each dimension of the target knowledge information is added on the basis of the target question is predicted, so as to calculate the beneficial score corresponding to each dimension of the target knowledge information through the target change information. For details, refer to the beneficial score calculation steps under the sample question, which will not be repeated here.

[0124] The method provided in this embodiment updates the knowledge generation model in real time through a logical closed loop of real-time updating, so that the knowledge generation model has an automatic updating capability, and provides comprehensive protection for the integrity, effectiveness and high quality of knowledge generation. The automatic updating of the model capability enables the question and answer technology to continuously evolve to adapt to market changes and new business demands, is easy to expand and upgrade, and has good adaptability, functional expansibility and operational flexibility.

[0125] The following specific examples illustrate the complete process of the question and answer method based on the knowledge base and the language model provided in this embodiment.

[0126] Figure 2 The second flowchart of the question and answer method based on the knowledge base and the language model provided in this embodiment is as follows: Figure 2The complete process of the method includes:

[0127] Data preparation steps: Obtain the first knowledge tuple (h, r, t) from the knowledge base, such as (owl, ability, flight); based on the type of relation entity r in the first knowledge tuple, call the corresponding verbalization template to transform the head entity and relation entity (h, r) into the first candidate question. For example, "What can an owl do?", and the tail entity t is taken as the correct answer a. * Sample several second knowledge tuples (h, r, t) that have the same relation as entity r but have different head and tail entities. i ′ ,r,t i ′ ), t i ′ As a distractor to the question, then the question stem The tail entity of the first knowledge tuple and the randomly ordered distractors are concatenated to obtain the second candidate question q, such as "What can an owl do? (A) Fly (B) Talk"; the source first knowledge tuple (h,r,t) is transformed into the basic form of first sample knowledge information k through a verbalization template. source For example, "Owls can fly", a large language model is called to generate supplementary evidence, and multi-dimensional candidate sample knowledge information is obtained, such as "Owls are birds, and birds are good at flying", and the corresponding answer, such as "(A) flying". The multi-dimensional candidate sample knowledge information is then filtered for consistency and usefulness to obtain the second sample knowledge information.

[0128] Model training steps: In the first stage, the small-sized language model is pre-trained with common sense knowledge using the first candidate question, such as "What can owls do?" and the first sample knowledge information, such as "Owls can fly", as labels. In the second stage, the pre-trained language model is trained with the second candidate question, such as "What can owls do? (A) Fly (B) Talk", and the second sample knowledge information, such as "Owls are birds, and birds are good at flying", as labels, so that the trained knowledge generation model (hereinafter referred to as LEROS) can imitate the high-quality knowledge information generation of the large language model.

[0129] The inference step involves jointly deploying a knowledge generation model and a question-answering model to generate multiple knowledge information for common sense questions, which are then integrated and predicted by the question-answering model.

[0130] The feedback and adjustment process involves using real-world common-sense question-and-answer training data as feedback. By leveraging the inference results of the generative model and the question-and-answer model, the knowledge generation model is optimized and adjusted through reinforcement learning, further improving its knowledge generation performance.

[0131] And the question and answer method provided in the embodiment is simulated by experiment to verify the effectiveness of the question and answer method provided in the embodiment.

[0132] During the experiment, a large common sense reasoning dataset ATOMIC-2020 is used as a source knowledge base for synthetic common sense question and answer data, and five common sense question and answer benchmark datasets CommonsenseQA (common common sense question and answer CSQA), QASC (Question Answering in Context, scientific question and answer challenge), PhysicIQA (physics common sense question and answer, PIQA), SocialIQA (social common sense question and answer, SIQA), and WINOGRANDE (WG) test sets are used as question and answer test data in the experiment.

[0133] Subsequently, the effectiveness of the existing method and the method provided in the embodiment (hereinafter referred to as Ours) is evaluated by average result (Average), average gain (Avg. Gain) to illustrate its effectiveness. Among them, the existing method includes: a general question and answer model (UnifiedQA-large, UQA); directly fine-tuning the training UnifiedQA-large using synthetic questions and performing testing, a question and answer model (hereinafter referred to as UQA_syn) that does not introduce any knowledge basis during training and testing; using a small amount of artificially annotated question -> knowledge basis sample prompt, using a general pre-training converter-3.5-turbo to generate a knowledge basis according to a test question, and inputting the UQA model for testing (hereinafter referred to as Few-shot general pre-training converter-3.5-turbo); using a self-talk prompt format, guiding the general pre-training converter-3 (13B) to generate a knowledge basis according to a test question, and inputting the UQA model for testing (hereinafter referred to as Self-talk general pre-training converter-3 (13B)); using a small amount of artificially annotated question -> knowledge basis sample prompt, guiding the general pre-training converter-3 (13B) to generate a knowledge basis according to a test question, and inputting the UQA model for testing (hereinafter referred to as Few-shot general pre-training converter-3 (13B)); using a real question and answer data + reinforcement learning feedback optimization question and answer model (hereinafter referred to as RAINIER (770M)); a question and answer model (hereinafter referred to as LEROS_RL) introduced on the basis of LEROS after feedback optimization.

[0134] As shown in Table 1, the comparative results on the CSQA, QASC, PIQA, SIQA, and WG datasets show that the small-size knowledge generation model (LEROS 770 million parameters) trained by fusing the knowledge base and the language model provided in the embodiment outperforms the few-shot prompt large-scale language model (Universal Pretrained Transformer-3, 13 billion parameters) on the CSQA, QASC, PIQA, SIQA, and WG datasets, and the performance can be further improved through feedback adjustment. Therefore, the knowledge enhancement effect of the method provided in the embodiment on different common sense question and answer task datasets is obviously improved compared with existing methods. Thus, it is shown that the method provided in the embodiment can effectively extract the advantages of the knowledge base and the large language model into a small-size knowledge generation model and generate helpful knowledge information in the common sense question and answer task, effectively improving the question and answer accuracy.

[0135] Table 1 simulation comparison table

[0136]

[0137] The knowledge base and language model-based question and answer device provided in the present application is described below. The knowledge base and language model-based question and answer device described below can be mutually corresponding to the knowledge base and language model-based question and answer method described above.

[0138] Figure 3 The structure diagram of the knowledge base and language model-based question and answer device provided in the present embodiment is shown in FIG. 3. Figure 3 As shown in FIG. 3, the device comprises:

[0139] The knowledge generation unit 310 is configured to input a target question into a knowledge generation model to obtain at least one-dimensional target knowledge information of the target question.

[0140] The information acquisition unit 320 is configured to acquire target input information according to the target question and each dimension of the target knowledge information.

[0141] The answer prediction unit 330 is configured to input the target input information into a question and answer model to obtain answer prediction information corresponding to the target question.

[0142] The knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information, and second sample knowledge information of a sample question; and the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information, and the second sample knowledge information, and an answer label of the sample question.

[0143] The first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by prompting information learning of the second candidate question by a large language model; the first knowledge tuple comprises a knowledge tuple associated with the sample question; and the second knowledge tuple comprises a knowledge tuple having a same relationship entity, a different question entity and a different answer entity as the first knowledge tuple.

[0144] The device provided by the embodiment automatically synthesizes common sense question and answer data from knowledge tuples in a knowledge base, supplements diverse knowledge information by prompt learning of a large language model, synthesizes a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question, and trains a small-size knowledge generation model capable of generating high-quality knowledge information based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and the knowledge generation model and a question and answer model are jointly deployed, the knowledge generation model is used to learn and generate high-quality multi-dimensional knowledge information for a target question requiring common sense question and answer, the question and answer model is provided with high-quality and high-explanation knowledge basis information, and thus the question and answer model can conveniently and efficiently predict an effective, reliable and high-precision answer.

[0145] The device provided by the embodiment is used to execute the above-mentioned method embodiments, and specific processes and detailed contents are referred to the above-mentioned embodiments, which will not be described herein.

[0146] Figure 4 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 4As shown, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 complete mutual communication through the communications bus 440. The processor 410 can invoke a logic instruction in the memory 430 to execute a knowledge base and language model-based question answering method, which includes: inputting a target question into a knowledge generation model to obtain at least one-dimensional target knowledge information of the target question; obtaining target input information according to the target question and each-dimensional target knowledge information; inputting the target input information into a question and answer model to obtain answer prediction information corresponding to the target question; wherein the knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information, and second sample knowledge information of a sample question; the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information, and the second sample knowledge information, and an answer label of the sample question; the first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple includes a knowledge tuple associated with the sample question; and the second knowledge tuple includes a knowledge tuple having the same relationship entity, different question entities, and different answer entities as the first knowledge tuple.

[0147] In addition, the logic instruction in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0148] In another aspect, the present application also provides a computer program product comprising a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being capable of executing the knowledge base and language model based question answering method provided by the above method when executed by a processor, the method comprising: inputting a target question into a knowledge generation model to obtain at least one-dimensional target knowledge information of the target question; obtaining target input information according to the target question and each dimension of the target knowledge information; inputting the target input information into a question and answer model to obtain answer prediction information corresponding to the target question; wherein the knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question; the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and an answer label of the sample question; the first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple comprises a knowledge tuple associated with the sample question; and the second knowledge tuple comprises a knowledge tuple having the same relationship entity, different question entity and different answer entity as the first knowledge tuple.

[0149] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the knowledge base and language model based question answering method provided by the above method, the method comprising: inputting a target question into a knowledge generation model to obtain at least one-dimensional target knowledge information of the target question; obtaining target input information according to the target question and each-dimensional target knowledge information; inputting the target input information into a question answering model to obtain answer prediction information corresponding to the target question; wherein the knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question; the question answering model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and an answer label of the sample question; the first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple comprises a knowledge tuple associated with the sample question; and the second knowledge tuple comprises a knowledge tuple having the same relationship entity, different question entity and different answer entity as the first knowledge tuple.

[0150] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0151] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus necessary universal hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0152] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A knowledge base and language model based question answering method, characterized by, The method comprises the following steps: inputting a target question into a knowledge generation model to obtain at least one-dimensional target knowledge information of the target question; acquiring target input information according to the target question and each dimension of the target knowledge information; inputting the target input information into a question and answer model to obtain answer prediction information corresponding to the target question; wherein the knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question; the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and an answer label of the sample question; the first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple comprises a knowledge tuple associated with the sample question; the second knowledge tuple comprises a knowledge tuple having the same relationship entity, different question entities and different answer entities as the first knowledge tuple; the second sample knowledge information is obtained based on the following steps: inputting the second candidate question into the large language model to obtain multi-dimensional candidate knowledge information corresponding to each of the second candidate questions, and answer prediction information corresponding to each dimension of the candidate knowledge information; performing consistency calculation according to the answer prediction information corresponding to each dimension of the candidate knowledge information and the answer label of the sample question to obtain a consistency score corresponding to each dimension of the candidate knowledge information; predicting sample change information of the answer prediction information before and after adding each dimension of the candidate knowledge information on the basis of the second candidate question; obtaining a beneficial score corresponding to each dimension of the candidate knowledge information according to the sample change information; obtaining the second sample knowledge information from the multi-dimensional candidate knowledge information according to the consistency score and the beneficial score. 2.The knowledge base and language model based question answering method according to claim 1, characterized in that, the first candidate question is obtained based on the following steps: extracting the first knowledge tuple from the knowledge base; determining a target verbalization template according to the relationship type of the relationship entity in the first knowledge tuple; converting the question entity and the relationship entity in the first knowledge tuple into the first candidate question according to the target verbalization template. 3.The knowledge base and language model based question answering method according to claim 1, characterized in that, the second candidate question is obtained based on the following steps: extracting at least one second knowledge tuple from the knowledge base; synthesizing the second candidate question according to the first candidate question, the answer entity in the first knowledge tuple, and the answer entity in each of the second knowledge tuples.

4. The knowledge base and language model based question answering method according to any one of claims 1-3, characterized in that, The training steps of the knowledge generation model comprise: pre-training the small language model based on the first candidate question and the first sample knowledge information to obtain a pre-trained knowledge model; The pre-training knowledge model is trained based on the second candidate question and the second sample knowledge information, to obtain the knowledge generation model.

5. The knowledge base and language model based question answering method according to any one of claims 1-3, characterized in that, The method further comprises: inputting the target input information into the question and answer model to predict target change information of answer prediction information before and after each dimension of the target knowledge information is added to the target question; obtaining a beneficial score corresponding to each dimension of the target knowledge information according to the target change information; performing reinforcement learning update on the knowledge generation model by taking the beneficial score corresponding to each dimension of the target knowledge information as a reward value and taking the knowledge generation model as a strategy network.

6. The knowledge base and language model based question answering method according to any one of claims 1-3, characterized in that, The target input information is obtained according to the target question and each dimension of the target knowledge information, comprising: generating blank knowledge information corresponding to the target question; generating a knowledge set according to the blank knowledge information and at least one dimension of the target knowledge information; splicing the target question and each knowledge information in the knowledge set to obtain the target input information.

7. A knowledge base and language model based question answering apparatus, characterized by, comprising: a knowledge generation unit configured to input a target question into a knowledge generation model to obtain at least one dimension of target knowledge information of the target question; an information obtaining unit configured to obtain target input information according to the target question and each dimension of the target knowledge information; an answer prediction unit configured to input the target input information into a question and answer model to obtain answer prediction information corresponding to the target question; The knowledge generation model is obtained by training a small language model based on a first candidate question, a second candidate question, first sample knowledge information and second sample knowledge information of a sample question; the question and answer model is obtained by training based on the first candidate question, the second candidate question, the first sample knowledge information and the second sample knowledge information, and an answer label of the sample question; The first candidate question is generated according to a first knowledge tuple in a knowledge base; the second candidate question is generated according to the first candidate question and a second knowledge tuple in the knowledge base; the first sample knowledge information is converted from the first knowledge tuple; the second sample knowledge information is obtained by learning prompt information of the second candidate question through a large language model; the first knowledge tuple comprises a knowledge tuple associated with the sample question; the second knowledge tuple comprises a knowledge tuple having the same relationship entity, different question entities and different answer entities as the first knowledge tuple; The second sample knowledge information is obtained based on the following steps: inputting the second candidate question into the large language model to obtain multi-dimensional candidate knowledge information corresponding to each of the second candidate questions, and answer prediction information corresponding to each dimension of the candidate knowledge information; performing consistency calculation according to the answer prediction information corresponding to each dimension of the candidate knowledge information and the answer label of the sample question to obtain a consistency score corresponding to each dimension of the candidate knowledge information; predicting sample change information of answer prediction information before and after each dimension of the candidate knowledge information is added to the second candidate question; According to the sample change information, a beneficial score corresponding to each dimension of the candidate knowledge information is obtained; According to the consistency score and the beneficial score, the second sample knowledge information is obtained from the candidate knowledge information in multiple dimensions.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the knowledge base and language model based question answering method according to any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the knowledge base and language model based question answering method according to any one of claims 1 to 6 when executed by the processor.

Images