A conversational knowledge graph question-answering method, device, electronic device, and medium driven by a large language model
By acquiring and predicting the target focus entities and finding similar triples from the conversation knowledge graph, the problem of knowledge injection failure caused by the missing or pronounization of conversation topic entities in the large language model is solved, ensuring the normal progress of conversation questions and answers.
Patent Information
- Application Number
- CN202411268220.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-11
AI Technical Summary
When using the large language model, the missing or pronounization of topic entities in the conversation leads to the failure of knowledge graph injection, and the Q&A of this round of conversation cannot be completed.
By obtaining the questions to be answered in the current session and entities in the historical question and answer, using the focus entity prediction model to determine the target focus entity, and looking for triples similar to the questions to be answered from the pre-established session knowledge graph to respond.
It avoids the failure of knowledge injection caused by the missing target focus entities or pronounization, and ensures that the Q&A of this round of conversation can be carried out normally.
Smart Images

Figure CN118779436B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of large language models, and in particular to a conversational knowledge graph question-answering method, device, electronic device and medium driven by a large language model. Background Art
[0002] Large language models (LLMs) pre-trained on large-scale corpora have shown excellent performance in various natural language processing tasks. They can solve complex reasoning tasks by gradually generating instructions from questions to answers. However, when using LLMs, researchers have also discovered the problem of hallucination. Existing studies have shown that by injecting information from knowledge graphs into LLMs in different forms, the model's hallucination problem can be significantly improved. Generally, a knowledge-enhanced prompt method based on zero-shot learning is used to provide factual information for LLMs. The core idea of this method is to use knowledge graphs to retrieve triple knowledge related to entities explicitly mentioned in the input question, thereby providing LLMs with rich factual information. These key factual knowledge are then combined with questions as prompts as input to LLMs, which can effectively overcome the hallucination problem without retraining or fine-tuning the model.
[0003] However, when the subject entity is missing or pronounized in the questions of a certain round of conversation, the subject entity of this round cannot be determined, and thus the knowledge graph triples related to the subject entity cannot be obtained, resulting in failure of knowledge injection and inability to complete the question and answer of this round of conversation. Summary of the invention
[0004] In view of this, the purpose of the present application is to provide a conversational knowledge graph question-answering method, device, electronic device and medium driven by a large language model, which can answer questions to be answered and avoid the problem of failure of knowledge injection and inability to complete the question-answering of this round of conversation due to the missing or pronounization of the target focus entity.
[0005] In a first aspect, an embodiment of the present application provides a conversational knowledge graph question answering method driven by a large language model, the method comprising:
[0006] Obtain the first candidate focus entity in the question to be answered in the current session, and the second candidate focus entity in the historical questions and answers before the question to be answered; the historical questions and answers include the questions and answers in each round of historical questions and answers;
[0007] Input the unanswered question, historical questions and answers, the first candidate focus entity and the second candidate focus entity into the focus entity prediction model to obtain the target focus entity;
[0008] From all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph, find the target triple that is similar to the question to be answered;
[0009] Among them, the similarity between the question to be answered and the target triplet is greater than the similarity between the question to be answered and any triplet in the initial triplet except the target triplet;
[0010] Answer the questions to be answered based on the target triplet.
[0011] In a possible implementation, the focus entity prediction model is trained through the following steps:
[0012] Obtain candidate focus entity samples in the sample questions to be answered in the sample session and candidate focus entity samples in the historical question and answer samples before the sample questions to be answered;
[0013] Determine a target focus entity sample from candidate focus entity samples;
[0014] The focus entity prediction model is trained by taking the sample questions to be answered, historical question and answer samples, candidate focus entity samples as samples, and the target focus entity samples as labels.
[0015] In a possible implementation, determining a target focus entity sample from candidate focus entity samples includes:
[0016] For each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the candidate focus entity sample is determined as the target focus entity sample.
[0017] In a possible implementation, searching for a target triplet similar to the question to be answered from all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph includes:
[0018] Calculate the similarity between each initial triple and the question to be answered;
[0019] According to the similarity between each initial triple and the question to be answered, a target triple is selected from the initial triples.
[0020] In a possible implementation, selecting a target triple from the initial triples according to the similarity between each initial triple and the question to be answered includes:
[0021] A preset number of initial triples with the greatest similarity to the question to be answered are determined as target triples.
[0022] In a possible implementation, selecting a target triple from the initial triples according to the similarity between each initial triple and the question to be answered further includes:
[0023] An initial triplet whose similarity to the question to be answered is greater than a preset similarity is determined as a target triplet.
[0024] In a possible implementation, answering the question to be answered according to the target triplet includes:
[0025] After the preset text information, the target triplet, and the question to be answered are spliced in a preset order, the spliced text information, the target triplet, and the question to be answered are input into the answer text generation model to obtain the answer text of the question to be answered;
[0026] Output the response text in the session interface of the current session.
[0027] In a second aspect, the embodiment of the present application further provides a conversational knowledge graph question-answering device driven by a large language model, the device comprising:
[0028] An acquisition module is used to acquire a first candidate focus entity in the question to be answered in the current session, and a second candidate focus entity in the historical questions and answers before the question to be answered; the historical questions and answers include questions and answers in each round of historical questions and answers;
[0029] An input module, used to input the question to be answered, the historical question and answer, the first candidate focus entity and the second candidate focus entity into the focus entity prediction model to obtain the target focus entity;
[0030] A search module is used to search for a target triplet similar to the question to be answered from all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph;
[0031] Among them, the similarity between the question to be answered and the target triplet is greater than the similarity between the question to be answered and any triplet in the initial triplet except the target triplet;
[0032] The answering module is used to answer the question to be answered according to the target triplet.
[0033] In a possible implementation, the device also includes: a training module; a training module specifically used to obtain candidate focus entity samples in the sample questions to be answered in the sample conversation, and candidate focus entity samples in the historical question and answer samples before the sample questions to be answered; determine the target focus entity samples from the candidate focus entity samples; use the sample questions to be answered, the historical question and answer samples, and the candidate focus entity samples as samples, and the target focus entity samples as labels to train the focus entity prediction model.
[0034] In a possible implementation, the training module is further used to:
[0035] For each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the candidate focus entity sample is determined as the target focus entity sample.
[0036] In a possible implementation, the search module is specifically configured to calculate the similarity between each initial triple and the question to be answered; and select a target triple from the initial triples according to the similarity between each initial triple and the question to be answered.
[0037] In a possible implementation, the search module is further used to:
[0038] A preset number of initial triples with the greatest similarity to the question to be answered are determined as target triples.
[0039] In a possible implementation, the search module is further used to:
[0040] An initial triplet whose similarity to the question to be answered is greater than a preset similarity is determined as a target triplet.
[0041] In one possible implementation, the response module is specifically used to splice the preset text information, the target triplet, and the question to be answered in a preset order, and input them into a response text generation model to obtain the response text of the question to be answered; and output the response text in the conversation interface of the current conversation.
[0042] In the third aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, when the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the conversational knowledge graph question and answer method driven by a large language model as described in any one of the first aspects.
[0043] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the conversational knowledge graph question-and-answer method driven by a large language model as described in any one of the first aspects are executed.
[0044] The embodiment of the present application provides a conversational knowledge graph question-answering method, device, electronic device and medium driven by a large language model, the method comprising: obtaining a first candidate focus entity in the question to be answered in the current conversation, and a second candidate focus entity in the historical questions and answers before the question to be answered; the historical questions and answers include questions and answers in each round of historical questions and answers; the question to be answered, the historical questions and answers, the first candidate focus entity and the second candidate focus entity are input into the focus entity prediction model to obtain a target focus entity; from all the initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph, a target triple similar to the question to be answered is searched; wherein the similarity between the question to be answered and the target triple is greater than the similarity between the question to be answered and any triple in the initial triples except the target triple; according to the target triple, the question to be answered is answered. The present application determines the target focus entity from the first candidate focus entity in the question to be answered and the second candidate focus entity in the historical questions and answers, thereby avoiding the problem of knowledge injection failure and inability to complete the question and answer of the current round of conversation due to the missing or pronounization of the target focus entity. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0046] Figure 1 A flowchart of a conversational knowledge graph question-answering method driven by a large language model provided in an embodiment of the present application is shown;
[0047] Figure 2 A flowchart of another large language model-driven conversational knowledge graph question-answering method provided in an embodiment of the present application is shown;
[0048] Figure 3 A schematic diagram of the structure of a conversational knowledge graph question-answering device driven by a large language model provided in an embodiment of the present application is shown;
[0049] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0050] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.
[0051] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0052] In order to enable those skilled in the art to use the contents of this application, the following implementation is provided in conjunction with a specific application scenario, the "field of large language model technology". For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the "field of large language model technology", it should be understood that this is only an exemplary embodiment.
[0053] The following is a detailed description of a conversational knowledge graph question-answering method driven by a large language model provided in an embodiment of the present application.
[0054] Reference Figure 1 As shown, it is a flow chart of a conversational knowledge graph question-answering method driven by a large language model provided in an embodiment of the present application. The specific execution process of the method is as follows:
[0055] S101, obtaining a first candidate focus entity in the question to be answered in the current session, and a second candidate focus entity in the historical questions and answers before the question to be answered.
[0056] S102: Input the unanswered question, historical questions and answers, the first candidate focus entity and the second candidate focus entity into a focus entity prediction model to obtain a target focus entity.
[0057] S103. Search for a target triplet similar to the question to be answered from all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph.
[0058] S104. Answer the question to be answered according to the target triplet.
[0059] The embodiment of the present application provides a conversational knowledge graph question-answering method driven by a large language model, the method comprising: obtaining a first candidate focus entity in a question to be answered in a current conversation, and a second candidate focus entity in a historical question and answer before the question to be answered; the historical question and answer includes questions and answers in each round of historical question and answer; the question to be answered, the historical question and answer, the first candidate focus entity, and the second candidate focus entity are input into a focus entity prediction model to obtain a target focus entity; from all initial triples corresponding to the target focus entity in a pre-established conversation knowledge graph, a target triple similar to the question to be answered is searched; wherein the similarity between the question to be answered and the target triple is greater than the similarity between the question to be answered and any triple in the initial triples except the target triple;
[0060] Answer the question to be answered according to the target triple. This application determines the target focus entity from the first candidate focus entity in the question to be answered and the second candidate focus entity in the historical questions and answers, avoiding the problem of knowledge injection failure and inability to complete the question and answer of this round of conversation due to the missing or pronounization of the target focus entity.
[0061] The following is a description of the exemplary steps of the present application embodiment:
[0062] S101, obtaining a first candidate focus entity in the question to be answered in the current session, and a second candidate focus entity in the historical questions and answers before the question to be answered.
[0063] In an implementation manner of the present application, one or more rounds of questions and answers may be conducted in a conversation, and a complete round of questions and answers includes a question and a corresponding answer; if the round corresponding to the question to be answered in the current conversation is greater than 1, the first candidate focus entity in the question to be answered in the current conversation and the second candidate focus entity in the historical questions and answers before the question to be answered in the current conversation are obtained; if the round corresponding to the question to be answered in the current conversation is equal to 1, all entities in the question to be answered are determined as target focus entities.
[0064] Among them, the unanswered questions refer to the questions that the user has entered in the current session and has not yet answered; the historical questions and answers refer to the questions and answers in each round of the current session before the unanswered questions, and the historical questions and answers include the questions and answers in each round of historical questions and answers. The first candidate focus entity refers to all entities in the unanswered questions, and the second candidate focus entity refers to all entities in each round of historical questions and answers.
[0065] Here, since only the second round and above of questions and answers in a conversation will have the situation where the target focus entity of this round cannot be determined, and the large language model LLMs has a certain memory function, it can remember the content mentioned in the previous conversation and generate appropriate replies accordingly. Therefore, if the round corresponding to the question to be answered in the current conversation is greater than 1, the first candidate focus entity in the question to be answered in the current conversation and the second candidate focus entity in the historical questions and answers before the question to be answered are obtained to determine the target focus entity corresponding to the question to be answered.
[0066] S102: Input the unanswered question, historical questions and answers, the first candidate focus entity and the second candidate focus entity into a focus entity prediction model to obtain a target focus entity.
[0067] In the implementation manner of the present application, the first candidate focus entity and the second candidate focus are both candidate focus entities, and the target focus entity refers to the focus entity corresponding to the question to be answered. The entity prediction model is used to determine whether each candidate focus entity is a target focus entity, that is, the entity prediction model solves a binary classification problem, so the entity prediction model should be a binary classification model. The focus entity prediction model includes a word segmentation encoding layer BERT, a word segmentation vectorization layer, a pooling layer, a fully connected layer, and an output layer.
[0068] Specifically, the following steps are used to input the unanswered questions, historical questions and answers, and candidate focus entities into the focus entity prediction model to obtain the target focus entity:
[0069] Step 1: Concatenate the question to be answered, historical questions and answers, and candidate focus entities to obtain the concatenated sequence context, which is expressed as:
[0070] ;
[0071] in, represents the splicing sequence, For the first round of history quiz, For splicing operation, is the answer to the first round of historical questions and answers, t is the round corresponding to the question to be answered, is the answer to the t-1th round of the history quiz, For questions to be answered, is the first candidate focus entity, is the nth candidate focus entity, and n is the number of candidate focus entities.
[0072] Step 2: Input the concatenated sequence context into the word segmentation encoding layer BERT to obtain the subword encoding sequence H, which includes the subword encoding of the question to be answered, the subword encoding of the historical question and answer, and the subword encoding of the candidate focus entity; the subword encoding of the candidate focus entity contains information in the question to be answered and the historical question and answer.
[0073] The expression of H is: ;
[0074] in, , is the first subword code in the subword code of the question to be answered and the subword code of the historical question and answer. is the kth subword code in the subword code of the question to be answered and the subword code of the historical question and answer, is the total number of subword codes for the questions to be answered and the subword codes for the historical questions and answers, The first candidate focus entity The first subword encoding of is the i-th candidate focus entity The j-th subword encoding of is the nth candidate focus entity The mth subword encoding of , n is the number of candidate focus entities, and m is the nth candidate focus entity The number of subword encodings.
[0075] Step 3: Input each subword code of the candidate focus entity in the subword code sequence H into the word segmentation vectorization layer respectively, vectorize each subword code of each candidate focus entity, and obtain each subword vector of each candidate focus entity.
[0076] In the implementation mode of the present application, in order to improve the generalization ability of the model and prevent overfitting, each subword encoding is subjected to layer normalization and Dropout processing. In order to prevent the gradient vanishing problem that may occur in the model during training, the output after layer normalization and the original output of the BERT model are residually connected. This helps to retain the information of the original input and alleviate the problem of gradient vanishing during back propagation, thereby improving the performance and training efficiency of the model.
[0077] Vectorized expression: ;
[0078] in, is the i-th candidate focus entity The j-th subword vector of is layer normalization, is the i-th candidate focus entity The j-th subword encoding of .
[0079] Step 4: Input all subword vectors corresponding to each candidate focus entity into the pooling layer for pooling to obtain each candidate focus entity vector.
[0080] In the implementation manner of the present application, the hidden expression (subword vector) of each subword of the candidate focus entity spliced by residual connection is also pooled to obtain the expression of the candidate focus entity (candidate focus entity vector).
[0081] The representation of the pooling layer is: ;
[0082] Among them, among them, is the i-th candidate focus entity The candidate focal entity vector of For pooling, is the i-th candidate focus entity The j-th subword vector of .
[0083] Step 5: Input each candidate focal entity vector into the fully connected layer and the output layer to obtain the probability of each candidate focal entity.
[0084] Here, each candidate focal entity vector is input into the following formula to obtain the probability of each candidate focal entity:
[0085] ;
[0086] in, is the probability of the i-th candidate focal entity, is the activation function, , b are model parameters, is the i-th candidate focus entity The candidate focal entity vector of , T is the transpose.
[0087] Step six: determine the candidate focus entity whose probability is greater than a preset value as the target focus entity.
[0088] Furthermore, the focus entity prediction model is trained by the following steps:
[0089] Step 1: Obtain candidate focus entity samples in the sample questions to be answered in the sample session and candidate focus entity samples in the historical question and answer samples before the sample questions to be answered.
[0090] In the implementation manner of the present application, the sample question to be answered refers to the question that the user inputs in the sample session and has not yet completed the answer, and the round corresponding to the question in the sample session is greater than 1; the historical question and answer sample refers to the questions and answers in each round of the sample session before the sample question to be answered, and the historical question and answer sample includes the questions and answers in each round of historical question and answer. The candidate focus entity sample refers to all entities in the sample question to be answered and the historical question and answer sample.
[0091] Step 2: Determine the target focus entity sample from the candidate focus entity samples.
[0092] In the implementation mode of the present application, a target focus entity sample is determined from candidate focus entity samples using a long-distance supervision method, and the target focus entity sample is then used as a supervision loop; the long-distance supervision method will label the candidate focus entity samples of the target focus entity sample as pseudo labels of the target focus entity, and label the candidate focus entity samples that are not the target focus entity sample as pseudo labels of the non-target focus entity. Specifically, the following two methods are included:
[0093] The first method: for each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the candidate focus entity sample is determined as the target focus entity sample.
[0094] The second method: for each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the number of jumps from the candidate focus entity sample to the answer entity corresponding to the sample question to be answered is counted; the answer entity corresponding to the sample question to be answered is used to characterize the entity in the answer corresponding to the sample question to be answered; the candidate focus entity sample with a jump number less than the preset number is determined as the target focus entity sample.
[0095] Step 3: Use the sample questions to be answered, historical question and answer samples, candidate focus entity samples as samples, and target focus entity samples as labels to train the focus entity prediction model.
[0096] Furthermore, the prior art generally uses a cross entropy loss function in the model training stage. However, as the number of rounds in the session increases, the proportion of target focus entity samples in the candidate focus entity samples is smaller, resulting in an unbalanced distribution of labels for positive and negative examples. This unbalanced data may cause serious deviations in training, thereby affecting the performance of the model. In order to alleviate this problem, the embodiment of the present application uses a weighted cross entropy loss function to deal with the imbalance of categories, so that the focus entity prediction model pays more attention to positive samples, thereby improving the performance of the focus entity prediction model for unbalanced data.
[0097] ;
[0098] Among them, loss is the loss value of the focus entity prediction model, represents the number of candidate focal entity samples, Indicates The probability distribution of candidate focal entity samples, is the first annotation through distant supervision pseudo labels of candidate focal entity samples, is the weight of the positive sample, is the weight of negative samples, and It is determined by the ratio of positive and negative samples.
[0099] S103. Search for a target triplet similar to the question to be answered from all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph.
[0100] In the implementation manner of the present application, the similarity between the question to be answered and the target triple is greater than the similarity between the question to be answered and any triple in the initial triples except the target triple; since there may be many paths starting from the target focus entity, too many triples are retrieved, and most of them may not be relevant to the question to be answered. This situation may cause the length limit of the LLMs context to be exceeded, and a large amount of irrelevant knowledge injection may mislead LLMs to generate irrelevant answers. In order to solve this problem, it is necessary to further screen out the target triples that are more relevant to the question to be answered based on the similarity between the question to be answered and the triples. The conversation knowledge graph contains multiple triples. ,in, is the head entity in the triple, is the tail entity in the triple, for and The relationship between.
[0101] Specifically, the target triples similar to the question to be answered are found through the following steps:
[0102] a. Calculate the similarity between each initial triple and the question to be answered.
[0103] In an implementation manner of the present application, each initial triplet and the question to be answered are respectively input into the focus entity prediction model, and the subword coding sequence corresponding to each initial triplet and the subword coding sequence corresponding to the question to be answered output by the word segmentation coding layer BERT are extracted; the cosine similarity between the subword coding sequence corresponding to each initial triplet and the subword coding sequence corresponding to the question to be answered is calculated to obtain the similarity between each initial triplet and the question to be answered.
[0104] b. Selecting a target triple from the initial triples based on the similarity between each initial triple and the question to be answered.
[0105] In an embodiment of the present application, the initial triples whose similarity to the question to be answered is greater than a preset similarity are determined as target triples; or a preset number of initial triples with the greatest similarity to the question to be answered are determined as target triples.
[0106] S104. Answer the question to be answered according to the target triplet.
[0107] Specifically, the preset text information, the target triplet, and the question to be answered are spliced in a preset order and input into a response text generation model to obtain the response text of the question to be answered; and the response text is output in the conversation interface of the current conversation.
[0108] In the implementation of the present application, since LLMs process text input, and fact triples are symbolically represented on the knowledge graph. Therefore, before injecting the symbolic facts in the knowledge graph into LLMs, first The triples are converted to text strings.
[0109] For example, the preset text information may be “Below are facts in the forms of the triple meaningful to answer the question:”.
[0110] Reference Figure 2 As shown, it is a flow chart of another large language model driven conversational knowledge graph question answering method provided in an embodiment of the present application. The exemplary steps of the embodiment of the present application are described below:
[0111] S201. Obtain candidate focus entity samples in a sample question to be answered in a sample conversation, and candidate focus entity samples in a historical question-and-answer sample before the sample question to be answered.
[0112] In the implementation manner of the present application, the sample question to be answered refers to the question that the user inputs in the sample session and has not yet completed the answer, and the round corresponding to the question in the sample session is greater than 1; the historical question and answer sample refers to the questions and answers in each round of the sample session before the sample question to be answered, and the historical question and answer sample includes the questions and answers in each round of historical question and answer. The candidate focus entity sample refers to all entities in the sample question to be answered and the historical question and answer sample.
[0113] S202: Determine a target focus entity sample from candidate focus entity samples.
[0114] In the implementation mode of the present application, a target focus entity sample is determined from candidate focus entity samples using a long-distance supervision method, and the target focus entity sample is then used as a supervision loop; the long-distance supervision method will label the candidate focus entity samples of the target focus entity sample as pseudo labels of the target focus entity, and label the candidate focus entity samples that are not the target focus entity sample as pseudo labels of the non-target focus entity. Specifically, the following two methods are included:
[0115] The first method: for each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the candidate focus entity sample is determined as the target focus entity sample.
[0116] The second method: for each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the number of jumps from the candidate focus entity sample to the answer entity corresponding to the sample question to be answered is counted; the answer entity corresponding to the sample question to be answered is used to characterize the entity of the answer corresponding to the sample question to be answered; the candidate focus entity sample with a jump number less than the preset number is determined as the target focus entity sample.
[0117] S203: Use the sample questions to be answered, historical question and answer samples, candidate focus entity samples as samples, and target focus entity samples as labels to train the focus entity prediction model.
[0118] Furthermore, the prior art generally uses a cross entropy loss function in the model training stage. However, as the number of rounds in the session increases, the proportion of target focus entity samples in the candidate focus entity samples is smaller, resulting in an unbalanced distribution of labels for positive and negative examples. This unbalanced data may cause serious deviations in training, thereby affecting the performance of the model. In order to alleviate this problem, the embodiment of the present application uses a weighted cross entropy loss function to deal with the imbalance of categories, so that the focus entity prediction model pays more attention to positive samples, thereby improving the performance of the focus entity prediction model for unbalanced data.
[0119] ;
[0120] Among them, loss is the loss value of the focus entity prediction model, represents the number of candidate focal entity samples, Indicates The probability distribution of candidate focal entity samples, is the first annotation through distant supervision pseudo labels of candidate focal entity samples, is the weight of the positive sample, is the weight of negative samples, and It is determined by the ratio of positive and negative samples.
[0121] An embodiment of the present application provides another large language model-driven conversational knowledge graph question-answering method, which is capable of training a focus entity prediction model.
[0122] Based on the same inventive concept, the embodiments of the present application also provide a conversational knowledge graph question and answer device driven by a large language model corresponding to the conversational knowledge graph question and answer method driven by a large language model. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the conversational knowledge graph question and answer method driven by a large language model in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0123] Reference Figure 3 As shown, a conversational knowledge graph question-answering device driven by a large language model is provided in an embodiment of the present application, and the device includes:
[0124] The acquisition module 301 is used to acquire the first candidate focus entity in the question to be answered in the current session, and the second candidate focus entity in the historical questions and answers before the question to be answered; the historical questions and answers include the questions and answers in each round of historical questions and answers;
[0125] An input module 302 is used to input the unanswered question, the historical question and answer, the first candidate focus entity and the second candidate focus entity into the focus entity prediction model to obtain a target focus entity;
[0126] A search module 303 is used to search for a target triplet similar to the question to be answered from all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph;
[0127] Among them, the similarity between the question to be answered and the target triplet is greater than the similarity between the question to be answered and any triplet in the initial triplet except the target triplet;
[0128] The answering module 304 is used to answer the question to be answered according to the target triplet.
[0129] In a possible implementation, the device also includes: a training module 305; a training module, specifically used to obtain candidate focus entity samples in the sample questions to be answered in the sample conversation, and candidate focus entity samples in the historical question and answer samples before the sample questions to be answered; determine the target focus entity samples from the candidate focus entity samples; use the sample questions to be answered, the historical question and answer samples, and the candidate focus entity samples as samples, and the target focus entity samples as labels, to train the focus entity prediction model.
[0130] In a possible implementation, the training module 305 is further configured to:
[0131] For each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the candidate focus entity sample is determined as the target focus entity sample.
[0132] In a possible implementation, the search module 303 is specifically configured to calculate the similarity between each initial triple and the question to be answered; and select a target triple from the initial triples according to the similarity between each initial triple and the question to be answered.
[0133] In a possible implementation manner, the search module 303 is further configured to:
[0134] A preset number of initial triples with the greatest similarity to the question to be answered are determined as target triples.
[0135] In a possible implementation manner, the search module 303 is further configured to:
[0136] An initial triplet whose similarity to the question to be answered is greater than a preset similarity is determined as a target triplet.
[0137] In one possible implementation, the response module 304 is specifically used to splice the preset text information, the target triple, and the question to be answered in a preset order, and input them into the response text generation model to obtain the response text of the question to be answered; and output the response text in the conversation interface of the current conversation.
[0138] An embodiment of the present application provides a conversational knowledge graph question-and-answer device driven by a large language model, the device comprising: an acquisition module 301, used to acquire a first candidate focus entity in a question to be answered in a current conversation, and a second candidate focus entity in a historical question and answer before the question to be answered; the historical question and answer include questions and answers in each round of historical question and answer; an input module 302, used to input the question to be answered, the historical question and answer, the first candidate focus entity and the second candidate focus entity into a focus entity prediction model to obtain a target focus entity; a search module 303, used to search for a target triple similar to the question to be answered from all initial triples corresponding to the target focus entity in a pre-established conversation knowledge graph; wherein the similarity between the question to be answered and the target triple is greater than the similarity between the question to be answered and any triple in the initial triples except the target triple; and a response module 304, used to respond to the question to be answered according to the target triple. This application determines the target focus entity from the first candidate focus entity in the question to be answered and the second candidate focus entity in the historical questions and answers, thereby avoiding the problem of knowledge injection failure and inability to complete the questions and answers in this round of conversation due to the missing or pronounization of the target focus entity.
[0139] like Figure 4 As shown, an electronic device 400 provided in an embodiment of the present application includes: a processor 401, a memory 402 and a bus, the memory 402 stores machine-readable instructions executable by the processor 401, and when the electronic device is running, the processor 401 communicates with the memory 402 through the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the conversational knowledge graph question-answering method driven by the large language model as described above.
[0140] Specifically, the above-mentioned memory 402 and processor 401 can be general-purpose memories and processors, which are not specifically limited here. When the processor 401 runs the computer program stored in the memory 402, it can execute the above-mentioned large language model-driven conversational knowledge graph question and answer method.
[0141] Corresponding to the above-mentioned large language model-driven conversational knowledge graph question-answering method, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned large language model-driven conversational knowledge graph question-answering method are executed.
[0142] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0143] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0144] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0145] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the information processing method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.
[0146] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A conversational knowledge graph question answering method driven by a large language model, characterized by: The method comprises: Obtaining a first candidate focus entity in the question to be answered in the current session, and a second candidate focus entity in the historical questions and answers before the question to be answered; the historical questions and answers include questions and answers in each round of historical questions and answers; Inputting the unanswered question, the historical questions and answers, the first candidate focus entity, and the second candidate focus entity into a focus entity prediction model to obtain a target focus entity; Searching for a target triple that is similar to the question to be answered from all initial triples corresponding to the target focus entity in a pre-established conversation knowledge graph; wherein the similarity between the question to be answered and the target triple is greater than the similarity between the question to be answered and any triple in the initial triple except the target triple; Answer the question to be answered according to the target triple; The first candidate focus entity and the second candidate focus entity are both candidate focus entities; the step of inputting the unanswered question, the historical questions and answers, the first candidate focus entity, and the second candidate focus entity into a focus entity prediction model to obtain a target focus entity includes: concatenating the unanswered question, the historical questions and answers, and the candidate focus entity to obtain a concatenated sequence context, which is expressed as: ;in, represents the splicing sequence, For the first round of history quiz, For splicing operation, is the answer to the first round of historical question answering, t is the round corresponding to the question to be answered, is the answer to the t-1 round of the history quiz, To answer questions, is the first candidate focus entity, is the nth candidate focus entity, and n is the number of candidate focus entities; the concatenated sequence context is input into the word segmentation encoding layer BERT to obtain a subword encoding sequence H, which includes the subword encoding of the question to be answered, the subword encoding of the historical question and answer, and the subword encoding of the candidate focus entity; the subword encoding of the candidate focus entity contains information from the question to be answered and the historical question and answer; the expression of H is: ;in, , The first subword code in the subword code of the question to be answered and the subword code of the historical question answering, is the kth subword encoding in the subword encoding of the question to be answered and the subword encoding of the historical question and answer, is the total number of subword codes for questions to be answered and subword codes for historical questions and answers, The first candidate focus entity The first subword encoding of is the i-th candidate focus entity The j-th subword encoding of is the nth candidate focus entity The mth subword encoding of , n is the number of candidate focus entities, and m is the nth candidate focus entity The number of subword codes of the candidate focus entity in the subword code sequence H is input into the word segmentation vectorization layer to vectorize the subword codes of each candidate focus entity to obtain the subword vectors of each candidate focus entity; the vectorized expression is: ;in, is the i-th candidate focus entity The j-th subword vector of is layer normalization, is the i-th candidate focus entity The j-th subword encoding of ; all subword vectors corresponding to each candidate focus entity are input into the pooling layer for pooling to obtain each candidate focus entity vector; the pooling layer is represented as: ;in, is the i-th candidate focus entity The candidate focus entity vector of For pooling, is the i-th candidate focus entity The j-th subword vector of ; input each candidate focus entity vector into the following formula to obtain the probability of each candidate focus entity: ;in, is the probability of the i-th candidate focal entity, is the activation function, , b are model parameters, is the i-th candidate focus entity The candidate focus entity vector of , T is the transpose; the candidate focus entity with a probability greater than a preset value is determined as the target focus entity; The focus entity prediction model is trained by the following steps: obtaining candidate focus entity samples in the sample questions to be answered in the sample conversation, and candidate focus entity samples in the historical question and answer samples before the sample questions to be answered; determining the target focus entity samples from the candidate focus entity samples, including: for each of the candidate focus entity samples, if in the pre-established conversation knowledge graph, the answer entity corresponding to the sample question to be answered can be linked from the candidate focus entity sample, then counting the number of jumps from the candidate focus entity sample to the answer entity corresponding to the sample question to be answered; the answer entity corresponding to the sample question to be answered is used to characterize the entity in the answer corresponding to the sample question to be answered; the candidate focus entity sample with the number of jumps less than the preset number is determined as the target focus entity sample; the sample question to be answered, the historical question and answer samples, the candidate focus entity samples are used as samples, and the target focus entity samples are used as labels to train the focus entity prediction model.
2. The conversational knowledge graph question answering method driven by a large language model according to claim 1 is characterized in that: The determining of a target focus entity sample from the candidate focus entity samples includes: For each candidate focus entity sample, if in the pre-established conversation knowledge graph, the candidate focus entity sample can be linked to the answer entity corresponding to the sample question to be answered, then the candidate focus entity sample is determined as the target focus entity sample.
3. The conversational knowledge graph question answering method driven by a large language model according to claim 1 is characterized in that: The step of searching for a target triple that is similar to the question to be answered from all initial triples corresponding to the target focus entity in the pre-established conversation knowledge graph includes: Calculating the similarity between each initial triple and the question to be answered; A target triple is selected from the initial triples according to the similarity between each initial triple and the question to be answered.
4. The conversational knowledge graph question answering method driven by a large language model according to claim 3 is characterized in that: The selecting a target triple from the initial triples according to the similarity between each initial triple and the question to be answered includes: A preset number of initial triples with the greatest similarity to the question to be answered are determined as target triples.
5. The conversational knowledge graph question answering method driven by a large language model according to claim 3 is characterized in that: The selecting a target triple from the initial triples according to the similarity between each initial triple and the question to be answered further comprises: An initial triplet whose similarity to the question to be answered is greater than a preset similarity is determined as a target triplet.
6. The conversational knowledge graph question answering method driven by a large language model according to claim 1 is characterized in that: The step of answering the unanswered question according to the target triplet includes: After splicing the preset text information, the target triple, and the question to be answered in a preset order, the splicing is input into the answer text generation model to obtain the answer text of the question to be answered; The response text is output on the session interface of the current session.
7. A conversational knowledge graph question-answering device driven by a large language model, characterized in that: The device comprises: An acquisition module is configured to acquire a first candidate focus entity in a question to be answered in a current session, and a second candidate focus entity in a historical question and answer session before the question to be answered; the historical question and answer session includes questions and answers in each round of historical question and answer sessions; An input module, configured to input the question to be answered, the historical questions and answers, the first candidate focus entity, and the second candidate focus entity into a focus entity prediction model to obtain a target focus entity; A search module, configured to search for a target triple similar to the question to be answered from all initial triples corresponding to the target focus entity in a pre-established conversation knowledge graph; wherein the similarity between the question to be answered and the target triple is greater than the similarity between the question to be answered and any triple in the initial triple except the target triple; An answering module, configured to answer the question to be answered based on the target triple; The first candidate focus entity and the second candidate focus are both candidate focus entities; the input module is specifically used to splice the unanswered question, the historical questions and answers, and the candidate focus entity to obtain a spliced sequence context, which is expressed as: ;in, represents the splicing sequence, For the first round of history quiz, For splicing operation, is the answer to the first round of historical question answering, t is the round corresponding to the question to be answered, is the answer to the t-1 round of the history quiz, To answer questions, is the first candidate focus entity, is the nth candidate focus entity, and n is the number of candidate focus entities; the concatenated sequence context is input into the word segmentation encoding layer BERT to obtain a subword encoding sequence H, which includes the subword encoding of the question to be answered, the subword encoding of the historical question and answer, and the subword encoding of the candidate focus entity; the subword encoding of the candidate focus entity contains information from the question to be answered and the historical question and answer; the expression of H is: ;in, , The first subword code in the subword code of the question to be answered and the subword code of the historical question answering, is the kth subword encoding in the subword encoding of the question to be answered and the subword encoding of the historical question and answer, is the total number of subword codes for questions to be answered and subword codes for historical questions and answers, The first candidate focus entity The first subword encoding of is the i-th candidate focus entity The j-th subword encoding of is the nth candidate focus entity The mth subword encoding of , n is the number of candidate focus entities, and m is the nth candidate focus entity The number of subword codes of the candidate focus entity in the subword code sequence H is input into the word segmentation vectorization layer to vectorize the subword codes of each candidate focus entity to obtain the subword vectors of each candidate focus entity; the vectorized expression is: ;in, is the i-th candidate focus entity The j-th subword vector of is layer normalization, is the i-th candidate focus entity The j-th subword encoding of ; all subword vectors corresponding to each candidate focus entity are input into the pooling layer for pooling to obtain each candidate focus entity vector; the pooling layer is represented as: ;in, is the i-th candidate focus entity The candidate focus entity vector of For pooling, is the i-th candidate focus entity The j-th subword vector of ; input each candidate focus entity vector into the following formula to obtain the probability of each candidate focus entity: ;in, is the probability of the i-th candidate focal entity, is the activation function, , b are model parameters, is the i-th candidate focus entity The candidate focus entity vector of , T is the transpose; the candidate focus entity with a probability greater than a preset value is determined as the target focus entity; A training module is specifically used to obtain candidate focus entity samples in the sample questions to be answered in the sample conversation, and candidate focus entity samples in the historical question and answer samples before the sample questions to be answered; determine the target focus entity samples from the candidate focus entity samples, including: for each of the candidate focus entity samples, if in the pre-established conversation knowledge graph, the answer entity corresponding to the sample question to be answered can be linked from the candidate focus entity sample, then count the number of jumps from the candidate focus entity sample to the answer entity corresponding to the sample question to be answered; the answer entity corresponding to the sample question to be answered is used to characterize the entity in the answer corresponding to the sample question to be answered; the candidate focus entity sample with the number of jumps less than a preset number is determined as the target focus entity sample; the sample question to be answered, the historical question and answer samples, the candidate focus entity samples are used as samples, and the target focus entity samples are used as labels to train the focus entity prediction model.
8. An electronic device, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the large language model-driven conversational knowledge graph question-answering method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the conversational knowledge graph question-answering method driven by a large language model as described in any one of claims 1 to 6.
