Knowledge rewriting method for complex knowledge graph question and answer task
Through the knowledge rewriting method combined with thinking chain and supervision training, the problems of redundancy and information loss in the knowledge graph question-and-answer task are solved, and knowledge representation forms that conform to the semantics of the question are generated, which improves the performance and user experience of the question-and-answer model.
Patent Information
- Application Number
- CN202411177700.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-26
AI Technical Summary
The existing knowledge rewriting methods have redundancy and information loss in complex knowledge graph Q&A tasks, resulting in the generated knowledge representation mismatch with the semantics of the problem, affecting the accuracy and logic of the Q&A model.
Using thinking chains to enhance the comprehensiveness and logic of knowledge representation forms, through an open source big model training framework with supervised training and preference fine-tuning, design knowledge rewriting methods for complex knowledge graph Q&A tasks, including sub-graph retrieval, knowledge rewriting, quality evaluation, and candidates Knowledge representation form sampling, preference annotation and DPO algorithm fine-tuning to generate natural language text that conforms to the semantics of the problem.
It effectively filters information that is not related to the semantics of the problem, reduces noise and inference costs, and the generated knowledge representation is clear and logical, which significantly improves the performance and user experience of the question-and-answer model, ensuring the accuracy and logic of the answers.
Smart Images

Figure CN119025683A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of natural language processing and relates to a knowledge rewriting method for complex knowledge graph question-answering tasks. Background Art
[0002] Large language models have achieved remarkable results on multiple natural language processing tasks and are an important milestone in this field. Although they have performed well in zero-shot scenarios, they still have factual errors, that is, hallucination problems, when faced with fact-intensive tasks, especially for question-answering tasks. Some recent retrieval enhancement works have attempted to use knowledge graphs as knowledge sources to improve the capabilities of large language models in knowledge graph question answering tasks (KGQA). Unlike general question-answering tasks, the key challenge of KGQA under the RAG paradigm is that the subgraphs related to the question need to be converted into natural language text that can be understood by the large language model as context information, and the structural information of the subgraph should be preserved as much as possible, that is, knowledge rewriting. Previous works usually simply concatenate the subject, predicate and object of the triple to form a text in the form of triples. In addition, considering that the large language model has been pre-trained on a large text corpus and has difficulty understanding structured triple text, other works are committed to converting triples into natural language text using KG-to-Text, or summarizing question-related knowledge from triples to generate question-related knowledge descriptions.
[0003] Although existing knowledge rewriting methods have been proven to be effective, they still have the following problems: (1) The rewritten knowledge is redundant or lost, which causes the generated knowledge representation to face usability challenges in actual knowledge graph question-answering applications. Existing knowledge rewriting methods convert question-related subgraphs into natural language text using simple linear concatenation or KG-to-Text methods, thereby generating redundant information that is irrelevant to the question semantics. This redundant information may interfere with the question-answering model, introduce irrelevant information, and affect the user experience. In addition, other works based on question and retrieval subgraphs have been Figure 1 The summary is generated in the first step. This approach lacks fine-grained decomposition and analysis of the question, which makes it easy to lose key information when dealing with complex questions that require a lot of contextual knowledge. This results in the rewritten knowledge representation lacking the key knowledge points required to answer the question, making it difficult for the question-answering model to provide users with satisfactory answers. (2) The generated knowledge representation does not match the semantics of the question. Existing work ignores the semantics of the question and lacks logical organization that matches the reasoning path of the question. This not only affects the question-answering model's ability to effectively use relevant contextual knowledge to provide users with accurate and concise answers, but also causes the answers generated by the model to lack the necessary logic and orderliness, increasing users' confusion and incomprehension about the answers.
[0004] Based on this, the thinking chain is used to improve the performance of the knowledge rewriting task, and a knowledge rewriting method for complex knowledge graph question-answering tasks is designed based on this strategy. The core of the knowledge rewriter is to alternately reason and summarize for a given question, and use the thinking chain to enhance the comprehensiveness and logic of the knowledge representation. In the reasoning step, the knowledge rewriter disassembles the problem and indicates the knowledge required for this step of reasoning. In the summarizing step, the knowledge rewriter summarizes the corresponding knowledge from the retrieved triples based on the content of the reasoning. In this way, the knowledge rewriting method for complex knowledge graph question-answering tasks filters out information that is irrelevant to the semantics of the question, reduces the noise and increase in reasoning cost caused by excessive contextual information. At the same time, it can generate natural language text that is clear and logically consistent with the semantics of the question. In addition, an open source large model training framework containing supervised training and preference fine-tuning is proposed to implement the knowledge rewriting method for complex knowledge graph question-answering tasks. The supervised training mainly uses a closed-source large model with strong performance to generate a reference knowledge representation to supervise the training of the open source large model so that it can initially master the ability of knowledge rewriting. Preference fine-tuning samples candidate knowledge representations from the open source large model after supervised training, builds a preference dataset, and finally further fine-tunes the open source large model through the DPO algorithm. Through the above two stages of training, a knowledge rewriter can be obtained that can generate contextual knowledge that meets the preferences of the question-answering model. Summary of the invention
[0005] In order to address the deficiencies in the prior art, the purpose of the present invention is to design a knowledge rewriting method for complex knowledge graph question answering tasks, so as to improve the knowledge representation of knowledge graph question answering tasks.
[0006] In order to achieve the above object, the solution of the present invention is as follows: a knowledge rewriting method for complex knowledge graph question answering tasks, the method comprising the following steps:
[0007] Step 1: Construct a supervised training dataset based on GLM-4. For question q and answer entity a, first retrieve the subgraph G related to the question, then use GLM-4 to rewrite the knowledge of the subgraph G to generate a reference knowledge representation for subsequent supervised training. The process mainly includes the following three stages:
[0008] Subgraph retrieval: First, extract the two-hop subgraph of the head entity corresponding to question q as the preliminary retrieval result. Then, calculate the semantic similarity between the triples in the subgraph and the question, and select the triples whose semantic similarity with the question exceeds the preset threshold as the result of the subgraph retrieval.
[0009] Knowledge rewriting: Question q and subgraph G are concatenated using a prompt template as the input x of the knowledge rewriting task. Then, x is input into GLM-4 to obtain the rewritten knowledge representation y. In order to improve the quality of the knowledge representation y and ensure that it conforms to the expected format, three examples are written as examples to guide GLM-4 to generate better knowledge representations under the paradigm of contextual learning.
[0010] Quality assessment: The knowledge representation form y is used as the contextual knowledge to answer question q, and the answer r is generated through the question-answering model. If the answer r contains all the answer entities a, the question q is considered to be answered correctly, and the input-output pair (x, y) is included in the supervised training data set. Otherwise, the knowledge representation form is judged to be unhelpful to the question-answering model and the data is discarded. Finally, the supervised training data set D is constructed. T ={(x1,y1),(x2,y2),…,(x T ,y T )}.
[0011] Step 2: Supervise the training of the open source large model based on the constructed training dataset.
[0012] In the training data set D T For each pair of input and output (x i ,y i ), open source large model R θ is trained based on the input x i Generate the corresponding output y i , the objective function used in the training is defined as:
[0013]
[0014] Among them, θ represents the open source large model R θ The parameter, p θ (y i |x i ) means that given an input x i Under the conditions of open source large model R θ Generate output y i Through the training at this stage, the open source big model has initially mastered the ability to rewrite knowledge.
[0015] Step 3: Sample candidate knowledge representations from the open source large model after supervised training.
[0016] The question q and the subgraph G are concatenated using the prompt template as input x, and the supervised training open source large model R θ Sample M candidate knowledge representations, y1, y2, ..., y M :
[0017] yi =R θ (x)i=1,2,...,M
[0018] Step 4: Preference annotation of candidate knowledge representations based on the feedback of the question-answering model. First, from the M candidate knowledge representations, the semantic similarity between each other is calculated, and the two candidate knowledge representations y1 and y2 with the lowest semantic similarity are selected. Then, these two candidate knowledge representations are used as the context of the question-answering task to obtain the corresponding answers r1 and r2. Finally, by evaluating the pros and cons of answers r1 and r2, preference annotation is performed on y1 and y2. This process mainly includes the following three stages:
[0019] Semantic similarity calculation: For the M candidate knowledge representation forms, the semantic similarity between them is calculated pairwise, and the two candidate knowledge representation forms y1 and y2 with the largest semantic gap are selected. The reason for designing this stage is that the knowledge rewriter after supervised training usually generates candidate knowledge representation forms with similar semantics, resulting in a small difference between the preferred and non-preferred knowledge representation forms in the constructed dataset, thus affecting the effect of preference fine-tuning.
[0020] Answer reasoning: The feedback from the question-answering model is used to label preferences, thereby avoiding direct evaluation of the two knowledge representations, which may lead to differences in preferences with the question-answering model. In this stage, the two knowledge representations y1 and y2 are used as contextual knowledge for the question-answering task, and the answers r1 and r2 of the question-answering model are obtained.
[0021] Preference labeling: GLM-4 is used to evaluate the quality of answers r1 and r2 from the two dimensions of accuracy and relevance. Specifically, question q, subgraph G, answer entity a, and answers r1 and r2 of the question-answering model are spliced using a prompt template and input into GLM-4 to select the answer with higher quality. + The corresponding knowledge representation is used as the preferred knowledge representation y + , low quality answers - The corresponding knowledge representation is the non-preferred knowledge representation y - .
[0022] Step 5: Use GLM-4 to rewrite the preferred knowledge representation to improve the quality and diversity of the data.
[0023] Knowledge representation for preferences y +, GLM-4 is used to rewrite it to improve the quality and diversity of the dataset. Different from the knowledge rewriting task, when GLM-4 is used for knowledge rewriting, not only the question q and the subgraph G are provided, but also the answer entity a is given. This allows GLM-4 to consider not only the question and the related subgraphs during the rewriting process, but also organize the related knowledge around the answer entity of the question, thereby effectively improving the accuracy of the rewritten knowledge representation and significantly reducing the redundancy of information. Based on this, GLM-4 represents the preferred knowledge y + Rewrite into a better knowledge representation form y ++ . ++ ,y - ) as a preference data pair and construct a preference data set
[0024] Step 6: Use preference data to fine-tune the open source large model based on the DPO algorithm.
[0025] The Direct Preference Optimization (DPO) algorithm is used to further fine-tune the open source large model R θ , to obtain the knowledge rewriter after preference alignment It aims to minimize the following objective function:
[0026]
[0027] Among them, σ represents the Sigmoid function, p θ (y i |x i ) represents the knowledge rewriter R after supervised training θ Based on the input x i Generate y i The probability of Knowledge rewriter after fine-tuning representative preferences Based on the input x i Generate y i After DPO fine-tuning, the knowledge rewriter Tends to generate knowledge representations k that are more beneficial to question answering models ++ , while avoiding the generation of knowledge representations k that are not beneficial to the question answering model -, Given that different question-answering models have different preferences for contextual knowledge, when fine-tuning preferences, it is recommended to build different preference fine-tuning datasets for different question-answering models as much as possible, and use them to fine-tune the open source large model to align it with the preferences of a specific question-answering model. Through preference fine-tuning, the knowledge rewriter converts the subgraph related to the question into a comprehensive and comprehensive knowledge summary that is consistent with the question logic. This knowledge summary is used as the context of the question, which significantly improves the performance of the large model in processing complex knowledge graph question-answering tasks.
[0028] Compared with the prior art, the advantages of the present invention are as follows:
[0029] 1. This solution proposes a knowledge rewriting method for complex knowledge graph question-answering tasks, aiming to optimize the relevant knowledge representation of problems in complex knowledge graph question-answering tasks. This method not only handles simple problems solved by a single fact, but also can deal with complex problems that require the participation of multiple facts. The present invention can effectively organize and rewrite the complex factual knowledge involved in actual application scenarios and convert it into a knowledge representation that is easy to handle for downstream question-answering models. In addition, this method can be widely used in existing retrieval-enhanced knowledge graph question-answering frameworks, and can rewrite the retrieved question-related subgraphs into comprehensive knowledge summaries that are consistent with the semantics of the question. This rewriting process significantly improves the question-answering model's ability to solve various problems in actual application scenarios, and enhances the practicality of the question-answering system in real-world scenarios;
[0030] 2. The solution designs a training framework that includes two stages: supervised training and preference fine-tuning. First, the training framework enables the open source big model to initially master the ability of knowledge rewriting through supervised training. Then, through preference fine-tuning, the knowledge representation generated by the open source big model is aligned with the preference of the question-answering model. This training strategy shows significant practical value. Compared with the method that only relies on supervised fine-tuning, preference fine-tuning enhances the adaptability of the open source big model in actual application scenarios and optimizes the knowledge representation generated by the model to better serve the downstream question-answering system. In addition, through meticulous preference adjustment, the question-answering system can provide answers that meet user needs more accurately and ensure that the generated content meets human ethical standards. This not only improves the functionality of the system, but also strengthens its sense of social responsibility. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a flow chart of the knowledge rewriting method for complex knowledge graph question answering tasks. Figure 2 Input and output examples of the knowledge rewriting method for complex knowledge graph question answering tasks. DETAILED DESCRIPTION
[0032] The present application is further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present application.
[0033] Example 1: The knowledge rewriting method proposed by the present invention for complex knowledge graph question answering tasks, such as Figure 1 As shown in the figure, it mainly includes two parts: supervised training and preference fine-tuning. The supervised training aims to use the closed-source big model to construct a supervised training corpus for supervised training of the open-source big model, so that it can initially master the ability of knowledge rewriting. Afterwards, in view of the preference difference between the open-source big model after supervised training and the question-answering model, candidate knowledge representation forms are sampled from the open-source big model after supervised training to construct a preference data set. Subsequently, the open-source big model is further fine-tuned through the DPO algorithm, and finally a knowledge rewriter that has both knowledge rewriting capabilities and is consistent with the question-answering model preferences is realized. It includes the following steps:
[0034] Step 1: Construct a supervised training dataset based on GLM-4. For question q and answer entity a, first retrieve the subgraph G related to the question, then use GLM-4 to rewrite the knowledge of the subgraph G to generate a reference knowledge representation for subsequent supervised training. The process mainly includes the following three stages:
[0035] Subgraph retrieval: First, extract the two-hop subgraph of the head entity corresponding to question q as the preliminary retrieval result. Then, calculate the semantic similarity between the triples in the subgraph and the question, and select the triples whose semantic similarity with the question exceeds the preset threshold as the result of the subgraph retrieval.
[0036] Knowledge rewriting: Question q and subgraph G are concatenated using a prompt template as the input x of the knowledge rewriting task. Then, x is input into GLM-4 to obtain the rewritten knowledge representation y. In order to improve the quality of the knowledge representation y and ensure that it conforms to the expected format, three examples are written as examples to guide GLM-4 to generate better knowledge representations under the paradigm of contextual learning.
[0037] Quality assessment: The knowledge representation form y is used as the contextual knowledge to answer question q, and the answer r is generated through the question-answering model. If the answer r contains all the answer entities a, the question q is considered to be answered correctly, and the input-output pair (x, y) is included in the supervised training data set. Otherwise, the knowledge representation form is judged to be unhelpful to the question-answering model and the data is discarded. Finally, the supervised training data set D is constructed. T ={(x1,y1),(x2,y2),…,(x T ,y T )}.
[0038] Step 2: Supervise the training of the open source large model based on the constructed training dataset.
[0039] In the training data set D T For each pair of input and output (x i ,y i ), open source large model R θ is trained based on the input x i Generate the corresponding output y i , the objective function used in the training is defined as:
[0040]
[0041] Among them, θ represents the open source large model R θ The parameter, p θ (y i |x i ) means that given an input x i Under the conditions of open source large model R θ Generate output y i Through the training at this stage, the open source big model has initially mastered the ability to rewrite knowledge.
[0042] Step 3: Sample candidate knowledge representations from the open source large model after supervised training.
[0043] The question q and the subgraph G are concatenated using the prompt template as input x, and the supervised training open source large model R θ Sample M candidate knowledge representations, y1, y2, ..., y M :
[0044] y i =R θ (x)i=1,2,...,M
[0045] Step 4: Preference annotation of candidate knowledge representations based on the feedback of the question-answering model. First, from the M candidate knowledge representations, the semantic similarity between each other is calculated, and the two candidate knowledge representations y1 and y2 with the lowest semantic similarity are selected. Then, these two candidate knowledge representations are used as the context of the question-answering task to obtain the corresponding answers r1 and r2. Finally, by evaluating the pros and cons of answers r1 and r2, preference annotation is performed on y1 and y2. This process mainly includes the following three stages:
[0046] Semantic similarity calculation: For the M candidate knowledge representation forms, the semantic similarity between them is calculated pairwise, and the two candidate knowledge representation forms y1 and y2 with the largest semantic gap are selected. The reason for designing this stage is that the knowledge rewriter after supervised training usually generates candidate knowledge representation forms with similar semantics, resulting in a small difference between the preferred and non-preferred knowledge representation forms in the constructed dataset, thus affecting the effect of preference fine-tuning.
[0047] Answer reasoning: The feedback from the question-answering model is used to label preferences, thereby avoiding direct evaluation of the two knowledge representations, which may lead to differences in preferences with the question-answering model. In this stage, the two knowledge representations y1 and y2 are used as contextual knowledge for the question-answering task, and the answers r1 and r2 of the question-answering model are obtained.
[0048] Preference labeling: GLM-4 is used to evaluate the quality of answers r1 and r2 from the two dimensions of accuracy and relevance. Specifically, question q, subgraph G, answer entity a, and answers r1 and r2 of the question-answering model are spliced using a prompt template and input into GLM-4 to select the answer with higher quality. + The corresponding knowledge representation is used as the preferred knowledge representation y + , low quality answers - The corresponding knowledge representation is the non-preferred knowledge representation y - .
[0049] Step 5: Use GLM-4 to rewrite the preferred knowledge representation to improve the quality and diversity of the data.
[0050] Knowledge representation for preferences y + , GLM-4 is used to rewrite it to improve the quality and diversity of the dataset. Different from the knowledge rewriting task, when GLM-4 is used for knowledge rewriting, not only the question q and the subgraph G are provided, but also the answer entity a is given. This allows GLM-4 to consider not only the question and the related subgraphs during the rewriting process, but also organize the related knowledge around the answer entity of the question, thereby effectively improving the accuracy of the rewritten knowledge representation and significantly reducing the redundancy of information. Based on this, GLM-4 represents the preferred knowledge y + Rewrite into a better knowledge representation form y ++ . ++ ,y - ) as a preference data pair and construct a preference data set
[0051] Step 6: Use preference data to fine-tune the open source large model based on the DPO algorithm.
[0052] The Direct Preference Optimization (DPO) algorithm is used to further fine-tune the open source large model R θ , to obtain the knowledge rewriter R after preference alignment θ* , which aims to minimize the following objective function:
[0053]
[0054] Among them, σ represents the Sigmoid function, p θ (y i |x i ) represents the knowledge rewriter R after supervised training θ Based on the input x i Generate y i The probability of Knowledge rewriter after fine-tuning representative preferences Based on the input x i Generate y i After DPO fine-tuning, the knowledge rewriter Tends to generate knowledge representations k that are more beneficial to question answering models ++ , while avoiding the generation of knowledge representations k that are not beneficial to the question answering model - , given that different question-answering models have different preferences for contextual knowledge, when performing preference fine-tuning, it is recommended to build different preference fine-tuning datasets for different question-answering models as much as possible, which are used to fine-tune the open source large model to align it with the preferences of a specific question-answering model. Through preference fine-tuning, the knowledge rewriter converts the subgraph related to the question into a comprehensive and comprehensive knowledge summary that is consistent with the question logic. This knowledge summary is used as the context of the question, which significantly improves the performance of the large model in processing complex knowledge graph question-answering tasks.
[0055] Example 2: Healthcare is a key application area of knowledge graph question answering system in real life. The system uses the fact triples related to the question in the knowledge graph to provide users with accurate and concise answers. This method has application potential in the healthcare field and can support large language models to provide high-quality services to users.
[0056] Step 1: Use GLM-4 as a corpus generator to construct training corpus for the knowledge rewriting task based on the medical knowledge graph and question-answering dataset.
[0057] For question q and the corresponding head entity e, first retrieve the triples related to question q in the two-hop subgraph around the head entity e from the domain knowledge graph to form a subgraph G. Specifically, taking the question "What is the relationship between retinopathy and diabetes, and how to avoid it through treatment?" as an example, first determine the head entity of the question as "diabetes", and then retrieve the two-hop subgraph around the entity in the knowledge graph. In this process, the triples related to the semantics of the question are screened out to form a subgraph G, including the following triples: (diabetes, common symptoms, high blood sugar), (diabetes, common symptoms, frequent urination), (diabetes, potential complications, retinopathy), (diabetes, potential complications, nephropathy), (retinopathy, treatment methods, laser photocoagulation), (nephropathy, treatment methods, hemodialysis), (high blood sugar, treatment methods, insulin injection), (frequent urination, treatment methods, behavioral therapy).
[0058] Then, concatenate the question q and the retrieved subgraph G as input x. Take x as the input of the knowledge rewriting task, use GLM-4 to perform knowledge rewriting, and obtain the corresponding knowledge summary y. For example, for the above example, the retrieved subgraph G can be rewritten as "Reasoning 1: I need to know the relationship between diabetes and retinopathy. Summary 1: Potential complications of diabetes include retinopathy. Reasoning 2: I need to know the treatment of retinopathy. Summary 2: Retinopathy can be treated with laser photocoagulation." (x, y) as the input and output pair of the knowledge rewriting task.
[0059] Step 2: supervise the training of the knowledge rewriter based on the constructed training corpus. After obtaining a certain amount of training data, use the open source large language model for supervised training. For example, use llama-3-chinese-8b-instruct-v3 as the basic model for supervised fine-tuning to cultivate the model's initial adaptability and execution capabilities for knowledge rewriting tasks. Through this process, the model has mastered basic knowledge rewriting skills, laying the foundation for subsequent more in-depth function development and optimization.
[0060] Step 3, construction of preference fine-tuning dataset.
[0061] In order to ensure that the knowledge rewrite matches the preferences of the question-answering model in the actual application scenario and facilitate the application of the question-answering model, this step constructs a dataset for preference fine-tuning. For the knowledge rewriter trained in step 2, different outputs for the same input are sampled. For example, for the question "What is the relationship between retinopathy and diabetes, and how can it be avoided through treatment?" and its corresponding subgraph, the supervised knowledge rewriter can generate multiple expressions. For example, knowledge representation form 1 ("Reasoning 1: I need to know the relationship between diabetes and retinopathy. Summary 1: Potential complications of diabetes include retinopathy. Reasoning 2: I need to know the treatment of retinopathy. Summary 2: Laser photocoagulation can treat retinopathy.") and knowledge representation form 2 ("Reasoning 1: I need to know what complications of diabetes are. Summary 1: Complications of diabetes include retinopathy and nephropathy. Reasoning 2: How can these complications be treated. Summary 2: Retinopathy can be treated by laser photocoagulation, and nephropathy can be treated by hemodialysis."). Then, the rewritten knowledge representation form is used as the context for answering the question to obtain the answer to the question. For example, for the above two knowledge representation forms, two answers can be obtained respectively: Answer 1 ("There is a potential complication relationship between diabetes and retinopathy, which can be treated by laser photocoagulation.") and Answer 2 ("Potential complications of diabetes include retinopathy and nephropathy. Among them, retinopathy can be treated by laser photocoagulation, and nephropathy can be treated by hemodialysis."). For the above two answers, compared with the standard answer "There is a potential complication relationship between retinopathy and diabetes, which can be avoided by laser photocoagulation.", the knowledge representation form 1 corresponding to answer 1 is considered to be a better representation form, while the knowledge representation form 2 corresponding to answer 2 is considered to be a worse representation form.
[0062] Step 4: Use GLM-4 to rewrite the better knowledge representation to improve the quality of the knowledge representation.
[0063] In the above example, GLM-4 is used to rewrite the better knowledge representation to improve the quality. For example, the better knowledge representation above is rewritten as "Reasoning 1: I need to know the relationship between diabetes and retinopathy. Conclusion 1: There is a potential complication relationship between diabetes and retinopathy. Reasoning 2: I need to know the treatment of retinopathy. Conclusion 2: Retinopathy can be treated with laser photocoagulation."
[0064] Step 5: Use the collected preference fine-tuning dataset to fine-tune the open source large model based on the DPO algorithm to align with the preferences of the downstream question-answering model.
[0065] Through the above steps, a thought chain enhanced knowledge rewriter can be obtained. Compared with the traditional knowledge rewriting method, which simply converts the triples related to the question into text, the present invention can rewrite the triples related to the question into a comprehensive and comprehensive knowledge summary that conforms to the semantics of the question. This rewriting significantly improves the performance of the question-answering model. Especially in the field of medical health, the proposed knowledge rewriting method can effectively convert the medical field triples related to the question into knowledge summaries related to the question, making it easier for the question-answering model to provide users with more accurate and refined answers, thereby enhancing the practicality of the question-answering system in the field of medical health.
[0066] Example 3: In the field of e-commerce, the application of knowledge graph question-answering systems is completely changing consumers' shopping experience. By integrating and structuring product data, user reviews, supply chain information and other knowledge, knowledge graphs enable consumers to quickly obtain accurate product information and recommendations through natural language queries. This method can be applied in the field of e-commerce to provide consumers with information about products and enhance their shopping experience.
[0067] Step 1: Based on consumer demand and interaction data, build a knowledge graph and question-answering dataset for the e-commerce field. The e-commerce field contains a wide range of product categories and user evaluation information, and these data sources enrich the content of the knowledge graph. According to the characteristics of different application scenarios and specific user needs, custom-built knowledge graphs and question-answering datasets are designed to improve consumer experience and satisfaction through accurate information services.
[0068] Step 2: Based on the knowledge graph and question-answering dataset in the e-commerce field, construct the supervised training corpus for the knowledge rewriting task.
[0069] For question q and its corresponding head entity e, we first retrieve the two-hop subgraph of head entity e from the domain knowledge graph, and select the triples related to question q to construct subgraph G. For example, for question “iPhon e 12Who is the founder of the manufacturer? ", retrieve the head entity "iPhon e12". Then, the triplets related to the question semantics are screened out to form a subgraph G, including: (iPhone 12, manufacturer, Apple), (iPhone 12, screen size, 6.1 inches), (iPhone 12, operating system, iOS14), (Apple, founder, Steve Jobs), (Apple, founding date, April 1, 1976), (Apple, headquarters location, Cupertino, California). The question q and the subgraph G are combined as input x and input into a closed-source large language model (such as GLM-4) to obtain the rewritten knowledge representation y. For example, for the above question, the knowledge representation rewritten by the large language model is: "Reasoning 1: I need to know the manufacturer of iPhone 12. Summary 1: The manufacturer of iPhone 12 is Apple. Reasoning 2: I need to know who is the founder of Apple. Summary 2: The founder of Apple is Steve Jobs. ". (x, y) is used as the training corpus for the knowledge rewriting task, which is specifically used for knowledge rewriting tasks in the e-commerce field.
[0070] Step 3: supervised training of the open source large language model. The open source large language model is supervised and trained using the knowledge rewriting task dataset in the e-commerce field. The training process is designed to ensure that the model can initially master the skills of knowledge rewriting, laying the foundation for the subsequent preference fine-tuning step.
[0071] Step 4: Construction of preference fine-tuning dataset in the e-commerce field. Although supervised training has enabled the knowledge rewriter to initially master the technology of knowledge rewriting, the knowledge representation it generates has not yet been fully aligned with the preferences of the downstream question-answering model. In the e-commerce field, although the rewritten knowledge structure conforms to the predetermined format, the actual application of the content may be limited by the processing capacity of the question-answering model. Therefore, preference fine-tuning is used to bridge the preferences between the knowledge rewriter and the question-answering model, so that the generated knowledge representation can be fully adopted by the downstream question-answering model to provide consumers with more accurate answers. The specific operations include: First, for the same question and question-related subgraphs, multiple knowledge representations are sampled from the open source large language model completed by supervised fine-tuning. Then, for different knowledge representations, they are used as the context of the question-answering model to generate corresponding answers. After that, these answers are compared with the standard answers, and the knowledge representation corresponding to the higher-quality answers is used as the preferred knowledge representation, and the knowledge representation corresponding to the lower-quality answers is used as the non-preferred knowledge representation. Finally, a closed-source large language model (such as GLM-4) is used to rewrite the preferred knowledge representation to improve its quality and ensure the maximum fine-tuning effect and rapid convergence of the fine-tuning process.
[0072] Step 5: Use the DPO algorithm to further fine-tune the open source large language model based on the constructed preference fine-tuning dataset. The purpose of this step is to achieve preference alignment between the knowledge rewriter and the question-answering model to optimize the performance of the model in practical applications.
[0073] The present invention has significant advantages over traditional knowledge graph question-and-answer frameworks, especially in dealing with complex knowledge and user needs in the e-commerce field. First, the knowledge in the e-commerce field is complex and user needs are numerous. Traditional knowledge graph question-and-answer frameworks often find it difficult to comprehensively organize a large amount of factual knowledge related to user queries, thereby providing consumers with satisfactory answers. The knowledge rewriting method designed by the present invention is specifically designed for complex knowledge graph question-and-answer challenges. It can organize a large amount of factual knowledge related to questions into concise summaries that conform to the semantics of the questions, significantly improving the service quality of the question-and-answer system. (2) User queries sometimes involve common sense questions, while traditional knowledge graph question-and-answer frameworks are usually limited to factual knowledge in the knowledge graph. It is difficult to give accurate answers to questions that are beyond the scope of the knowledge graph, especially common sense questions. The knowledge rewriting method proposed in the present invention is suitable for a retrieval-enhanced knowledge graph question-and-answer framework. For questions that lack corresponding factual knowledge in the knowledge graph, the framework can use the knowledge in the large language model to provide users with more accurate and refined answers. (3) In view of the rapid update speed of knowledge in the e-commerce field, the corresponding knowledge graph also needs to be updated in real time. The knowledge rewriting method of the present invention has good generalization ability and can be directly applied to the updated knowledge graph without additional training. This feature shows that the present invention has strong practical value and broad application prospects.
[0074] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.
Claims
1. A knowledge rewriting method for complex knowledge graph question answering tasks, characterized by: The method comprises the following steps: Step 1: Use GLM-4 as a data generator to construct a dataset for supervised training. Step 2: Based on the constructed data set, supervised training is performed on the open source large model to enable it to initially master the ability to rewrite knowledge. Step 3: Sample multiple outputs of the same knowledge rewriting task from the supervised trained large model as candidate knowledge representations. Step 4: For the candidate knowledge representation forms, use them as the context of the question-answering task to obtain the answer corresponding to the question. By evaluating the advantages and disadvantages of the answers corresponding to different knowledge representation forms, the candidate knowledge representation forms are labeled with preferences and a preference dataset is constructed. Step 5: Use GLM-4 to rewrite the preferred knowledge representation to improve the quality and diversity of the data set. Step 6: Use the preference dataset to fine-tune the open source large model based on the DPO algorithm to align it with the preferences of the question-answering model.
2. The knowledge rewriting method for complex knowledge graph question answering tasks according to claim 1 is characterized in that: The step 1 is as follows: for question q and answer entity a, first retrieve the subgraph G related to the question, then use GLM-4 to rewrite the subgraph G to generate a reference knowledge representation for subsequent supervised training. The process mainly includes the following three stages: Subgraph retrieval: First, extract the two-hop subgraph of the head entity corresponding to question q as the preliminary retrieval result. Then, calculate the semantic similarity between the triples in the subgraph and the question, and select the triples whose semantic similarity with the question exceeds the preset threshold as the result of the subgraph retrieval. Knowledge rewriting: Question q and subgraph G are concatenated using a prompt template as the input x of the knowledge rewriting task. Then, x is input into GLM-4 to obtain the rewritten knowledge representation y. In order to improve the quality of the knowledge representation y and ensure that it conforms to the expected format, three examples are written as examples to guide GLM-4 to generate better knowledge representations under the paradigm of contextual learning. Quality assessment: The knowledge representation form y is used as the contextual knowledge to answer question q, and the answer r is generated through the question-answering model. If the answer r contains all the answer entities a, the question q is considered to be answered correctly, and the input-output pair (x, y) is included in the supervised training data set. Otherwise, the knowledge representation form is judged to be unhelpful to the question-answering model and the data is discarded. Finally, the supervised training data set D is constructed. T ={(x1,y1),(x2,y2),…,(x T ,y T )}.
3. The knowledge rewriting method for complex knowledge graph question answering tasks according to claim 1 is characterized in that: The step 2 is specifically as follows: In the training data set D T For each pair of input and output (x i ,y i ), open source large model R θ is trained based on the input x i Generate the corresponding output y i , the objective function used in the training is defined as: Among them, θ represents the open source large model R θ The parameter, p θ (y i |x i ) means that given an input x i Under the conditions of open source large model R θ Generate output y i Through the training at this stage, the open source big model has initially mastered the ability to rewrite knowledge.
4. The knowledge rewriting method for complex knowledge graph question answering tasks according to claim 1 is characterized in that: The step 3 is as follows: The question q and the subgraph G are concatenated using the prompt template as input x, and the supervised training open source large model R θ Sample M candidate knowledge representations, y1, y2, ..., y M : y i =R θ (x) i=1,2,...,M。 5. The knowledge rewriting method for complex knowledge graph question answering tasks according to claim 1 is characterized in that: The step 4 is as follows: First, the semantic similarity between the two candidate knowledge representations is calculated from the M candidate knowledge representations, and the two candidate knowledge representations y1 and y2 with the lowest semantic similarity are selected. Then, the two candidate knowledge representations are used as the context of the question-answering task to obtain the corresponding answers r1 and r2. Finally, by evaluating the pros and cons of the answers r1 and r2, preference annotation is performed on y1 and y2. The process mainly includes the following three stages: Semantic similarity calculation: For the M candidate knowledge representation forms, the semantic similarity between them is calculated pairwise, and the two candidate knowledge representation forms y1 and y2 with the largest semantic gap are selected. The reason for designing this stage is that the knowledge rewriter after supervised training usually generates candidate knowledge representation forms with similar semantics, resulting in a small difference between the preferred and non-preferred knowledge representation forms in the constructed dataset, thus affecting the effect of preference fine-tuning. Answer reasoning: The feedback from the question-answering model is used to label preferences, so as to avoid direct evaluation of the two knowledge representations, which may lead to differences in preferences with the question-answering model. In this stage, the two knowledge representations y1 and y2 are used as the contextual knowledge of the question-answering task, and the answers r1 and r2 of the question-answering model are obtained. Preference labeling: GLM-4 is used to evaluate the quality of answers r1 and r2 from the two dimensions of accuracy and relevance. Specifically, question q, subgraph G, answer entity a, and answers r1 and r2 of the question-answering model are spliced using a prompt template and input into GLM-4 to select the answer with higher quality. + The corresponding knowledge representation is used as the preferred knowledge representation y + , low quality answers - The corresponding knowledge representation is the non-preferred knowledge representation y - .
6. The knowledge rewriting method for complex knowledge graph question answering tasks according to claim 1 is characterized in that: The step 5 is specifically as follows: Knowledge representation for preferences y + , GLM-4 is used to rewrite it to improve the quality and diversity of the data set. When GLM-4 is used for knowledge rewriting, not only the question q and the subgraph G are provided, but also the answer entity a is given. This allows GLM-4 to not only consider the question and the related subgraphs during the rewriting process, but also organize the related knowledge around the answer entity of the question, thereby effectively improving the accuracy of the rewritten knowledge representation and significantly reducing the redundancy of information. Based on this, GLM-4 represents the preferred knowledge y + Rewrite into a better knowledge representation form y ++ , and (x, y ++ ,y - ) as a preference data pair and construct a preference data set 7. The knowledge rewriting method for complex knowledge graph question answering tasks according to claim 1 is characterized in that: The step 6 is specifically as follows: The Direct Preference Optimization (DPO) algorithm is used to further fine-tune the open source large model R θ , to obtain the knowledge rewriter after preference alignment It aims to minimize the following objective function L DPO : Among them, σ represents the Sigmoid function, p θ (y i |x i ) represents the knowledge rewriter R after supervised training θ Based on the input x i Generate y i The probability of Knowledge rewriter after fine-tuning representative preferences Based on the input x i Generate y i The probability of DPO fine-tuning is the knowledge rewriter Tends to generate knowledge representations k that are more beneficial to question answering models ++ , while avoiding the generation of knowledge representations k that are not beneficial to the question answering model - ,Given that different question and answer models have different preferences for contextual knowledge, when fine-tuning preferences, it is recommended to build different ,preference fine-tuning datasets for different question and answer models as much as possible, which can be used to fine-tune the open source large ,model to align it with the preferences of a specific question and answer model.
Citation Information
Patent Citations
Question rewriting method and device oriented to dialogue knowledge graph questions and answers
CN116127085A
Knowledge graph generation type question answering method and system based on large language model
CN117033608A
Care question and answer model training method based on transfer learning and expert feedback
CN117542471A
Cited By
Large language model prompt learning method and system for engineering emergency response
CN121365128A
