Program knowledge summary-based common sense question and answer method, system, equipment and medium
By summarizing the key points and thinking process of skill category problems in the common sense question and answer system, procedural knowledge is generated, and high cost and data leakage problems are solved in large language models when generating common sense knowledge, and efficient and low-cost common sense question and answer are achieved.
Patent Information
- Application Number
- CN202411185852.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-06
AI Technical Summary
When using large language models to generate common sense knowledge, existing common sense questions and answer methods have high usage costs and potential data leakage risks, and it is difficult to effectively summarize procedural knowledge to assist in answering common sense questions.
By extracting multiple questions in skill categories, using large language models to summarize and analyze key points and thinking processes, generate procedural knowledge at the key point level and process level, and map them into prefix forms to assist common sense question-and-answer model to answer questions. Reduce calls to large language models, reduce costs and avoid data breaches.
On the premise of ensuring the correctness of answers, the calls to large language models are significantly reduced, the cost of use is reduced, and the risk of data leakage is eliminated, while improving the performance and interpretability of common sense question and answer systems.
Smart Images

Figure CN119088926A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a common sense question-answering method, system, device and medium based on procedural knowledge summarization. Background Art
[0002] Commonsense Question Answering (CQA) is about letting the model answer questions that require common sense knowledge. Many studies have shown that introducing knowledge to help answer common sense questions can help improve the accuracy of answers, so it has attracted a lot of attention. At the same time, due to the demand for common sense knowledge in common sense question answering tasks, many methods and systems introduce external knowledge to help answer common sense questions.
[0003] Since large language models (LLMs) use a large amount of corpus in the pre-training process, a large amount of common sense knowledge is stored in the parameters of large language models, making them a good source of external common sense knowledge. In common sense question answering tasks, the mainstream approach to using large language models to provide common sense knowledge is to let the large language model generate relevant common sense knowledge for each question, thereby helping the answering model to answer common sense questions. One common approach focuses on improving the model's capabilities to generate more useful knowledge. For example, some scholars use examples to let large language models generate common sense knowledge related to multiple questions. Another common approach focuses on better selection and utilization of generated knowledge. For example, after a large language model generates common sense knowledge related to a question, some scholars alternately train the knowledge selection model and the question answering model to select more useful common sense knowledge to assist in answering questions.
[0004] However, this method of generating relevant common sense knowledge for each question has two shortcomings. First, the high cost of use. Since large language models need to generate common sense knowledge for each question, this requires a large number of calls to large language models, and the cost of using large language models is very high, making the previous method expensive to use. Second, the potential risk of data leakage, because large language models sometimes directly give predicted answers, which means that the answering model may give results based on the answers predicted by the large language model rather than based on knowledge, so there is a risk of data leakage.
[0005] In summary, in order to save costs, eliminate potential data leakage risks, achieve the goal of effectively summarizing procedural knowledge and helping to answer common sense questions, it is very necessary to design a common sense question answering method and system that can overcome the above challenges, which is of great significance to the development of this field.
[0006] In view of this, the present invention is proposed. Summary of the invention
[0007] The purpose of the present invention is to provide a common sense question and answer method, system, device and medium based on procedural knowledge summary, which can save usage costs and avoid the risk of data leakage while ensuring the correctness of answers.
[0008] The objective of the present invention is achieved through the following technical solutions:
[0009] A common sense question answering method based on procedural knowledge summarization, including:
[0010] For each skill category, a set number of questions are extracted from the data set; all the extracted questions are passed to the large language model, and the large language model compares and analyzes the correct and incorrect option explanations during the answering process to obtain the key points of answering questions for each skill category as procedural knowledge at the key point level; and the large language model summarizes and analyzes the thinking process during the correct answering process to obtain the thinking process of each skill category question as procedural knowledge at the process level;
[0011] For the common sense questions to be answered, their corresponding skill categories are determined through classification, and the procedural knowledge at the key point level and the procedural knowledge at the process level of the corresponding skill category are extracted and mapped into prefix form. They are then input into the question-answering model together with the common sense questions to be answered to obtain the question-answering results.
[0012] A common sense question answering system based on procedural knowledge summarization, comprising:
[0013] A question extraction module is used to extract a set number of questions from the data set for each skill category;
[0014] The key points summary module is used to input all the extracted questions into the large language model, which compares and analyzes the correct and incorrect option explanations during the answering process to obtain the key points of the answer for each skill category question as the procedural knowledge at the key point level;
[0015] The thinking process summary module is used to input all the extracted questions into the large language model, and the large language model summarizes and analyzes the thinking process in the correct answering process to obtain the thinking process of each skill category question as process-level procedural knowledge;
[0016] The question-answering module based on knowledge selection is used to determine the corresponding skill category of the common sense question to be answered by classification, and extract the key-level procedural knowledge and process-level procedural knowledge of the corresponding skill category, and then map them into prefix form. After that, they are input into the question-answering model together with the common sense question to be answered to obtain the question-answering result.
[0017] A processing device, comprising: one or more processors; a memory for storing one or more programs;
[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.
[0019] A readable storage medium stores a computer program, which implements the above method when the computer program is executed by a processor.
[0020] It can be seen from the technical solution provided by the present invention that by summarizing a very small number of questions of the same type through a large language model to generate procedural knowledge, compared with the previous common sense question-answering system based on a large language model, it can greatly reduce the call to the large language model while ensuring similar or even better performance, thereby reducing the cost of use and eliminating potential data leakage risks. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0022] Figure 1 A flowchart of a common sense question answering method based on procedural knowledge summary provided by an embodiment of the present invention;
[0023] Figure 2 A schematic diagram of the overall framework of a common sense question answering method based on procedural knowledge summary provided by an embodiment of the present invention;
[0024] Figure 3 A schematic diagram of a common sense question answering system based on procedural knowledge summarization provided by an embodiment of the present invention;
[0025] Figure 4 A schematic diagram of a processing device provided by an embodiment of the present invention; DETAILED DESCRIPTION
[0026] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.
[0027] First, the terms that may be used in this article are explained as follows:
[0028] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.
[0029] The term "consisting of..." means excluding any technical feature elements not explicitly listed. If this term is used in a claim, it will make the claim closed, so that it does not contain technical feature elements other than the technical feature elements explicitly listed, except for the conventional impurities related to them. If this term only appears in a clause of a claim, it only limits the elements explicitly listed in the clause, and the elements recorded in other clauses are not excluded from the overall claim.
[0030] The following is a detailed description of a common sense question answering method, system, device and medium based on procedural knowledge summary provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If no specific conditions are specified in the embodiments of the present invention, they are carried out according to the conventional conditions in the field or the conditions recommended by the manufacturer.
[0031] Embodiment 1
[0032] The embodiment of the present invention provides a common sense question answering method based on procedural knowledge summary, such as Figure 1 As shown, it mainly includes the following steps:
[0033] Step 1: Question extraction.
[0034] In the embodiment of the present invention, for each skill category, a set number of questions are extracted from the data set.
[0035] People in this field can understand that skills are a general term in the field of common sense question answering. Skills are a response pattern acquired through learning, which can provide directional guidance for the model in the problem-solving process and help the model focus on the key points and precautions corresponding to the problem.
[0036] Exemplarily, for each skill category, 5 questions may be extracted respectively.
[0037] Step 2: Summarize the key points.
[0038] In this step, all the extracted questions are passed into a large language model, which performs comparative analysis based on the explanations of correct options and incorrect options during the answering process to obtain the key points of answering questions for each skill category as procedural knowledge at the key point level.
[0039] Specifically: for the current skill category, all extracted questions are passed into the large language model; for each question, the large language model analyzes the explanation of each option in the question, and then, given the correct options and incorrect options, the large language model summarizes the key points of each question based on the explanation of each option; the key points of all questions are summarized to obtain the key points for answering the questions of the current skill category; after processing all skill categories separately, the key points for answering the questions of each skill category are obtained.
[0040] Step 3: Summarize your thinking process.
[0041] In this step, all the extracted questions are passed into a large language model, which summarizes and analyzes the thinking process in the correct answering process to obtain the thinking process of each skill category question as process-level procedural knowledge.
[0042] Specifically: for the current skill category, all extracted questions are passed to the large language model; for each question, a correct option is given, and the large language model uses the correct option to infer the correct answer process and obtain the corresponding thinking process; then, the large language model summarizes the thinking process of all questions to obtain the thinking process of the current skill category questions; after processing all skill categories separately, the thinking process of each skill category question is obtained.
[0043] Step 4: Answering process based on knowledge selection.
[0044] In this step, the corresponding skill category of the common sense question to be answered is determined by classification, and the key-level procedural knowledge and process-level procedural knowledge of the corresponding skill category are extracted and mapped into prefix form. After that, they are input into the question-answering model together with the common sense question to be answered to obtain the question-answering result.
[0045] In the above-mentioned scheme provided by the embodiment of the present invention: a large language model is used to effectively summarize procedural knowledge, thereby improving the performance of the common sense question answering system. In the field of common sense question answering, the present invention enables a large language model to generate various types of procedural knowledge independent of specific questions, thereby realizing large language model-assisted common sense question answering at extremely low cost, and providing favorable support for improving the interpretability and performance of the common sense question answering model.
[0046] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the method provided by the embodiment of the present invention is described in detail with reference to specific embodiments below.
[0047] 1. Overview of the principles of the plan.
[0048] The present invention observes that declarative knowledge refers to knowledge that describes factual information, and it emphasizes "what it is". For example, "the sun rises from the east every morning" belongs to declarative knowledge. Procedural knowledge, on the other hand, is knowledge that introduces the processing method in a specific task or activity, and it emphasizes "how to do it". Procedural knowledge is obtained through declarative knowledge processing, and it can be reused in the same task and play an important role therein. Therefore, the present invention attempts to summarize and generate procedural knowledge, and use it to assist the question-answering model in answering common sense questions.
[0049] However, in order to use procedural knowledge to assist the common sense question answering process, there are two main challenges that need to be addressed: 1) Generate effective procedural knowledge. Models think differently from humans, and it is not easy to generate procedural knowledge that is effective for downstream question answering models. Therefore, it is necessary to design corresponding modules to generate effective procedural knowledge; 2) Identify and use procedural knowledge to assist in answering common sense questions. Procedural knowledge is obtained by summarizing the answering process of some questions and applied to the answering process of other questions. Therefore, how to enable the model to efficiently identify and reuse procedural knowledge in other questions needs to be solved. In summary, in order for large language models to generate reusable procedural knowledge independent of specific questions, it is very important. A common sense question answering solution is needed that enables large language models to generate procedural knowledge and can identify and use procedural knowledge to assist in answering common sense questions.
[0050] The solution provided by the present invention is divided into three parts: key point summary, thinking process summary and answering questions based on knowledge selection. Each part is implemented by a corresponding module, which will be specifically introduced in the following embodiment. Specifically, the present invention uses the key point summary part to compare and analyze the correct option explanations and the incorrect option explanations in the process of answering multiple questions of the same type, so as to obtain the key points of answering questions of each type; at the same time, the thinking process summary part is used to summarize and analyze the thinking process in the correct answering process of multiple questions of the same type, so as to obtain the thinking process of answering questions of each type; finally, the present invention uses the answering part based on knowledge selection to identify the procedural knowledge that may be involved in the current question, so as to guide the answering model to answer the question.
[0051] In the implementation process, the present invention collects existing common sense question and answer data sets on the Internet to learn and test the proposed solutions. The samples in these common sense question and answer data sets mainly include the following contents: question ID, question, question options (only included in multiple-choice questions), and answers (for multiple-choice questions, the answers are the corresponding correct options, and for true or false questions, the answers are correct or wrong).
[0052] In addition, since procedural knowledge emphasizes "how to do it", different skills have different processing methods (i.e., both involve completing tasks), and the present invention generates different procedural knowledge based on skills. Considering that the learning process of procedural knowledge can be divided into two categories: pattern recognition and action sequence, the present invention uses a key point summary module and a thinking process summary module to model these two learning processes, respectively. Exemplarily, the present invention can generate seven categories of procedural knowledge: "comparison", "negation", "cause and effect", "ability", "rationality", "timing", and "whole-part relationship".
[0053] Key points summary part: Because procedural knowledge is obtained through summarization, it is very important to accurately summarize the key points of answering questions for correctly answering various types of questions. First, for each skill category, the present invention randomly selects n questions (for example, n=5); then the present invention allows a large language model to imitate the learning process of students, and summarizes the key points of answering the current question by comparing and analyzing the wrong answers with the correct answers; finally, by summarizing and analyzing the key points of answering questions of the same skill type, the key points of answering questions of the current skill category are summarized and used as procedural knowledge at the key point level.
[0054] Thinking process summary part: In order to let the model answer questions according to the correct thinking process, the present invention proposes a skill thinking process summary module. Specifically, first for each skill category, the present invention randomly selects n questions (for example, n = 5); then the present invention allows the large language model to generate the reasoning process of the correct answer, so as to prepare for the subsequent summary of the similarities; finally, the present invention summarizes and analyzes the thinking process of questions of the same skill type, obtains the recommended thinking process of the current skill category question, and uses it as the procedural knowledge at the process level.
[0055] Answering part based on knowledge selection: In order to use the procedural knowledge auxiliary model to answer common sense questions, the present invention proposes an answering module based on knowledge selection. This module first uses common words or phrases for each skill to extract some questions. When the question involves words or phrases of the skill category, it is labeled with the corresponding skill category; then use these labeled questions to train a multi-label classification model, and use the multi-classification model to classify the questions in the data set to determine the skill category involved in each question; finally, according to the skill category involved in each question, the corresponding procedural knowledge (answering points and answering thinking process) is extracted, and the procedural knowledge is spliced with the question, and passed into the question-answering model for training and reasoning.
[0056] The application process of the common sense question answering method based on procedural knowledge summary of the present invention is as follows:
[0057] (1) Extract questions containing skill-related keywords or phrases, classify them according to the skill-related keywords or phrases, and extract n questions from each skill category (n = 5 in the test) for the subsequent summary of procedural knowledge;
[0058] (2) Generate procedural knowledge at the key point level for each skill type through the key point summary module;
[0059] (3) Generate procedural knowledge at the thinking process level for each skill type through the thinking process summary module;
[0060] (4) Use the multi-label classification model in the knowledge selection-based answering module to determine the skill categories that the current question may involve, extract the corresponding procedural knowledge, map it into a prefix form, and pass it into the question-answering model.
[0061] In this way, the present invention generates procedural knowledge that is independent of specific questions and can be reused with less calls to large language models, thereby improving the performance of the common sense question answering system.
[0062] 2. Detailed introduction of the plan.
[0063] 1. Problem definition and formalization.
[0064] In the embodiment of the present invention, the common sense question answering task mainly requires answering given common sense questions. Specifically, the common sense questions are organized in the form of multiple-choice questions or true-false questions. For the common sense question answering task in the form of multiple-choice questions, given a question q and a series of options O = [o1, o2, ..., o L ], where each o is an option, L is the number of options, and the goal of the task is to select the correct option from them; for common sense question-answering tasks in the form of judgment questions, a question q is given, and it is required to judge whether the current question is correct.
[0065] 2. Data collection.
[0066] The present invention uses the widely used open source common sense question and answer datasets CSQA and CSQA2 as test datasets. The CSQA dataset is a common sense question and answer dataset containing 12,247 multiple-choice questions. It uses the connection relationship of entities in the ConceptNet common sense knowledge graph to construct questions and various options. The CSQA2 dataset is a common sense question and answer dataset containing 14,343 judgment questions, in which the questions are constructed by playing a game between humans and AI. Of course, the common sense question and answer datasets shown here are only examples, and users can use other common sense question and answer datasets with the same organizational form according to their needs.
[0067] 3. General knowledge question and answer program.
[0068] In the embodiment of the present invention, the common sense question-answering scheme mainly includes three parts: summary of key points, summary of thinking process, and answering questions based on knowledge selection; the summary of key points is used to summarize procedural knowledge at the key point level; the summary of thinking process is used to summarize procedural knowledge at the process level; the answering part based on knowledge selection first uses common words or phrases of each skill to extract some questions, and uses them to train a multi-label classification model, and then uses the multi-label classification model to determine the skill category involved in the current question, thereby extracting procedural knowledge of the corresponding category to assist in answering questions. Considering that each part can be implemented through corresponding modules, the following is provided: Figure 2 The overall framework of the common sense question answering solution is shown.
[0069] 1) Summary of key points.
[0070] The present invention uses a key point summary module to model pattern recognition, and uses this as a way to learn procedural knowledge. Pattern recognition learning is mainly achieved through two methods: discrimination and generalization. First, discrimination refers to using different processing methods for different types of stimuli, among which comparing positive and negative samples is a very effective method. For this, the present invention uses option explanation acquisition to obtain positive and negative samples, and obtains through question key point summary to achieve discrimination learning. Second, generalization refers to using the same response method for the same stimulus. For this, the present invention uses skill category key point acquisition to summarize the key points of questions in the same skill category, thereby achieving generalization learning. Finally, the key points of the answer are obtained and used as part of the procedural knowledge.
[0071] 1.1) Option explanation acquisition.
[0072] In an embodiment of the present invention, for each question, the current option is defined as the correct option, and then a large language model is used to analyze the reason to obtain the answering perspective of the current option and the explanation of the current option; all options are processed in the same way to obtain the explanation of each option.
[0073] Specifically, when answering questions, large language models tend to focus on the options they think are correct, which has a negative impact on the attention of other options, which is not conducive to simulating the situation where the model answers incorrectly. The present invention addresses this problem by making the model believe that the current option is correct. For each skill category, the extracted question q and the corresponding options [o1, o2, ..., o L ] is passed into a large language model. Specifically, each option o in the question x They are passed into the big model as the correct answer together with the question to let it analyze the reasons and provide the answer perspective of the current option, x∈{1,2,…,L}.
[0074] Exemplarily, the prompt words used in the present invention are as follows:
[0075] The answer to question q is o x , please briefly explain your reasons in one paragraph.
[0076] Where q refers to the current problem, o x Refers to the current option; Finally, the present invention can obtain the explanation of each option in the question [e1, e2, ..., e L ], L is the number of options contained in the current question.
[0077] 1.2) Summarize the key points of the problem.
[0078] After obtaining explanations for each option of the question, the present invention allows the large language model to imitate the student's summary of correct and incorrect options, and passes the explanations of the correct options and incorrect options of the question into the large language model, and allows it to conduct comparative analysis, so as to summarize the key points of answering the current question (the reason why the correct option is correct and the reason why the incorrect option is incorrect); the correct options and incorrect options involved here are the information that comes with the question itself, that is, real information.
[0079] Exemplarily, the prompt words used in the present invention are as follows:
[0080] You are a student who is learning. You need to learn from various opinions. For question q, the correct answer is a. Here are other opinions. Correct opinion: {Some people think the answer is o k :e k}. Wrong view: {Some people think the answer is o j :e j ,…,Some people think the answer is o j′ :e j′ Your focus is to learn the correct way of thinking. Please analyze why the correct option is correct and why the wrong option is wrong. Please analyze from the perspective of principle and answer in one paragraph.
[0081] Among them, the subscript of o is the serial number of the option, and e is the explanation of the corresponding option. Finally, the present invention can obtain the key points summary of multiple problems in the current skill category [k1, k2, ..., k n ].
[0082] 1.3) Obtain the key points of skill categories.
[0083] This part summarizes the key points of the questions in the same skill category obtained previously again to generate the key points of the current skill category. In addition, considering the input length limit of the answering model, the present invention limits the length of the generated skill category key points.
[0084] Exemplarily, the prompt words used in the present invention are as follows:
[0085] The following are some i Skill-related questions and summary of answer points: {question: q1, answer point: k1}, {question: q2, answer point: k2}, …, {question: q n ,Answer key points:k n Please use a paragraph to summarize the key points to pay attention to when answering this type of skill question (within 30 words).
[0086] Among them, s i is the name of skill category i. The following method is used to process the questions of each skill category, and finally the key points of answering each skill can be obtained [matter1, matter2,…, matter m ], where each matter represents the key point of answering a skill, and m is the number of skill categories.
[0087] 2. Summary of the skill thinking process.
[0088] The present invention uses a thinking process summary module to model action sequence learning. Action sequence learning mainly has two stages: program decomposition and program synthesis. First, program decomposition refers to obtaining a clear action sequence. For this, the present invention uses the problem thinking process acquisition process to decompose the thinking process of a specific problem, thereby serving as an action sequence. Second, program synthesis refers to synthesizing simple action sequences into complex action sequences. For this, the present invention uses the skill category thinking process acquisition to summarize problems of the same skill category, thereby obtaining the skill level thinking process and using it as another part of procedural knowledge.
[0089] 2.1) Acquisition of problem thinking process.
[0090] In order to prevent interference from other options, the present invention directly informs the large language model of the correct answer. Specifically, the present invention extracts n questions for each skill category, and then passes the correct answer and question to the large language model, so that it generates a thinking process.
[0091] Exemplarily, the prompt words used in the present invention are as follows:
[0092] The answer to question q is o k , please briefly describe the reasoning process and present it in a path form similar to a knowledge graph (for example: ABCD).
[0093] After that, for each skill category, the present invention obtains the corresponding thinking process [p1, p2, ..., p n ].
[0094] 2.2) Skill category thinking process acquisition.
[0095] This part summarizes the problem thinking process in the same skill category obtained previously again to generate the key points of the current skill category. Specifically, the present invention passes the current skill type, problem and corresponding thinking process into a large language model to generate the thinking process of each skill category.
[0096] Exemplarily, the prompt words used in the present invention are as follows:
[0097] The following are some i Skill-related questions and the thinking process of the correct answer: {question: q1, thinking process: p1}, {question: q2, thinking process: p2}, …, {question: q n ,Thinking process:p n Please use a paragraph (30 words or less) to summarize your thought process for answering this skill-type question.
[0098] After that, the present invention obtains the thinking process of each skill category problem [path1, path2, ..., path m ], each path represents the thinking process of a skill category problem, and m is the number of skill categories.
[0099] 3) Knowledge-based selection part.
[0100] This part uses the previously acquired procedural knowledge (answering points, answering thinking process) to help the question-answering model answer common sense questions. To this end, first train a multi-label classification model to judge the skill category involved in each question; then extract the corresponding procedural knowledge to assist in answering trial questions. Specifically, this part includes two parts: knowledge selection and common sense question-answering combined with procedural knowledge.
[0101] 3.1) Knowledge selection.
[0102] In order to utilize the summarized procedural knowledge, the present invention first needs to identify the skill categories involved in the current question. The present invention pre-uses common words or phrases for each skill to assign pseudo-labels to some questions (i.e., when a skill-related word or phrase appears in a question, it is assigned a corresponding skill category label) to obtain labeled questions; then the present invention uses these labeled questions to train a multi-label classification model, and uses the trained multi-label classification model to judge the possibility that the question involves each skill category.
[0103] First, calculate the scores of the common sense questions to be answered in each skill category, expressed as:
[0104]
[0105] in, It represents the score of the common sense question q′ to be answered in skill category i, and classifier represents the multi-label classification model.
[0106] Then, the score is compared with the set threshold τ to determine whether the current common sense question q′ to be answered involves skill category i, which is expressed as:
[0107]
[0108] in, is the identifier of whether the common sense question q′ to be answered involves skill category i, Indicates that the common sense question q′ to be answered involves skill category i, It means that the common sense question q′ to be answered currently does not involve skill category i.
[0109] 3.2) Common sense question answering combined with procedural knowledge.
[0110] This part is to use procedural knowledge to assist in answering common sense questions. The present invention encodes the extracted procedural knowledge (answering points, answering thinking process), and then passes it into the multi-layer perceptron (MLP) to map it into a prefix form, and then inputs it into the question-answering model together with the common sense question to be answered. Specifically: the number of skill categories corresponding to the common sense question to be answered is recorded as p, and for each skill category, the procedural knowledge at the key point level and the procedural knowledge at the process level are encoded through the context encoder, and then the encoding results corresponding to the p skill categories are spliced, mapped into a prefix form through a multi-layer perceptron, and then spliced to the key information K and value information V of each attention layer of the encoder and decoder of the question-answering model, expressed as:
[0111] E = ContextEncoder (knowledge)
[0112] P=MLP([E1:E2:…E p ])
[0113] K′ l =[P l,k :K l ],V′ l =[P l,v :V l ],
[0114] Among them, E represents the encoding result of a single skill category, knowledge represents the procedural knowledge at the key point level and the procedural knowledge at the process level of a single skill category, and ContextEncoder represents the context encoder; E1:E2:…E p represents the encoding results corresponding to p skill categories; [·∶·] represents the concatenation operation, l represents the lth attention layer of the encoder and decoder, L l 、V l is the original key information and value information of the lth attention layer, K l 、V l is the key information and value information of the lth attention layer after concatenating the prefix; P is the vector mapped to the prefix form by the multilayer perceptron, that is, the prefix vector, P l,k is the vector of key information concatenated to the lth attention layer, P l,v is the vector of information concatenated to the value of the attention layer at the lth layer, P l,k With P l,v are all part of the prefix vector P.
[0115] Considering that the specific scheme of the question-answering model here can be implemented with reference to the existing technology, the internal processing scheme of the question-answering model involved also belongs to the scope of the existing technology, so it will not be repeated; at the same time, the knowledge related to the prefix vector can also refer to the conventional technology, so it will not be repeated. In specific applications, the spliced key information and value information can be input into the question-answering model through the above scheme, and the model can be trained with the existing scheme, and then the model can be inferred. Considering that this part of the process can also be implemented with reference to the conventional technology, it will not be repeated.
[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0117] Embodiment 2
[0118] The present invention also provides a common sense question answering system based on procedural knowledge summary, which is mainly used to implement the method provided in the above embodiment, such as Figure 3 As shown, the system mainly includes:
[0119] A question extraction module is used to extract a set number of questions from the data set for each skill category;
[0120] The key points summary module is used to input all the extracted questions into the large language model, which compares and analyzes the correct and incorrect option explanations during the answering process to obtain the key points of the answer for each skill category question as the procedural knowledge at the key point level;
[0121] The thinking process summary module is used to input all the extracted questions into the large language model, and the large language model summarizes and analyzes the thinking process in the correct answering process to obtain the thinking process of each skill category question as process-level procedural knowledge;
[0122] The question-answering module based on knowledge selection is used to determine the corresponding skill category of the common sense question to be answered by classification, and extract the key-level procedural knowledge and process-level procedural knowledge of the corresponding skill category, and then map them into prefix form. After that, they are input into the question-answering model together with the common sense question to be answered to obtain the question-answering result.
[0123] Considering that the main technical details involved in the system have been introduced in detail in the above embodiments, they will not be repeated here.
[0124] Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.
[0125] Embodiment 3
[0126] The present invention also provides a processing device, such as Figure 4 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the methods provided in the aforementioned embodiments.
[0127] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.
[0128] In the embodiment of the present invention, the specific types of the memory, input device and output device are not limited; for example:
[0129] The input device may be a touch screen, an image acquisition device, a physical button or a mouse, etc.;
[0130] The output device may be a display terminal;
[0131] The memory may be a random access memory (RAM) or a non-volatile memory, such as a disk memory.
[0132] Embodiment 4
[0133] The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.
[0134] In the embodiment of the present invention, the readable storage medium is a computer-readable storage medium and can be set in the aforementioned processing device, for example, as a memory in the processing device. In addition, the readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk, etc., which can store program codes.
[0135] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A common sense question answering method based on procedural knowledge summarization, It is characterized in that include: For each skill category, a set number of questions are extracted from the dataset; All the extracted questions are passed to a large language model, which compares and analyzes the correct and incorrect explanations of the options during the answering process to obtain the key points of the answers for each skill category question as procedural knowledge at the key point level; Furthermore, the large language model is combined with the thinking process in the correct answering process to summarize and analyze, and the thinking process of each skill category question is obtained as procedural knowledge at the process level; For the common sense question x to be answered, its corresponding skill category is determined through classification, and the key-level procedural knowledge and process-level procedural knowledge of the corresponding skill category are extracted and mapped into prefix form. After that, they are input into the question-answering model together with the common sense question to be answered to obtain the question-answering result.
2. A common sense question answering method based on procedural knowledge summary according to claim 1, It is characterized in that The large language model is combined with the correct option explanation and the incorrect option explanation in the answering process to perform comparative analysis to obtain the answer key points of each skill category question, including: For the current skill category, all the extracted questions are passed to the large language model. For each question, the large language model analyzes the explanation of each option in the question. Then, given the correct and incorrect options, the large language model summarizes the key points of each question based on the explanation of each option. The key points of all questions are combined to obtain the key points of the answer to the current skill category question. After processing all skill categories separately, the key points for answering questions in each skill category are obtained.
3. A common sense question answering method based on procedural knowledge summary according to claim 1, It is characterized in that For each question, the explanation of each option in the question is analyzed by a large language model, including: For each question, the current option is defined as the correct option, and then a large language model is used to analyze the reason to obtain the answering perspective of the current option and the explanation of the current option; all options are processed in the same way to obtain an explanation for each option.
4. A common sense question answering method based on procedural knowledge summarization according to claim 1, It is characterized in that The large language model is combined with the thinking process in the correct answering process to summarize and analyze, and the thinking process of each skill category question is obtained, including: For the current skill category, all extracted questions are passed to the large language model. For each question, the correct option is given, and the large language model uses the correct option to infer the correct answer process and obtain the corresponding thinking process. After that, the large language model summarizes the thinking process of all questions and obtains the thinking process of the current skill category question. After processing all skill categories separately, the thinking process of each skill category problem is obtained.
5. A common sense question answering method based on procedural knowledge summary according to claim 1, It is characterized in that The method further includes: Pre-label the questions using words or phrases of each skill category to obtain labeled questions; Training the multi-label classification model using the labeled questions; The common sense questions to be answered are classified through the trained multi-label classification model to determine the corresponding skill category.
6. A common sense question answering method based on procedural knowledge summary according to claim 1 or 5, It is characterized in that The skill categories corresponding to the common sense questions to be answered are determined by classification, including: Calculate the scores of the common sense questions to be answered in each skill category, expressed as: in, represents the score of the common sense question q′ to be answered in skill category i, and classifier represents the multi-label classification model; Compare the score with the set threshold τ to determine whether the current common sense question q′ to be answered involves skill category i, expressed as: in, is the identifier of whether the common sense question q′ to be answered involves skill category i, Indicates that the common sense question q′ to be answered involves skill category i, It means that the common sense question q′ to be answered currently does not involve skill category i.
7. A common sense question answering method based on procedural knowledge summarization according to claim 1, It is characterized in that The extraction of the key-level procedural knowledge and process-level procedural knowledge of the corresponding skill category, mapping them into prefix forms, and inputting them into the question-answering model together with the common sense questions to be answered currently includes: The number of skill categories corresponding to the common sense question to be answered is recorded as p. For each skill category, the procedural knowledge at the key point level and the procedural knowledge at the process level are encoded through the context encoder. The encoding results corresponding to the p skill categories are concatenated and mapped into a prefix form through a multi-layer perceptron. After that, they are concatenated to the key information K and value information V of each attention layer of the encoder and decoder of the question-answering model, expressed as: E = ContextEncoder (knowledge) P=MLP([E 1 :E 2 :…E p ]) K′ l =[P l,k :K l ],V′ l =[P l,v :V l ], Among them, E represents the encoding result of a single skill category, knowledge represents the procedural knowledge at the key point level and the procedural knowledge at the process level of a single skill category, and ContextEncoder represents the context encoder; E 1 :E 2 :…E p represents the encoding results corresponding to p skill categories; [·∶·] represents the concatenation operation, l represents the lth attention layer of the encoder and decoder, K l 、V l is the original key information and value information of the lth attention layer, K l 、V l is the key information and value information of the lth attention layer after concatenating the prefix; P is the vector mapped to the prefix form by the multilayer perceptron, that is, the prefix vector, P l,k is the vector of key information concatenated to the lth attention layer, P l,v is the vector of information concatenated to the value of the attention layer at the lth layer, P l,k With P l,v are all part of the prefix vector P.
8. A common sense question answering system based on procedural knowledge summarization, It is characterized in that include: A question extraction module is used to extract a set number of questions from the data set for each skill category; The key points summary module is used to input all the extracted questions into the large language model, which compares and analyzes the correct and incorrect option explanations during the answering process to obtain the key points of the answer for each skill category question as the procedural knowledge at the key point level; The thinking process summary module is used to input all the extracted questions into the large language model, and the large language model summarizes and analyzes the thinking process in the correct answering process to obtain the thinking process of each skill category question as process-level procedural knowledge; The question-answering module based on knowledge selection is used to determine the corresponding skill category of the common sense question to be answered by classification, and extract the key-level procedural knowledge and process-level procedural knowledge of the corresponding skill category, and then map them into prefix form. After that, they are input into the question-answering model together with the common sense question to be answered to obtain the question-answering result.
9. A processing device, It is characterized in that include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A readable storage medium storing a computer program, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
