Reading type examination question generation system and method based on common sense reasoning
Patent Information
- Application Number
- CN202280094780.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-24
- Publication Date
- 2024-12-03
AI Technical Summary
It is difficult to generate common sense reasoning test questions with correct grammar and consistent content using existing technology, and traditional methods lack modeling of the reasoning process and common sense contained in them, resulting in logically unreasonable test questions and poor user experience.
The reasoning clue graph extraction module is used to identify entities and relationships from the text, and the external knowledge base is combined to expand the entity graph, and the graph-guided question generation model is used to generate test questions. The posterior constraint learning module of language knowledge is used to optimize the model parameters to ensure the generated test questions. Logically sound and grammatically correct.
It realizes the generation of common sense reasoning test questions that are grammatically correct, consistent in content and logically reasonable, improving user experience and generation quality.
Smart Images

Figure CN119072687A_ABST
Abstract
Description
A reading test question generation system and method based on common sense reasoning Technical Field
[0001] The present invention relates to the field of artificial intelligence, and more specifically, to a reading test question generation system based on common sense reasoning. Background Art
[0002] Creating exam questions is a labor-intensive and resource-intensive endeavor, especially for exams that attract significant public attention, such as the Gaokao (National College Entrance Examination). The integrity of those who create the questions is highly demanding, with zero tolerance for violations like leaking test questions. Furthermore, manual question creation is subject to significant subjectivity. This demand has led to the rapid development of automated question generation. This task not only significantly reduces labor costs but also ensures greater confidentiality and objectivity, thereby enhancing exam fairness. Machine question generation has become a research hotspot in artificial intelligence and natural language processing. It requires generating coherent and relevant questions based on a given text. As a dual task of question-answering, questioning offers a different perspective on a machine's ability to understand the semantics of a given text, thus supporting numerous useful applications. For example, questioning can serve as a data augmentation strategy to reduce the cost of manual annotation of question-answering corpora. Interactive questioning can also generate new topics for cold-start conversational systems, and question-based feedback can enhance information acquisition in search engines. Questioning is a cognitively demanding process, requiring varying levels of comprehension. Simple questions often only touch upon the superficial meaning of a text, which can be effectively addressed through context-based word matching. In practical applications, complex, comprehensive questions have greater application value. For example, in education, to achieve quality education, the Ministry of Education requires that simple literal matching questions account for no more than 5% of elementary school exams. The Ministry encourages the addition of comprehensive logical thinking questions, particularly common sense reasoning questions. Compared to simple literal matching questions, these logical and common sense-based questions can better assess learning outcomes and stimulate students' independent learning abilities. However, automatically generating these common sense reasoning questions is not easy. Machines must deeply consider and reason about multiple entity clues scattered throughout the text. They even need to understand external knowledge and common sense common in people's daily lives, which they lack. For example, as shown in Figure 1, the question asks about a place related to a fountain in a park. Unlike simple literal matching questions, there is no literal connection between the question and the answer. However, by linking multiple evidence clues in the given text (i.e., fountain, Statue of Liberty) and external common sense relationships (i.e., Statue of Liberty, located in New York City), (New York City, part of, United States), (United States, capital, Washington), and (White House, located in, Washington), the question and answer can be logically linked. Such multi-hop reasoning chains are crucial to the question direction and answering process.
[0003] Traditional generative approaches rely primarily on manually crafted rules or templates to convert input text into questions. These rules and templates are easily over-engineered, resulting in poor model generalization and scalability. Therefore, current mainstream approaches employ data-driven neural models. These models frame the generation task as a sequence mapping problem similar to translation. Specifically, they map, or translate, the input text into questions by learning sequence mapping patterns from a large amount of training data. However, this approach is suitable for generating simple literal matching questions, but struggles with commonsense reasoning questions that require comprehensive understanding. This is because commonsense reasoning questions involve generation, not simply converting or summarizing the given content into a semantically equivalent form, but rather are subject to various grammatical and semantic constraints. Generated questions must not only be fluent but also answerable and reasonable. Specifically, the questions must be solvable, and the resulting solutions must be consistent with the given answers. Alternatively, the solutions must go beyond simple literal matching and involve commonsense reasoning. Traditional methods lack modeling of this critical reasoning process and the underlying common sense, and research on how to use this knowledge to guide questioning is weak. This leads to a logical gap, meaning the machine cannot know what to ask and how to ask it, ultimately only outputting superficial and simple questions. Furthermore, exam questions must meet numerous language requirements, such as correct grammar, accurate semantics, and valid and solvable questions. Otherwise, the generated questions will be riddled with typos, lack logic, or even incoherent or unsolvable, resulting in a poor user experience. These languages require discrete representation, making it difficult for conventional methods to integrate them into neural networks with continuous representations.
[0004] Summary of the Invention
[0005] The present invention provides a reading test question generation system based on common sense reasoning, which can generate test questions with correct grammar and consistent content.
[0006] Another object of the present invention is to provide a method for generating reading test questions based on common sense reasoning.
[0007] In order to achieve the above technical effects, the technical solutions of the present invention are as follows:
[0008] A reading test question generation system based on common sense reasoning, characterized by comprising:
[0009] The reasoning clue graph extraction module is used to infer the reasoning clue graph based on the text context starting from the given answer. It identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph. It selects the question content from the entity graph and retrieves relevant entities from the external common sense database to expand the entity graph to form the reasoning clue graph.
[0010] A graph-guided question generation model module that combines all text and entity graphs, as well as multi-hop and commonsense knowledge in the entity graph, to generate high-quality exam questions;
[0011] The posterior constraint learning module of language knowledge is used to train the graph-guided question generation model to learn the optimal model parameters.
[0012] A method for generating reading test questions based on common sense reasoning, comprising the following steps:
[0013] S1: The reasoning clue graph extraction module starts with a given answer and infers a reasoning clue graph based on the text context: it identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph; it selects the question content from the entity graph and retrieves related entities from an external common sense database to expand the entity graph to form a reasoning clue graph;
[0014] S2: The graph-guided question generation model module combines all text and entity graphs, as well as multi-hop and common sense knowledge in the entity graph to generate high-quality test questions;
[0015] S3: The posterior constraint learning module of language knowledge trains the graph-guided question generation model to learn the optimal model parameters.
[0016] Furthermore, in step S1, the reasoning clue graph extraction module represents each input text sentence as a parse tree, each parse tree node contains several entities and edges, where the edges represent contextual relationships, filters out punctuation marks and stop words in each node, aggregates equivalent nodes and co-reference nodes in the parse tree; adds connecting edges between trees between similar nodes in adjacent sentences to obtain an entity graph with potential clues.
[0017] Furthermore, in step S1, the process of selecting the question content from the entity graph is:
[0018] Mark two sentences that are relevant to the answer and remove the rest from the entity graph:
[0019] Sentences containing answer keywords are identified by exact word matching or related sentences are identified using the classifier based on the Rouge-L similarity metric, where keywords are the remaining words after filtering stop words from the answer.
[0020] Furthermore, in step S1, the method of retrieving relevant entities from the external common sense library to expand the entity graph is: using the entity of the input text as the query condition, and retrieving relevant entities from the external open source common sense library through word matching.
[0021] Furthermore, in step S2, a graph-enhanced encoder is designed to combine all text and entity graphs to complete the fusion of heterogeneous features of text and graphs. The graph-enhanced encoder consists of six cascade layers, each of which includes:
[0022] Text self-attention sublayer: responsible for encoding the text content and performing nonlinear transformation on the given text input representation vector to obtain a new vector as the input of the next sublayer:
[0023] Given a text input X=(x1,…x T ), where x is the distributed embedding representation of each word, and each sub-layer is linearly transformed to obtain three vectors, including the key vector k i =x i W K , value vector v i =x i W V and query vector q i =x i W Q , where W K 、W V and W Q is a learnable matrix parameter; then, the interaction score r between the query vector and the key vector is calculated by the dot product ij ,Right now The scores are normalized by the logistic regression function Softmax, and the attention coefficient α is calculated by formula (1) ij , where d x represents the dimension of the key vector; by weighted summing of the attention and value vectors, we can get the context-aware output z i The first sub-layer is initialized by the representation vector of the input text, where the corresponding vector is retrieved from the pre-trained vector library for each word in the text, and the vectors of each word are combined into a vector to represent the text; the output of the last layer is collected and To represent the input text paragraph and the answer respectively:
[0024]
[0025]
[0026] Graph attention sublayer: Considering that the nodes of the graph contain multiple words, Characterize each node, where is the distributed embedding representation of the jth word of the i-th node, m and n are the start and end positions of the text segment in the node, respectively. Represents the attention distribution of the node, which is used to indicate the importance of the node; Defined as softmax(ReLU(W R [G;w j ])), where g i is the i-th column of matrix G, W R is a trainable parameter; subsequently, the contextual representation of the node is enriched by weighted aggregation of the relevant semantics of adjacent nodes, where the weights are dynamically determined by the attention mechanism; in order to obtain this structural context information, the present invention obtains the relevance score of the edge by calculating the dot product between adjacent nodes i and j, Among them, τ ij Represents the relationship between nodes, which is learned from the corresponding relationship type. and It is a trainable parameter; by normalizing the relevant scores of all connected edges of the node, the attention of each node can be calculated Refer to formula (2), where Represents the neighboring nodes of node i; by weighted summing the attention, the output of graph structure perception is obtained where d x is the dimension size of the key vector, and Represents learnable parameters:
[0027]
[0028]
[0029] Feedforward sublayer: transform the text vector z i and graph vector Since splicing will introduce a lot of noise, a gating mechanism is used to obtain significant features and reduce noise, as shown in formula (3), where ⊙ represents element multiplication, f is a fusion vector, and η is a learnable gate used to selectively control features at different angles:
[0030]
[0031] z i =η⊙f+(1-η)⊙z i ;
[0032]
[0033] In addition, in order to enhance the generalization ability of the model, a nonlinear transformation is introduced, which is implemented by a double-layer multilayer perceptron MLP with RELU activation function. i =max(0,z iW1+b1)W2+b2, where W1, W2, b1 and b2 are trainable parameters, and the output code h obtained by this transformation i It is regarded as the input of the next layer; after multiple layers of operation, the final representation is obtained.
[0034] Furthermore, in step S2, the output matrix h of the last layer of the graph enhancement encoder is regarded as a representation vector that integrates all input contents, and another converter is used to perform autoregressive decoding to generate test questions; the test questions are based on probability Generated word by word, similar to the encoder of graph enhancement, the decoder consists of multiple layers cascaded, at step t, the lth th Output of the layer Two parts are considered, including the The decoding result from attention, and the encoding content from The attention representation of These two parts are fused through a multi-head attention layer and expressed as After multiple layers of nonlinear transformation, the output vector can be obtained By normalizing the vector on the predefined vocabulary V through the logistic regression function Softmax, the output probability of the word can be obtained Where W o and b o Represents trainable parameters;
[0035] In order to solve the problem of generating new words outside the vocabulary, a copy mechanism is used according to the distribution Generate test questions by copying new words from the input text, where α is the attention of the input text; define a balance factor By sampling this distribution, we can generate the words of the test questions one by one, where k is the balance factor, f(·) is a feedforward neural network with an S-type activation function Sigmoid, and y t-1 Represents the distributed embedding representation vector of the word generated in the t-1th step; in order to avoid the semantic drift problem, that is, the problem that the answer is inconsistent with the question, the answer encoding is used To initialize the decoder; a special marker is also introduced <eos>To indicate when the generation process is terminated:
[0036]
[0037] Furthermore, in step S3, supervised learning is used to train the graph-guided question generation model, that is, the optimal model parameters are learned by maximizing the log-likelihood probability, referring to formula (5), where Represents a training set containing N samples, K represents the size of the words in the test questions, and through a supervised teacher-guided learning model, the results generated by the model are close to the manual annotations in the samples:
[0038]
[0039] However, this approach cannot guarantee that the generated results are common sense reasoning questions. To solve this problem, this paper introduces a series of linguistic knowledge as regularization constraints to regulate the output probability distribution of the results. Regularization is achieved by satisfying the expected distribution d(·) of the constraints and the output distribution p of the model. θ (·) is realized by the KL divergence between them; a hybrid objective is adopted to fuse the supervision loss and the posterior loss into formula (6), where Is of the form The constraint set is b; φ(·) is the constraint feature function bounded by b; a, c, y represent the answer, text paragraph, and question respectively; λ is a parameter that weighs the confidence:
[0040]
[0041]
[0042] Since the above optimization objective is a convex function, it has a closed-form solution, refer to formula (7), where Z is the normalization factor, δ is the regularization factor, and f j (a,c,y)=b j -φ j (a, c, q) represents the constraint function; that is, when (a, c, y) satisfies the constraint, f j (·)>0; optimal distribution d * (·) Not only close to the distribution p learned from the labeled training data θ (·), and most of the constraints are satisfied. This posterior regularization can flexibly inject discrete constraint knowledge into the continuous model:
[0043]
[0044] Furthermore, in step S3, three constraint functions are designed to improve the generation quality of the model: common sense answerability, content relevance consistency, and expression grammatical accuracy.
[0045] The construction process of the common sense answerability constraint function is as follows: extract the most relevant paragraph sentences to the question by matching similarity, extract entities such as verbs and noun phrases from the answers and the extracted sentences using the Spacy toolkit, and group the results of the question into pairs. By introducing the learnable parameter ω1, the entity constraint knowledge is expressed as If the kth th Entity pairs are semantically similar, then vice versa, is the weight of each entity pair obtained from the attention network, where the network parameter is ω1; through this representation, when there are no similar entities between the sentences related to the answer and the question, then f 1 (·) is positive; otherwise, it is negative;
[0046] The process of constructing the content association consistency constraint function is: using a data-driven classifier, f 2 (a,c,y)=F(v c ,v y ;ω2), where v c represents the entity extracted from paragraph c, v y is the entity extracted from question y, ω2 is a parameter. When the two pairs of entities are semantically similar, the function F outputs a positive value, otherwise it outputs a negative value. By referring to the performance ranking of the evaluation dataset, the best performing question-answering model, the Unicorn model, is selected to predict the answer. By penalizing samples with inconsistent answers, the model can be encouraged to generate consistent answers, including in It is a judgment function that verifies the consistency between the predicted result and the labeled answer;
[0047] The construction process of the grammatical accuracy constraint function is to measure the fluency of the generated results by calculating the perplexity of the language model. Among them, P LM It is based on the pre-trained Roberta language model, where K is the number of question words. For the same answer, the generated result is often similar to the annotated result in terms of semantic structure and grammatical pattern. The word movement distance WMD is used to measure the semantic similarity between two texts. 5 (a,c,y)=WMD(y,y * ) / Length(y), where Length(·) is a normalized function, y * is the result of annotation; the similarity of grammatical structure is calculated by dependency syntax tree (DPTS), and the number of common substructures between two grammatical analysis trees is calculated by attention vector tree kernel (ACVT), so as to calculate the relevance of grammar, f 6 (a,c,y)=DPTS ACVT (y,y * ).
[0048] Furthermore, in step S3, the KL divergence optimization objective function in formula (6) is regarded as a knowledge extraction problem, that is, transferring knowledge from the teacher model d with constraints to the student's question generation model p θ Therefore, this objective function is solved using the expectation-maximization EM algorithm; in the t-th expectation calculation step, by formula Calculate the probability distribution d of the teacher model; then, calculate the maximum expectation, that is, update the probability distribution θ of the student model through formula (8) to approximate the probability distribution d of the teacher model, where is a trade-off factor, o is the labeled question probability distribution, and E is the expected accumulated error. This distillation objective can achieve a balance between the soft prediction of simulation d and the prediction of the true result;
[0049]
[0050] In addition to the student's question generation model θ, we also need to learn the parameters of the constraint f and its confidence The target in formula (6) It is found that when y is the labeled result, the constraint expectation h(a,c,y;ω)=exp{δ·∑ l λ l f l (a,c,y;ω l )} should be larger; h(·) is regarded as a likelihood function indicating the quality of the result, which makes the target similar to the variational lower bound of the corresponding model, which is achieved by labeling the result h based on the mean square error loss MSE. * To train ω, refer to formula (9). In addition, the constraint confidence λ is learned according to the probability distribution d of the teacher model, refer to formula (10):
[0051]
[0052]
[0053] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:
[0054] The present invention discovered that contextual connections between sentences help connect reasoning threads scattered throughout the sentences. By leveraging the context of the text, the chain of reasoning threads can be inferred from the answer, incorporating various underlying common sense knowledge into this chain. This chain of threads is then used to guide the generation of inferential test questions. Furthermore, the present invention incorporates various grammatical and semantic knowledge as a posteriori constraints into the model, thereby generating grammatically correct and consistent test questions. Logically sound test questions are generated by mastering a priori reasoning processes such as underlying common sense entities and relationships. Using these constraints as a posteriori knowledge, various linguistic knowledge is flexibly integrated into the generative model through regularization, resulting in smoother, more consistent, and more answerable results. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 is an example of a test question that requires common sense reasoning;
[0056] FIG2 is a block diagram of the system of the present invention;
[0057] FIG3 is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0058] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;
[0059] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;
[0060] It is understood by those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.
[0061] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.
[0062] Example 1
[0063] As shown in FIG2 , a reading test question generation system based on common sense reasoning is characterized by including:
[0064] The reasoning clue graph extraction module is used to infer the reasoning clue graph based on the text context starting from the given answer. It identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph. It selects the question content from the entity graph and retrieves relevant entities from the external common sense database to expand the entity graph to form the reasoning clue graph.
[0065] A graph-guided question generation model module that combines all text and entity graphs, as well as multi-hop and commonsense knowledge in the entity graph, to generate high-quality exam questions;
[0066] The posterior constraint learning module of language knowledge is used to train the graph-guided question generation model to learn the optimal model parameters.
[0067] Example 2
[0068] As shown in FIG3 , a method for generating reading test questions based on common sense reasoning includes the following steps:
[0069] S1: The reasoning clue graph extraction module starts with a given answer and infers a reasoning clue graph based on the text context: it identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph; it selects the question content from the entity graph and retrieves related entities from an external common sense database to expand the entity graph to form a reasoning clue graph;
[0070] S2: The graph-guided question generation model module combines all text and entity graphs, as well as multi-hop and common sense knowledge in the entity graph to generate high-quality test questions;
[0071] S3: The posterior constraint learning module of language knowledge trains the graph-guided question generation model to learn the optimal model parameters.
[0072] In step S1, the reasoning clue graph extraction module represents each input text sentence as a parse tree. Each parse tree node contains several entities and edges, where the edges represent contextual relationships. Punctuation marks and stop words in each node are filtered out, and equivalent nodes and co-reference nodes in the parse tree are aggregated; connecting edges between trees are added between similar nodes in adjacent sentences to obtain an entity graph with potential clues.
[0073] In step S1, the process of selecting question content from the entity graph is:
[0074] Mark two sentences that are relevant to the answer and remove the rest from the entity graph:
[0075] Sentences containing answer keywords are identified by exact word matching or related sentences are identified using the classifier based on the Rouge-L similarity metric, where keywords are the remaining words after filtering stop words from the answer.
[0076] In step S1, the method of retrieving relevant entities from the external common sense database to expand the entity graph is: using the entities of the input text as query conditions, and retrieving relevant entities from the external open source common sense database through word matching.
[0077] In step S2, a graph-enhanced encoder is designed to combine all text and entity graphs to fuse the heterogeneous features of text and graphs. The graph-enhanced encoder consists of six cascaded layers, each of which includes:
[0078] Text self-attention sublayer: responsible for encoding the text content and performing nonlinear transformation on the given text input representation vector to obtain a new vector as the input of the next sublayer:
[0079] Given a text input X=(x1,…x T ), where x is the distributed embedding representation of each word, and each sub-layer is linearly transformed to obtain three vectors, including the key vector k i =x i W K , value vector v i =x i W V and query vector q i =x i W Q , where W K 、W V and W Q is a learnable matrix parameter; then, the interaction score r between the query vector and the key vector is calculated by the dot product ij ,Right now The scores are normalized by the logistic regression function Softmax, and the attention coefficient α is calculated by formula (1) ij , where d x represents the dimension of the key vector; by weighted summing of the attention and value vectors, we can get the context-aware output z i The first sub-layer is initialized by the representation vector of the input text, where the corresponding vector is retrieved from the pre-trained vector library for each word in the text, and the vectors of each word are combined into a vector to represent the text; the output of the last layer is collected and To represent the input text paragraph and the answer respectively:
[0080]
[0081]
[0082] Graph attention sublayer: Considering that the nodes of the graph contain multiple words, Characterize each node, where is the distributed embedding representation of the jth word of the i-th node, m and n are the start and end positions of the text segment in the node, respectively. Represents the attention distribution of the node, which is used to indicate the importance of the node; Defined as softmax(ReLU(W R [G;w j ])), where g i is the i-th column of matrix G, W R is a trainable parameter; subsequently, the contextual representation of the node is enriched by weighted aggregation of the relevant semantics of adjacent nodes, where the weights are dynamically determined by the attention mechanism; in order to obtain this structural context information, the present invention obtains the relevance score of the edge by calculating the dot product between adjacent nodes i and j, Among them, τ ij Represents the relationship between nodes, which is learned from the corresponding relationship type. and It is a trainable parameter; by normalizing the relevant scores of all connected edges of the node, the attention of each node can be calculated Refer to formula (2), where Represents the neighboring nodes of node i; by weighted summing the attention, the output of graph structure perception is obtained where d x is the dimension size of the key vector, and Represents learnable parameters:
[0083]
[0084]
[0085] Feedforward sublayer: transform the text vector z i and graph vector Since splicing will introduce a lot of noise, a gating mechanism is used to obtain significant features and reduce noise, as shown in formula (3), where ⊙ represents element multiplication, f is a fusion vector, and η is a learnable gate used to selectively control features at different angles:
[0086]
[0087] z i =η⊙f+(1-η)⊙z i ;
[0088]
[0089] In addition, in order to enhance the generalization ability of the model, a nonlinear transformation is introduced, which is implemented by a double-layer multilayer perceptron MLP with RELU activation function. i =max(0,z i W1+b1)W2+b2, where W1, W2, b1 and b2 are trainable parameters, and the output code h obtained by this transformation i It is regarded as the input of the next layer; after multiple layers of operation, the final representation is obtained.
[0090] In step S2, the output matrix h of the last layer of the graph enhancement encoder is regarded as a representation vector that integrates all input contents, and another transformer is used to perform autoregressive decoding to generate test questions; the test questions are based on probability Generated word by word, similar to the encoder of graph enhancement, the decoder consists of multiple layers cascaded, at step t, the lth th Output of the layer Two parts are considered, including the The decoding result from attention, and the encoding content from The attention representation of These two parts are fused through a multi-head attention layer and expressed as After multiple layers of nonlinear transformation, the output vector can be obtained By normalizing the vector on the predefined vocabulary V through the logistic regression function Softmax, the output probability of the word can be obtained Where W o and b o Represents trainable parameters;
[0091] In order to solve the problem of generating new words outside the vocabulary, a copy mechanism is used according to the distribution Generate test questions by copying new words from the input text, where α is the attention of the input text; define a balance factor By sampling this distribution, we can generate the words of the test questions one by one, where k is the balance factor, f(·) is a feedforward neural network with an S-type activation function Sigmoid, and y t-1 Represents the distributed embedding representation vector of the word generated in the t-1th step; in order to avoid the semantic drift problem, that is, the problem that the answer is inconsistent with the question, the answer encoding is used To initialize the decoder; a special marker is also introduced <eos>To indicate when the generation process is terminated:
[0092]
[0093] In step S3, supervised learning is used to train the graph-guided question generation model, that is, the optimal model parameters are learned by maximizing the log-likelihood probability, referring to formula (5), where Represents a training set containing N samples, K represents the size of the words in the test questions, and through a supervised teacher-guided learning model, the results generated by the model are close to the manual annotations in the samples:
[0094]
[0095] However, this method cannot guarantee that the generated results are common sense reasoning questions. A series of linguistic knowledge is introduced as additional constraints to regulate the output probability distribution of the results. Regularization is achieved by satisfying the expected distribution d(·) of the constraints and the output distribution p of the model. θ (·) is realized by the KL divergence between them; a hybrid objective is adopted to fuse the supervision loss and the posterior loss into formula (6), where Is of the form The constraint set is b; φ(·) is the constraint feature function bounded by b; a, c, y represent the answer, text paragraph, and question respectively; λ is a parameter that weighs the confidence:
[0096]
[0097]
[0098] Since the above optimization objective is a convex function, it has a closed-form solution, refer to formula (7), where Z is the normalization factor, δ is the regularization factor, and f j (a,c,y)=b j -φ j (a, c, q) represents the constraint function; that is, when (a, c, y) satisfies the constraint, f j (·)>0; optimal distribution d * (·) Not only close to the distribution p learned from the labeled training data θ (·), and most of the constraints are satisfied. This posterior regularization can flexibly inject discrete constraint knowledge into the continuous model:
[0099]
[0100] In step S3, three constraint functions are designed to improve the generation quality of the model: common sense answerability, content relevance consistency, and expression grammatical accuracy.
[0101] The construction process of the common sense answerability constraint function is as follows: extract the most relevant paragraph sentences to the question by matching similarity, extract entities such as verbs and noun phrases from the answers and the extracted sentences using the Spacy toolkit, and group the results of the question into pairs. By introducing the learnable parameter ω1, the entity constraint knowledge is expressed as If the kth th Entity pairs are semantically similar, then vice versa, is the weight of each entity pair obtained from the attention network, where the network parameter is ω1; through this representation, when there are no similar entities between the sentences related to the answer and the question, then f 1 (·) is positive; otherwise, it is negative;
[0102] The process of constructing the content association consistency constraint function is: using a data-driven classifier, f 2 (a,c,y)=F(v c ,v y ;ω2), where v c represents the entity extracted from paragraph c, v y is the entity extracted from question y, ω2 is a parameter. When the two pairs of entities are semantically similar, the function F outputs a positive value, otherwise it outputs a negative value. By referring to the performance ranking of the evaluation dataset, the best performing question-answering model, the Unicorn model, is selected to predict the answer. By penalizing samples with inconsistent answers, the model can be encouraged to generate consistent answers, including in It is a judgment function that verifies the consistency between the predicted result and the labeled answer;
[0103] The construction process of the grammatical accuracy constraint function is to measure the fluency of the generated results by calculating the perplexity of the language model. Among them, P LM It is based on the pre-trained Roberta language model, where K is the number of question words. For the same answer, the generated result is often similar to the annotated result in terms of semantic structure and grammatical pattern. The word movement distance WMD is used to measure the semantic similarity between two texts. 5 (a,c,y)=WMD(y,y * ) / Length(y), where Length(·) is a normalized function, y * is the result of annotation; the similarity of grammatical structure is calculated by dependency syntax tree (DPTS), and the number of common substructures between two grammatical analysis trees is calculated by attention vector tree kernel (ACVT), so as to calculate the relevance of grammar, f 6 (a,c,y)=DPTS ACVT (y,y * ).
[0104] In step S3, the KL divergence optimization objective function in formula (6) is regarded as a knowledge distillation problem, that is, transferring knowledge from the teacher model d with constraints to the student's question generation model p θ Therefore, this objective function is solved using the expectation-maximization EM algorithm; in the t-th expectation calculation step, by formula Calculate the probability distribution d of the teacher model; then, calculate the maximum expectation, that is, update the probability distribution θ of the student model through formula (8) to approximate the probability distribution d of the teacher model, where is a trade-off factor, o is the labeled question probability distribution, and E is the expected accumulated error. This distillation objective can achieve a balance between the soft prediction of simulation d and the prediction of the true result;
[0105]
[0106] In addition to the student's question generation model θ, we also need to learn the parameters of the constraint f and its confidence The target in formula (6) It is found that when y is the labeled result, the constraint expectation h(a,c,y;ω)=exp{δ·∑ l λ l f l (a,c,y;ω l )} should be larger; h(·) is regarded as a likelihood function indicating the quality of the result, which makes the target similar to the variational lower bound of the corresponding model, which is achieved by labeling the result h based on the mean square error loss MSE. * To train ω, refer to formula (9). In addition, the constraint confidence λ is learned according to the probability distribution d of the teacher model, refer to formula (10):
[0107]
[0108]
[0109] Example 3
[0110] As shown in FIG2 , a reading test question generation system based on common sense reasoning is characterized by including:
[0111] The reasoning clue graph extraction module is used to infer the reasoning clue graph based on the text context starting from the given answer. It identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph. It selects the question content from the entity graph and retrieves relevant entities from the external common sense database to expand the entity graph to form the reasoning clue graph.
[0112] A graph-guided question generation model module that combines all text and entity graphs, as well as multi-hop and commonsense knowledge in the entity graph, to generate high-quality exam questions;
[0113] The posterior constraint learning module of language knowledge is used to train the graph-guided question generation model to learn the optimal model parameters.
[0114] As shown in FIG3 , a method applied to the above-mentioned common sense reasoning-based reading test question generation system includes the following steps:
[0115] S1: The reasoning clue graph extraction module starts with a given answer and infers a reasoning clue graph based on the text context: it identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph; it selects the question content from the entity graph and retrieves related entities from an external common sense database to expand the entity graph to form a reasoning clue graph;
[0116] S2: The graph-guided question generation model module combines all text and entity graphs, as well as multi-hop and common sense knowledge in the entity graph to generate high-quality test questions;
[0117] S3: The posterior constraint learning module of language knowledge trains the graph-guided question generation model to learn the optimal model parameters.
[0118] In step S1, the reasoning clue graph extraction module represents each input text sentence as a parse tree. Each parse tree node contains several entities and edges, where the edges represent contextual relationships. Punctuation marks and stop words in each node are filtered out, and equivalent nodes and co-reference nodes in the parse tree are aggregated; connecting edges between trees are added between similar nodes in adjacent sentences to obtain an entity graph with potential clues.
[0119] In step S1, the process of selecting question content from the entity graph is:
[0120] Mark two sentences that are relevant to the answer and remove the rest from the entity graph:
[0121] Sentences containing answer keywords are identified by exact word matching or related sentences are identified using the classifier based on the Rouge-L similarity metric, where keywords are the remaining words after filtering stop words from the answer.
[0122] In step S1, the method of retrieving relevant entities from the external common sense database to expand the entity graph is: using the entities of the input text as query conditions, and retrieving relevant entities from the external open source common sense database through word matching.
[0123] In step S2, a graph-enhanced encoder is designed to combine all text and entity graphs to fuse the heterogeneous features of text and graphs. The graph-enhanced encoder consists of six cascaded layers, each of which includes:
[0124] Text self-attention sublayer: responsible for encoding the text content and performing nonlinear transformation on the given text input representation vector to obtain a new vector as the input of the next sublayer:
[0125] Given a text input X=(x1,…x T ), where x is the distributed embedding representation of each word, and each sub-layer is linearly transformed to obtain three vectors, including the key vector k i =x i W K , value vector v i =x i W V and query vector q i =x i W Q , where W K 、W V and W Q is a learnable matrix parameter; then, the interaction score r between the query vector and the key vector is calculated by the dot product ij ,Right now The scores are normalized by the logistic regression function Softmax, and the attention coefficient α is calculated by formula (1) ij , where d x represents the dimension of the key vector; by weighted summing of the attention and value vectors, we can get the context-aware output z i The first sub-layer is initialized by the representation vector of the input text, where the corresponding vector is retrieved from the pre-trained vector library for each word in the text, and the vectors of each word are combined into a vector to represent the text; the output of the last layer is collected and To represent the input text paragraph and the answer respectively:
[0126]
[0127]
[0128] Graph attention sublayer: Considering that the nodes of the graph contain multiple words, Characterize each node, where is the distributed embedding representation of the jth word of the i-th node, m and n are the start and end positions of the text segment in the node, respectively. Represents the attention distribution of the node, which is used to indicate the importance of the node; Defined as softmax(ReLU(W R [G;w j ])), where g i is the i-th column of matrix G, W R is a trainable parameter; subsequently, the contextual representation of the node is enriched by weighted aggregation of the relevant semantics of adjacent nodes, where the weights are dynamically determined by the attention mechanism; in order to obtain this structural context information, the present invention obtains the relevance score of the edge by calculating the dot product between adjacent nodes i and j, Among them, τ ij Represents the relationship between nodes, which is learned from the corresponding relationship type. and It is a trainable parameter; by normalizing the relevant scores of all connected edges of the node, the attention of each node can be calculated Refer to formula (2), where Represents the neighboring nodes of node i; by weighted summing the attention, the output of graph structure perception is obtained where d x is the dimension size of the key vector, and Represents learnable parameters:
[0129]
[0130]
[0131] Feedforward sublayer: transform the text vector z i and graph vector Since splicing will introduce a lot of noise, a gating mechanism is used to obtain significant features and reduce noise, as shown in formula (3), where ⊙ represents element multiplication, f is a fusion vector, and η is a learnable gate used to selectively control features at different angles:
[0132]
[0133] z i =η⊙f+(1-η)⊙z i ;
[0134]
[0135] In addition, in order to enhance the generalization ability of the model, a nonlinear transformation is introduced, which is implemented by a double-layer multilayer perceptron MLP with RELU activation function. i =max(0,z i W1+b1)W2+b2, where W1, W2, b1 and b2 are trainable parameters, and the output code h obtained by this transformation i It is regarded as the input of the next layer; after multiple layers of operation, the final representation is obtained.
[0136] In step S2, the output matrix h of the last layer of the graph enhancement encoder is regarded as a representation vector that integrates all input contents, and another transformer is used to perform autoregressive decoding to generate test questions; the test questions are based on probability Generated word by word, similar to the encoder of graph enhancement, the decoder consists of multiple layers cascaded, at step t, the lth th Output of the layer Two parts are considered, including the The decoding result from attention, and the encoding content from The attention representation of These two parts are fused through a multi-head attention layer and expressed as After multiple layers of nonlinear transformation, the output vector can be obtained By normalizing the vector on the predefined vocabulary V through the logistic regression function Softmax, the output probability of the word can be obtained Where W o and b o Represents trainable parameters;
[0137] In order to solve the problem of generating new words outside the vocabulary, a copy mechanism is used according to the distribution Generate test questions by copying new words from the input text, where α is the attention of the input text; define a balance factor By sampling this distribution, we can generate the words of the test questions one by one, where k is the balance factor, f(·) is a feedforward neural network with an S-type activation function Sigmoid, and y t-1 Represents the distributed embedding representation vector of the word generated in the t-1th step; in order to avoid the semantic drift problem, that is, the problem that the answer is inconsistent with the question, the answer encoding is used To initialize the decoder; a special marker is also introduced <eos>To indicate when the generation process is terminated:
[0138]
[0139] In step S3, supervised learning is used to train the graph-guided question generation model, that is, the optimal model parameters are learned by maximizing the log-likelihood probability, referring to formula (5), where Represents a training set containing N samples, K represents the size of the words in the test questions, and through a supervised teacher-guided learning model, the results generated by the model are close to the manual annotations in the samples:
[0140]
[0141] However, this method cannot guarantee that the generated results are common sense reasoning questions. A series of linguistic knowledge is introduced as additional constraints to regulate the output probability distribution of the results. Regularization is achieved by satisfying the expected distribution d(·) of the constraints and the output distribution p of the model. θ (·) is realized by the KL divergence between them; a hybrid objective is adopted to fuse the supervision loss and the posterior loss into formula (6), where Is of the form The constraint set is b; φ(·) is the constraint feature function bounded by b; a, c, y represent the answer, text paragraph, and question respectively; λ is a parameter that weighs the confidence:
[0142]
[0143]
[0144] Since the above optimization objective is a convex function, it has a closed-form solution, refer to formula (7), where Z is the normalization factor, δ is the regularization factor, and f j (a,c,y)=b j -φ j (a, c, q) represents the constraint function; that is, when (a, c, y) satisfies the constraint, f j (·)>0; optimal distribution d * (·) Not only close to the distribution p learned from the labeled training data θ (·), and most of the constraints are satisfied. This posterior regularization can flexibly inject discrete constraint knowledge into the continuous model:
[0145]
[0146] In step S3, three constraint functions are designed to improve the generation quality of the model: common sense answerability, content relevance consistency, and expression grammatical accuracy.
[0147] The construction process of the common sense answerability constraint function is as follows: extract the most relevant paragraph sentences to the question by matching similarity, extract entities such as verbs and noun phrases from the answers and the extracted sentences using the Spacy toolkit, and group the results of the question into pairs. By introducing the learnable parameter ω1, the entity constraint knowledge is expressed as If the kth th Entity pairs are semantically similar, then vice versa, is the weight of each entity pair obtained from the attention network, where the network parameter is ω1; through this representation, when there are no similar entities between the sentences related to the answer and the question, then f 1 (·) is positive; otherwise, it is negative;
[0148] The process of constructing the content association consistency constraint function is: using a data-driven classifier, f 2 (a,c,y)=F(v c ,v y ;ω2), where v c represents the entity extracted from paragraph c, v y is the entity extracted from question y, ω2 is a parameter. When the two pairs of entities are semantically similar, the function F outputs a positive value, otherwise it outputs a negative value. By referring to the performance ranking of the evaluation dataset, the best performing question-answering model, the Unicorn model, is selected to predict the answer. By penalizing samples with inconsistent answers, the model can be encouraged to generate consistent answers, including in It is a judgment function that verifies the consistency between the predicted result and the labeled answer;
[0149] The construction process of the grammatical accuracy constraint function is to measure the fluency of the generated results by calculating the perplexity of the language model. Among them, P LM It is based on the pre-trained Roberta language model, where K is the number of question words. For the same answer, the generated result is often similar to the annotated result in terms of semantic structure and grammatical pattern. The word movement distance WMD is used to measure the semantic similarity between two texts. 5 (a,c,y)=WMD(y,y * ) / Length(y), where Length(·) is a normalized function, y * is the result of annotation; the similarity of grammatical structure is calculated by dependency syntax tree (DPTS), and the number of common substructures between two grammatical analysis trees is calculated by attention vector tree kernel (ACVT), so as to calculate the relevance of grammar, f 6 (a,c,y)=DPTS ACVT (y,y * ).
[0150] In step S3, the KL divergence optimization objective function in formula (6) is regarded as a knowledge distillation problem, that is, transferring knowledge from the teacher model d with constraints to the student's question generation model p θ Therefore, this objective function is solved using the expectation-maximization EM algorithm; in the t-th expectation calculation step, by formula Calculate the probability distribution d of the teacher model; then, calculate the maximum expectation, that is, update the probability distribution θ of the student model through formula (8) to approximate the probability distribution d of the teacher model, where is a trade-off factor, o is the labeled question probability distribution, and E is the expected accumulated error. This distillation objective can achieve a balance between the soft prediction of simulation d and the prediction of the true result;
[0151]
[0152] In addition to the student's question generation model θ, we also need to learn the parameters of the constraint f and its confidence The target in formula (6) It is found that when y is the labeled result, the constraint expectation h(a,c,y;ω)=exp{δ·∑ l λ l f l (a,c,y;ω l )} should be larger; h(·) is regarded as a likelihood function indicating the quality of the result, which makes the target similar to the variational lower bound of the corresponding model, which is achieved by labeling the result h based on the mean square error loss MSE. * To train ω, refer to formula (9). In addition, the constraint confidence λ is learned according to the probability distribution d of the teacher model, refer to formula (10):
[0153]
[0154]
[0155] The same or similar reference numerals correspond to the same or similar components;
[0156] The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.
[0157] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. A person skilled in the art would be able to make other variations or modifications based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the claims of the present invention.< / eos> < / eos> < / eos>
Claims
1. A reading test question generation system based on common sense reasoning, characterized by: include: The reasoning clue graph extraction module is used to infer the reasoning clue graph based on the text context starting from the given answer. It identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph. It selects the question content from the entity graph and retrieves relevant entities from the external common sense database to expand the entity graph to form the reasoning clue graph. A graph-guided question generation model module that combines all text and entity graphs, as well as multi-hop and commonsense knowledge in the entity graph, to generate high-quality exam questions; The posterior constraint learning module of language knowledge is used to train the graph-guided question generation model to learn the optimal model parameters.
2. A method used in the common sense reasoning-based reading test question generation system as claimed in claim 1, characterized in that: The following steps are involved: S1: The reasoning clue graph extraction module starts with a given answer and infers a reasoning clue graph based on the text context: it identifies all entities and relationships in the input text and uses contextual dependencies to build an entity graph; it selects the question content from the entity graph and retrieves related entities from an external common sense database to expand the entity graph to form a reasoning clue graph; S2: The graph-guided question generation model module combines all text and entity graphs, as well as multi-hop and common sense knowledge in the entity graph to generate high-quality test questions; S3: The posterior constraint learning module of language knowledge trains the graph-guided question generation model to learn the optimal model parameters.
3. The method for generating reading test questions based on common sense reasoning according to claim 2, characterized in that: In step S1, the reasoning clue graph extraction module represents each input text sentence as a parse tree, where each parse tree node contains several entities and edges, where the edges represent contextual relationships, filters out punctuation marks and stop words in each node, aggregates equivalent nodes and co-reference nodes in the parse tree; and adds connecting edges between trees between similar nodes in adjacent sentences to obtain an entity graph with potential clues.
4. The method for generating reading test questions based on common sense reasoning according to claim 3, characterized in that: In step S1, the process of selecting question content from the entity graph is: Mark two sentences that are relevant to the answer and remove the rest from the entity graph: Sentences containing answer keywords are identified by exact word matching or related sentences are identified using the classifier based on the Rouge-L similarity metric, where keywords are the remaining words after filtering stop words from the answer.
5. The method for generating reading test questions based on common sense reasoning according to claim 4, characterized in that: In step S1, the method of retrieving relevant entities from the external common sense database to expand the entity graph is: using the entity of the input text as the query condition, and retrieving relevant entities from the external open source common sense database through word matching.
6. The method for generating reading test questions based on common sense reasoning according to claim 5, characterized in that: In step S2, a graph-enhanced encoder is designed to combine all text and entity graphs to fuse the heterogeneous features of text and graphs. The graph-enhanced encoder consists of six cascaded layers, each of which includes: Text self-attention sublayer: responsible for encoding the text content and performing nonlinear transformation on the given text input representation vector to obtain a new vector as the input of the next sublayer: Given a text input X=(x1,…x T ), where x is the distributed embedding representation of each word, and each sub-layer is linearly transformed to obtain three vectors, including the key vector k i =x i W K , value vector v i =x i W V and query vector q i =x i W Q , where W K 、W V and W Q is a learnable matrix parameter; then, the interaction score r between the query vector and the key vector is calculated by the dot product ij ,Right now The scores are normalized by the logistic regression function Softmax, and the attention coefficient α is calculated by formula (1) ij , where d x represents the dimension of the key vector; by weighted summing of the attention and value vectors, we can get the context-aware output z i The first sub-layer is initialized by the representation vector of the input text, where the corresponding vector is retrieved from the pre-trained vector library for each word in the text, and the vectors of each word are combined into a vector to represent the text; the output of the last layer is collected and To represent the input text paragraph and the answer respectively: Graph attention sublayer: Considering that the nodes of the graph contain multiple words, Characterize each node, where is the distributed embedding representation of the jth word of the i-th node, m and n are the start and end positions of the text segment in the node, respectively. Represents the attention distribution of the node, which is used to indicate the importance of the node; Defined as softmax(ReLU(W R [G;w j ])), where g i is the i-th column of matrix G, W R is a trainable parameter; subsequently, the contextual representation of the node is enriched by weighted aggregation of the relevant semantics of adjacent nodes, where the weights are dynamically determined by the attention mechanism; in order to obtain this structural context information, the present invention obtains the relevance score of the edge by calculating the dot product between adjacent nodes i and j, Among them, τ ij Represents the relationship between nodes, which is learned from the corresponding relationship type. and It is a trainable parameter; by normalizing the relevant scores of all connected edges of the node, the attention of each node can be calculated Refer to formula (2), where Represents the neighboring nodes of node i; by weighted summing the attention, the output of graph structure perception is obtained where d x is the dimension size of the key vector, and Represents learnable parameters: Feedforward sublayer: transform the text vector z i and graph vector Since splicing will introduce a lot of noise, a gating mechanism is used to obtain significant features and reduce noise, as shown in formula (3), where ⊙ represents element multiplication, f is a fusion vector, and η is a learnable gate used to selectively control features at different angles: With i =η⊙f+(1-η)⊙z i ; In addition, in order to enhance the generalization ability of the model, a nonlinear transformation is introduced, which is implemented by a double-layer multilayer perceptron MLP with RELU activation function. i =max(0,z i W1+b1)W2+b2, where W1, W2, b1 and b2 are trainable parameters, and the output code h obtained by this transformation i It is regarded as the input of the next layer; after multiple layers of operation, the final representation is obtained.
7. The method for generating reading test questions based on common sense reasoning according to claim 6, characterized in that: In step S2, the output matrix h of the last layer of the graph enhancement encoder is regarded as a representation vector that integrates all input contents, and another converter is used to perform autoregressive decoding to generate test questions; the test questions are based on probability Generated word by word, similar to the encoder of graph enhancement, the decoder consists of multiple layers cascaded, at step t, the lth th Output of the layer Two parts are considered, including the The decoding result from attention, and the encoding content from The attention representation of These two parts are fused through a multi-head attention layer and expressed as After multiple layers of nonlinear transformation, the output vector can be obtained By normalizing the vector on the predefined vocabulary V through the logistic regression function Softmax, the output probability of the word can be obtained Where W o and b o Represents trainable parameters; In order to solve the problem of generating new words outside the vocabulary, a copy mechanism is used according to the distribution Generate test questions by copying new words from the input text, where α is the attention of the input text; define a balance factor By sampling this distribution, we can generate the words of the test questions one by one, where k is the balance factor, f(·) is a feedforward neural network with an S-type activation function Sigmoid, and y t-1 Represents the distributed embedding representation vector of the word generated in the t-1th step; in order to avoid the semantic drift problem, that is, the problem that the answer is inconsistent with the question, the answer encoding is used To initialize the decoder; a special marker is also introduced <eos> To indicate when the generation process is terminated:< / eos> 8. The method for generating reading test questions based on common sense reasoning according to claim 7, characterized in that: In step S3, supervised learning is used to train the graph-guided question generation model, that is, the optimal model parameters are learned by maximizing the log-likelihood probability, referring to formula (5), where Represents a training set containing N samples, K represents the size of the words in the test questions, and through a supervised teacher-guided learning model, the results generated by the model are close to the manual annotations in the samples: However, this approach cannot guarantee that the generated results are test questions that can be reasoned with common sense. To solve this problem, the present invention introduces a series of linguistic knowledge as regularization constraints to regulate the output probability distribution of the results, where regularization is achieved by satisfying the expected distribution d(·) of the constraints and the output distribution p of the model. θ (·) is realized by the KL divergence between them; a hybrid objective is adopted to fuse the supervision loss and the posterior loss into formula (6), where Is of the form The constraint set is b; φ(·) is the constraint feature function bounded by b; a, c, y represent the answer, text paragraph, and question respectively; λ is a parameter that weighs the confidence: Since the above optimization objective is a convex function, it has a closed-form solution, refer to formula (7), where Z is the normalization factor, δ is the regularization factor, and f j (a,c,y)=b j -φ j (a, c, q) represents the constraint function; that is, when (a, c, y) satisfies the constraint, f j (·)>0; optimal distribution d * (·) Not only close to the distribution p learned from the labeled training data θ (·), and most of the constraints are satisfied. This posterior regularization can flexibly inject discrete constraint knowledge into the continuous model:
9. The method for generating reading test questions based on common sense reasoning according to claim 8, characterized in that: In step S3, three constraint functions are designed to improve the generation quality of the model: common sense answerability, content relevance consistency, and expression grammatical accuracy. The construction process of the common sense answerability constraint function is as follows: extract the most relevant paragraph sentences to the question by matching similarity, extract entities such as verbs and noun phrases from the answers and the extracted sentences using the Spacy toolkit, and group the results of the question into pairs. By introducing the learnable parameter ω1, the entity constraint knowledge is expressed as If the kth th Entity pairs are semantically similar, then vice versa, is the weight of each entity pair obtained from the attention network, where the network parameter is ω1; through this representation, when there are no similar entities between the sentences related to the answer and the question, then f 1 (·) is positive; otherwise, it is negative; The process of constructing the content association consistency constraint function is: using a data-driven classifier, f 2 (a,c,y)=F(v c ,v y ;ω2), where v c represents the entity extracted from paragraph c, v y is the entity extracted from question y, ω2 is a parameter, when the two pairs of entities are semantically similar, the function F outputs a positive value, otherwise it outputs a negative value; By referring to the performance rankings of the evaluation dataset, we selected the Unicorn model, the best question-answering model currently, to predict the answer. By penalizing samples with inconsistent answers, we can encourage the model to generate consistent answers, including in It is a judgment function that verifies the consistency between the predicted result and the labeled answer; The construction process of the grammatical accuracy constraint function is to measure the fluency of the generated results by calculating the perplexity of the language model. Among them, P LM It is based on the pre-trained Roberta language model, where K is the number of question words. For the same answer, the generated result is often similar to the annotated result in terms of semantic structure and grammatical pattern. The word movement distance WMD is used to measure the semantic similarity between two texts. 5 (a,c,y)=WMD(y,y * ) / Length(y), where Length(·) is a normalized function, y * is the result of annotation; the similarity of grammatical structure is calculated by dependency syntax tree (DPTS), and the number of common substructures between two grammatical analysis trees is calculated by attention vector tree kernel (ACVT), so as to calculate the relevance of grammar, f 6 (a,c,y)=DPTS ACVT (y,y * ).
10. The method for generating reading test questions based on common sense reasoning according to claim 9, characterized in that: In step S3, the KL divergence optimization objective function in formula (6) is regarded as a knowledge distillation problem, that is, transferring knowledge from the teacher model d with constraints to the student's question generation model p θ Therefore, this objective function is solved using the expectation-maximization EM algorithm; in the t-th expectation calculation step, by formula Calculate the probability distribution d of the teacher model; then, calculate the maximum expectation, that is, update the probability distribution θ of the student model through formula (8) to approximate the probability distribution d of the teacher model, where is a trade-off factor, o is the labeled question probability distribution, and E is the expected accumulated error. This distillation objective can achieve a balance between the soft prediction of simulation d and the prediction of the true result; In addition to the student's question generation model θ, we also need to learn the parameters of the constraint f and its confidence The target in formula (6) It is found that when y is the labeled result, the constraint expectation h(a,c,y;ω)=exp{δ·∑ l λ l f l (a,c,y;ω l )} should be larger; h(·) is regarded as a likelihood function indicating the quality of the result, which makes the target similar to the variational lower bound of the corresponding model, which is achieved by labeling the result h based on the mean square error loss MSE. * To train ω, refer to formula (9). In addition, the constraint confidence λ is learned according to the probability distribution d of the teacher model, refer to formula (10):
Citation Information
Patent Citations
Reasoning-based difficulty controllable problem generation method
CN111563146A
Inference question-answering method, device and equipment
CN111597314A
Question answer extraction method based on multi-layer perception and electronic device
CN112380326A
Semisupervised autoencoder for sentiment analysis
US20180165554A1
Method and apparatus for automatically generating inference questions and answers
WO2021184311A1
