A method and system for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency

Abstract

The present invention discloses a semantic recognition and reasoning method and system for mathematical word problems with enhanced grammatical dependency. The system extracts information from the question text through a tree structure, integrates the local and global semantics of the question text, accurately recognizes the details of the mathematical word problem, effectively describes the semantics and details of the question, and improves the effect of practical application of intelligent education. In addition, the semantic vector generated by the encoder can provide certain technical support for the practice of intelligent education in the field of providing better question representation, automatic question labeling, and personalized question recommendation for many educational platforms, and can also bring certain potential economic benefits.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning, artificial intelligence and intelligent education technology, and in particular to a semantic recognition and reasoning method and system for mathematical word problems with enhanced grammatical dependency. Background Art

[0002] In intelligent education, automatic reasoning and solving of math word problems is a challenging task. Mathematical word problems contain a story described in natural language and a related math problem. The title of the math word problem is given in natural language, and the answer is obtained by mathematical reasoning. Therefore, solving math word problems requires algorithms to have the ability to understand the semantics of natural language and mathematical reasoning.

[0003] In existing methods, when reasoning about math word problems, the logical structure of the mathematical expression is combined to convert the problem into a computable mathematical expression. However, these works only focus on how to make computers perform mathematical reasoning, but do not have a deep understanding of the semantics of the question text, ignore the grammatical dependencies between the elements in the question (such as quantity and the modified entity), and find it difficult to accurately identify the details of the math word problem, and cannot effectively describe the semantics and details of the question, resulting in poor results in the actual application of intelligent education. Summary of the invention

[0004] The purpose of the present invention is to provide a semantic recognition and reasoning method and system for mathematical word problems with enhanced grammatical dependence, which can accurately identify the details of mathematical word problems, effectively describe the semantics and details of the problems, and improve the effect in the actual application of intelligent education.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A semantic recognition and reasoning method for mathematical word problems with enhanced grammatical dependency, comprising:

[0007] The question text is divided into multiple sentences, and a hierarchical word-sentence-question encoder is used to learn the semantic information representation of the question from local to global. The sentence semantics is enhanced by using the intra-sentence association. A grammatical dependency tree is established to save the question text structure and model the structural dependency between the elements in the sentence, and finally the representation of the global semantic information corresponding to the question text is obtained.

[0008] A tree-structured decoder is used in combination with the representation of global semantic information to recursively generate the mathematical expression corresponding to the question text.

[0009] It can be seen from the technical solution provided by the present invention that, compared with the traditional serialization processing method, the semantics and details of the question can be described more effectively, thereby improving the reasoning accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0011] Figure 1 A diagram describing a mathematical word problem provided by an embodiment of the present invention;

[0012] Figure 2 A framework diagram of a semantic recognition and reasoning method for mathematical word problems with enhanced grammatical dependency provided by an embodiment of the present invention;

[0013] Figure 3 A framework diagram of a sentence representation module with enhanced grammatical dependency provided in an embodiment of the present invention;

[0014] Figure 4 A schematic diagram of a semantic recognition and reasoning system for mathematical word problems with enhanced grammatical dependency provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0016] The embodiment of the present invention provides a semantic recognition and reasoning method for mathematical word problems with enhanced grammatical dependency, which mainly includes:

[0017] The question text is divided into multiple sentences, and a hierarchical word-sentence-question encoder is used to learn the semantic information representation of the question from local to global. The sentence semantics is enhanced by using the intra-sentence association, and a grammatical dependency tree is established to save the question structure and model the structural dependency between the elements in the sentence, finally obtaining the representation of the global semantic information corresponding to the question text;

[0018] A tree-structured decoder is used in combination with the representation of global semantic information to recursively generate the mathematical expression corresponding to the question text.

[0019] The above scheme of the embodiment of the present invention uses clauses as semantic units, uses a grammatical dependency tree to represent the sentence structure and hierarchically models the semantic information of the question text, and uses a tree-structured recursive sequence model to capture the structural dependency relationship between the elements of the mathematical expression (such as numbers, operators). By integrating the local details of the question text, the global goal, and the dependency relationship between the elements, the above scheme improves the accuracy of the algorithm's mathematical reasoning.

[0020] The above scheme actually covers two stages:

[0021] The first stage is the question understanding stage: simulating human reading and comprehension habits, the question text is divided into multiple sentences, and then a hierarchical word-sentence-question encoder is designed to learn the semantic information representation of the question text from local to global. Furthermore, in order to enhance the sentence semantics by using the intra-sentence association, in the sentence understanding stage, a grammatical dependency tree is established to save the question text structure and a module based on the dependency tree is designed to model the structural dependency between the elements in the sentence.

[0022] The second stage is the reasoning stage: a tree-structured decoder is proposed to recursively generate mathematical expressions. First, a hierarchical attention mechanism is used to explicitly integrate information at different levels to enhance semantics and extract relevant contextual information. Then, a pointer-generation network is used to guide the model to extract existing information and infer additional knowledge to predict each position symbol.

[0023] For ease of understanding, the above solution is described in detail below with reference to the accompanying drawings.

[0024] In the implementation of the present invention, mathematical word problems in a broad sense are used as data sets. Figure 1 As shown in the figure, it is a description diagram of a math word problem; the text content of the problem needs to contain natural text (such as words) used to describe the problem information and numerical values ​​(such as Figure 1 "3" and "4"), each data includes the title text (such as Figure 1 "Problem" part), mathematical expressions (such as Figure 1 in the "Expression" section) and numerical answers (such as Figure 1 The “Answer” part in the question is required for this solution, the mathematical expression is only used for the training of this solution, and the numerical answer is only used for the effect evaluation of this solution. In actual use, only the question text is needed. Examples of such data are the open source math word problem dataset (Math23K) released by Tencent and the open source dataset (MAWPS) released by Microsoft. In addition, the input dataset can also be obtained by crawling the web or collecting math problem sets for primary and secondary school students' homework or exams offline.

[0025] The purpose of this invention is to model the semantic information of the text, so as to perform deep reasoning and generate mathematical expressions. In this stage, the semantic vector can provide better test question representation, automatic test question annotation and personalized test question recommendation services to many educational platforms, bringing certain potential economic benefits. At the same time, the answer can be solved according to the generated mathematical expression.

[0026] In the embodiment of the present invention, the text P of each question is represented as a sequence P consisting of n words or values ​​= {p1, p2, ..., p n}, where p s are all words in the title text (such as Figure 1 ”length” and “width” in the question text) or a value (such as “3” and “4”), s=1,…,n; the value set of the question text P is defined as N P (For example: {3, 4}). Since the specific values ​​of these numbers are not concerned in the present invention, all numerical values ​​can be mapped to the special word "NUM". Therefore, the p in the above sequence P is P Can be understood as a word.

[0027] The expression E corresponding to question P P It can be defined as a sequence E consisting of G operators, constants, or numerical values ​​in the question text. P ={y1,y2,…,y G}, where each y i All of them are the decoding symbol set V of question P P An element of the symbol set V P By the operator set V O (e.g. {+,-,×,÷}), constant set V C (e.g. {1,2,π}) and a set of values ​​N P Composition, defined as V P =V O ∪V C ∪N P (e.g. {+,-,×,÷,1,2,π,3,4}), since N in each question P Different, each question's V P Also different.

[0028] Given an input sequence of question P, the goal of semantic recognition and reasoning of mathematical word problems is to learn a model that reads the token of P as input and predicts the corresponding expression through semantic recognition and reasoning.

[0029] The embodiment of the present invention constructs a network model to perform semantic recognition and reasoning of mathematical word problems.

[0030] The network model needs to be trained, and the training data is a pre-collected data set. In order to ensure the effect of the model, the collected data set needs to be preprocessed: 1) Data filtering: The example of the present invention is mainly aimed at the application scenario where the mathematical word problem contains only one question, the answer to the question is a numerical value and can be directly calculated by a mathematical expression. Therefore, it is necessary to select questions that only contain one mathematical expression and a numerical answer in the data set, and filter out questions that lack expressions or have two or more expressions and answers. 2) Sampling: Random sampling is performed in each data set, and a subset of the original data set is selected to train the model.

[0031] In addition, similar to human problem solving, semantic recognition and reasoning of math word problems require machines to have two key capabilities: natural language understanding to understand the relationship between multiple elements in the semantics of the question text, and logical reasoning to generate correct mathematical expressions through fine-grained logical reasoning. Most existing research works only focus on mathematical reasoning to generate expressions, but simply model the question text as a sequence of words and only focus on the direct impact of the relative position of words (e.g.: Figure 1 In the example above, “longer” comes after “cm” and before “than”, but ignores the complex structure between words that is highly related to semantics (e.g., the relationship between “3”, “longer”, and “width”). This is inconsistent with the way humans accurately understand the question text, and lacks the understanding of the deep semantics of the question text and the accuracy of reasoning. In fact, there are many difficulties and challenges in imitating human reading habits to accurately understand the semantics of the question text and improve the reasoning effect: (1) Humans usually read each part of the question text (e.g., sentence) first to pay attention to local details (e.g., Figure 1 ), and then combine them to obtain the global target (e.g., the perimeter of a rectangle). How to simulate this process is a challenging problem; (2) Humans can easily grasp the semantic dependencies between words to understand their detailed descriptions and local semantics (e.g., Figure 1 (3) In reasoning, how to convert local logic into corresponding expressions (e.g., “3cmlonger” in the question text corresponds to “+3” in the expression) is an important task. (4) Problem solving may require human mathematical knowledge outside the question text (e.g., Figure 1 In addition to the "length" and "width" in the question text, the rectangle perimeter formula "2×(NUM+NUM)" is also required, where the "2" cannot be directly generated from the question text). How to use this knowledge is still an unsolved problem.

[0032] The network model provided by the embodiment of the present invention can better simulate the human reading process to improve semantic comprehension ability and enhance reasoning effect. The entire method of the present invention is a network model that understands mathematical word problems and reasoning mathematical expressions based on a sequence-to-sequence (Seq2Seq) framework, such as Figure 2-Figure 3 As shown in the figure, the network model mainly consists of three parts: a hierarchical encoder module, a dependency-enhanced clause representation module and a tree-based decoder module.

[0033] The hierarchical encoder module is responsible for dividing the question text into sentences and learning the semantic representation of the question text from local to global. The sentence representation module with enhanced grammatical dependency is responsible for combining the structural dependency within the sentence to learn the representation of each sentence for the hierarchical encoder module. The tree-structured decoder module recursively generates mathematical expressions based on the semantic representation of the question text through a tree-structured neural network, and uses a hierarchical attention mechanism to integrate semantic information at different levels and a pointer-generation network to guide the model to extract existing information and derive additional knowledge, thereby accurately predicting the symbols of each node. The specific introduction of each module is as follows:

[0034] 1. Hierarchical encoder module.

[0035] Hierarchical encoder modules such as Figure 2 As shown in the dashed box above, given a sequence of questions P = {p1, p2, ..., p n}, imitate the human habit of reading and understanding from local to global, and divide the title text into m sentences according to commas and periods As a unit of local semantic representation, each clause C k is a word subsequence of the topic text P (including several p s ), k = 1, ..., m, m is the number of sentences. Then, based on the sentence division, the encoder sequentially learns the semantic representation of words, sentences and the entire question text in a bottom-up approach from local to global:

[0036] 1. Context-enhanced word representation.

[0037] At the word level ( Figure 2 "Word-Level" layer), each word p s Mapped to word vector x of equal dimension s(The word vector is pre-trained by word2vec technology), and each word p is obtained by a bidirectional gated recurrent unit (Gated Recurrent Unit, GRU). s The semantic representation of h s :

[0038]

[0039] Among them, GRU f and GRU b They are gated recurrent units that model previous and subsequent context information, and are the words p containing the previous and following information respectively. s The semantic representation of and Calculate for GRU separately Semantic representation and calculation of the previous word required The semantic representation of the next word required, so the semantic representation h s Contains the word p s The semantic information itself and the contextual information in both the previous and next directions.

[0040] 2. Sentence representation enhanced by grammatical dependency.

[0041] At the clause level ( Figure 2 According to the division of clauses, extract each clause C k Each word p s The semantic representation of h s , using the intra-sentence association to enhance the sentence semantics, building a grammatical dependency tree to save the question structure and model the structural dependency between the elements in the sentence, and combining the structural dependency relationship to obtain the sentence C k Semantic representation of This stage is implemented by the sentence representation module enhanced with grammatical dependencies, which will be explained in detail later.

[0042] 3. Question text representation with enhanced correlation between clauses.

[0043] At the topic level ( Figure 2 The "Problem-Level" layer), based on each clause C k Semantic representation of Combine the semantic dependencies between clauses (descriptions of different aspects of the same entity, such as Figure 1 The value of "width" in the first clause and the relationship between "width" and "length" in the second clause) enhances the clause semantics and obtains the representation of the global semantic information corresponding to the question text.

[0044] First, for each clause C k Semantic representation of The learnable positional encoding PE is used to model the order relationship between sentences and obtain the position-enhanced sentence representation.

[0045]

[0046] Among them, PE(k) is the encoding vector representing the position of the kth clause. Then, the self-attention mechanism is used to extract relevant semantic information from other clauses to enhance the clause semantics: Use neural network to model the clause representation With each other clause representation The correlation between as :

[0047]

[0048] Among them, ReLU is the linear rectification function, [.] represents the concatenation operation of multiple vectors, T is the vector transposition operation, and W ss , W sa and b sa is the learnable parameter of the linear module of the neural network (W is the weight parameter and b is the bias parameter). During model training, the training data is fitted by adjusting the content of the learnable matrix / vector in the linear module.

[0049] Those skilled in the art can understand that a neural network includes a linear module and a nonlinear module. The linear module includes a weight parameter W and a bias parameter b, and the nonlinear module mainly includes ReLU. The structures of the neural networks involved in various parts of the present invention may be the same or different.

[0050] Representation from each other clause based on the correlation between clauses Extract relevant semantic information according to certain weights

[0051]

[0052]

[0053] Among them, s is the index of each sentence when extracting semantic information, α s is the weight of extracting semantic information from the sth sentence, and k′ is the sentence index required to calculate the weight.

[0054] Through relevant semantic information Enhanced Clause C k Semantic representation of Get a sentence representation that contains local semantic information within the sentence and global semantic information of other sentences

[0055]

[0056] Among them, W so and b so is the learnable parameter of the linear module of the neural network, with the last clause C m The semantic representation of the title text is used as the representation of the global semantic information corresponding to the title text. The last clause is usually the problem part of the word problem, that is, the goal of solving the word problem.

[0057] 2. Sentence representation module with enhanced grammatical dependency.

[0058] This module can be understood as an independent module, and can also be understood as an internal submodule of the hierarchical encoder module. Both understandings do not affect the implementation of the present invention.

[0059] like Figure 3 As shown in Figure 1, it is a framework diagram of the sentence representation module with grammatical dependency enhancement. Given each sentence C k and each word p in the clause s The semantic representation of h s , through the syntax analysis tool to establish each clause C k The grammatical dependency tree T k ={t k1 ,t k1 ,…,t kL}( Figure 3 The upper left corner captures the grammatical dependencies between words in a clause, the dependency tree T k Each node t kl Both represent a word in the clause. s , kl=k1,…,kL, L is the clause C k The number of words, k1~kL represent the dependency tree T of the kth clause respectively. k The index of each node in the node; define the parent-child relationship between nodes, the child node depends on the parent node, and provide detailed descriptions for the parent node; such as: Figure 3 Here, "rectangle" depends on the parent node "perimeter" and describes the type of "perimeter".

[0060] In the embodiment of the present invention, based on the grammatical dependency tree T k , using the details provided by child nodes to enhance the semantics of the parent node.

[0061] For leaf node t f (e.g., “rectangle”), f∈[k1,kL], and the corresponding word represents h f As an enhanced node semantic representation

[0062] For non-leaf nodes t p (e.g.: “perimeter”), p∈[k1,kL], with the semantic representation of the corresponding word h p Initialize the node semantic representation, and extract relevant details from the child nodes through the attention mechanism to enhance the node semantic representation:

[0063] First, calculate the non-leaf node t p With each of its child nodes t c Enhanced node semantic representation The semantic relevance S ac :

[0064]

[0065] Where W cs , W ca and b ca is the learnable parameter of the linear module of the neural network, according to the semantic relevance S ac Extract relevant detail semantics from child nodes according to certain weights

[0066]

[0067] Among them, c is the index of each child node when extracting detailed semantics, α c To extract the weight of the detailed semantics from the cth child node, p′ is the index of the child node required to calculate the weight.

[0068] The detailed semantics of the child nodes are integrated with the initialized node semantic representation to obtain an enhanced node semantic representation.

[0069]

[0070] Among them, W co and b co is the learnable parameter of the linear module of the neural network, based on the grammatical dependency tree T k The structure of the node is to calculate the enhanced node semantic representation of each node in a bottom-up order from leaf node to root node; finally, the root node (such as Figure 3 Enhanced node semantic representation of "is" in It contains the local semantics of all nodes in the clause and the structural information of the grammatical dependency tree, with the enhanced node semantic representation of the root node As a local semantic representation of clauses

[0071] 3. Tree-structured decoder module.

[0072] The tree-based decoder module is Figure 2 As shown in the two dotted boxes below, its input is the semantic representation of each level of the question text P. Since each valid mathematical expression can be uniquely converted into a corresponding expression tree, the decoder converts the prediction and generation of mathematical expressions into the prediction and generation of expression trees. First, through a tree-structure-based method, each node of the required expression tree is recursively predicted and generated in order from the root node to the leaf node from top to bottom. (including symbols on nodes ), forming a complete and valid expression tree The symbol on each leaf node in the expression tree is a predicted constant or a value in the question text, and the symbol on each non-leaf node is a predicted operator, indicating the operation result of the two child nodes; then, the symbols on each node are arranged in the order generated. Get with Equivalent mathematical expressions with the required prefix

[0073] In the embodiment of the present invention, each node Contains the corresponding symbols Target vector q p With the context vector c p ; The target vector of the root node is represented by the global semantic information corresponding to the title text h s Initialization, the target vectors of the remaining nodes are generated by their parent nodes, and the context vector c p Through the target vector q p Combined with the hierarchical attention mechanism, the symbol of the node Then the target vector q of the node is p With the context vector c p Predicted, the symbols of all predicted nodes are combined and arranged in order to get the prefix mathematical expression The preferred implementations of the above parts are as follows:

[0074] 1. Target vector generation.

[0075] The target vector of the root node is represented by the global semantic information corresponding to the title text h s Initialization, the target vectors of the remaining nodes are generated by their parent nodes. The process of the parent node generating the target vector of the child node is as follows: For any parent node The target vector is denoted as q p , considering the node generation order, q p Already by The decoder first generates the parent node according to the target vector q p Generate Node The context vector c P and predict its sign (This process will be described below), if the predicted symbol is an operator, then it generates The two child nodes of the target are decomposed (this process is called node expansion). When the target is decomposed, the target vector q p Generate target vectors for two child nodes: First, pass the gated neural network according to the node The target vector q p , context vector c p ,symbol The word vector of the left child node generates the target vector q l And perform symbol prediction and expansion of the left child node, and this process is performed recursively; after all predictions of the left subtree are completed, the gated neural network is used to generate the semantic representation vector t of the partial mathematical expression corresponding to the left subtree in a bottom-up manner using all the nodes on the left subtree l , t l It will be used to generate the target vector of the right child node of the parent node; then according to Target vector q p , context vector c p ,symbol Word vector and semantic representation vector t of the left subtree l Generate the target vector q of the right child node through the gated neural network r And predict and expand the right child node. When the symbol predicted by a node is not an operator (constant or value), the target decomposition process on the node ends, and the node can correspond to the left subtree of a parent node (if the node is a left child node, it corresponds to the left subtree of its parent node; if the node is a right child node, it corresponds to the left subtree of a higher-level parent node), which means that the left subtree is fully predicted, and the parent node is located by backtracking the target decomposition process. According to the above process, the semantic representation of the left subtree is established, and the prediction and expansion of the right subtree of the parent node are started.

[0076] 2. Context extraction.

[0077] Node-based target vector q p , through the hierarchical attention mechanism, the relevant local semantic information is explicitly extracted from the semantic representations of different levels of the topic, and the context vector c that integrates the semantic information of relevant sentences and words is obtained p .

[0078] First, evaluate the target vector q p Each sentence representation in the title text The correlation S c :

[0079]

[0080] Among them, W sc , W ac and b ac is the learnable parameter of the linear module of the neural network, then, evaluate the target vector q p Each clause of the title text C k The semantic representation h of each word in k,t (like Figure 2 As shown by the arrow, the semantic representation of the word is obtained by the word layer of the hierarchical encoder module) w :

[0081]

[0082] Among them, W sw , W aw and b aw is the learnable parameter of the linear module of the neural network, based on the target vector q p With each clause C k The correlation between the related clauses and words is extracted from different clauses with different weights to generate a context vector c that integrates the semantic information of the related clauses and words. p :

[0083]

[0084] in, To represent the k-th clause The weight of extracting semantic information from To represent h from the tth word in the kth clause k,t The weight of extracting semantic information from C, k is the index of the sentence when extracting semantic information from each sentence, k′ is the sentence index required to calculate the weight, and t is the sentence index from sentence C k The word index when extracting semantic information from each word, t′ is the sentence C required to calculate the weight k The word index in is .

[0085] 3. Symbol prediction.

[0086] Node-based target vector q p and the context vector c p , prediction symbol Specifically, based on the node's target vector q p , context vector c, through the pointer generator network to copy the numerical variables in the question text (such as: Figure 1The algorithm uses the knowledge of "3", "4" that appear in both the Problem and Expression parts of the expression, and also infers additional knowledge (constants and operators in the external symbol set, such as "2" and "×" that appear only in the Expression part) to predict the symbol of the node.

[0087] First, the symbol to be predicted It may be a numerical variable in the question text, or a constant or operator in the external symbol set. By calculating the following three probabilities, a value probability P is calculated for each possible value of the symbol to be predicted. c :

[0088] Calculate the symbol to be predicted The probability that is a constant or an operator in the external symbol set:

[0089] P gen =σ(w pg [q p , c p ]+b pg )

[0090] The first case: If the symbol to be predicted is a numerical variable from the question text, and the symbol to be predicted in this case is calculated The probability distribution of is:

[0091]

[0092] The second case: If the symbol to be predicted is a constant or operator from an external symbol set, and the symbol to be predicted in this case is computed The probability distribution of is:

[0093]

[0094] in, is the symbol to be predicted in the first case The word representation of the possible values ​​of is the symbol to be predicted in the second case The possible word vectors, W pg , b pg , W ps , W pa , b pa , W gs , W ga and b ga are the learnable parameters of the linear module of the neural network;

[0095] The total probability for:

[0096]

[0097] The model is the symbol to be predicted Calculate a probability P for each possible symbol value (problem variable, constant and operator) c During training, the model adjusts the learnable parameters to maximize the probability P of the correct symbol value (the real symbol in the training data) y c (y), making it more likely to be selected in the prediction; the probability of all possible symbol values ​​in the prediction Prediction node symbol Generally, the symbol with the highest probability is selected, or a symbol is selected based on probability using a beam search method.

[0098] 4. Expression generation.

[0099] After all nodes of the expression tree are predicted, the symbols of all predicted nodes are integrated to generate a mathematical expression in the form of a prefix expression according to the node position or the order in which the nodes are generated.

[0100] The above is the main principle of the above solution of the embodiment of the present invention. In addition, for the question text P and the corresponding target expression E in the data set P ={y1,y2,…,y G}, the model corresponding to the whole method is trained by minimizing the following loss function L:

[0101]

[0102] The loss function represents the sum of the negative logarithms of the probabilities of the true symbols calculated by the model at each position of the target expression, where each y represents a true symbol in the target expression, P c (y g |y1,y2,…,y g-1 ,P) means that given the topic text P and the first g-1 symbols y1,y2,…,y g-1 In the case of the g-th position, the model calculates the true symbol y g The probability of (obtained from the previous "symbol prediction" stage), so the purpose of training is to maximize the probability of the correct symbol (the real symbol in the training data) so that it is more likely to be selected in the prediction. When using the trained model for prediction, only the question text P is used, and the symbol predicted with the probability calculated by the model Generate mathematical expressions and evaluate the accuracy of model predictions using the corresponding true answers in the dataset.

[0103] Those skilled in the art will appreciate that the training phase can optimize the learnable parameters in the model according to the loss function, such as the weight parameter W and the bias parameter b in each neural network linear module mentioned above.

[0104] The above scheme of the present invention extracts information from the question text through a tree structure, integrates the local and global semantics of the question text, and enhances the semantic understanding and reasoning capabilities of the method; in addition, the semantic vector generated by the encoder can provide certain technical support for the practice of smart education such as better question representation, automatic question labeling and personalized question recommendation for many educational platforms, and can also bring certain potential economic benefits.

[0105] Another embodiment of the present invention also provides a hierarchical semantic recognition and reasoning system for mathematical word problems with enhanced grammatical dependency, such as Figure 4 As shown, it mainly includes: processing equipment and display equipment; wherein:

[0106] The processing device uses the method provided in the above embodiment to perform semantic recognition and reasoning of mathematical word problems;

[0107] The display device is used to display the results obtained at various stages in the process of reasoning and solving mathematical word problems.

[0108] In an embodiment of the present invention, the display device may be a touch screen, which can not only display the results obtained in each stage, but also output control commands to the processing device, for example, clicking a related button to perform data preprocessing, or clicking a related button to control model operation, etc.

[0109] In the embodiment of the present invention, the semantic recognition and reasoning process involved in the processing device has been described in detail in the previous embodiment, so it will not be repeated here.

[0110] Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0112] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A semantic recognition and reasoning method for mathematical word problems with enhanced grammatical dependency, characterized in that: include: The question text is divided into multiple sentences, and a hierarchical word-sentence-question encoder is used to learn the semantic information representation of the question from local to global. The sentence semantics is enhanced by using the intra-sentence association. A grammatical dependency tree is established to save the question text structure and model the structural dependency between the elements in the sentence, and finally the representation of the global semantic information corresponding to the question text is obtained. A tree-structured decoder is used in combination with the representation of global semantic information to recursively generate the mathematical expression corresponding to the question text; Among them, the intra-sentence association is used to enhance the sentence semantics, a grammatical dependency tree is established to save the question structure and model the structural dependency between the elements in the sentence, and the sentence C is obtained by combining the structural dependency relationship. k The semantic representation includes: Given each clause C k and each word p in the clause s The semantic representation of h s , through the syntax analysis tool to establish each clause C k The grammatical dependency tree T k ={t k1 , t k2 , ..., t kL } captures the grammatical dependencies between words in a clause, dependency tree T k Each node t kl Both represent a word in the clause. s , kl=k1,...,kL,L is the clause C k The number of words; define the parent-child relationship between nodes, the child node depends on the parent node, and provides detailed descriptions for the parent node; For leaf node t f , f∈[k1,kL], with the corresponding word representation h f As an enhanced node semantic representation For non-leaf nodes t p , p∈[k1,kL], with the semantic representation of the corresponding word h p Initialize the node semantic representation, and extract relevant details from the child nodes through the attention mechanism to enhance the node semantic representation: First, calculate the non-leaf node t p With each of its child nodes t c Enhanced node semantic representation The semantic relevance S ac : Among them, W cs , W ca and b ca is a learnable parameter; According to semantic relevance S ac Extract relevant details from child nodes Among them, c is the index of each child node when extracting detailed semantics, α c is the weight of extracting the detailed semantics from the cth child node, and p′ is the index of the child node required to calculate the weight; The detailed semantics of the child nodes are integrated with the initialized node semantic representation to obtain an enhanced node semantic representation. Among them, W co and b co is a learnable parameter; Based on the grammatical dependency tree T k The structure of the enhanced node semantic representation of each node is calculated in a bottom-up order from leaf node to root node; finally, the enhanced node semantic representation of the root node It contains the local semantics of all nodes in the clause and the structural information of the grammatical dependency tree, with the enhanced node semantic representation of the root node As a local semantic representation of clauses The tree-structured decoder is combined with the representation of global semantic information and the local semantic representation of each sentence to recursively generate the mathematical expression corresponding to the question text, including: Through a tree-structured approach, each node of the required expression tree is predicted and generated recursively from the root node to the leaf node in order from top to bottom. Construct a complete and valid expression tree The symbol on each leaf node in the expression tree is a predicted constant or a value in the question text, and the symbol on each non-leaf node is a predicted operator, indicating the operation result of two child nodes; each node is arranged in the order of generation Symbols on Get the expression tree Equivalent mathematical expressions with the required prefix 2. The method for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency according to claim 1, characterized in that: The method of dividing the topic text into multiple sentences and using a hierarchical word-sentence-topic encoder to learn the semantic information representation of the topic from local to global includes: For the topic text P, it is represented as a sequence of n words P = {p1, p2, ..., p n }, where p s are all one word, s = 1, ..., n; divide the question P into m clauses according to commas and periods Each clause C k Contains several words p s , k = 1,…,m, m is the number of clauses; At the word level, each word p s Mapped to word vector x of equal dimension s , and obtain each word p through a bidirectional gated recurrent unit s The semantic representation of h s : Among them, GRU f and GRU b They are gated recurrent units that model previous and subsequent context information, and are the words p containing the previous and following information respectively. s The semantic representation of and Calculated for the gated recurrent unit The semantic representation of the previous word and the semantic representation of the next word required, the semantic representation h s Contains p s The semantic information of the text and the context information of the context. At the sentence level, according to the division of sentences, extract each sentence C k Each word p s The semantic representation of h s , enhance the semantics of clauses through intra-clause association, establish a grammatical dependency tree to save the question structure and model the structural dependency between elements in the clause, and combine the structural dependency relationship to obtain the clause C k Semantic representation of At the topic level, based on each clause C k Semantic representation of Combine the semantic dependencies between clauses to enhance the semantics of the clauses and obtain the representation of the global semantic information corresponding to the question text; first, for each clause C k Semantic representation of The learnable positional encoding PE is used to model the order relationship between sentences and obtain the position-enhanced sentence representation. Among them, PE(k) is the encoding vector representing the position of the kth sentence; Then, the self-attention mechanism is used to extract relevant semantic information from other clauses to enhance the clause semantics: Modeling inter-clause representation With each other clause representation The correlation S as : Where s = 1,…,m; ReLU is a linear rectification function, [.] represents a vector concatenation operation, T is a vector transposition operation, and W ss , W sa and b sa is a learnable parameter; Extract relevant semantic information from other clauses based on the correlation between clauses Among them, s is the index of each sentence when extracting semantic information, α s is the weight of extracting semantic information from the sth clause, and k′ is the clause index required to calculate the weight; Through relevant semantic information Enhanced Clause C k Semantic representation of Get a sentence representation that contains local semantic information within the sentence and global semantic information of other sentences Among them, W so and b so is a learnable parameter; With the last clause C m The semantic representation of the title text is used as the representation of the global semantic information corresponding to the title text.

3. The method for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency according to claim 1, characterized in that: Each node Contains the corresponding symbols Target vector q p With the context vector c p ; The target vector of the root node is represented by the global semantic information corresponding to the title text h s Initialization, the target vectors of the remaining nodes are generated by their parent nodes, and the context vector c p Through the target vector q p Combined with the hierarchical attention mechanism, the symbol of the node Then the target vector q of the node is p With the context vector c p Predicted.

4. A method for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency according to claim 3, characterized in that: Ways to generate a node target vector include: remember is the parent node, and the target vector is denoted as q p , according to the target vector q p Generate context vector c P and predict its sign If the predicted sign is an operator, two child nodes are generated and the target is decomposed. When the target is decomposed, the target vector q p Generate target vectors for two child nodes; first, use the gated neural network to calculate the target vector q of the node p , context vector c p ,symbol The word vector of the left child node generates the target vector q l The symbol prediction and target decomposition of the left child node are performed recursively to obtain the left subtree, and the semantic representation vector t of the partial mathematical expression corresponding to the left subtree is generated in a bottom-up manner through a gated neural network. l , and then according to the target vector q p , context vector c p ,symbol Word vector and left subtree semantic representation vector t l Generate the target vector q of the right child node through the gated neural network r And perform prediction and target decomposition of the right child node; when the symbol predicted by the node is not an operator, the target decomposition of the corresponding left subtree ends and the target decomposition of the right subtree begins.

5. A method for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency according to claim 3 or 4, characterized in that: Node-based target vector q p , through the hierarchical attention mechanism, the relevant local semantic information is explicitly extracted from the semantic representations of different levels of the topic, and the context vector c that integrates the semantic information of relevant sentences and words is obtained p , the steps include: Evaluate the target vector q p Each sentence representation in the title text The correlation S c : Among them, W sc , W ac and b ac is a learnable parameter; Evaluate the target vector q p Each clause of the title text C k The semantic representation h of each word in k,t The correlation S w : Among them, W sw , W aw and b aw is a learnable parameter; Based on the target vector q p With each clause C k The correlation between the related clauses and words is extracted from different clauses with different weights to generate a context vector c that integrates the semantic information of the related clauses and words. p : in, To represent the k-th clause The weight of extracting semantic information from To represent h from the tth word in the kth clause k,t The weight of extracting semantic information from C, k is the index of the sentence when extracting semantic information from each sentence, k′ is the sentence index required to calculate the weight, and t is the sentence index from sentence C k The word index when extracting semantic information from each word, t′ is the sentence C required to calculate the weight k The word index in .

6. A method for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency according to claim 3 or 4, characterized in that: Node-based target vector q p And the context vector c, the predicted symbol The steps include: Node-based target vector q p , context vector c, which replicates the numerical variables in the question text through a pointer-generating network, while inferring additional knowledge to predict the symbol of the node First, the symbol to be predicted For the numerical value in the question text, or the constant or operator in the external symbol set, a value probability P is calculated for each possible value of the symbol to be predicted by calculating the following three probabilities: c : Calculate the symbol to be predicted The probability that is a constant or an operator in the external symbol set: P gen =σ(W pg [q p ,c]+b pg ) The first case: If the symbol to be predicted is a numerical variable from the question text, calculating the symbol to be predicted The probability distribution of is: The second case: If the symbol to be predicted is a constant or operator from an external symbol set that computes the symbol to be predicted The probability distribution of is: in, is the symbol to be predicted in the first case The word representation of the possible values ​​of is the symbol to be predicted in the second case The possible word vectors, W pg 、b pg , W ps , W pa 、b pa , W gs , W ga and b ga are the learnable parameters of the linear module of the neural network; Then, calculate the total probability Finally, according to the probability of all possible values Symbols of prediction nodes 7. The method for semantic recognition and reasoning of mathematical word problems with enhanced grammatical dependency according to claim 1, characterized in that: For the question P and the corresponding target expression E P ={y1,y2,…,y G }, the model corresponding to the whole method is trained by minimizing the following loss function L: The loss function represents the sum of the negative logarithms of the probabilities of the true symbols calculated by the model at each position of the target expression, where each y represents a true symbol in the target expression, P c (y g |y1,y2,…,y g-1 ,P) means that given the topic text P and the first g-1 symbols y1,y2,…,y g-1 In the case of the g-th position, the model calculates the true symbol y g probability; When using the trained model to predict, only the title text P is used, and the predicted symbol Generates mathematical expressions.

8. A semantic recognition and reasoning system for mathematical word problems with enhanced grammatical dependency, characterized in that: Processing equipment and display equipment; wherein: The processing device uses the method described in any one of claims 1 to 7 to perform semantic recognition and reasoning of mathematical word problems; The display device is used to display the results obtained at various stages in the process of reasoning and solving mathematical word problems.

Images