A pseudo-dual learning method for automatically solving mathematical problems

Abstract

The present invention discloses a pseudo-dual learning method for automatically solving math problems. During the training and optimization process of the automatic solver for math problems, a pseudo-dual task is introduced to simulate the review process. That is, after obtaining the solution expression, the numbers in the expression are filled back into the original text problem description, thereby improving the automatic solver's ability to capture the number of math problems and its mathematical logic ability. In addition, during the joint training optimization process, the present invention also designs a plan fusion strategy, which uses the correct expression for number filling in the early stage of training, and smoothly transitions to using the predicted expression for number filling in the later stage of training, to solve the problem that the predicted expression is not ideal in the early stage of training and will mislead the pseudo-dual learning. At the same time, the plan fusion strategy can balance the learning of the solution process and the pseudo-dual filling process during the training process. Through experimental research, the present invention significantly improves the performance of multiple representative automatic solvers for math problems.

Description

Technical Field

[0001] The present invention belongs to the technical field of training automatic math problem solvers, and more specifically, relates to a pseudo-dual learning method for automatically solving math problems. Background Art

[0002] Automatically solving math problems is an important sub-problem of machine reasoning. Its goal is to solve a math problem consisting of a text description. The solver needs to understand the problem description, generate a standardized and computable solution equation, and use this expression to calculate the final answer to the problem.

[0003] With the rapid advancements in deep learning, existing methods for automatically solving math problems typically employ neural network-based encoder-decoder architectures, with a focus on upgrading and improving the encoder and decoder modules. A representative approach uses sequence-to-sequence models based on RNNs (recurrent neural networks). These models encode the semantic information in the problem description and generate arithmetic expressions sequentially from left to right. However, sequence-to-sequence models struggle to extract the structural information of mathematical expressions.

[0004] The tree structure has the advantage of representing mathematical expressions. The depth of the tree can correspond to the priority of the operations in the mathematical expression, so that the operations with higher priority are placed at the lower level, and the upper level nodes of the tree are the operators with lower priority. Therefore, a decoder based on the tree structure is proposed, which generates the decoder in a top-down manner. Target Expression In addition, from the perspective of relationship extraction, identifying the operational relationships between different quantities and iteratively constructing solution expressions through reasoning steps is also a mainstream method.

[0005] Although the existing automatic mathematical problem solving methods can effectively capture the semantic information of the problem description and Generate Expression However, the improvement scheme only focuses on the module technology upgrade of the encoder or decoder, and lacks a generalized training scheme applicable to various model frameworks. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a pseudo-dual learning method for automatically solving mathematical problems. The two-way task of solving mathematical problems and their inverse filling process is integrated into the pseudo-dual learning scheme, thereby improving the automatic mathematical problem solver's ability to capture the quantity and mathematical logic of mathematical problems and improving the performance of the automatic mathematical problem solver.

[0007] To achieve the above-mentioned object of the invention, the pseudo-dual learning method for automatically solving mathematical problems of the present invention is characterized by comprising the following steps:

[0008] (1) Obtain prediction expression

[0009] For a math problem P containing N words, N} is input into the automatic math problem solver for automatic solution, and the prediction expression S={s1,s2,...,s K}, where w n Represents the nth word, n = 1, 2, ..., N, number Q = {q1, q2, ..., q M},q m Represents the mth number, m = 1, 2, ..., M, s k represents the kth expression element;

[0010] The math problem automatic solver consists of a math problem encoder and an expression decoding generator. The math problem P to be solved is input into the math problem encoder to obtain the semantic features E of the math problem P to be solved. p , semantic feature E p Then input it into the expression decoding generator and output the predicted expression S with the maximum probability, which satisfies:

[0011]

[0012] Among them, Pr means getting character s k The probability value, Θ en and Θ dec The learning parameters of the math problem encoder and expression decoder generator respectively;

[0013] (2) Digital filling

[0014] 2.1) Resample the prediction expression S to obtain the continuous approximation of the prediction expression S

[0015] 2.2) The continuous approximation S g Input to the expression encoder to obtain the semantic features E of the predicted expression S g , the correct target expression of the math problem P to be solved Input to the expression encoder to get the target expression S t The semantic feature E t ;

[0016] 2.3) From the semantic feature E g Extract the semantic features of the numbers in the prediction expression S From the semantic feature E t Extract the target expression S t Semantic Features of Numbers in Chinese Then perform weighted summation to obtain the digital semantic features of the expression after planned fusion

[0017] E f_num =εE t_num +(1-ε)E g_num

[0018] Among them, the weight ε is used to adjust the ratio of the two parts, and ε decays exponentially from 1 to 0 during the training optimization process;

[0019] From the semantic feature E p Extracting semantic features of numbers in math problems in, represents the M×D dimensional real space, where D is the dimension of the hidden layer of the expression encoder;

[0020] 2.4) Use a pointer network to calculate the probability distribution vector of the mth number in the math problem P to be solved matching the M numbers in the prediction expression S:

[0021]

[0022] Among them, the probability distribution vector is Prob m is the M dimension, is the semantic feature E p In the mth row, W1 and W2 are the learning weight matrices of the pointer network, and u is the column weight vector;

[0023] In this way, we get a set of probability distribution vectors {Prob1, Prob2, ..., Prob M};

[0024] (3) Training optimization

[0025] 3.1) During the training and optimization process, for the automatic math problem solver, the correct target expression S t The negative log-likelihood loss function L is calculated between the supervisory signal and the prediction expression S G ;

[0026] For digital filling, the number q in the math problem P to be solved m The label is When it is at the jth position in the corresponding prediction expression S, o m The j-th dimension value of is 1, and the other dimension values ​​are 0. The corresponding filling loss function is calculated as follows:

[0027]

[0028] 3.2) Construct the loss function L = L G +L I, by minimizing the loss function L to train the automatic solver of optimization mathematical problems, thus completing pseudo-dual learning.

[0029] The object of the present invention is achieved in this way.

[0030] Considering that when humans solve mathematical problems, they substitute the solution expression into the original problem for review to improve the accuracy of the solution. The present invention introduces a pseudo-dual task to simulate the review process, that is, after obtaining the solution expression, the numbers in the expression are filled back into the original text problem description. Furthermore, the present invention innovatively proposes a training and learning scheme independent of the model, integrating the two-way task of solving mathematical problems and their reverse filling process into the pseudo-dual learning scheme, thereby improving the automatic solver's ability to capture the number of mathematical problems and the mathematical logic ability. In the joint training optimization process, in order to effectively jointly learn the solution process and the pseudo-dual filling process, the present invention also designs a plan fusion strategy, which uses the correct expression for number filling in the early stage of training, and smoothly transitions to using the predicted expression for number filling in the later stage of training to solve the problem that the predicted expression is not ideal in the early stage of training and will mislead the pseudo-dual learning. At the same time, the plan fusion strategy can balance the learning of the solution process and the pseudo-dual filling process during the training process. Through experimental research, the present invention significantly improves the performance of multiple representative automatic solvers for mathematical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the principle of a specific implementation method of the pseudo-dual learning method for automatically solving mathematical problems of the present invention;

[0032] Figure 2 This is a flow chart of a specific implementation of the pseudo-dual learning method for automatically solving math problems of the present invention. DETAILED DESCRIPTION

[0033] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0034] In the prior art, for a math problem P containing N words, N} can be input into the automatic math problem solver for automatic solution, and the prediction expression S={s1,s2,...,s} is composed of the number set Q, operators {+,-,×,÷,^} and constants {1,π,…}. K}, where w n Represents the nth word, n = 1, 2, ..., N, number Q = {q1, q2, ..., qM},q m Represents the mth number, m = 1, 2, ..., M, s k Represents the kth expression element.

[0035] The present invention designs a reverse number-filling process to simulate review, mapping the numbers in the predicted expression to the numbers in the math problem to be solved, P. This invention combines math problem solving with the reverse number-filling process, comprehensively considering the mapping of math problems to solution expressions and solution expressions to math problems. Through the present invention's pseudo-dual learning, the present invention improves the mathematical understanding and problem-solving capabilities of the automatic math problem solver.

[0036] The present invention innovatively proposes a pseudo-dual learning method for automatically solving mathematical problems. Figure 1 As shown, the model designed by the present invention consists of an automatic math problem solver and a number filling module. In addition, the present invention designs a plan fusion strategy to promote effective pseudo-dual learning of the automatic math problem solver and the number filling module during the process of training and optimizing the automatic math problem solver.

[0037] Figure 2 This is a flow chart of a specific implementation of the pseudo-dual learning method for automatically solving math problems of the present invention.

[0038] In this embodiment, if Figure 1 As shown, the pseudo-dual learning method for automatically solving math problems of the present invention includes the following steps:

[0039] Step S1: Get the prediction expression

[0040] For a math problem P containing N words, N} is input into the mathematical automatic solver for automatic solution, and the prediction expression S={s1,s2,...,s K}, where w n Represents the nth word, n = 1, 2, ..., N, number Q = {q1, q2, ..., q M},q m Represents the mth number, m = 1, 2, ..., M, s k Represents the kth expression element.

[0041] like Figure 1 As shown in the figure, the automatic math problem solver includes a math problem encoder and an expression decoding generator. The math problem P to be solved is input into the math problem encoder to obtain the semantic features E of the math problem P to be solved. p , semantic feature E p Then input it into the expression decoding generator and output the predicted expression S with the maximum probability, which satisfies:

[0042]

[0043] Among them, Pr means getting character s k The probability value, Θ en and Θ dec Learning parameters of the math problem encoder and expression decoder generator, respectively.

[0044] Step S2: Digital filling

[0045] After obtaining the solution expression through the automatic math problem solver, the present invention introduces a number filling module to form pseudo-dual learning, capturing the digital operation relationship between the expression and the math problem, thereby enhancing the solving module's ability to understand the mathematical relationship.

[0046] Step S2.1: Resample the prediction expression S to obtain a continuous approximation S g

[0047] Resample the prediction expression S to obtain a continuous approximation of the prediction expression S

[0048] The string of the prediction expression S obtained by argmax output in formula (1) is not differentiable, which cuts off the gradient between the prediction expression and the automatic math problem solver, thereby blocking the gradient propagation from the number filling module to the automatic math problem solver, and making it impossible to complete pseudo-dual learning. To solve this problem, in this embodiment, gumbel resampling is used to obtain a continuous approximation of the prediction expression S

[0049] Step S2.2: Obtain the semantic features of the predicted expression and the target expression

[0050] The continuous approximation S g Input to the expression encoder to obtain the semantic features E of the predicted expression S g , the correct target expression of the math problem P to be solved Input to the expression encoder to get the target expression S t The semantic feature E t .

[0051] In this embodiment, two expression encoders are used for illustration: a sequence model and a graph convolutional model.

[0052] For the sequence model, a bidirectional gated recurrent neural network (BiGRU) is used to train the predicted expression S and the target expression S t The characters (numbers, operators and constants) are sequentially encoded. Here we use the prediction expression S to illustrate and obtain its semantic features. Where D is the dimension of the hidden layer of the expression encoder:

[0053]

[0054]

[0055]

[0056] If we use the graph convolution model to predict the expression S and the target expression S t To encode, first convert the expression into a binary tree and get the adjacency matrix A, then embed A and the words of the expression into the matrix X v Inject into the parameter W v The graph convolutional network (GCN) with the activation function as relu is used to obtain the semantic features of the expression. Taking the predicted expression S as an example, the definition of the graph convolution operation GConv and the semantic features of the expression are The acquisition is as follows:

[0057]

[0058]

[0059] Therefore, the semantic features E of the predicted expression S are obtained through the expression encoder g , where E g for or

[0060] The same method can be used to obtain the target expression S t The semantic feature E t .

[0061] Step S2.3: Planned fusion of expression numerical semantic features

[0062] In the early stages of joint training of the automatic math problem solver and the number-filling module, the automatic math problem solver cannot perfectly predict the solution expression. Using a less accurate expression as input to the number-filling module will mislead the entire pseudo-dual model training process. One possible solution is to use the correct target expression as the input to the number-filling module, but this strategy isolates the solution module and the number-filling module during training and prevents feedback signals from being propagated to the solution module. Therefore, the present invention designs a plan fusion strategy to address this problem.

[0063] From the semantic feature E g Extract the semantic features of the numbers in the prediction expression S From the semantic feature E t Extract the target expression S tSemantic Features of Numbers in Chinese Then perform weighted summation to obtain the digital semantic features of the expression after planned fusion

[0064] E f_num =εE t_num +(1-ε)E g_num (7)

[0065] Among them, the weight ε is used to adjust the ratio of the two parts, and ε decays exponentially from 1 to 0 during the training optimization process;

[0066] From the semantic feature E p Extracting semantic features of numbers in math problems in, Represents the M×D-dimensional real number space.

[0067] Through planned fusion, the number filling module obtains more information from the target expression as input in the early stages of training optimization. As training optimization progresses, the solution module can predict a more ideal expression, at which point it smoothly switches to the predicted expression as input to the number filling module. Both the automatic math problem solver and the number filling module are stably optimized at the same time.

[0068] Step S2.4: Obtain the set of probability distribution vectors that match the M numbers in the math problem P to be solved with the M numbers in the prediction expression S

[0069] Use a pointer network to calculate the probability distribution vector that the mth number in the math problem P to be solved matches the M numbers in the prediction expression S:

[0070]

[0071] Among them, the probability distribution vector is Prob m is the M dimension, is the semantic feature E p In the mth row, W1 and W2 are the learning weight matrices of the pointer network, and u is the column weight vector;

[0072] In this way, we get a set of probability distribution vectors {Prob1, Prob2, ..., Prob M};

[0073] Step S3: Training optimization

[0074] Step S3.1: Calculate negative log-likelihood loss function and padding loss function

[0075] During the training and optimization process, for the automatic math problem solver, the correct target expression S tThe negative log-likelihood loss function L is calculated between the supervisory signal and the prediction expression S G ;

[0076] For number filling, the label of the number qm in the math problem P to be solved is When it is at the jth position in the corresponding prediction expression S, o m The j-th dimension value of is 1, and the other dimension values ​​are 0. The corresponding filling loss function is calculated as follows:

[0077]

[0078] Step S3.2: Construct loss function and perform training optimization

[0079] Construct loss function L=L G +L I , by minimizing the loss function L to train the automatic solver of optimization mathematical problems, thus completing pseudo-dual learning.

[0080] Experimental verification

[0081] The pseudo-dual learning method proposed in this invention for automatically solving mathematical problems can effectively optimize the training of the joint solution process and the pseudo-dual filling process, smooth the pseudo-dual learning process through the planned fusion strategy, further enhance the mathematical understanding ability of the model, and improve the accuracy of automatically solving mathematical problems.

[0082] In order to evaluate the effectiveness of pseudo-dual learning in strengthening the model's ability to solve mathematical problems, the present invention conducted a large number of experiments on two commonly used public data sets, Math23k and MathQA. Math23k contains 2,300 elementary school math word problems, and MathQA is an English data set that involves more mathematical operations and is more complex in scope. It is more difficult to solve than Math23k. Both data sets include annotations of mathematical problems, solution expressions, answer values, and stem numbers. The present invention uses expression accuracy (Expression) and answer accuracy (Value) as evaluation indicators. Expression indicates whether the predicted solution expression is the same as the annotation of the data set, and Value indicates whether the final answer calculated based on the predicted expression is equal to the target value.

[0083] The experimental results on the Math23k and MathQA datasets are shown in Table 1.

[0084]

[0085] Table 1

[0086] This paper conducts experiments based on five solution models. The five models differ in their use of different encoders and decoders. DNS indicates that the encoder uses a bidirectional gated recurrent neural network and the decoder uses a long short-term memory network; GTS indicates that the encoder uses a bidirectional gated recurrent neural network and the decoder uses a tree network; Graph2Tree indicates that the encoder uses a graph neural network and the decoder uses a tree network; BERT-Tree indicates that the encoder uses the BERT pre-trained language model and the decoder uses a tree network; and RE-Deduction indicates that the encoder uses the BERT pre-trained language model and the decoder uses an inference network. The performance reported by the original method is *, and the performance achieved by the code in this paper is *. +PseDual indicates that the solution model combines pseudo-dual learning, GRU indicates the use of bidirectional gated recurrent neural networks to encode the expression, and GCN indicates the use of graph convolutional networks to encode the expression.

[0087] The experimental results in Table 1 show that all automatic math problem solvers significantly improved their performance on both datasets after incorporating pseudo-dual learning. The accuracy of answers on Math23k and MathQA increased by an average of 1.14% and 0.98%, respectively. These experimental results validate the effectiveness and robustness of the pseudo-dual learning scheme. Furthermore, using GCN to encode expressions outperforms GRU, indicating that the binary tree structure of expressions is more reasonable than the sequence structure. Furthermore, BERT-Tree and RE-Deduction achieved superior results, benefiting from the superior representation capabilities of the pre-trained language model.

[0088] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. A pseudo-dual learning method for automatically solving mathematical problems, characterized by: The following steps are involved: (1) Obtain prediction expression For a math problem P containing N words, N } is input into the automatic math problem solver for automatic solution, and the prediction expression S={s1,s2,...,s K }, where w n Represents the nth word, n = 1, 2, ..., N, number Q = {q1, q2, ..., q M },q m Represents the mth number, m = 1, 2, ..., M, s k represents the kth expression element; The math problem automatic solver consists of a math problem encoder and an expression decoding generator. The math problem P to be solved is input into the math problem encoder to obtain the semantic features E of the math problem P to be solved. p , semantic feature E p Then input it into the expression decoding generator and output the predicted expression S with the maximum probability, which satisfies: Among them, Pr means getting character s k The probability value, Θ en and Θ dec The learning parameters of the math problem encoder and expression decoder generator respectively; (2) Digital filling 2.1) Resample the prediction expression S to obtain the continuous approximation of the prediction expression S 2.2) The continuous approximation S g Input to the expression encoder to obtain the semantic features E of the predicted expression S g , the correct target expression of the math problem P to be solved Input to the expression encoder to get the target expression S t The semantic feature E t ; 2.3) From the semantic feature E g Extract the semantic features of the numbers in the prediction expression S From the semantic feature E t Extract the target expression S t Semantic Features of Numbers in Chinese Then perform weighted summation to obtain the digital semantic features of the expression after planned fusion E f_num ANDE t_num +(1-ε)E g_num in, The weight ε is used to adjust the ratio of the two parts, and ε decays exponentially from 1 to 0 during the training optimization process; From the semantic feature E p Extracting semantic features of numbers in math problems in, represents the M×D dimensional real space, where D is the dimension of the hidden layer of the expression encoder; 2.4) Use a pointer network to calculate the probability distribution vector of the mth number in the math problem P to be solved matching the M numbers in the prediction expression S: Among them, the probability distribution vector is Prob m is the M dimension, is the semantic feature E p In the mth row, W1 and W2 are the learning weight matrices of the pointer network, and u is the column weight vector; In this way, we get a set of probability distribution vectors {Prob1, Prob2, ..., Prob M }; (3) Training optimization 3.1) During the training and optimization process, for the automatic math problem solver, the correct target expression S t The negative log-likelihood loss function L is calculated between the supervisory signal and the prediction expression S G ; For number filling, the label of the number qm in the math problem P to be solved is When it is at the jth position in the corresponding prediction expression S, o m The j-th dimension value of is 1, and the other dimension values ​​are 0. The corresponding filling loss function is calculated as follows: 3.2) Construct the loss function L = L G +L I , by minimizing the loss function L to train the automatic solver of optimization mathematical problems, thus completing pseudo-dual learning.

Images