An Automatic Solution Method and System for Mathematical Application Problems Based on Dual-Decoder Fusion

Abstract

This invention provides an automatic solution method and system for mathematical word problems based on dual-decoder fusion. Based on the fusion of Seq2Seq and tree decoder, it first extracts attribute words associated with numbers from the problem text according to grammatical dependency relationships. Then, it introduces a pre-trained language model (PLM) encoder with digital entity embedding to extract relationships between digital entities. The decoder is initialized based on the PLM encoder enhanced with digital characteristics, addressing the problem of missing mathematical knowledge in the Seq2Seq decoder. Furthermore, it shares the weights of the encoder-decoder corresponding parameters, allowing the encoder parameters to simultaneously affect both problem parsing and expression generation. Finally, it integrates the structure learning of Seq2Seq and tree decoder to generate multi-form expressions, enhancing the encoder's extraction of diverse structural information of numbers and the representation of digital features, thereby improving the accuracy of solving mathematical word problems.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing, and in particular relates to an automatic solution method and system for mathematical word problems based on dual decoder fusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, applications based on Natural Language Processing (NLP) such as problem solving, automated proofs, and human-like question answering have become research hotspots. Solving math word problems is a crucial scenario for assessing machine reading comprehension abilities, requiring machines to generate solution expressions from brief text descriptions—a process involving complex and intensive reasoning. In primary education, math word problems are an important part of K-12 curriculum development. If machines can perfectly understand these problems and be used to tutor students, it will have a significant impact on the application of artificial intelligence in education, while simultaneously promoting the development of education. Furthermore, the databases of educational support companies (such as Xiaoyuan Search, TiPaipai, and AnswerMeow) contain many user-uploaded or web-crawled math word problems, many of which have missing or incorrect answers. Currently, these companies mainly rely on crowdfunding for human solutions or verification, which is inefficient and costly, highlighting the urgent need for an automated method for solving math word problems.

[0004] Automated solving of Math Word Problems (MWPs) has long been a focus of NLP research both domestically and internationally. Early research employed statistical machine learning and semantic parsing methods to solve MWPs; however, these methods require significant effort to design complex expression templates and features, are difficult to extend, cannot handle unfamiliar expressions or operational relations, lack universality, and fail to generate expressions effectively. With the development of deep learning, more and more deep learning methods have been introduced to handle MWP solving tasks. Deep learning-based methods benefit from their ability to automatically learn features, enabling them to handle novel, unseen expressions and exhibiting better performance in MWP solving. Existing deep learning methods can be mainly categorized into three types based on their neural network model structure and the way they solve problems: sequence-to-sequence (Seq2Seq) methods, tree decoder-based methods, and relation extraction-based methods. In 2017, researchers first introduced Seq2Seq for solving MWPs. Since then, Seq2Seq-based methods have often used recurrent neural networks (RNNs), long short-term memory recurrent neural networks (LSTMs), or gated recurrent units (GRUs) to extract information from the problem description to solve MWPs. However, solving MWPs often involves a lot of additional knowledge, and extracting the representation of the problem from only a short text limits the model's reasoning ability. Later, some scholars proposed tree decoders based on binary tree structures to generate prefix expressions in a goal-driven manner. The node filling operator in the binary tree and the leaf filling number ensure that the generated expression is always computable, improving the model's performance in solving MWPs. However, the encoder's information extraction and text representation capabilities remain bottlenecks. With the emergence and application of large-scale pre-trained language models (PLMs) such as BERT and RoBERTa, PLMs have significantly improved their performance in MWP solving compared to LSTM and GRU, thanks to the implicit language knowledge within them. However, compared to some methods based on improvements to LSTM or GRU, PLMs lack information such as the extraction of information about the structure of digital entities in MWPs. Digital entities consist of numbers and important attribute words related to numbers, representing units, increases, or decreases, which play a crucial role in the model's extraction of logical relationships between numbers. Recently, some scholars have viewed MWP solving as a process of extracting complex relationships. Inspired by humans solving problems through deductive reasoning, they have proposed iterative relationship extraction methods that are more in line with human cognition, gradually generating solution expressions from local relationships to the global relationship. However, handling new variables generated during the reasoning process and determining the end point of reasoning remain challenges.

[0005] Currently, research in the field of automated solutions to mathematical word problems mainly relies on Seq2Seq or tree decoders to generate corresponding structured expressions and calculate the final answer. This research area faces three main challenges:

[0006] (1) The word problem text contains many words related to numbers. These words often describe the units of numbers or represent the logical relationships between numbers. The digital entity structure formed by numbers is an important attribute for extracting the logical relationships between numbers. However, existing PLM-based encoders lack the ability to extract digital entity-level structural information.

[0007] (2) Existing PLM-based methods have proposed tasks to enhance the logical reasoning and numerical characteristics of PLMs with encoder structures such as BERT and RoBERTa, in order to obtain encoders with implicit understanding of numerical properties and other additional knowledge. However, there is currently a lack of Seq2Seq decoders containing implicit mathematical knowledge or related targeted enhancement tasks to improve the performance of their generated expressions.

[0008] (3) Currently, single-structure models based on Seq2Seq or tree decoders follow top-down or top-up expression generation methods. However, different forms of expressions imply different relationships and structural information between numbers. Generating expressions in only one direction limits the encoder's ability to extract diverse relationships and structural information between numbers in the text, thus affecting the encoder's problem representation capabilities. Summary of the Invention

[0009] To address the technical problems mentioned above, this invention provides an automatic solution method and system for mathematical word problems based on dual-decoder fusion. Based on digital entity information, encoder-decoder weight sharing, and multi-structure expression generation strategies, the method enhances the encoder's ability to extract multiple implicit relationships between numbers, thereby improving the accuracy of solving mathematical word problems.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] The first aspect of this invention provides an automatic solution method for mathematical word problems based on dual-decoder fusion, comprising the following steps:

[0012] The text of the problem to be solved is preprocessed and divided into multiple short text segments according to punctuation.

[0013] Using grammatical dependency parsing, numbers and number-related attribute words are extracted from the preprocessed question text;

[0014] By introducing digital entity embedding, a pre-trained language model based on mathematical corpus augmentation is reconstructed as an encoder to obtain the question representation of the input data and the final hidden layer state of the encoder.

[0015] Based on the encoder structure and parameters, a sequence-to-sequence decoder with shared parameter weights is constructed. The final hidden layer state of the encoder is input, and inference is performed to generate infix and postfix form solution expressions.

[0016] Based on sequence-to-sequence decoder and tree decoder structures, a multi-form expression generation model integrating the two is constructed. Combining preprocessed question text, numbers, and their related attribute words, the model learns to generate multi-form expressions and calculates the answer to the question.

[0017] Furthermore, the preprocessing includes text segmentation, data error correction, number replacement, expression cleaning, and data augmentation.

[0018] Furthermore, the digital entity embedding includes digital location embedding and attribute word location embedding.

[0019] Furthermore, the reconstruction, based on a pre-trained language model augmented with mathematical corpus as the encoder, yields the question representation of the input data and the final hidden layer state of the encoder as follows:

[0020] The preprocessed text data and the extracted list of numbers and attribute word positions are input into the encoder. The encoder processes the data and outputs the problem representation and the encoder's final hidden layer state.

[0021] Furthermore, the specific steps of the fused multi-form expression generation model are as follows:

[0022] The numbers and related attribute words are transformed into their corresponding representations in the preprocessed question text, and then transformed into number entity embeddings.

[0023] Overlaying digital entity embeddings, word embeddings, and positional embeddings of text as input encoders for question embeddings;

[0024] The encoder extracts and parses the semantic information in the question embedding, transforms it into a latent representation, and generates the question output.

[0025] The sequence-to-sequence decoder learns to generate infix and postfix form solution expressions based on the implicit representation output by the encoder;

[0026] The tree decoder takes the problem output representation as the global goal and learns to generate a prefix form solution expression based on the implicit representation output by the encoder.

[0027] Extracting prefix expressions to calculate the final answer to the problem.

[0028] Furthermore, the numbers and number-related attribute words are obtained by filtering and extracting short text grammatical dependency analysis according to rules.

[0029] Furthermore, the hidden representation and problem output are generated by the extraction encoder's final hidden layer and the average final hidden layer.

[0030] A second aspect of the present invention provides an automatic solution system for mathematical word problems based on dual-decoder fusion, comprising:

[0031] The preprocessing module is configured to preprocess the text of the problem to be solved and divide it into multiple short text segments based on punctuation.

[0032] The feature extraction module is configured to extract numbers and number-related attribute words from the preprocessed question text using syntactic dependency analysis.

[0033] The encoder building module is configured to: introduce digital entity embeddings, reconstruct a PLM based on mathematical corpus enhancement as an encoder, and obtain the problem representation of the input data and the final hidden layer state of the encoder;

[0034] The decoder building module is configured to: build a sequence-to-sequence decoder with shared parameter weights based on the encoder structure and parameters, input the final hidden layer state of the encoder, and infer to generate infix and postfix form solution expressions;

[0035] The reasoning and computation module is configured to: build a multi-form expression generation model that integrates the sequence-to-sequence decoder structure and the tree decoder structure; combine the preprocessed question text, numbers and their related attribute words; learn to generate multi-form expressions and calculate the answer to the question.

[0036] A third aspect of the present invention provides a computer-readable storage medium.

[0037] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method for automatically solving mathematical word problems based on dual-decoder fusion.

[0038] A fourth aspect of the present invention provides a computer device.

[0039] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the above-described method for automatically solving mathematical word problems based on dual-decoder fusion.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. Based on the characteristics of text data in MWPs, this invention preprocesses data such as numbers and units to reduce data noise.

[0042] 2. This invention segments text, analyzes the grammatical dependencies of short texts, and extracts the related attribute words of numbers more accurately.

[0043] 3. This invention introduces a digital entity embedding reconstruction PLM encoder, which improves the model's ability to learn the logical relationships between numbers in MWP.

[0044] 4. This invention constructs a weight-sharing Seq2Seq model, applies encoder parameters to multiple tasks, improves the encoder's extraction of numerical relationships, and introduces mathematical knowledge into the Seq2Seq decoder, enhancing the performance of sequence structure expression generation.

[0045] 5. This invention adopts a strategy of integrating Seq2Seq and tree decoder to generate multi-form expressions, which improves the encoder's ability to extract and represent multiple structural information between numbers, such as global to local and local to global.

[0046] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0047] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0048] Figure 1 This is a flowchart of an automatic solution method for mathematical word problems based on dual-decoder fusion according to an embodiment of the present invention;

[0049] Figure 2 This is an overall framework diagram of the automatic solution method for mathematical application problems based on dual decoder fusion according to an embodiment of the present invention;

[0050] Figure 3 This is an encoder input diagram according to an embodiment of the present invention;

[0051] Figure 4 This is an encoder-decoder block parameter weight sharing diagram according to an embodiment of the present invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0054] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0055] Example 1

[0056] like Figure 1-4 As shown, this embodiment provides an automatic solution method for mathematical word problems based on dual-decoder fusion, which specifically includes the following steps:

[0057] Dual decoder fusion is based on the fusion of Seq2Seq and tree decoder.

[0058] S1: Preprocess the text of the problem to be solved and divide it into multiple short text segments according to punctuation;

[0059] Specifically, the data is preprocessed, transforming the question text into text data where numbers are replaced by logical identifiers. The expression labels are then transformed to generate prefix, infix, and postfix expressions.

[0060] Data preprocessing includes the following steps:

[0061] S101: Text segmentation: Use jieba segmentation to segment the problem text.

[0062] S102: Data correction: Corrects misclassified decimals, fractions, and other numerical values ​​in the problem text after word segmentation, and separates numbers connected to units.

[0063] S103: Number Replacement: Use regular expressions to extract integers, decimals, fractions, and percentages from the text. Replace the numbers with the logical identifier "#n" according to the order in which they appear in the text. At the same time, build a number stack to store the original numbers for answer calculation. Fractions and percentages are converted to decimals for calculation.

[0064] S104: Expression cleanup: Remove redundant parentheses from expressions, keeping only their simplest form.

[0065] S105: Data Augmentation: Generate prefix and postfix expressions from the original infix expression, and remove data containing unequal expressions that cannot be converted. Add specific identifiers to the original issue text. <post>As new text data, it is used as input to generate postfix expressions.

[0066] S2: Use syntactic dependency parsing to extract numbers and number-related attribute words from the preprocessed question text;

[0067] Feature extraction includes the following steps:

[0068] S201: Divide the original text into multiple short texts based on punctuation marks.

[0069] S202: Use Stanford CoreNLP to extract the part-of-speech tags and grammatical dependencies of words in the text.

[0070] S203: First, extract the numbers from the text, and then extract the nouns and verbs related to the numbers based on the relationships nmod:range, nmod:assmod, and dep. Extract the units related to the numbers based on the relationship mark:clf. Then, for the extracted nouns, extract the associated adjectives and verbs based on the relationships amod and dobj respectively; for the extracted verbs, extract their associated nouns based on the relationship nsubj.

[0071] S204: Construct a list of number positions and attribute word positions with the same length as the text. Based on the positions of the numbers and the extracted related attribute words in the text, set the corresponding element in the list to 2, and set the others to 1.

[0072] S3: Introduce digital entity embedding, reconstruct a pre-trained language model based on mathematical corpus enhancement as an encoder, and obtain the question representation of the input data and the final hidden layer state of the encoder;

[0073] Pre-trained language models are also known as PLM (Pre-trained language models) encoders.

[0074] Specifically, the MWP-BERT encoder is reconstructed by embedding digital entities (numbers and attribute words). The input is preprocessed text data and the extracted list of numbers and attribute words. The encoder processes the data and outputs the problem representation and the final hidden layer state of the encoder.

[0075] Specifically, the following steps are included:

[0076] S301: Add digit entity embeddings to the embedding module of MWP-BERT, which is the sum of digit location embeddings and attribute word location embeddings. Set the input dimension of the digit location embeddings and attribute word location embeddings to 3, the output dimension to the hidden_size of MWP-BERT, and the padding flag to 0. Superimpose the original word embeddings, location embeddings, and the added digit entity embeddings as the output of the embedding module.

[0077] S302: Add a list of positions of numbers and their associated attribute words to the forward function of MWP-BERT, and replace the original embedding module with the embedding module reconstructed using the number entity information.

[0078] like Figure 3 As shown, the encoder embedding module generates digit position embeddings and attribute word position embeddings based on digit positions and attribute word positions. The original MWP-BERT word embeddings, position embeddings, and digit and attribute word position embeddings are superimposed to form the final question embedding representation.

[0079] S303: Input the preprocessed text data and the extracted list of numbers and attribute word positions into the encoder, extract the final hidden layer state H from the encoder output, and take the average as the problem representation.

[0080] Given P = {w1, w2, ..., w n } represents the input text containing n characters, constructing NumberPos = {z1, z2, ..., zn} n } and AttributePos = {c1, c2, ..., c n } represents a list of numbers and a list of attribute words, respectively, and assigns the value 1 to each element. When z... i The corresponding w i When it is a number, assign it the value 2, and similarly, c i The corresponding w i When extracting attribute words, assign the value c. i 2. Input the text P, NumberPcs, and AttributePos into the encoder embedding module to obtain the embedded problem representation X. p ={x1, x2, ..., x n The encoder takes the question embedding as input and obtains the final hidden layer state H = {h1, h2, ..., h}. n }∈R n ×d And find the problem representation.

[0081]

[0082] Where d represents the dimension of the latent vector.

[0083] S4: Based on the encoder structure and parameters, construct a sequence-to-sequence decoder with shared parameter weights, input the final hidden layer state of the encoder, and infer to generate infix and postfix form solution expressions;

[0084] The decoder is a Seq2Seq decoder, and specifically includes the following steps:

[0085] S401: Modify the MWP-BERT bidirectional self-attention layer to unidirectional, add a cross-attention layer before the multi-head attention layer, and add an LM-Head layer after the encoder for word classification to achieve autoregressive generation of the decoder.

[0086] S402: Due to the similarity in the structure of the encoder and decoder, except for the newly added cross-attention layer, LM-Head layer, and entity embedding layer, the corresponding parts of the encoder and decoder share parameter weights. Furthermore, since the encoder's vocabulary contains the tokens that make up the solution expression, the encoder's word embedding layer and LM-Head layer share parameter weights.

[0087] like Figure 4 As shown, within the corresponding encoder-decoder block, the parameter weights of the feedforward layer and the self-attention layer are shared, while the weights of the parameters of the cross-attention layer of the decoder are randomly initialized.

[0088] S5: Based on the sequence-to-sequence decoder structure and the tree decoder structure, a multi-form expression generation model integrating the two is constructed. Combining the preprocessed question text, numbers and their related attribute words, the model learns to generate multi-form expressions and calculates the answer to the question.

[0089] In step one, the tree decoder takes the question representation as the global target, connects the final hidden layer state of the encoder through a self-attention layer to obtain the contextual representation of the question, and iteratively predicts the node symbols according to the preorder traversal to construct the prefix expression.

[0090] Specifically, the following steps are included:

[0091] S501: Represent the problem The solution objective q0, which serves as the root node of the tree, is associated with the final hidden state H of the problem via the encoder. It is then input into the self-attention layer of the tree decoder to generate the current context representation c. i .

[0092]

[0093] where t sl The left subtree embedding of the parent node to which the current node belongs is represented by equation (5).

[0094] Given E denotes the infix solution expression of the problem, assume Let V be the m values ​​arranged in order in problem P. op = {+, -, x, ÷, ∧} are mathematical operators, V con Let E represent constants such as 1 and π. Furthermore, E can be expressed as... The ordered set E = {s1, s2, ..., sn} consisting of k elements. k The goal of the tree decoder is to generate the solution expression E in prefix form. pre .

[0095] S502: Solve the objective q i The current context represents c. i An embedding representation N of a set V is input to the prediction layer to predict and generate the token I of the current node. i .

[0096] , i =Predict(q i c i (3)

[0097] when When the current node's reasoning ends, the inference process ends; otherwise, it's necessary to further generate the target and context representations for the child nodes. For the left child node, the solution target q of the parent node is... p Context representation c p Predicting and generating tokens I p The embedding representation is input to the left child node generation layer and transformed to obtain the solution objective q of the left child node. l Then predict the token I of the left child node. l .

[0098]

[0099] For the right child node, when generating the solution objective, q r In addition to considering the solution objective of the left child node Context representation c l and the generated token I l In addition, the embedding representation of the left subtree also needs to be considered. In the embedding representation of a computation tree, when When taking the corresponding I in N i The embedding of a position is used as the embedding representation of a tree; when I i ∈V op At that time, embed its left and right subtrees and node tokens I i The embedding represents the input merging layer, which is transformed to generate the embedding representation of the current tree.

[0100]

[0101] Therefore, for the right child node, the left subtree embedding representation t is used. l Solve for the objective q l Context representation c l and generating symbol I l The embedding representation is given by inputting the right child node generation layer to obtain the solution objective q of the right child node. r Further predict and generate its token I r .

[0102]

[0103] Step 2: The Seq2Seq decoder and the tree decoder share the encoder output, generate corresponding representations, and fuse them to train and optimize the model.

[0104] like Figure 2 The overall framework diagram shown illustrates the following steps in the fusion training:

[0105] S503: The Seq2Seq decoder generates infix and postfix expressions based on the final hidden state H output by the encoder, calculates the sum of the negative log-likelihoods of the two parts as the Seq2Seq partial loss function, and optimizes the Seq2Seq model.

[0106] Given a dataset D, for each pair of problem P and solution expression E, the prefix and suffix forms obtained after transforming E into its infix form are respectively E pre E post Add the <post> tag to question P to obtain the result used to generate E. posost New copy P post .

[0107]

[0108]

[0109] J seq2seq =J infix +J post (9)

[0110] Where Pr(E|P) represents the conditional probability of expression E given problem P. infix and J post Let J represent the loss functions of the Seq2Seq model for infix and postfix expressions, respectively. seq2seq It is the sum of two loss functions, which is the overall loss function of the Seq2Seq model.

[0111] S504: The tree decoder combines the problem representation of the encoder's final hidden layer state and output, iteratively generates a prefix form expression from top to bottom, and calculates the negative log-likelihood of this part as the Seq2Tree partial loss function to optimize the Seq2Tree model.

[0112]

[0113]

[0114] Among them, E pre For prefix expressions, J seq2tree This represents the loss function of the Seq2Tree model for the prefix expression.

[0115] S505: Given a problem and expression in a dataset D, optimize the overall model by minimizing the sum of the loss functions of the two parts of the model and save it.

[0116] J = J seq2seq +J seq2tree (12)

[0117] The reasoning steps for answering the question are as follows:

[0118] Step (1): Preprocess the input problem and extract relevant features according to steps S101→S104 and S201→S204 in the model training.

[0119] Step (2): Load the trained model and infer to generate the solution expression.

[0120] Step (3): Extract the prefix form expression and calculate the final answer.

[0121] This invention provides an automatic solution method and system for mathematical word problems based on the fusion of Seq2Seq and tree decoder. It enhances the encoder's ability to extract multiple implicit relationships between numbers by leveraging digital entity information, encoder-decoder weight sharing, and multi-structure expression generation strategies. First, attribute words associated with numbers are extracted from the problem text based on grammatical dependency relationships. Then, digital entities (numbers and attribute words) are embedded to reconstruct the PLM encoder, extracting relationships between digital entities. The decoder is initialized based on a digital characteristic-enhanced PLM encoder (MWP-BERT) to address the lack of mathematical knowledge in the Seq2Seq decoder. Furthermore, the weights of corresponding encoder-decoder parameters are shared, allowing encoder parameters to simultaneously affect both problem parsing and expression generation. Finally, the Seq2Seq and tree decoder structure learning are fused to generate multi-form expressions, enhancing the encoder's extraction of diverse structural information and representation of digital features, thereby improving the accuracy of solving mathematical word problems.

[0122] Example 2

[0123] This embodiment provides an automatic solution system for mathematical word problems based on dual-decoder fusion, which specifically includes:

[0124] The preprocessing module is configured to preprocess the text of the problem to be solved and divide it into multiple short text segments based on punctuation.

[0125] The feature extraction module is configured to extract numbers and number-related attribute words from the preprocessed question text using syntactic dependency analysis.

[0126] The encoder building block is configured to: introduce digital entity embeddings, reconstruct a pre-trained language model based on mathematical corpus augmentation as an encoder, and obtain the question representation of the input data and the final hidden layer state of the encoder;

[0127] The decoder building module is configured to: build a sequence-to-sequence decoder with shared parameter weights based on the encoder structure and parameters, input the final hidden layer state of the encoder, and infer to generate infix and postfix form solution expressions;

[0128] The reasoning and computation module is configured to: build a multi-form expression generation model that integrates the sequence-to-sequence decoder structure and the tree decoder structure; combine the preprocessed question text, numbers and their related attribute words; learn to generate multi-form expressions and calculate the answer to the question.

[0129] Example 3

[0130] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the above-described method for automatically solving mathematical word problems based on dual-decoder fusion.

[0131] Example 4

[0132] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the above-described method for automatically solving mathematical application problems based on dual-decoder fusion.

[0133] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / post>

Claims

1. An automatic solution method for mathematical word problems based on dual-decoder fusion, characterized in that, include: The text of the problem to be solved is preprocessed and divided into multiple short text segments according to punctuation. Using grammatical dependency parsing, numbers and number-related attribute words are extracted from the preprocessed question text; By introducing digital entity embedding, a pre-trained language model based on mathematical corpus augmentation is reconstructed as an encoder to obtain the question representation of the input data and the final hidden layer state of the encoder. Based on the encoder structure and parameters, a sequence-to-sequence decoder with shared parameter weights is constructed. The final hidden layer state of the encoder is input, and inference is performed to generate infix and postfix form solution expressions. The negative log-likelihood of the two parts is calculated as the sequence-to-sequence partial loss function to optimize the sequence-to-sequence model. Based on a sequence-to-sequence decoder structure and a tree decoder structure, a multi-form expression generation model integrating the two is constructed. Combining preprocessed question text, numbers, and their related attribute words, the model learns to generate multi-form expressions and calculates the question answer. This includes the tree decoder combining the final hidden layer state and output question representation of the encoder, iteratively generating prefix form expressions from top to bottom, and calculating the negative log-likelihood of this part as the loss function of the tree decoder part to optimize the tree decoder model. Given a question and expression in a dataset, the overall model is optimized and saved by minimizing the sum of the loss functions of the two parts of the model. The trained model is then loaded, inference is performed to generate solution expressions, prefix form expressions are extracted, and the final answer is calculated.

2. The automatic solution method for mathematical word problems based on dual-decoder fusion as described in claim 1, characterized in that, The preprocessing includes text segmentation, data error correction, number replacement, expression cleanup, and data augmentation.

3. The automatic solution method for mathematical word problems based on dual-decoder fusion as described in claim 1, characterized in that, The digital entity embedding includes digital location embedding and attribute word location embedding.

4. The automatic solution method for mathematical word problems based on dual-decoder fusion as described in claim 1, characterized in that, The reconstruction, based on a pre-trained language model augmented with mathematical corpus as the encoder, yields the question representation of the input data and the final hidden layer state of the encoder as follows: The preprocessed text data and the extracted list of numbers and attribute word positions are input into the encoder. The encoder processes the data and outputs the problem representation and the encoder's final hidden layer state.

5. The automatic solution method for mathematical word problems based on dual-decoder fusion as described in claim 1, characterized in that, The specific steps of the fused multi-form expression generation model are as follows: The numbers and related attribute words are transformed into their corresponding representations in the preprocessed question text, and then transformed into number entity embeddings. Overlaying digital entity embeddings, word embeddings, and positional embeddings of text as input encoders for question embeddings; The encoder extracts and parses the semantic information in the question embedding, transforms it into a latent representation, and generates the question output. The sequence-to-sequence decoder learns to generate infix and postfix form solution expressions based on the implicit representation output by the encoder; The tree decoder takes the problem output representation as the global goal and learns to generate a prefix form solution expression based on the implicit representation output by the encoder. Extracting prefix expressions to calculate the final answer to the problem.

6. The automatic solution method for mathematical word problems based on dual-decoder fusion as described in claim 5, characterized in that, The numbers and number-related attribute words are obtained by filtering and extracting short texts based on rules and performing grammatical dependency analysis.

7. The automatic solution method for mathematical word problems based on dual-decoder fusion as described in claim 5, characterized in that, The hidden representation and problem output are generated by the extraction encoder's final hidden layer and the average final hidden layer.

8. An automatic solution system for mathematical word problems based on dual-decoder fusion, characterized in that, include: The preprocessing module is configured to preprocess the text of the problem to be solved and divide it into multiple short text segments based on punctuation. The feature extraction module is configured to extract numbers and number-related attribute words from the preprocessed question text using syntactic dependency analysis. The encoder building block is configured to: introduce digital entity embeddings, reconstruct a pre-trained language model based on mathematical corpus augmentation as an encoder, and obtain the question representation of the input data and the final hidden layer state of the encoder; The decoder building module is configured to: build a sequence-to-sequence decoder with shared parameter weights based on the encoder structure and parameters; input the final hidden layer state of the encoder; infer and generate infix and postfix form solution expressions; and calculate the sum of the negative log-likelihoods of the two parts as the sequence-to-sequence partial loss function to optimize the sequence-to-sequence model. The inference and computation module is configured to: construct a multi-form expression generation model based on a sequence-to-sequence decoder structure and a tree decoder structure, combining the preprocessed question text, numbers, and their related attribute words, learn to generate multi-form expressions and calculate the question answer, including the tree decoder combining the final hidden layer state and output question representation of the encoder, iteratively generating prefix form expressions from top to bottom, calculating the negative log-likelihood of this part as the loss function of the tree decoder part to optimize the tree decoder model, inputting the question and expression from the given dataset, optimizing the overall model by minimizing the sum of the loss functions of the two parts of the model and saving it, loading the trained model, inferring and generating the solution expression, extracting the prefix form expression, and calculating the final answer.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the automatic solution method for mathematical application problems based on dual decoder fusion as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the automatic solution method for mathematical application problems based on dual decoder fusion as described in any one of claims 1-7.

Images