Mathematics application question intelligent solution method and system fusing large model knowledge

Abstract

The application discloses a kind of mathematical application question intelligent solution method and system of fusing big model knowledge, necessary knowledge is excited, verified, injected in each reasoning step, so as to focus on how to ensure the rationality of the knowledge excited in each reasoning step, and how to apply the knowledge to the symbol generation of current step, finally realize the improvement of reasoning process explainability and accuracy.The scheme provided by the application can be extended to include various machine learning tasks and use various large language models, providing a feasible solution for artificial intelligence with strong reasoning ability.It also provides high explainability problem-solving logic for personalized online learning, with high educational and practical value.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent solution method and system for mathematical application problems that integrates knowledge from large-scale models. Background Technology

[0002] Automated problem-solving of Math Word Problems (MWPs) is a crucial task in exploring general artificial intelligence. Its goal is to enable machines to read a mathematical problem described in natural language and deduce the corresponding numerical answer. This process requires machines to understand mathematical logic, learn, and apply mathematical knowledge for reasoning, much like humans do. Therefore, the ability to automatically solve mathematical word problems is considered an important indicator of machine intelligence and has always been a fundamental yet vital research direction in the field of artificial intelligence.

[0003] Currently, the main methods for automatically solving math word problems are as follows:

[0004] (1) Rule-based automatic solution method.

[0005] Rule-based automatic problem-solving methods rely on manually constructed patterns or formal languages. By matching problem text with predefined problem templates, the problem is mapped into a specific propositional form, and then the reasoning results are generated step by step using manually defined rules.

[0006] (2) Statistical automatic solution method.

[0007] Statistical automatic solution methods utilize traditional machine learning models such as support vector machines and log-linear models to identify entities, quantities, and operators from the problem text, then substitute them into a predefined expression template, and derive the answer through simple symbolic reasoning.

[0008] (3) Automatic solution method based on semantic parsing.

[0009] Automatic solution methods based on semantic parsing map problem text into structured semantic representations (such as semantic parsing trees), and transform semantic information into mathematical expressions through certain logical rules.

[0010] (4) Automatic solution method based on deep learning.

[0011] Deep learning-based automatic problem-solving methods primarily employ an encoder-decoder structure. The encoder automatically mines and understands the features of the problem text, while the decoder uses reasoning to solve the expression. Based on this structure, existing work mainly focuses on improving the semantic understanding ability of the encoder and the reasoning ability of the decoder. Methods focusing on semantic understanding enrich the mining of problem information by incorporating semantic structure, numerical magnitude, and numerical relationships from the problem text, achieving a more accurate understanding of the problem and obtaining more reasonable reasoning results. Methods focusing on reasoning ability design different neural network architectures based on different reasoning modes. For example, in the sequence-to-sequence (Seq2Seq) architecture, a recurrent neural network is used to directly generate symbol sequences; in the sequence-to-tree (Seq2Tree) architecture, based on the human problem decomposition mechanism, a goal-oriented tree-structure decoder is designed to generate expression trees from top to bottom; and in the deductive reasoning architecture, a relation extraction decoder is used to progressively combine values ​​and operators to form sub-expressions, iteratively generating expression trees from bottom to top.

[0012] (5) Automatic solution method based on large language model.

[0013] Large Language Models (LLMs) are general-purpose deep neural network models with massive parameters trained on large-scale corpora, possessing strong contextual understanding capabilities. Existing research has utilized techniques such as chain-of-thought and tree-of-thought to enhance their thinking and planning abilities, gradually achieving excellent results in tasks such as solving mathematical word problems.

[0014] However, the aforementioned automatic solution methods are still some distance from human-level artificial intelligence, and they are prone to logical errors and factual errors in reasoning.

[0015] In view of this, the present invention is hereby proposed. Summary of the Invention

[0016] The purpose of this invention is to provide an intelligent solution method and system for mathematical word problems that integrates knowledge from large-scale models. This overcomes the shortcomings of existing technologies, such as the inability to model the gradual application of knowledge during the reasoning process, lack of interpretability, and significant distance from the goal of general artificial intelligence. This invention can achieve more accurate intelligent solutions to mathematical word problems.

[0017] The objective of this invention is achieved through the following technical solution:

[0018] A method for intelligently solving mathematical word problems that integrates knowledge from large-scale models includes:

[0019] Step 1: Enter the question text;

[0020] Step 2: In the current reasoning step, based on the question text and the generated symbols, stimulate the knowledge required for the current reasoning step based on the large language model;

[0021] Step 3: Encode the knowledge generated in Step 2. Use the obtained knowledge encoding and the reasoning target vector of the current reasoning step to verify and filter the knowledge, obtain the filtered knowledge, and calculate the attention weight of the filtered knowledge by combining it with the reasoning target vector of the current reasoning step, and calculate the attention weight of the question text by combining the knowledge encoding and the text representation vector of the question text; then update the reasoning target vector of the current reasoning step by combining the two calculated attention weights; use the updated reasoning target vector of the current reasoning step to calculate the symbol of the current reasoning step, and generate the reasoning target vector of the next reasoning step; where, if the current reasoning step is the first step, the reasoning target vector of the current reasoning step is the initial reasoning target vector obtained by using the question text; by repeating Step 2 and Step 3, generate all symbols, and combine them in order to form the solution expression;

[0022] Step 4: Solve the expression to get the answer.

[0023] A smart solution system for mathematical word problems that integrates knowledge from large-scale models, comprising:

[0024] The input unit is used to input the question text;

[0025] A knowledge system is used to generate the knowledge required for the current reasoning step based on the question text and the generated symbols, using a large language model.

[0026] The reasoning system encodes the triggered knowledge, verifies and filters the knowledge using the obtained knowledge encoding and the reasoning target vector of the current reasoning step, obtains the filtered knowledge, and calculates the attention weight of the filtered knowledge by combining the reasoning target vector of the current reasoning step, and calculates the attention weight of the question text by combining the knowledge encoding and the text representation vector of the question text; updates the reasoning target vector of the current reasoning step by combining the two calculated attention weights; calculates the symbol of the current reasoning step by combining the updated reasoning target vector of the current reasoning step, and generates the reasoning target vector of the next reasoning step; wherein, if the current reasoning step is the first step, the reasoning target vector of the current reasoning step is the initial reasoning target vector obtained using the question text; by repeatedly executing steps 2 and 3, all symbols are generated and combined in order to form the solution expression;

[0027] The solver unit is used to solve for the answer using the solver expression.

[0028] A processing device includes: one or more processors; and a memory for storing one or more programs;

[0029] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0030] A readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0031] As can be seen from the technical solution provided by this invention, necessary knowledge is stimulated, verified, and injected in each reasoning step. This focuses on ensuring the rationality of the stimulated knowledge and how to apply it to the symbol generation in the current step, ultimately improving the interpretability and accuracy of the reasoning process. The solution provided by this invention can not only be extended to include various machine learning tasks and use various large language models, providing a feasible solution for realizing artificial intelligence with powerful reasoning capabilities; it can also provide highly interpretable problem-solving logic for personalized online tutoring, possessing high educational and practical value. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart illustrating an intelligent solution method for mathematical application problems that integrates knowledge from a large model, provided as an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the intelligent problem-solving framework provided in an embodiment of the present invention;

[0035] Figure 3 A schematic diagram illustrating the principles of knowledge activation, knowledge verification, and knowledge injection provided in embodiments of the present invention;

[0036] Figure 4 A schematic diagram of an intelligent solution system for mathematical application problems that integrates knowledge from a large model, provided as an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of a processing device provided in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0039] First, the following explanations are provided for the terms that may be used in this article:

[0040] The terms “including,” “comprising,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0041] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0042] The following is a detailed description of the intelligent solution method and system for mathematical application problems that integrates large-scale model knowledge, provided by this invention. Contents not described in detail in the embodiments of this invention belong to prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, conventional conditions in the art or conditions recommended by the manufacturer shall apply. Reagents or instruments used in the embodiments of this invention, unless otherwise specified by the manufacturer, are all conventional products that can be purchased commercially.

[0043] Example 1

[0044] This invention provides an intelligent solution method for mathematical word problems that integrates knowledge from large-scale models, such as... Figure 1 As shown, it mainly includes the following steps:

[0045] Step 1: Enter the question text.

[0046] Step 2: Knowledge Activation Process.

[0047] In this embodiment of the invention, a knowledge system is constructed to generate the necessary knowledge for the current reasoning step based on the question text and the generated symbols, using a large language model. This knowledge system mainly includes a large language model and a knowledge invoker; the large language model serves as a knowledge base storing common sense, mathematical knowledge, etc., and the knowledge invoker generates knowledge from the large language model at the reasoning step level.

[0048] Step 3: Reasoning process.

[0049] In this embodiment of the invention, knowledge verification and injection based on stimulated knowledge are used to guide logical thinking and ensure the rationality of reasoning.

[0050] The reasoning process is as follows: The knowledge generated in step 2 is encoded. The knowledge is then verified and filtered using the obtained knowledge encoding and the reasoning target vector of the current reasoning step to obtain the filtered knowledge. The attention weight of the filtered knowledge is calculated by combining the reasoning target vector of the current reasoning step with the knowledge encoding, and the attention weight of the question text is calculated by combining the knowledge encoding and the text representation vector of the question text. The reasoning target vector of the current reasoning step is then updated using the two calculated attention weights. The symbol of the current reasoning step is calculated using the updated reasoning target vector of the current reasoning step, and the reasoning target vector of the next reasoning step is generated. If the current reasoning step is the first step, the reasoning target vector of the current reasoning step is the initial reasoning target vector obtained using the question text. By repeatedly executing steps 2 and 3, all symbols are generated and combined in order to form the solution expression.

[0051] Step 4: Solve the expression to get the answer.

[0052] To more clearly demonstrate the technical solution and its effects provided by the present invention, the method provided by the embodiments of the present invention will be described in detail below with reference to specific examples.

[0053] I. Overall Overview of the Plan

[0054] Existing automated problem-solving methods still fall short of human-level artificial intelligence, prone to logical and factual errors in mathematical reasoning. Even large models can produce results that violate basic common sense and mathematical principles during the reasoning process, a phenomenon known as hallucination. Therefore, comparing human mathematical reasoning to this process, cognitive psychology theory suggests that reasoning is a step-by-step process, requiring each step to stimulate, verify, and inject necessary knowledge to guide logical thinking and ensure the rationality of the reasoning. Thus, enabling models to progressively apply knowledge as in the reasoning process is a key research area for improving their intelligence level and a core goal of achieving general-purpose artificial intelligence.

[0055] To overcome the shortcomings of existing technologies, such as their inability to model the gradual application of knowledge by humans during reasoning, lack of interpretability, and significant distance from the goals of general artificial intelligence, this invention provides an intelligent solution method for mathematical word problems that integrates knowledge from a large model. A "knowledge system-reasoning system" is constructed as the intelligent problem-solving framework. The knowledge system uses a large language model as a knowledge base storing common sense, mathematical knowledge, etc., and includes a knowledge invoker that incites knowledge from the large language model at the reasoning step level. The inference system adopts an encoder-decoder structure, progressively applying knowledge from the knowledge system to reason mathematical expressions. Specifically, the decoder includes a knowledge verifier and a knowledge injector, focusing on how to ensure the rationality of the incited knowledge in the current reasoning step and how to apply that knowledge to the symbol generation in the current step, respectively, ultimately improving the interpretability and accuracy of the reasoning process. This invention can also be extended to include various machine learning tasks and use various large language models, providing a feasible solution for realizing artificial intelligence with powerful reasoning capabilities.

[0056] II. Detailed introduction of the plan.

[0057] 1. Problem definition and formalization.

[0058] The goal of the automatic solution task for mathematical word problems is to generate the answer to the problem by performing logical reasoning based on the word problem text. In this embodiment of the invention, the problem text X corresponding to the mathematical word problem P is... P Defined as a sequence X consisting of n words P =[x1,x2,...,x n Each 'x' represents a word, and the subscript is the word's index. P The set of numerical values ​​appearing in N is defined as N. P (e.g., {400, 6}). The solution expression for problem P is Y. P Defined as a sequence Y consisting of m operators, constants, or numerical values ​​from the problem (collectively referred to as symbols). P =[y1,y2,…,y m ], where each symbol y is a set of decoded symbols V for a mathematical word problem P. P An element of the set of decoded symbols, with its index being the symbol's ordinal number. P From the set of operators V O (e.g., {+,-,×,÷}), constant set V C (e.g., {1,π}) and the numerical set NP Composition, i.e., V P =V O ∪V C ∪N P Since N is the number of problems... P Different, V for each question P They are also different. The answer to question P is a. P The value of the unknown variable to be determined (e.g., weight "550") is expressed by the mathematical expression Y. P The derivation shows that the goal of the automatic problem-solving task for mathematical word problems is to learn a model that can read the problem text X. P Generate expression Y P And based on Y P The problem is to calculate the variable a. P .

[0059] 2. Data collection and preprocessing.

[0060] (1) Data collection.

[0061] This invention patent uses mathematical word problems in a broad sense as the input dataset. The problems must include the problem text, mathematical expression, and numerical answer. Examples of such data include the open-source mathematical word problem dataset (Math23K) released by Tencent and the open-source dataset (MAWPS) released by Microsoft. Alternatively, the input dataset can be obtained by web scraping or collecting homework or exam math problems from primary and secondary school students offline.

[0062] (2) Data preprocessing

[0063] Before building a model, the collected data needs to be preprocessed to ensure the model's effectiveness. Preprocessing mainly includes the following:

[0064] (2.1) Data filtering.

[0065] The embodiments of the present invention are mainly aimed at mathematical application problems that contain only one question, have a numerical answer, and can be directly calculated by a mathematical expression. Therefore, it is necessary to select questions in the dataset that contain only one mathematical expression and a numerical answer, and filter out questions that lack an expression or have two or more expressions and answers.

[0066] (2.2) Sampling.

[0067] Random sampling is performed in each dataset class, and a subset of the original dataset is selected to train the model.

[0068] 3. Intelligent problem-solving framework.

[0069] In this embodiment of the invention, a "knowledge system-reasoning system" is constructed as an intelligent problem-solving framework, such as... Figure 2As shown, the intelligent problem-solving framework comprises a knowledge system and a reasoning system. The knowledge system is primarily responsible for implementing step 2, which utilizes a large language model as a knowledge repository and employs a knowledge activator to activate the necessary knowledge for each reasoning step. The reasoning system is primarily responsible for implementing step 3, which uses an encoder-decoder approach to reason the solution expression for a given mathematical problem. This invention designs a knowledge verifier and a knowledge injector within the decoder, used respectively to filter irrelevant knowledge activated by the knowledge system and to apply that knowledge to guide symbolic reasoning in the current step.

[0070] like Figure 3 As shown, the main principles of knowledge stimulation, knowledge verification, and knowledge injection are presented.

[0071] (1) Knowledge system.

[0072] The knowledge system is responsible for storing and activating knowledge. Since large language models have implicitly stored knowledge in their parameters through training on massive amounts of data, the knowledge system directly uses the current large language model (such as GPT4, ChatGPT) as the knowledge base K.

[0073] In this embodiment of the invention, the current reasoning step is defined as step t (reasoning step t), and given a knowledge base K, the knowledge activator is based on the question text X. P and the expressions generated before step t. The knowledge K required to stimulate the reasoning in step t t This process utilizes prompt engineering techniques to piece together X. P , For a query Q t Input a large language model, and use the text output by the large language model as K. t '(For example Figure 3 The text in the document states, "This problem requires applying the formula 'Total weight = Quantity multiplied by unit weight,' which is calculated by multiplying the unit weight of the bananas by the total number of boxes." This is denoted as... r represents the length of the knowledge.

[0074] (2) Reasoning system.

[0075] The reasoning system aims to apply knowledge generated within a knowledge system to reason about the expression of a given mathematical word problem. For example... Figure 2 As shown, it uses an encoder-decoder approach, first encoding the question text and then decoding it to generate an expression tree.

[0076] (2.1) Encoder-decoder.

[0077] Given a mathematical word problem P, encoder Enc reads in the problem text X. P =[x1,x2,...,xn Generate a text representation vector sequence H = [h1, h2, ..., h] n ] and the initial inference target vector s1:

[0078] (H,s1)=Enc([x1,x2,...,x n ])

[0079] Enc can be implemented using RNN, BERT, the encoder of the existing MWP model, and large language models.

[0080] Based on H and s1, the decoder gradually generates the solution expression Y. P =[y1,y2,…,y m Specifically, step t (t = 1, 2, ..., m) comprises two stages: 1) Symbolic prediction stage: utilizing the inference target vector s from step t. t Calculation symbol y t The distribution p(y) t |y1,y2,...,y t-1 ); 2) Target generation stage: based on the selected symbol y t Generate the inference target vector s for the next step. t+1 These two stages alternate, and all the symbols generated are combined sequentially to form the solution expression Y. P .

[0081] This invention primarily focuses on the decoding process, progressively utilizing knowledge generated by a knowledge system to guide the symbol prediction stage in the decoder. Within the decoder, this invention designs two modules: a knowledge verifier module and a knowledge injector. In step t, the knowledge verifier processes the knowledge K obtained from the knowledge generator. t ′ The rationality of the data is verified and filtered. The knowledge injector then injects the filtered knowledge into the symbol prediction at step t, which is used to assist in the generation of y. t .

[0082] (2.2) Knowledge Verifier.

[0083] Knowledge K Inspired by Large Language Models t ′ There may be redundancy, for example Figure 2 The system might simultaneously trigger correct but irrelevant knowledge such as "bananas and oranges are both fruits," which could introduce noise into the model's knowledge application. Therefore, a knowledge validator is used to encode the knowledge obtained from the triggers. This encoded knowledge is then used in conjunction with the reasoning target vector of the current reasoning step to validate and filter the knowledge, resulting in filtered knowledge.

[0084] First, the encoding is performed in the following manner:

[0085]

[0086] in, Representing knowledge K t The first word in ′ is related to the rth word; for The corresponding encoding vector, f θ For text encoders (such as BERT, T5, etc.); To define the symbol, K t For knowledge K t The encoded vector of ′, k t Encoding knowledge.

[0087] Then, combined with the inference target vector s t and knowledge encoding k t Computational knowledge K t ′ The fit score and fit probability p for the current inference step t :

[0088] score = tanh(W) v1 ·[s t ,k t ]+b v1 )

[0089] p t =σ(W v2 ·score+b v2 )

[0090] Where σ is the sigmoid function, W v1 W v2 Let b be the weight matrix. v1 ,b v2 is the bias term, and tanh is the hyperbolic activation function.

[0091] Based on the above-mentioned fit probability, this invention uses a threshold ε>0 to match knowledge K. t ′ Perform filtering if p t If the value is greater than ε, then the filtered knowledge is the knowledge encoding vector K. t Otherwise, the filtered knowledge will be a zero vector.

[0092] To train the knowledge validator, we need to minimize the following training objective L. v , where δ t Is it appropriate to filter knowledge K in step t if ∈{0,1}? t ′ The true label is δ.t =1, otherwise 0.

[0093]

[0094] Among them, L v Let m be the loss of the knowledge verifier, and m be the number of reasoning steps.

[0095] (2.3) Knowledge injector.

[0096] The knowledge injector injects filtered knowledge into symbol prediction. It is mainly used to calculate the attention weight of the filtered knowledge by combining the inference target vector of the current inference step, and to calculate the attention weight of the question text by combining the knowledge encoding and the text representation vector of the question text. Then, it updates the inference target vector of the current inference step by combining the two calculated attention weights.

[0097] Inspired by the dual mechanisms of human cognition, on the one hand, filtered knowledge is expressed in textual form, which is often sparse. During reasoning, one tends to focus on the most important parts (e.g., the formula "total weight = quantity multiplied by unit weight"). On the other hand, based on knowledge, humans can, in turn, find the most relevant information in the question text (e.g., find "the unit weight of bananas" and "the total number of boxes"), which can further reduce the difficulty of reasoning.

[0098] Based on this idea, the present invention designs the following dual attention reasoning mechanism to calculate the attention weight of the filtered knowledge and the attention weight of the question text:

[0099]

[0100] in, The attention weight for the i-th word in the filtered knowledge. The attention weight of the j-th word in the question text. W is the encoding vector of the i-th word in the filtered knowledge. k1 W k2 W h1 W h2 All are weight matrices; softmax is the normalized exponential function, tanh is the hyperbolic activation function, t indicates that the current inference step is step t, and s t Let k be the inference target vector. t Encoding knowledge; h j Let be the text representation vector of the j-th word in the question text.

[0101] This invention combines the two attention weights mentioned above to update the inference target vector s. t The update method is as follows:

[0102]

[0103] in, The attention weight for the i-th word in the filtered knowledge. The attention weight of the j-th word in the question text. h is the encoding vector of the i-th word in the filtered knowledge. j Let be the text representation vector of the j-th word in the question text. β1, β2 > 0 are hyperparameters that balance the two attention weights. If β1 is larger, it means that more attention needs to be paid to the knowledge that needs to be stimulated during reasoning; if β2 is larger, it means that more attention needs to be paid to the information already present in the question. ′ t This is the updated inference target vector for the current inference step t.

[0104] (2.4) Decoding module.

[0105] The decoding module is also an important component of the decoder. It is used to calculate the symbol of the current inference step using the updated inference target vector of the current inference step, and to generate the inference target vector of the next inference step.

[0106] In this embodiment of the invention, the updated inference target vector s is combined with ′ t Calculate the sign of the current inference step t, denoted as:

[0107] p(y t |y1,y2,...,y t-1 = softmax(Dec1(y1,y2,...,y t-1 ,s′ t ))

[0108] Where y1, y2, ..., y t-1 This represents a symbol that has already been generated, with the subscript corresponding to the sequence number of the reasoning step, p(y t |y1,y2,...,y t-1 () indicates generating the symbol y for the current inference step t, given the already generated symbols. t The probability; softmax is the normalized exponential function, and Dec1 is a decoding module.

[0109] The reasoning target vector s for the next reasoning step is generated using the following formula. t+1 :

[0110] s t+1 =Dec2(s t ′,e(y t ))

[0111] Dec2 is another decoding module; e(yt ) represents the symbol y t The embedding vector.

[0112] For example, Dec1 and Dec2 can be implemented using the output gate and forget gate in the LSTM model.

[0113] 4. Model training.

[0114] In this embodiment of the invention, the intelligent problem-solving framework needs to be trained in advance, and the training loss function L is expressed as:

[0115] L=∑ P ∑ t -logp(y t |y1,y2,...,y t-1 )+β·L v

[0116] Where P represents the mathematical application problem to which the problem text belongs; t represents the current reasoning step as step t, y1, y2, ..., y t-1 This represents a symbol that has already been generated, with the subscript corresponding to the sequence number of the reasoning step, p(y t |y1,y2,...,y t-1 () indicates generating the symbol y for the current inference step t, given the already generated symbols. t The probability of L; v Let β be the loss function of the knowledge validator, and β>0 be the hyperparameter balancing the symbol prediction loss function (i.e., the first term in the above equation) and the knowledge filter loss function.

[0117] The model parameters are optimized based on the training loss function until a stopping condition is met (e.g., training reaches the set maximum number of iterations or the loss function converges). After that, the solution task can be completed based on the aforementioned steps 1-4. The parameter optimization process involved in this part can be implemented with reference to conventional techniques, so it will not be described in detail.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0119] Example 2

[0120] This invention also provides an intelligent solution system for mathematical word problems that integrates knowledge from large-scale models. This system is primarily used to implement the methods provided in the foregoing embodiments, such as... Figure 4 As shown, the system mainly includes:

[0121] The input unit (performing step 1 above) is used to input the question text;

[0122] The knowledge system (which performs step 2 above) is used to generate the knowledge required for the current reasoning step based on the question text and the generated symbols, using a large language model.

[0123] The reasoning system (performing step 3 above) encodes the triggered knowledge, verifies and filters the knowledge using the obtained knowledge encoding and the reasoning target vector of the current reasoning step, obtains the filtered knowledge, calculates the attention weight of the filtered knowledge by combining the reasoning target vector of the current reasoning step, and calculates the attention weight of the question text by combining the knowledge encoding and the text representation vector of the question text; updates the reasoning target vector of the current reasoning step by combining the two calculated attention weights; calculates the symbol of the current reasoning step by combining the updated reasoning target vector of the current reasoning step, and generates the reasoning target vector of the next reasoning step; wherein, if the current reasoning step is the first step, the reasoning target vector of the current reasoning step is the initial reasoning target vector obtained using the question text; all symbols are generated by repeated execution and combined in order to form the solution expression;

[0124] The solver unit (which performs step 4 above) is used to solve for the answer using the solver expression.

[0125] Since the main technical details involved in this system have been described in detail in previous embodiments, they will not be repeated here.

[0126] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0127] Example 3

[0128] The present invention also provides a processing device, such as Figure 5 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the foregoing embodiments.

[0129] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0130] In this embodiment of the invention, the specific types of the memory, input device, and output device are not limited; for example:

[0131] Input devices can be touchscreens, image acquisition devices, physical buttons, or mice, etc.

[0132] The output device can be a display terminal;

[0133] The memory can be random access memory (RAM) or non-volatile memory, such as disk storage.

[0134] Example 4

[0135] The present invention also provides a readable storage medium storing a computer program that, when executed by a processor, implements the method provided in the foregoing embodiments.

[0136] In this embodiment of the invention, the readable storage medium is a computer-readable storage medium and can be disposed in the aforementioned processing device, for example, as a memory in the processing device. Furthermore, the readable storage medium can also be any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0137] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An intelligent solution method for mathematical word problems that integrates knowledge from large-scale models, characterized in that, include: Step 1: Enter the question text; Step 2: In the current reasoning step, based on the question text and the generated symbols, stimulate the knowledge required for the current reasoning step based on the large language model; Step 3: Encode the knowledge generated in Step 2, verify and filter the knowledge using the obtained knowledge code and the reasoning target vector of the current reasoning step, obtain the filtered knowledge, calculate the attention weight of the filtered knowledge in combination with the reasoning target vector of the current reasoning step, and calculate the attention weight of the question text in combination with the knowledge code and the text representation vector of the question text. Then, the inference target vector of the current inference step is updated by combining the two calculated attention weights; using the updated inference target vector of the current inference step, the symbol of the current inference step is calculated, and the inference target vector of the next inference step is generated; where, if the current inference step is the first step, the inference target vector of the current inference step is the initial inference target vector obtained using the question text; by repeatedly executing steps 2 and 3, all symbols are generated and combined in order into the solution expression; Step 4: Solve the expression to get the answer.

2. The intelligent solution method for mathematical application problems integrating large-scale model knowledge as described in claim 1, characterized in that, Encoding the knowledge generated in step 2 includes: Let the current reasoning step be denoted as step t, and the knowledge triggered by step 2 be denoted as K. t ′, is encoded in the following way: in, Representing knowledge K t The first word in ′ is related to the rth word; for The corresponding encoding vector, f θ For text encoders; To define the symbol, K t For knowledge K t The encoded vector of ′, k t Encoding knowledge.

3. The intelligent solution method for mathematical application problems that integrates large-scale model knowledge according to claim 2, characterized in that, The obtained knowledge encoding is used in conjunction with the reasoning target vector of the current reasoning step to verify and filter the knowledge, resulting in the following filtered knowledge: Let the current reasoning step be denoted as step t, and the reasoning target vector be denoted as s. t The knowledge encoding is denoted as k. t ; Computational knowledge K′ t The fit score and fit probability p for the current inference step t t : score=tanh(W v1 ·[s t ,k t ]+b v1 ) p t =σ(W v2 ·score+b v2 ) Where σ is the sigmoid function, W v1 W v2 Let b be the weight matrix. v1 ,b v2 Here, tanh is the bias term, and tanh is the hyperbolic activation function. If p t If the value is greater than ε, then the filtered knowledge is the knowledge encoding vector K. t Otherwise, the filtered knowledge will be a zero vector.

4. The intelligent solution method for mathematical application problems integrating large-scale model knowledge as described in claim 1, characterized in that, The calculation of the attention weight of the filtered knowledge by combining the reasoning target vector of the current reasoning step, and the calculation of the attention weight of the question text by combining the knowledge encoding and the text representation vector of the question text, include: The following dual attention reasoning mechanism is designed to calculate the attention weights of the filtered knowledge and the question text: in, The attention weight for the i-th word in the filtered knowledge. The attention weight of the j-th word in the question text. W is the encoding vector of the i-th word in the filtered knowledge. k1 W k2 W h1 W h2 All are weight matrices; softmax is the normalized exponential function, tanh is the hyperbolic activation function, t indicates that the current inference step is step t, and s t Let k be the inference target vector. t Encoding knowledge; h j Let be the text representation vector of the j-th word in the question text.

5. The intelligent solution method for mathematical application problems integrating large-scale model knowledge as described in claim 1, characterized in that, The method of updating the inference target vector for the current inference step by combining the two calculated attention weights includes: Let the current reasoning step be denoted as step t, and the reasoning target vector be denoted as s. t The update method is as follows: in, The attention weight for the i-th word in the filtered knowledge. The attention weight of the j-th word in the question text. h is the encoding vector of the i-th word in the filtered knowledge. j Let be the text representation vector of the j-th word in the question text, and β1 and β2 be hyperparameters balancing the two attention weights; s′ t This is the updated inference target vector for the current inference step t.

6. The intelligent solution method for mathematical application problems integrating large-scale model knowledge as described in claim 1, characterized in that, The step of combining the updated inference target vector of the current inference step, calculating the sign of the current inference step, and generating the inference target vector of the next inference step includes: Let the current inference step be denoted as step t, and the updated inference target vector for the current inference step t be denoted as s′. t The symbolic representation of calculating the current inference step t is: p(and t |y1,y2,...,y t-1 )=softmax(Dec1(y1,y2,...,y t-1 ,s′ t )) Where y1, y2, ..., y t-1 This represents a symbol that has already been generated, with the subscript corresponding to the sequence number of the reasoning step, p(y t |y1,y2,...,y t-1 () indicates generating the symbol y for the current inference step t, given the already generated symbols. t The probability; softmax is the normalized exponential function, and Dec1 is a decoding module; The reasoning target vector s for the next reasoning step is generated using the following formula. t+1 : s t+1 =Dec2(s′ t ,e(y t )) Dec2 is another decoding module; e(y t ) represents the symbol y t The embedding vector.

7. The intelligent solution method for mathematical application problems integrating large-scale model knowledge as described in claim 1, characterized in that, Step 2 is implemented through a knowledge system, and step 3 is implemented through a reasoning system. The reasoning system adopts an encoder-decoder structure. The encoder is responsible for encoding the question text to obtain the text representation vector of the question text and the initial reasoning target vector. The decoder includes a knowledge verifier, a knowledge injector, and a decoding module. The knowledge verifier is used to encode the knowledge triggered in step 2, and uses the obtained knowledge encoding and the reasoning target vector of the current reasoning step to verify and filter the knowledge to obtain filtered knowledge. The knowledge injector is used to calculate the attention weight of the filtered knowledge by combining the reasoning target vector of the current reasoning step, and to calculate the attention weight of the question text by combining the knowledge encoding and the text representation vector of the question text. The inference target vector for the current inference step is updated by combining the two calculated attention weights; the decoding module is used to calculate the symbol for the current inference step and generate the inference target vector for the next inference step using the updated inference target vector for the current inference step. The knowledge system and reasoning system together form an intelligent problem-solving framework; this intelligent problem-solving framework is pre-trained, and the training loss function L is expressed as: L=∑ p ∑ t -log p(y t |y1,y2,...,y t-1 )+β·L v Where P represents the mathematical application problem to which the problem text belongs; t represents the current reasoning step as step t, y1, y2, ..., y t-1 This represents a symbol that has already been generated, with the subscript corresponding to the sequence number of the reasoning step, p(y t |y1,y2,...,y t-1 () indicates generating the symbol y for the current inference step t, given the already generated symbols. t The probability of L; v Let β be the loss function of the knowledge validator, and β>0 be a hyperparameter. When the knowledge validator verifies and filters knowledge, it generates a fit score p for the knowledge triggered in step 2 to the current reasoning step t. t The loss function calculated from this is expressed as: Among them, L v The loss of the knowledge verifier is denoted by δ, where m is the number of reasoning steps and δ is the value of δ. t ∈{0,1} is the true label of the current inference step t.

8. An intelligent solution system for mathematical word problems that integrates large-scale model knowledge, characterized in that, include: The input unit is used to input the question text; A knowledge system is used to generate the knowledge required for the current reasoning step based on the question text and the generated symbols, using a large language model. The reasoning system is used to encode the knowledge triggered in step 2, verify and filter the knowledge using the obtained knowledge encoding and the reasoning target vector of the current reasoning step, obtain the filtered knowledge, calculate the attention weight of the filtered knowledge in combination with the reasoning target vector of the current reasoning step, and calculate the attention weight of the question text in combination with the knowledge encoding and the text representation vector of the question text. The inference target vector of the current inference step is updated by combining the two calculated attention weights; the symbol of the current inference step is calculated by combining the updated inference target vector of the current inference step, and the inference target vector of the next inference step is generated; if the current inference step is the first step, the inference target vector of the current inference step is the initial inference target vector obtained by using the question text; all symbols are generated by repeating the process and combined in order to form the solution expression. The solver unit is used to solve for the answer using the solver expression.

9. A processing device, characterized in that, include: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images