Numerical reasoning method, system and storage medium based on reasoning step concatenation

Abstract

The application provides a numerical reasoning method and system based on reasoning step concatenation and a storage medium, and comprises the following steps: constructing a node representation of a calculation graph according to a numerical reasoning problem in a text form based on a numerical reasoning model trained based on a target function; decoding an operation type according to the node representation of the graph to determine an operation type distribution probability; concatenating operation numbers matched with each operation to determine the probability of the operation numbers and generate a directed acyclic graph; and selecting a numerical operation result of a graph node as an answer to the problem by the numerical reasoning model. The application solves the problem that the existing numerical reasoning model is not flexible enough in modeling the logic dependency between reasoning steps and has a high error rate.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a numerical reasoning method, system, and storage medium based on the concatenation of reasoning steps. Background Technology

[0002] Numerical reasoning is an important research area in artificial intelligence, capable of efficiently extracting and utilizing the information value of the vast numerical data existing in natural language text, and has enormous application potential in fields such as education and financial decision-making. Existing datasets for evaluating the numerical reasoning capabilities of models generally take the form of question-and-answer questions: given a natural language question, the model needs to perform numerical reasoning on the text, execute necessary mathematical operations, and then provide an answer. The challenge of this task lies in the fact that the model needs to perform comprehensive and structured parsing of the text, while simultaneously achieving accurate and complex inferences.

[0003] Existing technologies discussing numerical reasoning models based on neural networks and deep learning typically rely on the following fundamental elements: Encoding representation of the numerical reasoning problem. Typical solutions often employ pre-trained language models to obtain the encoded representation of the text problem. Some solutions design model structures or pre-trained tasks to enhance the numerical representation in the problem. Decoding methods for mathematical operations. Typical solutions include serialization decoding and structured decoding: serialization decoding treats structured mathematical expressions as sequences and performs autoregressive generation from left to right; structured decoding converts mathematical expressions into directed acyclic computation graphs and autoregressively generates mathematical operation steps according to a pre-defined graph traversal order (such as preorder traversal). Experiments have shown that structured decoding has advantages over serialization decoding.

[0004] At present, such typical technical solutions have achieved certain results, but there are still two drawbacks: (1) the preset decoding order will cause unnecessary burden on the logical dependence between mathematical operations learned by the model; (2) usually only one mathematical operation process is decoded, and the accuracy of a single decoding is required to be high. If the intermediate steps are wrong, the result will be wrong.

[0005] This patent proposes a numerical reasoning device based on the concatenation of reasoning steps. By using parallel decoding of mathematical operations, it eliminates the pre-defined decoding dependency. The model itself can more flexibly capture the logical relationships between operations. In addition, by comparing diverse operation steps and concatenating the most relevant operations to form the final reasoning scheme, the error rate can be significantly reduced. Summary of the Invention

[0006] This invention provides a numerical reasoning method, system, and storage medium based on the concatenation of reasoning steps, in order to solve the problems of insufficient flexibility and high error rate in the modeling of logical dependencies between reasoning steps in existing numerical reasoning models.

[0007] This invention provides a numerical reasoning method based on the concatenation of reasoning steps, comprising:

[0008] Based on the numerical reasoning model trained with the objective function, a node representation of the computational graph is constructed according to the numerical reasoning problem in text form.

[0009] Decode the operation type based on the graph nodes to determine the probability distribution of the operation type;

[0010] In each step of the operation, the operands that match it are concatenated to determine the probability of the operands and generate a directed acyclic graph;

[0011] The numerical reasoning model selects the numerical computation result of a graph node as the answer to the question.

[0012] According to the present invention, a numerical reasoning method based on the concatenation of reasoning steps is provided, wherein the method constructs a node representation of a computational graph based on a numerical reasoning problem in text form using a numerical reasoning model trained with an objective function, specifically including:

[0013] Given a numerical reasoning problem, determine the set of numerical values, encode the problem using a pre-trained encoder, and extract the encoded representation of the corresponding positions of the numerical values ​​as the semantic vector of the numerical values.

[0014] Based on the encoded representation of the numerical reasoning problem, multiple vector representations are generated through a pre-defined directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes and encoding the semantics of multiple mathematical operations.

[0015] According to the present invention, a numerical reasoning method based on the concatenation of reasoning steps is provided. The method involves generating multiple vector representations using a preset directed acyclic graph decoder, based on the encoded representation of the numerical reasoning problem, as semantic vectors for multiple graph nodes, encoding the semantics of multiple mathematical operations. Specifically, this includes:

[0016] The type of operation, the semantics of the operands, and the semantics of the computation result;

[0017] Specific computational subgraphs are formed using the semantic vectors of graph nodes.

[0018] According to the present invention, a numerical reasoning method based on the concatenation of reasoning steps is provided, which decodes the operation type based on the graph nodes and determines the operation type distribution probability, specifically including:

[0019] A computational operation is predicted at each graph node using a numerical inference model;

[0020] Predict the probability distribution of operation types on the set graph nodes.

[0021] According to the present invention, a numerical reasoning method based on the concatenation of reasoning steps is provided, wherein in each step of the calculation, the operands matching it are concatenated to determine the probability of the operands and generate a directed acyclic graph, specifically including:

[0022] Define a matrix representing all values, and concatenate each operation with the best-matching operand;

[0023] Predict the probability distributions of the first operand and the second operand on the defined graph nodes;

[0024] By using a probabilistic mask to ensure that each graph node cannot use itself or a graph node with a larger index as an operand, a directed acyclic graph is generated.

[0025] According to the present invention, a numerical reasoning method based on the concatenation of reasoning steps is provided, wherein the numerical calculation result of a graph node is selected as the answer to the problem through the numerical reasoning model, specifically including:

[0026] Based on the directed acyclic graph, select the result of the operation on one of the graph nodes as the answer to the problem;

[0027] The selected graph nodes and their dependent descendant nodes constitute the final computational subgraph, representing the numerical reasoning process of the problem, determining the predicted distribution probability, and calculating the numerical result.

[0028] The present invention also provides a numerical reasoning system based on the concatenation of reasoning steps, the system comprising:

[0029] The text question and graph node representation module is used to construct node representations of computational graphs based on numerical reasoning questions in text form, using a numerical reasoning model trained with an objective function.

[0030] The operation type decoding module is used to decode the operation type based on the graph nodes and determine the operation type distribution probability.

[0031] The concatenation inference module is used to concatenate the operands that match it in each step of the calculation, determine the probability of the operands, and generate a directed acyclic graph.

[0032] The subgraph determination module is used to select the numerical calculation result of a graph node as the answer to the question through the numerical reasoning model.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the numerical reasoning method based on the concatenation of reasoning steps as described above.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the numerical reasoning method based on the concatenation of reasoning steps as described above.

[0035] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a numerical reasoning method based on the concatenation of reasoning steps as described above.

[0036] The present invention provides a numerical reasoning method, system and storage medium based on the concatenation of reasoning steps. By using parallel decoding of mathematical operations, the pre-defined decoding dependency is eliminated. The model itself can more flexibly capture the logical relationships between operations. In addition, by comparing diverse operation steps and concatenating the most relevant operations to form the final reasoning scheme, errors can be significantly reduced and the accuracy of reasoning logic can be improved. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is one of the flowcharts of a numerical reasoning method based on the concatenation of reasoning steps provided by the present invention;

[0039] Figure 2 This is the second flowchart of a numerical reasoning method based on the concatenation of reasoning steps provided by the present invention;

[0040] Figure 3 This is the third flowchart of a numerical reasoning method based on the concatenation of reasoning steps provided by the present invention;

[0041] Figure 4 This is the fourth flowchart of a numerical reasoning method based on the concatenation of reasoning steps provided by the present invention;

[0042] Figure 5 This is the fifth flowchart of a numerical reasoning method based on the concatenation of reasoning steps provided by the present invention;

[0043] Figure 6 This is a schematic diagram of the module connection of a numerical reasoning system based on the sequential connection of reasoning steps provided by the present invention;

[0044] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0045] Figure label:

[0046] 110: Text Problem and Graph Node Representation Module; 120: Operation Type Decoding Module; 130: Serial Reasoning Module; 140: Subgraph Determination Module;

[0047] 710: Processor; 720: Communication interface; 730: Memory; 740: Communication bus. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] The following is combined with Figures 1-5 This invention describes a numerical reasoning method based on the concatenation of reasoning steps, comprising:

[0050] S100. Based on the numerical reasoning model trained with the objective function, construct the node representation of the computational graph according to the numerical reasoning problem in text form;

[0051] S200. Decode the operation type based on the graph nodes to determine the operation type distribution probability;

[0052] S300. In each step of the operation, the operands that match it are concatenated to determine the probability of the operands and generate a directed acyclic graph.

[0053] S400. Select the numerical calculation result of a graph node as the answer to the question through the numerical reasoning model.

[0054] This invention eliminates the pre-defined decoding dependency by using parallel decoding mathematical operations. The model itself can capture the logical relationships between operations more flexibly. In addition, by comparing diverse operation steps and connecting the most relevant operations to form the final reasoning scheme, the error rate can be significantly reduced.

[0055] Based on the numerical inference model trained with the objective function, a node representation of the computational graph is constructed according to the textual question, specifically including:

[0056] S101. Obtain a given numerical reasoning problem, determine the set of numerical values, encode the problem using a pre-trained encoder, and extract the encoded representation of the corresponding position of the numerical values ​​as the semantic vector of the numerical values.

[0057] S102. Based on the encoded representation of the numerical reasoning problem, multiple vector representations are generated through a preset directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes, and encoding the semantics of multiple mathematical operations.

[0058] In this invention, a numerical reasoning problem X is given, and it is assumed that the set of numbers it contains is N = {n1, n2, ... n}. |N| The model first encodes the question using a pre-trained encoder (such as RoBERTa) and extracts the encoded representations of the corresponding positions of the numerical values ​​as semantic vectors for the values. A predefined set of constant numerical values ​​C = {c1, c2, ..., c...} |C| The vector is represented using random initialization.

[0059] Based on the encoded representation of problem X, the model uses a Transformer-based Directed Acyclic Graph (DAG) decoder to generate L vector representations V = {v1, v2, ..., v...} in parallel. L}, as the semantic vector of L graph nodes, is responsible for encoding the semantics of L mathematical operations: including the type of operation, the semantics of the operands, and the semantics of the calculation result.

[0060] Based on the encoded representation of the numerical reasoning problem, multiple vector representations are generated through a pre-defined directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes to encode the semantics of multiple mathematical operations, specifically including:

[0061] The type of operation, the semantics of the operands, and the semantics of the computation result;

[0062] Specific computational subgraphs are formed using the semantic vectors of graph nodes.

[0063] Subsequently, the semantic vectors of the graph nodes are used to form a specific computational subgraph Z = {z1, z2, ..., z}. |Z|},in Representing the pth j Binary operations on graph nodes It refers to the type of operation, such as addition, subtraction, multiplication, and division. It has two operands. The operation z j The position index of the node in the graph satisfies 1 ≤ p1 <p2<…<p |Z| ≤L, p |Z| It is the index of the root node of the directed acyclic graph Z.

[0064] Based on the graph nodes, the operation type is decoded to determine the probability distribution of the operation type, specifically including:

[0065] S201. Predict an operation on each graph node using a numerical reasoning model;

[0066] S202. Predict the probability distribution of operation types on the set graph nodes.

[0067] The numerical inference model predicts one computational operation at each graph node. At the p-th node... j Prediction operation type on each graph node The probability distribution is defined as:

[0068]

[0069] In each step of the computation, the operands that match it are concatenated to determine the probability of the operands, generating a directed acyclic graph, specifically including:

[0070] S301. Define a representation matrix for all values, and concatenate each operation with the best-matching operand.

[0071] S302. Predict the probability distribution of the first operand and the probability distribution of the second operand on the defined graph nodes;

[0072] S303. By using a probability mask, ensure that each graph node cannot use itself or a graph node with a larger index as an operand, and generate a directed acyclic graph.

[0073] In this invention, each operation is concatenated with its best-matching operand. The representation matrix for all numerical values ​​is defined as Q = [V, C, N], where C and N are the semantic representation matrices of the predefined numerical constant C and the numerical value N contained in the problem, respectively. At the p-th... j Predicting the first operand on each graph node. The probability distribution is defined as:

[0074]

[0075] Predict the second operand The probability distribution can be similarly defined as:

[0076]

[0077] To prevent the computation graph from forming cycles, a probability mask is used to ensure that each graph node cannot use itself or a graph node with a larger index as an operand.

[0078] The numerical reasoning model selects the numerical computation result of a graph node as the answer to the question, specifically including:

[0079] S401. Based on the directed acyclic graph, select the result of the operation on one of the graph nodes as the answer to the problem;

[0080] S402. The selected graph nodes and their dependent descendant nodes constitute the final computational subgraph, representing the numerical reasoning process of the problem, determining the predicted distribution probability, and calculating the numerical result.

[0081] The numerical inference model obtains a directed acyclic graph of size L. It then selects the computation result of one of the graph nodes as the answer to the problem. The selected graph node and its dependent descendant nodes constitute the final computational subgraph Z, representing the numerical inference process of problem X. The predicted probability distribution of the indices of the selected graph node—the root node of Z—is defined as follows:

[0082]

[0083] Where v L+1 It is a vector representation used for root node selection, obtaining v L+1 The method is the same as the method for representing and calculating graph nodes.

[0084] In this invention, after determining the numerical reasoning model, the numerical reasoning model is trained, and a new training method is defined to make it more in line with actual use needs.

[0085] First, define the method for generating probability decomposition, given the labeled correct computation process Y = {y1, y2, ..., y...} |Y|},in It is a binary operation. It is an operation type. There are two operands, and the model generation probability can be decomposed as follows:

[0086]

[0087] Where P θ (Y|Z,X) is 1 if and only if Z is equivalent to the operation y i Mapped to the A graph node, otherwise P θ (Y|Z,X) is 0. Therefore, P θ (Y|X) can be rewritten as follows:

[0088]

[0089] Any Z∈Γ is a directed acyclic graph representation of Y. For a given Y, Γ can be obtained as follows: (1) Enumerate all possible combinations of indices {p1,…,p |Y|} such that 1≤p1<… <p |Y| ≤L, (2) will y i Mapped to p i On each graph node Therefore, the size of Γ

[0090] P θ (Z|X) can be decomposed into:

[0091]

[0092]

[0093] Next, we define the objective function as follows:

[0094]

[0095] When the number of graph nodes L exceeds the actual number of operations required (such as the number of operations contained in Y), optimizing the objective function described above allows different graph nodes to capture a wider variety of operations (reasoning steps). By comparing these diverse operations, the model is more likely to form a logically correct computational process.

[0096] However, since Γ may be large, directly optimizing the above Loss is computationally expensive, so an approximate Loss function optimization model can be used.

[0097] In this specific example, the pre-trained encoder uses RoBERTa-base, and the decoder consists of two TransformerLayer layers. The hyperparameter settings of each layer are the same as those of the decoder layer of the pre-trained model T5-base.

[0098] The training method is as follows: the numerical inference model is trained using an approximate marginal likelihood maximization approach. This is because directly calculating P... θ The marginal likelihood of (Z|X) is costly, so an independence assumption is introduced, which allows P to be approximated in polynomial time using dynamic programming algorithms. θ The marginal likelihood of (Z|X). Specifically, for each computational step y in the target computation process Y. i Assuming and The computational steps in the two subgraphs of the root node are independently mapped to the graph nodes.

[0099] Training was performed using the AdamW optimizer with a learning rate of 2e-5, 32 samples per update, and the number of graph nodes L was preset to 60.

[0100] The numerical reasoning model is used as follows: Input question X is fed into the question encoder, and the DAG decoder obtains the computational subgraph Z. The result of the operation at the root node of Z is used as the answer to the question.

[0101] The numerical reasoning model was trained on the MAWPS and ASDiv-A datasets and tested on the SVAMP dataset. Compared to the best model, DeductReasoner, which does not use this patented method, the answer accuracy improved from 45.1% to 49.6%, and the reasoning logic accuracy improved from 44.6% to 49.2%.

[0102] In the embodiment based on annealing Hard EM, unlike the edge likelihood maximization algorithm, the objective function of Hard EM is:

[0103] Loss = -log P θ (Z * |X),Z * =arg max Z∈Γ P θ (Z|X)

[0104] According to P θ The decomposition of (Z|X) is approximated by using a cluster search method to determine the optimal subgraph Z. * Due to the early stages of model training, Z * The quality of the data may be poor. To avoid affecting the model performance, we use MML as the initialization for Hard EM: first use MML to optimize for τ=2000 steps, and then use Hard EM to optimize the model.

[0105] The model was trained and tested on the MathQA dataset. Compared to the best model, DeductReasoner, which does not use this patented method, the answer accuracy improved from 78.6% to 82.9%, and the reasoning accuracy improved from 74.7% to 79.2%.

[0106] The numerical reasoning method based on the concatenation of reasoning steps provided by this invention eliminates the pre-defined decoding dependency by decoding mathematical operations in parallel. The model itself can more flexibly capture the logical relationships between operations. In addition, by comparing diverse operation steps and concatenating the most relevant operations to form the final reasoning scheme, errors can be significantly reduced and the accuracy of reasoning logic can be improved.

[0107] refer to Figure 6 The present invention also discloses a numerical reasoning system based on the concatenation of reasoning steps, the system comprising:

[0108] The text question and graph node representation module 110 is used to construct a node representation of a computational graph based on a numerical reasoning model trained with an objective function and a numerical reasoning question in text form.

[0109] The operation type decoding module 120 is used to decode the operation type based on the graph nodes and determine the operation type distribution probability.

[0110] The concatenated inference module 130 is used to concatenate the operands that match it in each step of the calculation, determine the probability of the operands, and generate a directed acyclic graph.

[0111] The subgraph determination module 140 is used to select the numerical calculation result of a graph node as the answer to the question through the numerical reasoning model.

[0112] Among them, the text question and graph node representation module 110 obtains a given numerical reasoning question, determines the set of numerical values, encodes the question through a pre-trained encoder, and extracts the encoded representation of the corresponding position of the contained numerical values ​​as the semantic vector of the numerical values.

[0113] Based on the encoded representation of the numerical reasoning problem, multiple vector representations are generated through a pre-defined directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes and encoding the semantics of multiple mathematical operations.

[0114] The operation type decoding module 120 predicts an operation on each graph node through a numerical reasoning model.

[0115] Predict the probability distribution of operation types on the set graph nodes.

[0116] The inference module 130 is connected in series, which defines the representation matrix of all values ​​and connects each operation with the best matching operand.

[0117] Predict the probability distributions of the first operand and the second operand on the defined graph nodes;

[0118] By using a probabilistic mask to ensure that each graph node cannot use itself or a graph node with a larger index as an operand, a directed acyclic graph is generated.

[0119] The subgraph determination module 140 selects the operation result of one of the graph nodes as the answer to the problem based on the directed acyclic graph.

[0120] The selected graph nodes and their dependent descendant nodes constitute the final computational subgraph, representing the numerical reasoning process of the problem, determining the predicted distribution probability, and calculating the numerical result.

[0121] The numerical reasoning system based on the concatenation of reasoning steps provided by this invention eliminates the pre-defined decoding dependency by decoding mathematical operations in parallel. The model itself can more flexibly capture the logical relationships between operations. In addition, by comparing diverse operation steps and concatenating the most relevant operations to form the final reasoning scheme, errors can be significantly reduced and the accuracy of reasoning logic can be improved.

[0122] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a numerical reasoning method based on the concatenation of reasoning steps. This method includes: constructing a node representation of a computational graph based on a numerical reasoning model trained with an objective function and a text-based numerical reasoning problem;

[0123] Decode the operation type based on the graph nodes to determine the probability distribution of the operation type;

[0124] In each step of the operation, the operands that match it are concatenated to determine the probability of the operands and generate a directed acyclic graph;

[0125] The numerical reasoning model selects the numerical computation result of a graph node as the answer to the question.

[0126] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute a numerical reasoning method based on the concatenation of reasoning steps provided by the above methods, the method including: constructing a node representation of a computational graph based on a numerical reasoning problem in text form according to a numerical reasoning model trained by an objective function;

[0128] Decode the operation type based on the graph nodes to determine the probability distribution of the operation type;

[0129] In each step of the operation, the operands that match it are concatenated to determine the probability of the operands and generate a directed acyclic graph;

[0130] The numerical reasoning model selects the numerical computation result of a graph node as the answer to the question.

[0131] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a numerical reasoning method based on the concatenation of reasoning steps provided by the above methods, the method comprising: constructing a node representation of a computational graph based on a numerical reasoning model trained with an objective function and a numerical reasoning problem in text form;

[0132] Decode the operation type based on the graph nodes to determine the probability distribution of the operation type;

[0133] In each step of the operation, the operands that match it are concatenated to determine the probability of the operands and generate a directed acyclic graph;

[0134] The numerical reasoning model selects the numerical computation result of a graph node as the answer to the question.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A numerical reasoning method based on the concatenation of reasoning steps, characterized in that, include: Based on the numerical reasoning model trained with the objective function, a node representation of the computational graph is constructed according to the numerical reasoning problem in text form. Decode the operation type based on the graph nodes to determine the probability distribution of the operation type; In each step of the operation, the operands that match it are concatenated to determine the probability of the operands and generate a directed acyclic graph; The numerical reasoning model selects the numerical calculation result of a graph node as the answer to the question. The node representation of the computational graph constructed based on the textual question, using a numerical inference model trained with an objective function, specifically includes: Given a numerical reasoning problem, determine the set of numerical values, encode the problem using a pre-trained encoder, and extract the encoded representation of the corresponding positions of the numerical values ​​as the semantic vector of the numerical values. Based on the encoded representation of the numerical reasoning problem, multiple vector representations are generated through a preset directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes to encode the semantics of multiple mathematical operations. The step of concatenating the operands that match it in each operation step, determining the probability of the operands, and generating a directed acyclic graph specifically includes: Define a matrix representing all values, and concatenate each operation with the best-matching operand; Predict the probability distributions of the first operand and the second operand on the defined graph nodes; By using a probabilistic mask to ensure that each graph node cannot use itself or a graph node with a larger index as an operand, a directed acyclic graph is generated.

2. The numerical reasoning method based on the concatenation of reasoning steps according to claim 1, characterized in that, The step of generating multiple vector representations based on the encoded representation of the numerical reasoning problem using a preset directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes, and encoding the semantics of multiple mathematical operations, specifically includes: The type of operation, the semantics of the operands, and the semantics of the computation result; Specific computational subgraphs are formed using the semantic vectors of graph nodes.

3. The numerical reasoning method based on the concatenation of reasoning steps according to claim 1, characterized in that, Based on the graph nodes, the operation type is decoded to determine the probability distribution of the operation type, specifically including: A computational operation is predicted at each graph node using a numerical inference model; Predict the probability distribution of operation types on the set graph nodes.

4. The numerical reasoning method based on the concatenation of reasoning steps according to claim 1, characterized in that, The numerical reasoning model selects the numerical computation result of a graph node as the answer to the question, specifically including: Based on the directed acyclic graph, select the result of the operation on one of the graph nodes as the answer to the problem; The selected graph nodes and their dependent descendant nodes constitute the final computational subgraph, representing the numerical reasoning process of the problem, determining the predicted distribution probability, and calculating the numerical result.

5. A numerical reasoning system based on the concatenation of reasoning steps, characterized in that, The system includes: The text problem and graph node representation module is used to construct node representations of computational graphs based on numerical reasoning problems in text form, using a numerical reasoning model trained with an objective function. The operation type decoding module is used to decode the operation type based on the graph nodes and determine the operation type distribution probability. The concatenation inference module is used to concatenate the operands that match it in each step of the calculation, determine the probability of the operands, and generate a directed acyclic graph. The subgraph determination module is used to select the numerical calculation result of a graph node as the answer to the question through the numerical reasoning model; The node representation of the computational graph constructed based on the textual question, using a numerical inference model trained with an objective function, specifically includes: Given a numerical reasoning problem, determine the set of numerical values, encode the problem using a pre-trained encoder, and extract the encoded representation of the corresponding positions of the numerical values ​​as the semantic vector of the numerical values. Based on the encoded representation of the numerical reasoning problem, multiple vector representations are generated through a preset directed acyclic graph decoder, serving as semantic vectors for multiple graph nodes to encode the semantics of multiple mathematical operations. The step of concatenating the operands that match it in each operation step, determining the probability of the operands, and generating a directed acyclic graph specifically includes: Define a matrix representing all values, and concatenate each operation with the best-matching operand; Predict the probability distributions of the first operand and the second operand on the defined graph nodes; By using a probabilistic mask to ensure that each graph node cannot use itself or a graph node with a larger index as an operand, a directed acyclic graph is generated.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the numerical reasoning method based on the concatenation of reasoning steps as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the numerical reasoning method based on the concatenation of reasoning steps as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the numerical reasoning method based on the concatenation of reasoning steps as described in any one of claims 1 to 4.

Images