Method, device, equipment, medium and product for generating arithmetic expression
By obtaining tables and text, determining heterogeneous graphs and using graph processing models and tree decoders to generate arithmetic expressions, the problem of low accuracy of arithmetic expressions in the prior art is solved, and effective processing and accurate answers to complex problems are achieved. Sex improvement.
Patent Information
- Application Number
- CN202210754115.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-06-28
Smart Images

Figure CN115309858B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical fields of natural language processing and artificial intelligence, and in particular to a method, device, equipment, medium and product for generating an arithmetic expression. Background Art
[0002] Question answering system is a system that can use various types of knowledge to answer natural language questions. It is an important system related to the field of natural language processing (NLP). At present, the table-text hybrid question answering system processes mainly two types of questions: fact retrieval type and numerical reasoning type.
[0003] In the related art, for numerical reasoning problems, the following method is used to obtain answers: predict the type of problem, select an operator based on the type, obtain an empty expression to be filled with data based on the selected operator, search for data in table data and text data, obtain data to be filled in the empty expression, fill the data into the empty expression, obtain an arithmetic expression; and calculate the arithmetic expression to obtain the answer.
[0004] In the above-mentioned related technologies, the type of the problem is predicted, an operator is selected based on the type, and an empty expression to be filled with data is obtained based on the selected operator, resulting in an erroneous empty expression, which in turn makes the accuracy of the arithmetic expression low. Summary of the invention
[0005] The present invention provides a method, device, equipment, medium and product for generating an arithmetic expression, so as to solve the problem of low accuracy of arithmetic expressions in the prior art.
[0006] The present invention provides a method for generating an arithmetic expression, comprising:
[0007] Obtaining a table and text corresponding to the target problem; the table and text are used to determine an arithmetic expression of the target problem;
[0008] Based on the table, text and target question, determine the heterogeneous graph of the target question;
[0009] Based on the graph processing model, determine the target embedding matrix of the heterogeneous graph;
[0010] Through the tree decoder, the target embedding matrix and the preset symbol dictionary are processed to obtain the arithmetic expression of the target problem.
[0011] According to a method for generating an arithmetic expression provided by the present invention, a heterogeneous graph of a target problem is determined based on a table, a text and a target problem, including: determining multiple nodes based on the table, the text and the target problem; the multiple nodes include cells in the table, words in the text, sentence identifiers of sentences in the text and words in the target problem; performing structural recognition on the table and the text to obtain structural information; creating multiple edges corresponding to multiple nodes based on the types, structural information and pre-defined edge relationships of multiple nodes; and creating a heterogeneous graph of the target problem based on the multiple nodes and the multiple edges.
[0012] According to a method for generating an arithmetic expression provided by the present invention, a target embedding matrix of a heterogeneous graph is determined based on a graph processing model, including: determining an initial embedding matrix corresponding to the heterogeneous graph based on the edge relationship of multiple edges and multiple nodes included in the heterogeneous graph through a graph input layer in the graph processing model; and processing the initial embedding matrix through a graph hidden layer in the graph processing model to obtain a target embedding matrix of the heterogeneous graph.
[0013] According to a method for generating an arithmetic expression provided by the present invention, an initial embedding matrix includes node initial vectors of each node and edge initial vectors of each edge in a heterogeneous graph; an initial embedding matrix corresponding to the heterogeneous graph is determined through a graph input layer in a graph processing model based on edge relations of multiple edges and multiple nodes included in the heterogeneous graph, including: for each edge, the graph input layer obtains a vector corresponding to the edge relation from a preset parameter matrix based on the edge relation of the edge, and determines the vector corresponding to the edge relation as the edge initial vector of the edge; for each node, when the node is a sentence identifier of a sentence in a text, a word in the text, or a word in a target question, a pre-trained language model (Pre-trained Language Model, PLM) in the graph input layer is used to process N sub-nodes of the node to obtain respective vectors of the N sub-nodes; a word vector generation formula is used to process respective vectors of the N sub-nodes to obtain a node initial vector of the node; when the node is a cell in a table, if the node includes multiple words, a word vector aggregation formula is used to aggregate the node initial vectors of the multiple words to obtain a node initial vector of the node.
[0014] According to a method for generating an arithmetic expression provided by the present invention, the word vector generation formula is: Among them, w is the node initial vector of the node, softmax is the normalization operation, tanh is the inverse tangent function, e i is the vector of the ith subnode in the node, W s and is a trainable parameter;
[0015] The word vector aggregation formula is: Where x represents the node initial vector of the node, W s and W n is a trainable parameter, w i Represents the node initialization vector of the i-th word in the node.
[0016] According to a method for generating an arithmetic expression provided by the present invention, a target embedding matrix and a preset symbol dictionary are processed through a tree decoder to obtain an arithmetic expression of a target problem, including: processing the target embedding matrix and the preset symbol dictionary through a tree decoder to determine the Polish form corresponding to the target problem; converting the Polish form into an operation tree corresponding to the target problem; and determining the arithmetic expression of the target problem based on the operation tree.
[0017] The present invention also provides a device for generating an arithmetic expression, comprising:
[0018] An acquisition module is used to acquire a table and text corresponding to a target problem; the table and text are used to determine an arithmetic expression of the target problem;
[0019] A determination module, for determining a heterogeneous graph of a target problem based on a table, text, and the target problem;
[0020] The determination module is also used to determine the target embedding matrix of the heterogeneous graph based on the graph processing model;
[0021] The determination module is also used to process the target embedding matrix and the preset symbol dictionary through the tree decoder to obtain the arithmetic expression of the target problem.
[0022] According to a device for generating an arithmetic expression provided by the present invention, a determination module is specifically used to: determine multiple nodes based on a table, a text and a target problem; the multiple nodes include cells in the table, words in the text, sentence identifiers of sentences in the text and words in the target problem; perform structural recognition on the table and the text to obtain structural information; create multiple edges corresponding to the multiple nodes based on the types, structural information and pre-defined edge relationships of the multiple nodes; and create a heterogeneous graph of the target problem based on the multiple nodes and the multiple edges.
[0023] According to a device for generating an arithmetic expression provided by the present invention, the determination module is also specifically used to: determine, through a graph input layer in a graph processing model, an initial embedding matrix corresponding to the heterogeneous graph based on the edge relationship of multiple edges and multiple nodes included in the heterogeneous graph; and process the initial embedding matrix through a graph hidden layer in the graph processing model to obtain a target embedding matrix of the heterogeneous graph.
[0024] According to a device for generating an arithmetic expression provided by the present invention, the initial embedding matrix includes the node initial vector of each node in the heterogeneous graph and the edge initial vector of each edge; the determination module is also specifically used for: for each edge, the graph input layer obtains the vector corresponding to the edge relationship from the preset parameter matrix based on the edge relationship of the edge, and determines the vector corresponding to the edge relationship as the edge initial vector of the edge; for each node, when the node is a sentence identifier of a sentence in a text, a word in the text, or a word in a target question, the N sub-nodes of the node are processed by the pre-trained language model (Pre-trained Language Model, PLM) in the graph input layer to obtain the vectors of the N sub-nodes respectively; the vectors of the N sub-nodes are processed by the word vector generation formula to obtain the node initial vector of the node; when the node is a cell in a table, if the node includes multiple words, the node initial vectors of the multiple words are aggregated by the word vector aggregation formula to obtain the node initial vector of the node.
[0025] According to a device for generating an arithmetic expression provided by the present invention, the word vector generation formula is: Among them, w is the node initial vector of the node, softmax is the normalization operation, tanh is the inverse tangent function, e i is the vector of the ith subnode in the node, W s and is a trainable parameter;
[0026] The word vector aggregation formula is: Where x represents the node initial vector of the node, W s and W n is a trainable parameter, w i Represents the node initialization vector of the i-th word in the node.
[0027] According to a device for generating an arithmetic expression provided by the present invention, the determination module is also specifically used to: process the target embedding matrix and the preset symbol dictionary through a tree decoder to determine the Polish form corresponding to the target problem; convert the Polish form into an operation tree corresponding to the target problem; and determine the arithmetic expression of the target problem based on the operation tree.
[0028] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for generating any arithmetic expression as described above is implemented.
[0029] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for generating an arithmetic expression.
[0030] The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned methods for generating arithmetic expressions when executed by a processor.
[0031] The present invention provides a method, device, equipment, medium and product for generating an arithmetic expression. In the method for generating an arithmetic expression, a target embedding matrix of a heterogeneous graph is determined based on a graph processing model; the target embedding matrix and a preset symbol dictionary are processed through a tree decoder to obtain an arithmetic expression of a target problem, without selecting an operator based on the type of problem, thereby avoiding the problem of obtaining an erroneous empty expression and improving the accuracy of the arithmetic expression. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0033] Figure 1 A schematic diagram of an application scenario provided by the present invention;
[0034] Figure 2 A schematic diagram of a flow chart of a method for generating an arithmetic expression provided for the invention;
[0035] Figure 3 Schematic diagram of tables and text provided for the present invention;
[0036] Figure 4 A schematic diagram of the structure of a heterogeneous graph provided by the present invention;
[0037] Figure 5 A schematic diagram of the structure of the graph processing model provided for the invention;
[0038] Figure 6 A schematic diagram of the structure of the operation tree provided by the present invention
[0039] Figure 7 It is a schematic diagram of the structure of the device for generating an arithmetic expression provided by the present invention;
[0040] Figure 8 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] In the related art, the type of question is predicted, an operator is selected based on the type, an empty expression to be filled with data is obtained based on the selected operator, the data corresponding to the question is searched in the table data and text data, the data to be filled in the empty expression is obtained, the data is filled in the empty expression, an arithmetic expression is obtained, and the arithmetic expression is calculated to obtain the answer. For example, in the case where the type of question is to find the product, the operator "×" is selected, and the empty expression obtained is "_×_", and if the data corresponding to the question is 2 and 3, the arithmetic expression is "2×3".
[0043] In the above-mentioned related technologies, the type of problem is predicted, an operator is selected based on the type, and an empty expression to be filled with data is obtained based on the selected operator. This is only suitable for expressions of simple problems, and cannot obtain expressions of complex problems (such as "(_+_+_)×_"). Therefore, for complex problems, an erroneous empty expression may be obtained, which in turn makes the accuracy of the arithmetic expression low.
[0044] In the present invention, in order to improve arithmetic expressions, the inventors have come up with a method for generating arithmetic expressions. Based on tables, texts and target problems, a heterogeneous graph of the target problem is determined, and then an arithmetic expression is obtained based on the heterogeneous graph, thereby avoiding the problem of obtaining an erroneous empty expression due to selecting an operator based on the type of problem, and improving the accuracy of the arithmetic expression.
[0045] Combine the following Figure 1 The application scenario of the method for generating an arithmetic expression provided by the present invention is described. Figure 1 Schematic diagram of application scenarios provided by the present invention. Figure 1 As shown, it includes: a question-answering system, a target question, a table and a text. The question-answering system is used to process the target question, the table and the text to obtain an arithmetic expression of the target question.
[0046] The method for generating an arithmetic expression provided by the present invention is described below in conjunction with specific embodiments.
[0047] Figure 2 A flow chart of the method for generating an arithmetic expression provided by the invention. Figure 2 As shown, the method includes:
[0048] Step S201, obtaining a table and text corresponding to the target problem; the table and text are used to determine the arithmetic expression of the target problem.
[0049] Optionally, the execution subject of the embodiment of the present invention may be an electronic device, or a device for generating an arithmetic expression disposed on the electronic device, and the device for generating an arithmetic expression may be implemented by a combination of software and / or hardware. The electronic device may be, for example, a computer or other device.
[0050] Combine the following Figure 3 Describe the tables and texts corresponding to the target questions. Figure 3 Schematic diagram of the tables and text provided by the present invention. Figure 3 As shown, it exemplarily includes: multiple tables and multiple texts. Figure 3 In the figure, a solid line box is used to indicate a table, and a dotted line box is used to indicate a text. Based on 3, the target question is, for example, at least one of the following:
[0051] Question 1: What percentage of the total price of restricted stock granted from 2017 to 2018 was the compensation expense for outstanding restricted stock in 2018?
[0052] Question 2: What was the percentage change in the total price of non-vested shares from December 31, 2018 to 2019?
[0053] Step S202: Determine a heterogeneous graph of the target problem based on the table, the text, and the target problem.
[0054] A heterogeneous graph includes multiple nodes and multiple edges. Multiple nodes include cells in a table, sentence tags in a text, words in a text, and words in a target question. Multiple edges are edges between multiple nodes. Figure 4 A heterogeneous graph is described by way of example. Figure 4 This is a schematic diagram of the structure of the heterogeneous graph provided by the present invention. Figure 4 As shown, the exemplary heterogeneous graph includes: 17 nodes.
[0055] Among them, 5 nodes (from the target question) are words in the target question, 6 nodes (from the text) are sentence tags and words in the text, and 6 nodes (from the table) are cells in the table.
[0056] Exemplarily, in a heterogeneous graph, there may be edges represented by solid lines between nodes with the same source, and there may be edges represented by dotted lines between nodes with different sources.
[0057] Step S203: Determine a target embedding matrix of the heterogeneous graph based on the graph processing model.
[0058] The graph processing model is used to process heterogeneous graphs and obtain the target embedding matrix of the heterogeneous graphs. For a detailed description of the graph processing model, see Figure 5 Example.
[0059] The target embedding matrix includes multiple target vectors, wherein the multiple target vectors include node target vectors corresponding to each node and edge target vectors corresponding to each edge.
[0060] Step S204: Process the target embedding matrix and the preset symbol dictionary through the tree decoder to obtain an arithmetic expression of the target problem.
[0061] Among them, the preset symbol dictionary is V op ∪V con , where V con Represents a constant, V op Indicates an operator. For example, V con ={1, AVG}, AVG represents the average of the answers based on the obtained arithmetic expression, V op ={+,-,×,÷}.
[0062] For example, based on the target problem being problem 1, Figure 3 , the arithmetic expression of question 1 is obtained through the tree decoder: 109.7 / (0.41×278.29). Further, 109.7 / (0.41×278.29) is calculated and the answer to question 1 is 0.96.
[0063] For example, based on the target problem 2, Figure 3 , through the tree decoder, we can get the arithmetic expression of question 2 as ((0.709×275.00)-(0.739×225.93)) / (0.739×225.93). Further, by calculating ((0.709×275.00)-(0.739×225.93)) / (0.739×225.93), we get the answer to question 2 as 16.78.
[0064] exist Figure 2 In the method for generating an arithmetic expression provided in the embodiment, a heterogeneous graph of the target problem is determined based on a table, text and target problem; a target embedding matrix of the heterogeneous graph is determined based on a graph processing model; the target embedding matrix and a preset symbol dictionary are processed through a tree decoder to obtain an arithmetic expression of the target problem, without selecting an operator based on the type of problem, thereby avoiding the problem of obtaining an erroneous empty expression and improving the accuracy of the arithmetic expression.
[0065] Based on the above embodiment, step S202 is described in detail below:
[0066] Optionally, step S202 specifically includes:
[0067] Based on a table, text and a target question, multiple nodes are determined; the multiple nodes include cells in the table, words in the text, sentence tags of sentences in the text and words in the target question; structural recognition is performed on the table and the text to obtain structural information; based on the types, structural information and pre-defined edge relationships of the multiple nodes, multiple edges corresponding to the multiple nodes are created; and based on the multiple nodes and the multiple edges, a heterogeneous graph of the target question is created.
[0068] Optionally, based on the table, the text and the target question, a plurality of nodes are determined, including: based on space characters, the table, the text and the target question are split respectively to obtain a plurality of nodes.
[0069] Optionally, structural recognition is performed on tables and texts to obtain structural information, including: based on a table structure perception algorithm, the structure of the table is recognized and processed to obtain table structural information; based on a paragraph recognition algorithm, the structure of the text is recognized and processed to obtain text structural information; based on a string matching algorithm, the structural matching relationship between the table and the text is recognized and processed to obtain matching structural information; the table structural information, the text structural information and the matching structural information are determined as structural information.
[0070] Optionally, based on different sources, the types of nodes can be divided into the following three types: table type, question type, and text type.
[0071] Optionally, a table type node may be divided into, for example, a table row name (T row ), table column name (T column ), table cell (T cell ) and special signs (T time ); question type nodes are divided into question words (Q word ), text type nodes are divided into text words (P word ) and sentence marker (P sentence ).
[0072] In the present invention, since the special row and column names may be time (such as 2019), a special flag T is set. time It should be noted that the table row names and column names are composed of text, and the table cells are composed of numbers.
[0073] The predefined edge relations include intra-type relations and inter-type relations. Intra-type relations refer to the edge relations between nodes of the same type. For example, row 、T column 、T cell and T timeThese four types of nodes are all table types, so the edge relationships between these four types of nodes can be called intra-type edge relationships. Among them, inter-type relationships belong to the edge relationships between nodes of different types. For example, Q word (of the type of question) and T row (Belongs to table type) The edge relationship between these two types of nodes belongs to the inter-type edge relationship. For example, the intra-type relationship is shown in Table 1 below, and the inter-type relationship is shown in Table 2 below.
[0074] Table 1 Relationships within types
[0075]
[0076] Table 2 Relationship between types
[0077]
[0078]
[0079] The heterogeneous graph G in the present invention can be expressed as:
[0080] G=(V,R),V=Q∪T∪P,R=R i ∪R C ;
[0081] Among them, Q represents the question type, T represents the table type, P represents the text type, and R i Represents the intra-type relationship, R C Represents relationships between types.
[0082] The total number of multiple nodes is expressed as:
[0083]
[0084] Among them, |Q| represents the total number of words in the target question, |T| represents the total number of cells, and |s i | represents the total number of words included in the i-th sentence in the text.
[0085] Figure 5 The structural diagram of the graph processing model provided by the invention. Figure 5 As shown, the graph processing model includes: a graph input layer and a graph hidden layer.
[0086] Optionally, based on the graph processing model, a target embedding matrix of the heterogeneous graph is determined, including: through a graph input layer, based on the edge relationships of multiple edges and multiple nodes included in the heterogeneous graph, an initial embedding matrix corresponding to the heterogeneous graph is determined; through a graph hidden layer, the initial embedding matrix is processed to obtain a target embedding matrix of the heterogeneous graph.
[0087] The initial embedding matrix includes the edge initial vector of each edge and the node initial vector of each node in the heterogeneous graph.
[0088] For each edge, an initial edge vector is obtained based on the following method: based on the edge relationship of the edge, a vector corresponding to the edge relationship is obtained from a preset parameter matrix; the preset parameter matrix includes vectors corresponding to each edge relationship; the vector corresponding to the edge relationship is determined as the initial edge vector.
[0089] It should be noted that the initial edge vector is Among them, V e represents the edges included in the heterogeneous graph, d represents the dimension of the initial vector of the edge, Represents the matrix composed of edge initial vectors.
[0090] The node initial vector of each node is obtained based on the following method:
[0091] For each node, when the node is a sentence identifier of a sentence in the text, a word in the text, or a word in the target question, the pre-trained language model (PLM) in the graph input layer processes the N sub-nodes of the node to obtain the vectors of the N sub-nodes; the vectors of the N sub-nodes are processed through the word vector generation formula to obtain the node initial vector of the node;
[0092] In the case where the node is a cell in a table, if the node includes multiple words, the node initial vectors of the multiple words are aggregated using the word vector aggregation formula to obtain the node initial vector of the node.
[0093] Optionally, in the present invention: multiple nodes may be grouped into a node sequence, and the node sequence may be discretized to obtain N secondary nodes of each node; N is an integer greater than or equal to 1.
[0094] For example, the node sequence is:
[0095] [CLS]q1q2…q |Q| [SEP]t1t2…t |T| [SEP]s 10 s 11 …s |P|0 s |P|1 …[SEP]; where q |Q| represents the |Q|th word in the target question, t |T| represents the |T|th cell in the table, s |P|0It represents the first word in the |P|th sentence in the text, [CLS] represents the starting position of the node sequence, and [SEP] is the spacing symbol between nodes of different types (including table type, question type and text type).
[0096] For example, in the case where the node is integration and transformation-related costs, the N sub-nodes of the node include integration, and, transformation, related, costs.
[0097] The pre-trained language model is a Robustly Optimized Bidirectional Encoder Representation from Transformers (RoBERTa).
[0098] The word vector generation formula is:
[0099]
[0100] Among them, w is the node initial vector of the node, softmax is the normalization operation, tanh is the inverse tangent function, e i is the vector of the ith subnode in the node, W s and is a trainable parameter.
[0101] The word vector aggregation formula is:
[0102]
[0103] Where x represents the node initial vector of the node, W s and W n is a trainable parameter, w i represents the node initial vector of the i-th word in the node. i It is obtained based on the word vector generation formula.
[0104] After each vector in the initial embedding matrix corresponding to the heterogeneous graph passes through the graph hidden layer, a target vector is obtained. For example, after the node initial vector in the initial embedding matrix passes through the graph hidden layer, a node target vector is obtained, and after the edge initial vector passes through the graph hidden layer, an edge target vector is obtained.
[0105] Optionally, the graph hidden layer is composed of M relation-aware graph attention networks (R-GATs) connected in series, where M is an integer greater than or equal to 1. For the mth R-GAT among the M R-GATs, the mth R-GAT obtains the mth output vector based on Formula 3:
[0106]
[0107] in, represents the attention weight of the mth R-GAT, represents the normalized attention weight in the mth R-GAT, represents the mth output vector, represents the parameter matrix that the mth R-GAT can learn, i represents the i-th node, j represents the node that has an edge relationship with the i-th node, x i represents the vector corresponding to the i-th node input to the m-th R-GAT, x j represents the vector corresponding to the jth node of the mth R-GAT, r ij represents the edge vector between the i-th node and the j-th node, represents the receptive field of the i-th node, d represents the dimension of the vector, and H represents the number of attention heads included in the m-th R-GAT.
[0108] Optionally, the target embedding matrix is processed by a tree decoder to obtain an arithmetic expression of the target problem, including: processing the target embedding matrix and a preset symbol dictionary by a tree decoder to determine the Polish form corresponding to the target problem; converting the Polish form into an operation tree corresponding to the target problem; and determining the arithmetic expression of the target problem based on the operation tree.
[0109] In the present invention, a target embedding matrix and a preset symbol dictionary are processed through a tree decoder to determine the Polish form corresponding to the target problem; the Polish form is converted into an operation tree corresponding to the target problem; based on the operation tree, the arithmetic expression of the target problem is determined without selecting an operator based on the type of problem, and an empty expression is generated, thereby improving the accuracy of the arithmetic expression.
[0110] Among them, the Polish form includes nodes. For example, based on the target problem being problem 2, the nodes determined in the Polish form include: / -×0.709 275.00×0.739 225.93×0.739 225.93. Further, the operation tree obtained based on the Polish form is as follows Figure 6 shown. Figure 6 This is a schematic diagram of the structure of the operation tree provided by the present invention.
[0111] Optionally, the tree decoder may include a multi-layer perceptron, an arithmetic generation model, and a text generation model; the target embedding matrix is subjected to binary classification by the multi-layer perceptron to obtain the processing type corresponding to the target embedding matrix; the processing type is a span extraction class or an arithmetic class; when the processing type is an arithmetic class, the arithmetic generation model in the tree decoder is used to determine the arithmetic expression corresponding to the target question based on the target embedding matrix; and the arithmetic expression is calculated to obtain the numerical answer to the target question. When the processing type is a span extraction class, the text generation model is used to obtain the text answer to the target question based on the target embedding matrix. Among them, the span extraction class includes: span class, multi-span class, and counting class.
[0112] Optionally, the tree decoder may be a bidirectional gated recurrent unit (GRU).
[0113] The tree decoder obtains the nodes in the Polish form based on the following formula 4:
[0114]
[0115] Among them, p c represents the probability of selecting a node (i.e., determining the node in the Polish formula among multiple nodes), σ represents the sigmoid function, and W z represents the trainable parameter matrix, s t+1 represents the state of the tree decoder at the t+1th time step, c t Represents the mixed context state, r t represents the context state representation, P c (y t ) represents the probability of selecting a node at the tth time step, y t Represents the character or number generated at the tth time step, α ti represents the attention weight of the i-th node at the t-th time step, P g (y t ) represents the probability of selecting an operator or a constant at the tth time step, f(·) is the perceptron layer in the tree decoder, and P(y t |y <t ,X) represents the generation of y t The joint probability of , X represents the target problem.
[0116] The tree decoder updates the state at each time step t based on the following formula 5:
[0117] s t+1 =BiGRU([c t :g t :E(y t )],s t ) Formula 5;
[0118] Among them, s t+1 represents the state of the t+1th time step, BiGRU represents a bidirectional gated recurrent network, and g t represents the expression tree generation state at the tth time step, E(y t ) represents y t The embedded representation of .
[0119] E(y t ) is calculated as:
[0120]
[0121] Among them, M op represents the embedding vector of the operator, M con represents a constant trainable embedding vector, V num Indicates the value in the table or text. For the value V num , whose embedding vector is the state of the hidden layer of the graph Related, loc(y t ,T,P) represents the index position y of the i-th node in the table and article t .
[0122] Based on automatic solution of math word problems (Math Word Problem Solving), g t The calculation formula is:
[0123] g t+1 =σ(W g [g t :g g,p :g t,l :g t,r ]) Formula 7;
[0124] Among them, g t+1 represents the expression tree generation state at the t+1th time step, σ represents the sigmoid function, W g is a trainable weight for the generated node g t , g g,p Indicates the parent node of the node, g t,l Indicates the left child of the node, g t,r Represents the right child of this node.
[0125] Based on the attention mechanism, c t The calculation formula is:
[0126]
[0127] Among them, α ti is the node target vector z of the ith nodei The attention distribution, W h and W s is a trainable parameter, z i The node target vector of the i-th node, and m represents the total number of all nodes included in the heterogeneous graph.
[0128] In order to verify the correctness of the method for generating arithmetic expressions provided by the present invention, the present invention is verified based on a tabular and textual dataset for question answering (Tabular And Textual dataset for Question Answering, TAT-QA) (i.e., a large-scale question answering dataset in a mixed form of tabular and text). The problems in TAT-QA include: numerical reasoning problems (arithmetic problems) and span extraction problems (non-arithmetic problems). TAT-QA is extracted from financial financial statements, and TAT-QA contains 2757 mixed data sources and 16552 corresponding question and answer pairs. TAT-QA contains four types of questions, namely span class, multi-span class, counting class and arithmetic class. TAT-QA is shown in Table 3 below.
[0129] Table 3
[0130] sheet text Tables and text Span Class 1801 3496 1842 Multi-span class 777 258 1037 Counting Class 106 5 266 Arithmetic 4747 143 2074
[0131] In the present invention, the span class, the multi-span class, the counting class and the arithmetic class are divided into two categories, namely, the arithmetic class and the span extraction class.
[0132] In the present invention, based on EM value and F1 value, the accuracy of the answers obtained by the method provided in the present invention (RegHNT) and the other five existing methods are shown in Table 4. Among them, the other five methods include: method 1 (human method, Human), method 2 (such as TAGOP), method 3 (such as LETTER), method 4 (such as KIQA), and method 5 (such as GSReasoner).
[0133] Table 4
[0134]
[0135]
[0136] Among them, DEV means EM and F1 are obtained based on the experimental set, and Test means EM and F1 are obtained based on the test set.
[0137] It can be seen from the calculated data in Table 4 that based on the method in the present invention, the maximum EM value and F1 value are obtained, so the method in the present invention can achieve the best performance (SOTA).
[0138] Furthermore, by comparing the method in the present invention with TAGOP, the EM value obtained by the method in the present invention is more than 20% higher than the EM value obtained by TAGOP; the F1 value obtained by the method in the present invention is more than 20% higher than the F1 value obtained by TAGOP. Therefore, the method proposed in this application can answer more questions with higher accuracy.
[0139] Table 5 exemplarily illustrates the EM value and F1 value obtained by RegHNT provided by the present invention and by TAGOP in the prior art.
[0140] Table 5
[0141]
[0142] In Table 5, in the following three cases, the accuracy of the answers obtained by RegHNT is significantly improved compared with TAGOP:
[0143] For span and multi-span categories, when the answer comes from a table, text, or a table and text;
[0144] For counting, when the answers come from tables and texts;
[0145] For Arithmetic Class, where the answers are derived from tables, tables and texts.
[0146] Specifically, compared with TAGOP, due to the existence of the tree decoder in RegHNT, the accuracy of the answers to arithmetic problems is improved by about 30%.
[0147] For span extraction questions, it can be observed from Table 5 that the accuracy of the answers obtained by RegHNT is improved compared with TAGOP.
[0148] For the case where the answer comes from tables and texts, when the type of the target question is arithmetic and span extraction, the accuracy of the answer obtained by RegHNT is improved by about 20% compared with TAGOP, which shows that the arithmetic expression generation method provided by the present invention can well solve the problem of mixed forms of tables and texts.
[0149] Table 6
[0150] Arithmetic expressions percentage(%) TAGOP RegHNT Simple Expression 41.8 45.0 79.3 Complex expressions 42.8 60.5 85.3 Undefined expression 15.4 0.0 61.8
[0151] For numerical reasoning problems, in order to verify RegHNT more accurately, arithmetic expressions are divided into simple expressions, complex expressions and undefined expressions based on their complexity. Simple expressions are expressions with only one operator, complex expressions are expressions containing multiple operators, and are usually used to calculate averages or change rates. Undefined expressions refer to expressions for which no template is defined in TAGOP.
[0152] Based on Table 6, it can be seen that for simple expressions and complex expressions, compared with TAGOP, the accuracy of the answers obtained by RegHNT is significantly improved (about 41.8% for simple expressions and about 42.8% for complex expressions).
[0153] Especially for undefined expressions, RegHNT can solve the expressions of undefined templates in TAGOP, which demonstrates the flexibility and generalization ability of RegHNT.
[0154] The device for generating an arithmetic expression provided by the present invention is described below. The device for generating an arithmetic expression described below and the method for generating an arithmetic expression described above can be referred to each other in correspondence.
[0155] Figure 7 Schematic diagram of the structure of the device for generating arithmetic expressions provided by the present invention. Figure 7 As shown, the generating device of the arithmetic expression includes:
[0156] An acquisition module 710 is used to acquire a table and text corresponding to the target problem; the table and text are used to determine the arithmetic expression of the target problem;
[0157] A determination module 720, for determining a heterogeneous graph of a target problem based on the table, the text, and the target problem;
[0158] The determination module 720 is further used to determine a target embedding matrix of the heterogeneous graph based on the graph processing model;
[0159] The determination module 720 is further used to process the target embedding matrix and the preset symbol dictionary through a tree decoder to obtain an arithmetic expression of the target problem.
[0160] According to a device for generating an arithmetic expression provided by the present invention, the determination module 720 is specifically used to: determine multiple nodes based on a table, a text and a target problem; the multiple nodes include cells in the table, words in the text, sentence identifiers of sentences in the text and words in the target problem; perform structural recognition on the table and the text to obtain structural information; create multiple edges corresponding to multiple nodes based on the types, structural information and pre-defined edge relationships of multiple nodes; and create a heterogeneous graph of the target problem based on the multiple nodes and the multiple edges.
[0161] According to a device for generating an arithmetic expression provided by the present invention, the determination module 720 is also specifically used to: determine the initial embedding matrix corresponding to the heterogeneous graph based on the edge relationship of multiple edges and multiple nodes included in the heterogeneous graph through the graph input layer in the graph processing model; and process the initial embedding matrix through the graph hidden layer in the graph processing model to obtain the target embedding matrix of the heterogeneous graph.
[0162] According to a device for generating an arithmetic expression provided by the present invention, the initial embedding matrix includes the node initial vector of each node and the edge initial vector of each edge in the heterogeneous graph; the determination module 720 is also specifically used for: for each edge, the graph input layer obtains the vector corresponding to the edge relationship from the preset parameter matrix based on the edge relationship of the edge, and determines the vector corresponding to the edge relationship as the edge initial vector of the edge; for each node, when the node is a sentence identifier of a sentence in a text, a word in the text, or a word in a target question, the pre-trained language model Pre-trained LanguageModel, PLM in the graph input layer is used to process the N sub-nodes of the node to obtain the respective vectors of the N sub-nodes; the vectors of the N sub-nodes are processed by the word vector generation formula to obtain the node initial vector of the node; when the node is a cell in a table, if the node includes multiple words, the node initial vectors of the multiple words are aggregated by the word vector aggregation formula to obtain the node initial vector of the node.
[0163] According to a device for generating an arithmetic expression provided by the present invention, the word vector generation formula is: Among them, w is the node initial vector of the node, softmax is the normalization operation, tanh is the inverse tangent function, e i is the vector of the ith subnode in the node, W s and is a trainable parameter;
[0164] The word vector aggregation formula is: Where x represents the node initial vector of the node, W s and W n is a trainable parameter, w i Represents the node initialization vector of the i-th word in the node.
[0165] According to a device for generating an arithmetic expression provided by the present invention, the determination module 720 is also specifically used to: process the target embedding matrix and the preset symbol dictionary through a tree decoder to determine the Polish form corresponding to the target problem; convert the Polish form into an operation tree corresponding to the target problem; and determine the arithmetic expression of the target problem based on the operation tree.
[0166] Figure 8 Schematic diagram of the physical structure of the electronic device provided by the present invention. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the method for generating an arithmetic expression, the method comprising: obtaining a table and text corresponding to the target problem; the table and the text are used to determine the arithmetic expression of the target problem; based on the table, the text and the target problem, a heterogeneous graph of the target problem is determined; based on the graph processing model, a target embedding matrix of the heterogeneous graph is determined; and the target embedding matrix and the preset symbol dictionary are processed by a tree decoder to obtain the arithmetic expression of the target problem.
[0167] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0168] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for generating arithmetic expressions provided by the above methods, which includes: obtaining a table and text corresponding to the target problem; using the table and text to determine the arithmetic expression of the target problem; determining a heterogeneous graph of the target problem based on the table, text and target problem; determining a target embedding matrix of the heterogeneous graph based on a graph processing model; and processing the target embedding matrix and a preset symbol dictionary through a tree decoder to obtain the arithmetic expression of the target problem.
[0169] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for generating an arithmetic expression provided by the above-mentioned methods, the method comprising: obtaining a table and text corresponding to the target problem; using the table and text to determine the arithmetic expression of the target problem; determining a heterogeneous graph of the target problem based on the table, text and target problem; determining a target embedding matrix of the heterogeneous graph based on a graph processing model; and processing the target embedding matrix and a preset symbol dictionary through a tree decoder to obtain the arithmetic expression of the target problem.
[0170] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0171] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.
[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for generating arithmetic expressions, It is characterized in that include: Get the table and text corresponding to the target question; The table and the text are used to determine an arithmetic expression of the target problem; Determining a heterogeneous graph of the target problem based on the table, the text, and the target problem; Determining a target embedding matrix of the heterogeneous graph based on a graph processing model; Processing the target embedding matrix and the preset symbol dictionary through a tree decoder to obtain an arithmetic expression of the target problem; The determining the target embedding matrix of the heterogeneous graph based on the graph processing model includes: Determining, through a graph input layer in the graph processing model, an initial embedding matrix corresponding to the heterogeneous graph based on edge relationships of a plurality of edges and a plurality of nodes included in the heterogeneous graph; Processing the initial embedding matrix through the graph hidden layer in the graph processing model to obtain a target embedding matrix of the heterogeneous graph; The initial embedding matrix includes node initial vectors of each node and edge initial vectors of each edge in the heterogeneous graph; the initial embedding matrix corresponding to the heterogeneous graph is determined based on the edge relationships of multiple edges and multiple nodes included in the heterogeneous graph through the graph input layer in the graph processing model, including: For each edge, the graph input layer obtains a vector corresponding to the edge relationship from a preset parameter matrix based on the edge relationship of the edge, and determines the vector corresponding to the edge relationship as the edge initial vector of the edge; For each node, when the node is a sentence identifier of a sentence in the text, a word in the text, or a word in the target question, the N sub-nodes of the node are processed by the pre-trained language model Pre-trained Language Model PLM in the graph input layer to obtain respective vectors of the N sub-nodes; the vectors of the N sub-nodes are processed by the word vector generation formula to obtain the node initial vector of the node; when the node is a cell in the table, if the node includes multiple words, the node initial vectors of the multiple words are aggregated by the word vector aggregation formula to obtain the node initial vector of the node; The word vector generation formula is: Wherein, w is the node initial vector of the node, softmax is the normalization operation, tanh is the inverse tangent function, e i is the vector of the ith subnode in the node, W s and is a trainable parameter; The word vector aggregation formula is: Where x represents the node initial vector of the node, W s and W n is a trainable parameter, w i A node initialization vector representing the i-th word in the node; The graph hidden layer is composed of M relation-aware graph attention networks (R-GATs) connected in series, where M is an integer greater than or equal to 1; for the mth R-GAT among the M R-GATs, the mth R-GAT obtains the mth output vector based on the following formula: in, represents the attention weight of the mth R-GAT, represents the normalized attention weight in the mth R-GAT, represents the mth output vector, represents the parameter matrix that the mth R-GAT can learn, i represents the i-th node, j represents the node that has an edge relationship with the i-th node, x i represents the vector corresponding to the i-th node input to the m-th R-GAT, x j represents the vector corresponding to the jth node of the mth R-GAT, r ij represents the edge vector between the i-th node and the j-th node, represents the receptive field of the i-th node, d represents the dimension of the vector, and H represents the number of attention heads included in the m-th R-GAT; The target embedding matrix and the preset symbol dictionary are processed by the tree decoder to obtain an arithmetic expression of the target problem, including: The target embedding matrix and the preset symbol dictionary are processed by the tree decoder to determine the Polish form corresponding to the target problem; the Polish form is converted into an operation tree corresponding to the target problem; and an arithmetic expression of the target problem is determined based on the operation tree.
2. The method for generating an arithmetic expression according to claim 1, It is characterized in that The determining, based on the table, the text and the target problem, a heterogeneous graph of the target problem comprises: Based on the table, the text and the target question, a plurality of nodes are determined; the plurality of nodes include cells in the table, words in the text, sentence identifiers of sentences in the text and words in the target question; Performing structural recognition on the table and the text to obtain structural information; Based on the types of the multiple nodes, the structural information and the predefined edge relationships, multiple edges corresponding to the multiple nodes are created; and based on the multiple nodes and the multiple edges, a heterogeneous graph of the target problem is created.
3. A device for generating an arithmetic expression, It is characterized in that include: The acquisition module is used to obtain the table and text corresponding to the target question; The table and the text are used to determine an arithmetic expression of the target problem; A determination module, configured to determine a heterogeneous graph of the target problem based on the table, the text, and the target problem; The determination module is further used to determine a target embedding matrix of the heterogeneous graph based on a graph processing model; The determination module is further configured to process the target embedding matrix and the preset symbol dictionary through a tree decoder to obtain an arithmetic expression of the target problem; The determination module is also specifically used to: determine the initial embedding matrix corresponding to the heterogeneous graph based on the edge relationship of multiple edges and multiple nodes included in the heterogeneous graph through the graph input layer in the graph processing model; process the initial embedding matrix through the graph hidden layer in the graph processing model to obtain the target embedding matrix of the heterogeneous graph; The initial embedding matrix includes the node initial vector of each node in the heterogeneous graph and the edge initial vector of each edge; the determination module is also specifically used for: for each edge, the graph input layer obtains the vector corresponding to the edge relationship from the preset parameter matrix based on the edge relationship of the edge, and determines the vector corresponding to the edge relationship as the edge initial vector of the edge; for each node, when the node is a sentence identifier of a sentence in a text, a word in a text, or a word in a target question, the pre-trained language model Pre-trained Language Model (PLM) in the graph input layer processes the N sub-nodes of the node to obtain the vectors of the N sub-nodes; the vectors of the N sub-nodes are processed by the word vector generation formula to obtain the node initial vector of the node; when the node is a cell in a table, if the node includes multiple words, the node initial vectors of the multiple words are aggregated by the word vector aggregation formula to obtain the node initial vector of the node; The word vector generation formula is: Among them, w is the node initial vector of the node, softmax is the normalization operation, tanh is the inverse tangent function, e i is the vector of the ith subnode in the node, W s and is a trainable parameter; The word vector aggregation formula is: Where x represents the node initial vector of the node, W s and W n is a trainable parameter, w i Represents the node initial vector of the i-th word in the node; The graph hidden layer is composed of M relation-aware graph attention networks (R-GATs) connected in series, where M is an integer greater than or equal to 1; for the mth R-GAT among the M R-GATs, the mth R-GAT obtains the mth output vector based on the following formula: in, represents the attention weight of the mth R-GAT, represents the normalized attention weight in the mth R-GAT, represents the mth output vector, represents the parameter matrix that the mth R-GAT can learn, i represents the i-th node, j represents the node that has an edge relationship with the i-th node, x i represents the vector corresponding to the i-th node input to the m-th R-GAT, x j represents the vector corresponding to the jth node of the mth R-GAT, r ij represents the edge vector between the i-th node and the j-th node, represents the receptive field of the i-th node, d represents the dimension of the vector, and H represents the number of attention heads included in the m-th R-GAT; The determination module is also specifically used to: process the target embedding matrix and the preset symbol dictionary through a tree decoder to determine the Polish form corresponding to the target problem; convert the Polish form into an operation tree corresponding to the target problem; and determine the arithmetic expression of the target problem based on the operation tree.
4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the method for generating an arithmetic expression according to any one of claims 1 to 2 is implemented.
5. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for generating an arithmetic expression as claimed in any one of claims 1 to 2 is implemented.
6. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method for generating an arithmetic expression as claimed in any one of claims 1 to 2 is implemented.
