Indicator knowledge graph guided large language model financial numerical reasoning method and system

Abstract

The application discloses a kind of index knowledge graph guided big language model financial numerical reasoning method and system, method includes: fusion multi-source financial knowledge constructs financial index knowledge graph, provides semantic and structural constraint for big language model;Financial query and document context are parsed, and the structured execution plan including reasoning type, aggregation function, subgoal and constraint condition is generated;Based on financial index knowledge graph, using parallel graph constraint reasoning mechanism, evidence search, index decomposition and program execution are recursively executed to each subgoal, and numerical calculation is completed along multiple candidate derivation paths;Cross path verification is carried out to candidate result, and final answer or refusal mark is output according to verification result.The application can effectively constrain the free generation behavior of big language model, avoid producing value fabrication without evidence when evidence is insufficient, and be suitable for financial report analysis, research report question and answer and financial text intelligent analysis etc.

Description

Technical Field

[0001] This invention belongs to the field of financial technology, specifically relating to a financial numerical reasoning method and system guided by an indicator knowledge graph and a large language model. Background Technology

[0002] In the field of intelligent financial text analysis, numerical reasoning tasks based on unstructured documents such as financial reports, research reports, and announcements are attracting widespread attention. These tasks are typical knowledge-intensive tasks, requiring models to understand the relevant knowledge and calculation formulas of the indicators to be calculated in financial problems, correctly extract key financial indicators from lengthy and complex texts, and derive accurate answers through complex multi-hop calculations (such as calculating growth rates, profit margins, and inter-period comparisons). This task places extremely high demands on the fidelity of numerical evidence within the context and the interpretability of the reasoning process; erroneous numerical outputs can lead to serious investment decision errors.

[0003] Existing financial numerical inference techniques primarily enhance model reasoning capabilities through two main approaches. One approach involves collecting a large number of numerical reasoning question-and-answer pairs for supervised training or fine-tuning, enabling the model to learn numerical computation patterns in the financial domain. However, this method essentially learns statistical patterns from the data rather than true reasoning ability, lacking the necessary generalization ability for long-tail indicators or complex multi-hop calculations not present in the training data. The other approach enhances large-scale model reasoning by introducing external knowledge sources such as knowledge graphs. However, the integration of external knowledge with large models is mostly for retrieval and supplementation, rather than providing structured guidance for the reasoning process, failing to fundamentally constrain the model's free-generative behavior. Both approaches typically provide calculation results end-to-end, making it difficult to output the multi-hop calculation process of complex indicators. Furthermore, neither training nor enhancement methods explicitly guide the model to refuse to answer unanswerable questions, leading to issues such as fabricating values ​​and forcibly outputting results when evidence is lacking, and an inability to distinguish between missing data and reasoning failures.

[0004] In summary, existing technologies have not yet solved two core problems in financial numerical reasoning: first, the lack of structured guidance in the model makes it difficult to backtrack the multi-hop calculation process; second, the lack of a secure rejection mechanism leads to the model fabricating values ​​and generating illusory answers when there is insufficient evidence. Summary of the Invention

[0005] To address the shortcomings of existing financial numerical reasoning techniques, this invention provides a large language model-based financial numerical reasoning method guided by an indicator knowledge graph. This method integrates the semantic generalization ability of a large language model with the structured constraint capabilities of a financial indicator knowledge graph. It designs four core steps: financial indicator knowledge graph construction, problem parsing, parallel graph-constrained reasoning, and cross-path verification, achieving accurate numerical reasoning and secure rejection of financial documents. This method ensures that the reasoning process is strictly limited to the scope of available evidence. By exhaustively enumerating all potential computational paths, it distinguishes between genuine data missingness and retrieval failures, preventing the large language model from fabricating unfounded numerical data when faced with ambiguous or missing evidence. This method has significant application value in the field of financial intelligent analysis.

[0006] According to one aspect of the present invention, a financial numerical reasoning method guided by an indicator knowledge graph is provided, comprising: By integrating multi-source financial knowledge, a financial indicator knowledge graph is constructed, which includes atomic nodes, derived nodes, dependency edges, candidate formula sets, and alias mapping sets. Using a problem parsing operator built on a large language model, unstructured financial queries and document contexts are parsed into structured execution plans, which include inference type, aggregation function, set of sub-objectives to be computed, and accuracy requirements. A parallel graph-constrained reasoning algorithm is used to organize evidence search operators, index decomposition operators, and program execution operators. Specifically: the index decomposition operator retrieves candidate derivation paths from the knowledge graph based on the sub-objective; the evidence search operator calls a large language model along the path to extract numerical evidence; and the program execution operator converts the calculation formulas and verified evidence in the candidate derivation paths into executable code and performs the calculation. For each sub-objective, the evidence search operator, index decomposition operator, and program execution operator are recursively called. When no valid evidence can be obtained from any of the derivation paths, a safe rejection is triggered, and a rejection flag is output. For candidate results generated by different derivation paths, the evidence basis, constraint consistency and calculation correctness are verified using a large language model, and the final answer is output through consensus voting; if no candidate result passes the verification, a rejection flag is output.

[0007] As a further technical solution, the construction of the financial indicator knowledge graph further includes: Financial terminology and computational logic were collected from function libraries, financial encyclopedias, and academic resources; Embedsion-based semantic clustering is performed on the collected term set, and semantic verification is performed using a large language model; Verified semantically similar terms are merged into canonical nodes, and the large language model completes the aliases, calculation formulas, and code snippets. Implement a hybrid verification process that combines human verification with large language models. A depth-first search strategy based on node access status is used to detect and eliminate loops.

[0008] As a further technical solution, the financial indicator knowledge graph is formalized as a directed acyclic graph G = (V, E, Φ, A), where V represents the set of financial variable nodes, divided into a set of atomic nodes and a set of derived nodes, E represents the set of dependency edges between nodes, Φ is the mapping from derived nodes to a set of candidate formulas, and A is the mapping from nodes to a set of language aliases; each derived node is associated with a set of candidate formulas; each node is mapped to a set of language aliases, used to align symbol nodes with unstructured text fragments.

[0009] As a further technical solution, the problem analysis includes: Receive financial documents and queries, construct question parsing prompts, and call a large language model to perform question parsing; Map queries to one of five predefined reasoning types: direct retrieval, arithmetic calculation, comparative reasoning, statistical aggregation, and temporal reasoning; and parse the reasoning type. Select the corresponding aggregate function from the predefined set of aggregate functions based on the reasoning type; Extract the set of operand sub-objectives, where each sub-objective includes a numerical indicator to be retrieved or calculated and its constraints, wherein the constraints include information in three dimensions: time range, entity range, and meta-attribute. Extract the accuracy requirements, output a structured execution plan, and pass it to the subsequent parallel graph constraint inference steps.

[0010] As a further technical solution, the parallel graph constraint reasoning includes: The recursive parsing function is called in parallel for each sub-objective output by the problem parsing step; In the recursive parsing function, the evidence search operator is first called to locate numerical evidence that satisfies the constraints in the document context; if numerical evidence is successfully found, the value is returned; if no numerical evidence is found and the target node is an atomic node, a rejection flag is returned; if no numerical evidence is found and the target node is a derived node, the index decomposition operator is called to obtain all candidate derivation paths. For each derivation path, recursive parsing of child nodes is performed in parallel; if all child nodes of the current path are successfully parsed, the program is called to execute operators to generate executable code and calculate candidate results; if at least one child node fails to obtain valid evidence, the path is marked as failed. If candidate results exist, proceed to the cross-path verification step; if no complete evidence can be obtained from any of the derivation paths, return a rejection flag. After all sub-goals are parsed, the aggregate function is applied to obtain the final answer; if there is a rejection flag in a sub-goal, the rejection is returned with a reason.

[0011] As a further technical solution, the evidence search operator utilizes the semantic capabilities of a large language model, combined with the alias set in the financial indicator knowledge graph, to search for numerical evidence that meets the constraints in financial documents. It strictly limits the process to only extracting evidence without performing any calculations. If the context does not contain a numerical value that meets the constraints, it returns a rejection flag and reports the missing indicator name. The indicator decomposition operator retrieves candidate paths from the financial indicator knowledge graph through bidirectional search. The main search direction is to traverse downwards to collect the child nodes of the target indicator and their decomposition formulas, while the auxiliary search direction is to explore upwards, limited to the range of one parent node. The program execution operator translates the symbolic derivation path and verified evidence into an executable program and executes it in a sandbox environment to obtain deterministic calculation results. If any value in the input evidence set is a rejection flag, the execution stops immediately and the rejection flag is propagated upwards.

[0012] As a further technical solution, the cross-path verification includes: Receive a set of candidate results, in which each candidate path includes a list of sub-targets and their constraints, evidence variable bindings and corresponding document evidence fragments, the applied calculation formulas and intermediate values, and the final numerical results and their meta-attributes; For each candidate path, a verification prompt is constructed, and a large language model is called as the judge to check each item from three dimensions: evidence basis, constraint consistency and calculation correctness. The decision of whether each candidate path is valid or invalid is output. Remove candidate paths deemed invalid and retain those deemed valid; If a valid candidate path exists, a consensus vote is performed to arrive at the final answer; if no candidate path is valid, a security rejection is triggered, and a rejection flag is returned.

[0013] According to one aspect of the present invention, a large language model financial numerical reasoning system guided by an indicator knowledge graph is provided, comprising: Financial Indicator Knowledge Graph Construction Module: Used to construct a financial indicator knowledge graph that includes atomic nodes, derived nodes, dependency edges, candidate formula sets, and alias mapping sets; Problem parsing module: Utilizes problem parsing operators built on a large language model to parse unstructured financial queries and documents into structured execution plans, which include inference type, aggregation function, sub-objective set, and accuracy requirements; Parallel graph-constrained reasoning module: used to coordinate the evidence search operator, index decomposition operator and program execution operator. The evidence search operator calls the large language model, the index decomposition operator retrieves candidate paths, the program execution operator generates code, and explores heterogeneous inference paths in parallel to achieve secure rejection. Cross-path validation module: Used to perform multi-dimensional validation of candidate results using a large language model, and output the final answer or a rejection mark.

[0014] According to one aspect of the present invention, a financial numerical reasoning device for large language models guided by an indicator knowledge graph is provided, comprising a memory and a processor, wherein the memory stores program instructions that are executed by the processor, and the processor invokes the program instructions to perform the method described thereon.

[0015] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described herein.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: First, the financial indicator knowledge graph explicitly models the computational dependencies and candidate formulas between indicators, providing clear knowledge guidance and strict structured constraints for the reasoning process of the large language model, thereby improving the accuracy of multi-hop computation of the model. Second, unstructured financial problems are analyzed into structured execution plans that include reasoning types, aggregation functions, and sets of sub-objectives. The reasoning process unfolds along the dependencies of the knowledge graph. The evidence binding, formula application, and intermediate calculation results of each derivation path can be fully recorded and traced back, which significantly improves the transparency and interpretability of financial numerical reasoning. Third, by using the parallel graph constraint reasoning method, the large language model is guided to explore multiple heterogeneous derivation paths at the same time. When all derivation paths fail to obtain valid evidence or fail to pass verification, the system clearly returns a rejection mark, clearly distinguishing between real data missingness and errors in the reasoning process, thus improving the credibility of the financial text intelligent analysis system. Fourth, a cross-path verification mechanism is introduced to independently review candidate results from multiple derivation paths and derive the final answer through a consensus voting mechanism. This eliminates unreliable paths that rely on false evidence or erroneous calculations, ensuring that the output answer has sufficient evidentiary support and semantic consistency. Compared to existing methods, the method of this invention has higher computational accuracy, stronger interpretability, and more reliable security, meeting the needs of practical financial numerical reasoning applications. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the overall implementation of the financial numerical reasoning method for indicator knowledge graph-guided big language models provided in this embodiment of the invention; Figure 2 A flowchart for constructing a financial indicator knowledge graph provided in this embodiment of the invention; Figure 3 A schematic diagram of the topology of the financial indicator knowledge graph FMKG provided in this embodiment of the invention; Figure 4 A flowchart for problem analysis provided in the embodiments of the present invention; Figure 5 The flowchart of the parallel graph constraint inference algorithm provided in the embodiment of the present invention is shown below; Figure 6 A flowchart for cross-path verification provided in an embodiment of the present invention. Detailed Implementation

[0019] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0021] According to one embodiment of the present invention, a financial numerical reasoning method guided by an indicator knowledge graph using a large language model is proposed, such as... Figure 1 As shown, it includes the following steps:

[0022] The steps for constructing a financial indicator knowledge graph are as follows: By integrating financial terminology and computational logic from open-source function libraries, financial encyclopedias, and academic resources, and through semantic clustering, large language model-assisted completion, and expert verification, a financial indicator knowledge graph (FMKG) is constructed to provide strict semantic and structural constraints for subsequent reasoning processes.

[0023] Problem parsing steps: Using the problem parsing operator, unstructured financial queries and document contexts are parsed into structured execution plans, including inference type, aggregation function, set of sub-objectives to be computed, and accuracy requirements, providing structured input for downstream executable inference;

[0024] Parallel graph-constrained inference steps: The parallel graph-constrained inference algorithm organizes evidence-searching operators, indicator decomposition operators, and program execution operators. Simultaneously, it explores heterogeneous derivation paths within the financial indicator knowledge graph. For each sub-objective, evidence searching and indicator decomposition are recursively performed. Finally, the program execution operator generates executable code to complete accurate calculations. A secure rejection is triggered when all derivation paths fail to obtain valid evidence.

[0025] Cross-path verification steps: Verify the candidate results generated by multiple derivation paths in three dimensions: evidence basis, constraint consistency, and calculation correctness. The final answer is obtained from the verified candidate results through a consensus voting mechanism. If no candidate result passes the verification, a rejection flag is output to distinguish between true data missingness and reasoning failure.

[0026] As one implementation method, the steps for constructing the financial indicator knowledge graph include: acquiring financial terms and computational logic from existing function libraries, authoritative financial encyclopedias, and unstructured academic resources; performing semantic clustering based on embedding and semantic verification assisted by a large language model on the original collected terminology set to eliminate terminology redundancy and ambiguity; merging the verified term clusters into the same node and automatically aggregating related attributes; and ensuring the accuracy and completeness of the graph through a verification process involving both manual verification and a large language model, ultimately obtaining the financial indicator knowledge graph FMKG.

[0027] Furthermore, the financial indicator knowledge graph FMKG is formalized as a directed acyclic graph:

[0028] G = (V, E, Φ, A)

[0029] Where V represents the set of financial variable nodes, E represents the set of dependency edges between nodes, Φ is the mapping from derived nodes to the set of candidate formulas, and A is the mapping from nodes to the set of language aliases, denoted as A: V → , Let S represent the power set of set S (i.e., the set of all subsets of S), where S represents the universal set of all possible language aliases. The node set V is divided into two disjoint subsets, one of which is the set of atomic nodes V0. atomic The first represents the indivisible basic financial concept, which serves as the termination node in the dependency graph; the second is the set of derived nodes V. derived , which represents a composite financial indicator defined by one or more calculation formulas.

[0030] For dependent edges and formulas, the directed edge (d, a) E represents the derived node d. V derived The computation depends on node a, where a can be a derived node (a V derived ), or it can be an atomic node (a V atomic To accommodate the characteristic that the same indicator can be calculated in different ways in the financial field, each derived node d V derived Associate a set of candidate formulas:

[0031]

[0032] in, Indicates derived nodes The total number of candidate formulas available. This represents the deterministic computation function corresponding to the k-th candidate formula. This represents the subset of child nodes required for the k-th candidate formula. A derived node d is considered resolved if and only if it has at least one candidate formula. It can be successfully computed, that is... Each child node has been parsed or numerical evidence can be obtained.

[0033] For alias mapping, each node v maps to a set of language aliases:

[0034]

[0035] in, , The different representations of node v in unstructured financial text are represented by aliases. Alias ​​mapping serves as an alignment bridge between symbol nodes and unstructured text fragments, resolving semantic ambiguity in financial terms.

[0036] Specifically, the steps for constructing the financial indicator knowledge graph are as follows:

[0037] Step 1.1 Obtain the core calculation patterns from the existing financial reasoning dataset function library, establish a basic financial terminology classification system, and proceed to Step 1.2;

[0038] Step 1.2 Crawl and parse entries from authoritative online financial encyclopedias to obtain standardized definitions, aliases, and calculation formulas for financial terms, thereby expanding the scope of terminology coverage. Proceed to Step 1.3.

[0039] Step 1.3 Extract long-tail domain knowledge from unstructured documents, including entities and calculation rules from financial textbooks and professional exam materials, and supplement professional indicators not covered by encyclopedia sources. Proceed to Step 1.4.

[0040] Step 1.4 Perform embedding-based semantic clustering on the original term set collected in Steps 1.1 to 1.3 to group synonymous terms. Then, use a large language model to perform semantic verification on each cluster, strictly determining whether terms within the same category refer to the same financial concept. If the determination is inconsistent, the terms are submitted to manual verification, and proceed to Step 1.5; if the determination is consistent, the cluster is retained, and proceed to Step 1.6.

[0041] Step 1.5 For financial concepts that are inconsistent in the large model validation, the financial concepts in the same category are manually divided into different terminology categories, and then proceed to Step 1.6;

[0042] Step 1.6 Merge the validated semantically similar terms into canonical nodes. For each canonical node, the large language model merges and completes its associated aliases, calculation formulas, and Python code snippets to form a unified standardized representation. Derived indicator nodes contain four attributes: alias set, definition text, candidate formula set, and code implementation. Atomic indicator nodes contain only two attributes: alias set and definition text. Proceed to Step 1.7.

[0043] Step 1.7 Execute a hybrid verification process combining human and large language model collaboration. First, use the large language model to verify the extracted fields. If the verification passes, directly store them in the financial indicator knowledge graph; otherwise, domain experts correct the errors before storing them in the financial indicator knowledge graph, then proceed to step 1.8.

[0044] Step 1.8 employs a depth-first search strategy based on node access states to traverse the nodes in the graph. When an edge pointing to a node on the current search path is encountered during traversal, a cycle is identified, and a continuous sequence of nodes from the target node to the current node is immediately extracted from the current search path as a specific cycle. The output cycle information is analyzed by domain experts, who remove cycles generated by invertible formulas, retaining only unidirectional dependencies. The final result is the Financial Indicator Knowledge Graph (FMKG).

[0045] It's important to note that FMKG only encodes the dependency direction of the specification, i.e., the unidirectional dependency hierarchy from basic variables to higher-order derived indicators. The inverse of the formula is processed during the search and execution phases rather than stored as additional graph edges. This ensures the graph maintains a directed acyclic graph structure, avoiding the introduction of directed cycles due to reversible formula relationships, which could lead to topological sorting failures and ambiguities in recursive decomposition. The financial knowledge graph construction process is as follows: Figure 2 As shown in the diagram, the topology of the Financial Indicator Knowledge Graph (FMKG) is illustrated below. Figure 3 As shown.

[0046] As one implementation, the problem parsing step includes: receiving a financial document D and a query Q, and using a problem parsing operator. The query is mapped to one of five predefined inference types, and the aggregate function, operand sub-objectives, their constraints and accuracy requirements are parsed to generate a structured execution plan.

[0047] Furthermore, the problem parsing operator The formal definition is as follows:

[0048]

[0049] Where D represents financial documents, Q represents user queries, and T represents the identified reasoning type. This represents the identified aggregation function, where N represents the number of operand sub-targets. This represents the i-th operand node. Let P represent the set of constraints for the i-th operand node, and let P represent the precision requirement of the final answer.

[0050] Constraints To disambiguate, specify information in the following three dimensions: first, the time range, referring to the fiscal year, quarter, or specific timestamp; second, the entity range, referring to the target company entity or subsidiary; and third, the meta-attributes, referring to the required currency, unit of measurement, and scale. Constraints should be populated primarily using evidence from the document context, including company name, date, currency symbol, etc. If some information is unavailable in both the query and the document, it should be marked as unknown rather than speculative.

[0051] Furthermore, the five types of reasoning and their corresponding symbolic semantic definitions are as follows:

[0052] The first method is direct retrieval, which retrieves information directly from the context and strictly satisfies the constraints. For the metric instance V, the final aggregation logic is as follows:

[0053]

[0054] Result represents the final aggregate result under the current inference type. No arithmetic operations are allowed under this type.

[0055] The second type is arithmetic calculation, which extracts the set of explicit operands defined by the arithmetic relations in the query. , The final aggregation logic is as follows:

[0056]

[0057] in, It belongs to the set of addition, subtraction, multiplication, and division operations: {+, -, ×, ÷}.

[0058] The third type is comparative reasoning, which extracts different operational values ​​through dimensional transformation. and Dimensional transformation includes time dimension transformation (comparison of time t with time t-1) and entity dimension transformation (entity... With entity (Comparison). When calculating the ratio, the final aggregation logic is:

[0059]

[0060] When calculating the difference, the final aggregation logic is as follows:

[0061]

[0062] When performing a direct comparison, the final aggregation logic is as follows:

[0063]

[0064] in, This is a comparison function that returns the relative size of two operands. This represents the numerical value of the operation within the target dimension. This represents the operational value under the baseline dimension.

[0065] The fourth type is statistical aggregation, which retrieves a set of values ​​𝒱 in batches along a specified dimension. The final aggregation logic is as follows:

[0066]

[0067] Where 𝒱 represents the set of values ​​retrieved across a specified dimension, and Mean, Sum, and Count represent the mean, summation, and count aggregation operations, respectively.

[0068] The fifth type is temporal reasoning, which defines time boundaries. and To capture the financial scope. When the query explicitly requests a time span, the final aggregation logic is:

[0069]

[0070] When time normalization is required, the final aggregation logic is as follows:

[0071]

[0072] in, Indicates the starting point of time. The endpoint represents the time, and Δ represents the time difference function. ΔT represents the value to be normalized, Scale represents the target dimension, and ΔT represents the time interval.

[0073] Specifically, the problem-solving steps are as follows:

[0074] Step 2.1 Receive financial document D and query Q, construct question parsing prompts, concatenate document D and query Q into the prompt template, and call the large language model to perform question parsing. The question parsing prompts require the model to use both the query and document context to determine what needs to be computed, extract only the required operand metrics as sub-objectives without computing the final answer, and utilize evidence from the document context to fill in the constraints as much as possible. Proceed to Step 2.2;

[0075] Step 2.2 Based on the output of the large language model, parse the inference type T, and proceed to step 2.3;

[0076] Step 2.3 Determine the aggregation function based on the inference type T It selects the corresponding function from the predefined set of aggregation functions {sum, difference, product, division, ratio, change ratio, range, compare, average, time, counting}, where sum represents summing over the set of operands; difference represents finding the difference between two operands; product represents finding the product of multiple operands; division represents finding the quotient of two operands; and ratio is used to aggregate the relative values ​​of entity dimensions in comparison reasoning, corresponding to the formula ( - ) / The change ratio is used to aggregate relative changes in the time dimension in comparative inference. It is mathematically consistent with the ratio, but semantically specifically refers to the rate of change across the time dimension, reflecting the dimensional differences of the operand sub-objectives. In this invention, the range represents the aggregation of differences in comparative inference, i.e. - This differs from the concept of range in statistics; `compare` directly compares two operands and returns the comparison result; `average` is equivalent to aggregating the mean, and the corresponding formula is... ;time represents the time difference or time normalization aggregation under time inference, corresponding to the formula Δ( , ) or × Scale / ΔT; counting is equivalent to calculating the aggregation of counts, corresponding to the formula Proceed to step 2.4;

[0077] Step 2.4 Extract the operand sub-target set Each sub-objective includes only the numerical indicators that need to be retrieved or subsequently calculated, along with their constraints. Constraints Information includes three dimensions: time range, entity range, and meta-attributes. If the corresponding information is not described in the context, the constraint is unknown, and proceed to step 2.5.

[0078] Step 2.5 Extract the precision requirement P and determine the output format of the final answer. After the problem analysis step is completed, output the structured execution plan (…). , , , This information is passed to subsequent parallel graph constraint inference steps. The problem resolution flowchart is as follows: Figure 4 As shown.

[0079] As one implementation method, the parallel graph-constrained reasoning step includes: performing recursive parsing in parallel on each sub-objective output by the problem parsing step, utilizing evidence search operators, index decomposition operators, and program execution operators to explore heterogeneous derivation paths along the dependency structure of the financial index knowledge graph. When all derivation paths for a certain sub-objective fail to obtain valid evidence, the system returns a rejection flag ⊥, achieving secure rejection.

[0080] Furthermore, the evidence search operator Used to locate numerical evidence that satisfies constraints from the document context. Its formal definition is as follows:

[0081]

[0082] Where v represents the target node, C represents the set of constraints, D represents the financial document, A(v) represents the set of aliases for node v in the financial indicator knowledge graph, val represents the verified numerical evidence, and ⊥ represents the rejection flag. This operator leverages the semantic capabilities of a large language model, combined with the alias set in the FMKG, to search for numerical evidence in document D that satisfies constraint C. This operator strictly limits itself to performing only evidence extraction without any computation. If the context does not contain directly labeled numerical values ​​that satisfy the constraints, it returns the rejection flag ⊥ and reports the missing indicator name. The rejection flag ⊥, as an explicit state, triggers indicator decomposition for derived nodes and path termination for atomic nodes.

[0083] Furthermore, the index decomposition operator This method is used to decompose complex financial indicators into computable subproblems. Its formal definition is as follows:

[0084]

[0085] Where d represents the derived node to be decomposed (d ∈ ), where Φ represents the set of formulas in the financial indicator knowledge graph. This represents the total number of candidate formulas for the derived node d. This represents the symbolic computation formula applied to the k-th derivation path, corresponding to the candidate formula in FMKG. The representation of This represents the set of child nodes required to parse node d in the k-th path. For brevity, this invention will refer to this set in the following description. This is called the symbolic derivation path.

[0086] The indicator decomposition operator retrieves candidate paths from the financial indicator knowledge graph through bidirectional search. The primary search direction is downward traversal, collecting child nodes of the target indicator and their corresponding decomposition formulas for subsequent execution. The secondary search direction is upward exploration, limited to a single hop of parent nodes, and is only used when the formula of the parent node can be algebraically rearranged to solve the target indicator and the required variables can be obtained from the context. Regardless of whether it is a downward or upward path, the obtained formulas and evidence are passed to the program execution operator for executable code generation and final calculation.

[0087] Furthermore, the program execution operator It is used to transform the results of reasoning into deterministic computation. Its formal definition is as follows:

[0088]

[0089] in, Indicates the decomposition operator by index The symbolic computation formula on the k-th derivation path (i.e., the k-th candidate formula) is output. (specific form) represents the numerical value of each child node in the verified evidence set, and y represents the final calculation result. This operator will... The verified evidence is translated into an executable Python program and executed in a sandbox environment to obtain a deterministic computation result y. If any value in the input evidence set is a rejection flag ⊥, execution stops immediately and ⊥ is propagated upwards.

[0090] Specifically, the parallel graph constraint inference steps are as follows:

[0091] Step 3.1 Receive the inference type T and aggregate function output from the problem analysis step. Sub-target set:

[0092]

[0093] in, This represents the i-th operand node. Let N represent the set of constraints for the i-th operand node, and N represent the number of operand sub-objectives. Initialization result set. If the set is empty, proceed to step 3.2;

[0094] Step 3.2 For Each sub-target in Parallel calls to recursive parsing functions (Within the recursive function, local variables v and C represent the currently processed node and constraints), proceed to step 3.3. Obtain the parsed results of each returned sub-objective and add them to... After all sub-targets have been parsed, proceed to step 3.9;

[0095] Step 3.3 In the recursive analytic function In the middle, the evidence search operator is invoked first. Get the value of subtarget v in the context. .like A value not equal to the rejection flag ⊥ indicates that numerical evidence satisfying the constraints was successfully found in the document, and the response is returned. Proceed to step 3.2. If Equal to ⊥ and v belongs to the set of atomic nodes Since atomic nodes are fundamental, indivisible concepts, there is no further derivation path; therefore, return directly to ⊥ and proceed to step 3.2. If... Equal to ⊥ and v belongs to the set of derived nodes This indicates that although the indicator is not directly reported in the document, it may be derived from other variables. Proceed to step 3.4.

[0096] Step 3.4 Call the index decomposition operator Obtain the set of all candidate derivation paths corresponding to the indicator (Paths), and initialize the candidate result set. If the set is empty, proceed to step 3.5;

[0097] Step 3.5 For each derivation path ( , Perform the following operations in parallel: Initialize the set of child node values ​​for this path. If the set is empty, initialize the valid flags for this path. If true, proceed to step 3.6;

[0098] Step 3.6 For the current path Each child node u in the process is called in parallel recursively:

[0099]

[0100] Where u represents a child node required by the current path, and C represents the set of constraints inherited from the parent node. If the response is equal to the rejection flag (⊥), then... Set to false. Add to After all child nodes of the current path have been resolved, proceed to step 3.7;

[0101] Step 3.7 Judgment The value of . If A value of true indicates that all child nodes of the current path have been successfully resolved, and the program is invoked to execute the operator. The symbolic formula for the current path is passed in. and child node value set (in = { , , ..., }, that is, the set of values ​​of all child nodes of the k-th path, and The evidence set in the formal definition is equivalent, and the result of the current path is calculated. .Will Add to In the middle. If If the result is false, it indicates that at least one child node in the current path failed to obtain valid evidence. This path is marked as failed and no candidate results are generated. After all paths have been processed, proceed to step 3.8.

[0102] Step 3.8 Judgment Is it an empty set? If A non-empty result indicates that at least one derivation path has successfully generated a candidate result, proceeding to the cross-path verification step (step 4.1). Perform the verification and return the final parsed value. .like An empty set indicates that no complete evidence can be obtained from any of the derivation paths; return the rejection flag ⊥. Proceed to step 3.2;

[0103] Step 3.9 If all sub-targets are successfully parsed, in the result set Apply aggregate functions The final answer y is obtained; if there is ⊥ in the sub-objective, the index cannot be calculated, and a rejection is returned with a reason.

[0104] The complete flowchart of the parallel graph constraint inference algorithm is as follows: Figure 5 As shown.

[0105] As one implementation method, the cross-path verification step includes: verifying the set of candidate results generated in the parallel graph constraint reasoning step, with a large language model acting as the judge, reviewing each candidate path one by one from three dimensions: evidentiary basis, constraint consistency, and computational correctness, and outputting a decision of validity or invalidity. The final answer is obtained through consensus voting among the verified candidate paths. If no candidate path passes verification, the system triggers a safe rejection.

[0106] Furthermore, the verification checklist in the cross-path verification includes the following three dimensions:

[0107] The first dimension is the verification of the basis of evidence, which requires that every operand used in the computation must be traceable to a piece of evidence in the path, and that the evidence must match the constraints specified by the query in terms of time span, entity scope, and meta-attributes. If the path uses alias mapping, the mapping must be explicitly supported by evidence or metadata in the path, rather than based on assumptions.

[0108] The second dimension is constraint consistency verification, which requires that the time range, entity range, and meta-attributes bound to all variables in the path satisfy the constraints specified in the query. Fields marked as unknown in the constraints are considered missing information and must not be filled by guesswork.

[0109] The third dimension is the verification of calculation correctness, which requires that the formulas and arithmetic operations used are correct, that the numerical relationship between intermediate values ​​and the final result is consistent, and that there are no arithmetic errors or missing operands.

[0110] Furthermore, the cross-path verification is adjudicated by the large language model through a single call. Its input is an instantiated inference path, including logical constraints (entities, time, and meta-attributes), variable bindings for the evidentiary basis (names, values, and evidence fragments), the applied inference path, and the computation result. The adjudicator outputs a decision of valid VALID or invalid INVALID for each candidate path, along with a brief reason. A candidate path is deemed invalid if its operands lack supporting evidence, the evidence violates specified constraints, or the computation contains arithmetic errors or missing operands.

[0111] Specifically, the cross-path verification steps are as follows:

[0112] Step 4.1 Receive the candidate result set Each candidate path includes the following information: a list of sub-targets and their constraints, evidence variable bindings and corresponding document evidence fragments, the applied calculation formula or program and intermediate values, and the final numerical result and its meta-attributes. Proceed to step 4.2;

[0113] Step 4.2 For For each candidate path, a verification prompt is constructed. The query question, document context, and complete information about the candidate path are concatenated into the verification prompt template. A large language model is then invoked as the judge to perform the verification. The judge checks each item on a verification checklist according to three dimensions: evidentiary basis, constraint consistency, and computational correctness. For each candidate path, the judge outputs a decision of whether it is a valid VALID or an invalid INVALID, along with a brief reason. Proceed to step 4.3;

[0114] Step 4.3 Statistical verification results. Remove the candidate paths determined to be invalid from the set, retain the candidate paths determined to be valid, and proceed to step 4.4;

[0115] Step 4.4 Determine if there are any verified candidate paths. If one or more verified candidate paths exist, perform a consensus vote on these paths to arrive at the final answer. If no candidate path passes verification, the system triggers a security rejection and returns a rejection flag ⊥. The cross-path verification flowchart is as follows: Figure 6 As shown.

[0116] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a large language model financial numerical reasoning system guided by an indicator knowledge graph, comprising:

[0117] The Financial Indicator Knowledge Graph Construction Module is used to comprehensively acquire financial terms and computational logic from function libraries, financial encyclopedias, and academic resources. Through semantic clustering, large language model-assisted completion, and expert verification, it constructs a financial indicator knowledge graph formalized as a directed acyclic graph structure. This graph contains atomic nodes, derived nodes, dependency edges, candidate formula sets, and alias mapping sets.

[0118] The question parsing module is used to jointly parse unstructured financial queries and document contexts into a structured execution plan that includes inference type, aggregation function, operand sub-objective set, and accuracy requirements using question parsing operators built based on a large language model.

[0119] Parallel Graph Constrained Reasoning Module: This module coordinates three atomic operators—evidence search operator, indicator decomposition operator, and program execution operator—using a parallel graph constrained reasoning algorithm. It simultaneously explores heterogeneous derivation paths along the financial indicator knowledge graph, recursively performs evidence search and indicator decomposition for each sub-objective, and uses the program execution operator to generate executable code to complete accurate calculations. When all derivation paths fail to obtain valid evidence, a secure rejection is triggered.

[0120] Cross-path verification module: It is used to verify the candidate results generated by multiple derivation paths in three dimensions: evidence basis, constraint consistency and calculation correctness. It outputs the final answer from the candidate results that pass the verification through a consensus voting mechanism. If no candidate result passes the verification, it outputs a rejection flag.

[0121] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a large language model financial numerical reasoning device guided by an indicator knowledge graph, including a memory and a processor. The memory stores program instructions that are executed by the processor, and the processor calls the program instructions to execute the aforementioned large language model financial numerical reasoning method guided by an indicator knowledge graph.

[0122] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the aforementioned indicator knowledge graph-guided large language model financial numerical reasoning method.

[0123] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] In summary, this invention discloses a financial numerical reasoning method and system for large language models guided by an indicator knowledge graph, belonging to the fields of financial technology and artificial intelligence. The method includes: constructing a financial indicator knowledge graph by integrating multi-source financial knowledge to provide semantic and structural constraints for the large language model; parsing financial queries and document context to generate a structured execution plan containing reasoning type, aggregation function, sub-objectives, and constraints; based on the financial indicator knowledge graph, employing a parallel graph-constrained reasoning mechanism to recursively perform evidence search, indicator decomposition, and program execution for each sub-objective, completing numerical calculations along multiple candidate derivation paths; performing cross-path verification on candidate results, and outputting the final answer or a rejection flag based on the verification results. This invention can effectively constrain the free generation behavior of large language models, improve the accuracy, interpretability, and security of financial numerical reasoning, and avoid unfounded numerical fabrication when evidence is insufficient. It is applicable to scenarios such as financial statement analysis, research report Q&A, and intelligent analysis of financial texts.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A financial numerical reasoning method using a large language model guided by an indicator knowledge graph, characterized in that, include: By integrating multi-source financial knowledge, a financial indicator knowledge graph is constructed, which includes atomic nodes, derived nodes, dependency edges, candidate formula sets, and alias mapping sets. Using a problem parsing operator built on a large language model, unstructured financial queries and document contexts are parsed into structured execution plans, which include inference type, aggregation function, set of sub-objectives to be computed, and accuracy requirements. A parallel graph-constrained reasoning algorithm is used to organize evidence search operators, index decomposition operators, and program execution operators. Specifically: the index decomposition operator retrieves candidate derivation paths from the knowledge graph based on the sub-objective; the evidence search operator calls a large language model along the path to extract numerical evidence; and the program execution operator converts the calculation formulas and verified evidence in the candidate derivation paths into executable code and performs the calculation. For each sub-objective, the evidence search operator, index decomposition operator, and program execution operator are recursively called. When no valid evidence can be obtained from any of the derivation paths, a safe rejection is triggered, and a rejection flag is output. For candidate results generated by different derivation paths, a large language model is used to verify the evidence basis, constraint consistency and calculation correctness, and the final answer is output through consensus voting; If no candidate result passes the validation, a rejection flag will be output.

2. The method according to claim 1, characterized in that, The construction of the financial indicator knowledge graph further includes: Financial terminology and computational logic were collected from function libraries, financial encyclopedias, and academic resources; Embedsion-based semantic clustering is performed on the collected term set, and semantic verification is performed using a large language model; Verified semantically similar terms are merged into canonical nodes, and the large language model completes the aliases, calculation formulas, and code snippets. Implement a hybrid verification process that combines human verification with large language models. A depth-first search strategy based on node access status is used to detect and eliminate loops.

3. The method according to claim 1, characterized in that, The financial indicator knowledge graph is formalized as a directed acyclic graph G = (V, E, Φ, A), where V represents the set of financial variable nodes, divided into a set of atomic nodes and a set of derived nodes, E represents the set of dependency edges between nodes, Φ is the mapping from derived nodes to a set of candidate formulas, and A is the mapping from nodes to a set of language aliases. Each derived node is associated with a set of candidate formulas; each node is mapped to a set of language aliases, used to align symbol nodes with unstructured text fragments.

4. The method according to claim 1, characterized in that, The problem analysis includes: Receive financial documents and queries, construct question parsing prompts, and call a large language model to perform question parsing; Map queries to one of five predefined reasoning types: direct retrieval, arithmetic calculation, comparative reasoning, statistical aggregation, and temporal reasoning; and parse the reasoning type. Select the corresponding aggregate function from the predefined set of aggregate functions based on the reasoning type; Extract the set of operand sub-objectives, where each sub-objective includes a numerical indicator to be retrieved or calculated and its constraints, wherein the constraints include information in three dimensions: time range, entity range, and meta-attribute. Extract the accuracy requirements, output a structured execution plan, and pass it to the subsequent parallel graph constraint inference steps.

5. The method according to claim 1, characterized in that, The parallel graph constraint reasoning includes: The recursive parsing function is called in parallel for each sub-objective output by the problem parsing step; In the recursive parsing function, the evidence search operator is first called to locate numerical evidence that satisfies the constraints in the document context; if numerical evidence is successfully found, the value is returned; if no numerical evidence is found and the target node is an atomic node, a rejection flag is returned; if no numerical evidence is found and the target node is a derived node, the index decomposition operator is called to obtain all candidate derivation paths. For each derivation path, recursive parsing of child nodes is performed in parallel; if all child nodes of the current path are successfully parsed, the program is called to execute operators to generate executable code and calculate candidate results; if at least one child node fails to obtain valid evidence, the path is marked as failed. If candidate results exist, proceed to the cross-path verification step; if no complete evidence can be obtained from any of the derivation paths, return a rejection flag. After all sub-goals are parsed, the aggregate function is applied to obtain the final answer; if there is a rejection flag in a sub-goal, the rejection is returned with a reason.

6. The method according to claim 5, characterized in that, The evidence search operator utilizes the semantic capabilities of a large language model, combined with the alias set in the financial indicator knowledge graph, to search for numerical evidence that meets the constraints in financial documents. It strictly limits the process to only extracting evidence without performing any calculations. If the context does not contain a numerical value that meets the constraints, it returns a rejection flag and reports the missing indicator name. The indicator decomposition operator retrieves candidate paths from the financial indicator knowledge graph through bidirectional search. The main search direction is to traverse downwards to collect the child nodes of the target indicator and their decomposition formulas, while the auxiliary search direction is to explore upwards, limited to the range of one parent node. The program execution operator translates the symbolic derivation path and verified evidence into an executable program and executes it in a sandbox environment to obtain deterministic calculation results. If any value in the input evidence set is a rejection flag, execution stops immediately and the rejection flag is propagated upwards.

7. The method according to claim 1, characterized in that, The cross-path verification includes: Receive a set of candidate results, in which each candidate path includes a list of sub-targets and their constraints, evidence variable bindings and corresponding document evidence fragments, the applied calculation formulas and intermediate values, and the final numerical results and their meta-attributes; For each candidate path, a verification prompt is constructed, and a large language model is called as the judge to check each item from three dimensions: evidence basis, constraint consistency and calculation correctness. The decision of whether each candidate path is valid or invalid is output. Remove candidate paths deemed invalid and retain those deemed valid; If a valid candidate path exists, a consensus vote is performed to arrive at the final answer; if no candidate path is valid, a security rejection is triggered, and a rejection flag is returned.

8. A financial numerical reasoning system based on a large language model guided by an indicator knowledge graph, characterized in that, include: Financial Indicator Knowledge Graph Construction Module: Used to construct a financial indicator knowledge graph that includes atomic nodes, derived nodes, dependency edges, candidate formula sets, and alias mapping sets; Problem parsing module: Utilizes problem parsing operators built on a large language model to parse unstructured financial queries and documents into structured execution plans, which include inference type, aggregation function, sub-objective set, and accuracy requirements; Parallel graph-constrained reasoning module: used to coordinate the evidence search operator, index decomposition operator and program execution operator. The evidence search operator calls the large language model, the index decomposition operator retrieves candidate paths, the program execution operator generates code, and explores heterogeneous inference paths in parallel to achieve secure rejection. Cross-path validation module: Used to perform multi-dimensional validation of candidate results using a large language model, and output the final answer or a rejection mark.

9. A financial numerical reasoning device using a large language model guided by an indicator knowledge graph, characterized in that, It includes a memory and a processor, the memory storing program instructions that are executed by the processor, the processor invoking the program instructions to perform the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 7.

Images