An explainable generative reordering and business semantic level gate control orchestration method and device for a financial intelligent agent system, equipment and medium

Abstract

The application provides an explainable generative reordering and business semantic level gate control arrangement method and device for a financial intelligent agent system, and relates to the technical field of artificial intelligence. The method upgrades the traditional reordering module that only outputs a score or a ranking list to an explainable generative reordering mechanism. The mechanism first outputs a scoring basis and then outputs a correlation level, so that the selection decision of each candidate unit has traceability and explainability. By quantifying the correlation into discrete business semantic levels and directly mapping them into executable gate action instructions, the evidence can be stably and controllably screened, the tools can be controllably called, and the strategies can be controllably triggered, so that the conversion from a fuzzy similarity to an explicit business action is realized. Finally, the method not only improves the accuracy of RAG evidence selection, but also reduces context interference and answer illusion.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an interpretable generative reordering and business semantic-level gating orchestration method, apparatus, device and medium for financial intelligent agent systems. Background Technology

[0002] In scenarios such as financial question answering, investment research assistance, or investment advisory services, financial intelligent agent systems (financial LLM-Agents) based on Large Language Models (LLMs) and combined with capabilities such as retrieval, tool invocation, memory, and strategies are being widely applied. Here, "Agent" refers to an intelligent agent. These systems typically employ a Retrieval Augmented Generation (RAG) architecture, where a retrieval component provides the generative model with external document context to reduce illusions and improve the traceability of answers. In the RAG process, the Rerank module is responsible for refining the candidate document set obtained in the recall phase to select the most relevant evidence for use by the LLM.

[0003] Existing reordering schemes typically suffer from the following drawbacks: 1) Lack of business interpretability: Traditional Rerank models only output continuous relevance scores or ranking lists, failing to provide the scoring basis or reasons why the document was selected. In the financial field, answers must be based on accurate subjects, time, indicators, and definitions. The lack of interpretability makes the system's decision-making process unauditable and unreplayable, making it difficult to meet the engineering requirements of financial compliance and quality inspection.

[0004] 2) Lack of business semantic operability: Continuous score thresholds are difficult to set and transfer. The system usually can only rely on Top-K or experience thresholds for evidence screening, which can easily lead to context bloat and answer illusions.

[0005] 3) Difficult to support unified orchestration of generalized candidate units: When an agent needs to make unified selections and combinations among generalized candidate units such as document fragments, tool capability descriptions, historical memories, and policy templates, traditional reordering outputs lack unified decision-making scales and executable rules, making it difficult to directly output gating and orchestration results, resulting in fragmented system policies and difficulty in locating the source of errors.

[0006] Therefore, there is an urgent need for a reordering scheme that can provide interpretability, have clear business semantics, and drive unified gating and orchestration to solve the above-mentioned technical problems. Summary of the Invention

[0007] In view of the above problems, this application is made to provide an interpretable generative reordering and business semantic-level gating orchestration method, apparatus, device, and medium for financial intelligent agent systems that overcomes or at least partially solves the above problems. The technical solution is as follows: Firstly, an interpretable generative reordering and business semantic-level gating orchestration method for financial intelligent agent systems is provided, the method comprising: Obtain a set of candidate units, wherein the types of candidate units in the set include one or more of the following: document fragments, tools, memories, and strategies; The user question and each candidate unit in the candidate unit set are input into a pre-trained generative reordering model for processing. The model generates a scoring basis and a relevance level for each candidate unit. The scoring basis is generated before the relevance level and is used to explain the reasons for assigning a relevance level. The output includes a structured result that includes the scoring basis and relevance level for each candidate unit. The structured output results are subjected to consistency verification. Consistency verification is used to determine whether there is a logical conflict between the scoring criteria and the relevance level. After the consistency check passes, the gating action instruction corresponding to the relevance level is determined according to the preset mapping rules between business semantics and gating actions. The gating action instruction is used to indicate the processing method of the corresponding candidate unit in context assembly and task orchestration. Based on gating action instructions, candidate units in the candidate unit set are assembled into context and orchestrated into tasks to generate orchestration results that drive the financial intelligent agent system to perform tasks.

[0008] In one possible implementation, the candidate unit set is constructed through the following steps: Acquire multi-source heterogeneous data related to user issues, including document library data, tool capability description data, historical interaction memory data, and strategy template data; Multi-source heterogeneous data is formatted in a unified manner to generate multiple candidate units with a unified data structure. The unified data structure includes at least a candidate unit type field, a candidate unit content field, and a metadata field.

[0009] In one possible implementation, the user question and each candidate unit in the candidate unit set are input into a pre-trained generative reordering model for processing, generating a scoring criterion and relevance level for each candidate unit, including: The user question and each candidate unit in the candidate unit set are input into a pre-trained generative rearrangement model for processing. The generative rearrangement model analyzes the matching between the user question and each candidate unit and generates a scoring criterion for each candidate unit. The scoring criterion includes matching point information, missing element information, and risk information. Based on the scoring criteria for each candidate unit, a relevance level is generated for each candidate unit.

[0010] In one possible implementation, the relevance level is a level value selected from a set of preset discrete levels, each of which has a predefined corresponding business semantics.

[0011] In one possible implementation, the multi-level discrete grade set is a five-point grade set, and the grade values ​​in the five-point grade set include 0 points, 1 point, 2 points, 3 points, and 4 points. The pre-defined mapping rules between business semantics and gating actions include: When the relevance level is 4, it is mapped to a gating action instruction that indicates the corresponding candidate unit as core evidence to enter the context and trigger a direct response action; When the relevance level is 3, it is mapped to a gating action instruction that indicates that the corresponding candidate unit is used as the main basis to enter the context and triggers the attached policy constraint action. The attached policy constraint action includes at least one of risk warning, caliber verification, supplementary evidence retrieval or tool verification. When the relevance level is 2, it is mapped to a gating action instruction that indicates that the corresponding candidate unit will be included in the context as supplementary information given that there is already 4 or 3 points of evidence. When the relevance level is 1, it is mapped to a gating action instruction that indicates the corresponding candidate unit to be placed in the backtracking pool; When the relevance level is 0, it is mapped to a gating action instruction that indicates the removal of the corresponding candidate unit.

[0012] In one possible implementation, consistency checks are performed on the structured output, including: The format of the structured results is validated to determine whether the relevance level is a valid value in the preset set of multiple discrete levels; Semantic recognition is performed on the scoring criteria in the structured results. When semantic features that represent missing key information, inability to locate evidence, inconsistent standards, or mismatch with the question are identified, it is determined whether the corresponding relevance level is higher than a preset threshold. If the corresponding relevance level is higher than the preset threshold, it is determined that there is a logical conflict, and the corresponding relevance level is downgraded or a retry mechanism is triggered.

[0013] In one possible implementation, after generating the orchestration result, the method further includes: Generate audit log records. The audit log records are used to store the complete link information of this task execution. The complete link information includes user questions, candidate unit set, structured results corresponding to each candidate unit, gating action instructions corresponding to each candidate unit, tool call process and results, and final answer content.

[0014] Secondly, an interpretable generative reordering and business semantic-level gating orchestration apparatus for financial intelligent agent systems is provided, the apparatus comprising: The acquisition module is used to acquire a set of candidate units, wherein the types of candidate units in the candidate unit set include one or more of the following: document fragments, tools, memories, and strategies; The generative re-ranking module is used to input the user question and each candidate unit in the candidate unit set into the pre-trained generative re-ranking model for processing, and to generate a scoring basis and relevance level for each candidate unit. The scoring basis is generated before the relevance level, and the scoring basis is used to explain the reasons for assigning the relevance level. The output includes a structured result including the scoring basis and relevance level for each candidate unit. The verification module is used to perform consistency verification on the output structured results. The consistency verification is used to determine whether there is a logical conflict between the scoring criteria and the relevance level. The gating mapping module is used to determine the gating action instruction corresponding to the relevance level according to the preset mapping rules between business semantics and gating actions after the consistency verification is passed. The gating action instruction is used to indicate the processing method of the corresponding candidate unit in context assembly and task orchestration. The orchestration and execution module is used to perform context assembly and task orchestration on candidate units in the candidate unit set based on gating action instructions, so as to generate orchestration results for driving the financial intelligent agent system to execute tasks.

[0015] Thirdly, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the interpretable generative reordering and business semantic-level gating orchestration method for financial intelligent agent systems as described in any of the preceding claims.

[0016] Fourthly, a storage medium is provided that stores a computer program, wherein the computer program is configured to execute, at runtime, the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems as described above.

[0017] By employing the aforementioned technical solutions, this application provides an interpretable generative reordering and business semantic-level gating orchestration method, apparatus, device, and medium for financial intelligent agent systems. This method upgrades traditional reordering modules that only output scores or ranking lists to an interpretation-first generative reordering mechanism. This mechanism first outputs the scoring criteria, then the relevance level, ensuring that the selection decision for each candidate unit is traceable and interpretable. By quantifying relevance into discrete business semantic levels and directly mapping them to executable gating action instructions, it can stably and controllably screen evidence, invoke tools, and trigger strategies, thereby achieving the transformation from fuzzy similarity to explicit business actions. Ultimately, this method not only improves the accuracy of RAG evidence selection and reduces contextual interference and answer illusions, but also achieves unified selection and orchestration of generalized candidate units such as documents, tools, memories, and strategies, meeting the stringent requirements of auditability, replayability, and high controllability in financial scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0019] Figure 1 The flowchart illustrates the interpretable generative reordering and business semantic-level gating orchestration method for financial intelligent agent systems provided in this application embodiment. Figure 2 This paper shows a structural diagram of an interpretable generative reordering and business semantic-level gating orchestration device for financial intelligent agent systems provided in an embodiment of this application. Figure 3 This invention provides a structural diagram of an interpretable generative reordering and business semantic level gating orchestration apparatus for financial intelligent agent systems, according to another embodiment of this application. Figure 4 A structural diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0020] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0022] Before describing the embodiments of this application, the following technical terms will be explained: 1) Financial LLM-Agent: In scenarios such as financial Q&A, investment research assistance, or investment advisory services, it is an intelligent agent system that completes information acquisition, reasoning, generation, and execution based on a large language model and combined with retrieval, tool calling, memory, and strategy capabilities.

[0023] Large language models here refer to generative models with a large parameter scale and strong general language understanding and generation capabilities. Typically, the network parameters of large language models can reach tens of billions, hundreds of billions, or even higher. Their generality is reflected in their ability to handle multiple natural language tasks that are not limited to a single specialized problem or a single domain. Their emergent nature is reflected in the fact that, as the model size, data scale, or training method changes, the model may exhibit reasoning, induction, instruction understanding, or complex task processing capabilities that are not obvious in smaller models.

[0024] The parameter scale of the large language model used in this application can be flexibly configured according to actual application needs, computing resource conditions, and performance requirements. For example, models of different magnitudes such as 671 billion, 14 billion, 32 billion, 7 billion, or 1.5 billion can be used. This application does not limit the specific model structure, parameter scale, or model source of the large language model.

[0025] 2) RAG: Retrieval Enhanced Generation, which provides external document context to the generative model through retrieval components to reduce illusions and improve traceability.

[0026] 3) Rerank: A module that scores and ranks the relevance of candidates to the problem on the recalled candidate set.

[0027] 4) LLM-Rerank: Generative interpretable reranking uses a large language model as the reranking model to infer and generate "question-candidate units" and output structured results, which at least include the scoring basis (reason) and relevance level (score).

[0028] 5) Candidate Units / Generalized Documents: Objects selected by the financial LLM-Agent. In addition to document fragments, they may also include candidate units such as tools, memories, and strategies, and participate in rearrangement and gating in a unified format.

[0029] 6) Relevance Level: The level value selected from a pre-defined set of multiple discrete levels. Each level in the set has a predefined business semantics to drive subsequent gating and orchestration actions. For example, the set of multiple discrete levels is a five-point level set, and the level values ​​in the five-point level set include 0 points, 1 point, 2 points, 3 points and 4 points.

[0030] 7) Gating: Based on relevance level and scoring criteria, determine whether candidate units enter the context, whether they can be answered independently, whether risk warnings or supplementary evidence are triggered, whether only supplementary evidence is required, and whether they are eliminated.

[0031] 8) Orchestration: Making decisions on the selection, combination, and execution order among multiple candidate units, such as document evidence selection, tool invocation priority, strategy triggering, and memory injection.

[0032] 9) Audit / Replay: Records the link information of candidates, scoring basis, level, gating decision, tool calls and final output, which is used for compliance review, quality inspection spot checks and online problem location.

[0033] To address the aforementioned technical problems, embodiments of this application provide an interpretable generative reordering and business semantic-level gating orchestration method for financial intelligent agent systems, such as... Figure 1 As shown, the interpretable generative reordering and business semantic-level gating orchestration method for financial intelligent agent systems may include the following steps S101 to S105: Step S101: Obtain a candidate unit set, wherein the candidate units in the candidate unit set include one or more of the following types: document fragments, tools, memories, and strategies; Step S102: Input the user question and each candidate unit in the candidate unit set into the pre-trained generative rearrangement model for processing, generate a scoring basis and relevance level for each candidate unit, the scoring basis is generated before the relevance level, and the scoring basis is used to explain the reasons for assigning the relevance level; the output includes a structured result including the scoring basis and relevance level for each candidate unit. Step S103: Perform consistency verification on the output structured results. Consistency verification is used to determine whether there is a logical conflict between the scoring criteria and the relevance level. Step S104: After the consistency check passes, determine the gate action instruction corresponding to the relevance level according to the preset mapping rules between business semantics and gate actions. The gate action instruction is used to indicate the processing method of the corresponding candidate unit in context assembly and task orchestration. Step S105: Based on the gating action instructions, perform context assembly and task orchestration on the candidate units in the candidate unit set to generate an orchestration result for driving the financial intelligent agent system to perform tasks.

[0034] This embodiment upgrades the traditional reordering module, which only outputs scores or ranking lists, to an interpretation-first generative reordering mechanism. This mechanism first outputs the scoring criteria, then the relevance level, making the selection decision of each candidate unit traceable and interpretable. By quantifying relevance into discrete business semantic levels and directly mapping them to executable gating action instructions, it can stably and controllably screen evidence, invoke tools, and trigger strategies, thereby realizing the transformation from fuzzy similarity to explicit business actions. Ultimately, this method not only improves the accuracy of RAG evidence selection and reduces contextual interference and answer illusions, but also achieves unified selection and arrangement of generalized candidate units such as documents, tools, memories, and strategies, meeting the stringent requirements of auditability, replayability, and high controllability in financial scenarios.

[0035] This application provides a possible implementation method in which the candidate unit set is constructed through the following steps A1 and A2: Step A1: Obtain multi-source heterogeneous data related to user issues. The multi-source heterogeneous data includes document library data, tool capability description data, historical interaction memory data, and strategy template data. Step A2 involves performing unified formatting on the multi-source heterogeneous data to generate multiple candidate units with a unified data structure. The unified data structure includes at least a candidate unit type field, a candidate unit content field, and a metadata field.

[0036] In one specific implementation, the process begins by acquiring multi-source heterogeneous data related to the financial question raised by the user. For example, a user's question might be, "What was the net profit attributable to the parent company of a certain company in 2024?" This multi-source heterogeneous data includes not only document fragments from research reports, announcements, and financial statements, but also tool capability descriptions from tool libraries (e.g., functional descriptions of individual stock analysis tools), historical interaction memory data (e.g., confirmed company names and user preferences), and strategy template data (e.g., risk warning templates and caliber verification rules). Subsequently, this multi-source heterogeneous data undergoes a unified formatting process to generate a candidate unit with a unified data structure for each candidate option. This unified data structure includes at least a candidate unit type field (identifying whether it is a document, tool, memory, or strategy), a candidate unit content field (the specific text content or tool description), and metadata fields (such as source, date, company, etc.).

[0037] This embodiment performs unified formatting on heterogeneous data from multiple sources, including document libraries, tool libraries, memory libraries, and policy libraries, transforming it into a set of candidate units with a unified data structure. This preprocessing step enables downstream generative rearrangement models and gating mapping modules to process different types of data sources in a consistent manner, laying the data foundation for subsequent unified selection, combination, and orchestration decisions, and solving the policy fragmentation problem caused by different data types in traditional solutions.

[0038] This application provides a possible implementation method. In step S102, the user question and each candidate unit in the candidate unit set are input into a pre-trained generative rearrangement model for processing to generate a scoring basis and relevance level for each candidate unit. The specific construction steps are shown in B1 and B2 below: Step B1: Input the user question and each candidate unit in the candidate unit set into the pre-trained generative rearrangement model for processing. Analyze the matching between the user question and each candidate unit through the generative rearrangement model to generate scoring criteria for each candidate unit. The scoring criteria include matching point information, missing element information, and risk information. Step B2: Based on the scoring criteria for each candidate unit, generate the relevance level for each candidate unit.

[0039] In one specific implementation, a pre-trained generative reranking model (LLM-Rerank) is invoked, pairing the user's question with each candidate unit as input. The model first analyzes the matching between the question and the candidate units and generates a scoring criterion (reason) in natural language. This criterion clearly indicates where the matching point is, which key elements (such as company, year, indicator, and scope) are complete or missing, and whether there are risk information such as inconsistencies in scope. Then, based on the generated scoring criterion, the model derives and generates a relevance level selected from a pre-defined set of multiple discrete levels. Finally, the model outputs the scoring criterion and relevance level in a structured format (e.g., JSON).

[0040] This embodiment specifies that the reasoning process of the generative rearrangement model is interpretation-first; the model first analyzes and generates a scoring basis containing matching points, missing elements and risk information, and then derives the relevance level based on the basis; this mechanism forces the model to perform logical reasoning, rather than directly outputting an uninterpretable black box score, thereby ensuring that every step of the rearrangement decision is based on evidence, providing key information for subsequent auditing, review and problem localization.

[0041] This application provides a possible implementation method in which the relevance level is a level value selected from a set of preset discrete levels, and each level in the set of discrete levels has a predefined corresponding business semantics.

[0042] This embodiment discretizes continuous relevance scores into multiple levels and assigns clear business semantics to each level, transforming fuzzy mathematical similarity into clear and actionable business judgments. This enables system developers and business personnel to configure and optimize system behavior based on a shared semantic understanding, significantly improving the portability of gating rules and the maintainability of the system.

[0043] This application provides a possible implementation method in which the multi-level discrete grade set is a five-point grade set, and the grade values ​​in the five-point grade set include 0 points, 1 point, 2 points, 3 points, and 4 points; The pre-defined mapping rules between business semantics and gating actions include: When the relevance level is 4, it is mapped to a gating action instruction that indicates the corresponding candidate unit as core evidence to enter the context and trigger a direct response action; When the relevance level is 3, it is mapped to a gating action instruction that indicates that the corresponding candidate unit is used as the main basis to enter the context and triggers the attached policy constraint action. The attached policy constraint action includes at least one of risk warning, caliber verification, supplementary evidence retrieval or tool verification. When the relevance level is 2, it is mapped to a gating action instruction that indicates that the corresponding candidate unit will be included in the context as supplementary information given that there is already 4 or 3 points of evidence. When the relevance level is 1, it is mapped to a gating action instruction that indicates the corresponding candidate unit to be placed in the backtracking pool; When the relevance level is 0, it is mapped to a gating action instruction that indicates the removal of the corresponding candidate unit.

[0044] This embodiment defines specific five-point system business semantics and their precise mapping rules to gating actions. For example, a score of 4 allows for a direct response, a score of 3 allows for a response but requires policy constraints, a score of 2 is only supplementary, a score of 1 enters the backtracking pool, and a score of 0 is directly eliminated. This explicit mapping relationship directly transforms the reordered output into instructions that can be executed by downstream orchestration modules, solving the problems of context bloat and unstable responses caused by threshold ambiguity in traditional solutions. Furthermore, by distinguishing between direct responses and responses requiring constraints, it effectively improves the rigor and security of the system's responses.

[0045] This application provides a possible implementation method in which the consistency verification of the output structured result in step S103 may specifically include the following steps C1 to C3: Step C1: Validate the format of the structured results to determine whether the relevance level is a valid value in the preset set of multiple discrete levels. Step C2: Semantic recognition is performed on the scoring criteria in the structured results. When semantic features that represent missing key information, inability to locate evidence, inconsistent standards, or mismatch with the question are identified, it is determined whether the corresponding relevance level is higher than the preset threshold. In step C3, if the corresponding relevance level is higher than the preset threshold, it is determined that there is a logical conflict, and the corresponding relevance level is downgraded or a retry mechanism is triggered.

[0046] This embodiment introduces an interpretation-level consistency check step to perform a logical consistency check on the output of the generative rearrangement model. When the scoring criteria clearly indicate missing or mismatched information, but the model incorrectly assigns a high level, this step will automatically perform a downgrade or trigger a retry. This mechanism effectively filters out possible logical errors or illusions in the model, ensuring the reliability of gating instructions and the stability of the overall system output.

[0047] This application embodiment provides a possible implementation method. In step S105, based on the gating action instruction, the candidate units in the candidate unit set are performed on context assembly and task orchestration. The specific steps are shown in D1 to D3: Step D1: Based on the gating action command, determine whether each candidate unit enters the context and the priority of entering the context; Step D2: When context capacity is limited, priority is given to retaining candidate units with higher relevance levels and more sufficient scoring basis; Step D3: Based on the tool call instructions contained in the arrangement results, prioritize calling the tool candidate units corresponding to the elements to be verified in the user's question.

[0048] This embodiment further improves the decision-making efficiency and response quality of the financial intelligent agent system under limited context capacity by concretizing the gating action instructions into executable rules in the context assembly and task orchestration process.

[0049] This application embodiment provides a possible implementation method. After generating the arrangement result in step S105, the method further includes: Generate audit log records. The audit log records are used to store the complete link information of this task execution. The complete link information includes user questions, candidate unit set, structured results corresponding to each candidate unit, gating action instructions corresponding to each candidate unit, tool call process and results, and final answer content.

[0050] This embodiment provides the financial intelligent agent system with powerful auditable and replayable capabilities by generating and storing audit logs containing the complete decision-making chain. When an answer error occurs or a compliance check is required, operation and maintenance personnel and quality inspection personnel can completely trace back every step of the process from question input, candidate recall, reordering basis, gating decision to final answer generation. This is of vital value for meeting the strict compliance requirements of the financial industry, locating online problems, and continuously optimizing system strategies.

[0051] This application provides a possible implementation method, which may also include online policy adaptation and replay-driven parameter tuning steps. Specifically, the gating policy can be iterated in a versioned manner based on online feedback (such as manual sampling results, user error correction behavior, and error sample replay). For example, the triggering conditions for "3-point independent answer" and risk warning templates can be adjusted, thereby continuously improving the system stability without changing the overall framework.

[0052] The above introduces Figure 1 The embodiments shown have various implementation methods for each stage. The following will further explain the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems according to specific embodiments.

[0053] In a specific embodiment, steps S1–S6 constitute the basic process, while S7 is an optional enhancement (online call-up and playback).

[0054] S1, Candidate Unit Construction and Unified Representation (Preprocessing).

[0055] Input: The financial question posed by the user (e.g., "What was the net profit attributable to the parent company of Company X in 2024?"), and available data sources (such as research reports, announcements, financial statements, news documents, etc.); In a broader sense, tool libraries, memory libraries, and strategy libraries can also be considered a unified reference tool database. For example, a stock analysis tool's function description might be: "A stock analysis tool provides an analysis report on the overall situation of a stock through common dimensions such as fundamentals, technicals, and capital flows, helping clients to comprehensively analyze and understand target stocks." This function description can serve as a reference document for the tool. If the user's needs closely match the function descriptions of one or more tools, it indicates that the tool is helpful in solving the user's current problem.

[0056] Processing: Generate a "candidate cell list". Candidate cells include not only document fragments (chunks), but also: ① Tool candidates (such as the capability description and calling parameters of financial indicator query, valuation calculation, and announcement retrieval interfaces); ② Memory candidates (such as confirmed companies, definitions, preferences, and last query results); ③ Strategy candidates (such as risk warning templates, definition verification rules, and citation rules).

[0057] Output: A set of candidate units in a uniform format. Each candidate unit includes a type (document, tool, memory, strategy), content (text or tool capability description), and necessary metadata (source, date, company, chapter, etc.) to facilitate subsequent unified rearrangement and gating.

[0058] S2, Explains the generative rearrangement reasoning with an explanation in mind (explainable output).

[0059] For each candidate unit, a generative reranking model (LLM-Rerank) is invoked for evaluation.

[0060] The output adopts an "explanation-first" approach: first, the scoring criteria (reasons) are generated, and then the relevance level (0 to 4) is generated. The scoring criteria need to explain: where the candidate unit matches the question, whether the key elements (company, year, indicator, scope) are complete, and whether there are risks such as inconsistencies in scope or insufficient information.

[0061] The output format is recommended to be structured text (e.g., JSON) to facilitate subsequent validation, gating execution, and audit playback.

[0062] Example of output: For example, the following document has a relevance score of 2. The user's question is "What is the product code for the Northbound Capital Strategy?". This document introduces a lot of information about the Northbound Capital Strategy, but does not provide the code. Therefore, it scores 2 points, which corresponds to a 2-point business definition.

[0063] 2 points: Cannot support an answer, but can supplement / support. This candidate unit lacks key elements or only provides background information, and is generally not allowed to be answered alone; it is only included as supplement to the context when 3 or 4 points of evidence have already been selected.

[0064] {"query":"What is the product code for the Northbound Capital strategy?"} "doc":"Northbound Capital Strategy is a product that..." "score":2.0, "reason": "The document mentions the northbound capital strategy, but does not explicitly provide the product code." } Below is a reference document with a relevance score of 4, which means it can completely solve the problem.

[0065] {"query":"What are the nine premium benefits of Dianjintou?",} "doc":"Subscribers to Dianjintou can enjoy services from nine specific products, including Panzhongbao, Magic Radio Wave..." "score":4.0, "reason": "The document directly answers the nine premium benefits of Dianjintou, including products such as Intraday Treasure, Magic Wave, Magic Nine Turns, Main Force Code, Closing Price Digging, Northbound Capital Strategy, Advanced Stock Diagnosis, Safe Investment, and Margin Trading." } S3, Structural Verification and Interpretation - Level Consistency Verification (Degradation Core).

[0066] Perform two types of checks on the rearranged output to ensure that the results are executable and reproducible.

[0067] (1) Structure validation: whether the output format is correct, whether the fields are complete, and whether the relevance level is a valid value from 0 to 4.

[0068] (2) Explanation-level consistency verification: If the reason clearly points out that "key information is missing, evidence cannot be located, the standards are inconsistent, or the problem is not matched", then it is not allowed to give too high a level; if necessary, it will be automatically downgraded and marked with action tags such as "need to supplement evidence / need risk warning / need tool verification".

[0069] If the verification fails, one or more retries can be triggered; if it still fails, it can be downgraded to a rule-based strategy (such as keyword or element matching, similarity threshold) and an alarm can be recorded to ensure stable operation.

[0070] The above steps are also completed using a large model and prompt words.

[0071] S4, Business semantic definition and gating mapping of relevance levels 0–4 (core).

[0072] The relevance is defined as "five levels from 0 to 4", with each level having a clear business semantic and directly corresponding to gating actions: 4 points: Fully answerable. This candidate unit directly and completely supports the answer to the question; it is usually allowed to enter the context as core evidence and can trigger an "instant response".

[0073] 3 points: Can be partially answered; when there is no 4 points of evidence, it can also be used as the main basis for an independent answer, but it needs to be accompanied by strategic constraints, such as: risk warning, caliber verification, supplementary evidence retrieval or tool verification.

[0074] 2 points: Cannot support an answer, but can supplement / assist. This candidate unit lacks key elements or only provides background information, and is generally not allowed to be answered alone; it is only included in the context as supplementary evidence when the system has already selected 3 or 4 points of evidence.

[0075] 1 point: Same domain but irrelevant. The content belongs to the same domain or a similar topic, but does not match the key elements of the question; it is not included in the context by default, but can be entered into the "backtracking pool" for subsequent search disambiguation or question rewriting.

[0076] 0 points: Completely irrelevant. Irrelevant to the topic or elements of the question, and therefore eliminated.

[0077] For example, a user might ask: What will be Company A's net profit attributable to the parent company in 2024? Candidate document fragment 1 (from Company A's 2024 annual report summary): contains "2024 net profit attributable to parent company was 50.745 billion yuan". Model rationale: The company, year, and indicator are a perfect match, and the figure can be located in the original text; the level is 4 points. Gating: Core evidence is included, allowing for direct answers.

[0078] Candidate document fragment 2 (from the revenue section of the same report): mainly describes "revenue of 362.013 billion yuan," which is inconsistent with the question's metrics. Model rationale: The company and year match, but the metrics do not; rating: 2 points. Gating: only supplementary, not as the basis for the answer.

[0079] Candidate document fragment 3 (profit paragraph from peer company B): Similar topic but inconsistent content. Model rationale: Same industry domain but mismatched content; score is 1 or 0. Gating: Remove or add to the backtesting pool.

[0080] This example not only shows "which is more relevant," but also "whether it is sufficient to answer, whether a risk warning or supplementary evidence is needed, and whether only supplementary evidence is required," and each step has a reason for auditing.

[0081] The entire process not only accurately answered the questions, but each decision (why segment 1 was chosen, why segment 2 was only used as a supplement) was supported by clear scoring criteria and was fully recorded in the audit log, meeting the high requirements of financial scenarios for explainability and compliance.

[0082] S5, based on gated context assembly and agent orchestration execution (generalized document uniform selection).

[0083] Organize the context and execution plan based on the gating results: (1) Evidence selection (RAG gate): Prioritize 4-point and 3-point as core evidence; when the context length is limited, prioritize the retention of higher-level and more well-reasoned candidate units; 2-point is only included as a supplement when there are 3-point or 4-point; 1-point and 0-point are not included.

[0084] (2) Tool and strategy arrangement: When the candidate unit contains tools or strategies, priority can be determined by level. For example: if the highest score is only 3 points and the reason prompts "the value or scope needs to be verified", then the financial indicator query or scope verification tool will be called first, and then the final answer will be generated.

[0085] (3) Unified standards: Documents, tools, memories, and strategies all follow the same "reason + level + gating" process to avoid inconsistencies in arrangement caused by different sources having their own set of rules.

[0086] S6, Output and Audit Logs (Compliance Review).

[0087] Output: Generates the final answer, which may include citation information (such as the source of the cited document, chapter or excerpt number).

[0088] Audit: Simultaneously record the entire process for compliance and review, including: candidate unit list, reasons and levels for each candidate, gating decisions, tool usage process and results, and final response content.

[0089] This audit chain enables subsequent answers to questions such as: why which evidence was selected at the time, why risk warnings or tool verifications were triggered, and where exactly the error occurred.

[0090] S7, online policy adaptation and playback-driven parameter tuning.

[0091] The gating strategy is iterated and versioned based on online feedback (such as manual sampling, user error correction, and playback of erroneous samples).

[0092] For example, adjustments were made to the triggering conditions for "3-point independent answers", the risk warning template, the upper limit for including 2-point supplementary answers, and the explanation-level consistency verification rules, thereby continuously improving stability without changing the overall framework.

[0093] It should be noted that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In practical applications, all the above possible implementation methods can be arbitrarily combined in a combined manner to form possible embodiments of this application, which will not be described in detail here.

[0094] Based on the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems provided in the above embodiments, and based on the same inventive concept, this application also provides an interpretable generative reordering and business semantic level gating orchestration device for financial intelligent agent systems.

[0095] Figure 2This is a structural diagram of the interpretable generative reordering and business semantic-level gating orchestration device for financial intelligent agent systems provided in this application embodiment. Figure 2 As shown, the interpretable generative reordering and business semantic level gating orchestration device for financial intelligent agent systems may specifically include an acquisition module 210, a generative reordering module 220, a verification module 230, a gating mapping module 240, and an orchestration execution module 250.

[0096] The acquisition module 210 is used to acquire a set of candidate units, wherein the types of candidate units in the candidate unit set include one or more of document fragments, tools, memories, and strategies; Generative re-ranking module 220 is used to input the user question and each candidate unit in the candidate unit set into a pre-trained generative re-ranking model for processing, generate a scoring basis and relevance level for each candidate unit, the scoring basis is generated before the relevance level, and the scoring basis is used to explain the reasons for assigning the relevance level; the output includes a structured result including the scoring basis and relevance level for each candidate unit. The verification module 230 is used to perform consistency verification on the output structured results. The consistency verification is used to determine whether there is a logical conflict between the scoring criteria and the relevance level. The gating mapping module 240 is used to determine the gating action instruction corresponding to the relevance level according to the preset mapping rules between business semantics and gating actions after the consistency verification is passed. The gating action instruction is used to indicate the processing method of the corresponding candidate unit in context assembly and task orchestration. The orchestration execution module 250 is used to perform context assembly and task orchestration on candidate units in the candidate unit set based on gating action instructions, so as to generate orchestration results for driving the financial intelligent agent system to perform tasks.

[0097] This application provides one possible implementation method, such as... Figure 3 As shown above, Figure 2 The demonstrated apparatus may also include a building module 310 for constructing a set of candidate units through the following steps: Acquire multi-source heterogeneous data related to user issues, including document library data, tool capability description data, historical interaction memory data, and strategy template data; Multi-source heterogeneous data is formatted in a unified manner to generate multiple candidate units with a unified data structure. The unified data structure includes at least a candidate unit type field, a candidate unit content field, and a metadata field.

[0098] This application embodiment provides a possible implementation, wherein the generative reordering module 220 is further configured to: The user question and each candidate unit in the candidate unit set are input into a pre-trained generative rearrangement model for processing. The generative rearrangement model analyzes the matching between the user question and each candidate unit and generates a scoring criterion for each candidate unit. The scoring criterion includes matching point information, missing element information, and risk information. Based on the scoring criteria for each candidate unit, a relevance level is generated for each candidate unit.

[0099] This application provides a possible implementation method in which the relevance level is a level value selected from a set of preset discrete levels, and each level in the set of discrete levels has a predefined corresponding business semantics.

[0100] This application provides a possible implementation method in which the multi-level discrete grade set is a five-point grade set, and the grade values ​​in the five-point grade set include 0 points, 1 point, 2 points, 3 points, and 4 points; The pre-defined mapping rules between business semantics and gating actions include: When the relevance level is 4, it is mapped to a gating action instruction that indicates the corresponding candidate unit as core evidence to enter the context and trigger a direct response action; When the relevance level is 3, it is mapped to a gating action instruction that indicates that the corresponding candidate unit is used as the main basis to enter the context and triggers the attached policy constraint action. The attached policy constraint action includes at least one of risk warning, caliber verification, supplementary evidence retrieval or tool verification. When the relevance level is 2, it is mapped to a gating action instruction that indicates that the corresponding candidate unit will be included in the context as supplementary information given that there is already 4 or 3 points of evidence. When the relevance level is 1, it is mapped to a gating action instruction that indicates the corresponding candidate unit to be placed in the backtracking pool; When the relevance level is 0, it is mapped to a gating action instruction that indicates the removal of the corresponding candidate unit.

[0101] This application embodiment provides a possible implementation, wherein the verification module 230 is further configured to: The format of the structured results is validated to determine whether the relevance level is a valid value in the preset set of multiple discrete levels; Semantic recognition is performed on the scoring criteria in the structured results. When semantic features that represent missing key information, inability to locate evidence, inconsistent standards, or mismatch with the question are identified, it is determined whether the corresponding relevance level is higher than a preset threshold. If the corresponding relevance level is higher than the preset threshold, it is determined that there is a logical conflict, and the corresponding relevance level is downgraded or a retry mechanism is triggered.

[0102] This application embodiment provides a possible implementation, wherein the orchestration and execution module 250 is further configured to: After generating the orchestration results, an audit log is generated. The audit log is used to store the complete link information of this task execution. The complete link information includes the user question, the candidate unit set, the structured result corresponding to each candidate unit, the gating action instruction corresponding to each candidate unit, the tool call process and result, and the final answer content.

[0103] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems of any of the above embodiments.

[0104] In an exemplary embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 400 includes a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, for example, via a bus 402. Optionally, the electronic device 400 may also include a transceiver 404. It should be noted that in practical applications, the transceiver 404 is not limited to one type, and the structure of this electronic device 400 does not constitute a limitation on the embodiments of this application.

[0105] Processor 401 may be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0106] Bus 402 may include a pathway for transmitting information between the aforementioned components. Bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 402 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0107] The memory 403 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0108] The memory 403 stores computer program code that executes the scheme of this application, and its execution is controlled by the processor 401. The processor 401 executes the computer program code stored in the memory 403 to implement the content shown in the foregoing method embodiments.

[0109] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0110] Based on the same inventive concept, embodiments of this application also provide a storage medium storing a computer program, wherein the computer program is configured to execute, at runtime, the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems of any of the above embodiments.

[0111] Those skilled in the art will clearly understand that the specific working process of the systems, devices, and modules described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0112] Those skilled in the art will understand that the technical solution of this application, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several program instructions to cause an electronic device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of this application when running the program instructions. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0113] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as electronic devices like personal computers, servers, or network devices) associated with program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of this application.

[0114] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of this application, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to leave the protection scope of this application.

Claims

1. A method for interpretable generative reordering and business semantic-level gating orchestration for financial intelligent agent systems, characterized in that, The method includes: Obtain a set of candidate units, wherein the types of candidate units in the set include one or more of the following: document fragments, tools, memories, and strategies; The user question and each candidate unit in the candidate unit set are input into a pre-trained generative reordering model for processing. The model generates a scoring basis and a relevance level for each candidate unit. The scoring basis is generated before the relevance level and is used to explain the reasons for assigning a relevance level. The output includes a structured result that includes the scoring basis and relevance level for each candidate unit. The structured output results are subjected to consistency verification. Consistency verification is used to determine whether there is a logical conflict between the scoring criteria and the relevance level. After the consistency check passes, the gating action instruction corresponding to the relevance level is determined according to the preset mapping rules between business semantics and gating actions. The gating action instruction is used to indicate the processing method of the corresponding candidate unit in context assembly and task orchestration. Based on gating action instructions, candidate units in the candidate unit set are assembled into context and orchestrated into tasks to generate orchestration results that drive the financial intelligent agent system to perform tasks.

2. The method according to claim 1, characterized in that, The candidate unit set is constructed through the following steps: Acquire multi-source heterogeneous data related to user issues, including document library data, tool capability description data, historical interaction memory data, and strategy template data; Multi-source heterogeneous data is formatted in a unified manner to generate multiple candidate units with a unified data structure. The unified data structure includes at least a candidate unit type field, a candidate unit content field, and a metadata field.

3. The method according to claim 1, characterized in that, The user question and each candidate unit in the candidate unit set are input into a pre-trained generative reordering model for processing, generating a scoring criterion and relevance level for each candidate unit, including: The user question and each candidate unit in the candidate unit set are input into a pre-trained generative rearrangement model for processing. The generative rearrangement model analyzes the matching between the user question and each candidate unit and generates a scoring criterion for each candidate unit. The scoring criterion includes matching point information, missing element information, and risk information. Based on the scoring criteria for each candidate unit, a relevance level is generated for each candidate unit.

4. The method according to claim 1, characterized in that, The relevance level is a level value selected from a set of preset discrete levels, and each level in the set of discrete levels has a predefined corresponding business semantics.

5. The method according to claim 4, characterized in that, The multi-level discrete grade set is a five-point grade set, and the grade values ​​in the five-point grade set include 0 points, 1 point, 2 points, 3 points, and 4 points; The pre-defined mapping rules between business semantics and gating actions include: When the relevance level is 4, it is mapped to a gating action instruction that indicates the corresponding candidate unit as core evidence to enter the context and trigger a direct response action; When the relevance level is 3, it is mapped to a gating action instruction that indicates that the corresponding candidate unit is used as the main basis to enter the context and triggers the attached policy constraint action. The attached policy constraint action includes at least one of risk warning, caliber verification, supplementary evidence retrieval or tool verification. When the relevance level is 2, it is mapped to a gating action instruction that indicates that the corresponding candidate unit will be included in the context as supplementary information given that there is already 4 or 3 points of evidence. When the relevance level is 1, it is mapped to a gating action instruction that indicates the corresponding candidate unit to be placed in the backtracking pool; When the relevance level is 0, it is mapped to a gating action instruction that indicates the removal of the corresponding candidate unit.

6. The method according to claim 4, characterized in that, Perform consistency checks on the structured output results, including: The format of the structured results is validated to determine whether the relevance level is a valid value in the preset set of multiple discrete levels; Semantic recognition is performed on the scoring criteria in the structured results. When semantic features that represent missing key information, inability to locate evidence, inconsistent standards, or mismatch with the question are identified, it is determined whether the corresponding relevance level is higher than a preset threshold. If the corresponding relevance level is higher than the preset threshold, it is determined that there is a logical conflict, and the corresponding relevance level is downgraded or a retry mechanism is triggered.

7. The method according to claim 1, characterized in that, After generating the orchestration results, the method further includes: Generate audit log records. The audit log records are used to store the complete link information of this task execution. The complete link information includes user questions, candidate unit set, structured results corresponding to each candidate unit, gating action instructions corresponding to each candidate unit, tool call process and results, and final answer content.

8. An interpretable generative reordering and business semantic-level gating orchestration device for financial intelligent agent systems, characterized in that, The device includes: The acquisition module is used to acquire a set of candidate units, wherein the types of candidate units in the candidate unit set include one or more of the following: document fragments, tools, memories, and strategies; The generative re-ranking module is used to input the user question and each candidate unit in the candidate unit set into the pre-trained generative re-ranking model for processing, and to generate a scoring basis and relevance level for each candidate unit. The scoring basis is generated before the relevance level, and the scoring basis is used to explain the reasons for assigning the relevance level. The output includes a structured result including the scoring basis and relevance level for each candidate unit. The verification module is used to perform consistency verification on the output structured results. The consistency verification is used to determine whether there is a logical conflict between the scoring criteria and the relevance level. The gating mapping module is used to determine the gating action instruction corresponding to the relevance level according to the preset mapping rules between business semantics and gating actions after the consistency verification is passed. The gating action instruction is used to indicate the processing method of the corresponding candidate unit in context assembly and task orchestration. The orchestration and execution module is used to perform context assembly and task orchestration on candidate units in the candidate unit set based on gating action instructions, so as to generate orchestration results for driving the financial intelligent agent system to execute tasks.

9. An electronic device, characterized in that, The system includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute, at runtime, the interpretable generative reordering and business semantic level gating orchestration method for financial intelligent agent systems as described in any one of claims 1 to 7.

Images