A runtime context management method, device, medium and program product

Abstract

The application discloses a runtime context management method, device, medium and program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: obtaining a target analysis task and a corresponding offline common sense base; the offline common sense base contains preset rules, which are used for calibrating a runtime context in the process of executing the target analysis task; recording an execution track in the process of an intelligent agent executing the target analysis task, and generating experience rules by analyzing the execution track; the experience rules comprise trigger conditions and rule contents; and the runtime context is updated according to the experience rules, so that the intelligent agent executes the target analysis task according to the runtime context. The accuracy and integrity of the runtime context can be maintained in the long-chain analysis process at a lower cost, and the correctness of the analysis result is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a runtime context management method, device, medium, and program product. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, agent-based complex task analysis systems are widely used in long-link analysis scenarios such as financial analysis, legal due diligence, and academic literature review. Unlike simple question-answering scenarios, long-link analysis tasks are characterized by long texts and long links.

[0003] To support research and analysis involving long timeframes, context management techniques can be broadly categorized into two types: One is the ultra-long context approach, which expands the model's context window to input all historical records and source data into the model at once. However, each time a decision is made, all the long historical texts need to be re-inputted into the model, resulting in high inference costs. Furthermore, the model may struggle to accurately capture key instructions when processing ultra-long texts. The other is the runtime dynamic context approach, which continuously rewrites and compresses historical information to preserve the most critical information for subsequent steps. However, dynamic compression schemes are prone to losing crucial details, leading to numerous illusions and logical errors. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a runtime context management method, device, medium, and program product that can maintain the accuracy and integrity of the runtime context during long-link analysis at a lower cost, thereby improving the accuracy of analysis results. The specific solution is as follows: Firstly, this application discloses a runtime context management method, including: Obtain the target analysis task and the corresponding offline common sense base; the offline common sense base contains preset rules for calibrating the runtime context during the execution of the target analysis task; During the process of the agent performing the target analysis task, the execution trajectory is recorded, and empirical rules are generated by analyzing the execution trajectory; the empirical rules include triggering conditions and rule content. The runtime context is updated according to the empirical rules so that the agent can perform the target analysis task based on the runtime context.

[0005] Optionally, the preset rule is a structured preset rule constructed according to the target field structure; The step of generating empirical rules by analyzing the execution trajectory includes: Based on the tool call steps, input parameters, return results, success status or failure status in the execution trajectory, structured empirical rules are generated according to the target field structure. The target field structure includes: rule content, triggering condition, confidence level, usage effect index, token usage, and lifecycle status.

[0006] Optionally, the runtime context management method further includes: Monitor the token usage status of the runtime context; When the token usage state meets the preset pruning trigger condition, the value score of each empirical rule in the runtime context is calculated; The empirical rules in the runtime context are pruned based on the value score.

[0007] Optionally, the preset pruning triggering conditions include: The condition C+O+S≥W is satisfied; where C is the token usage of the current runtime context, O is the reserved output space, S is the safety margin, and W is the maximum context window of the model. And / or, the token usage of all rules in the current runtime context reaches the preset rule budget threshold.

[0008] Optionally, the value score is calculated based on the number of times the rule is used, the success rate of the rule association, the confidence level of the rule, the most recent hit time of the rule, the matching degree of the rule's applicable scope, and the amount of rule tokens occupied.

[0009] Optionally, before recording the execution trajectory during the intelligent agent's execution of the target analysis task, the method further includes: The target analysis task is broken down into multiple sub-tasks; Select target preset rules corresponding to the subtask from the offline common sense base, and construct the runtime context of the current subtask based on the target preset rules; Save the empirical rules and evidence summaries generated during the execution of the subtasks for use in subsequent subtasks.

[0010] Optionally, the runtime context management method further includes: Obtain user feedback on the analysis results of the target analysis task; Candidate rules are generated based on the feedback information, and when a candidate rule meets the inclusion criteria, it is added to the offline common sense database.

[0011] Optionally, before updating the runtime context according to the empirical rule, the method further includes: The empirical rules are used to perform conflict detection with the preset rules in the offline common sense base; the conflict detection dimensions include any one or more of the following dimensions: indicator selection dimension, business analysis caliber dimension, tool call dimension, and compliance constraint dimension. When a conflict is detected, conflict handling is performed according to the confidence levels corresponding to the empirical rules and the preset rules, respectively.

[0012] Optionally, the conflict handling based on the confidence levels corresponding to the empirical rules and the preset rules includes: When the confidence level of the preset rule is greater than the preset high confidence threshold, and the confidence level of the empirical rule is less than the preset low confidence threshold, the confidence level of the empirical rule is reduced to the preset lower limit and marked as a rule to be verified; the rule to be verified does not participate in task analysis. If the confidence level of the empirical rule is greater than the preset low confidence threshold and less than the preset high confidence threshold, then the empirical rule is added to the runtime context and its priority is reduced. When the confidence difference between the empirical rule and the preset rule is less than the preset difference threshold, a user confirmation prompt is generated, and the current empirical rule is processed based on user feedback.

[0013] Secondly, this application discloses a runtime context management device, including: The common sense base acquisition module is used to acquire the target analysis task and the corresponding offline common sense base; the offline common sense base contains preset rules for calibrating the runtime context during the execution of the target analysis task; An experience rule generation module is used to record the execution trajectory during the process of the agent performing the target analysis task, and generate experience rules by analyzing the execution trajectory; the experience rules include triggering conditions and rule content. The context update module is used to update the runtime context according to the empirical rules, so that the agent can perform the target analysis task according to the runtime context.

[0014] Thirdly, this application discloses an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned runtime context management method.

[0015] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program implements the aforementioned runtime context management method when executed by a processor.

[0016] Fifthly, this application discloses a computer program product, including a computer program that, when executed by a processor, implements the aforementioned runtime context management method.

[0017] In this application, a target analysis task and a corresponding offline common sense base are obtained; the offline common sense base contains preset rules for calibrating the runtime context during the execution of the target analysis task; the execution trajectory is recorded during the execution of the target analysis task by the agent, and empirical rules are generated by analyzing the execution trajectory; the empirical rules include triggering conditions and rule content; the runtime context is updated according to the empirical rules so that the agent can execute the target analysis task according to the runtime context.

[0018] It is evident that by analyzing the execution trajectory to generate empirical rules, the experience learned from the execution trajectory is extracted into rules that can be directly invoked in subsequent steps, rather than simply compressing text summaries, thus effectively preserving key details. At the same time, during execution, the runtime context is calibrated using an offline common sense base to ensure that the generated empirical rules do not violate domain common sense. This approach can maintain the accuracy and integrity of the runtime context during long-link analysis at a lower cost, reducing the occurrence of illusions and improving the accuracy of analysis results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart of a runtime context management method provided in this application; Figure 2 A flowchart of a specific dynamic pruning method provided in this application; Figure 3 A flowchart illustrating a specific subtask context construction method provided in this application; Figure 4 A flowchart of a specific rule conflict detection method provided in this application; Figure 5 A schematic diagram of a runtime context management device is provided in this application; Figure 6 This application provides a structural diagram of an electronic device. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] In related technologies, a runtime dynamic context approach is used. This approach continuously rewrites and compresses historical information to retain the most critical information for subsequent steps. However, dynamic compression schemes are prone to losing key details, resulting in numerous illusions and logical errors. To overcome these technical problems, this application proposes a runtime context management method that can maintain the accuracy and integrity of the runtime context during long-link analysis at a lower cost, thereby improving the accuracy of analysis results.

[0023] This application discloses a runtime context management method, which can be applied to long-chain analysis tasks such as generating lengthy financial analysis reports, legal due diligence reports, and academic literature reviews.

[0024] See Figure 1 As shown, the method may include the following steps: Step S11: Obtain the target analysis task and the corresponding offline common sense base.

[0025] First, acquire the target analysis task, its corresponding offline common sense base, and the basic data for the target analysis task, such as financial data for a financial analysis task. The target analysis task can be a complex analysis task that requires the agent to complete through multiple steps and the invocation of multiple tools. The offline common sense base is pre-built and contains at least one preset rule. This preset rule can be unstructured natural language rules or structured rules built according to the target field structure (such as JSON format or Markdown lists). The preset rule is used to calibrate the runtime context during the execution of the target analysis task. The offline common sense base is pre-built according to the domain of the analysis task, such as an offline common sense base for finance in the financial domain.

[0026] For example, if the objective analysis task is to generate an annual financial analysis report for a company in a given year, then the task and its corresponding offline common sense base are retrieved. Preset rules may include: profitability analysis of banking companies should not use gross profit margin as a metric, but should focus on net interest margin. These rules calibrate the runtime context during the execution of the objective analysis task to ensure that the agent does not incorrectly use the gross profit margin metric in banking analysis.

[0027] In this embodiment, the offline common sense base is constructed through two methods: cold start and offline feedback update. During the cold start phase, basic rules from the current business domain are used as the initial context, such as banking analysis rules and tool invocation rules. During the offline feedback update phase, large-scale online usage data is collected, and high-confidence rules are formed based on positive feedback such as user likes and acceptances, and negative feedback such as modifications and rewrites. These rules are then periodically updated to the offline common sense base.

[0028] In some embodiments, the preset rule feedback update includes the following steps: obtaining user feedback information on the analysis results of the target analysis task; generating candidate rules based on the feedback information; and adding the candidate rules to the offline common sense base when the candidate rules meet the inclusion conditions.

[0029] After generating the analysis results for the target analysis task, the system provides these results to the user and records user feedback. Feedback includes, but is not limited to, acceptance, modification, deletion, supplementation, and rewriting. For example, when analyzing banking clients, the system detects that over 80% of users, upon receiving a report containing gross profit margin analysis, delete or modify that section to net interest margin analysis. Based on this feedback, a candidate rule is generated: the profitability analysis of the banking industry should not include gross profit margin indicators and should be replaced with net interest margin analysis. The inclusion criteria can be a threshold number of feedback responses or administrator confirmation, etc., which are not limited in this embodiment. By collecting user feedback and aggregating it into candidate rules, continuous iterative updates to the offline common sense base are achieved; as usage increases, the rules in the offline common sense base will become more accurate and complete.

[0030] Step S12: Record the execution trajectory during the process of the intelligent agent performing the target analysis task, and generate empirical rules by analyzing the execution trajectory; the empirical rules include triggering conditions and rule content.

[0031] An execution trajectory refers to the complete operation record generated by an agent during task execution. Empirical rules are generated through analysis and reflection on the execution trajectory, serving as rule units for subsequent analysis and reference. During execution, the execution trajectory of each step is recorded in real time, including the tool name, input parameters, return result, and success or failure status. Empirical rules are generated through reflective analysis; these rules include specific triggering conditions and rule content for subsequent guidance and analysis.

[0032] For example, the execution trajectory records: Step 1: The "company_search" tool is called with the input parameter {"name": "Company A"}, and the result is null (not found), with a failure status; Step 2: The "stock_code_search" tool is called with the input parameter {"code": "000xxx"}, and the result is that Company A's data has been successfully retrieved, with a success status. Analyzing the above execution trajectory, an empirical rule can be extracted: When querying data related to Company A, prioritize using the stock code (000xxx) rather than the Chinese name for retrieval. This empirical rule includes the triggering condition, i.e., the query target contains Company A, and the rule content, i.e., using the stock code 000xxx instead of the Chinese name for retrieval.

[0033] It should be noted that the empirical rules in this embodiment can be either unstructured natural language rules or structured rules, as long as they contain triggering conditions and rule content. When stored in a structured format, each rule can include fields such as rule content, triggering conditions, confidence level, usage effect indicators, token usage, and lifecycle status. A token is the smallest semantic unit in text processing.

[0034] In some preferred embodiments, the preset rules in the offline common sense base are structured preset rules constructed according to the target field structure. The step of generating empirical rules by analyzing the execution trajectory includes: generating structured empirical rules according to the target field structure based on the tool call steps, input parameters, return results, success or failure status in the execution trajectory; the target field structure includes: rule content, triggering conditions, confidence level, usage effect indicators, token usage, and lifecycle status.

[0035] Specifically, after each tool call completed by the agent, the input parameters, return results, and success / failure status involved in this step are analyzed and semantically reflected upon using a large language model. Specifically, the above trajectory can be input into the large language model, along with a reflection instruction (Prompt) such as: "Reviewing the above execution trajectory, your previous attempt failed, but the subsequent attempt succeeded. Please summarize this experience to avoid similar failures in subsequent steps." The rules should include triggering conditions and rule content.

[0036] The aforementioned target field structure includes: rule content, triggering conditions, confidence level, usage effect metrics, token usage, and lifecycle status; it may also include, but is not limited to, rule number, scope of application, source, most recent hit time, successful association degree, and evidence citation. Token usage refers to the actual number of tokens used by the rule in the context, automatically calculated based on the text length of the rule content and the word segmentation results. Lifecycle status includes active, pending verification, merged, and expired states, used to mark the current lifecycle stage of the rule.

[0037] In the preferred embodiment, the structured preset rules and structured experience rules use the same target field structure, enabling unified scoring, sorting, pruning, and conflict detection operations to be performed on both types of rules without needing to write different processing logic for different types of rules. It is understood that by using structured field definitions to define rules, rules are no longer indivisible natural language text, but rather structured data objects that can be automatically parsed and managed, providing a data foundation for subsequent dynamic pruning and conflict calibration. This means that value scores can be calculated based on metadata such as rule usage frequency, successful association degree, and confidence level; budget control can be performed based on token usage; precise matching and loading can be achieved through rule content and triggering conditions; and automated management can be performed, such as merging similar rules, eliminating low-value rules, and protecting high-confidence rules. If rules are maintained only in natural language text form without structured storage, the accuracy of relevant information acquisition cannot be guaranteed, making it impossible to score, sort, and prune rules. All rule management would rely on manual work, leading to a sharp decline in rule management efficiency when the number of rules grows rapidly in long-link analysis tasks.

[0038] Step S13: Update the runtime context according to empirical rules so that the agent can perform the target analysis task based on the runtime context.

[0039] The runtime context is a dynamically maintained set of information by the agent during a single analysis task, based on the execution trajectory, tool return results, and reflection results; specifically, an offline common sense base can be used as the initial runtime context. During execution, the generated empirical rules are added to the runtime context so that when the agent needs to query the company's data again during subsequent analysis, it can directly apply the rules and use the stock code for retrieval, avoiding repeated failures.

[0040] Meanwhile, the offline common sense base continuously calibrates the runtime context during execution; when an empirical rule conflicts with a preset rule in the offline common sense base, conflict resolution is performed based on the confidence level of each rule to ensure that the empirical rule does not violate domain common sense.

[0041] This embodiment extracts execution trajectories into empirical rules, achieving a transformation from lossy compression to lossless extraction of context, effectively preserving key details in the execution trajectory. Simultaneously, calibration using an offline common sense base prevents the propagation of erroneous experiences. Compared to existing technologies, the technical solution of this embodiment significantly improves the accuracy of long-link analysis tasks and reduces the incidence of illusions caused by information loss.

[0042] As can be seen from the above, this embodiment generates empirical rules by analyzing the execution trajectory, extracting the experience learned from the execution trajectory into rules that can be directly called in subsequent steps, rather than simply compressing text summaries, thereby effectively preserving key details. At the same time, during execution, an offline common sense base is used to calibrate the runtime context to ensure that the generated empirical rules do not violate domain common sense. This can maintain the accuracy and integrity of the runtime context in long-link analysis at a lower cost, reduce the occurrence of illusions, and improve the accuracy of analysis results.

[0043] Based on the above embodiments, this application discloses a specific dynamic pruning method, for example... Figure 2 As shown, the dynamic pruning process includes the following steps: Step S21: Monitor the token usage status of the runtime context.

[0044] Continuously monitor the token usage of the current runtime context. A token is the smallest semantic unit in text processing. The context window of an LLM is measured in terms of the number of tokens. For example, if the context window of a certain model is 128K tokens, it means that the model can process a maximum of about 128,000 tokens at a time.

[0045] In this embodiment, C (token occupancy in the current runtime context), O (reserved output space, i.e., the space required for the model to generate the next inference or analysis results, used to ensure the generation of the next inference or report paragraph), and B (token occupancy of all rules in the current runtime context) are calculated in real time. The token occupancy of the current runtime context includes system prompts, user questions, historical execution trajectories, rule sets, retrieval evidence summaries, intermediate conclusions, etc.

[0046] Step S22: When the token usage status meets the preset pruning trigger condition, calculate the value score of each empirical rule. The system determines that the pruning trigger condition has been met when any of the following conditions are met.

[0047] Triggering condition 1: C+O+S≥W. Here, S is a safety margin to account for uncertainties caused by fluctuations in the length returned by the tool and word segmentation errors; W is the maximum context window of the model. This constraint ensures that the overall context does not exceed the model window limit.

[0048] Triggering condition two: The token usage of all rules in the current runtime context reaches a preset rule budget threshold. For example, let the preset rule budget threshold Bmax = 20,000 tokens. When the token usage B of all empirical rules in the runtime context reaches 20,000 tokens, pruning is triggered. This rule budget constraint ensures that the rule set does not excessively expand and crowd out space for other critical information, such as user questions, historical data, and evidence summaries.

[0049] Once pruning is triggered, the value score of each empirical rule in the current runtime context is calculated. The value score is calculated based on the rule's usage frequency, rule success correlation, rule confidence, rule's most recent hit time, rule's applicability scope matching degree, and rule token usage. Rule success correlation refers to the success rate of related analysis steps after the rule is applied; rule applicability scope matching degree refers to the degree to which the rule's applicability scope matches the current task.

[0050] In the preferred embodiment, the value score is calculated using a weighted summation method: Score(i) = w1 × Number of uses (i) + w2 × Success correlation (i) + w3 × Confidence (i) + w4 × Recent hit time (i) + w5 × Applicable scope matching degree (i) - w6 × Token usage (i); Among them, w1 to w6 are the weight coefficients of each item, which can be dynamically adjusted according to the actual application scenario.

[0051] For example, suppose there are the following two rules in the runtime context: Rule A: 5 uses, success rate 1.0, confidence level 0.85, recent hit time (normalized) 0.8, applicable scope matching degree 1.0, token usage 120. Rule B: 8 uses, success rate 0.9, confidence level 0.80, recent hit time (normalized) 0.6, applicable scope matching degree 1.0, token usage 110.

[0052] Therefore, Score(A) = 0.25×5 + 0.25×1.0 + 0.20×0.85 + 0.15×0.8 + 0.15×1.0 - 0.0005×120 = 1.88; Score(B) = 0.25×8 + 0.25×0.9 + 0.20×0.80 + 0.15×0.6 + 0.15×1.0 - 0.0005×110 = 2.57. Rule B has a higher value score than Rule A, therefore Rule A is prioritized for elimination during pruning.

[0053] Step S23: Prune the empirical rules in the runtime context based on the value score.

[0054] In this embodiment, rules are sorted from low to high according to their value scores, and pruning is performed. Pruning includes any one or more of the following: merging similar rules, eliminating low-value rules, retaining high-value rules, and protecting high-confidence rules.

[0055] The similarity rule merging process identifies semantically similar rules and generates a new, more generalized rule using LLM (Local Rule Management). Low-value rule elimination removes the rules with the lowest value scores, determined by the rule's popularity (frequency of invocation). High-value rule retention ensures that rules with value scores above a preset threshold are not eliminated. High-confidence rule protection ensures that high-confidence rules belonging to the offline common sense base are not eliminated; specifically, it checks if a rule belongs to the high-confidence rules in the cold-start common sense base, and if so, it is retained with reduced weight rather than directly deleted.

[0056] For example, the context before pruning includes the following rules: Rule A (usages: 5, confidence: 0.85): Photovoltaic industry analysis must include capacity utilization; Rule B (usages: 8, confidence: 0.80): Semiconductor manufacturing companies should focus on analyzing capacity utilization and yield; Rule C (usages: 2, confidence: 0.90): For manufacturing companies, the changes in the debt-to-asset ratio over the past three years should be compared; Rule D (usages: 0, confidence: 0.50): Retail companies' cash flow analysis should focus on quarterly volatility. After calculation, the value scores for Rules A, B, C, and D are 85, 88, 75, and 12, respectively. Furthermore, since Rules A and B both focus on capacity utilization and have a semantic similarity higher than the preset threshold (e.g., 0.85), they are merged into Rule E: Manufacturing companies (especially those in the photovoltaic and semiconductor industries) must analyze capacity utilization, and semiconductor companies additionally need to focus on yield. After merging, Rule E is used 13 times. Subsequently, the system eliminates rule D, which has the lowest value score, by removing rule D from the runtime context. Rule C, although having a low value score, has a high confidence level and is protected from elimination.

[0057] This embodiment monitors the token usage status and triggers pruning when the triggering conditions are met, avoiding the computational waste caused by compression in every round of dialogue; at the same time, it calculates value scores through multi-dimensional factors, so that rules with high usage frequency and high success rate are retained, while low-value rules are eliminated, ensuring that the context window is always occupied by high-value content.

[0058] Based on the above embodiments, this application discloses a specific subtask context construction method, applicable to long-chain analysis scenarios such as generating lengthy financial analysis reports. For example... Figure 3 As shown, before recording the execution trajectory during the agent's execution of the target analysis task, the following steps are performed: S31: Divide the target analysis task into multiple sub-tasks.

[0059] For example, if the objective analysis task is to generate an annual financial analysis report for a company in a certain year, it can be broken down into several sub-tasks, such as company overview, revenue structure analysis, profitability analysis, solvency analysis, cash flow analysis, industry comparison analysis, risk warning, and conclusions and recommendations.

[0060] S32: Select target preset rules corresponding to the subtask from the offline common sense base, and construct the runtime context of the current subtask based on the target preset rules.

[0061] Taking the profitability analysis subtask as an example, when the target company is a bank, preset rules that include profitability analysis as a trigger condition and are applicable to industries including banking are selected from the offline common sense base. The selected target preset rules are then loaded into the runtime context of the profitability analysis subtask as the initial context of the subtask.

[0062] S33: Save the empirical rules and evidence summaries generated during the execution of the subtask for subsequent subtask invocation.

[0063] If new rules of thumb are generated through reflection and summarization during the execution of the profitability analysis subtask, these rules and their corresponding evidence summaries are saved in the global context. When performing the subsequent industry comparative analysis subtask, these rules and evidence summaries can be retrieved from the global context without needing to be re-searched and calculated. This binding of rules of thumb with evidence summaries such as tool calls, financial statement excerpts, and indicator calculation results facilitates conclusion traceability and manual review.

[0064] In this embodiment, each subtask independently constructs a context window, and each subtask's context window only contains rules and evidence directly related to its analysis objective, significantly reducing the number of input tokens required for a single model call. Key information is passed between different subtasks through empirical rules and evidence summaries, eliminating the need for each subtask to repeat the same retrieval and calculation steps, thus significantly improving processing efficiency.

[0065] Based on the above embodiments, this application discloses a specific rule conflict detection method. This method performs conflict detection between empirical rules and an offline common sense base before the empirical rules are updated to the runtime context. For example... Figure 4 As shown, it includes the following steps: Step S41: Perform conflict detection between the empirical rules and the preset rules in the offline common sense base.

[0066] The dimensions of conflict detection mentioned above include any one or more of the following: indicator selection dimension, business analysis caliber dimension, tool usage dimension, and compliance constraint dimension.

[0067] In this context, the indicator selection dimension refers to whether the empirical rules recommend indicators that are not applicable to the current industry or analytical objectives. For example, when performing a profitability analysis of the banking industry, the agent generates an empirical rule through reflection: the profitability analysis of the banking industry should analyze the trend of gross profit margin changes. However, there is a preset rule in the offline common sense base: the profitability analysis of the banking industry should not use gross profit margin as the core indicator, but should focus on net interest margin. That is, there is a conflict between the two in terms of indicator selection dimensions.

[0068] The business analysis caliber dimension refers to checking whether the rule of thumb uses a calculation caliber or comparison benchmark that is inconsistent with domain conventions. For example, the rule of thumb stipulates that a year-on-year caliber should be used when calculating revenue growth, while the offline common sense base stipulates that the industry has seasonal characteristics and a month-on-month caliber should be used.

[0069] The tool invocation dimension refers to detecting whether the empirical rules specify data acquisition methods with low success rates or that are not authoritative. For example, the empirical rule may specify using the Chinese name to search for company information, while the offline common sense base may specify that the company's stock code has a higher success rate.

[0070] Among them, the compliance constraint dimension refers to whether the empirical rules for detection violate laws, regulations, regulatory requirements, or industry standards.

[0071] Step S42: When a conflict is detected, conflict handling is performed according to the confidence levels corresponding to the empirical rules and the preset rules, respectively.

[0072] Upon detecting a conflict, conflict handling is further performed based on the confidence levels corresponding to both empirical rules and preset rules. This embodiment, through a multi-dimensional conflict detection mechanism, can comprehensively identify deviations between empirical rules and domain common sense from multiple perspectives, including indicator selection, analysis criteria, tool invocation, and compliance constraints, ensuring that erroneous rules generated by the agent's reflection that violate domain common sense or compliance requirements are not used.

[0073] In some embodiments, conflict handling based on the confidence levels corresponding to the empirical rule and the preset rule includes: S421: When the confidence level of the preset rule is greater than the preset high confidence threshold, and the confidence level of the empirical rule is less than the preset low confidence threshold, the confidence level of the empirical rule is reduced to a preset lower limit and marked as a rule to be verified; the rule to be verified does not participate in task analysis. In other words, when the confidence level of a preset rule is greater than a preset high confidence threshold, and the confidence level of an empirical rule is less than a preset low confidence threshold, the preset rule is considered to have a much higher confidence level than the empirical rule. In this case, the empirical rule should be excluded and not included in the context. The aforementioned preset high confidence threshold and preset low confidence threshold can be flexibly configured according to actual circumstances.

[0074] S422: When the confidence level of the empirical rule is greater than the preset low confidence threshold and less than the preset high confidence threshold, the empirical rule is added to the runtime context and its priority is reduced. When the confidence level of an empirical rule is between a preset low confidence threshold and a preset high confidence threshold, it indicates that although the rule is not globally applicable, it still has local applicability value in specific scenarios and should be retained but with a reduced priority.

[0075] S423: When the confidence difference between the empirical rule and the preset rule is less than the preset difference threshold, generate a user confirmation prompt message and process the current empirical rule according to the user feedback.

[0076] When the empirical rule conflicts with the preset rule, and the confidence difference is less than the preset difference threshold, a user confirmation prompt is generated, and the user makes the decision.

[0077] Through the above conflict handling, erroneous rules of experience can be eliminated in a timely manner, while rules with local value can be retained. At the same time, users can make decisions on rules with doubts, thus achieving efficient and accurate conflict handling of rules of experience.

[0078] Accordingly, embodiments of this application also disclose a runtime context management device, see [link to relevant documentation]. Figure 5 As shown, the device includes: The common sense base acquisition module 11 is used to acquire the target analysis task and the corresponding offline common sense base; the offline common sense base contains preset rules for calibrating the runtime context during the execution of the target analysis task; The experience rule generation module 12 is used to record the execution trajectory during the process of the intelligent agent performing the target analysis task, and generate experience rules by analyzing the execution trajectory; the experience rules include triggering conditions and rule content. The context update module 13 is used to update the runtime context according to the empirical rules, so that the agent can perform the target analysis task according to the runtime context.

[0079] In one specific embodiment, the preset rule is a structured preset rule constructed according to the target field structure; the empirical rule generation module 12 specifically includes: The structured rule generation unit is used to generate structured empirical rules based on the tool call steps, input parameters, return results, success status or failure status in the execution trajectory, according to the target field structure; the target field structure includes: rule type, triggering condition, confidence level, usage effect index, token usage and lifecycle status.

[0080] In one specific embodiment, the device further includes: The token monitoring module is used to monitor the token usage status of the runtime context; The value scoring module is used to calculate the value score of each empirical rule in the runtime context when the usage status of the token meets the preset pruning triggering conditions. The pruning module is used to prune the empirical rules in the runtime context based on the value score.

[0081] In one specific embodiment, the preset pruning trigger condition includes: The condition C+O+S≥W is satisfied; where C is the token usage of the current runtime context, O is the reserved output space, S is the safety margin, and W is the maximum context window of the model. And / or, the token usage of all rules in the current runtime context reaches the preset rule budget threshold.

[0082] In one specific embodiment, the value score is calculated based on the number of times the rule is used, the success rate of rule association, the confidence level of the rule, the most recent hit time of the rule, the matching degree of the rule's applicable scope, and the amount of rule tokens occupied.

[0083] In one specific embodiment, the pruning process includes any one or more of the following: merging similar rules, eliminating low-value rules, retaining high-value rules, and protecting high-confidence rules.

[0084] In one specific embodiment, the device further includes: The task splitting module is used to split the target analysis task into multiple sub-tasks before the experience rule generation module records the execution trajectory; The subtask context construction module is used to filter out target preset rules corresponding to the subtask from the offline common sense base, and construct the runtime context of the current subtask based on the target preset rules; The information storage module is used to store the empirical rules and evidence summaries generated during the execution of the subtasks for subsequent subtasks to call.

[0085] In one specific embodiment, the device further includes: The feedback acquisition module is used to acquire user feedback information on the analysis results of the target analysis task; A candidate rule generation module is used to generate candidate rules based on the feedback information; The rule entry module is used to add the candidate rule to the offline common sense database when the candidate rule meets the entry conditions.

[0086] In one specific embodiment, the device further includes: The conflict detection module is used to perform conflict detection between the empirical rules and the preset rules in the offline common sense base before the context update module updates the runtime context according to the empirical rules; the conflict detection dimensions include any one or more of the following dimensions: indicator selection dimension, business analysis caliber dimension, tool call dimension, and compliance constraint dimension. The conflict handling module is used to handle conflicts based on the confidence levels corresponding to the empirical rules and the preset rules when a conflict is detected.

[0087] In one specific embodiment, the conflict handling module specifically includes: The first processing unit is configured to reduce the confidence level of the empirical rule to a preset lower limit and mark it as a rule to be verified when the confidence level of the preset rule is greater than a preset high confidence threshold and the confidence level of the empirical rule is less than a preset low confidence threshold; the rule to be verified does not participate in task analysis. The second processing unit is configured to add the empirical rule to the runtime context and reduce its priority when the confidence level of the empirical rule is less than the preset low confidence threshold and greater than the preset high confidence threshold. The third processing unit is used to generate a user confirmation prompt when the confidence difference between the empirical rule and the preset rule is less than a preset difference threshold, and to process the current empirical rule according to user feedback.

[0088] Furthermore, this application also discloses an electronic device, see [link to relevant documentation]. Figure 6 As shown, the content in the figure should not be considered as any limitation on the scope of use of this application.

[0089] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the runtime context management method disclosed in any of the foregoing embodiments.

[0090] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0091] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include operating system 221, computer program 222 and data 223 including execution trace, etc., and the storage method can be temporary storage or permanent storage.

[0092] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the runtime context management method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0093] Furthermore, this application also discloses a computer storage medium storing computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, they implement the runtime context management method steps disclosed in any of the foregoing embodiments.

[0094] Furthermore, embodiments of this application also disclose a computer program product, including a computer program that, when executed by a processor, implements the runtime context management method steps disclosed in any of the foregoing embodiments.

[0095] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0096] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0097] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0098] The above provides a detailed description of the runtime context management method, device, medium, and program product provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A runtime context management method, characterized in that, include: Obtain the target analysis task and the corresponding offline common sense database; The offline common sense base contains preset rules for calibrating the runtime context during the execution of the target analysis task; During the process of the agent performing the target analysis task, the execution trajectory is recorded, and empirical rules are generated by analyzing the execution trajectory; the empirical rules include triggering conditions and rule content. The runtime context is updated according to the empirical rules so that the agent can perform the target analysis task based on the runtime context.

2. The runtime context management method according to claim 1, characterized in that, The preset rules are structured preset rules constructed according to the target field structure; The step of generating empirical rules by analyzing the execution trajectory includes: Based on the tool call steps, input parameters, return results, success status or failure status in the execution trajectory, structured empirical rules are generated according to the target field structure. The target field structure includes: rule content, triggering condition, confidence level, usage effect index, token usage, and lifecycle status.

3. The runtime context management method according to claim 1, characterized in that, Also includes: Monitor the token usage status of the runtime context; When the token usage state meets the preset pruning trigger condition, the value score of each empirical rule in the runtime context is calculated; The empirical rules in the runtime context are pruned based on the value score.

4. The runtime context management method according to claim 3, characterized in that, The preset pruning trigger conditions include: The condition C+O+S≥W is satisfied; where C is the token usage of the current runtime context, O is the reserved output space, S is the safety margin, and W is the maximum context window of the model. And / or, the token usage of all rules in the current runtime context reaches the preset rule budget threshold.

5. The runtime context management method according to claim 3, characterized in that, The value score is calculated based on the number of times the rule is used, the success rate of the rule's association, the confidence level of the rule, the most recent hit time of the rule, the matching degree of the rule's applicable scope, and the amount of rule tokens used.

6. The runtime context management method according to claim 1, characterized in that, Before recording the execution trajectory during the intelligent agent's execution of the target analysis task, the method further includes: The target analysis task is broken down into multiple sub-tasks; Select target preset rules corresponding to the subtask from the offline common sense base, and construct the runtime context of the current subtask based on the target preset rules; Save the empirical rules and evidence summaries generated during the execution of the subtasks for use in subsequent subtasks.

7. The runtime context management method according to claim 1, characterized in that, Also includes: Obtain user feedback on the analysis results of the target analysis task; Candidate rules are generated based on the feedback information, and when a candidate rule meets the inclusion criteria, it is added to the offline common sense database.

8. The runtime context management method according to claim 1, characterized in that, Before updating the runtime context according to the empirical rule, the method further includes: The empirical rules are used to perform conflict detection with the preset rules in the offline common sense base; the conflict detection dimensions include any one or more of the following dimensions: indicator selection dimension, business analysis caliber dimension, tool call dimension, and compliance constraint dimension. When a conflict is detected, conflict handling is performed according to the confidence levels corresponding to the empirical rules and the preset rules, respectively.

9. The runtime context management method according to claim 8, characterized in that, The conflict handling based on the confidence levels corresponding to the empirical rules and the preset rules includes: When the confidence level of the preset rule is greater than the preset high confidence threshold, and the confidence level of the empirical rule is less than the preset low confidence threshold, the confidence level of the empirical rule is reduced to the preset lower limit and marked as a rule to be verified; the rule to be verified does not participate in task analysis. If the confidence level of the empirical rule is greater than the preset low confidence threshold and less than the preset high confidence threshold, then the empirical rule is added to the runtime context and its priority is reduced. When the confidence difference between the empirical rule and the preset rule is less than the preset difference threshold, a user confirmation prompt is generated, and the current empirical rule is processed based on user feedback.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the runtime context management method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein the computer programs, when executed by a processor, implement the runtime context management method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the runtime context management method according to any one of claims 1 to 9.

Images