Large model agent dynamic configuration system and method based on SpringBoot

Abstract

The invention discloses a large-model intelligent agent dynamic configuration system and method based on SpringBoot, and relates to the technical field of configuration management, and the method comprises the steps: constructing an interface layer for packaging large-model reasoning, knowledge retrieval and tool calling in SpringBoot, generating a configuration description object based on attribute binding, and generating a configuration description object; selecting an effective configuration according to the session context to determine a target model and a knowledge base; and a session prompt and a tool sequence are generated according to a prompt template and a tool chain rule, and reasoning and dialogue management are executed by combining a delay sensitive condition and a dynamic routing selection calling line. Through the technical characteristics, decoupling of large model capability and service codes, user-oriented and scene-oriented assembly according to configuration and time delay adaptive routing are realized, and the response stability of the intelligent agent and the SLA guarantee capability are improved.

Description

Technical Field

[0001] This invention relates to the field of configuration management technology, and in particular to a dynamic configuration system and method for large-scale intelligent agents based on Spring Boot. Background Technology

[0002] With the increasing application of large-scale pre-trained language models (LLMs) and agents in scenarios such as question answering retrieval, process automation, and enterprise knowledge assistants, the composite orchestration of "model inference + knowledge retrieval + tool invocation" has become the main paradigm for building intelligent applications. In the Java ecosystem, Spring Boot, relying on its automatic configuration, component scanning, and integration with configuration centers, has become an important runtime environment for large model inference, vector retrieval, and tool orchestration. In practice, configuration files and remote configuration centers are often used to centrally manage large model parameters, vector indexes, business knowledge bases, and prompt templates. Combined with session persistence and caching mechanisms, contextual retrieval is supported, driving the evolution of large model agents from "single service" to an "configurable and scalable" engineering form.

[0003] However, existing large model application integration based on Spring Boot still has shortcomings: First, it lacks an abstract interface layer and a unified configuration description object that simultaneously supports large model inference, embedded models, retrieval post-processing, and tool calls. Model configuration, knowledge base selection, prompt templates, and tool links are mostly fixed in a fixed call process, making it difficult to flexibly orchestrate and quickly switch between capability combinations under different users, business scenarios, or agent identifiers based solely on configuration. It is also difficult to finely select the target large model and target knowledge base based on the session context at runtime. Second, it lacks a latency-sensitive assembly and dynamic routing mechanism that combines expected latency levels, resource load, and fault status. Furthermore, it does not uniformly incorporate tool call order, multi-turn dialogue memory, and configuration version management into the execution context, making it difficult to simultaneously meet service SLAs and long dialogue context continuity in scenarios with multiple agents concurrently, resource fluctuations, and frequent configuration changes. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a dynamic configuration method for large-scale intelligent agents based on Spring Boot to solve the problems of inflexible configuration of intelligent agent capability combinations and lack of latency awareness and resource awareness optimization in the existing technology.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for dynamic configuration of large model intelligent agents based on Spring Boot, which includes building a multi-dimensional abstract interface layer in Spring Boot, encapsulating large model reasoning, knowledge retrieval and external tool calling capabilities, and assigning dimension identifiers to each abstract interface; Based on configuration attribute binding rules, the large model, knowledge base, prompt template, and memory strategy fields are read from the configuration source and a configuration description object is generated. Based on the configuration parsing rules, select the effective configuration from the set of configuration description objects according to the session context and determine the target large model and target knowledge base; The system utilizes a prompt template configuration mechanism and tool call chain generation rules to generate prompt text adapted to the current session and constructs an external tool call sequence. Based on latency-sensitive conditions and dynamic routing strategies, select the large model call line and assemble the corresponding implementation to perform inference. The dialogue records and configuration versions are written through a multi-turn dialogue memory storage mechanism and configuration change persistence rules.

[0007] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on Spring Boot according to the present invention, the construction of the multi-dimensional abstract interface layer includes defining an abstract interface in the Spring Boot runtime environment for encapsulating large model-related capabilities; The abstract interface includes a first abstract interface for encapsulating large model inference calls, a second abstract interface for encapsulating text vectorization and vector retrieval, a third abstract interface for encapsulating retrieval result post-processing logic, and a fourth abstract interface for encapsulating external tool calls and function call post-processing logic. When the application starts, it scans the implementation classes with identifier annotations, loads each implementation class that implements the abstract interface into the container, and establishes a key-value mapping relationship according to the dimension type and implementation identifier. Bind each abstract interface implementation to a unique dimension type identifier and implementation identifier; When a configuration description object generated based on configuration attribute binding rules is received, the identifiers in the configuration description object are mapped to the corresponding abstract interface implementations in the multidimensional abstract interface layer.

[0008] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on Spring Boot according to the present invention, the generation of configuration description objects includes configuring large model intelligent agent configuration information in Spring Boot configuration resources in a hierarchical namespace manner, and assigning a unique configuration prefix to each intelligent agent in the local configuration file and the remote configuration center. Bind configuration items starting with the configuration prefix to a predefined configuration description object; The configuration description object includes large model configuration fields, embedded model configuration fields, knowledge base configuration fields, prompt template configuration fields, toolset configuration fields, and memory strategy fields; When the application starts and the configuration is refreshed, the configuration description object is instantiated according to the mapping relationship between the field name and the configuration key, and the instantiated configuration description object is registered in the container.

[0009] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on Spring Boot described in this invention, the step of selecting the effective configuration and determining the target large model and the target knowledge base includes parsing session context information from the request message when receiving an external request; The session context information includes user identifier, business scenario identifier, session identifier, expected latency level, and description of available resources; The parsed session context information is encapsulated into a context object and passed into the configuration parsing process. The configuration parsing rules then match the intelligent agent identifier in the context object against the set of bound configuration description objects. When there are multiple candidate configuration description objects, they are sorted according to a priority strategy. The configuration resolution rules select the first candidate configuration description object as the effective configuration for the current session according to the priority value from largest to smallest; After obtaining the effective configuration, the configuration parsing rules determine the target large model inference abstract interface implementation based on the large model configuration field in the configuration description object, and determine the target embedded model implementation, target knowledge base, and retrieval method based on the embedded model configuration field and knowledge base configuration field.

[0010] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on SpringBoot described in this invention, the step of generating prompt text adapted to the current session includes: after the configuration parsing rules determine the effective configuration, reading the prompt template configuration field and the toolset configuration field from the configuration description object, and mapping the template identifier stored in the prompt template configuration field to the specific template content in the prompt template storage medium. When generating prompts, user input, historical dialogue summaries, and knowledge content from the session context are injected into the prompt template according to variable names. The conditional expressions in the prompt template are evaluated. When the conditional expressions are calculated based on the session context, if the result is true, the corresponding fragment is retained; if the result is false, the corresponding fragment is deleted, resulting in specific prompt text adapted to the current session.

[0011] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on Spring Boot described in this invention, the construction of the external tool call sequence includes: querying the external tool call abstract interface implementation in the multidimensional abstract interface layer at runtime according to the tool list, tool call order and tool dependency relationship defined in the tool set configuration field; encapsulating each tool implementation as a chain node object; and connecting multiple chain objects end to end according to the configuration order to form a call sequence.

[0012] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on Spring Boot described in this invention, the construction of the external tool call sequence further includes determining the execution mode of each chain node in the call sequence according to the call mode field in the tool configuration; When the calling mode is serial, the calling session context is passed sequentially. When the calling mode is parallel, the calling session context is copied for each chain node, and the results of each chain node are aggregated after all chain nodes have been executed. When the tool configuration sets a chain node to depend on the execution results of other chain nodes, the chain nodes are topologically sorted according to the dependency relationship to ensure that the dependent nodes of any chain node have completed execution before it is executed.

[0013] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on Spring Boot described in this invention, the step of selecting the large model call line and assembling the corresponding implementation to perform inference includes, in the application configuration phase, presetting the mapping relationship between the latency level and the large model call line; Each large model call line consists of at least one large model inference abstract interface implementation and several tool call nodes; In the Spring Boot environment, for each large model call line, define a latency-sensitive conditional assembly expression for the implementation of the large model inference abstract interface and the external tool call abstract interface. When the conditional assembly expression is true based on the current latency level, resource load and fault status, the corresponding implementation is allowed to be assembled into the container. If the result is false, it remains unassembled. By calculating the expected response time of each candidate large model call line, the candidate large model call line that meets the response threshold and has the smallest expected response time is selected as the actual execution line for the current request.

[0014] As a preferred embodiment of the dynamic configuration method for large model intelligent agents based on SpringBoot described in this invention, the step of writing dialogue records and configuration versions includes: after each request is processed, extracting dialogue round information and key information summary from the large model inference results, encapsulating the session identifier, round number, question content, response content and summary content into a dialogue record object, and selecting the target storage medium according to the memory strategy configuration in the currently effective configuration. When the memory policy is configured for short-term memory mode, the conversation record object is written to a cache that supports expiration time control. When the memory strategy is configured to long-term recording mode, the conversation record object is written to persistent storage, and a composite index is built based on the session identifier and timestamp during the writing process. When a new request arrives, the multi-turn dialogue memory storage mechanism queries the most recent dialogue record from the target storage medium based on the session identifier, and truncates and merges the query results according to the maximum context length configured in the configuration, injecting the merged historical content into the template variable used by the prompt template configuration mechanism.

[0015] Secondly, the present invention provides a dynamic configuration system for large model intelligent agents based on Spring Boot, including an interface construction module, a configuration generation module, a target selection module, a call chain generation module, an inference execution module, and a persistence module; The interface building module is used to build a multi-dimensional abstract interface layer in Spring Boot, encapsulate large model reasoning, knowledge retrieval and external tool calling capabilities, and assign dimension identifiers to each abstract interface; The configuration generation module is used to read the large model, knowledge base, prompt template and memory strategy field from the configuration source based on the configuration attribute binding rules and generate a configuration description object. The target selection module is used to select the effective configuration from the set of configuration description objects according to the configuration parsing rules and the session context, and to determine the target large model and the target knowledge base. The call chain generation module is used to generate prompt text adapted to the current session using the prompt template configuration mechanism and tool call chain generation rules, and to construct an external tool call sequence; The inference execution module is used to select large model call lines and assemble the corresponding implementations to perform inference based on latency-sensitive conditions and dynamic routing strategies. The persistence module is used to write dialogue records and configuration versions through a multi-turn dialogue memory storage mechanism and configuration change persistence rules.

[0016] The beneficial effects of this invention are as follows: By constructing a multi-dimensional abstract interface layer in the Spring Boot runtime environment, the decoupling of large model-related capabilities from business code and constant-time positioning of target implementation are achieved; by generating configuration description objects through attribute binding rules, differentiated target large models, knowledge bases, and retrieval methods can be assembled for different users and business scenarios at runtime, relying solely on configuration; by using latency-sensitive condition assembly expressions and expected response time calculation rules, dynamic routing and assembly of large model call paths based on expected latency levels, resource load, and health status are achieved, ensuring the service SLA is met; and by mapping template identifiers to template content through a prompt template configuration mechanism, automatic trimming and generation of prompt text based on the current session context and parallel tool call sequences that satisfy dependency constraints are achieved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for dynamically configuring large model agents based on Spring Boot.

[0019] Figure 2 This is a schematic diagram of a dynamic configuration system for large model intelligent agents based on Spring Boot.

[0020] Figure 3 A flowchart for determining the target knowledge base.

[0021] Figure 4 A flowchart for generating prompt text and the call sequence. Figure 5 This is a graph showing the SLA satisfaction rate under different expected delay levels.

[0022] Figure 6 This is a graph showing the relationship between the expected response time and the actual response time. Detailed Implementation

[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0025] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0026] Reference Figures 1-6 This is one embodiment of the present invention, which provides a method for dynamic configuration of large model intelligent agents based on Spring Boot, including the following steps: S1. In Spring Boot, build a multi-dimensional abstract interface layer to encapsulate large model reasoning, knowledge retrieval, and external tool calling capabilities, and assign dimension identifiers to each abstract interface.

[0027] Furthermore, several abstract interfaces are defined in the Spring Boot runtime environment to encapsulate capabilities related to large models.

[0028] The abstract interface includes a first abstract interface for encapsulating large model inference calls, a second abstract interface for encapsulating text vectorization and vector retrieval, a third abstract interface for encapsulating retrieval result post-processing logic, and a fourth abstract interface for encapsulating external tool calls and function call post-processing logic.

[0029] When the application starts, the component scanning mechanism scans the implementation classes with identifier annotations, loads each implementation class that implements the above abstract interface into the container, and establishes a key-value mapping relationship according to the dimension type and implementation identifier. The dimension type identifier is a string enumeration, such as "LLM_INFER" or "EMBEDDING"; the implementation identifier can be the implementation class Bean name or the implementation code declared in the annotation; the key-value mapping relationship is a mapping from [dimensional type identifier, implementation identifier] to the implementation object.

[0030] Each abstract interface implementation is bound to a unique dimension type identifier and implementation identifier.

[0031] When a configuration description object generated based on configuration attribute binding rules is received, the large model identifier, embedded model identifier, knowledge base identifier, retrieval action identifier, and tool identifier in the configuration description object are mapped to the corresponding abstract interface implementations in the multi-dimensional abstract interface layer.

[0032] A lookup table is constructed in memory using key-value mapping to map dimension type identifiers to abstract interface implementations. This allows the target implementation class to be located in constant time complexity during request processing based on the dimension type and implementation identifier. The lookup table can be maintained using either a HashMap or a ConcurrentHashMap.

[0033] S2. Based on the configuration attribute binding rules, read the large model, knowledge base, prompt template and memory strategy fields from the configuration source and generate a configuration description object.

[0034] Furthermore, the configuration information of large model agents is configured in the Spring Boot configuration source in a hierarchical namespace manner. A unique configuration prefix is ​​assigned to each agent in the local configuration file and the remote configuration center. The configuration items starting with the agent configuration prefix are bound to the predefined configuration description object through the property binding component.

[0035] The configuration description object includes large model configuration fields, embedded model configuration fields, knowledge base configuration fields, prompt template configuration fields, toolset configuration fields, and memory strategy fields.

[0036] When the application starts or the configuration is refreshed, the property binding component traverses the configuration source and instantiates the configuration description object according to the mapping relationship between field names and configuration keys; and registers the instantiated configuration description object into the container.

[0037] When the configuration source changes, the attribute binding process is re-executed by listening for configuration change events, replacing the old configuration description object with the new one, and assigning an incrementing configuration version number to each configuration description object; the incrementing method is global increment.

[0038] During the attribute binding process, the configuration attribute binding rules limit the use of default values ​​to supplement missing configuration items, and constrain the sampling verification rules for illegal configurations. When a required field is detected to be empty or the field value does not meet the format constraints, an exception is thrown to prevent the configuration description object corresponding to the corresponding agent from becoming invalid.

[0039] For example, missing items are read from the global default configuration, and if they are still missing, hard-coded default values ​​are used; a list of required fields is declared using annotations or enumerations, and a specific exception is thrown when a field is empty.

[0040] During the runtime phase, the configuration attribute binding rules specify that the mapping is between the configuration identifier and the agent identifier in the session context. When the agent identifier carried in the request cannot be found in the set of bound configuration description objects, the configuration attribute binding rules fall back to the scenario default configuration and the global default configuration in order of priority. The binding result records the configuration source, version number, and effective time corresponding to each configuration description object. Specifically, when no configuration is found based on the agent identifier, the scenario default configuration is searched based on the business scenario identifier. If no configuration is found, the global default configuration is used. Missing fields are filled in step by step in the order of scenario default and global default.

[0041] It should be noted that the configuration attribute binding rule refers to taking the key-value set in the configuration source and the field definition of the configuration description object as input, and outputting the bound configuration description object by traversing the configuration items, performing format verification and default value completion logic.

[0042] S3. Based on the configuration parsing rules, select the effective configuration from the set of configuration description objects according to the session context and determine the target large model and target knowledge base.

[0043] Furthermore, when an external request is received, session context information is parsed from the request message.

[0044] Session context information includes user identifier, business scenario identifier, session identifier, expected latency level, and description of available resources.

[0045] The parsed session context information is encapsulated into a context object and passed into the configuration parsing process. The configuration parsing rules match the bound configuration description objects in the context object based on the agent identifier. When there are multiple candidate configuration description objects, they are sorted according to the priority strategy.

[0046] The priority strategy is expressed by calculating a priority value. Specifically, the total value of the Boolean value representing whether the agent identifier matches the user-level configuration, the Boolean value representing whether the agent identifier matches the scene-level configuration, and the Boolean value representing whether it is the default configuration is used as the priority value.

[0047] It should be noted that when the agent identifier matches the user-level configuration or scenario-level configuration, the Boolean value is represented as 1 in the numerical field; if they do not match, the Boolean value is represented as 0 in the numerical field.

[0048] It should also be noted that if the candidate configuration description object is the default configuration, the Boolean value is represented as 1 in the numerical field; if it is not the default configuration, the Boolean value is represented as 0 in the numerical field.

[0049] The configuration resolution rules select the first candidate configuration description object as the effective configuration for the current session according to the priority value from largest to smallest.

[0050] After obtaining the effective configuration, the configuration resolution rules determine the target large model inference abstract interface implementation based on the large model configuration field in the configuration description object, and determine the target embedded model implementation, target knowledge base, and retrieval method based on the embedded model configuration field and knowledge base configuration field. When the large model configuration field indicates that it is necessary to switch to a backup large model or call multiple large models in parallel, the configuration resolution rules decide whether to enable parallel inference or cascaded inference based on the expected latency level and available resource description in the session context. The specific judgment rules for parallel inference and cascaded inference are as follows: for example, when the expected latency is strict and the CPU and memory utilization is lower than the utilization threshold, parallel inference is started; when the expected latency is more lenient and resources are scarce, cascaded inference is preferred.

[0051] It should be noted that the CPU and video memory utilization thresholds can be set by the system developers based on offline stress testing, with typical values ​​ranging from 0.6 to 0.8.

[0052] Based on the index name, retrieval method, and filtering conditions contained in the knowledge base configuration fields, the specific knowledge retrieval process and post-retrieval processing logic are determined. The configuration parsing rules also encapsulate the parsing results, prompt template configuration fields, toolset configuration fields, and memory strategy configuration fields into an execution context. Among them, when there are multiple knowledge bases or retrieval methods, the knowledge base corresponding to the business scenario identifier is selected first. If there are multiple corresponding knowledge bases, they are selected according to the update time in the configuration. If there are multiple corresponding retrieval methods, they are selected according to the vector retrieval priority, or they can be selected by adding keyword retrieval to supplement the selection.

[0053] It should be noted that the configuration resolution rule refers to taking the session context object and the set of configuration description objects as input, and outputting the effective configuration and encapsulated execution context corresponding to the current session through a priority strategy.

[0054] S4. Generate prompt text adapted to the current session using the prompt template configuration mechanism and tool call chain generation rules, and construct an external tool call sequence.

[0055] Furthermore, after the configuration parsing rules determine the effective configuration, the prompt template configuration field and toolset configuration field are read from the configuration description object, and the template identifier stored in the prompt template configuration field is mapped to the specific template content in the prompt template storage medium.

[0056] The template content includes placeholder variable symbols and conditional expressions used to control branches.

[0057] When generating prompts, user input from the conversation context, historical conversation summaries recovered based on the multi-turn dialogue memory storage mechanism, and knowledge content retrieved from the target knowledge base are injected into the prompt template according to variable names. The conditional expressions in the prompt template are evaluated. When the conditional expressions are calculated based on the conversation context, if the result is true, the corresponding fragment is retained; if the result is false, the corresponding fragment is deleted, resulting in specific prompt text adapted to the current conversation.

[0058] Furthermore, to ensure that conditional expressions in the prompt template can be parsed and executed at runtime, the Spring Expression Language is used as the syntax specification for conditional expressions. Conditional expressions are stored as strings within the prompt template content and marked with placeholders, such as "#{…}" as start and end markers. Conditional expressions are allowed to reference attribute fields from the session context object, the history summary object, and the knowledge retrieval result object. For example, `context.userLevel` can represent the user level, `history.turnCount` can represent the number of rounds in the current session, and `kb.hitCount` can represent the number of documents hit in this knowledge retrieval. When generating the prompt text, the session context, history summary, and… The knowledge retrieval results are encapsulated in the expression evaluation environment and bound to the variable names context, history, and kb respectively. The SpEL expression parsing component is then called to evaluate the conditional expression. When the evaluation result of the conditional expression is a Boolean true value, the template fragment associated with the conditional expression is retained. When the evaluation result of the conditional expression is a Boolean false value or the expression parsing fails, the corresponding template fragment is deleted. For example, the current fragment is only retained in the final prompt text when the conditional expression #{context.userLevel=='VIP'} is configured in the template. Or, for example, the template fragment containing the conditional expression is only injected into the target prompt text when the current template is configured with the conditional expression #{history.turnCount>5&&kb.hitCount>0}.

[0059] It should be noted that the prompt template configuration mechanism refers to taking template identifiers, variable names and context field mapping relationships, and conditional expressions as inputs, and outputting target prompt text strings that are adapted to the current session through template analysis, variable injection, and condition evaluation.

[0060] Furthermore, the tool call chain generation rules, based on the tool list, tool call order, and tool dependencies defined in the toolset configuration fields, query the external tool call abstract interface implementations in the multidimensional abstract interface layer at runtime, encapsulate each tool implementation as a chain node object, and connect multiple chain objects end to end according to the configuration order to form a call sequence.

[0061] The execution mode of each chain node in the call sequence is determined by the call mode field in the tool configuration. When the call mode is serial, the call session context is passed sequentially. When the call mode is parallel, the call session context is copied for each chain node, and the results of each chain node are aggregated after all chain nodes have been executed.

[0062] When the tool configuration sets a chain node to depend on the execution results of other chain nodes, the chain nodes are topologically sorted according to the dependency relationship to ensure that the dependent nodes of any chain node have completed execution before it is executed. Finally, the generated prompt text and call sequence are injected into the abstract interface implementation corresponding to the target inference model, driving the large model to trigger external tool calls in a predetermined order during the inference process.

[0063] Furthermore, to support topological sorting of tool call chains based on tool dependencies, this embodiment models tool dependencies as a directed acyclic graph (DAG). Specifically, each tool in the toolset configuration field is assigned a unique tool identifier, and each tool is abstracted as a tool node in the DAG. The tool identifier and a list of preceding dependent tool identifiers are recorded in the DAG representation of the tool node. In the configuration source, the dependency relationship is described by the correspondence between the current tool identifier and the set of preceding tool identifiers. When parsing the toolset configuration field, the attribute binding component constructs a dependency relationship storage structure in the form of an adjacency list based on the configuration content. The adjacency list uses the tool identifier as the key and the set of subsequent tool identifiers of the dependent tool as the value, forming a DAG that describes the tool dependency relationship.

[0064] Furthermore, during runtime, the topological sorting process is executed based on a directed acyclic graph structure and implemented using the Kahn algorithm. Specifically, the in-degree value of each tool node is counted, and tool nodes with an in-degree of zero are added to the execution queue. Then, tool nodes are sequentially retrieved from the execution queue and appended to the end of the call sequence. The in-degree values ​​of all successor nodes pointed to by the tool nodes are decremented. When the in-degree value of a successor node is decremented to zero, the current successor node is added to the execution queue. This process is repeated until the queue is empty or all tool nodes have been processed.

[0065] It should be noted that each external tool call abstract interface implementation is encapsulated as a tool node object. The tool node object can be used as a chain node in the call chain, or as a graph node when modeling dependencies.

[0066] It should be noted that if tool nodes with an in-degree greater than zero are detected during the sorting process, it is determined that there is a circular dependency in the current dependency configuration, and the current tool call chain is prevented from taking effect through an exception mechanism.

[0067] It should be noted that the tool call chain generation rule refers to taking the tool set configuration fields and the external tool implementations available in the multidimensional abstract interface layer as input, and outputting an ordered tool chain node sequence that satisfies the dependency constraints through dependency modeling, topological sorting, and call pattern parsing.

[0068] S5. Based on latency-sensitive conditions and dynamic routing strategies, select the large model call line and assemble the corresponding implementation to perform inference.

[0069] Furthermore, during the application configuration phase, a mapping relationship between several latency levels and large model call lines is constructed. Each large model call line consists of at least one large model inference abstract interface implementation and several tool call nodes. The average response time and resource consumption indicators during historical runtime are recorded for each large model call line.

[0070] At runtime, based on the expected latency level in the session context and the real-time latency values ​​and available resources of the major model call lines currently being monitored, the available major model call lines are filtered out, and major model call lines that are currently unavailable or whose predicted latency exceeds the upper limit are removed.

[0071] The latency-sensitive condition assembly and dynamic routing strategy calculates the expected response time for each candidate large model call line, compares the expected response time with the response threshold corresponding to the expected latency level, and selects the candidate large model call line that meets the response threshold and has the smallest expected response time as the actual execution line for the current request. Specifically, the expected response time is calculated as the weighted average of the historical average response time calculated based on historical call records, the short-term moving average response time calculated based on the most recent time window call, the queuing waiting time estimated based on the current number of queued requests and the average processing rate per request, and the tool call time estimated based on the average execution cost of the tool call chain corresponding to the current line.

[0072] It should be noted that, in a preferred embodiment, when the system has stable offline statistical data, configurable weights can be introduced to the duration component without changing the core technical idea of ​​the present invention; however, in this embodiment, in order to ensure the determinism, stability and interpretability of the calculation rules, the equal weighted average method is preferred.

[0073] It should also be noted that the response threshold can be configured according to the SLA requirements of the business for different latency levels. For example, it can be set to within 500ms for the strict level, within 1s for the standard level, and within 2s for the lenient level.

[0074] In the Spring Boot environment, for each large model call path, define a latency-sensitive conditional assembly expression for the implementation of the large model inference abstract interface and the external tool call abstract interface. When the conditional assembly expression is true based on the current latency level, resource load, and fault status, the corresponding implementation is allowed to be assembled into the container. If the result is false, it remains unassembled.

[0075] Furthermore, the logic for determining the latency-sensitive conditional assembly expression specifically involves the Spring Boot runtime environment maintaining and monitoring metrics and status fields related to latency for each large model call line. These monitoring metrics include, but are not limited to, the expected latency level `delayLevel` carried by the current session context, the real-time resource load metrics `cpuLoad` and `memoryLoad` corresponding to the line, and the line's fault status metrics `errorRate` and `healthStatus`. Here, `delayLevel` represents one of three levels: strict, standard, or lenient; `cpuLoad` represents the CPU utilization of the compute node where the current call line resides; `memoryLoad` represents the memory utilization of the compute node where the current call line resides; `errorRate` represents the call failure rate of the line within the statistical window; and `healthStatus` represents the line availability indicator given by the health check logic. The latency-sensitive conditional assembly expression is configured using Boolean expressions to indicate whether the implementation is allowed to be assembled into the container under different latency levels and resource states.

[0076] For example, an expression of the following form: enable=(delayLevel=='STRICT'&&cpuLoad<0.7&&errorRate<0.01&&healthStatus=='UP')||(delayLevel=='RELAXED'&&cpuLoad<0.9&&healthStatus=='UP').

[0077] Furthermore, during application startup or configuration refresh, the property binding component reads the Boolean expression as a string from the configuration source and parses it into executable decision logic using the SpEL condition parsing component. During the execution of route selection and assembly, the delayLevel from the current session context, the monitored cpuLoad, memoryLoad, errorRate, and healthStatus are injected into the expression evaluation environment to evaluate the assembly expression for delay-sensitive conditions. When the evaluation result is true, it is determined that the corresponding large model inference abstract interface implementation and the external tool call abstract interface implementation meet the current delay and resource constraints, allowing them to be assembled and participate in this large model call route. If the evaluation result is false, the current implementation remains in an unassembled state and does not participate in the execution of the current request.

[0078] At the same time, the route identifier and key monitoring indicators used in this routing decision are recorded in the execution context.

[0079] It should be noted that dynamic routing strategy refers to taking the expected latency level, historical indicators and real-time monitoring indicators of candidate large model call lines as input, and outputting the selected target large model call line identifier and related routing decisions through expected response time calculation and availability filtering.

[0080] In this embodiment, to verify the effect of the latency-sensitive condition assembly and dynamic routing strategy of the present invention on the Service Level Agreement (SLA) under different expected latency levels and different load QPS (queries per second), a general-purpose server with a Linux operating system, a multi-core general-purpose CPU, 32GB of memory, and a gigabit Ethernet network environment is used. The business gateway and large-scale intelligent agent orchestration service are deployed based on the Spring Boot framework, calling three typical large-scale model service paths in the backend to simulate low-latency, medium-latency, and high-computing-power high-latency paths, respectively. Different basic response time, concurrency carrying capacity, and error rate parameters are set for each path. The request side is divided into three categories according to the business scenario: strict latency level, standard latency level, and lenient latency level, and the expected latency level is used as the input of the latency-sensitive condition assembly expression.

[0081] System A is a control scheme that statically selects a fixed call path based solely on the expected latency level; System B is an embodiment of the present invention that enables dynamic routing by assembling latency-sensitive conditions, calculating expected response time rules, and using runtime resource load and health status indicators.

[0082] During the test, multiple rounds of session requests were continuously injected under different request intensity levels. Statistical data on response time, SLA fulfillment, predicted time, and actual time were collected for each level, resulting in the following: Figure 5The SLA satisfaction rate curves shown are for different expected delay levels and Figure 6 The graph shown illustrates the relationship between the expected response time and the actual response time.

[0083] Figure 5 As shown in the figure above, at low QPS levels, both solutions maintain a high SLA satisfaction rate across all levels, with their curves largely overlapping. As QPS gradually increases, the SLA satisfaction rate of the control solution system A decreases significantly, especially at the strict latency level, where the decrease is the fastest. In contrast, the corresponding curve of system B in the present invention is generally above the curve of system A, indicating that in medium-to-high load scenarios, the present invention can better ensure that the response latency does not exceed the SLA constraints at each level. Figure 5 The figure below is a magnified view of a high-load range. As can be seen from the magnified view, even under the most stringent latency constraints and when the resource load is close to the bottleneck, the embodiment of the present invention can still improve the SLA satisfaction rate, verifying that the delay-sensitive condition assembly expression and the expected response time calculation rule can improve and stably maintain the SLA satisfaction rate.

[0084] Figure 6 As can be seen from the figure above, most of the scatter points are distributed near the diagonal, indicating that the calculated expected response time is highly correlated with the actual response time. The prediction results can reflect the latency change trend under different routing paths and their load conditions. Figure 6 The figure below is a magnified view of a medium response time range. In the local view, the sample points with large deviations between the prediction and the actual values ​​are marked by bidirectional arrows, and the quantitative values ​​of the local feature differences are given. This shows that there may still be some deviations in extreme cases, but the overall error range is controlled. This verifies that the expected response time calculation rule can provide a reliable delay estimation basis for dynamic routing and delay-sensitive condition assembly, and is a key supporting means to achieve fine-grained path selection according to the expected delay level.

[0085] S6. Write the dialogue record and configuration version through a multi-turn dialogue memory storage mechanism and configuration change persistence rules.

[0086] Furthermore, after each request is processed, dialogue turn information and key information summaries are extracted from the large model inference results. The session identifier, turn number, question content, response content, and summary content are encapsulated into a dialogue record object. The target storage medium is selected according to the memory policy configuration in the current effective configuration. When the memory policy is configured as short-term memory mode, the dialogue record object is written to a high-speed cache that supports expiration time control, such as Redis. When the memory policy is configured as long-term recording mode, the dialogue record object is written to persistent storage, such as a document database, and a composite index is built based on the session identifier and timestamp during the writing process.

[0087] When a new request arrives, the multi-turn dialogue memory storage mechanism queries the most recent dialogue record from the target storage medium based on the session identifier, and truncates and merges the query results according to the maximum context length configured in the configuration, injecting the merged historical content into the template variable used by the prompt template configuration mechanism.

[0088] It should be noted that in this embodiment, the maximum context length is defined as the upper limit of the length counted by the number of tokens. Specifically, after retrieving historical dialogue records from the target storage medium by session identifier, the maximum context token limit value maxContextTokens matching the target large model is read according to the memory strategy configuration in the configuration description object. The question content, reply content, and summary content in each historical dialogue record are segmented and tokenized, the corresponding token count is calculated, and the historical dialogue records are traversed and accumulated in reverse order from the most recent round to the earliest round. When the total number of tokens accumulated does not exceed maxContextTokens, the dialogue content of the current round is merged into the historical context to be injected. When the number of tokens exceeds maxContextTokens after accumulating the next historical record, the accumulation stops, and the accumulated part is used as the historical context fragment available for this request. Within a single dialogue record, when the number of tokens in the text itself exceeds maxContextTokens, the record is truncated according to sentence boundaries or paragraph boundaries, and only the first half of the content that meets the token limit is retained.

[0089] For configuration changes, the configuration change persistence rules generate a configuration version object each time the configuration property binding result changes.

[0090] The configuration version object includes the version number, change time, and a summary of the configuration content.

[0091] The configuration version object is written to persistent storage, and a version list is established according to the agent identifier. When the configuration parsing rules are running, the latest version or the specified version of the configuration is selected according to the version number. When the latest version is detected to be inconsistent with the version used in the current session and the hot switching conditions are not met, the current version is maintained until the session ends. When the hot switching conditions are met, the latest configuration version is switched at the session boundary, and the version numbers before and after the switch are written to the dialogue record object.

[0092] It should be noted that the hot-swap condition determination rules are specifically implemented in the configuration parsing rules, which maintain state fields related to configuration version switching for each session. These include the session-level hot-swap permission flag `allowHotSwitch`, the flag indicating whether the session is in a critical transaction phase (`criticalFlag`), the stability flag for the latest configuration version (`newVersionStable`), and the release time of the latest configuration version (`versionPublishTime`). `allowHotSwitch` is explicitly provided by the memory policy configuration or business scenario configuration in the configuration description object, indicating whether the current session allows switching configuration versions during the session. `criticalFlag` is set to true by the business system when entering critical transaction processes such as payment or order placement, and set to false after the transaction is completed or rolled back. `newVersionStable` is calculated by the configuration change persistence rules in conjunction with the monitoring component based on the error rate and alarm status of the new version within the observation window; it is set to true when there are no fatal alarms. `versionPublishTime` is the timestamp of the latest configuration version being written to persistent storage. The hot-swap condition is determined using a Boolean expression.

[0093] Hot-switching conditions are expressed using Boolean expressions, for example: hotSwitch=(allowHotSwitch==true)&&(criticalFlag==false)&&(newVersionStable==true)&&(now-versionPublishTime>=minPropagationWindow).

[0094] When the Boolean expression for the hot-swap condition evaluates to true, the hot-swap condition is determined to be met. When the current session completes the current round of request processing and reaches the session boundary, the configuration version bound to the session is switched to the latest configuration version. When the expression evaluates to false, even if an updated version is detected, the old configuration version used by the current session is maintained until the session ends.

[0095] It should be noted that the multi-turn dialogue memory storage mechanism refers to taking the conversation identifier, the current round's question content, the response content, and the summary content as input, and outputting the dialogue record and historical context fragments that can be injected into the prompt template through memory strategy selection, token-level context truncation and merging.

[0096] It should also be noted that the configuration change persistence rule refers to taking the new configuration description object, the old configuration version number, and the change time as input, and outputting a new configuration version object by incrementing the version number, generating a version content summary, and maintaining the version linked list, and updating the version association relationship in the persistent storage.

[0097] This embodiment also provides a dynamic configuration system for large model intelligent agents based on Spring Boot, including: an interface construction module, a configuration generation module, a target selection module, a call chain generation module, an inference execution module, and a persistence module.

[0098] The interface building module is used to build a multi-dimensional abstract interface layer in Spring Boot, encapsulating large model reasoning, knowledge retrieval, and external tool invocation capabilities, and assigning dimension identifiers to each abstract interface.

[0099] The configuration generation module is used to read the large model, knowledge base, prompt template and memory strategy fields from the configuration source based on the configuration attribute binding rules and generate a configuration description object.

[0100] The target selection module is used to select the effective configuration from the set of configuration description objects according to the configuration parsing rules and the session context, and to determine the target big model and target knowledge base.

[0101] The call chain generation module is used to generate prompt text adapted to the current session by utilizing the prompt template configuration mechanism and tool call chain generation rules, and to construct the external tool call sequence.

[0102] The inference execution module is used to select the large model call line based on latency-sensitive conditions and dynamic routing strategies, and to assemble the corresponding implementation to execute inference.

[0103] The persistence module is used to write dialogue records and configuration versions through a multi-turn dialogue memory storage mechanism and configuration change persistence rules.

[0104] In summary, this invention achieves decoupling of large model-related capabilities from business code and constant-time positioning of target implementation by constructing a multi-dimensional abstract interface layer in the Spring Boot runtime environment; it enables the generation of configuration description objects through attribute binding rules, allowing for the configuration of differentiated target large models, knowledge bases, and retrieval methods for different users and business scenarios at runtime, relying solely on configuration; it achieves dynamic routing and assembly of large model call paths based on expected latency levels, resource load, and health status through latency-sensitive condition assembly expressions and expected response time calculation rules, ensuring service SLA satisfaction; and it achieves automatic trimming and generation of prompt text based on the current session context and parallel tool call sequences that satisfy dependency constraints by mapping template identifiers to template content through a prompt template configuration mechanism.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic configuration of large-scale intelligent agents based on Spring Boot, characterized in that: include, In Spring Boot, a multi-dimensional abstract interface layer is built to encapsulate large model reasoning, knowledge retrieval, and external tool invocation capabilities, and dimension identifiers are assigned to each abstract interface. Based on configuration attribute binding rules, the large model, knowledge base, prompt template, and memory strategy fields are read from the configuration source and a configuration description object is generated. Based on the configuration parsing rules, select the effective configuration from the set of configuration description objects according to the session context and determine the target large model and target knowledge base; The system utilizes a prompt template configuration mechanism and tool call chain generation rules to generate prompt text adapted to the current session and constructs an external tool call sequence. Based on latency-sensitive conditions and dynamic routing strategies, select the large model call line and assemble the corresponding implementation to perform inference. The dialogue records and configuration versions are written through a multi-turn dialogue memory storage mechanism and configuration change persistence rules.

2. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 1, characterized in that: The construction of the multidimensional abstract interface layer includes defining abstract interfaces in the Spring Boot runtime environment for encapsulating capabilities related to large models; The abstract interface includes a first abstract interface for encapsulating large model inference calls, a second abstract interface for encapsulating text vectorization and vector retrieval, a third abstract interface for encapsulating retrieval result post-processing logic, and a fourth abstract interface for encapsulating external tool calls and function call post-processing logic. When the application starts, it scans the implementation classes with identifier annotations, loads each implementation class that implements the abstract interface into the container, and establishes a key-value mapping relationship according to the dimension type and implementation identifier. Bind each abstract interface implementation to a unique dimension type identifier and implementation identifier; When a configuration description object generated based on configuration attribute binding rules is received, the identifiers in the configuration description object are mapped to the corresponding abstract interface implementations in the multidimensional abstract interface layer.

3. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 2, characterized in that: The generated configuration description object includes configuring the configuration information of the large model intelligent agent in the Spring Boot configuration resource in a hierarchical namespace manner, and assigning a unique configuration prefix to each intelligent agent in the local configuration file and the remote configuration center; Bind configuration items starting with the configuration prefix to a predefined configuration description object; The configuration description object includes large model configuration fields, embedded model configuration fields, knowledge base configuration fields, prompt template configuration fields, toolset configuration fields, and memory strategy fields; When the application starts and the configuration is refreshed, the configuration description object is instantiated according to the mapping relationship between the field name and the configuration key, and the instantiated configuration description object is registered in the container.

4. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 3, characterized in that: The step of selecting the effective configuration and determining the target large model and target knowledge base includes parsing session context information from the request message when an external request is received; The session context information includes user identifier, business scenario identifier, session identifier, expected latency level, and description of available resources; The parsed session context information is encapsulated into a context object and passed into the configuration parsing process. The configuration parsing rules then match the intelligent agent identifier in the context object against the set of bound configuration description objects. When there are multiple candidate configuration description objects, they are sorted according to a priority strategy. The configuration resolution rules select the first candidate configuration description object as the effective configuration for the current session according to the priority value from largest to smallest; After obtaining the effective configuration, the configuration parsing rules determine the target large model inference abstract interface implementation based on the large model configuration field in the configuration description object, and determine the target embedded model implementation, target knowledge base, and retrieval method based on the embedded model configuration field and knowledge base configuration field.

5. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 4, characterized in that: The process of generating prompt text adapted to the current session includes, after the configuration parsing rules determine the effective configuration, reading the prompt template configuration field and the toolset configuration field from the configuration description object, and mapping the template identifier stored in the prompt template configuration field to the specific template content in the prompt template storage medium. When generating prompts, user input, historical dialogue summaries, and knowledge content from the session context are injected into the prompt template according to variable names. The conditional expressions in the prompt template are evaluated. When the conditional expressions are calculated based on the session context, if the result is true, the corresponding fragment is retained; if the result is false, the corresponding fragment is deleted, resulting in specific prompt text adapted to the current session.

6. The dynamic configuration method for large model intelligent agents based on Spring Boot as described in claim 5, characterized in that: The construction of the external tool call sequence includes querying the external tool call abstract interface implementation in the multidimensional abstract interface layer at runtime according to the tool list, tool call order and tool dependency relationship defined in the tool set configuration field, encapsulating each tool implementation as a chain node object, and connecting multiple chain objects end to end according to the configuration order to form a call sequence.

7. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 6, characterized in that: The construction of the external tool call sequence also includes determining the execution method of each chain node in the call sequence based on the call mode field in the tool configuration; When the calling mode is serial, the calling session context is passed sequentially. When the calling mode is parallel, the calling session context is copied for each chain node, and the results of each chain node are aggregated after all chain nodes have been executed. When the tool configuration sets a chain node to depend on the execution results of other chain nodes, the chain nodes are topologically sorted according to the dependency relationship to ensure that the dependent nodes of any chain node have completed execution before it is executed.

8. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 7, characterized in that: The selection of large model call lines and the corresponding implementation of inference execution include, during the application configuration phase, presetting the mapping relationship between latency levels and large model call lines; Each large model call line consists of at least one large model inference abstract interface implementation and several tool call nodes; In the Spring Boot environment, for each large model call line, define a latency-sensitive conditional assembly expression for the implementation of the large model inference abstract interface and the external tool call abstract interface. When the conditional assembly expression is true based on the current latency level, resource load and fault status, the corresponding implementation is allowed to be assembled into the container. If the result is false, it remains unassembled. By calculating the expected response time of each candidate large model call line, the candidate large model call line that meets the response threshold and has the smallest expected response time is selected as the actual execution line for the current request.

9. The method for dynamic configuration of large model intelligent agents based on Spring Boot as described in claim 8, characterized in that: The process of writing dialogue records and configuration versions includes extracting dialogue round information and key information summaries from the large model inference results after each request is processed, encapsulating the session identifier, round number, question content, response content and summary content into a dialogue record object, and selecting the target storage medium according to the memory strategy configuration in the currently effective configuration. When the memory policy is configured for short-term memory mode, the conversation record object is written to a cache that supports expiration time control. When the memory strategy is configured to long-term recording mode, the conversation record object is written to persistent storage, and a composite index is built based on the session identifier and timestamp during the writing process. When a new request arrives, the multi-turn dialogue memory storage mechanism queries the most recent dialogue record from the target storage medium based on the session identifier, and truncates and merges the query results according to the maximum context length configured in the configuration, injecting the merged historical content into the template variable used by the prompt template configuration mechanism.

10. A dynamic configuration system for large model agents based on Spring Boot, based on the dynamic configuration method for large model agents based on Spring Boot according to any one of claims 1 to 9, characterized in that: It includes an interface construction module, a configuration generation module, a target selection module, a call chain generation module, an inference execution module, and a persistence module; The interface building module is used to build a multi-dimensional abstract interface layer in Spring Boot, encapsulate large model reasoning, knowledge retrieval and external tool calling capabilities, and assign dimension identifiers to each abstract interface; The configuration generation module is used to read the large model, knowledge base, prompt template and memory strategy field from the configuration source based on the configuration attribute binding rules and generate a configuration description object. The target selection module is used to select the effective configuration from the set of configuration description objects according to the configuration parsing rules and the session context, and to determine the target large model and the target knowledge base. The call chain generation module is used to generate prompt text adapted to the current session using the prompt template configuration mechanism and tool call chain generation rules, and to construct an external tool call sequence; The inference execution module is used to select large model call lines and assemble the corresponding implementations to perform inference based on latency-sensitive conditions and dynamic routing strategies. The persistence module is used to write dialogue records and configuration versions through a multi-turn dialogue memory storage mechanism and configuration change persistence rules.

Images