Isomer generation model cluster scheduling apparatus and method, and storage medium

Abstract

The present disclosure relates to the field of generative artificial intelligence computing and scheduling, and provides a heterogeneous generative model cluster scheduling device, method and storage medium. The device comprises a processor which acquires instruction data, determines a target generative model and its tool information capacity; determines a task tool according to the instruction data, and determines a candidate tool set comprising the task tool and its preceding tools according to tool association information; determines a target tool set under the tool information capacity constraint and generates tool context data containing tool description data; provides the instruction data and the tool context data to the target generative model, so that the target generative model calls the tools in the target tool set in a dependency order and generates response data. The present disclosure can control the size of the tool context data while retaining the preceding tool information related to the task tool call, reduce the possibility of interruption of the tool call chain due to the absence of the preceding tool information, and improve the continuity and execution reliability of the multi-tool call process.

Description

Technical Field

[0001] This disclosure relates to the field of generative artificial intelligence computing and scheduling technology, and in particular to a heterogeneous generative model cluster scheduling device, method and storage medium. Background Technology

[0002] Generative AI services can deploy generative models on various computing resources and invoke external tools such as search, database queries, code execution, and file processing when handling user requests. As the number of tools increases, a single request may involve multiple tools with data transfer relationships. Existing scheduling methods typically select tools based on the direct correlation between the request and the tool description. However, when the number of tools is large or there are continuous calling relationships between tools, the tool information provided to the generative model may consume significant context resources. Direct filtering may miss information dependent on subsequent calls, thus affecting the continuity and reliability of the multi-tool calling process. Therefore, how to balance the scale of tool information and tool calling relationships when the generative model's capacity to receive tool information is limited is a technical problem that needs to be solved in the scheduling of generative AI services. Summary of the Invention

[0003] This disclosure provides a heterogeneous generative model cluster scheduling device. The device includes a processor that acquires instruction data and determines the target generative model for processing the instruction data, as well as the tool information capacity corresponding to the target generative model. After determining the task tool involved in the current instruction data, the processor combines tool association information to determine a candidate tool set. This candidate tool set includes the task tool and its prerequisite tools. Under the constraint of the tool information capacity of the target generative model, the processor further determines a target tool set based on the candidate tool set and generates tool context data based on the target tool set. The target tool set includes at least the task tool and its necessary prerequisite tools, where the output of the necessary prerequisite tools is a required input for the corresponding subsequent tool. The tool context data includes tool description data corresponding to the tools in the target tool set. Subsequently, the processor provides the instruction data and tool context data to the target generative model. The target generative model, based on the dependencies represented by the tool association information, calls the tools in the target tool set in the corresponding dependency order and generates response data based on the corresponding processing procedure.

[0004] This disclosure also provides a method for scheduling a heterogeneous generative model cluster. The method acquires instruction data and determines the target generative model for processing the instruction data, as well as the tool information capacity corresponding to the target generative model. After determining the task tool involved in the current instruction data, a candidate tool set is determined based on tool association information. This candidate tool set includes the task tool and its predecessor tools that have a prerequisite relationship with the task tool. Under the constraint of the tool information capacity of the target generative model, a target tool set is further determined based on the candidate tool set, and tool context data is generated based on the target tool set. The target tool set includes at least the task tool and its necessary predecessor tools, where the output of the necessary predecessor tools is a required input for the corresponding successor tool. The tool context data contains tool description data corresponding to the tools in the target tool set. Subsequently, the instruction data and tool context data are provided to the target generative model, enabling the target generative model to call the tools in the target tool set according to the dependency relationships represented by the tool association information, in the corresponding dependency order, and generate response data based on the corresponding processing procedure.

[0005] This disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the above-described heterogeneous generation model cluster scheduling method.

[0006] This disclosure provides a heterogeneous generative model cluster scheduling device, method, and storage medium. By constraining the tool information capacity of the target generative model, and combining tool association information to determine the target tool set corresponding to the current instruction data, tool context data is generated based on the target tool set. This ensures that while the tool description data provided to the target generative model is limited by the tool information capacity, it retains the preceding tool information related to task tool calls. Therefore, by controlling the size of the tool context data and reducing the resource consumption of tool information on the generative model context, the possibility of tool call chain interruption due to missing preceding tool information can be reduced. This provides the corresponding data foundation for the target generative model to execute multiple tool calls in dependency order, thereby improving the continuity and execution reliability of the multi-tool call process. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the heterogeneous generative model cluster scheduling system provided in this disclosure.

[0008] Figure 2 This is a flowchart illustrating the heterogeneous generation model cluster scheduling method provided in this disclosure.

[0009] Figure 3 A schematic diagram illustrating the process of determining the target generation model provided in this disclosure.

[0010] Figure 4A schematic diagram illustrating the calculation process for the requested uncertainty value provided in this disclosure.

[0011] Figure 5 This is a schematic diagram illustrating the process for determining the processing capacity requirements provided in this disclosure.

[0012] Figure 6 This is a schematic diagram of the model selection process based on historical interaction data provided in this disclosure.

[0013] Figure 7 This is a schematic diagram illustrating the process of generating tool-related information provided in this disclosure.

[0014] Figure 8 This is a schematic diagram of the pruning process for the candidate toolkit provided in this disclosure.

[0015] Figure 9 This is a schematic diagram of the conflict resolution process for the multi-source tools provided in this disclosure. Detailed Implementation

[0016] The technical solutions in this disclosure will now be clearly and completely described with reference to the accompanying drawings.

[0017] Figure 1 This is a schematic diagram of the heterogeneous generative model cluster scheduling system provided in this disclosure. Figure 1 As shown, the heterogeneous generation model cluster scheduling system 100 includes a user terminal 110, a heterogeneous generation model cluster scheduling device 120, a heterogeneous generation model cluster 130, and a tool service environment 140. The user terminal 110 is used to initiate pending requests. The heterogeneous generation model cluster 130 includes lightweight generation models on the edge, medium-sized generation models in the cloud, and other generation models with different processing capabilities. Each generation model has different characteristics in terms of computational resource consumption, response speed, and processing accuracy. The tool service environment 140 provides search, database query, code execution, file processing, and other tool services that can be called through interfaces.

[0018] The heterogeneous generative model cluster scheduling device 120 includes a processor 122. The processor 122 receives requests from the user terminal 110 and interacts with the heterogeneous generative model cluster 130 and the tool service environment 140. When performing scheduling processing, the processor 122 can provide instruction data and tool context data to the generative model and obtain the execution results generated by the tool invocation.

[0019] The heterogeneous generative model cluster scheduling system 100 is equipped with a service access node 150 and a data storage environment 160. The service access node 150 is located between the user terminal 110 and the heterogeneous generative model cluster scheduling device 120, and is used to receive requests and send back responses; the data storage environment 160 is connected to the heterogeneous generative model cluster scheduling device 120, and is used to store model service information, tool basic information, historical interaction data, and tool association information.

[0020] The generative models in the heterogeneous generative model cluster 130 can handle different types of requests based on their deployment location and processing capabilities. For example, lightweight generative models on the edge can be used for request prediction or partial instruction processing, while medium-sized generative models and generative models with higher processing capabilities in the cloud can handle requests requiring more inference resources. Different generative models can accept different context sizes. After determining the generative model to handle the current request, the heterogeneous generative model cluster scheduling device 120 can also obtain the tool information capacity corresponding to that generative model and use that tool information capacity to constrain the organization of subsequent tool description data.

[0021] Tools in the tool service environment 140 can register their information according to the Model Context Protocol (MCP) or other methods that support structured interface descriptions. Tool information may include tool name, function description, input structure, output structure, required inputs, optional inputs, permission scope, provider, interface version, call success rate, timeout rate, error rate, and data expiration policy. The processor 122 can identify the tools related to the request based on the above information and establish data relationships between the tools.

[0022] Before the system is put into operation, the connection of generation models and tool services can be completed, and model service information and tool basic information can be established for processor 122 to read. When tool interfaces are added, deleted, or their versions change, the tool information can be updated accordingly; user feedback and tool operation records generated during request processing can also be used to update historical interaction data and tool relationship data. In the above operating environment, after user requests are processed by the heterogeneous generation model cluster scheduling device 120, subsequent processing can be completed by the corresponding generation models and tool services.

[0023] Figure 2 This is a flowchart illustrating the heterogeneous generative model cluster scheduling method provided in this disclosure. This method can be executed by the processor 122 of the heterogeneous generative model cluster scheduling device 120. Specifically, after receiving a request, the processor 122 performs the following steps: In step S210, processor 122 determines the target generative model and its corresponding tool information capacity. Specifically, processor 122 acquires instruction data and determines the target generative model used to process the instruction data and the tool information capacity of the target generative model. Instruction data refers to the data in the current request that needs to be understood, reasoned about, or generated by the generative model; the target generative model is the generative model in the heterogeneous generative model cluster 130 that undertakes the processing of the current request. Tool information capacity represents the upper limit of the amount of data that can be allocated to tool description data in the current request processing. When using tokens as the unit of measurement for model input, tool information capacity can be represented by the token budget allocated to tool description data in the context window of the target generative model.

[0024] The tool information capacity corresponds to the target generation model, but it is not equivalent to the complete context window of the target generation model. The input to the target generation model, in addition to tool description data, can also include instruction data and context information generated during processing. Therefore, the tool information capacity corresponds to the capacity boundary allocated to tool description data in the input resources of the target generation model. Let the tool information capacity be B, and the amount of tool description data corresponding to tool t be d(t). We can determine whether a toolset satisfies the tool information capacity constraint based on the total amount of d(t) corresponding to each tool in the toolset.

[0025] In step S220, processor 122 determines a candidate tool set. Specifically, processor 122 determines the task tool based on instruction data and the candidate tool set based on tool association information. The candidate tool set includes the task tool and its predecessor tools. The task tool is a tool directly related to the processing task corresponding to the current instruction and can be determined based on the semantic matching relationship between the instruction data and the tool's functional description. Tool association information indicates the data transfer relationship and direction between tools. Processor 122 queries forward from the task tool along the tool association information to obtain the predecessor tools that provide input data for the task tool or intermediate tool.

[0026] Let C be the set of task tools determined based on the instruction data. task The set of prerequisite tools obtained by recursively querying based on tool association information is Pre(C). task If the candidate tool set C is... cand It can be represented as: C cand = C task ∪ Pre(C task ).

[0027] When a preceding tool depends on earlier tool outputs, the processor 122 continues to search forward along the tool association information until the preceding relationships required by the current tool call chain have been completed. Therefore, the candidate tool set includes task tools directly related to instruction data, as well as preceding tools that need to be called in advance when executing task tools or intermediate tools.

[0028] In step S230, processor 122 determines the target tool set and generates tool context data. Specifically, under the constraint of tool information capacity, processor 122 determines the target tool set based on the candidate tool set and generates tool context data. The target tool set includes at least the task tool and its necessary prerequisite tools. Necessary prerequisite tools are those whose output data constitutes the required input data for their corresponding successor tools. The tool context data contains tool description data corresponding to the tools in the target tool set. Tool description data may include tool name, function description, input fields, output fields, and call interface information. The tool context data may also include the relationships between tools in the target tool set, used to represent tools with sequential relationships and their calling directions. The amount of tool description data corresponding to the candidate tool set can be expressed as: ; Where t represents the candidate tool set C cand In the context of tools, d(t) represents the amount of tool description data corresponding to tool t.

[0029] When the amount of tool description data corresponding to the candidate tool set does not exceed the tool information capacity, the candidate tool set can be determined as the target tool set; when the amount of tool description data corresponding to the candidate tool set exceeds the tool information capacity, the candidate tool set is processed according to the capacity status to determine the target tool set. The target tool set represents the range of tools provided to the target generative model in the current request, while the tool context data is the data generated based on the tools in the target tool set and provided to the target generative model.

[0030] In step S240, processor 122 instructs the target generation model to invoke tools and generate response data. Specifically, processor 122 provides instruction data and tool context data to the target generation model, enabling the model to invoke tools in the target tool set according to the dependency order indicated by the tool association information and generate response data. For tools with prerequisite relationships, the output of the prerequisite tool can be used as the input of the subsequent tool. Let the target tool set be C. target The tool association subgraph corresponding to the target tool set is G[C target ], can be done by Order=TopologicalSort(G[C targetThe execution sequence that satisfies dependencies is obtained; tools that do not have dependencies on each other are not restricted by their order. Here, TopologicalSort(·) represents performing a topological sort on the corresponding tool association subgraph, and Order represents the tool execution sequence obtained from the topological sort. After obtaining the execution results of the corresponding tools, the target generation model generates response data based on the instruction data and the tool execution results.

[0031] For other requests that do not involve tool calls, the target generation model can also directly generate a response based on the instruction data.

[0032] The tool information capacity limits the size of the tool description data provided to the target generative model, while the tool association information is used to determine the prerequisite relationships between task tools and the calling order between tools. Both contribute to the construction of the tool context data, ensuring that the tool information provided to the target generative model corresponds to the current tool call chain and is constrained by the tool information capacity of the target generative model. This improves the adaptability of task processing, the continuity of the tool calling process, and the reliability of execution.

[0033] Figure 3 This is a schematic diagram illustrating the process of determining the target generation model provided in this disclosure. Figure 3 As shown, the process of determining the target generation model includes the following steps: In step S310, the processor 122 obtains the predicted probability distribution output by the predictive generation model. Specifically, the processor 122 calls the predictive generation model to process the instruction data and obtains the predicted probability distribution output by the predictive generation model. The predictive generation model can be an edge-side lightweight generation model. This generation model performs forward inference on the instruction data and outputs predicted data used to form the predicted probability distribution.

[0034] In step S320, processor 122 determines the request uncertainty value and the processing capacity requirement value. In this step, the prediction probability distribution reflects the degree of concentration of the predictions of the predictive generation model for candidate tokens at the corresponding generation position. When the prediction probability distribution is concentrated on a small number of candidate tokens, the predictive generation model's judgment on the generation direction is relatively concentrated; when the prediction probability distribution is more dispersed, the distinction between different generation directions is relatively reduced. Processor 122 determines the request uncertainty value based on the prediction probability distribution and determines the processing capacity requirement value based on the request uncertainty value. The request uncertainty value characterizes the degree of prediction uncertainty of the predictive generation model when processing the current instruction, and the processing capacity requirement value characterizes the current request's demand on the generation model's processing capacity.

[0035] In step S330, processor 122 determines the target generation model. The predicted generation model and the target generation model can be the same; when the predicted generation model can handle the processing requirements of the current request, it can continue to be processed by that model; when the processing capacity requirements fall within the processing range of other models, a corresponding generation model can be selected from the heterogeneous generation model cluster 130 to undertake the subsequent processing.

[0036] The predicted probability distribution output by the predictive generation model is derived from the actual processing results of the current instruction data. Based on this predicted probability distribution, a request uncertainty value is formed, and the processing capacity requirement value is determined accordingly, so that the predicted state of the current request can serve as the basis for selecting the target generation model.

[0037] Figure 4 This is a schematic diagram illustrating the calculation process for the requested uncertainty value provided in this disclosure. For example... Figure 4 As shown, the process for calculating the uncertainty value of this request includes the following steps: In step S410, the processor 122 obtains the predicted score corresponding to the generation position. In this disclosure, the processor 122 obtains the predicted scores output by the predictive generation model for the instruction data at the first k generation positions, where k is a positive integer greater than 1. When the predictive generation model is an edge-side lightweight generation model, its output unnormalized predicted score (Logits) can be used as the predicted score. The predicted score at the j-th generation position is denoted as z. j (v), where v represents the candidate token. k can be 4 to 16, for example, 10. Furthermore, for the j-th generation position, the predicted score output by the predictive generation model can be represented as the predicted score vector z. j ={z j (v) | v∈V}, where V represents the set of candidate tokens.

[0038] In step S420, processor 122 determines the predicted probability distribution of the generated positions. In this disclosure, processor 122 determines the predicted probability distribution corresponding to the first k generated positions based on the predicted scores. For the j-th generated position, the Softmax function can be used to convert Logits into predicted probabilities: ; Where v′ represents the candidate tokens traversed during summation, and p j (v) represents the predicted probability of generating candidate token v at the j-th generation location.

[0039] In step S430, processor 122 determines the predicted entropy value of the generation position. In this disclosure, processor 122 determines the predicted entropy values ​​corresponding to the first k generation positions based on the predicted probability distribution. For the complete candidate token set, the predicted entropy H of the j-th generation position... j It can be represented as: .

[0040] When the candidate token set is large, the M candidate tokens with the highest predicted probabilities (referred to as Top-M) can be selected for calculation, where M can be between 10 and 100. Let the Top-M candidate set for the j-th generation position be... Renormalize the Top-M probabilities: ; Then, the normalized prediction entropy H in the range of 0 to 1 is obtained according to the following formula. j,norm : ; Here, log M is used to normalize the entropy value to a uniform numerical range.

[0041] In step S440, processor 122 determines the request uncertainty value. Specifically, processor 122 determines the request uncertainty value based on the prediction entropy values ​​corresponding to the first k generation positions. One approach is to calculate the arithmetic mean of the first k normalized prediction entropies: ; Among them, H avg This represents the sum of request uncertainties corresponding to the first k generation locations. This represents the normalized prediction entropy corresponding to the j-th generation position.

[0042] Another approach is to use a position decay weight ρ j Calculate the weighted average: ; ; Where η represents the position decay coefficient, and r represents the generation position index obtained by summing the first k generation positions. When the tokens generated in the early stages have a strong representation of the task type, task intent, and reasoning direction, the weight of the earlier generation positions in the request uncertainty value can be increased through position decay weighting. Arithmetic mean and position decay weighted average are two alternative calculation methods.

[0043] The prediction entropy of the first k generation positions collectively reflects the prediction state of the prediction generation model in the early stages of generation. When using Top-M candidate tokens, the number of candidate tokens participating in the entropy summation can be reduced, while retaining the prediction information corresponding to candidate tokens with higher probabilities.

[0044] Figure 5 This is a schematic diagram illustrating the process for determining the processing capacity requirements provided in this disclosure. Figure 5 As shown, the process for determining this processing capacity requirement includes the following steps: In step S510, processor 122 quantizes the length of instruction data, tool invocation requirements, and context dependency. Specifically, after obtaining the request uncertainty value, processor 122 quantifies the length of instruction data, the tool invocation requirements represented by the instruction data, and the dependency of the instruction data on context information, respectively, to obtain instruction length feature values, tool invocation requirement feature values, and context dependency values.

[0045] Instruction length L can be represented by the number of characters, words, or tokens. To reduce the numerical impact of excessively long instructions in the overall evaluation, the instruction length can be logarithmically normalized to obtain the instruction length characteristic value L. norm : ; Among them, L max The maximum preset length is indicated, and min(·) represents taking the smaller value from the corresponding parameters. The tool call requirement characteristic value can be determined based on whether the command involves external search, database query, code execution, file processing, API call, toolchain call, or the need for real-time data, user private data, or third-party system data. The context dependency value can be determined based on pronoun references in the command, multi-turn context dependencies, omitted expressions, missing parameters, and dependencies on historical files, historical sessions, or user profiles. Both the tool call requirement characteristic value and the context dependency value can be normalized to the [0,1] interval.

[0046] In step S520, processor 122 determines the processing capacity requirement value. Specifically, processor 122 determines the processing capacity requirement value based on the request uncertainty value, instruction length characteristic value, tool call requirement characteristic value, and context dependency value. When the request uncertainty value adopts H... avg The instruction length characteristic value is represented by L. norm When the tool invocation requirement characteristic value is represented by ToolIntent and the context dependency value is represented by ContextDependence, the processing capacity requirement value S can be expressed as: S = αH avg + βL norm + δToolIntent + θContextDependence; α + β + δ + θ = 1.

[0047] In simplified computing scenarios, the processing capacity requirement can also be calculated using two factors: request uncertainty and instruction length. S = αH avg + βL norm ; α + β = 1; Wherein, α, β, δ and θ represent the configurable weights corresponding to the processing capacity requirement value determined by the request uncertainty value, instruction length feature value, tool call requirement feature value and context dependency value, respectively.

[0048] The aforementioned weights and related parameters can be obtained through offline calibration. During calibration, a sample set D={(I i ,y i , q i , c i , l i , f i )}, where I i Indicates historical commands, y i Represents the optimal route label, q i Indicates the output quality score, c i Indicates the actual call cost, l i f represents the actual response delay. i This indicates a failure flag or a negative user feedback flag. Samples can come from publicly available evaluation sets, internal enterprise historical request sets, manually constructed complex task sets, and real user feedback data after the model is deployed. After offline replay of different parameter combinations, the corresponding routing strategy can be evaluated using the following formula: J = Quality - μCost - νLatency - ξFailurePenalty; Where Quality represents the output quality score, Cost represents the inference cost, Latency represents the response latency, and FailurePenalty represents the penalty for incorrect routing, failed tool calls, or negative user feedback; μ, ν, and ξ represent the business-configurable weights of cost, response latency, and failure penalty in the objective function, respectively.

[0049] Parameter optimization can employ grid search, Bayesian optimization, genetic algorithms, reinforcement learning, or periodic recalibration based on online feedback. Different business domains can also maintain separate parameter sets Θd={k}. d M d , α d , β d , δ d , θ d ,T base,d , γ d}; d represents the business domain, k d M d α d β d δ d θ d T base,d and γ dThese represent k, M, α, β, δ, θ, and the basic threshold T corresponding to the business domain d, respectively. base And the adjustment coefficient γ. When the model version, tool library structure, user group, or online feedback status changes, the parameters can be recalibrated to match the corresponding parameters with the quality, cost, response latency, and failure risk of the business scenario.

[0050] During offline calibration, the selection of parameters to be calibrated can also be expressed as a problem of maximizing the objective function J: ; in, , , , , and are α, β, δ, θ, T respectively base The calibration results for γ, and arg max represent the parameter combination selected from the given optional parameter combinations that makes the objective function J obtain a larger value.

[0051] Request uncertainty, command length, tool invocation requirements, and contextual dependencies characterize the processing demands of a current request from different perspectives. Using these characteristics together to determine processing capacity requirements can reduce the bias caused by solely relying on fixed keywords or text lengths for request judgment and provide a quantitative basis for selecting the processing capacity level of the generative model. The predictive generative model can be deployed on the client side and output corresponding prediction data before the target generative model processes the current request. Therefore, requests with lower processing capacity requirements can be handled by client-side or medium-sized generative models; requests with higher processing capacity requirements are then handled by generative models at the corresponding processing capacity level, ensuring that the computational resource consumption of the generative model corresponds to the processing demand of the current request.

[0052] Figure 6 This is a schematic diagram of the model selection process based on historical interaction data provided in this disclosure. For example... Figure 6 As shown, the model selection process includes the following steps: In step S610, the processor 122 acquires historical interaction data and determines the model selection threshold. Specifically, after obtaining the processing capacity requirement value, the processor 122 acquires historical interaction data corresponding to the account identifier associated with the instruction data, and determines the model selection threshold based on the historical interaction data, so that the model selection boundary can be adjusted in combination with the historical interaction status of the corresponding account.

[0053] Historical interaction data can include the proportion of complex tasks (R). complex The proportion of high-processing-capacity models selected (R) high Low processing capacity model retry rate R retry Professional field concentration Rdomain High-quality feedback percentage R accept and cost sensitivity R cost The proportion of complex tasks (R) complex The proportion of tasks such as multi-step reasoning, code generation and debugging, legal text writing, patent drafting, financial analysis, medical analysis, long document summarization, and toolchain invocation in historical requests is determined. The proportion of high-processing-capacity models selected (R) is also considered. high The proportion of retry attempts for the low-processing-capacity model is determined based on the percentage of accounts actively selecting high-processing-capacity models and accepting the system's recommendation of high-processing-capacity models. retry The concentration of expertise (R) is determined based on the proportion of regeneration, follow-up questions, requests for increased detail, rigor, or accuracy, and the switching to a high-capacity model after the output of the edge-based or medium-scale generation model. domain The percentage of high-quality feedback (R) is determined based on the concentration of requests in specialized tasks such as patents, legal matters, R&D, and finance within a preset timeframe. accept The cost sensitivity R is determined based on the frequency of adoption, replication, export, positive feedback, and follow-up questions of the high-processing-capacity model's output. As the follow-up question frequency decreases, the proportion of high-quality feedback increases accordingly. cost The expert preference score for the current period is determined by the proportion of users choosing low-cost or high-speed processing methods, refusing to use high-capacity models, and providing negative feedback on cost-related suggestions. After normalizing these features to a range of 0 to 1, the expert preference score can be calculated. ; .

[0054] Where σ(·) represents the Sigmoid normalization function, x represents the function input, e represents the natural constant, and w0 to w6 represent configurable weight coefficients.

[0055] To reduce the impact of a single interaction on account status, an exponential moving average can be used to update expert preference scores: ; Where λ represents the update step size of the exponential moving average, which is between 0 and 1, and can take values ​​from 0.05 to 0.3, P expert (u,t) represents the expert preference score for user u after the update in the current period t. This represents the provisional expert preference score obtained based on historical behavioral characteristics for the current period. The expert preference score can be updated after a single interaction, or after a cumulative N interactions; it can also be updated in batches daily, weekly, or monthly, or triggered by events such as model switching, negative feedback, or multiple retries. When historical data is insufficient, an initial expert preference score can be set, for example, 0.3 for ordinary accounts, 0.6 for professional accounts, and 0.7 for enterprise administrators or expert role accounts; as historical interaction data increases, the expert preference score is updated based on actual interaction data. When historical interaction data is insufficient, the initial expert preference score can be set according to account type, or determined based on the account's registered role, organization, service package type, initial task type, or system default configuration. As historical interaction data increases, an expert preference score calculated and updated based on actual interaction data can be gradually adopted.

[0056] Model selection threshold T u It can be determined based on expert preference ratings, cost sensitivity, and delay sensitivity: ; Among them, P expert Expert preference rating; P cost Select the cost sensitivity score to use in the threshold calculation for the model; P latency Select a latency sensitivity score for the threshold calculation in the model. P cost and P latency The range of values ​​for P can all be normalized to [0,1]; cost The larger the value, the more sensitive the corresponding account is to the cost of model invocation; P latency The larger the value, the more sensitive the corresponding account is to response latency. e γ c and γ l P expert P cost and P latency The corresponding adjustment coefficient, T min and T max These represent the lower and upper limits of the threshold, respectively. `clip(·)` restricts the calculation result to T. min To T max Functions within the specified range. The cost sensitivity score is determined based on the cost sensitivity mentioned above, while the delay sensitivity score is determined based on the historical interaction data of the corresponding account.

[0057] Cost sensitivity R cost Cost sensitivity score P cost This corresponds to a monotonically non-decreasing mapping relationship in R. cost Given that the normalization is already to [0,1], we can let: .

[0058] In another implementation, a normalized linear mapping can also be used: ; Among them, a c and b c For mapping parameters determined through historical samples or offline calibration, clip(·) restricts the calculation results to the range [0,1]. The more pronounced the cost-sensitive tendency exhibited in account history interactions, the higher the R... cost The larger the value, the higher the corresponding value of P. cost It is also larger. R cost It can also be used as a historical behavioral characteristic in the calculation of expert preference scores, P cost This directly participates in the calculation of the model selection threshold, and both make the triggering conditions for generating higher-cost models more stringent when the account is more sensitive to costs.

[0059] Delay sensitivity score P latency The determination is based on latency-sensitive behaviors in the historical interaction data of the corresponding account. Historical interaction data may include at least one of the following: history of the account actively selecting fast processing mode, low latency mode, or low latency generation model; history of the account canceling the request after the response wait time reaches a preset wait condition; and history of the account re-initiating the request or switching to another low latency generation model after the response wait time reaches the preset wait condition.

[0060] Let N be the number of valid interactions within a statistical period, and let N be the number of times an account actively selects either the fast processing mode or the low-latency generation model. fast The number of interactions that reach the preset waiting condition in terms of response time is N. slow The number of account cancellation requests is N. cancel The number of times the account re-initiates a request or switches to another low-latency generation model is N. switch Then we can obtain the following respectively: ; ; ; Here, ε0 is a very small positive number set to prevent the denominator from being zero. In one implementation, the delay sensitivity score can be calculated as follows: ; ; Among them, w l1 w l2 and w l3These represent the configurable weights for quick selection, cancel after waiting, and switch after waiting, respectively. These weights can be calibrated offline using historical samples or dynamically adjusted based on actual operational data. When historical interaction data is insufficient, P can be used. cost and P latency Set system preset values ​​and update them gradually as historical interaction data increases. cost and P latency It reflects the cost and latency preferences of the corresponding account based on historical interaction behavior, rather than the actual call cost or actual response time incurred by the current single request.

[0061] The model selection threshold can also be determined based on expert preference scores, using the following formula: .

[0062] When the expert preference score increases, the model selection threshold decreases accordingly; when cost sensitivity or latency sensitivity increases, the model selection threshold increases accordingly. The processor 122 also acquires model capability data for candidate generative models, including the processing capability level of the candidate generative models.

[0063] In step S620, the processor 122 determines the target generation model. Specifically, when the processing capacity requirement value is greater than the model selection threshold, the target generation model is determined from candidate generation models whose processing capacity level reaches the preset processing capacity level; when the processing capacity requirement value is less than or equal to the model selection threshold, the target generation model is determined from candidate generation models whose processing capacity level is lower than the preset processing capacity level. Different accounts may have different model selection thresholds; therefore, the same processing capacity requirement value may correspond to different model selection results.

[0064] By adjusting the model selection threshold using historical interaction data, the same processing capacity requirement can correspond to different model selection boundaries under different account states, thereby making the processing capacity level of the target generation model correspond to the account's historical preferences for processing quality, cost, and response latency.

[0065] Figure 7 This is a schematic diagram illustrating the process of generating associated information for the tools provided in this disclosure. For example... Figure 7 As shown, the process of generating and applying related information using this tool includes the following steps: In step S710, processor 122 obtains the output fields of the first tool and the required input fields of the second tool. The output fields represent the data items that the first tool can generate after execution, and the required input fields represent the data items that the second tool needs to obtain during execution.

[0066] In step S720, processor 122 determines the dependency matching score between the first tool and the second tool. Specifically, processor 122 determines the dependency matching score between the first tool and the second tool based on the field type matching degree and semantic similarity between the output field and the required input field. The dependency matching score can also be determined by combining the field value constraint matching degree between the output field and the required input field. Field type matching degree reflects whether the output type and input type are consistent or have a convertible relationship, such as the compatibility relationship between string and enum, integer and number, and date string and timestamp. Semantic similarity can be calculated based on the semantic representation formed by field name, field description, tool description, and parameter description. Field value constraint matching degree reflects the compatibility degree between the output field and the required input field in terms of field format, enumerated values, units, numerical range, or regular expression constraints. In addition to field type matching degree, semantic similarity, and field value constraint matching degree, field name format, permission range, and historical call chain can also be used as further matching criteria during field matching. Field names can include field names, aliases, camelCase, underscores, and abbreviations. Permission scope can include the tool's permissions, tenant, account identity, and data access range. The historical call chain reflects successful calls made by the first tool as a precursor to the second tool.

[0067] Let the first tool be a and the second tool be b. Then the dependency matching score between the first tool a and the second tool b can be expressed as: .

[0068] Where SchemaMatch(a,b) represents the field structure matching degree, SemanticSim(a,b) represents the semantic similarity between the field and the tool description, ConstraintMatch(a,b) represents the field value constraint matching degree, λ1 represents the weight corresponding to the field structure matching degree, λ2 represents the weight corresponding to the semantic similarity between the field and the tool description, and λ3 represents the weight corresponding to the field value constraint matching degree.

[0069] In step S730, processor 122 establishes associations between tools. Specifically, when the dependency matching score reaches the dependency generation threshold, processor 122 establishes an association from the first tool to the second tool in the tool association information. This association indicates that the output field of the first tool provides input data for the required input field of the second tool. Let the dependency generation threshold be τ. dep When Match(a,b)≥τ dep A directed association from a to b is established, depending on the generation threshold τ. dep It can be determined through offline calibration.

[0070] In step S740, processor 122 determines the dependency edge weights of the association. Specifically, after establishing the association, the dependency edge weight W of the association can be calculated. dep (a,b). Dependency edge weights can be a combination of field structure matching (SchemaMatch(a,b), semantic similarity between field and tool descriptions (SemanticSim(a,b), field value constraint matching (ConstraintMatch(a,b), historical call chain success rate (HistoricalSuccess(a,b)), authorization compatibility (AuthCompatibility(a,b)), and historical failure rate (FailureRate(a,b)). For example: ; Where λ1 to λ6 are the weights of the corresponding evaluation factors. The dependency matching score Match(a,b) is used to determine whether a directed association is established between the first tool a and the second tool b; the dependency edge weight W dep (a,b) represents the dependency edge weight corresponding to the association relationship from tool a to tool b. It is used to represent the relative strength of the association relationship and can be used as the basis for association updates, loop handling, or relationship priority judgment. The dependency edge weight Wdep(a,b) can be normalized to the interval [0,1] according to the business configuration.

[0071] Based on the interaction between tools, dependencies can be categorized into hard dependencies, soft dependencies, substitution relationships, validation relationships, and conflict relationships. A hard dependency indicates that the output of the preceding tool constitutes the input required for the execution of the subsequent tool; without this output, the subsequent tool cannot execute. A soft dependency indicates that the output of the preceding tool can improve the performance of the subsequent tool, but does not constitute the required input. A substitution relationship indicates that different tools can produce similar outputs, and one tool can be switched to when another is unavailable. A validation relationship indicates that the output of one tool can be used to verify the output of another tool. A conflict relationship indicates that different tools may return similar but inconsistent results, requiring conflict resolution after execution.

[0072] In step S750, processor 122 updates the tool association information. Specifically, the structured information of the tool node may also include the `side_effect` field and the `freshness_policy` field. The `side_effect` field describes whether the tool call may cause external state changes, and the `freshness_policy` field describes the timeliness policy of the tool's output data. When a tool interface is added, deleted, or updated, the tool association information can be incrementally updated. For a new tool, matching the input structure of the new tool with the output structure of an existing tool can generate an association pointing to the new tool; matching the output structure of the new tool with the input structure of an existing tool can generate an association from the new tool to an existing tool. When a tool fails, the weight of the relevant association can be reduced or the tool can be marked as unavailable. For a failed tool t... dead The relevant weights can be calculated as follows: Update, where κ represents the weight decay coefficient, and satisfies 0 ≤ κ < 1, t dead The 'h' represents the failed tool, and 'h' represents another tool that is associated with the failed tool. This indicates the weight of the dependent edges before the update. This represents the updated dependency edge weights. This disclosure can preserve historical relationships for rollback, auditing, and fault analysis.

[0073] After establishing or updating tool association information, processor 122 processes loops in the tool association graph according to the association type and association weight. Specifically, when loops exist in the tool association graph, they can be broken according to the association type and association weight. The association processing priority can be set in the order of hard dependency (hard_dep), verify relationship (verify), soft dependency (soft_dep), and alternative relationship (alternative), with priority given to retaining associations that maintain the required inputs for subsequent tools. When loops still form between necessary prerequisite relationships, related tools can be merged into a composite tool group, parameter confirmation can be triggered, loop-breaking edges can be selected based on historical successful execution paths, or the toolchain can be split into multiple stages for execution.

[0074] Once the tool association information is established based on the tool's input and output fields and their semantic relationships, it can simultaneously carry the necessary preconditions for forming the execution conditions of subsequent tools, as well as other associations for improving execution results, providing alternative outputs, verifying execution results, and triggering conflict handling. This provides an association basis for querying precondition tools, completing candidate tool sets, determining dependency order, and resolving subsequent conflicts.

[0075] Based on the tool association information mentioned above, the preceding tools that form a preceding relationship with the following tools include necessary preceding tools and auxiliary preceding tools. The output of necessary preceding tools is a required input for the corresponding following tool, while the output of auxiliary preceding tools is a non-required input that improves the execution effect of the corresponding following tool. Necessary preceding tools are used to form the execution conditions of the following tools, while auxiliary preceding tools are used to supplement the non-required information of the following tools.

[0076] When determining the candidate tool set containing the task tool and its corresponding prerequisite tools, a prerequisite query can be performed on tools directly related to the instruction data along necessary prerequisite relationships. Specifically, the association relationships that form the execution conditions of the subsequent tools can be considered as necessary prerequisite relationships. Let E be the set of directed association edges corresponding to necessary prerequisite relationships. nec The necessary set of prerequisite tools for tool t can be represented as: ; In the formula t pre Pre indicates the necessary prerequisite tool for tool t. nec (t) represents the set of necessary prerequisite tools for tool t.

[0077] For the necessary prerequisite tools obtained from the query, continue to perform prerequisite queries along the necessary prerequisite relationships until no more necessary prerequisite tools are obtained, thus forming the task tool set C. task The corresponding necessary preceding closure C nec C nec Indicates from the task tool set C task Starting from this point, recursively query the set of necessary prerequisite tools obtained along the necessary prerequisite relationships. C can be... task With C nec The union of these sets is used as the tool set after dependency completion, and the candidate tool set is determined accordingly.

[0078] In addition to completing the necessary prerequisite tools, this disclosure can also recursively query the set of auxiliary prerequisite tools C along the auxiliary prerequisite relationships based on task relevance, historical call success rate, and tool information capacity. aux According to the task tool set C task C, the necessary set of prerequisite tools nec With the set of auxiliary pre-tools C aux Determine the candidate tool set, i.e., candidate tool set C. cand It can be represented as: .

[0079] Using the above method, tool association information can be established based on the structural matching and semantic relationship between the input and output fields of the tools, so that the data dependency relationship between tools can be represented by a directed association relationship. On this basis, by recursively querying and supplementing the necessary prerequisite tools along the necessary prerequisite relationships, the candidate tool set can be made to include the prerequisite tools required to maintain the execution of the subsequent tools. This reduces the situation where the subsequent tools lack required input due to the lack of necessary prerequisite tools, and provides an association basis for subsequent calling of tools according to the dependency order and processing of the candidate tool set under the constraint of tool information capacity.

[0080] Figure 8 This is a schematic diagram of the candidate tool set pruning process provided in this disclosure. When the sum of the tool description data in the candidate tool set exceeds the tool information capacity, the processor 122 prunes the candidate tool set to obtain a pruned tool set. The pruning process includes retaining the task tools and necessary prerequisite tools, and removing at least one auxiliary prerequisite tool from the candidate tool set. Figure 8 As shown, when the sum of the tool description data in the candidate tool set exceeds the tool information capacity, the pruning process for the candidate tool set includes the following steps: In step S810, processor 122 determines the set of critical tools that need to be retained during the pruning process. To facilitate the description of subsequent pruning constraints, the task tools and necessary prerequisite tools that need to be retained in the current pruning process can be collectively referred to as critical tools, and the task tool set C is defined as follows: task With the necessary set of prerequisite tools C nec The union of the key tools set C is denoted as C. critical ,Right now: ; For the key toolset C critical The tool t in the middle can set the key tool tag C. ritical (t)=1, to indicate that tool t does not participate in deletion during the current pruning process.

[0081] Let the tool information capacity be B. In the key tool set C... critical When the corresponding tool description data size is no greater than the tool information capacity B, the pruning tool set C pruned Includes at least the key toolset C critical Furthermore, the sum of the tool description data corresponding to the pruning tool set is no greater than the tool information capacity B, that is: ; ; Where d(t) represents the amount of tool description data corresponding to tool t.

[0082] In step S820, processor 122 determines the retention order of auxiliary pre-tools. During pruning, the retention order of auxiliary pre-tools can be determined by combining their semantic relevance to instruction data, centrality in tool association information, historical call success rate, output data timeliness, data source authority, tool description data volume, and historical failure rate. Based on this, processor 122 calculates the tool retention benefit Utility(t): .

[0083] Where t represents the auxiliary pre-tool to be evaluated; I represents the current instruction data; Rel(t,I) represents the semantic relevance between tool t and the current instruction data; DepCentrality(t) represents the centrality or dependence of tool t in the tool association information; SuccessRate(t) represents the historical call success rate of tool t; Freshness(t) represents the timeliness of the output data of tool t; Authority(t) represents the authority of the data source corresponding to tool t; TokenCost(t) represents the amount of tool description data of tool t; FailureRate(t) represents the historical failure rate of tool t; and a to g represent the configurable weights corresponding to the evaluation factors. Auxiliary pre-tools with lower retention benefits can be preferentially removed from the current tool context. When determining the retention order of auxiliary pre-tools, the tool's timeout rate, permission availability status, and interface version status can also be considered. Tools with high timeout rates, unavailable permissions, or interface versions that do not meet the current call requirements can have their retention order reduced.

[0084] In step S830, processor 122 determines the target set of auxiliary pre-tools. Specifically, after pruning, the auxiliary pre-tools set C is selected based on tool retention benefits. aux The set of target-aid pre-tools retained in the middle is Then the set of target-aid pre-tools is It can be represented as: ; The sum of the tool description data in the key toolset and the target-supporting pre-toolset shall not exceed the tool information capacity B. This constraint can be expressed as: ; The final set of pruning tools can be represented as: .

[0085] For tools with a large amount of tool description data, the natural language function description of the tool can be compressed first, retaining the tool name, required input fields, output fields, and tool association information. Then, based on the amount of compressed tool description data, it can be determined whether the corresponding tool needs to be removed from the current tool context. After reducing the auxiliary pre-tools or compressing the tool description data, the processor 122 recalculates the amount of tool description data corresponding to the pruned tool set.

[0086] When the tool information capacity is sufficient, information corresponding to auxiliary relationships, verification relationships, or alternative tools can be added to the candidate context based on task relevance and historical success rate. When the tool information capacity decreases, auxiliary pre-tools with lower relevance can be reduced, verification tools can be adjusted to post-execution verification, or high-scoring alternative tools can be retained and other alternative tools can be used as runtime backups. This process ensures that the order of tool information reduction corresponds to the tool's role in the current call chain.

[0087] Step S840: Determine whether the pruning tool set satisfies the tool information capacity constraint, that is, determine whether the sum of the tool description data of the pruning tool set is less than or equal to the tool information capacity; if yes, proceed to step S850; if no, proceed to step S860.

[0088] In step S850, the processor 122 determines the pruning tool set as the target tool set. Specifically, when the sum of the tool description data of the pruning tool set is less than or equal to the tool information capacity, the pruning tool set is determined as the target tool set. During the pruning process, the task tools and necessary prerequisite tools are retained, so that the target tool set continues to contain the tools required to maintain the execution of subsequent tools.

[0089] In step S860, processor 122 determines the handling method for insufficient capacity. If the handling method is determined to be model switching, step S870 is executed; if the handling method is determined to be tool segmentation, step S880 is executed.

[0090] In step S870, the processor 122 reselects a target generative model that meets the capacity condition. Specifically, after completing the pruning of auxiliary pre-tools, if the pruning tool set consists of task tools and necessary pre-tools, further tool deletion may cause the post-tools to lack the required input. Therefore, when the sum of the tool description data of the pruning tool set is still greater than the tool information capacity, the processor 122 reselects a generative model whose tool information capacity is greater than or equal to the sum of the tool description data of the pruning tool set as the target generative model, and determines the pruning tool set as the target tool set.

[0091] Let the tool information capacity corresponding to the original target generation model be B1, and the pruning tool set be C. pruned And the pruning tool set C prunedThe sum of the corresponding tool description data D(C) pruned Processor 122 searches for the condition B from the heterogeneous generative model cluster 130. i ≥D(C pruned The original target generation model is replaced by a generation model that meets the capacity requirements. This adjustment changes the target generation model that carries the tool context and its tool information capacity, but does not change the tool association information already formed between the task tool and the necessary prerequisite tools.

[0092] After reselecting the target generation model, the pruning toolset will no longer be reduced due to insufficient tool information capacity of the original target generation model. Task tools and necessary prerequisite tools can continue to be used as tools in the target toolset, thus avoiding the need to delete necessary prerequisite tools due to insufficient capacity.

[0093] In step S880, the processor 122 divides the pruning tool set into tool subsets. Specifically, if the tool set still exceeds the tool information capacity after pruning, it can be further segmented while keeping the current target generation model unchanged. When the pruning tool set consists of task tools and necessary prerequisite tools, and the sum of their tool description data still exceeds the tool information capacity, the processor 122 divides the pruning tool set into at least two tool subsets according to the dependency order.

[0094] The division of tool subsets is determined based on the data dependencies between tools and the amount of descriptive data corresponding to each tool. The results generated after the execution of the previous tool subset can be used as the input for the next tool subset, so that the data transmission order indicated by the tool association information continues between different tool subsets.

[0095] Processor 122 generates corresponding tool context data segments based on the tool description data contained in each tool subset, and ensures that the data size of a single tool context data segment is less than or equal to the tool information capacity. Let the i-th tool subset be C. i The corresponding tool context data segment is Context. i Then the data size D of the tool's context data segment is... i )satisfy: ; Subsequently, processor 122 provides each tool context data segment to the target generation model in the order of dependency. After the previous stage completes the tool invocation and obtains the data required by the next stage, it provides the next tool context data segment. When providing tool context data segments in the above order, the amount of tool description data entering the target generation model at each stage satisfies the tool information capacity constraint. Therefore, while keeping the current target generation model unchanged, each stage can also satisfy the tool information capacity constraint.

[0096] In this disclosure, when the capacity is insufficient, the tool description can be compressed, retaining information such as the tool name, required parameters, output fields, and dependencies; it can also request the addition of missing parameters to reduce the number of candidate tools. When the current tool information capacity is insufficient to support continued execution, a corresponding prompt can be returned, indicating the missing necessary context.

[0097] Figure 9 This is a schematic diagram illustrating the process for resolving conflicting results from multiple source tools provided in this disclosure. After the target generation model calls the tools in the order of dependency, different tools may return different results for the same entity, field, fact, or metric. For example... Figure 9 As shown, the conflict resolution process of this multi-source tool includes the following steps: In step S910, processor 122 acquires candidate results generated by at least two tools for the same data field and converts the raw outputs of different tools into a unified structure r. i ={value, source, timestamp, tool_id, field_name, confidence, metadata}, to facilitate comparisons at the same field level. Here, value represents the specific value returned by the tool, source represents the data source, timestamp represents the data time or API response time corresponding to the result, tool_id represents the tool identifier that generated the result, field_name represents the data field being compared in conflict, confidence represents the confidence information returned by the tool itself, and metadata represents additional metadata. i This represents the i-th candidate result.

[0098] In step S920, processor 122 determines whether the differences between candidate results meet the conflict determination criteria. For structured fields, conflict can be determined based on the differences in field values ​​and the tolerance threshold; for the field value v corresponding to the i-th candidate result... i The field value v corresponding to the j-th candidate result j The following relative differences can be adopted: ; in, This indicates a positive number used to prevent the denominator from being zero, and ε represents the conflict determination threshold corresponding to the numerical field.

[0099] For text fields, when When τ is found to be a conflict among candidate results, it can be determined that there is a conflict. text SemanticSim(v) represents the semantic consistency threshold of the text. i ,v j ) represents the text field value v i With vj The semantic similarity between them. For time-sensitive fields, timestamp differences can also be used for judgment. When the difference between candidate results reaches the conflict judgment condition, it can be determined that there is a conflict between the candidate results, and steps S930 to S950 are executed; when the difference between candidate results does not reach the corresponding conflict judgment condition, step S960 is executed.

[0100] In step S930, processor 122 determines the credibility score of the candidate results. Specifically, processor 122 determines the corresponding credibility score based on at least one of the following: data source authority, result timeliness, tool reliability, and result consistency. Data source authority can be configured according to categories such as official interfaces, enterprise internal master data systems, certified partner interfaces, public third-party interfaces, web crawling results, user-uploaded content, and unknown sources. Result timeliness can be achieved using the following time decay function: ; Among them, Freshness(r i ) represents the candidate result r i Timeliness score, T now T represents the current time. ri Represents candidate result r i The corresponding data time or interface response time, τ represents the time decay parameter. τ can be set according to the task type. Tasks with rapidly changing timeframes, such as prices, inventory, weather, news, and schedules, can have a smaller τ setting. Tasks with slower timeframes, such as encyclopedic knowledge, historical data, and static documents, can have a larger τ setting.

[0101] Tool reliability can be calculated based on historical success rate, timeout rate, and error rate. Alternatively, average response time can be used as an evaluation factor for tool reliability, allowing it to participate in the reliability evaluation within the corresponding business scenario. An example calculation relationship is as follows: ; Among them, t i This indicates that candidate results r are generated. i Tools, ToolReliability(t i ) indicates tool t i Reliability, SuccessRate(t) i ) indicates the tool t i Historical success rate, TimeoutRate(t) i ) indicates the tool t i The timeout rate, ErrorRate(t) i ) indicates the tool t i The error rate is denoted by ρ1 and ρ2, which represent the penalty coefficients corresponding to the timeout rate and the error rate, respectively.

[0102] Result consistency can be calculated based on the consistency between a candidate result and results from other independent tools. Data source authority can be set to discrete levels, such as 1.00 for official interfaces, 0.95 for internal enterprise master data systems, 0.85 for certified partner interfaces, 0.70 for publicly available third-party interfaces, 0.50 for web crawling results, 0.40 for user-uploaded content, and 0.20 for unknown sources, and can be adjusted according to business scenarios. One way to calculate result consistency is as follows: ; Where n represents the number of candidate results corresponding to the same data field, l(·) represents the indicator function, and Consistent(r i ,r j ) represents the candidate result r i With r j Does it meet the preset consistency conditions?

[0103] The overall credibility score can be expressed as:

[0104] Among them, s i Represents candidate result r i The corresponding data source, Authority(s) i Freshness(r) represents the authority of the data source. i ) indicates the timeliness of the result, ToolReliability(t i CrossConsistency(r) represents the reliability of the tool. i ) indicates the consistency of the results, SchemaConfidence(r i ) represents the structural verification confidence level, confidence represents the confidence information returned by the tool or interface itself, and ErrorRisk(r i ) represents the error risk, and ω1 to ω7 represent the configurable weights of the corresponding evaluation factors.

[0105] Structure validation can include at least one of the following: field integrity validation, field type validation, enumeration value validation, unit consistency validation, and required field missing validation. Taking field integrity validation as an example, SchemaConfidence(r i The number of fields that pass validation (N) can be used as a reference. valid (r i ) and the number of fields to be validated N required (r i ) Determined, that is: ; Where, Nrequired (r i () is a positive number greater than 0.

[0106] ErrorRisk(r i The result can be obtained by weighting risk factors such as interface errors, insufficient permissions, null values, expired cache, abnormal units, abnormal formats, or truncated returned content. ; Where M represents the number of risk item categories included in the error risk calculation, m represents the risk item category number, and Riskm(r i ) represents the candidate result r i Does the corresponding type m risk exist or what is its corresponding risk level? χ m This represents the risk weight corresponding to the m-th type of risk.

[0107] The confidence field returned by the tool itself can also be used as confidence information on the model or interface side in the scoring.

[0108] The weights of each evaluation factor in the reliability score can be adjusted based on the task type or business domain. For example, in tasks that primarily rely on official data or internal enterprise master data, the weight corresponding to the authority of the data source can be increased; in tasks with high timeliness, such as those related to prices, inventory, weather, news, and schedules, the weight corresponding to the timeliness of the results can be increased. Different business scenarios can therefore adopt scoring weights corresponding to their data reliability requirements.

[0109] The weights of evaluation factors can also be adjusted according to task type or business domain d, as shown below. Where m represents the index of the evaluation factor in the credibility score, ω m This represents the weight of the m-th evaluation factor before adjustment. This indicates the weight adjusted for task type or business domain d. Adjust(·) indicates the processing of adjusting the weight of evaluation factors according to task type or business domain.

[0110] In step S940, processor 122 determines the conflict resolution result based on the credibility score. When the difference between the highest and second-highest score reaches a preset difference threshold Δ, the highest score can be used; when the difference is less than Δ, verification tools or alternative tools can be invoked to obtain additional results, or the numerical results can be weighted and fused based on the score, or multiple candidate results and their source information can be retained. In high-risk scenarios, user confirmation of candidate results can also be requested. The conflict result difference threshold Δ can be determined through offline calibration.

[0111] Specifically, when the sum of the confidence scores of the candidate results is not zero, the numerical results can be weighted and fused according to the following formula: ; Among them, Score(r i ) represents the candidate result r i Credibility rating, value i Represents candidate result r i The corresponding numerical result.

[0112] To avoid negative scores from being included in the weighting, normalized positive weights can also be used: ; Here, max(·,0) represents truncating negative values ​​to 0; when the denominator of the normalized positive weights is equal to 0, the candidate result with the highest confidence score can be used. The numerical results are then weighted and fused after applying the normalized positive weights: .

[0113] For text-based candidate results, the result with the highest credibility score can be used as the primary result, with other highly credible results serving as supplements. When multiple highly credible text results meet preset semantic consistency conditions, they can be merged to form a unified result. If multiple highly credible text results still have substantial disagreements, a structured conflict resolution result can be formed. This conflict resolution result can include the recommendation result `final_answer`, result confidence information, conflicting_sources, and the selection or fusion reason `resolution_reason`. The conflicting_sources can also store the result values, credibility scores, and time information of each source.

[0114] In step S950, processor 122 provides the conflict resolution results to the target generation model to generate response data. The system can also record conflict fields, candidate results, scores, final selection results, user feedback, and subsequent verification results. If subsequent verification indicates that a tool's results are incorrect, the reliability of that tool can be reduced. ; Among them, t i ToolReliability is a tool that indicates the reliability of data to be updated. old (t i The ) indicates the tool's reliability before the feedback update, or ToolReliability. new (t i FeedbackScore(t) represents the reliability of the updated tool. i ) represents the feedback score determined based on subsequent verification results or user feedback, λ r This represents the reliability update step size, and 0 ≤ λ.r ≤1.

[0115] At the same time, according to Update the associated weights of relevant tools to allow user feedback and tool execution results to participate in subsequent scheduling. This indicates the weight of the dependent edges before the feedback update; This indicates the updated dependency edge weights. The associated edge feedback value, obtained based on at least one of the following: the actual execution of tool a and tool b, subsequent verification results, and user feedback that can be linked to the tool's call chain, is used to characterize the degree to which the actual running data supports the validity of the dependency relationship between tool a and tool b. The value range is [range missing]. ; The closer it is to 1, the more the actual results support the existence of a valid dependency relationship between tool a and tool b; The closer the value is to 0, the less the actual execution result supports the dependency. w This indicates that the step size is updated based on the edge weights, and 0 ≤ λ. w ≤1.

[0116] Understandably, FeedbackScore(t) i ) used for evaluation tool t i Its own execution reliability The tool is used to evaluate whether the associated edge between tool a and tool b is supported in actual operation, and the update objects corresponding to the two tools are different.

[0117] Specifically, feedback F can be executed based on the tool. exec (a,b), subsequent verification feedback F verify (a,b) and user feedback F user At least one calculation in (a,b) Tool execution feedback F exec (a, b) represents the degree to which the output of tool a provides valid input to tool b and supports the normal execution of tool b during the actual invocation process. The value range can be normalized to [0, 1]. This feedback can be determined based on at least one of the following: whether tool a successfully outputs the required fields of tool b; whether the type, format, unit, or value constraints between the output fields of tool a and the input fields of tool b meet the actual invocation requirements; whether tool b executes normally based on the output provided by tool a; and whether tool b fails to execute due to missing output from tool a, field mismatch, format error, or unavailable data.

[0118] For example, suppose tool b requires m number of mandatory input parameters provided by the preceding tool. b The actual number of parameters effectively provided by tool a is m. aThe degree to which the parameters are satisfied can be expressed as: ; .

[0119] F param (a,b) can be used as F exec The higher the degree of support that tool a provides for the actual execution of tool b, the better the F value. exec The larger (a, b) is, the better. Subsequent verification feedback F verify (a,b) represents the degree of support of subsequent verification tools, other independent data sources, or subsequent processing results for the execution results generated by the association between tool a and tool b; the higher value is taken when the verification result is consistent with the output of the tool call chain, and the lower value is taken when they are inconsistent, and can be normalized to [0,1] according to the degree of consistency.

[0120] User feedback F user (a, b) represents the user's feedback on the final processing result containing the call relationship from tool a to tool b. A higher value can be assigned when the user explicitly accepts the feedback, provides positive feedback, or does not subsequently correct the result; a lower value can be assigned when the user explicitly rejects the feedback, corrects it, or requests a re-execution. User feedback only participates in the feedback calculation of the associated edge if it can be linked to a call chain containing the relationship between tool a and tool b; if there is no valid user feedback in the current interaction, this item will not participate in this update.

[0121] When multiple valid feedbacks are obtained simultaneously, the feedback value of the associated edge can be calculated using the following formula: ; Where μ1, μ2, and μ3 represent the weights corresponding to tool execution feedback, subsequent verification feedback, and user feedback, respectively, and M1, M2, and M3 represent whether the corresponding feedback data is valid within the current update cycle. For the th i Class feedback data, M i satisfy: ; When there is valid feedback data of type i, M i When M is set to 1, it means there is no valid feedback data of type i. i Set it to 0. Therefore, if a certain type of feedback data is missing, that type of feedback will not participate in this W operation. feedback (a, b) are calculated, and the remaining effective feedback metrics are normalized according to their effective weights. In one implementation, feedback F can be performed using a tool. exec (a,b) serves as the primary feedback basis, with subsequent verification feedback F verify (a,b) and user feedback F use r(a,b) serves as an auxiliary feedback basis.

[0122] Dependency edge weight W dep (a,b) is normalized to [0,1], and W feedback (a,b) and the step size λ of the dependent edge weight update w When all edges are within [0,1], the updated dependency edge weights obtained according to the above weighted update relationship still remain within the range of [0,1]. w When the value is small, the updated dependency edge weights retain more historical statistical results to reduce the impact of single abnormal executions or occasional feedback on tool dependencies; λ w When the dependency weights are large, the dependency edge weights respond more quickly to recent tool execution, subsequent verification, and user feedback results. By continuously refining the pre-built tool dependencies based on actual operational data, subsequent pre-built tool completion, tool association priority judgment, and tool scheduling can be adapted to the actual operational results.

[0123] In step S960, processor 122 does not perform the credibility scoring and conflict resolution process, and uses the candidate results as the tool execution results for the generation of response data.

[0124] In this disclosure, when candidate results conflict, the authority of the data source, the timeliness of the results, the reliability of the tool, and the consistency of the results can be used as evaluation criteria for the credibility of the candidate results. After selecting, verifying, or fusing candidate results based on the credibility score, the conflict resolution results are then provided to the target generation model, which can reduce the situation where abnormal or outdated tool results are directly used for response generation.

[0125] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the heterogeneous generative model cluster scheduling method described above. The computer program may include program instructions corresponding to request data acquisition, generative model determination, toolset processing, tool context data generation, tool invocation, and response generation, and may be configured to implement these corresponding processing procedures.

[0126] The above description is a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Those skilled in the art can make corresponding changes or substitutions within the technical scope disclosed in this disclosure.

Claims

1. A heterogeneous generation model cluster scheduling device, characterized in that, Includes a processor, the processor being configured to: Acquire instruction data, determine the target generation model used to process the instruction data, and the tool information capacity of the target generation model; The task tool is determined based on the instruction data, and a candidate tool set is determined based on the tool association information. The candidate tool set includes the task tool and the preceding tools of the task tool. Under the constraint of the tool information capacity, a target tool set is determined based on the candidate tool set, and tool context data is generated. The target tool set includes at least the task tool and the necessary prerequisite tools of the task tool. The output of the necessary prerequisite tools is the required input of the corresponding subsequent tools. The tool context data includes tool description data corresponding to the tools in the target tool set. The instruction data and the tool context data are provided to the target generation model so that the target generation model calls the tools in the target tool set in the dependency order indicated by the tool association information and generates response data.

2. The heterogeneous generation model cluster scheduling device according to claim 1, characterized in that, The pre-process tools include necessary pre-process tools and auxiliary pre-process tools. The output of the auxiliary pre-process tools is a non-mandatory input that improves the execution effect of the corresponding post-process tools. The processor is configured to: If the sum of the tool description data in the candidate tool set is greater than the tool information capacity, the candidate tool set is pruned to obtain a pruned tool set. The pruning process includes retaining the task tool and the necessary prerequisite tool, and deleting at least one of the auxiliary prerequisite tools from the candidate tool set. If the sum of the tool description data of the pruning tool set is less than or equal to the tool information capacity, the pruning tool set is determined as the target tool set.

3. The heterogeneous generation model cluster scheduling device according to claim 2, characterized in that, The processor is configured to: If the pruning tool set consists of the task tool and the necessary prerequisite tool, and the sum of the tool description data of the pruning tool set is greater than the tool information capacity, then a generative model with a tool information capacity greater than or equal to the sum of the tool description data of the pruning tool set is selected as the target generative model, and the pruning tool set is determined as the target tool set.

4. The heterogeneous generation model cluster scheduling device according to claim 2, characterized in that, The processor is configured to: When the pruning tool set consists of the task tool and the necessary prerequisite tool, and the sum of the tool description data of the pruning tool set is greater than the tool information capacity, the pruning tool set is divided into at least two tool subsets according to the dependency order. A tool context data segment is generated based on the tool description data contained in the tool subset, wherein the data size of a single tool context data segment is less than or equal to the tool information capacity; The tool context data segment is provided to the target generative model in the order of dependencies.

5. The heterogeneous generation model cluster scheduling device according to claim 1, characterized in that, The processor is configured to: The prediction generation model is invoked to process the instruction data, and the predicted probability distribution output by the prediction generation model is obtained; The request uncertainty value is determined based on the predicted probability distribution, and the processing capacity requirement value is determined based on the request uncertainty value. The target generation model is determined based on the processing capacity requirement value.

6. The heterogeneous generation model cluster scheduling device according to claim 5, characterized in that, The processor is configured to: Obtain the predicted scores of the prediction generation model for the instruction data at the first k generation positions, where k is a positive integer greater than 1; The predicted probability distribution corresponding to the first k generation positions is determined based on the predicted scores; The predicted entropy values ​​corresponding to the first k generation positions are determined based on the predicted probability distribution. The request uncertainty value is determined based on the prediction entropy values ​​corresponding to the first k generation positions.

7. The heterogeneous generation model cluster scheduling device according to claim 5, characterized in that, The processor is configured to: The length of the instruction data, the tool invocation requirement represented by the instruction data, and the degree of dependence of the instruction data on context information are quantified to obtain instruction length feature value, tool invocation requirement feature value, and context dependence value. The processing capacity requirement value is determined based on the request uncertainty value, the instruction length characteristic value, the tool call requirement characteristic value, and the context dependency value.

8. The heterogeneous generation model cluster scheduling device according to claim 5, characterized in that, The processor is configured to: Obtain historical interaction data corresponding to the account identifier associated with the instruction data, and determine the model selection threshold based on the historical interaction data; Obtain the model capability data of the candidate generation model, wherein the model capability data includes the processing capability level of the candidate generation model; If the processing capacity requirement value is greater than the model selection threshold, the target generation model is determined from the candidate generation models whose processing capacity level reaches the preset processing capacity level. If the processing capacity requirement is less than or equal to the model selection threshold, the target generation model is determined from the candidate generation models whose processing capacity level is lower than the preset processing capacity level.

9. The heterogeneous generation model cluster scheduling device according to claim 1, characterized in that, The processor is configured to: Retrieve the output fields of the first tool and the required input fields of the second tool; The dependency matching score between the first tool and the second tool is determined based on the field type matching degree and semantic similarity between the output field and the required input field; When the dependency matching score reaches the dependency generation threshold, an association relationship is established in the tool association information, pointing from the first tool to the second tool. The association relationship indicates that the output field of the first tool provides input data for the required input field of the second tool.

10. The heterogeneous generation model cluster scheduling device according to claim 1, characterized in that, The processor is configured to: Obtain candidate results from at least two tools for the same data field; If the differences between the candidate results reach the conflict determination criteria, a credibility score corresponding to the candidate result is determined based on at least one of the following: data source authority, result timeliness, tool reliability, and result consistency. The conflict resolution result is determined based on the aforementioned credibility score; The conflict resolution results are provided to the target generation model to generate the response data.

11. A method for scheduling heterogeneous generation model clusters, characterized in that, include: Acquire instruction data, determine the target generation model used to process the instruction data, and the tool information capacity of the target generation model; The task tool is determined based on the instruction data, and a candidate tool set is determined based on the tool association information. The candidate tool set includes the task tool and the preceding tools of the task tool. Under the constraint of the tool information capacity, a target tool set is determined based on the candidate tool set, and tool context data is generated. The target tool set includes at least the task tool and the necessary prerequisite tools of the task tool. The output of the necessary prerequisite tools is the required input of the corresponding subsequent tools. The tool context data includes tool description data corresponding to the tools in the target tool set. The instruction data and the tool context data are provided to the target generation model so that the target generation model calls the tools in the target tool set in the dependency order indicated by the tool association information and generates response data.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the heterogeneous generation model cluster scheduling method of claim 11.

Images