Error-aware multi-model scheduling method and device, equipment and storage medium

Abstract

This application provides an error-aware multi-model scheduling method, apparatus, device, and storage medium. The method includes: acquiring a task request, and the processing success probability and processing consumption parameters of each large model providing object; acquiring the task baseline coefficient of the task request, and calculating a scheduling priority value for each large model providing object based on the task baseline coefficient, processing success probability, and processing consumption parameters; allocating the task request to the target large model providing object with the highest scheduling priority value for processing, acquiring the task processing result for evaluation, and obtaining an evaluation coefficient; determining the reference scheduling priority value and evaluation coefficient of a reference large model providing object with a scheduling priority value lower than the target large model providing object, and calculating the result exchange coefficient of the target large model providing object; and returning resource exchange data to the target large model providing object based on the result exchange coefficient. This improves the accuracy and efficiency of routing resource allocation.

Description

Technical Field

[0001] This application relates to the field of task allocation technology, and in particular to a multi-model scheduling method, apparatus, device and storage medium based on error awareness. Background Technology

[0002] Large model routing technology is a request distribution mechanism for multi-model inference platforms. It aims to dynamically select the most suitable model for processing among multiple large language models according to the user's task requirements, so as to achieve a balance between inference cost, response latency and output quality.

[0003] In related technologies, centralized routing is generally used to implement multi-model routing. Specifically, based on the data uploaded by each model in response to the task request, the success rate of each model on the task request can be predicted. Then, the target model is selected based on the inference cost of each model. Finally, the corresponding resource exchange data is returned based on the task processing result of the target model, thus completing the routing loop.

[0004] However, the relevant technologies fail to consider the dual errors inherent in the platform's post-evaluation and the provider's pre-prediction: Large model providers need to estimate the probability of their bids being deemed successful by the platform's evaluator, but this estimation itself carries errors due to a lack of understanding of the platform's evaluation details and the actual data distribution. Simultaneously, the evaluator, which the platform relies on for task processing results, inevitably contains biases and random noise. When these dual errors exist, rational providers will tend to predict the platform evaluator's pass probability rather than the objective, true success rate, and accordingly overstate their probability of being deemed successful to maximize their own profits, leading to routing resource mismatch and reduced routing efficiency. Summary of the Invention

[0005] This application proposes an error-aware multi-model scheduling method, apparatus, device, and storage medium, which can improve the accuracy of routing resource allocation and routing efficiency.

[0006] To achieve the above objectives, a first aspect of this application proposes an error-aware multi-model scheduling method, the method comprising:

[0007] Obtain the task request, and for each large model, provide the object with the processing success probability and processing consumption parameters generated based on the task request;

[0008] Obtain the task baseline coefficient corresponding to the task request, and provide an object for each large model. Combine the task baseline coefficient, the corresponding processing success probability, and the processing consumption parameter to calculate the corresponding scheduling priority value. The scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameter.

[0009] The task request is assigned to the target large model object with the highest scheduling priority value for processing, so as to obtain the task processing result;

[0010] The results of the task processing are evaluated to obtain evaluation coefficients;

[0011] A reference large model providing object is determined whose scheduling priority value is lower than that of the target large model providing object. Based on the reference scheduling priority value corresponding to the reference large model providing object and the evaluation coefficient, the result exchange coefficient corresponding to the target large model providing object is calculated. The result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value.

[0012] Based on the exchange coefficients, the target large model is provided with corresponding resource exchange data.

[0013] Accordingly, a second aspect of the embodiments of this application proposes an error-aware multi-model scheduling device, the device comprising:

[0014] The acquisition module is used to acquire task requests, as well as the processing success probability and processing consumption parameters generated by each large model object based on the task request;

[0015] The calculation module is used to obtain the task baseline coefficient corresponding to the task request, and provide an object for each large model. Combining the task baseline coefficient, the corresponding processing success probability and the processing consumption parameter, the module calculates the corresponding scheduling priority value, wherein the scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameter.

[0016] The allocation module is used to allocate the task request to the target large model object with the highest scheduling priority value for processing, so as to obtain the task processing result;

[0017] The evaluation module is used to evaluate the task processing results and obtain evaluation coefficients;

[0018] The determination module is used to determine a reference large model providing object whose scheduling priority value is lower than that of the target large model providing object, and to calculate the result exchange coefficient corresponding to the target large model providing object based on the reference scheduling priority value corresponding to the reference large model providing object and the evaluation coefficient, wherein the result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value;

[0019] The return module is used to provide the target large model with the corresponding resource exchange data based on the result exchange coefficient.

[0020] In some embodiments, the acquisition module is further configured to:

[0021] Obtain a task request and determine the task description information and task baseline coefficient corresponding to the task request;

[0022] The task description information and the task benchmark coefficients are sent to each large model provider to obtain the processing success probability and processing consumption parameters generated by each large model provider based on the task description information and the task benchmark coefficients.

[0023] In some embodiments, the computing module is further configured to:

[0024] Obtain the access level identifier of the user object corresponding to the task request, and the task type identifier corresponding to the task request;

[0025] Based on the access level identifier, the task level coefficient corresponding to the task request is determined from a preset level mapping table;

[0026] Based on the task type identifier, the task type coefficient corresponding to the task request is determined from a preset task type mapping table;

[0027] Obtain the task description information corresponding to the task request, and determine the task complexity coefficient corresponding to the task request based on the task description information;

[0028] Based on the task level coefficient, the task type coefficient, and the task complexity coefficient, calculate the task baseline coefficient corresponding to the task request.

[0029] In some embodiments, the error-aware multi-model scheduling device further includes an adjustment module for:

[0030] Determine the preceding historical task requests and obtain the allocation time interval between the historical task requests and the task request;

[0031] Based on the allocated time interval, determine the time decay coefficient;

[0032] Obtain the cumulative number of unprocessed task requests in the queue, and determine the cumulative compensation coefficient based on the cumulative number of tasks;

[0033] Based on the time decay coefficient and the cumulative compensation coefficient, the task baseline coefficient is adjusted to obtain the target task baseline coefficient;

[0034] Then, for each large model, an object is provided, and the corresponding scheduling priority value is calculated by combining the task baseline coefficient, the corresponding processing success probability, and the processing consumption parameter, including:

[0035] For each large model, an object is provided, and the corresponding scheduling priority value is calculated by combining the target task baseline coefficient, the corresponding processing success probability, and the processing consumption parameter.

[0036] In some embodiments, the computing module is further configured to:

[0037] The expected exchange rate is obtained by multiplying the task baseline coefficient and the processing success probability.

[0038] Based on the difference between the expected exchange rate and the processing consumption parameter, the scheduling priority value of the corresponding large model-provided object is obtained.

[0039] In some implementations, the evaluation module is further configured to:

[0040] Obtain the task description information corresponding to the task request, and encode the task description information and the task processing result to generate input data;

[0041] The input data is fed into a pre-trained evaluation model to obtain the evaluation coefficients corresponding to the task processing results.

[0042] In some implementations, the determining module is further configured to:

[0043] The actual exchange rate is calculated based on the product of the task baseline coefficient and the evaluation coefficient in the task request.

[0044] The result exchange coefficient corresponding to the target large model object is calculated based on the difference between the actual exchange coefficient and the reference scheduling priority value corresponding to the object provided by the reference large model.

[0045] Accordingly, a third aspect of the embodiments of this application proposes a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the error-aware multi-model scheduling method of any one of the embodiments of the first aspect of this application.

[0046] Accordingly, a fourth aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the error-aware multi-model scheduling method of any one of the embodiments of the first aspect of this application.

[0047] This application embodiment obtains task requests and the processing success probability and processing consumption parameters generated by each large model provider based on the task requests; obtains the task baseline coefficient corresponding to the task request, and calculates the corresponding scheduling priority value for each large model provider by combining the task baseline coefficient, the corresponding processing success probability, and the processing consumption parameters. The scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameters. The task request is assigned to the target large model provider with the highest scheduling priority value for processing to obtain the task processing result. The task processing result is evaluated to obtain an evaluation coefficient. A reference large model provider with a scheduling priority value lower than the target large model provider is determined, and the result exchange coefficient corresponding to the target large model provider is calculated based on the reference scheduling priority value and evaluation coefficient of the reference large model provider. The result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value. Based on the result exchange coefficient, the corresponding resource exchange data is returned to the target large model provider. In this way, the responsibility for prediction can be transferred to the provider of private information (i.e., the provider of the large model) in advance, which can reduce the centralized prediction burden of the platform and the bias caused by information asymmetry. Furthermore, a closed-loop incentive mechanism is formed by linking the post-evaluation coefficient with the reference scheduling priority value, which can effectively suppress the provider's motivation to falsely report the probability of meeting the target. Specifically, in the allocation phase, this application can directly calculate the scheduling priority value based on the processing success probability and processing consumption parameters reported by the large model provider. This eliminates the need for the platform to maintain a centralized quality predictor across models, thus avoiding prediction bias caused by information asymmetry. In the settlement phase, the result exchange coefficient depends on both the post-evaluation coefficient and the reference scheduling priority value. This ensures that the final benefit of the target large model provider is constrained by both the actual output quality (evaluation coefficient) and the remaining level of the second-best competitors (reference scheduling priority value). When a provider falsely reports its processing success probability, it may increase its winning probability, but the post-evaluation coefficient often decreases due to the actual output not meeting the requirements. Simultaneously, the competitive pressure created by the reference scheduling priority value further compresses the space for falsely reported benefits, guiding the provider's optimal bidding strategy towards its true efficiency. Ultimately, this ensures that task requests are allocated to the large model provider with the largest actual remaining resources. In summary, this application can improve the accuracy and efficiency of routing resource allocation. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the architecture of the error-aware multi-model scheduling system provided in the embodiments of this application;

[0049] Figure 2 This is a flowchart of the error-aware multi-model scheduling method provided in the embodiments of this application;

[0050] Figure 3This is a schematic diagram of the functional modules of the error-aware multi-model scheduling device provided in the embodiments of this application;

[0051] Figure 4 This is a schematic diagram of the hardware structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0055] Large model routing technology is a request distribution mechanism for multi-model inference platforms. It aims to dynamically select the most suitable model for processing among multiple large language models according to the user's task requirements, so as to achieve a balance between inference cost, response latency and output quality.

[0056] In related technologies, centralized routing is generally used to implement multi-model routing. Specifically, based on the data uploaded by each model in response to the task request, the success rate of each model on the task request can be predicted. Then, the target model is selected based on the inference cost of each model. Finally, the corresponding resource exchange data is returned based on the task processing result of the target model, thus completing the routing loop.

[0057] However, the relevant technologies fail to consider the dual errors inherent in the platform's post-evaluation and the provider's pre-prediction: Large model providers need to estimate the probability of their bids being deemed successful by the platform's evaluator, but this estimation itself carries errors due to a lack of understanding of the platform's evaluation details and the actual data distribution. Simultaneously, the evaluator, which the platform relies on for task processing results, inevitably contains biases and random noise. When these dual errors exist, rational providers will tend to predict the platform evaluator's pass probability rather than the objective, true success rate, and accordingly overstate their probability of being deemed successful to maximize their own profits, leading to routing resource mismatch and reduced routing efficiency.

[0058] Based on this, embodiments of this application provide a multi-model scheduling method, apparatus, device, and storage medium based on error awareness, which can improve the accuracy of routing resource allocation and routing efficiency.

[0059] The error-aware multi-model scheduling method, apparatus, device, and storage medium provided in this application are specifically described through the following embodiments. First, the error-aware multi-model scheduling system in this application is described.

[0060] Please refer to Figure 1 In some implementations, embodiments of this application provide an error-aware multi-model scheduling system, including a terminal 11 and a server 12.

[0061] In some implementations, terminal 11 can be used to initiate a large model inference task request to server 12. For example, it can be a user-side hardware device such as a smartphone, personal computer, tablet computer, or Internet of Things device. Terminal 11 runs an application or interacts with server 12 through a web interface to receive task description information input by the user, and encapsulates the task description information and possible task baseline coefficients (such as access level, task type, complexity, etc.) into a task request and sends it to server 12. At the same time, it receives the task processing result or routing failure prompt returned by server 12.

[0062] Furthermore, the server-side 12 can be used to implement multi-model routing logic based on error-aware reverse auctions. For example, it can be a cloud server cluster, a gateway server, or a routing node deployed in a service mesh.

[0063] In some implementations, the server 12 is configured with a request acquisition module, a priority calculation module, an allocation module, an evaluation module, an exchange calculation module, and a resource exchange module. For example, the request acquisition module can be used to acquire task requests and the processing success probability and processing consumption parameters generated by each large model provider based on the task request; the priority calculation module can be used to calculate the scheduling priority value of each provider by combining the task baseline coefficient, the processing success probability, and the processing consumption parameters; the allocation module can be used to allocate tasks to the target large model provider with the highest priority value and acquire the task processing result obtained by the target large model provider's large model; the evaluation module can be used to evaluate the task processing result to obtain an evaluation coefficient; the exchange calculation module can be used to calculate the result exchange coefficient of the target large model provider based on the reference scheduling priority value and evaluation coefficient of the reference large model provider; and the resource exchange module can be used to return the corresponding resource exchange data to the target large model provider based on the result exchange coefficient.

[0064] For example, terminal 11 and server 12 collaborate via network communication protocols (such as HTTP / HTTPS or gRPC): terminal 11 can send a task request containing task description information and task baseline coefficients to server 12. Server 12 sequentially performs operations such as bid collection, priority sorting, allocation execution, post-evaluation and settlement payment, and finally returns the task processing results or error information output by the model to terminal 11, thereby achieving efficient, scalable and incentive-compatible large model inference services in noisy evaluation and prediction error environments.

[0065] The error-aware multi-model scheduling method in this application can be illustrated through the following embodiments.

[0066] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user will be obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent will the necessary user-related data for the normal operation of the embodiments of this application be obtained.

[0067] In this embodiment, the description will focus on an error-aware multi-model scheduling device, which can be integrated into a computer device. See [link to relevant documentation]. Figure 2 , Figure 2This is a flowchart illustrating the steps of the error-aware multi-model scheduling method provided in this application embodiment. Taking the integration of the error-aware multi-model scheduling device into a terminal or server as an example, the specific process when the processor on the terminal or server executes the program instructions corresponding to the error-aware multi-model scheduling method is as follows:

[0068] Step 101: Obtain the task request, as well as the processing success rate and processing consumption parameters generated by each large model object based on the task request.

[0069] In some implementations, to overcome the scalability deficiencies, information asymmetry, and incentive distortion caused by the need for the platform to continuously profile and calibrate the performance of each new model in a dual uncertainty environment where there are platform post-evaluation errors and model provider pre-prediction errors, each large model provider can autonomously predict and report the probability that its processing results will pass the platform's evaluation and the cost required to execute the task, based on its own private capabilities and task understanding. This transfers the responsibility and risk of pre-prediction from the platform to the provider, reducing the platform's dependence on centralized calibration and laying a data foundation for suppressing false alarms and improving resource allocation efficiency in noisy environments.

[0070] Task requests can be user input or system instructions received by the platform that require processing by a large model. For example, they can be text or structured data such as a user question, a dialogue context, or the intent of a tool to be invoked, which can be used to trigger subsequent bidding, allocation, and execution processes.

[0071] The large model provider can be a model service provider that connects to the platform, can perform inference tasks, and return results. For example, it can be an API service provider that provides private or open-source large language models, a self-deployed model instance, or an inference container containing a specific version of the model, which can be used to participate in task bidding, execute task requests, and receive payment.

[0072] The pass / fail probability can be a pre-prediction of the probability that the large model provider will pass or succeed in executing the current task request under the platform's post-evaluation criteria, based on its private data and capability evaluator. For example, it can be the pass probability value obtained by the large model provider inputting the task description information corresponding to the task request into its local predictor, which can be used to characterize the large model provider's confidence in the degree of fit between its own model and the task request.

[0073] Among them, the processing cost parameter can be an estimate of the cost that a large model provides for an object to perform the current task request.

[0074] In some implementations, error-aware multi-model scheduling methods can be applied to multi-model intelligent routing and resource scheduling platforms. Specifically, the platform is a centralized or semi-centralized task distribution and settlement system deployed in a cloud inference environment, service mesh, or API gateway layer. Under conditions of dual uncertainty—including platform post-evaluation errors and model provider pre-prediction errors—it receives user task requests, broadcasts task parameters, collects processing success probabilities and execution costs autonomously reported by each model provider based on private information, calculates the remaining scheduling priorities based on reports to select the optimal executor, calls the winner model to perform inference, runs a noisy post-evaluator to determine the quality of the output, and performs incentive-compatible automatic settlement based on the evaluation results and the second-highest score (i.e., the scheduling priority value of the second-best provider). This achieves low-cost, highly scalable, robust, and socially optimal multi-model routing and resource allocation without relying on the platform for centralized performance profiling and continuous calibration of each model.

[0075] For example, a task request can be an online inference request from an application programming interface (API) call, or it can be a pending item in a batch task queue. Specifically, after receiving user input, the platform gateway can generate a unique identifier t for the task request and extract task description information d. The task description information d can include structured or unstructured data such as the user's question text, historical dialogue context, retrieval results required for Retrieval-augmented Generation (RAG), tool call permission identifiers, and system instructions.

[0076] Furthermore, the platform can determine the task baseline coefficient V based on factors such as the user's service level agreement, task type, and current system load. For example, a code generation request from a paid enterprise user can be marked as a high-priority task and assigned a higher task baseline coefficient V=2.0; while a simple question-and-answer request from a free user can be assigned a lower task baseline coefficient V=0.5. Subsequently, the platform will assign the task identifier t, task description information d, task baseline coefficient V, and optional latency constraints... Information such as these are encapsulated into task announcement messages and sent to all large model providers (i.e., the set of model service providers participating in this auction) on the current access platform via message queues or broadcast mechanisms to ensure that all potential executors can make subsequent bidding decisions based on the exact same task information.

[0077] For example, each large model provides object i, upon receiving a task announcement, to perform a pre-prediction based on its locally maintained one. Calculate the probability of achieving the target. The probability of this processing meeting the standard. This can be used to characterize the likelihood that a large model provider's predictions of its deployed model outputs will pass the platform's post-evaluation criteria. Specifically, the large model provider can first input the received task description d (which, if necessary, is concatenated with system prompts or instruction templates that its model intends to use) into its local encoder. In this process, the task embedding vector is obtained. The encoder A pre-trained SentenceTransformer model (such as the all-MiniLM-L6-v2 model) can be used and kept frozen. Next, the embedding vectors... Input to a multilayer perceptron (MLP) classifier In the process, after the hidden layer transformation, the Sigmoid activation function outputs an original probability value between 0 and 1. .

[0078] Furthermore, to reduce the provider's internal prediction error, the provider can also modify the original probability value. Perform calibration, such as using temperature scaling, Pratt calibration, or bin calibration, to... The calibration probability is adjusted to more closely approximate the true distribution of the platform evaluator. That is, processing the probability of meeting the standard.

[0079] In some implementations, each large model provides object i with the ability to calculate the processing cost parameters required to execute the current task, while simultaneously generating the probability of achieving the processing target. The processing consumes parameters. It is a quantitative scalar that can be used to comprehensively reflect the marginal cost of the provider performing the task. The calculation typically involves multiple dimensions: First, the computational cost dimension, where large model providers can calculate the basic inference cost based on the unit cost of computational power for the current model instance (e.g., the inference cost per million tokens) and the estimated number of input and output tokens; second, the latency penalty dimension, where large model providers can calculate the basic inference cost based on task latency constraints. The model provider estimates timeout risk based on the current instance's queue length and average inference latency, quantifying potential SLA default penalties as incremental costs. Finally, regarding opportunity cost, the large model provider can also consider the increased concurrent request queuing latency that might result from undertaking this task, thus impacting overall throughput and quantifying this as an additional opportunity cost. For example, for this code generation task, the large model provider estimates 1500 input tokens and 800 output tokens. Based on its current computing power price of 0.002 yuan / thousand tokens, the base cost is calculated to be (1.5 + 0.8). 0.002 = 0.0046 yuan. Considering the current queue length and the estimated expected queuing delay is close to the SLA threshold, an additional 0.001 yuan is added for latency risk cost, ultimately yielding the processing consumption parameters. =0.0056 yuan (or converted to 56 points according to the platform's unified settlement unit). This cost Will handle the probability of achieving the target. The bid information is compiled and submitted to the platform.

[0080] By employing the above methods, the responsibility for pre-prediction can be transferred from the platform to the provider possessing private information at the initial stage of task routing. This avoids the scalability bottleneck and high integration costs caused by the platform needing to continuously perform centralized calibration for each new model. Furthermore, the provider's bids naturally include its internal capabilities and task suitability information, thereby alleviating the information asymmetry problem. This provides accurate and reliable input data for achieving incentive compatibility, suppressing false alarms, and improving routing efficiency through error-aware allocation and payment rules in noisy evaluation environments.

[0081] In some implementations, to enable model providers with private information to make targeted cost and success probability predictions based on the specific content of the task and the platform's value judgment in an environment with dual uncertainties, including platform post-evaluation errors and pre-prediction errors of large model providers, task description information and task benchmark coefficients representing task importance can be broadcast to all potential large model providers as a unified public basis for bid generation. For example, step 101 may include:

[0082] (101.1) Obtain the task request and determine the task description information and task baseline coefficient corresponding to the task request;

[0083] (101.2) Send the task description information and task baseline coefficients to each large model provider object to obtain the processing success probability and processing consumption parameters generated by each large model provider object based on the task description information and task baseline coefficients.

[0084] The task description information can be text or structured data received by the platform that represents the specific content of the user's request, such as the user's input question, dialogue history summary, contextual retrieval results, or tool call intent. It can be used to enable large model providers to accurately understand task requirements and, based on this, combine their own capabilities to perform localized pre-prediction and quotation generation.

[0085] The task baseline coefficient can be a quantitative weight assigned by the platform based on the priority of the task request, the service level agreement, or the importance of the business. For example, it can be a scalar value parameter calculated based on factors such as user access level, task type, and task complexity, used to characterize the value of the task request to the platform.

[0086] Specifically, when the platform gateway receives a task request, it extracts the core content as task description information d. Task description information d may include structured or unstructured data such as natural language questions entered by the user, multi-turn dialogue history, relevant document fragments obtained from context retrieval, descriptions of tools or functions to be called, and system-preset instruction templates.

[0087] Furthermore, the platform can calculate the task baseline coefficient V based on the contextual attributes of the task request to quantify the value weight of the task request to the platform. Specifically, the task baseline coefficient V can be determined based on various factors: for example, the corresponding level coefficient can be obtained from a preset level mapping table based on the access level of the user initiating the request (such as a paid enterprise user, developer trial user, or free user); the corresponding type coefficient can be obtained from a task type mapping table based on the task type identifier (such as code generation, text summarization, or mathematical reasoning); and a task complexity coefficient can also be calculated based on the complexity of the task description information d itself (such as text length or expected reasoning difficulty). Subsequently, the platform can integrate these coefficients through weighted summation or product to obtain the final task baseline coefficient V. For example, for a code generation request from a paid enterprise user, the platform may find a level coefficient of 1.5, a task type coefficient of 1.3, and a task complexity coefficient estimated by a lightweight model of 1.2. Then, the comprehensive task baseline coefficient V = 1.5 × 1.3 × 1.2 = 2.34.

[0088] Furthermore, after determining the task description information d and the task baseline coefficient V, the platform will combine these two information with the unique identifier t of the task request and the optional task completion time limit. The parameters are encapsulated into a task announcement message and broadcast to all currently connected large model providers (i.e., the set of model service providers participating in the auction) via the platform's message middleware or direct API call. The broadcast method can use a publish-subscribe pattern, where providers pre-subscribe to the task channel and the platform publishes the task announcement to that channel; or a polling method, where providers periodically query the platform for new tasks. To ensure the reliability and security of information transmission, the platform can digitally sign the task announcement and attach a timestamp to prevent replay attacks. Upon receiving the message, the provider can initiate its local pre-prediction process.

[0089] It should be noted that after each large model provides the object with the task description information d and the task baseline coefficient V, it generates the processing success probability based on this information. and processing consumption parameters The specific implementation method has been elaborated in detail in the above embodiments, and can be referred to the above description, which will not be repeated here.

[0090] Furthermore, the platform can collect all bids returned by the large model within the bid collection window and perform validity checks (e.g., 0 ≤ ... ≤1、 (≥0, signature authentication, etc.). For late or abnormal bids, the platform can choose to discard or demote them. At this point, the platform obtains the processing success probability and processing cost parameters of all large model objects participating in this auction, preparing data for subsequent calculation of scheduling priority values ​​and task allocation.

[0091] By using the above methods, key information (description and value) of the task can be accurately delivered to all potential executors in the initial stage of task routing. This ensures that each provider makes autonomous predictions based on the same and complete task context, thereby avoiding the scalability bottleneck and high integration costs caused by the platform needing to continuously perform centralized calibration for each new model.

[0092] Step 102: Obtain the task baseline coefficient corresponding to the task request, and provide an object for each large model. Combine the task baseline coefficient, the corresponding processing success probability and processing consumption parameter to calculate the corresponding scheduling priority value. The scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameter.

[0093] In some implementations, in order to transform the private information (processing success probability and processing consumption parameters) reported by each provider into a unified, comparable, and directly applicable decision-making basis for task allocation in an environment with dual uncertainties of provider pre-prediction errors and platform post-evaluation errors, a task benchmark coefficient representing task value can be introduced. For each provider, its reported residual score (i.e., the product of the task benchmark coefficient and the processing success probability minus the processing consumption parameter) can be calculated to obtain a scheduling priority value positively correlated with the provider's expected net contribution. This simplifies the complex multi-dimensional bidding information into a single ranking scale and fundamentally avoids the burden on the platform to perform centralized performance profiling and calibration for each new model.

[0094] The scheduling priority value is a quantitative ranking index calculated by the platform based on the processing success probability and processing consumption parameters reported by each large model object, combined with the task baseline coefficient. It can be used to measure the expected net contribution of the large model object in the current task and serve as the basis for the platform to select the task executor.

[0095] It should be noted that the method for obtaining the task baseline coefficient V corresponding to the task request has been explained above and will not be repeated here.

[0096] In some implementations, the platform obtains the processing success rate reported by object i for each large model. and processing consumption parameters Then, it can be combined with the task baseline coefficient V to calculate a unified scheduling priority value. This scheduling priority value can be used to measure the expected net contribution that providing object i with the current task can bring to the platform.

[0097] Specifically, the expected gross revenue for object i is the task value V multiplied by the probability of task success. (Right now ), and performing tasks requires incurring costs. Therefore, the expected net contribution is gross revenue minus costs. Thus, the scheduling priority value... With the probability of achieving the target There is a positive correlation; the higher the probability, the greater the expected gross profit, and the higher the priority. It is also related to processing consumption parameters. There is a negative correlation: the higher the cost, the lower the net contribution and the lower the priority. Through this quantitative indicator, the platform can unify the multi-dimensional information (probability and cost) reported by different providers onto the same scale for comparison, thereby selecting the executor that is most beneficial to the platform.

[0098] In some implementations, scheduling priority value It can be determined using the following formula:

[0099] ;

[0100] in, This indicates the task baseline coefficient corresponding to the task request, which is determined by the platform based on factors such as user level and task type. This represents the probability that the processing of the object report provided by the i-th large model meets the standard, that is, the likelihood that its subjective prediction output can pass the platform's post-evaluation, and the value is between 0 and 1; This represents the processing cost parameter reported by the i-th large model object, i.e., the cost estimate required for its task execution, and is a non-negative value. This formula integrates the task value from the platform side with the provider's private information, resulting in... This provides the scheduling priority value corresponding to the object in the current large model.

[0101] In some implementations, the platform can also introduce a risk preference factor to non-linearly adjust the formula. For example, in high-risk preference scenarios, the platform may prioritize providers with higher probabilities, and could use a power-law approach. ,in >1 can amplify the advantage of high probability. >1 can increase cost penalties. Specifically, and It can be dynamically configured according to the platform's operation strategy. Furthermore, when calculating scheduling priority values, the provider's historical reputation or historical calibration error can be considered. For example, the platform can maintain historical deviation statistics (such as average prediction error) for each provider and incorporate this deviation as a correction term into the formula: ,in This represents the historical average overreporting bias of provider i (i.e., the degree of average overreporting probability). This can further suppress the provider's tendency to overreport, making the scheduling priority value closer to the true expected contribution.

[0102] By using the above methods, the bidding data reported by each provider based on private information and containing subjective prediction errors can be normalized using the common scale of task benchmark coefficient. This allows for the rapid quantification of each provider's scheduling priority value for the current task, thereby enabling automated allocation decisions based on scheduling priority values ​​without requiring the platform to have a complete picture of the internal capabilities of each model.

[0103] In some implementations, to achieve a refined and quantifiable unified measurement of the multi-dimensional value attributes inherent in a task request in an environment with dual uncertainties—including platform post-evaluation errors and provider pre-prediction errors—this can be achieved by obtaining the user's access level, task type, and task complexity calculated based on the task description information, mapping these to corresponding level coefficients, type coefficients, and complexity coefficients, and then calculating a comprehensive task benchmark coefficient through weighted or aggregated calculations. This transforms abstract business value into a common metric that can be used for bidding calculations and payment settlements, fundamentally solving the routing decision inaccuracy problem caused by the ambiguity of task value, and providing a precise value benchmark for subsequent error-aware allocation and payment rules. For example, step 102, "obtaining the task benchmark coefficient corresponding to the task request," may include:

[0104] (102.a1) Obtain the access level identifier of the user object corresponding to the task request, and the task type identifier corresponding to the task request;

[0105] (102.a2) Based on the access level identifier, determine the task level coefficient corresponding to the task request from the preset level mapping table;

[0106] (102.a3) Based on the task type identifier, determine the task type coefficient corresponding to the task request from the preset task type mapping table;

[0107] (102.a4) Obtain the task description information corresponding to the task request, and determine the task complexity coefficient corresponding to the task request based on the task description information;

[0108] (102.a5) Calculate the task baseline coefficient corresponding to the task request based on the task level coefficient, task type coefficient and task complexity coefficient.

[0109] The user object can be the entity that initiates the task request, such as an application that calls an API, a developer account, or an end-user session. It can be used to identify the source of the request and associate it with its corresponding service level agreement and billing policy.

[0110] The access level identifier can be a label or code that represents the priority or service level of a user on the platform, such as low, medium, high, or ordinary member, high member, diamond member, etc. It can be used to query the corresponding task level coefficient from a preset level mapping table to reflect the difference in the value weight of different user requests.

[0111] The task type identifier can be a marker used to distinguish the business domain or processing category to which the task belongs, such as code generation, text summarization, mathematical reasoning, tool invocation, or the corresponding category ID. It can be used to query the corresponding task type coefficient from the preset task type mapping table to reflect the differences in the strategic value or processing difficulty of different task types to the platform.

[0112] The level mapping table can be a data structure pre-configured by the platform, used to map the access level identifier of a user object to the corresponding numerical task level coefficient. For example, a key-value pair table (diamond members are mapped to 1.5, senior members to 1.2, and ordinary members to 0.8) can be used to realize the quantitative conversion of user priority to task value weight.

[0113] The task level coefficient can be a numerical scalar used to represent the task value weight determined based on the user access level, such as 1.5 or 0.8.

[0114] The task type mapping table can be a pre-configured data structure on the platform, used to map task type identifiers to corresponding numerical task type coefficients, such as a lookup table (code generation is mapped to 1.3, text summarization to 1.0, and simple question and answer to 0.7).

[0115] The task type coefficient can be a numerical scalar used to represent the task value weight determined based on the task type, such as 1.3 or 0.7.

[0116] The task complexity coefficient can be a numerical scalar calculated based on task description information, used to characterize the technical difficulty, length, or expected resource consumption of the task itself. For example, it can be a complexity score obtained by length statistics of the task text, keyword extraction, or prediction by a lightweight model. It can be used to characterize the impact of the technical difficulty of the task itself on the task value when calculating the task baseline coefficient, avoiding treating simple tasks and complex tasks the same.

[0117] For example, after receiving a task request, the platform can first extract the user object's access level identifier and the task type identifier corresponding to the task request from the context information of the request. The user object's access level identifier can be obtained by parsing the authentication credentials carried in the request, such as the API Key or Token attached by the user when making an API call. Based on the user account information associated with the credentials, the platform can query the service level to which the user belongs (such as Platinum, Gold, or Free).

[0118] Furthermore, the task type identifier can be based on explicit specification in the user request (such as the request parameter containing type=code_generation) or automatically identified by the platform through rules (for example, matching keywords in the task description d; if it contains words such as code, function, or programming, it is determined to be a code generation task).

[0119] In some implementations, the platform pre-configures a tier mapping table, a data structure (such as a database table, configuration file, or key-value pairs in memory), to map different access tier identifiers to corresponding numerical task tier coefficients. The task tier coefficient quantifies the task value weight of different user groups; higher-tier users typically have requests corresponding to higher business value or SLA requirements, thus receiving a larger task tier coefficient. This tier mapping table can be dynamically configured by platform operators according to business strategies; for example, Platinum tier can be mapped to 1.5, Gold tier to 1.2, and Free tier to 0.8. Specifically, different task tier coefficients can be preset for different tiers.

[0120] For example, the platform can also pre-configure a task type mapping table to map different task type identifiers to corresponding numerical task type coefficients. The task type coefficient quantifies the task value weight of different business domains or processing categories. For instance, code generation tasks may be assigned a higher task type coefficient due to their high requirements for model capabilities and significant commercial value, while simple question-answering tasks may be assigned a lower coefficient. This mapping table can also be flexibly configured by the platform according to operational needs; for example, code generation can be mapped to 1.3, text summarization to 1.0, mathematical reasoning to 1.2, and simple question-answering to 0.7. For instance, the platform can look up the corresponding task type coefficient T=1.3 in the task type mapping table based on the obtained task type identifier (e.g., code_generation). This coefficient will also serve as a component factor of the task baseline coefficient.

[0121] In some implementations, the platform also needs to determine the task complexity coefficient C based on the task description information d. This coefficient is used to quantify the technical difficulty, expected resource consumption, or processing complexity of the task itself. The task description information d may include user-input text, context, etc. The platform can calculate the complexity coefficient in various ways: it can segment and map based on text length (such as the number of tokens), for example, setting tasks with 0-100 tokens as simple (C=0.8), 100-500 tokens as medium (C=1.0), and more than 500 tokens as complex (C=1.2); another approach is to embed the task description using a lightweight pre-trained model, then output a complexity score through a regressor, and normalize it to a preset range (such as [0.5, 1.5]). In addition, it can also consider whether the task description contains special requirements (such as requiring JSON output, calling multiple tools, etc.) as a basis for complexity weighting. For example, if the task description is to write a quicksort function in Python, which requires processing an input list and returning a sorted list, the platform will have about 50 tokens, which is considered simple. However, considering that the number of tools to be called during the processing exceeds the preset tool number threshold (e.g., 2), and that code needs to be generated, the platform can mark the task as medium complexity based on the rules, and finally determine the task complexity coefficient as 1.2, etc.

[0122] In some implementations, after obtaining the task level coefficient L, task type coefficient T, and task complexity coefficient C, the platform can fuse them to obtain a comprehensive task baseline coefficient V. The fusion method can be a product to reflect the synergistic amplification effect between the factors, or a weighted sum to reflect their independent contributions. In a preferred implementation, the task baseline coefficient is calculated using a product: V = L × T × C; where L is the task level coefficient, T is the task type coefficient, and C is the task complexity coefficient. The specific fusion method (direct fusion or weighted fusion) can be configured by the platform based on the actual business scenario and the degree of importance attached to each factor.

[0123] The task benchmark coefficients calculated using the above methods can accurately depict the true value weight of each task request to the platform. This allows the platform to calculate the remaining score of the provider's report based on this precise value benchmark during the subsequent bidding and allocation process, thereby guiding resources towards high-value tasks.

[0124] In some implementations, to quantify the expected gross revenue generated by each provider's task execution in an environment with dual uncertainties—including platform post-evaluation errors and provider pre-prediction errors—and thus provide an intermediate calculation basis for obtaining the net surplus after cost deduction and allocating tasks accordingly, the expected exchange rate can be obtained by calculating the product of the task baseline coefficient and the processing achievement probability. This integrates the platform value and provider predictions with a unified dimension through multiplication, thereby providing an accurate gross revenue metric for calculating the provider's expected net contribution and implementing incentive-compatible routing decisions based on reported surplus. For example, step 102, "combining the task baseline coefficient, the corresponding processing achievement probability, and the processing consumption parameter to calculate the corresponding scheduling priority value," may include:

[0125] (102.b1) The expected exchange rate is obtained based on the product of the task baseline coefficient and the processing success probability;

[0126] (102.b2) Based on the difference between the expected exchange rate and the processing consumption parameter, the scheduling priority value of the corresponding large model provided object is obtained.

[0127] The expected exchange rate can be an intermediate quantitative indicator obtained by multiplying the task baseline coefficient by the processing success probability reported by the corresponding large model object. It can be used to represent the expected value of the large model object in the current task, before deducting execution costs.

[0128] In some implementations, due to the processing success rate The model represents the likelihood that the subjective prediction of object i's output provided by the large model can pass the platform's post-evaluation, while the task baseline coefficient V represents the objective value of the task to the platform. Therefore, the product of the two is... This constitutes the expected gross revenue under conditions of uncertainty, also known as the expected exchange rate. The introduction of the expected exchange rate enables the platform to transform the provider's private predictive information into a quantifiable revenue indicator aligned with the platform's value benchmark, even in the presence of prior prediction errors (the provider's bias in estimating its own probability of success).

[0129] Furthermore, after obtaining the expected exchange rate for each provider, the platform can deduct the cost required to execute the task to obtain the expected net contribution of that provider to the platform. Specifically, while provider i may generate revenue (expected exchange rate) by executing the task, the provider itself also incurs execution costs. Therefore, from the platform's perspective, the true metric for evaluating the worthiness of an offering to be selected should be the net surplus after subtracting the cost (i.e., processing consumption parameter) from the expected exchange rate. From this, the scheduling priority value can be obtained. The calculation formula is as follows:

[0130] ;

[0131] Scheduling priority and processing success rate Positive correlation, The higher the value, the higher the gross profit and the higher the net surplus; this is related to the processing consumption parameters. Negative correlation The higher the value, the lower the net surplus. This quantitative indicator allows for the unification of multidimensional information (probability and cost) reported by different large models onto a single scale for ranking and comparison, thereby selecting the executor with the greatest expected net contribution to the platform.

[0132] In some implementations, each large model provides an object that can return a corresponding probability of success in processing a task request. Processing consumption parameters and latency costs It can Breaking down into computing power costs and latency costs And assign different weights to schedule priority values. ,in and This is a coefficient that the platform dynamically adjusts based on the current load. This allows scheduling decisions to respond more precisely to the real-time status of the system, further improving routing efficiency.

[0133] By using the above methods, the task value weight on the platform side and the success probability predicted by the provider based on private information can be quantitatively integrated. This allows the multi-dimensional bidding information of each provider to be transformed into a unified expected gross revenue metric without the platform needing to have a complete picture of the model's internal capabilities.

[0134] In some implementations, to prevent high-value tasks from becoming ineffective due to prolonged non-allocation or low-value tasks from accumulating resources due to backlog caused by static task baseline coefficients, the task baseline coefficients can be dynamically adjusted by introducing the allocation time interval for task requests and the accumulated number of unprocessed tasks in the queue. This allows routing decisions to perceive the system load status and historical frequency of task processing in real time, thereby guiding model providers to respond more actively to high-frequency or backlogged tasks in a multi-model environment with dual errors, thus optimizing the fairness and efficiency of task allocation. For example, after "obtaining the task baseline coefficient corresponding to the task request" in step 102, the following may also be included:

[0135] (A.1) Determine the preceding historical task requests and obtain the allocation time interval between the historical task requests and the task request;

[0136] (A.2) Determine the time decay coefficient based on the allocation time interval;

[0137] (A.3) Obtain the cumulative number of unprocessed task request queues and determine the cumulative compensation coefficient based on the cumulative number of tasks;

[0138] (A.4) Based on the time decay coefficient and the cumulative compensation coefficient, the task baseline coefficient is adjusted to obtain the target task baseline coefficient;

[0139] For each large model, an object is provided, and the corresponding scheduling priority value is calculated by combining the task baseline coefficient, the corresponding processing success probability, and the processing consumption parameters, including:

[0140] For each large model, an object is provided, and the corresponding scheduling priority value is calculated by combining the target task baseline coefficient, the corresponding processing success probability, and the processing consumption parameters.

[0141] Among them, historical task requests can be tasks that have entered the system and been completed or are in the process of being assigned before the current task request, such as the previous task or the previous batch of tasks.

[0142] The allocation time interval can be the time difference between the current task request and the previous historical task request, such as the difference in timestamps of two tasks arriving at the system. It can be used to quantify the waiting time of a task or the idle level of the system.

[0143] The time decay coefficient can be a numerical factor determined based on the allocation time interval. This coefficient is negatively correlated with the allocation time interval, that is, the larger the interval, the smaller the coefficient. It is used to represent the degree to which the value of the task decays over time, thereby reflecting the timeliness requirements in scheduling.

[0144] The cumulative number of tasks can be the number of tasks that have not yet been processed and are still waiting to be assigned in the queue at the current moment, which can be used to reflect the real-time load pressure of the system.

[0145] The cumulative compensation coefficient can be a numerical factor determined based on the cumulative number of tasks. This coefficient is positively correlated with the cumulative number of tasks, meaning the longer the queue, the larger the coefficient. It is used to represent the dynamic compensation adjustment made by the system to the task baseline coefficient to alleviate backlog, so as to incentivize the model provider to prioritize the processing of backlog tasks when the load is high.

[0146] The target task baseline coefficient can be a final value scalar obtained by dynamically adjusting the original task baseline coefficient by introducing a time decay coefficient and a cumulative compensation coefficient, and used for subsequent scheduling priority calculation. It can be used in dynamic load environments to enable the task baseline coefficient to respond in real time to changes in system state, thereby guiding resource allocation to better match actual operational needs.

[0147] In some implementations, to perceive the timing characteristics of system task processing in real time, upon receiving a current task request, the system can first determine its preceding historical task requests. Specifically, the system can maintain a global variable or key-value pair in memory or a distributed cache (such as Redis) to record the arrival timestamp or allocation timestamp of the most recently assigned task request. When a new task request arrives, the system reads the previous timestamp of the record and subtracts it from the arrival timestamp of the current task request to obtain the allocation time interval Δt. If the system is empty (i.e., no historical tasks), Δt can be preset to a large value (such as infinity) or the decay adjustment can be skipped directly. This allocation time interval Δt reflects the system's idle level or the density of task arrivals: the smaller the interval, the more frequent the tasks, and the system may be under high load; the larger the interval, the relatively idle the system. For example, assuming the system processed the previous task at time T1, and the current task arrives at time T2, then Δt = T2 - T1, in milliseconds or seconds. After the record is updated, the timestamp of the current task is written to this global variable for use by subsequent tasks.

[0148] Furthermore, after obtaining the allocation time interval Δt, it can be quantified as a decay factor that can be applied to the task baseline coefficient. The design principle of the time decay coefficient is: the shorter the task interval, the more the system wants to retain or enhance the attractiveness of the task, so the decay coefficient should approach 1; the longer the interval, the lower the task value may be over time, so the decay coefficient should approach 0. In some implementations, an exponential decay function can be used to determine the time decay coefficient. For example, it can be calculated using the following formula:

[0149] ;

[0150] in, The pre-set attenuation rate parameter can be configured according to the system's sensitivity to timeliness (e.g., =0.1 means it decays to approximately 0.37 every 10 seconds. This formula guarantees... It monotonically decreases as Δt increases, and always lies between 0 and 1. In other implementations, an inverse proportional function may also be used. =1 / (1+β·Δt) or a piecewise linear function, for example, when Δt is less than the threshold. =1, otherwise decreases linearly. In this way, tasks that arrive frequently in the near future will receive a higher baseline coefficient during subsequent scheduling, and are therefore more likely to be prioritized.

[0151] It's important to note that besides time, the current system load also directly impacts routing efficiency. Therefore, we can obtain the cumulative number L of unprocessed task requests in the queue, i.e., the number of tasks that haven't been assigned and are still waiting in the queue at the current moment. This value can be directly read from the task queue management module (such as the queue length statistics of the message middleware). A larger L indicates a more severe backlog, requiring incentives for larger models to provide objects that can process the backlog of tasks more quickly. For this purpose, a cumulative compensation coefficient is introduced. It is positively correlated with L. In some implementations, a linear compensation formula can be used:

[0152] =1+k·L;

[0153] Where k is the compensation strength coefficient (e.g., k=0.1), ensuring ≥1. When L=0 =1, no compensation; when L=10 =2, the baseline coefficient is doubled. To limit the compensation range, a logarithmic form can also be used. =1+k·ln(1+L) or a piecewise function. For example, suppose there are 5 tasks waiting in the queue, and k=0.1, then =1.5 indicates that the baseline coefficient for the task will be increased by 50% to attract more model providers to participate in the bidding and alleviate the backlog.

[0154] Furthermore, the time decay coefficient is obtained. and cumulative compensation coefficient Then, the original task baseline coefficient V can be adjusted to generate the target task baseline coefficient for subsequent scheduling priority calculation. The adjustment method must simultaneously reflect both time-related degradation and load compensation; a multiplicative combination can be used specifically:

[0155] ;

[0156] Wherein, V is the initial task baseline coefficient determined based on static factors such as user access level, task type, and complexity (see Task Value Determination Method). ∈(0,1] makes It decreases as the interval increases. ≥1 makes The value increases as the queue backlog increases. Both factors work together to dynamically adapt the task value to the system state.

[0157] For example, if V=100, =0.8, =1.5, then =120, higher than the original value; if the interval is large and the queue is empty. =0.3, =1, then =30, lower than the original value. The system will then use... Instead of the original V, incorporate the processing compliance probability reported by each model provider. and processing consumption parameters Calculate the scheduling priority value Based on this, tasks are allocated. In this way, routing decisions not only reflect the inherent value of the tasks, but also respond in real time to system load and timeliness requirements, improving overall efficiency and stability.

[0158] In some implementations, the determination of the cumulative compensation coefficient depends not only on the queue length but also on the distribution of task waiting times. For example, the waiting time of each task in the queue can be statistically analyzed to calculate the average or longest waiting time, and the compensation intensity can be adjusted accordingly. If the queue is long but consists entirely of newly arrived tasks, the compensation can be moderate; if there are tasks in the queue that have not been processed for a long time, a higher compensation is required to prioritize their processing. For example, the compensation coefficient can be set to... ,in This represents the average waiting time. This serves as a weighting coefficient. In this way, the system can more precisely perceive the quality of the task backlog, avoiding the problem of prioritizing short tasks while starving long tasks due to simple quantity compensation.

[0159] In some implementations, the task baseline coefficient can be adjusted based on the task complexity coefficient corresponding to the task request to obtain the target task baseline coefficient. Specifically, the task description information corresponding to the task request and the historical task description information of multiple historical task requests corresponding to the task description information can be obtained, wherein the similarity between each historical task description information and the task description information is greater than a preset similarity threshold; the historical task processing result corresponding to each historical task request is obtained, and the task complexity coefficient corresponding to the task request is calculated based on the differences between the multiple historical task processing results corresponding to multiple historical task requests.

[0160] Specifically, if a task request is semantically highly similar to several historical tasks, but the processing results of these historical tasks differ significantly (e.g., inconsistent answers, diverse output formats, or varying quality), it indicates that the task itself may be ambiguous, difficult, or sensitive to model capabilities, meaning it has high task complexity. In this case, the initial baseline coefficient for the task can be appropriately increased to obtain the baseline coefficient for the target task, thus incentivizing the model provider to invest more computational resources and adopt a more cautious generation strategy, thereby improving output quality and consistency. Conversely, if the results of historical tasks are highly consistent, it indicates that the task is relatively simple and clear, and the baseline coefficient can be maintained or slightly reduced to avoid resource waste caused by over-incentivization.

[0161] In its implementation, the system first obtains the task description information d corresponding to the current task request, and then searches the historical task database based on semantic similarity to find multiple historical task requests whose similarity to d exceeds a preset threshold (e.g., cosine similarity greater than 0.9), recording their historical task description information set D_sim. Subsequently, it obtains the task processing result set corresponding to these historical tasks. ={y_1,y_2,..., }. For quantification The differences between the results can be measured in various ways, such as calculating the variance of the semantic embedding distance between all pairs of results, the dispersion based on BLEU or ROUGE scores, or using clustering algorithms to determine the number of result clusters. In one implementation, the task complexity coefficient ω can be calculated using the following formula: ω=1+β·Var({E(y)|y∈ }), where E(·) is the encoder that maps the result to the embedding vector, Var is the variance calculation, and β is the scaling factor. The larger the variance, the larger ω. Finally, the original task baseline coefficient V is adjusted to V'=V·ω. For example, suppose the current task description is "explain what machine learning is." A search reveals that among the results of similar historical tasks, some answers emphasize supervised learning, some emphasize deep learning, and some even confuse the concept of artificial intelligence, resulting in a large variance in the result vector. In this case, ω=1.5, and V increases from 100 to 150, thus giving the task a higher priority in subsequent scheduling and attracting higher-quality models. In this way, task complexity can be perceived through differences in historical results, achieving dynamic and fine-tuned adjustment of task value.

[0162] By using the above methods, when the task request interval is short (i.e., similar tasks have been processed recently) or there is a large backlog in the queue, the task baseline coefficient is increased accordingly, thereby increasing the scheduling priority value of each model provider. This makes these tasks easier to be allocated first, thus shortening user waiting time and alleviating system congestion. Conversely, when the task interval is long or the queue is idle, the baseline coefficient returns to a normal level, avoiding over-incentivization. This achieves adaptive adjustment of system load and improves overall resource utilization and routing stability.

[0163] Step 103: Assign the task request to the target large model object with the highest scheduling priority value for processing, so as to obtain the task processing result.

[0164] In some implementations, in order to actually deliver the task to the calculated and confirmed optimal executor and obtain its raw output after completing the ranking decision based on the report remainder, thereby providing the necessary execution result data for subsequent post-evaluation and error-aware payment, the complete description of the task request and execution parameters (such as temperature, maximum token, timeout limit, etc.) can be encapsulated into an inference call request and sent to the API interface or service endpoint of the target large model providing object with the highest scheduling priority value, so as to wait for and receive the returned inference result, thereby completing the closed loop from bidding decision to physical execution, and thus providing real execution output for the subsequent settlement process based on noisy evaluation signals.

[0165] The target large model provider can be a large model provider determined as the optimal executor for the current task after sorting based on scheduling priority values, for example, reporting the remaining score among all participating providers. The highest-ranking winner, j, can be used to actually perform tasks and generate output, and receive the corresponding payment.

[0166] The task processing result can be the final output returned by the target large model after receiving the task request and performing local inference calculations through the local model, such as generated text answers, code snippets, structured data, etc.

[0167] In some implementations, after calculating the scheduling priority values ​​for each model provider, the system needs to allocate the current task request to the target large model provider with the highest priority value and perform inference to obtain the task processing result. Specifically, the allocation module first processes all valid bids... Sort the data, determine the largest model corresponding to the maximum value, provide object j as the target (i.e., the winner), and record the results. The second highest value (marked as H) will be used as the basis for subsequent settlement.

[0168] Specifically, if the highest scheduling priority value is ≤0, the system can trigger an empty allocation strategy, such as rolling back to the platform's own base model processing, returning a rejection, or delaying a retry. For non-empty allocations, the system can initiate a remote procedure call (such as an HTTP / gRPC request) to the target large model providing object j, passing in the task description information d, the task baseline coefficient V (or V'), and execution parameters (such as the maximum number of tokens generated, temperature coefficient, tool call permissions, timeout threshold T_deadline, etc.). After receiving the request, the target large model providing object j performs inference based on its local model and prompt word strategy to generate the output result. (Such as text responses, structured data, or tool call sequences), and return the results to the platform synchronously or asynchronously. After receiving the output results, the platform records billing and auditing information such as execution latency, token consumption, and return codes. At the same time, it temporarily stores the output results as the original task processing results for later use (such as post-event evaluation and settlement).

[0169] By using the above methods, the optimal provider selected based on the error-aware reverse auction mechanism can be seamlessly connected with the actual physical execution process. This ensures that the theoretically optimal allocation decision can be translated into observable and evaluable actual output, thereby providing a real data foundation for subsequent error-aware payments based on ex-post evaluation signals. Ultimately, this achieves incentive compatibility and efficient resource allocation in noisy environments.

[0170] Step 104: Evaluate the task processing results and obtain the evaluation coefficient.

[0171] In some implementations, the raw text or results output by the winner model can be transformed into an observable quantitative signal that can be used for subsequent error-aware settlement. This replaces the unobservable real task success status with a verifiable platform evaluation pass status, providing actual observations aligned with the predicted target at the time of bidding for subsequent payment based on the evaluation coefficient and the second highest score.

[0172] The evaluation coefficient can be a numerical signal obtained by the platform evaluating the output of the object provided by the winner after it has completed the task by running a pre-built post-evaluator. For example, it can be obtained by comparing the task description d with the task processing result. The binary indicator variable (with a value of 0 or 1) obtained by concatenating the input SentenceTransformer+MLP model and passing the threshold decision can be used to indicate whether the output has passed the platform's quality judgment.

[0173] In some implementations, after obtaining the task processing result returned by the target large model, the result can be quantitatively evaluated to obtain an evaluation coefficient. Specifically, firstly, the task description information corresponding to the task request (such as user input, context, tool invocation intent, etc.) is obtained, and then concatenated with the task processing result to construct the evaluation input data. In some implementations, a pre-trained sentence encoder (e.g., SentenceTransformer) can be used to... Mapped to a fixed-dimensional embedding vector e=E( E(·) represents the encoder function. Subsequently, the embedding vector e is input into a pre-trained multilayer perceptron (MLP) classifier, and the pass / fail probability is obtained through a sigmoid activation function. =sigmoid(MLP(e)), where the probability value is the evaluation coefficient for the continuous type.

[0174] For example, if the system needs to output a binary evaluation signal, a decision threshold can be set. (e.g., 0.5), when ≥ The evaluation coefficient is set to 1 (indicating pass) if the condition is met, and 0 (indicating failure) otherwise. This evaluation model is typically trained using historical task data (including manually labeled or rule-based evaluation labels), and its optimization objective is binary classification cross-entropy loss. To improve evaluation robustness, the platform can also employ a multi-evaluator fusion strategy, such as weighted voting or averaging of rule-based evaluation results, Large Language Model (LLM-as-judge) evaluations, and manual sampling results to obtain the final evaluation coefficient. For example, for the user question "How to treat a cold?", Model A returns text containing medical advice. The platform concatenates the question and answer and inputs the result into the evaluation model. =0.92, exceeding the threshold of 0.7, therefore the evaluation coefficient is... =1 indicates that the result is acceptable; if the other model returns empty content, the evaluation coefficient may be 0.

[0175] By using the above methods, the original output generated after the winner's execution can be transformed into an observable and quantifiable evaluation coefficient through a standardized evaluation model. This replaces the unobservable true success state with a platform-approved evaluation pass state, thereby providing accurate input for subsequent error-aware payment based on the evaluation coefficient and the second-highest score.

[0176] In some implementations, to transform the original task processing results into a standardized quantitative signal aligned with the predicted target during bidding, which can be used for subsequent error-aware settlement, in situations where there are inherent biases and noise in the platform's post-evaluator (i.e., post-evaluation errors) and the true quality of the winner's output cannot be directly observed, the task description information and the task processing results of the winner's output can be concatenated and encoded to generate a comprehensive representation vector (input data) containing the task context and model output. This vector is then input into a pre-trained evaluation model that can simulate the platform's evaluation criteria. After forward computation by the model, a numerical evaluation coefficient (e.g., through probability or binary decision results) is obtained. This replaces the noisy and uncontractually contractual real completion state with an observable and verifiable platform evaluation signal, thereby providing a target aligned with the pre-predicted objective for the subsequent payment process based on the evaluation coefficient and the second-highest score. A consistent settlement basis. For example, step 104 may include:

[0177] (104.1) Obtain the task description information corresponding to the task request, and encode the task description information and the task processing result to generate input data;

[0178] (104.2) Input the input data into the pre-trained evaluation model to obtain the evaluation coefficients corresponding to the task processing results.

[0179] The input data can be a structured or sequential representation constructed by the platform for post-event evaluation, such as the task description information d and the task processing results output by the winner. The data is concatenated and converted into a vector embedding e by an encoder (such as SentenceTransformer) to capture the degree of matching between the task requirements and the actual output.

[0180] The evaluation model can be a machine learning model pre-trained and deployed on the platform, used to determine the quality of task processing results. For example, it could be a binary classification neural network consisting of a SentenceTransformer encoder and an MLP classifier, which can be used to map input data to probability. and through threshold The judgment received a final evaluation coefficient. This is to simulate the platform's criteria for judging output quality.

[0181] In some implementations, the process of obtaining the task description information corresponding to the task request has been described above and will not be repeated here.

[0182] In some implementations, after obtaining the task processing result returned by the target large model, the result can be quantitatively evaluated to obtain an evaluation coefficient. Specifically, firstly, the task description information corresponding to the task request (such as user input, context, tool invocation intent, etc.) is obtained, and then concatenated with the task processing result to construct the evaluation input data. In some implementations, a pre-trained sentence encoder (e.g., SentenceTransformer) can be used to... Mapped to a fixed-dimensional embedding vector e=E( E(·) represents the encoder function. Subsequently, the embedding vector e is input into a pre-trained multilayer perceptron (MLP) classifier, and the pass / fail probability is obtained through a sigmoid activation function. =sigmoid(MLP(e)), where the probability value is the evaluation coefficient for the continuous type.

[0183] For example, if the system needs to output a binary evaluation signal, a decision threshold can be set. (e.g., 0.5), when ≥ The evaluation coefficient is set to 1 (indicating pass) if the condition is met, and 0 (indicating failure) otherwise. This evaluation model is typically trained using historical task data (including manually labeled or rule-based evaluation labels), and its optimization objective is binary classification cross-entropy loss. To improve evaluation robustness, the platform can also employ a multi-evaluator fusion strategy, such as weighted voting or averaging of rule-based evaluation results, Large Language Model (LLM-as-judge) evaluations, and manual sampling results to obtain the final evaluation coefficient. For example, for the user question "How to treat a cold?", Model A returns text containing medical advice. The platform concatenates the question and answer and inputs the result into the evaluation model. =0.92, exceeding the threshold of 0.7, therefore the evaluation coefficient is... =1 indicates that the result is acceptable; if the other model returns empty content, the evaluation coefficient may be 0.

[0184] Using the above methods, the original output can be transformed into a quantifiable and reproducible evaluation coefficient. This allows, in environments where platform evaluation errors exist, the unobservable actual task success status to be replaced with the predicted target at the time of bidding. The platform evaluation status is consistent with the bid, thus providing an observation signal aligned with the bid for the subsequent error-aware settlement process based on the evaluation coefficient and the second highest score.

[0185] Step 105: Determine the reference large model providing object whose scheduling priority value is lower than that of the target large model providing object, and calculate the result exchange coefficient corresponding to the target large model providing object based on the reference scheduling priority value and evaluation coefficient of the reference large model providing object. The result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value.

[0186] In some implementations, to construct an incentive-compatible payment rule to suppress false reporting and ensure allocation efficiency in an environment with dual uncertainties—including platform post-evaluation errors and provider pre-prediction errors—a reference scheduling priority value for the reference large model provider, whose scheduling priority value is second only to the winner in the allocation process, can be introduced as an external opportunity cost. Based on the actual evaluation coefficient obtained by the winner (rather than its self-reported probability), a result exchange coefficient that is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value can be calculated. This links the winner's final benefit to its actual performance and the level of market competition, thereby forcing the provider to maximize its expected utility by truthfully reporting its predicted evaluation pass probability when bidding. This fundamentally solves the incentive distortion problem caused by dual errors.

[0187] The reference large model provider can be another large model provider whose scheduling priority value is second only to the selected target large model provider (winner j) in the scheduling priority ranking. For example, the large model provider k that reports the second highest remaining score among all participating providers can be used as an external benchmark for calculating the winner's payout to reflect the level of market competition.

[0188] The reference scheduling priority value can be the scheduling priority value corresponding to the object provided by the reference large model, such as the second highest reported remaining score H=max_{k≠j}. , where j is the object provided by the target large model.

[0189] The evaluation coefficient can be a numerical signal obtained by the platform evaluating the output of the object after it performs the task provided by the target large model by running a post-evaluator. For example, it can be obtained by comparing the task description d with the task processing result. The binary indicator variable obtained after inputting into the evaluation model and passing a threshold decision. (Value can be 0 or 1).

[0190] The result exchange coefficient can be the amount that the platform calculates based on the evaluation coefficient actually obtained by the winner and the reference scheduling priority value of the object provided by the reference large model. It is the final amount paid to the winner and can be used as a reward for performing tasks on the object provided by the target large model. It ensures that when the evaluation is passed, the reward is the gross reward minus the external opportunity cost, and when the evaluation is failed, it may be negative to form a penalty.

[0191] In some implementations, the scheduling priority values ​​for all providers are calculated. At that time, the values ​​have been sorted and the target large model provider j corresponding to the maximum value and the large model provider k corresponding to the second largest value have been recorded. This large model provider k is the reference large model provider. If multiple providers are tied for the second highest value, one can be selected or the average value can be taken. If there is only one provider participating in the bidding, the reference scheduling priority value H can be preset to 0 or a very small positive value to ensure the validity of the payment formula. For example, assuming that the scheduling priority values ​​of providers A, B, and C are 85, 60, and 30 respectively, then the target large model provider is A, the reference large model provider is B, and its reference scheduling priority value H=60. The system temporarily stores this reference object and its priority value for subsequent calculations.

[0192] Understandably, due to the existence of ex-post assessment errors, this application introduces an assessment coefficient to retrospectively adjust the task value. Specifically, the payment received by the target should be equal to the ex-post assessed task value minus the externalities caused by its elimination, where the ex-post assessed task value is composed of the task baseline coefficient V and the assessment coefficient. The product of these values ​​represents the externalities, while the externalities are measured by the second-highest priority value H (i.e., the reference scheduling priority value). Thus, the final utility of the target object is the payment minus its true cost, and its optimal strategy remains to truthfully report its pre-predicted probabilities and costs, thereby achieving incentive compatibility in a double-error environment.

[0193] In some implementations, the resulting exchange rate is... It can be determined by the following formula:

[0194] ;

[0195] in, This indicates that the target large model provides the result exchange rate for object j; This represents the baseline coefficient for the task (which can be the baseline coefficient for the target task after dynamic adjustment). This reflects the inherent value weight of the task itself; The evaluation coefficient represents the result obtained after evaluating the task processing result of target object j, and can be binary (0 or 1) or continuous (0 to 1); H represents the reference scheduling priority value corresponding to the object provided by the reference large model, i.e., the second highest score. The above formula reflects the constraint relationship that the result exchange coefficient is positively correlated with the evaluation coefficient (the higher the evaluation, the more payment) and negatively correlated with the reference scheduling priority value (the greater the externality, the less payment).

[0196] when When H is negative, the calculation result is negative. At this time, the platform can truncate (set to 0) according to the rules or deduct it from the margin to ensure the balance of the system's revenue and expenditure.

[0197] Through the above method, by introducing the reference scheduling priority value as the external opportunity cost and combining the actual evaluation coefficient to construct the result conversion coefficient, the direct connection between the winner's (the object provided by the target large model) income and its self-reported probability can be cut off. Thus, the provider is forced to truthfully report the predicted evaluation passing probability and cost during the bidding stage. Otherwise, it will face the risk of being落选 or a decrease in expected income due to false reporting, thereby achieving incentive compatibility in an environment with double errors and further strengthening the provider's motivation to optimize its own prediction accuracy.

[0198] In some embodiments, in an environment with double uncertainties of the platform's ex-post evaluation error and the provider's ex-ante prediction error, to convert the noisy evaluation coefficient actually obtained by the winner into a payment amount linked to the market opportunity cost and available for final settlement, thereby constructing an incentive-compatible closed-loop mechanism, it can be achieved by first calculating the product of the task benchmark coefficient and the evaluation coefficient to obtain the actual conversion coefficient (i.e., the gross income actually created by the winner), and then subtracting the reference scheduling priority value of the reference large model provider (i.e., the second-highest reported surplus, representing the social opportunity cost), so as to obtain the result conversion coefficient finally paid to the winner, thereby strictly binding the winner's reward to its actual performance (although noisy) and the market competition level. This makes the optimal strategy of the provider only to truthfully report the predicted evaluation passing probability and cost, fundamentally solving the incentive distortion and efficiency loss problems caused by double errors. Exemplarily, "calculating the result conversion coefficient corresponding to the target large model provider based on the reference scheduling priority value and evaluation coefficient corresponding to the reference large model provider" in step 105 may include:

[0199] (105.1) Calculate the actual conversion coefficient based on the product between the task benchmark coefficient of the task request and the evaluation coefficient;

[0200] (105.2) Calculate the result conversion coefficient corresponding to the target large model provider based on the difference between the actual conversion coefficient and the reference scheduling priority value corresponding to the reference large model provider.

[0201] Among them, the actual conversion coefficient can be an intermediate quantitative index obtained by the platform multiplying the task benchmark coefficient by the evaluation coefficient actually obtained by the winner, which can be used to represent the observable gross income actually created by the winner based on the platform's evaluation result after executing the task, and serve as the basic input for subsequent calculation of the final payment amount.

[0202] In some embodiments, when obtaining the task benchmark coefficient V (or the target task benchmark coefficient after dynamic adjustment) The target large model provides the evaluation coefficients corresponding to the task processing results of object j. Next, the actual exchange rate can be calculated, which represents the actual value of the task after quality correction. Specifically, the actual exchange rate can be obtained by multiplying the task baseline rate and the evaluation rate, i.e. It can be used to characterize the realization value of the task for the platform after taking into account the actual output quality.

[0203] Furthermore, after obtaining the actual exchange coefficient, the final exchange coefficient (i.e., payment amount) obtained by the target large model provider can be calculated by combining the reference scheduling priority value H corresponding to the reference large model provider. Specifically, the payment obtained by the target large model provider should be equal to the social value it creates minus the externalities caused by its replacement. In some implementations, the exchange coefficient can be determined by calculating the following formula:

[0204] ;

[0205] in, V represents the exchange rate for the result provided by the target large model for object j; V is the task baseline coefficient. H represents the evaluation coefficient; H is the reference scheduling priority value corresponding to the object provided by the reference large model. This formula reflects the constraint relationship that the result exchange coefficient is positively correlated with the evaluation coefficient (the higher the evaluation, the more payment) and negatively correlated with the reference scheduling priority value (the greater the externality, the less payment). When the calculation result is negative, the platform can truncate it according to the rules (such as setting it to 0) or deduct it from the margin to ensure the system's revenue and expenditure balance.

[0206] By using the above methods, the winner's reward can be strictly linked to their actual performance (despite the noise) and the bidding level of competitors, thereby ensuring that the entire payment rule meets incentive compatibility. That is, no matter how the provider predicts the platform's evaluation error, its bidding strategy to maximize expected utility is to truthfully report its private information, ultimately achieving optimal resource allocation in a noisy environment.

[0207] Step 106: Based on the result exchange coefficient, provide the target large model with the corresponding resource exchange data returned to the object.

[0208] In some implementations, in order to translate the theoretical result exchange coefficient into a practical and executable resource transfer, thereby forming a complete closed-loop incentive and ensuring that the provider receives compensation commensurate with its actual contribution, the result exchange coefficient can be sent to the platform's settlement and billing system. This system will then generate corresponding payable vouchers, initiate resource exchange operations such as fund transfers or points distribution, and record them in the reconciliation log. This completes the entire economic closed loop from task release, bidding, allocation, execution, evaluation to final settlement, thereby ensuring that the error-aware reverse auction mechanism is implementable, repeatable, and auditable in the real system.

[0209] Among them, resource exchange data can be a reward certificate or transfer instruction generated by the platform based on the result exchange coefficient, used to actually pay the target large model provider. For example, an amount to be settled, points, tokens, or accounts payable recorded in the billing system can be used to complete the final economic compensation to the winner provider and be written into the reconciliation log as the closing link of the entire auction mechanism.

[0210] In some implementations, the exchange rate of the target large model provided by object j is calculated. Subsequently, the target large model can be provided with corresponding resource exchange data to achieve a closed loop of economic incentives. Specifically, resource exchange data can take various forms, such as increases in platform account balances, redeemable points, digital currency transfers, or on-chain token transfers. The settlement module first... Associated with the account identifier of provider j, if If the result is positive, the corresponding amount will be added to the account balance, and a record containing the task ID, provider ID, and... Evaluation coefficient Reconciliation records, including those with priority values ​​such as H, are written to a distributed billing system or blockchain ledger for auditing purposes; if If the value is negative (e.g., due to an unsatisfactory assessment and a relatively large second-highest value), the system can handle it according to preset risk control rules. This could involve deducting it from the provider's margin, recording it as accounts receivable and offsetting it against subsequent revenue, or directly truncating it to zero and recording the abnormal event, while simultaneously notifying the provider. In this way, the economic realization of the auction mechanism can be achieved through the actual flow of resource exchange data, and subsequent credit management and risk control can be supported.

[0211] This application embodiment obtains task requests and the processing success probability and processing consumption parameters generated by each large model provider based on the task requests; obtains the task baseline coefficient corresponding to the task request, and calculates the corresponding scheduling priority value for each large model provider by combining the task baseline coefficient, the corresponding processing success probability, and the processing consumption parameters. The scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameters. The task request is assigned to the target large model provider with the highest scheduling priority value for processing to obtain the task processing result. The task processing result is evaluated to obtain an evaluation coefficient. A reference large model provider with a scheduling priority value lower than the target large model provider is determined, and the result exchange coefficient corresponding to the target large model provider is calculated based on the reference scheduling priority value and evaluation coefficient of the reference large model provider. The result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value. Based on the result exchange coefficient, the corresponding resource exchange data is returned to the target large model provider. In this way, the responsibility for prediction can be transferred to the provider of private information (i.e., the provider of the large model) in advance, which can reduce the centralized prediction burden of the platform and the bias caused by information asymmetry. Furthermore, a closed-loop incentive mechanism is formed by linking the post-evaluation coefficient with the reference scheduling priority value, which can effectively suppress the provider's motivation to falsely report the probability of meeting the target. Specifically, in the allocation phase, this application can directly calculate the scheduling priority value based on the processing success probability and processing consumption parameters reported by the large model provider. This eliminates the need for the platform to maintain a centralized quality predictor across models, thus avoiding prediction bias caused by information asymmetry. In the settlement phase, the result exchange coefficient depends on both the post-evaluation coefficient and the reference scheduling priority value. This ensures that the final benefit of the target large model provider is constrained by both the actual output quality (evaluation coefficient) and the remaining level of the second-best competitors (reference scheduling priority value). When a provider falsely reports its processing success probability, it may increase its winning probability, but the post-evaluation coefficient often decreases due to the actual output not meeting the requirements. Simultaneously, the competitive pressure created by the reference scheduling priority value further compresses the space for falsely reported benefits, guiding the provider's optimal bidding strategy towards its true efficiency. Ultimately, this ensures that task requests are allocated to the large model provider with the largest actual remaining resources. In summary, this application can improve the accuracy and efficiency of routing resource allocation.

[0212] In some implementations, in order to facilitate a comprehensive understanding of the technical solutions of this application, an overall embodiment based on the solutions of this application is described below.

[0213] First, the platform (Buyer / Task Center) can receive a user-submitted task request consisting of a string of 6 characters long, composed of the numbers 1, 2, and 3, and determine the value (processing cost coefficient) of this task request as 10. The platform has a built-in ex-post evaluator, which has inherent evaluation errors. For example, its actual pass probability for judging task completion quality is 0.7, while the platform's observable predicted pass probability is only 0.6. This evaluation error constitutes the platform's hidden information.

[0214] Specifically, multiple large model providers (Sellers / LLM Providers) participated in this auction. Each large model provider internally employs an ex-ante predictor, which estimates its probability of completing the task (processing success probability) and cost (processing consumption parameters) based on proprietary information (such as model hidden ability, task difficulty, etc.). However, due to the pre-prediction noise inherent in large model providers, their reported processing success probability may deviate from their actual capabilities.

[0215] Furthermore, after collecting bids from all providers, the platform can calculate the remaining scores (i.e., scheduling priority values) of each party based on observable report information, and sort them according to their scores (e.g., 4.0 > 3.5 > 2.3), selecting the highest scorer (i.e., 4.0) as the target large model provider. After the target large model provider executes its task, the platform can use a post-evaluator to perform a noisy evaluation of the output results, obtaining an evaluation coefficient. Finally, the platform calculates the result exchange coefficient for the target evaluation coefficient based on the evaluation coefficient and the second-highest score (referencing the scheduling priority value, such as 3.5), for example, calculating 6.5, and completing the settlement of resource exchange data. Through the above closed-loop process of bidding-allocation-execution-evaluation-payment, this application achieves incentive-compatible task scheduling and resource allocation under the condition of dual errors (pre-prediction error and post-evaluation error).

[0216] Please see Figure 3 This application also provides an error-aware multi-model scheduling device, which can implement the above-mentioned error-aware multi-model scheduling method. The error-aware multi-model scheduling device includes:

[0217] The acquisition module 31 is used to acquire task requests, as well as the processing success rate and processing consumption parameters of each large model object generated based on the task request;

[0218] The calculation module 32 is used to obtain the task baseline coefficient corresponding to the task request, and provide an object for each large model. Combining the task baseline coefficient, the corresponding processing success probability and processing consumption parameters, the corresponding scheduling priority value is calculated. The scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameters.

[0219] The allocation module 33 is used to allocate task requests to the target large model object with the highest scheduling priority value for processing in order to obtain the task processing results;

[0220] Evaluation module 34 is used to evaluate the task processing results and obtain evaluation coefficients;

[0221] The determination module 35 is used to determine the reference large model providing object whose scheduling priority value is lower than that of the target large model providing object, and to calculate the result exchange coefficient corresponding to the target large model providing object based on the reference scheduling priority value and evaluation coefficient of the reference large model providing object. The result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value.

[0222] Return module 36 is used to provide the target large model with the corresponding resource exchange data based on the result exchange coefficient.

[0223] The specific implementation of this error-aware multi-model scheduling device is basically the same as the specific embodiment of the error-aware multi-model scheduling method described above, and will not be repeated here. Subject to meeting the requirements of the embodiments of this application, the error-aware multi-model scheduling device may also be equipped with other functional modules to implement the error-aware multi-model scheduling method in the above embodiments.

[0224] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned error-aware multi-model scheduling method. This computer device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0225] Please see Figure 4 , Figure 4 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes:

[0226] The processor 41 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0227] The memory 42 can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 42 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 42 and is called and executed by the processor 41 using the error-aware multi-model scheduling method of the embodiments of this application.

[0228] Input / output interface 43 is used to implement information input and output;

[0229] The communication interface 44 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0230] Bus 45 transmits information between various components of the device (e.g., processor 41, memory 42, input / output interface 43, and communication interface 44);

[0231] The processor 41, memory 42, input / output interface 43 and communication interface 44 are connected to each other within the device via bus 45.

[0232] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described error-aware multi-model scheduling method.

[0233] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0234] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0235] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0236] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0237] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0238] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0239] It should be understood that in this application, "at least one" and "several" refer to one or more, and "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0240] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0241] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0242] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0243] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0244] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A multi-model scheduling method based on error awareness, characterized in that, The method includes: Obtain a task request and determine the task description information and task baseline coefficient corresponding to the task request; send the task description information and task baseline coefficient to each large model providing object to obtain the processing success probability and processing consumption parameters generated by each large model providing object based on the task description information and task baseline coefficient; Obtain the task baseline coefficient corresponding to the task request, and for each large model, provide an object, and obtain the expected exchange coefficient based on the product between the task baseline coefficient and the processing success probability; obtain the scheduling priority value of the corresponding large model provided object based on the difference between the expected exchange coefficient and the processing consumption parameter, wherein the scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameter; The task request is assigned to the target large model object with the highest scheduling priority value for processing, so as to obtain the task processing result; The results of the task processing are evaluated to obtain evaluation coefficients; A reference large model providing object is determined whose scheduling priority value is lower than that of the target large model providing object. The actual exchange coefficient is calculated based on the product of the task baseline coefficient and the evaluation coefficient of the task request. The result exchange coefficient corresponding to the target large model providing object is calculated based on the difference between the actual exchange coefficient and the reference scheduling priority value corresponding to the reference large model providing object. The result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value. Based on the exchange coefficients, the target large model is provided with corresponding resource exchange data.

2. The error-aware multi-model scheduling method according to claim 1, characterized in that, The step of obtaining the task baseline coefficient corresponding to the task request includes: Obtain the access level identifier of the user object corresponding to the task request, and the task type identifier corresponding to the task request; Based on the access level identifier, the task level coefficient corresponding to the task request is determined from a preset level mapping table; Based on the task type identifier, the task type coefficient corresponding to the task request is determined from a preset task type mapping table; Obtain the task description information corresponding to the task request, and determine the task complexity coefficient corresponding to the task request based on the task description information; Based on the task level coefficient, the task type coefficient, and the task complexity coefficient, calculate the task baseline coefficient corresponding to the task request.

3. The error-aware multi-model scheduling method according to claim 1, characterized in that, After obtaining the task baseline coefficient corresponding to the task request, the method further includes: Determine the preceding historical task requests and obtain the allocation time interval between the historical task requests and the task request; Based on the allocated time interval, determine the time decay coefficient; Obtain the cumulative number of unprocessed task requests in the queue, and determine the cumulative compensation coefficient based on the cumulative number of tasks; Based on the time decay coefficient and the cumulative compensation coefficient, the task baseline coefficient is adjusted to obtain the target task baseline coefficient; Then, for each large model, an object is provided, and the corresponding scheduling priority value is calculated by combining the task baseline coefficient, the corresponding processing success probability, and the processing consumption parameter, including: For each large model, an object is provided, and the corresponding scheduling priority value is calculated by combining the target task baseline coefficient, the corresponding processing success probability, and the processing consumption parameter.

4. The error-aware multi-model scheduling method according to claim 1, characterized in that, The evaluation of the task processing results to obtain evaluation coefficients includes: Obtain the task description information corresponding to the task request, and encode the task description information and the task processing result to generate input data; The input data is fed into a pre-trained evaluation model to obtain the evaluation coefficients corresponding to the task processing results.

5. A multi-model scheduling device based on error awareness, characterized in that, The device includes: The acquisition module is used to acquire task requests and determine the task description information and task baseline coefficients corresponding to the task requests; and send the task description information and task baseline coefficients to each large model providing object to obtain the processing success probability and processing consumption parameters generated by each large model providing object based on the task description information and task baseline coefficients. The calculation module is used to obtain the task baseline coefficient corresponding to the task request, and for each large model providing object, to obtain the expected exchange coefficient based on the product between the task baseline coefficient and the processing success probability; and to obtain the scheduling priority value of the corresponding large model providing object based on the difference between the expected exchange coefficient and the processing consumption parameter, wherein the scheduling priority value is positively correlated with the processing success probability and negatively correlated with the processing consumption parameter. The allocation module is used to allocate the task request to the target large model object with the highest scheduling priority value for processing, so as to obtain the task processing result; The evaluation module is used to evaluate the task processing results and obtain evaluation coefficients; The determination module is used to determine a reference large model providing object whose scheduling priority value is lower than that of the target large model providing object, and to calculate the actual exchange coefficient based on the product of the task baseline coefficient and the evaluation coefficient of the task request; and to calculate the result exchange coefficient corresponding to the target large model providing object based on the difference between the actual exchange coefficient and the reference scheduling priority value corresponding to the reference large model providing object, wherein the result exchange coefficient is positively correlated with the evaluation coefficient and negatively correlated with the reference scheduling priority value; The return module is used to provide the target large model with the corresponding resource exchange data based on the result exchange coefficient.

6. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the error-aware multi-model scheduling method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the error-aware multi-model scheduling method according to any one of claims 1 to 4.

Images