A budget-driven token request scheduling method and system

Abstract

This invention discloses a budget-driven term request scheduling method and system, which achieves refined and automated management of enterprise-level AI call costs per unit cost while ensuring the effectiveness of multi-vendor large model calls. The method includes: configuring call parameters autonomously before initiating a call; automatically calculating and providing multiple differentiated call allocation schemes for users to choose from based on user funding constraints and model configurations; generating a unique private key for each user and strongly binding this unique private key to the user-selected call allocation scheme and the underlying routing rules; calculating the context length and threshold of a single call in real time and dynamically routing to the corresponding model; monitoring the fund consumption status in real time and executing corresponding control or blocking actions based on tiered thresholds; dynamically adjusting thresholds daily based on actual consumption to ensure controllable daily budget; step S7: at the end of the cycle, settlement and archiving.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and cloud computing resource management technology, specifically to a budget-driven lexical request scheduling method and system. Background Technology

[0002] In recent years, with the rapid development of Large Language Model (LLM) technology, more and more enterprises have begun to introduce multi-vendor large model APIs into their internal operations for unified access, in order to improve business efficiency and intelligence. Under this trend, AI gateways or model aggregation platforms have emerged, becoming the core hub for enterprises to manage and access AI resources.

[0003] However, existing AI gateways or model aggregation platforms still generally suffer from the following technical problems in practical applications, especially in scenarios aimed at refined cost control within enterprises: First, existing AI gateways lack the capability for token (term) invocation planning tailored to individual or user-level budgets. In multi-model aggregation scenarios, the token billing standards for models from different vendors and with varying capabilities differ significantly. Existing systems often only provide invocation interfaces and cannot automatically match and generate multiple cost-effectiveness balancing scheduling schemes based on the user's total budget. This makes it impossible for users to estimate the total available resources before use, hindering them from making reasonable plans between "ease of use" and "cost-effectiveness."

[0004] Secondly, in existing technologies, API keys are typically used only as authentication credentials and are not strongly linked to specific budget plans or calling schemes. Once a user obtains the private key, they can often arbitrarily call high-cost models within their authorized scope. This can easily lead to cross-model calls, causing actual costs to far exceed the expected budget and resulting in uncontrolled AI calling costs for enterprises.

[0005] Secondly, current gateway systems have relatively rigid scheduling mechanisms, lacking dynamic cost assessment and scheduling strategies for the two billing models provided by AI vendors: "Coding Plan (fixed-quota package pool)" and "pay-as-you-go API pool." In scenarios involving long texts or high-frequency conversations, the system cannot perform real-time calculations and route switching based on the context length of a single interaction and the equivalent unit price of the two billing models. This results in either the package quota being quickly exhausted by a small amount of long text, or the package advantages not being fully utilized in high-frequency short conversations, failing to achieve intelligent routing based on the principle of cost optimization.

[0006] Finally, existing platforms offer coarse-grained budget monitoring, often only providing post-event billing statistics or simple total alerts. They lack multi-level early warnings based on real-time consumption, dynamic rate limiting, and mandatory limit / disable mechanisms. This lagging monitoring approach cannot promptly block requests the moment overspending occurs (such as during high-concurrency calls or malicious consumption), easily leading to overspending and unnecessary financial losses for businesses. Summary of the Invention

[0007] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0008] The purpose of this invention is to solve the above problems and provide a budget-driven lexical request scheduling method and system, which achieves refined and automated management of enterprise-level AI call costs per unit cost while ensuring the effectiveness of multi-vendor large model calls.

[0009] The technical solution of this invention is as follows: This invention discloses a budget-driven lexical request scheduling method, the method comprising: Step S1: Configure the call parameters before initiating the call; Step S2: Based on the user's funding constraints and model configuration, automatically calculate and provide multiple differentiated call allocation schemes for the user to choose from; Step S3: Generate a unique private key for each user, and strongly bind the unique private key to the call allocation scheme selected by the user in step S2 and the underlying routing scheme rules; Step S4: Calculate the context length and threshold of a single call in real time, and dynamically route to the corresponding model; Step S5: Monitor the fund consumption status in real time and execute corresponding control or blocking actions based on tiered thresholds; Step S6: Dynamically adjust the threshold daily based on actual consumption to ensure that the daily budget is controllable; Step S7: End of cycle, settlement and archiving.

[0010] According to an embodiment of the budget-driven lexical request scheduling method of the present invention, the parameters configured autonomously in step S1 include: total periodic budget, usage period, and multiple large model service channels selected autonomously.

[0011] According to an embodiment of the budget-driven lexical request scheduling method of the present invention, multiple large model service channels include models with different billing and performance gradients, further including: a high-end on-demand API model, a mid-range on-demand API model, and a fixed-number fallback model.

[0012] According to an embodiment of the budget-driven lexical request scheduling method of the present invention, multiple differentiated call allocation schemes include effect priority, balanced general, and extreme cost control. Based on the set total periodic budget, the comprehensive converted unit price, daily available lexical quantity, and monthly / period total estimated lexical usage under each call allocation scheme are calculated.

[0013] According to an embodiment of the budget-driven lexical request scheduling method of the present invention, the critical value of the context length of a single call. for: , in, The total price of the package for the fixed-number guaranteed model. The API pay-as-you-go pricing for vendor-specific terms. This is a budget adjustment factor used to adapt to the cost control needs of different users.

[0014] According to an embodiment of the budget-driven lexical request scheduling method of the present invention, the dynamic switching rule in dynamic routing is as follows: when In this case, priority is given to routing to the fixed-number fallback model. ;when If the remaining credit for the fixed-number backup plan is insufficient, the system will automatically switch to a lexical API call, or switch to a lower-priced plan from another vendor. The context length for a single call. The total cost of using the lexical API in a single call. The equivalent cost of using a fixed-number fallback model for a single call.

[0015] According to an embodiment of the budget-driven lexical request scheduling method of the present invention, automated intervention is performed according to a set multi-level threshold strategy: when the lexical usage reaches the first threshold set by the budget, a front-end message reminder is triggered; when the lexical usage reaches the second threshold set by the budget, soft rate limiting is initiated; when the lexical usage reaches the third threshold set by the budget, a forced downgrade to the lowest cost model is initiated; when the lexical usage reaches or exceeds 100% of the budget, the user's private key is automatically disabled to reject subsequent calls, while high-frequency abnormal calls are temporarily frozen and alerted.

[0016] This invention also discloses a budget-driven lexical request scheduling system, the system comprising: The call parameter configuration module is used to configure the call parameters before the call is initiated. The differentiated scheduling scheme generation module is used to automatically calculate and provide multiple differentiated call allocation schemes for users to choose from based on the user's financial constraints and model configuration; The private key-scheme-route strong binding module is used to generate a unique private key for each user and to strongly bind this unique private key to the call allocation scheme selected by the user in the differentiated scheduling scheme generation module and the underlying routing scheme rules. The dynamic routing module is used to calculate the context length and threshold of a single call in real time and dynamically route to the corresponding model. The monitoring and intervention module is used to monitor the status of fund consumption in real time and execute corresponding control or blocking actions based on tiered thresholds. The daily budget threshold dynamic calibration module is used to dynamically adjust the threshold according to actual consumption every day to ensure that the daily budget is controllable. The data archiving module is used to settle accounts and archive data at the end of the period.

[0017] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the budget-driven lexical request scheduling method described above.

[0018] The present invention also discloses a computer program product that, when executed by a processor, implements the steps of the budget-driven lexical request scheduling method described above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: the present invention achieves budget planning, scheme selection, usage monitoring and over-limit blocking through budget-driven scheme generation, private key strong binding routing, dynamic switching of token pool and multi-level threshold monitoring mechanism.

[0020] Specifically, the present invention addresses the problems existing in the background art and has the following technical effects.

[0021] I. Regarding the issue of "lacking budget-oriented planning capabilities and being unable to generate cost-effectiveness solutions": The system receives the user's total periodic budget, usage period, and at least three selected major model service channels (including high-end on-demand API model, mid-range on-demand API model, and Coding Plan fixed-time fallback model) from the front end. Based on the above basic parameters, the unit price of each model's input / output tokens, and empirical ratios, the system automatically generates three differentiated scheduling schemes: "Effect Priority," "Balanced General," and "Ultimate Cost Control," and calculates the daily available token amount and the monthly total estimated token usage under each scheme.

[0022] This enables "budget planning and solution selection," breaking away from the previous model of "blindly calling and calculating costs afterward." It allows users to intuitively see the AI ​​workload (estimated total token amount) under different budget ratios before the call occurs, thus enabling them to make the most scientific plan that best suits their current budget based on actual business needs, balancing "model intelligence level" and "call cost."

[0023] II. Regarding the issue of "private keys not being bound to the invocation scheme, which easily leads to cross-model over-budget invocations": The AI ​​gateway generates a unique private key for each user and strongly binds this unique private key to the user's self-selected calling scheme (such as performance priority or extreme cost control) and the underlying routing scheme rules, and calculates and constrains it in real time based on the daily average cost.

[0024] This achieves "route binding and cost penetration prevention." This strong binding feature of "private key-scheme-route" ensures that a specific API key can only circulate within the established cost rules and model pool. It effectively prevents users or subsystems from arbitrarily exceeding their authority to call high-priced models after obtaining the private key, thus precisely implementing cost control at the individual or project level.

[0025] III. Regarding the issue that "the vendor package pool and the pay-as-you-go API pool lack a dynamic scheduling strategy, making it impossible to achieve optimal cost": The system establishes a method based on the length of a single call context. With critical value The dynamic switching algorithm. Among them, This threshold is calculated based on the cost balance principle of the vendor's Token API pay-as-you-go pricing and the Coding Plan's package pricing (i.e., when their costs are equal), combined with a budget adjustment factor. Each time it is called, if... If the request is not accepted, it will be routed to the Coding Plan package first; if... If the balance of the package is insufficient, it will automatically switch to another package with a lower unit price via TokenAPI call.

[0026] This achieves "dynamic optimal scheduling of the token pool," enabling the system to perform "intelligent accounting." It effectively avoids the risk of long text conversations instantly draining the fixed-price plan, while maximizing the use of the plan's low-price advantage in high-frequency short text conversations. Through real-time dynamic switching, it ensures that every API request travels through the "most cost-effective" billing channel.

[0027] IV. Regarding the issue of "coarse-grained budget monitoring, lack of multi-level early warning and intervention mechanisms, which easily leads to uncontrolled costs": The system provides real-time statistics on user token consumption, call count, and cumulative costs, and employs a tiered, multi-level threshold execution strategy: a front-end message notification is triggered when usage reaches 50%; soft rate limiting (limiting concurrency and single-call length) is implemented at 80%; a forced downgrade to the lowest-cost model is applied at 95%; and private keys are automatically disabled and calls are rejected at 100%. Simultaneously, abnormal call detection enables temporary freezing and alerts.

[0028] This achieves "monitorable usage and blocking of over-limits," transforming the previously crude post-event monitoring into refined and automated in-event intervention. Through four lines of defense—"alert-rate limiting-degradation-circuit breaking"—it completely eliminates the possibility of AI bill explosions caused by sudden traffic spikes, malicious calls, or code infinite loops, ensuring system stability while achieving true cost mitigation. Attached Figure Description

[0029] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0030] Figure 1 A flowchart of an embodiment of the budget-driven lexical request scheduling method of the present invention is shown.

[0031] Figure 2 A schematic diagram of an embodiment of the budget-driven lexical request scheduling system of the present invention is shown.

[0032] Figure 3 A schematic diagram of the dynamic switching rules of the present invention is shown. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0034] Figure 1The flowchart of an embodiment of the budget-driven lexical request scheduling method of the present invention is shown. Please refer to... Figure 1 The implementation steps of the method in this embodiment are described in detail below.

[0035] Step S1: User configuration: total budget for the period, number of days used, and selection of multiple model channels.

[0036] Before initiating a call, users can configure the call cycle, funding constraints, and the model service pool they wish to use through the front-end interface.

[0037] The system receives three basic parameters configured by the user on the front end: 1. Total periodic budget; 2. Usage period (usually in calendar days); 3. Three or more selected large model service channels. The selected model channels must include models with different billing and performance tiers, specifically covering high-end pay-as-you-go API models, mid-range pay-as-you-go API models, and the foundational Coding Plan fixed-time fallback model. Pay-as-you-go APIs are similar to "ordering food on demand," while Coding Plans are similar to a "monthly buffet," which can be understood as a fixed-rate programming plan.

[0038] Step S2: Automatically generate multiple scheduling schemes, such as the three schemes of effect priority / balanced / cost control, and calculate the total amount of tokens.

[0039] Based on the user's funding constraints and model configuration, the system automatically calculates and provides multiple differentiated call allocation schemes for the user to choose from.

[0040] The system combines the actual unit price of each model's input / output tokens with the average token ratio of each model's input / output in the enterprise's historical calls (e.g., the default 2:1) to automatically generate three differentiated scheduling allocation schemes: "Effect Priority," "Balanced General Use," and "Ultimate Cost Control." Subsequently, based on the set total periodic budget, the system calculates the comprehensive converted unit price, the daily available token quantity, and the monthly / period total estimated token usage for each scheme.

[0041] For example, see below.

[0042] Default rule: Input Token : Output Token = 2 : 1; The unit price per thousand tokens for each model is as follows: 1. High-end model (GLM-5.0): Input 0.0015 yuan / thousand tokens, output 0.006 yuan / thousand tokens; 2. Mid-range model (Qwen-3.6): Input 0.0008 yuan / thousand tokens, output 0.003 yuan / thousand tokens; 3. Coding Plan safety net model (GLM-4.7): 0.0005 yuan / thousand tokens.

[0043] Calculations are performed using example parameters: Total monthly budget = 500 yuan, number of days in the month = 31 days; Average daily budget = 500 ÷ 31 ≈ 16.13 yuan.

[0044] Configuration plan: Option A (Prioritizing Results): High-end models 70%, mid-range models 20%, Coding Plan package 10%; Option B is a balanced approach: 40% high-end models, 40% mid-range models, and 20% Coding Plan packages. Option C - Ultimate Cost Control: High-end models 10%, mid-range models 30%, Coding Plan package 60%; but: Option A (Prioritizing Results): Overall price per token: 0.002456 yuan / thousand tokens; Estimated total monthly tokens: 203.6 million. Option B (Balanced and Universal): Overall unit price 0.001912 yuan / thousand tokens, estimated total monthly tokens: 261.5 million; Option C: Ultimate Cost Control: Average unit price 0.001059 yuan / thousand tokens, estimated total monthly tokens: 472.1 million.

[0045] Step S3: The AI ​​gateway generates a unique private key for the user, which is strongly bound to the solution in step S2.

[0046] The system generates a calling credential for each user and strictly limits the calling rules and available budget of the credential in the underlying logic.

[0047] The cloud-native AI gateway generates a unique private key for each user. The system then strongly binds this unique private key to the specific invocation scheme (such as the "balanced general" scheme) selected by the user in step S2, as well as the underlying routing rules. This mechanism ensures that every request initiated with this private key must strictly adhere to the model allocation and total budget limits defined in the predetermined scheme.

[0048] Step S4: Calculate the context length in real time With critical value Dynamically route APIs / Coding Plans from various vendors.

[0049] When a single call is triggered, the system intelligently selects the most cost-effective request channel based on the text length and the cost balance point of different billing modes.

[0050] Each time a call is initiated, the system extracts the context length of the call in real time. (Unit: Token), and calculate the context length threshold. ( The calculation logic is based on the cost balance principle, that is, the cost of a single API call. Equivalent cost of Coding Plan The length of time can be supplemented by a budget fluctuation adjustment factor. ).

[0051] The specific calculation method is as follows: , = Vendor Token API pay-as-you-go pricing (unit: yuan / Token) = Manufacturer Coding Plan Package Unit Price (Unit: Yuan / Token, calculated as total package price ÷ total package token amount) , , The formula for calculating the cost of a single call is as follows: , .

[0052] Context length threshold The calculation method is as follows (based on the cost balance principle, i.e.) = The context length can be adjusted by a factor based on budget fluctuations in practical applications. , ∈[0.8,1.2]) , in, The total price of the Coding Plan package (unit: yuan). This is a budget adjustment factor used to adapt to the cost control needs of different users. When the value is less than 1, a coding plan is preferred. When the value is greater than 1, API calls can be appropriately relaxed.

[0053] refer to Figure 3 As shown, the dynamic switching rules based on the calculation results are as follows: When At this time, priority is given to routing to the Coding Plan package to achieve the best cost. ;when If the remaining credit in the Coding Plan is insufficient, the system will automatically switch to a Token API call or switch to a lower-priced plan from another provider to prevent long text messages from exhausting the credit limit.

[0054] Step S5: Real-time statistics of cost percentage, and execution of multi-level warnings / rate limiting / disabling.

[0055] The system continuously monitors the status of fund consumption and executes corresponding control or blocking actions based on tiered thresholds.

[0056] The system performs real-time statistics on token consumption, call count, and cumulative costs under a user's private key. It executes automated interventions based on a multi-level threshold strategy: for example, when usage reaches 50% of the budget (first threshold), a front-end message notification is triggered; when usage reaches 80% of the budget (second threshold), soft rate limiting is initiated (limiting concurrency and the maximum token length per call); when usage reaches 95% (third threshold), a forced downgrade to the lowest cost model is implemented to further control consumption; when usage reaches or exceeds 100% of the budget, the user's private key is automatically disabled to reject subsequent calls, preventing excessive expenditures, while high-frequency abnormal calls are temporarily frozen and alerted.

[0057] Step S6: Adjust the threshold dynamically based on actual consumption each day to ensure that the daily budget is controllable.

[0058] The system distributes the total periodic budget across the daily dimension and dynamically fine-tunes the daily control level based on the actual daily consumption.

[0059] The system calculates the "daily average budget" by dividing the total budget for the period by the number of days used. In actual operation, the system performs real-time calculations based on the daily average cost. If the actual token consumption fluctuates on a particular day, the system will dynamically adjust the trigger thresholds for soft rate limiting and degradation the following day or in real time. This fine-grained daily average control ensures stable budget consumption throughout the entire usage period, preventing early overspending from causing later shutdowns.

[0060] Step S7: End of cycle, settlement and archiving.

[0061] Once the set usage time limit is reached, the system performs closed-loop processing on the call bill for this period.

[0062] When the actual usage days reach the user-configured usage period, the system stops the current budget deduction process and accurately calculates the actual token consumption and costs for each vendor's API and Coding Plan within that period. The system generates the final periodic bill and archives the data, then releases the private key lock, awaiting the user to configure parameters for the next period and restart the entire process.

[0063] Figure 2 The budget-driven lexical request scheduling system of the present invention is illustrated. (Reference) Figure 2 The system in this embodiment includes: a call parameter configuration module, a differentiated scheduling scheme generation module, a private key-scheme-route strong binding module, a dynamic routing module, a monitoring and intervention module, a daily budget threshold dynamic calibration module, and a data archiving module.

[0064] The call parameter configuration module is used to configure the call parameters before the call is initiated.

[0065] The system receives three basic parameters configured by the user on the front end: 1. Total periodic budget; 2. Usage period (usually in calendar days); 3. Three or more selected large model service channels. The selected model channels must include models with different billing and performance tiers, specifically covering high-end pay-as-you-go API models, mid-range pay-as-you-go API models, and the foundational Coding Plan fixed-time fallback model. Pay-as-you-go APIs are similar to "ordering food on demand," while Coding Plans are similar to a "monthly buffet," which can be understood as a fixed-rate programming plan.

[0066] The differentiated scheduling scheme generation module is used to automatically calculate and provide multiple differentiated call allocation schemes for users to choose from, based on the user's financial constraints and model configuration.

[0067] The system combines the actual unit price of each model's input / output tokens with the average token ratio of each model's input / output in the enterprise's historical calls (e.g., the default 2:1) to automatically generate three differentiated scheduling allocation schemes: "Effect Priority," "Balanced General Use," and "Ultimate Cost Control." Subsequently, based on the set total periodic budget, the system calculates the comprehensive converted unit price, the daily available token quantity, and the monthly / period total estimated token usage for each scheme.

[0068] The Private Key-Scheme-Route Strong Binding module is used to generate a unique private key for each user and to strongly bind this unique private key to the call allocation scheme selected by the user in the Differentiated Scheduling Scheme Generation module and the underlying routing scheme rules.

[0069] The system generates a calling credential for each user and strictly limits the calling rules and available budget of the credential in the underlying logic.

[0070] The cloud-native AI gateway generates a unique private key for each user. The system then strongly binds this unique private key to the specific invocation scheme (such as the "balanced general" scheme) selected by the user in step S2, as well as the underlying routing rules. This mechanism ensures that every request initiated with this private key must strictly adhere to the model allocation and total budget limits defined in the predetermined scheme.

[0071] The dynamic routing module is used to calculate the context length and threshold of a single call in real time and dynamically route the call to the corresponding model.

[0072] When a single call is triggered, the system intelligently selects the most cost-effective request channel based on the text length and the cost balance point of different billing modes.

[0073] Each time a call is initiated, the system extracts the context length of the call in real time. (Unit: Token), and calculate the context length threshold. ( The calculation logic is based on the cost balance principle, that is, the cost of a single API call. Equivalent cost of Coding Plan The length of time can be supplemented by a budget fluctuation adjustment factor. ).

[0074] The specific calculation method is as follows: , = Vendor Token API pay-as-you-go pricing (unit: yuan / Token) = Manufacturer Coding Plan Package Unit Price (Unit: Yuan / Token, calculated as total package price ÷ total package token amount) , , The formula for calculating the cost of a single call is as follows: , .

[0075] Context length threshold The calculation method is as follows (based on the cost balance principle, i.e.) = The context length can be adjusted by a factor based on budget fluctuations in practical applications. , ∈[0.8,1.2]) , in, The total price of the Coding Plan package (unit: yuan). This is a budget adjustment factor used to adapt to the cost control needs of different users. When the value is less than 1, a coding plan is preferred. When the value is greater than 1, API calls can be appropriately relaxed.

[0076] refer to Figure 3 As shown, the dynamic switching rules based on the calculation results are as follows: When At this time, priority is given to routing to the Coding Plan package to achieve the best cost. ;when If the remaining credit in the Coding Plan is insufficient, the system will automatically switch to a Token API call or switch to a lower-priced plan from another provider to prevent long text messages from exhausting the credit limit.

[0077] The monitoring and intervention module is used to monitor the status of fund consumption in real time and execute corresponding control or blocking actions based on tiered thresholds.

[0078] The system continuously monitors the status of fund consumption and executes corresponding control or blocking actions based on tiered thresholds.

[0079] The system performs real-time statistics on token consumption, call count, and cumulative costs under a user's private key. It executes automated interventions based on a multi-level threshold strategy: for example, when usage reaches 50% of the budget (first threshold), a front-end message notification is triggered; when usage reaches 80% of the budget (second threshold), soft rate limiting is initiated (limiting concurrency and the maximum token length per call); when usage reaches 95% (third threshold), a forced downgrade to the lowest cost model is implemented to further control consumption; when usage reaches or exceeds 100% of the budget, the user's private key is automatically disabled to reject subsequent calls, preventing excessive expenditures, while high-frequency abnormal calls are temporarily frozen and alerted.

[0080] The daily budget threshold dynamic calibration module is used to dynamically adjust the threshold daily based on actual consumption, ensuring that the daily budget is controllable.

[0081] The system distributes the total periodic budget across the daily dimension and dynamically fine-tunes the daily control level based on the actual daily consumption.

[0082] The system calculates the "daily average budget" by dividing the total budget for the period by the number of days used. In actual operation, the system performs real-time calculations based on the daily average cost. If the actual token consumption fluctuates on a particular day, the system will dynamically adjust the trigger thresholds for soft rate limiting and degradation the following day or in real time. This fine-grained daily average control ensures stable budget consumption throughout the entire usage period, preventing early overspending from causing later shutdowns.

[0083] The data archiving module is used to settle accounts and archive data at the end of the period.

[0084] Once the set usage time limit is reached, the system performs closed-loop processing on the call bill for this period.

[0085] When the actual usage days reach the user-configured usage period, the system stops the current budget deduction process and accurately calculates the actual token consumption and costs for each vendor's API and Coding Plan within that period. The system generates the final periodic bill and archives the data, then releases the private key lock, awaiting the user to configure parameters for the next period and restart the entire process.

[0086] Furthermore, this invention also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the following... Figure 1 The steps of an embodiment of the budget-driven lexical request scheduling method are shown.

[0087] This invention also discloses a computer program product, which, when executed by a processor, implements the following: Figure 1 The steps of an embodiment of the budget-driven lexical request scheduling method are shown.

[0088] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0089] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0090] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0091] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0092] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0093] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A budget-driven lexical request scheduling method, characterized in that, The methods include: Step S1: Configure the call parameters before initiating the call; Step S2: Based on the user's funding constraints and model configuration, automatically calculate and provide multiple differentiated call allocation schemes for the user to choose from; Step S3: Generate a unique private key for each user, and strongly bind the unique private key to the call allocation scheme selected by the user in step S2 and the underlying routing scheme rules; Step S4: Calculate the context length and threshold of a single call in real time, and dynamically route to the corresponding model; Step S5: Monitor the fund consumption status in real time and execute corresponding control or blocking actions based on tiered thresholds; Step S6: Dynamically adjust the threshold daily based on actual consumption to ensure that the daily budget is controllable; Step S7: End of cycle, settlement and archiving.

2. The budget-driven lexical request scheduling method according to claim 1, characterized in that, The parameters that can be configured independently in step S1 include: total periodic budget, usage period, and multiple large model service channels that can be selected independently.

3. The budget-driven lexical request scheduling method according to claim 2, characterized in that, The various large-scale model service channels include models with different billing and performance tiers, further including: high-end pay-as-you-go API model, mid-range pay-as-you-go API model, and fixed-number fallback model.

4. The budget-driven lexical request scheduling method according to claim 1, characterized in that, Multiple differentiated call allocation schemes include performance-first, balanced and universal, and extreme cost control. Based on the set total periodic budget, the comprehensive converted unit price, daily available keyword quantity, and monthly / period total estimated keyword usage are calculated for each call allocation scheme.

5. The budget-driven lexical request scheduling method according to claim 3, characterized in that, Critical value for context length in a single call for: ， in, The total price of the package for the fixed-number guaranteed model. The API pay-as-you-go pricing for vendor-specific terms. This is a budget adjustment factor used to adapt to the cost control needs of different users.

6. The budget-driven lexical request scheduling method according to claim 5, characterized in that, The dynamic switching rules in dynamic routing are as follows: When In this case, priority is given to routing to the fixed-number fallback model. ;when If the remaining credit for the fixed-number backup plan is insufficient, the system will automatically switch to a lexical API call, or switch to a lower-priced plan from another vendor. The context length for a single call. The total cost of using the lexical API in a single call. The equivalent cost of using a fixed-number fallback model for a single call.

7. The budget-driven lexical request scheduling method according to claim 1, characterized in that, Automated intervention is executed based on the set multi-level threshold strategy: when the usage of keywords reaches the first threshold set by the budget, a front-end message reminder is triggered; when the usage of keywords reaches the second threshold set by the budget, soft rate limiting is initiated. When the usage of a word element reaches the third threshold set in the budget, it will be forcibly downgraded to the lowest cost model; when the usage of a word element reaches or exceeds 100% of the budget, the user's private key will be automatically disabled to reject subsequent calls, and high-frequency abnormal calls will be temporarily frozen and alerted.

8. A budget-driven lexical request scheduling system, characterized in that, The system includes: The call parameter configuration module is used to configure the call parameters before the call is initiated. The differentiated scheduling scheme generation module is used to automatically calculate and provide multiple differentiated call allocation schemes for users to choose from based on the user's financial constraints and model configuration; The private key-scheme-route strong binding module is used to generate a unique private key for each user and to strongly bind this unique private key to the call allocation scheme selected by the user in the differentiated scheduling scheme generation module and the underlying routing scheme rules. The dynamic routing module is used to calculate the context length and threshold of a single call in real time and dynamically route to the corresponding model. The monitoring and intervention module is used to monitor the status of fund consumption in real time and execute corresponding control or blocking actions based on tiered thresholds. The daily budget threshold dynamic calibration module is used to dynamically adjust the threshold according to actual consumption every day to ensure that the daily budget is controllable. The data archiving module is used to settle accounts and archive data at the end of the period.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the budget-driven lexical request scheduling method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the program is executed by the processor, it implements the steps of the budget-driven lexical request scheduling method as described in any one of claims 1-7.

Images