Digital dial-based distributed API key dynamic allocation method and system

Abstract

The application discloses a kind of distributed API key dynamic allocation method and system based on digital turntable, it is related to computer network and cloud computing technical field, method includes: initialization digital turntable;Available key list in memory in key pool is obtained by hierarchical digital turntable;Current key is selected by digital turntable pointer and API request is sent using current key;When normal response, API request is received and the key of required level of request is determined, the next available key is obtained from the turntable of corresponding level, and the selected key is used to send request and monitor response;When abnormal state, dynamically handle key exception and request failure, execute temporary dormancy strategy to current abnormal key and execute intelligent retry and failover mechanism to select new key retry, break through the limitation of single API key, also through intelligent failover and adaptive optimization mechanism, ensure high throughput, high availability and efficient API access experience.

Description

Technical Field

[0001] This invention relates to the fields of computer networks and cloud computing technology, and in particular to a distributed API key dynamic allocation method and system based on a digital turntable. Background Technology

[0002] With the widespread application of artificial intelligence services, the demand for accessing AI API services, such as OpenAI's GPT and Anthropic's Claude, has increased dramatically. In large-scale application scenarios, existing technologies face the following challenges: 1. Access restriction bottleneck: AI service providers impose strict request frequency limits and concurrency limits on individual API keys. When the application scales up, a single key cannot meet the demand; 2. Availability challenges: A single API key may become unavailable due to quota exhaustion, temporary network problems, or service provider restrictions; 3. Uneven resource allocation: Different levels and types of API keys have different performance characteristics and permissions, and simple polling cannot optimize usage efficiency; 4. Lack of adaptation mechanisms: It is impossible to dynamically adjust the allocation strategy based on the real-time status and historical performance of API keys; 5. Complexity of multi-tenant environments: It is difficult to achieve fair allocation and resource isolation while sharing an API key pool.

[0003] Current solutions typically employ basic random selection or simple polling mechanisms, lacking an intelligent allocation mechanism that comprehensively considers key status, service quality, and load conditions. This makes them unable to effectively address the complex needs of large-scale AI API access scenarios and can easily lead to system instability. Summary of the Invention

[0004] This application proposes a distributed API key dynamic allocation method and system based on digital turntables. By organizing API keys into multiple virtual turntables and adopting adaptive polling and intelligent failover mechanisms, it achieves efficient and reliable API request allocation, solves the bottleneck limitations, availability challenges and resource optimization problems faced in large-scale AI API access scenarios, breaks through the limitations of a single API key, and ensures a high-throughput, high-availability and high-efficiency API access experience through intelligent failover and adaptive optimization mechanisms.

[0005] This invention provides a distributed API key dynamic allocation method based on a digital turntable, comprising: Initialize the digital turntable; The list of available keys in the key pool in memory is obtained through a hierarchical digital turntable; Select the current key using the digital turntable pointer and send the API request using the current key; During a normal response, the system receives API requests and determines the key required for the request level. It then retrieves the next available key from the corresponding level's carousel, sends the request using the selected key, and monitors the response. This allows the system to expand the number of API service calls through a hierarchical polling mechanism of the digital carousel, enabling dynamic scheduling of multiple service providers or multiple keys. In abnormal situations, the system dynamically handles key abnormalities and request failures, implements a temporary sleep policy for the current abnormal key, and employs intelligent retry and failover mechanisms to select a new key for retrying when a request fails.

[0006] Preferably, the hierarchical digital carousel corresponds to an independent carousel instance at the API key level, and each digital carousel contains multiple API keys with similar characteristics, with a cascading relationship between each digital carousel; wherein, the digital carousel is a RoundRobin structure, used to implement cyclic allocation of objects, and the structure includes: an object slice for storing all objects to be allocated; a current index for recording the index of the currently allocated object; an object length for recording the length of the current object set; and a mutex lock for ensuring concurrency safety and thread safety when adding, deleting, and allocating objects.

[0007] Preferably, the step of obtaining the list of available keys in the key pool in memory via a hierarchical digital turntable includes: Obtain all available keys and create a list of available keys; The method call function in the ProxyHandler structure is used to count the distribution of API keys at different levels, and the statistical results are stored in the relevant fields of ProxyHandler; Use temporary storage newKeys to group keys by their level attribute, iterate through all keys, aggregate keys of the same level into the same group, and call the target timer to cancel the original timer for that key. Iterate through each level group, store the PartialKey of the current key into the memory mapping validKeys of the valid keys, and call the polling mechanism to create a new poller for the current level; Assign the newly generated poller mapping newRoundRobins to the actual running mapping p.roundRobins; Update the poller mapping in the system and record the last update time.

[0008] Preferably, the method also includes dynamically adjusting user permissions, priority rollback, and round-robin scheduling based on a smart key selection strategy to achieve efficient key allocation, and further includes: Check if a poller at the specified level exists; If it does not exist, check if the highest-level poller exists. If it does, roll back and update the user level and allowed level information. Update the global-level mapping table and remove the relevant mappings from the original level; The next key is obtained sequentially using a poller. Record request statistics to the channel and return the selected key and poller status.

[0009] Preferably, the temporary dormancy policy for the current abnormal key includes: Get or initialize a timer mapping, check if a timer mapping of type key exists, and initialize a new mapping if it does not exist; Check if a timer is already running for the current key. If a timer for this key already exists, return directly without repeating the sleep setting. The safety of concurrent operations on shared resources is ensured by a mutex lock (MuK), which includes pollers and timer mappings. If the poller at the current key level has multiple keys, remove the key from the poller; if there is only one key, do not sleep to ensure service availability. Create a timer to perform the recovery operation after a preset time; During the recovery operation, check if the current key is still valid. If it is valid, rejoin the poller and remove the timer reference.

[0010] Preferably, the step of performing intelligent retry and failover mechanisms to select a new key for retrying when the request fails includes: When a request failure is detected, determine whether a retry strategy needs to be executed based on the error type and business logic. One or more retries are performed based on the exponential backoff algorithm or a strategy that limits the number of retries. If the retries fail after exceeding the threshold number, a failover will be initiated. Switch the request route to the backup service node, or read data from the backup database; After retries and failover, the request is sent again. If it still fails, the final processing operation is initiated. The final processing operation includes resetting the user's state, sending an alarm notification, releasing the user lock, and returning.

[0011] Preferably, the method also includes achieving adaptive adjustment of the model and level by dynamically adjusting the level through model cancellation frequency, and further includes: The switchOver function is triggered when the requested model is canceled. Check if there is a configured switching target for the level corresponding to the current model; The number of cancellations for the current model within a specified time is counted, and if the number of cancellations exceeds the threshold, the switching process is initiated. Check if the time of the last level switch in the current model exists and if it is within the switching cooldown period; If the switch exists and the switchover cooldown period has not expired, the switchover is skipped; if the switch does not exist or the switchover cooldown period has expired, the switchover is allowed. Call the switching function to perform the level switch, update the level mapping in the configuration, and record logs and events; The system automatically switches back to the original level after a preset time using a timer, clears the switching status, resets the cancellation count and time, and completes one switching process.

[0012] This invention also provides a distributed API key dynamic allocation system based on a digital turntable, implementing the distributed API key dynamic allocation method based on a digital turntable as described in the embodiments of this invention, including: The initialization module is used to initialize the digital turntable; The key acquisition module is used to obtain the list of available keys in the key pool in memory via a hierarchical digital turntable. The polling allocation module is used to select the current key via a digital rotary pointer and send API requests using the current key. During a normal response, the system receives API requests and determines the key required for the request level. It then retrieves the next available key from the corresponding level's carousel, sends the request using the selected key, and monitors the response. This allows the system to expand the number of API service calls through a hierarchical polling mechanism of the digital carousel, enabling dynamic scheduling of multiple service providers or multiple keys. In abnormal situations, the system dynamically handles key abnormalities and request failures, implements a temporary sleep policy for the current abnormal key, and employs intelligent retry and failover mechanisms to select a new key for retrying when a request fails.

[0013] The present invention also provides an electronic device, comprising: The memory is used to store the processing program; The processor, when executing the processing program, implements the distributed API key dynamic allocation method based on a digital turntable as described in the embodiments of the present invention.

[0014] The present invention also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the distributed API key dynamic allocation method based on a digital turntable as described in the embodiments of the present invention.

[0015] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) This invention incorporates multiple API keys or service providers into a polling pool through a hierarchical digital carousel (such as a RoundRobin structure), and distributes requests evenly to different resources. For example, if a single key is limited to 100 calls per minute, using 5 keys can extend this to 500 calls, breaking the single point limitation; different levels of carousel instances manage keys independently, supporting dynamic allocation according to business needs (such as priority and permissions), further optimizing resource utilization.

[0016] (2) When a key or service provider is abnormal (such as a failed request), the system temporarily removes it from the polling pool through a timer to avoid reusing faulty resources; failed requests trigger exponential backoff retry or switch to a backup node, and are automatically routed to a normal service provider in combination with the polling mechanism to ensure service continuity.

[0017] (3) The present invention is based on the cancellation frequency level switching, the statistical model cancels the number of times, and automatically switches the key level (such as from low level to high level) when the threshold is exceeded, and avoids frequent switching through the cooling period; if the high level key is insufficient, it can fall back to the low level poller, flexibly adapting to resource changes.

[0018] (4) This invention ensures the safety of concurrent operations of pollers and timers by using mutex locks (MuK / MuL) to prevent data competition; records the success / failure status of requests and key switching events (such as Redis list storage) to support troubleshooting and performance optimization; when adding a key or service provider, only the corresponding level of the turntable instance needs to be updated, without changing the core logic; dormant keys are automatically re-added to the polling pool after the timer expires, reducing manual intervention.

[0019] (5) This invention, based on a digital turntable-based distributed API key dynamic allocation system, effectively solves the bottleneck limitations, availability challenges, and resource optimization problems faced in large-scale AI API access scenarios through innovative key organization and allocation mechanisms. This system not only breaks through the limitations of a single API key but also ensures a high-throughput, high-availability, and high-efficiency API access experience through intelligent failover and adaptive optimization mechanisms. The system's layered design and scalable architecture enable it to adapt to application scenarios of different scales and complexities, providing key technical support for the large-scale application of AI services. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of the distributed API key dynamic allocation method based on a digital turntable according to the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0023] like Figure 1 As shown, existing solutions typically employ basic random selection or simple round-robin mechanisms, lacking an intelligent allocation mechanism that comprehensively considers key status, service quality, and load conditions, thus failing to effectively address the complex requirements of large-scale AI API access scenarios. This invention provides a distributed API key dynamic allocation method based on a digital carousel. The core of this implementation lies in selecting currently stable API services through a digital carousel, dynamically adjusting the mechanism to handle anomalies, executing failover logic, and resolving stability issues. Specifically, it includes the following steps: Initialize the digital turntable; The list of available keys in the key pool in memory is obtained through a hierarchical digital turntable; Selecting the current key using the digital carousel pointer and sending an API request using the current key can be understood as retrieving the list of available keys in the key pool in memory via the digital carousel. Sending an API request using the current key selected by the digital carousel pointer results in either a normal response or an exception, in which case the current key goes to sleep.

[0024] During a normal response, the system receives API requests and determines the key required for the request level. It then retrieves the next available key from the corresponding level's carousel, sends the request using the selected key, and monitors the response. This allows the system to expand the number of API service calls through a hierarchical polling mechanism of the digital carousel, enabling dynamic scheduling of multiple service providers or multiple keys. In abnormal states, the system dynamically handles key anomalies and request failures. It implements a temporary sleep policy for the currently abnormal key and employs intelligent retry and failover mechanisms to select a new key for retrying when a request fails. This can be understood as achieving the following: Function 1: Increased resource call volume. If an API service only allows 100 calls per minute, using a numerical polling mechanism with five accounts increases this to 500 calls, thus increasing the call volume. Function 2: Automatically blocking faulty service providers. Whether there are four or five service providers, polling minimizes or avoids such issues, thus achieving resource optimization. For example, when requesting the Deepseekv3 model, the system forwards the request to the corresponding service provider processing list to obtain N service providers. Polling among these N service providers, if one service provider encounters a problem, it enters a sleep state, i.e., is temporarily skipped.

[0025] In one embodiment, the hierarchical digital carousel corresponds to an independent carousel instance for each API key level, and each digital carousel contains multiple API keys with similar characteristics. A cascading relationship is established between each digital carousel. The digital carousel is a RoundRobin structure used to implement cyclic allocation of objects. The structure includes: an object slice for storing all objects to be allocated; a current index for recording the index of the currently allocated object; an object length for recording the length of the current object set; and a mutex lock for ensuring concurrency safety and thread safety during object addition, deletion, and allocation. The hierarchical digital carousel structure is organized into multiple virtual "carousels" according to key levels and characteristics. Each carousel contains multiple API keys with similar characteristics, supporting dynamic addition, removal, and transfer of keys. The RoundRobin struct distributes API keys in object slices cyclically using the current index, ensuring equal call frequency for each key, avoiding overload of a single key, and extending key lifecycle. It organizes independent spinners according to key levels (such as permissions and functions), supporting differentiated allocation for different business scenarios. For example, high-priority requests use a high-level key pool, while ordinary requests use a basic key pool. This solution achieves efficient API key management, elastic resource scaling, and high service availability through layered polling, dynamic scheduling, concurrency safety, and fault isolation technologies. It is suitable for distributed scenarios with high-frequency calls and multi-service provider collaboration (such as AI model inference and microservice gateways).

[0026] In one embodiment, the step of obtaining the list of available keys in the key pool in memory through a hierarchical digital carousel, i.e., organizing them into multiple virtual carousels according to the level and characteristics of the API keys, includes: Obtain all available keys and create a list of available keys; The method call function in the ProxyHandler structure is used to count the distribution of API keys at different levels, and the statistical results are stored in the relevant fields of ProxyHandler; Use temporary storage newKeys to group keys by their level attribute, iterate through all keys, aggregate keys of the same level into the same group, and call the target timer to cancel the original timer for that key. Iterate through each level group, and generate the PartialKey of the current key, such as a fingerprint or prefix, by hashing the full key (e.g., k.ID) using a hash algorithm (e.g., SHA-256) or by extracting a portion of the characters (e.g., the first 8 characters). Store the unique PartialKey corresponding to each full key in the memory mapping validKeys of the valid keys, and call the polling mechanism to create a new poller for the current level. Assign the newly generated poller mapping newRoundRobins to the actual running mapping p.roundRobins; The poller mapping in the system is updated, and the last update time is recorded. This embodiment uses a generic RoundRobin component to implement object allocation, a thread-safe sequential selection mechanism to ensure uniform allocation, and internal state tracking and dynamic adjustment capabilities.

[0027] The above solution achieves layered resource isolation and precise matching, with independent management by level: keys are classified according to function, permission, or performance characteristics (e.g., lowLevel / highLevel), with each level corresponding to an independent carousel, ensuring that appropriate key resources are allocated for different business scenarios (e.g., ordinary requests, high-priority tasks), preventing low-level keys from being overwhelmed by high-load requests. Dynamic level adaptation: It supports dynamic adjustment of key levels based on model cancellation frequency (e.g., the switchOver function), for example, automatically switching to a high-availability level when there are high-frequency failures, improving resource utilization and business stability. The generic design of utils.RoundRobin enables uniform key polling, avoiding single-key overload and extending key lifecycle. Mutex locks (MuK / MuL) ensure concurrency safety and guarantee allocation consistency in a multi-threaded environment. When high-level keys are exhausted, they can fall back to a low-level carousel (e.g., mostLevel), achieving elastic resource expansion and balancing system load and response efficiency. The key pool is periodically synchronized through the syncKeys function, and valid keys are quickly marked through validKeys mapping (e.g., k.PartialKey). Combined with timer cancellation (cancelTimer), faulty keys are isolated instantly. By temporarily putting abnormal keys into a dormant state (e.g., re-adding them to the polling pool after a timer expires), combined with retry mechanisms (exponential backoff, standby node switching), the risk of service interruption is reduced. `validKeys` quickly determines key validity using `PartialKey` (e.g., hash or truncated prefix), reducing full verification overhead and improving request processing speed. Pollers are created in batches after grouping keys by level (`newKeys`), reducing repetitive operations and optimizing initialization and synchronization performance. Layered roundRobins are decoupled from level statistics (`levels`), so adding new key levels or service providers only requires extending the corresponding roundRobin, reducing code coupling. Dynamic addition / removal of keys at runtime (e.g., concurrent-safe operations in `sync.Map`), combined with a cooldown mechanism (`modelSwitchTime`), avoids frequent switching and improves system robustness.

[0028] In one embodiment, the allocation method further includes dynamically adjusting user permissions, priority rollback, and round-robin scheduling based on a smart key selection strategy to achieve efficient key allocation, and further includes: Check if a poller at a specified level exists. For example, to try to get a poller at a specified level: search for the corresponding poller rr in p.roundRobins based on the passed level. If it does not exist, check if the poller for the highest level p.highLevel exists. If it exists, fall back and update the user level and allowed level information. If the highest level poller still does not exist, return nil directly, indicating that there is no available key. Update the global level mapping table, removing related mappings for the original levels. For example, adjust user levels: set both UserLevel and VirtualLevel to the highest level, p.highLevel; update the allowed user level list AllowLevels: delete the original level (via lib.DelInt); add the new level, p.highLevel (via lib.AddInt). Clean up the global level mapping: lock using MuL.Lock() to ensure concurrency safety; iterate through the global configuration conf.LevelMapper, deleting all key-value pairs mapped to the original level; unlock using MuL.Unlock(). The poller retrieves the next key sequentially, such as by calling rr.Next(). If the poller is empty (key==nil), it returns nil to indicate that no key is available. The poller logic is as follows: the poller returns keys sequentially (e.g., keys[index%len(keys)]), rr.Index records the current index, and rr.Length is the total number of keys. This mechanism records request statistics to the channel and returns the selected key and poller status. For example, asynchronously sending statistics involves writing request information (user, key, poller index, etc.) to the `requestsChan` channel. If the channel is full, the data is discarded and a warning log is printed. Statistics include: User: User ID, Key: Selected key ID, Index: Current polling index, Length: Total number of polling keys, and RequestNum: Current request number. The return result is as follows: On success, it returns the selected key, the current poller index `rr.Index`, and the total number of keys `rr.Length`. On failure, it returns `nil, 0, 0`. This mechanism achieves intelligent key selection through priority fallback (specified level → highest level), dynamic permission adjustment (updating user levels and allowed lists), polling scheduling (evenly distributing keys), and asynchronous monitoring (statistical requests). The core objective is to balance resource utilization and system stability while adapting to dynamically changing key states. Requests are evenly distributed using a round-robin algorithm to prevent overuse of any single key and extend its lifespan. When a user's original level becomes unavailable, they are automatically upgraded to a higher level to ensure service continuity, while user permissions are updated to prevent chaos after downgrade. The system can fall back to a higher level when a specified level becomes unavailable, improving system robustness. Statistics for each request are recorded to aid in subsequent analysis and optimization. Mutex locks protect global configuration modifications, ensuring data consistency. User permission levels and global mappings are dynamically adjusted to adapt to different scenario requirements.

[0029] In one embodiment, the implementation of a temporary dormancy policy for the current abnormal key includes: Get or initialize a timer mapping, check if a timer mapping of type key exists, and initialize a new mapping if it does not exist; each key corresponds to a timer, stored in p.timers["key"], where the key is the key ID; Check if the current key already has a timer running. If a timer already exists for the key, return directly without repeating the sleep setting. If the key already has a timer (e.g., sleep is triggered multiple times), skip directly to avoid duplicate operations.

[0030] The safety of concurrent operations on shared resources is ensured by a mutex lock (MuK), which includes pollers and timer mappings. If the poller at the current key level has multiple keys, remove the key from the poller; if there is only one key, do not sleep to ensure service availability. Create a timer to perform the recovery operation after a preset time; During recovery, the system checks if the current key is still valid. If valid, it is re-added to the poller and the timer reference is removed. This can be understood as follows: the poller is dynamically adjusted: keys are grouped into pollers (roundRobins) according to their level (key.Level); a sleep condition is applied: an abnormal key is removed from the poller only if there are multiple keys within a level; single-key protection is implemented: if only one key remains at a level, the system does not sleep, ensuring uninterrupted service. The automatic recovery mechanism involves checking if the key is still valid after a timer is triggered (using validKeys or a Redis set). If valid, it is re-added to the poller; if invalid, it is completely removed, and the timer reference is cleaned up after recovery to prevent resource leaks.

[0031] The above scheme temporarily puts abnormal keys into hibernation to avoid potential security risks (such as key leakage and abuse) and reduce the attack surface. Before restoring a key, it verifies the ValidKeys set and Redis members to ensure that only legitimate keys can be reactivated, preventing malicious keys from being revived. Keys that have been hibernated multiple times (e.g., more than 3 times) are automatically deleted to prevent long-term invalid keys from remaining in the system. When a request fails, a multi-level retry strategy (current level → other levels allowed by the user → higher system levels) quickly switches available keys to reduce the probability of service interruption. When temporarily removing abnormal keys from the poller, they are only removed if multiple keys exist at the same level, avoiding single-key farms. Service unavailability in the context of a scenario; for special error codes such as 404, additional retries are performed based on user permissions (e.g., Gear>0) to enhance fault tolerance; timers are managed through keyTimers mapping to avoid duplicate creation and reduce resource consumption; the number of retries is dynamically adjusted based on user level (Gear) (maximum 3 times for ordinary users, maximum 7 times for advanced users) to balance resource consumption and success rate; dormant keys are automatically woken up through timers to reduce manual intervention and lower operation and maintenance costs; failed retry mapping relationships (e.g., ModelMapper) are automatically cleaned up to prevent dirty data from interfering with subsequent requests; retry and switching logic is transparent to users to ensure business continuity.

[0032] In one embodiment, the step of performing intelligent retry and failover mechanisms to select a new key for retrying when a request fails includes: When a request failure is detected, the system determines whether to execute a retry strategy based on the error type (e.g., 5xx server error, 4xx client error, network timeout, and business logic). If the request returns a non-successful status code (e.g., r.StatusCode>=400) or a timeout / network error occurs, it is marked as failed. Retrying is only performed on temporary errors (e.g., 5xx, network timeout) to avoid retrying permanent errors such as 4xx. Based on the exponential backoff algorithm, such as executing the first retry immediately, delaying the second retry by 1 second, and delaying the third retry by 2 seconds, or setting a maximum number of retries such as 3, to avoid infinite loops, one or more retries are performed. If the retries fail after exceeding the threshold number, a failover will be initiated. The request route is switched to the backup service node, or data is read from the backup database, or a fallback process is implemented: if the backup service is unavailable, cached data or default values ​​are returned to ensure that some functions are available; After retrying and failover, the system attempts to send the request again, such as once. If this also fails, it proceeds to the final processing operation, which includes resetting the user's state, sending an alarm notification, releasing the user lock, and returning. The logical relationship is as follows: Request failure → Determine retryability → Intelligent retry (exponential backoff) → [Exit on success] → Retry exhaustion → Switch to backup service → [Exit on success] → Failover failure → Reset user state → Send alarm → Return error. Through these steps, the system can automatically perform intelligent retries and failover when a request fails, maximizing availability, and simultaneously notifying manual intervention in a timely manner through an alarm mechanism.

[0033] In one embodiment, the allocation method further includes a dynamic level switching mechanism that monitors the number of model cancellations, configuration mapping, and cooldown period management to achieve adaptive adjustment of the model and level, as detailed below: When the requested model is canceled, the switchOver function is triggered, which first triggers a switch check: Entry point: switchOver(xModel, level) is called (e.g., model request fails or times out); Thread safety: MuK is used to ensure concurrency safety; Configuration check: the target level switchTo corresponding to the current level is searched in conf.SwitchOver, and if no mapping exists, it returns directly; Check if there is a configured switching target for the level corresponding to the current model; The system counts the number of cancellations of the current model within a specified time. If the number of cancellations exceeds a threshold, a switching process is initiated. For example, in a time window: the current time "now" is recorded; the counter is updated: if "modelCancelTime[xModel]" exists and its difference from the current time is greater than 1 minute, "modelCancelCount[xModel]" is reset to 1; otherwise, the counter "modelCancelCount[xModel]" is incremented. If this is the first time the system is triggered, the counter is initialized to 1; threshold judgment: when "modelCancelCount[xModel]" > 5, the switching process is initiated.

[0034] Check if the last time the current model switched levels exists and if it is in the switching cooldown period, such as checking if modelSwitchTime[xModel] exists and if the switching time is less than 1 hour; If the switchover exists and the switchover cooldown period has not expired, the switchover will be skipped to avoid frequent switchovers; if the switchover does not exist or the switchover cooldown period has expired, the switchover will be allowed. Call the switching function p.switchModelLevel(xModel, level, switchTo) to perform the level switch. After updating the level mapping in the configuration and locking the MuL, set conf.LevelMapper[xModel] to switchTo, record logs and events, and print the model switch log from the original level to the new level. The system automatically switches back to the original level after a preset time using a timer, clears the switch status, resets the cancellation count and time, completing one switchover process. For example, the switchover event is recorded by storing the switch information (time, model, original level, new level) in a Redis list `SwitchOver`, with a maximum length of 1024, and updating `p.switchOverCount` to the list length. Automatic recovery is set by executing the switchback logic after 1 hour using `time.AfterFunc`: calling `p.switchModelLevel(xModel, switchTo, level)` to restore the original level, deleting `modelSwitchTime[xModel]`, and ending the cooldown period. When resetting the status, the counters are cleared by deleting `modelCancelCount[xModel]` and `modelCancelTime[xModel]` to avoid interfering with subsequent statistics. The lock is released by unlocking `MuK`, ending the process. Its `conf.SwitchOver` is a mapping table that defines the switching targets for each level (e.g., level1 → level2); `modelCancelCount` records the number of cancellations for each model within a time window; `modelCancelTime` records the time of the last cancellation of the model; `modelSwitchTime` records the time of the last switch of the model, used for cooldown period determination; and the Redis list `SwitchOver` persists the switch events for easy monitoring and auditing. The logical relationship is: cancellation event → count cancellations → exceed threshold → check cooldown period → switch level → record event → timed recovery → reset state; the level is dynamically adjusted according to the model cancellation frequency to balance performance and resource consumption; the cooldown period and counter threshold are used to avoid frequent switching that could lead to system instability; and the switch history is recorded in logs and Redis for easy troubleshooting and analysis.

[0035] This invention utilizes a digital turntable mechanism to intelligently allocate API keys, offering the following significant technical advantages: 1. Overcoming service bottlenecks: By using multiple keys in parallel, the request limitations of a single API key are overcome, increasing the overall system throughput by 5-20 times; 2. High availability guarantee: The automatic key failure detection and transfer mechanism ensures that even if multiple keys fail, the system can continue to provide stable services, increasing the request success rate to over 99.9%; 3. Optimized resource utilization: Requests are intelligently allocated based on key level, status, and historical performance, maximizing resource utilization efficiency and reducing request costs by 20-40%; 4. Adaptability. The system offers several optimization features: 5. Elastic scaling: It supports dynamic addition and removal of keys, allowing for capacity adjustments without downtime to adapt to changes in business scale; 6. Multi-tenant isolation: It achieves precise resource isolation and fair scheduling for different users and permission levels, ensuring service quality; 7. Simplified operation and maintenance: Automated fault handling, health checks, and load balancing mechanisms significantly reduce the need for manual intervention and operational complexity; 8. Distributed consistency: Even in multi-node deployment environments, it maintains the consistency and efficiency of key allocation strategies. In practical applications, the system has been successfully applied to large-scale AI API proxy services, capable of handling API requests from tens of thousands of users simultaneously and ensuring service continuity and stability under peak load and partial key failure conditions.

[0036] Based on the same inventive concept, this invention also provides a distributed API key dynamic allocation system based on a digital turntable, comprising: The initialization module is used to initialize the digital turntable; The key acquisition module is used to obtain the list of available keys in the key pool in memory via a hierarchical digital turntable. The polling allocation module is used to select the current key via a digital rotary pointer and send API requests using the current key. During a normal response, the system receives API requests and determines the key required for the request level. It then retrieves the next available key from the corresponding level's carousel, sends the request using the selected key, and monitors the response. This allows the system to expand the number of API service calls through a hierarchical polling mechanism of the digital carousel, enabling dynamic scheduling of multiple service providers or multiple keys. In abnormal situations, the system dynamically handles key anomalies and request failures, implementing a temporary sleep strategy for the currently abnormal key and employing intelligent retry and failover mechanisms to select a new key for retrying when a request fails. The implementation principle of the above module can be found in the distributed API key dynamic allocation method based on a digital turntable described in the embodiment, and will not be repeated here.

[0037] In some embodiments of this application, an electronic device is also provided. This electronic device includes a memory and a processor, wherein the memory stores a processing program; when the processor executes the processing program, it implements the distributed API key dynamic allocation method based on a digital turntable as described in the embodiments of the present invention. When the processor executes the processing program, the distributed API key dynamic allocation method based on a digital turntable in the foregoing embodiments is implemented.

[0038] This invention provides a readable storage medium storing a processing program, which, when executed by a processor, implements the distributed API key dynamic allocation method based on a digital turntable as described in the embodiments of this invention. The computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusion structures storing instructions thereon, and any suitable combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0039] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A distributed API key dynamic allocation method based on a digital turntable, characterized in that, include: Initialize the digital turntable; The list of available keys in the key pool in memory is obtained through a hierarchical digital turntable; Select the current key using the digital turntable pointer and send the API request using the current key; During a normal response, the system receives API requests and determines the key required for the request level. It then retrieves the next available key from the corresponding level's carousel, sends the request using the selected key, and monitors the response. This allows the system to expand the number of API service calls through a hierarchical polling mechanism of the digital carousel, enabling dynamic scheduling of multiple service providers or multiple keys. In abnormal situations, the system dynamically handles key abnormalities and request failures, implements a temporary sleep policy for the current abnormal key, and employs intelligent retry and failover mechanisms to select a new key for retrying when a request fails. The temporary sleep strategy for the current abnormal key includes: acquiring or initializing a timer mapping; checking if a timer mapping of the key type exists, and if not, initializing a new mapping; checking if a timer for the current key is already running, and if a timer for the key already exists, returning directly without repeating the sleep setting; ensuring the safety of concurrent operations on shared resources through a mutex lock (MuK), the shared resources including pollers and timer mappings; if the poller at the current key's level has multiple keys, removing the key from the poller, and if there is only one key, not sleeping to ensure service availability; creating a timer and performing a recovery operation after a preset time; during the recovery operation, checking if the current key is still valid, and if valid, re-adding it to the poller and deleting the timer reference. The intelligent retry and failover mechanism for selecting a new key for retrying when a request fails includes: when a request failure is detected, determining whether a retry strategy needs to be implemented based on the error type and business logic; performing one or more retries based on an exponential backoff algorithm or a strategy that limits the number of retries; if the retries still fail after exceeding the threshold, initiating failover; switching the request route to a backup service node or reading data from a backup database; and after retrying and failover, attempting to send the request again. If this also fails, the final processing operation is initiated, which includes resetting the user's state, sending an alarm notification, releasing the user lock, and returning.

2. The distributed API key dynamic allocation method based on a digital turntable as described in claim 1, characterized in that, The hierarchical digital carousel corresponds to an independent carousel instance at each API key level, and each digital carousel contains multiple API keys with similar characteristics. A cascading relationship is set between each digital carousel. The digital carousel is a RoundRobin structure used to implement cyclic allocation of objects. The structure includes: an object slice for storing all objects to be allocated; a current index for recording the index of the currently allocated object; an object length for recording the length of the current object set; and a mutex lock to ensure concurrency safety and thread safety when adding, deleting, and allocating objects.

3. The distributed API key dynamic allocation method based on a digital turntable as described in claim 1, characterized in that, The step of obtaining the list of available keys in the key pool in memory via a hierarchical digital turntable includes: Obtain all available keys and create a list of available keys; The method call function in the ProxyHandler structure is used to count the distribution of API keys at different levels, and the statistical results are stored in the relevant fields of ProxyHandler; Use temporary storage newKeys to group keys by their level attribute, iterate through all keys, aggregate keys of the same level into the same group, and call the target timer to cancel the original timer for that key. Iterate through each level group, store the PartialKey of the current key into the memory mapping validKeys of the valid keys, and call the polling mechanism to create a new poller for the current level; Assign the newly generated poller mapping newRoundRobins to the actual running mapping p.roundRobins; Update the poller mapping in the system and record the last update time.

4. The distributed API key dynamic allocation method based on a digital turntable as described in claim 1, characterized in that, It also includes dynamically adjusting user permissions, priority rollback, and round-robin scheduling based on intelligent key selection strategies to achieve efficient key allocation, and further includes: Check if a poller at the specified level exists; If it does not exist, check if the highest-level poller exists. If it does, roll back and update the user level and allowed level information. Update the global-level mapping table and remove the relevant mappings from the original level; The next key is obtained sequentially using a poller. Record request statistics to the channel and return the selected key and poller status.

5. The distributed API key dynamic allocation method based on a digital turntable as described in claim 1, characterized in that, This also includes achieving adaptive adjustment of the model and levels by dynamically adjusting the levels through model cancellation frequency, and further includes: The switchOver function is triggered when the requested model is canceled. Check if there is a configured switching target for the level corresponding to the current model; The number of cancellations for the current model within a specified time is counted, and if the number of cancellations exceeds the threshold, the switching process is initiated. Check if the time of the last level switch in the current model exists and if it is within the switching cooldown period; If the switch exists and the switchover cooldown period has not expired, the switchover is skipped; if the switch does not exist or the switchover cooldown period has expired, the switchover is allowed. Call the switching function to perform the level switch, update the level mapping in the configuration, and record logs and events; The system automatically switches back to the original level after a preset time using a timer, clears the switching status, resets the cancellation count and time, and completes one switching process.

6. A distributed API key dynamic allocation system based on a digital turntable, characterized in that, Implementing the distributed API key dynamic allocation method based on a digital turntable as described in any one of claims 1 to 5, comprising: The initialization module is used to initialize the digital turntable; The key acquisition module is used to obtain the list of available keys in the key pool in memory via a hierarchical digital turntable. The polling allocation module is used to select the current key via a digital rotary pointer and send API requests using the current key. During a normal response, the system receives API requests and determines the key required for the request level. It then retrieves the next available key from the corresponding level's carousel, sends the request using the selected key, and monitors the response. This allows the system to expand the number of API service calls through a hierarchical polling mechanism of the digital carousel, enabling dynamic scheduling of multiple service providers or multiple keys. In abnormal situations, the system dynamically handles key abnormalities and request failures, implements a temporary sleep policy for the current abnormal key, and employs intelligent retry and failover mechanisms to select a new key for retrying when a request fails.

7. An electronic device, characterized in that, include: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the distributed API key dynamic allocation method based on a digital turntable as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, The readable storage medium stores a processing program, which, when executed by a processor, implements the distributed API key dynamic allocation method based on a digital turntable as described in any one of claims 1 to 5.

Images