Distributed short chain resource management and fault-tolerant method and system based on double-layer pool architecture

Abstract

This invention provides a distributed short-link resource management and fault-tolerance method and system based on a two-layer pool architecture, aiming to solve problems such as resource contention, cursor consistency, and insufficient system fault tolerance in short-link resource management in a distributed environment. The method includes creating a public short-link pool table and setting an auto-incrementing primary key and unique index to construct a public short-link pool as a central storage repository; replenishing resources in the public short-link pool through distributed lock control and batch write operations; creating a private short-link pool based on a concurrent linked list queue in the memory of each service instance, monitoring capacity through scheduled tasks and retrieving short links in batches to achieve short-link storage; establishing a dual-storage cursor mechanism to monitor cursor positions in real time and perform anomaly detection and self-repair; and achieving local persistence and fault tolerance processing for the private short-link pool. This invention achieves hierarchical management of short-link resources and local persistence through a two-layer pool architecture, improving system stability under abnormal conditions.

Description

Technical Field

[0001] This invention belongs to the field of distributed systems, and specifically relates to a distributed short-chain resource management and fault tolerance method and system based on a two-layer pool architecture. Background Technology

[0002] Shortened URL technology is a key technology in web applications used to convert long URLs into shorter strings. It is widely used in scenarios such as social media, instant messaging, and marketing promotions where saving character space is crucial. This technology establishes a unique mapping between a fixed-length string and the original long URL, allowing users to be automatically redirected to the target website when they visit it.

[0003] Traditional link shortening services primarily rely on a single database storage and real-time generation model. This includes methods such as encoding long URLs using hash algorithms to generate shorter links, or converting auto-incrementing IDs to a specific character set. While these methods typically meet basic needs in low-concurrency scenarios, frequent database interactions lead to increased system response latency when faced with high-frequency requests.

[0004] Currently, more advanced technical solutions employ a pre-generated short link resource pool design. This involves pre-generating a certain number of short links and storing them in a database. When a request is made, the links are retrieved directly from the resource pool, avoiding the computational overhead of real-time generation. However, this approach still has several problems in a distributed deployment environment: high risk of duplicate allocation due to resource contention, lack of effective conflict handling mechanisms, difficulty in handling high-concurrency scenarios with a single storage structure, insufficient efficiency in short link generation and storage, and limited system fault tolerance.

[0005] Especially in a distributed environment, concurrent access to the shared resource pool by multiple service instances can easily lead to problems such as inconsistent cursors and duplicate allocation of short links. At the same time, existing technologies lack effective protection for the state of the resource pool when the service restarts, resulting in low system stability and resource utilization. Furthermore, the conflict handling mechanism in the short link generation process is not perfect, which affects the overall service efficiency and reliability. Summary of the Invention

[0006] The purpose of this invention is to provide a distributed short-chain resource management and fault-tolerance method and system based on a two-layer pool architecture, which solves the problems of resource competition, cursor consistency, and insufficient system fault tolerance faced by short-chain resource management in a distributed environment in the prior art.

[0007] To achieve the above objectives, this invention provides a distributed short-chain resource management and fault tolerance method based on a two-layer pool architecture, comprising the following steps:

[0008] Create a public short link pool table, set an auto-incrementing primary key and a unique index for short links in the public short link pool table, and construct a public short link pool corresponding to the public short link pool table. The public short link pool serves as a central storage repository for short link resources.

[0009] Based on the capacity status of the public short link pool, the distributed lock control single instance is executed, a batch of short link resources are randomly generated using a preset character set, and a batch write operation is performed using the insert-ignore statement to replenish the resources of the public short link pool.

[0010] A private short link pool based on a concurrent linked list queue is created in the memory of each service instance. The capacity of the private short link pool is monitored by a scheduled task, and short links are retrieved in batches from the public short link pool to the private short link pool to realize the storage of short links. The private short link pool is used for local caching of short link resources.

[0011] Obtain the cursor position of the public short chain pool, monitor the unique index conflict anomaly during the short chain ingestion process, and trigger cursor anomaly detection and self-repair operation based on the unique index conflict anomaly to realize the dual storage cursor mechanism.

[0012] The system listens for service shutdown events and triggers the storage of resources from the private short link pool to local disk files. When the service starts, it loads the resources from the private short link pool to achieve local persistence and fault tolerance of the private short link pool.

[0013] Optionally, a public short link pool table is created, an auto-incrementing primary key and a unique index for short links are set on the public short link pool table, and a public short link pool corresponding to the public short link pool table is constructed. The public short link pool serves as a central storage repository for short link resources, including:

[0014] Based on the MySQL database environment, the public short link pool table is created, and the identifier field of the public short link pool table is set as an auto-incrementing primary key to establish the primary key constraint of the public short link pool table.

[0015] Set the short link field in the public short link pool table to a six-character variable-length type and create a unique index constraint for the short link field to ensure the uniqueness of short links;

[0016] Based on the primary key constraint and unique index constraint, the public short link pool table is configured as a transaction storage engine, and a public short link pool corresponding to the public short link pool table is constructed. The public short link pool serves as a central storage repository for short link resources.

[0017] Optionally, based on the capacity status of the public short link pool, a distributed lock control single instance is executed, and a batch of short link resources are randomly generated using a preset character set and batch write operations are performed using insert-ignore statements to replenish the resources of the public short link pool, including:

[0018] Use Redis's atomic set command to create a distributed lock identifier with an expiration time, obtain the distributed lock to control the single-instance execution of supplementary tasks, and ensure concurrency safety;

[0019] Define a preset character set containing 62 characters including numbers and uppercase and lowercase letters. Use a secure random number class to generate a random number and randomly extract 6 characters from the preset character set using a character acquisition method to generate a single short chain.

[0020] The single short chain is batch generated, and multiple short chains are generated in each batch to obtain the batch short chain resources;

[0021] Based on the batch short link resources, the multiple short links are inserted into the public short link pool table in batches using an insert-ignore statement, and the number of non-duplicate short links that were successfully inserted is obtained through a row count function, thereby replenishing the resources of the public short link pool.

[0022] Optionally, a private short link pool based on a concurrent linked list queue is created in the memory of each service instance. A scheduled task monitors the capacity of the private short link pool and retrieves short links from the public short link pool in batches to the private short link pool to achieve short link storage. The private short link pool is used for local caching of short link resources and includes:

[0023] A concurrent linked list queue is used as the data structure for the private short link pool in the memory of each service instance, and a maximum capacity and replenishment threshold are set for the private short link pool.

[0024] The scheduled execution service implements a timed task that checks the current length of the private short chain pool every second. When the current length of the private short chain pool is lower than the replenishment threshold, a replenishment operation is triggered, and replenishment triggering conditions are generated.

[0025] Based on the supplementary triggering conditions, a distributed lock is acquired to ensure the serial execution of the retrieval operation in a multi-service instance environment. Multiple short chains are retrieved from the public short chain pool each time, and the short chain resources retrieved in batches are output.

[0026] The short chain resources obtained in batches are added to the concurrent linked list queue of the private short chain pool, and the identifier of the last record is recorded as the new cursor value. The private short chain pool serves as a local cache for the short chain resources.

[0027] Optionally, multiple short chains are retrieved from the public short chain pool each time, and the short chain resources retrieved in batches are output, including:

[0028] The public short chain pool is divided into N logical partitions according to the identifier range. Each logical partition forms an independent resource path through different cursor ranges, thus constructing a multi-path short chain resource acquisition model, where N is a positive integer greater than 1.

[0029] For each resource path, four key indicators are monitored in real time: resource density, competition level, response time, and conflict rate. The path status table records the four key indicators accessed multiple times in the past, and the weighted average of the four key indicators is calculated as the path health score.

[0030] Based on the path health score, the weight of each path is dynamically adjusted using an exponential smoothing algorithm, and the amount of resources to be acquired is allocated proportionally according to the weight of each path to achieve multi-path resource scheduling.

[0031] The short chain resources are obtained in parallel from each path, and the obtained short chain resources are added to the private short chain pool. The status information and cursor position of each path are updated to obtain the short chain resources in batches based on multi-path scheduling.

[0032] When congestion is detected on a certain path, a congestion control mechanism is activated to temporarily reduce the weight of the congested path and increase the acquisition ratio of other paths, or to add the congested path to a cooling queue, thereby achieving adaptive multi-path resource scheduling optimization.

[0033] Optionally, for each resource path, four key indicators—resource density, competition level, response time, and conflict rate—are monitored in real time. A path status table records these four key indicators from previous visits, and a weighted average of these four key indicators is calculated as the path health score, including:

[0034] The number of available short links within a unit interval is calculated for each resource path as a resource density index, and the access frequency of multiple service instances to each resource path is counted as a competition level index to obtain the basic path status index, which includes the resource density index and the competition level index.

[0035] The average time taken to obtain resources from each of the resource paths is measured as a response time indicator, and the proportion of unique index conflicts that occur when obtaining resources from each of the resource paths is calculated as a conflict rate indicator to obtain a path performance indicator, which includes the response time indicator and the conflict rate indicator.

[0036] Based on the path basic status indicators and path performance indicators, the resource density, contention level, response time and conflict rate values ​​of the past N accesses are recorded in the path status table to obtain historical monitoring data, where N is a positive integer greater than 1.

[0037] A weighted average algorithm is applied to the historical monitoring data, with weights corresponding to resource density, the reciprocal of competition level, the reciprocal of response time, and the reciprocal of conflict rate, respectively, to calculate the comprehensive health score and obtain the path health score;

[0038] Based on the path health score, the performance trend of each resource path is periodically evaluated to identify paths with degraded or abnormal performance, thereby achieving comprehensive monitoring and evaluation of path status.

[0039] Optionally, when congestion is detected on a certain path, a congestion control mechanism is activated to temporarily reduce the weight of the congested path and increase the acquisition ratio of other paths, or the congested path is added to a cooling queue to achieve adaptive multi-path resource scheduling optimization, including:

[0040] The congestion index for each resource path is calculated using an exponentially weighted moving average algorithm. A smoothing factor is applied to the calculation based on the current conflict rate and the historical congestion index to obtain the real-time congestion index.

[0041] When the real-time congestion index exceeds a preset threshold, congestion detection is triggered. The weight of the congested path is temporarily reduced to decrease the amount of resources acquired, while the acquisition ratio of other healthy paths is increased. The access interval of the congested path is adjusted to avoid frequent competition. Alternatively, the congested path is added to a cooling queue and a cooling time is set. During the cooling period, the congested path is no longer accessed. After the cooling period ends, the status of the congested path is reassessed.

[0042] Optionally, the cursor position of the public short chain pool is obtained, and unique index conflict anomalies are monitored during the short chain insertion process. Based on the unique index conflict anomalies, cursor anomaly detection and self-repair operations are triggered to implement a dual-storage cursor mechanism, including:

[0043] In Redis, the cursor value is stored using a string type. In MySQL, a dedicated cursor table is created to store the historical values ​​and update times of the cursor, thus establishing a dual storage structure for the cursor.

[0044] The process of monitoring the short chain entry into the database involves a unique index conflict. When a conflict occurs, the identifier value A, the cursor value B in Redis, and the cursor value C in MySQL corresponding to the latest short chain in the private short chain pool are retrieved.

[0045] The maximum value of the three values—identifier A corresponding to the latest short link in the private short link pool, cursor value B in Redis, and cursor value C in MySQL—is used to determine the current correct cursor position. The correct cursor value is then synchronized and updated to Redis and MySQL to achieve cursor self-repair.

[0046] Calculate the difference between the correct cursor value and the minimum of the three values ​​to determine the number of short chains that are repeatedly claimed, and delete the corresponding number of short chains from the private short chain pool to achieve a dual-storage cursor mechanism.

[0047] Optionally, the method further includes:

[0048] A cursor management mechanism is established, which divides cursor management into three basic strategies: greedy mode, cooperative mode and random mode. In the greedy mode, the region with resource density higher than a first preset threshold and competition level lower than a second preset threshold is selected as the cursor position. In the cooperative mode, the cursor positions are selected in turn according to a predefined order. In the random mode, the cursor position is selected randomly within the available range.

[0049] A comprehensive competition index is obtained by calculating a weighted average of conflict rate, resource acquisition delay, and cursor update conflict, which is used to assess the competition status.

[0050] Based on the comprehensive competition index, the cursor management strategy is dynamically adjusted to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, thus outputting a hybrid strategy selection mechanism.

[0051] Under the hybrid strategy selection mechanism, a choice is made between the cooperative strategy and the greedy strategy according to the strategy selection parameters, and a random strategy is used to increase the exploratory nature under the first probability. The strategy execution results are recorded and the strategy effect score is updated to realize hybrid strategy cursor management.

[0052] Optionally, based on the comprehensive competition index, the cursor management strategy is dynamically adjusted to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, outputting a hybrid strategy selection mechanism, including:

[0053] Regularly analyze the performance of each strategy under different competitive environments, establish a strategy-environment-effect mapping relationship by statistically analyzing the strategy effects according to the competition index range, and obtain historical performance analysis results;

[0054] Based on the historical performance analysis results, when the current competition index falls into a specific range, the optimal strategy within that specific range is determined. The strategy selection parameters are adjusted to favor the optimal strategy and the internal parameters of the strategy are optimized to achieve self-tuning of the strategy parameters. This is to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, outputting a hybrid strategy selection mechanism. The degree of competition in the first competitive environment is greater than that in the second competitive environment.

[0055] The strategy selection mode of the first instance is broadcast to other instances to obtain the strategy selection information of other instances. Randomness is added to the strategy selection information to avoid strategy selection synchronization and achieve cluster strategy coordination.

[0056] Optionally, the system listens for service shutdown events and triggers the storage of resources from the private short link pool to a local disk file. When the service starts, it loads the resources from the private short link pool to achieve local persistence and fault tolerance of the private short link pool, including:

[0057] Use Spring framework's event listener annotations to listen for context shutdown events, capture signals of normal service shutdown, and generate service shutdown event triggering conditions;

[0058] Based on the service shutdown event triggering condition, the remaining short links in the private short link pool are obtained and converted into a list collection. The list collection is serialized and written to a local disk file using file input / output operations to achieve resource persistence of the private short link pool.

[0059] When the service starts, check if there are persistent files on the local disk, read the file content and deserialize it into a short link list, add the short links in the short link list to the private short link pool, and obtain the resource loading result.

[0060] Based on the resource loading result, the persistent file is deleted to avoid repeated loading. During the short link entry process, a unique key conflict exception is captured and a new short link is retrieved from the private short link pool to replace the conflicting item. A recursive retry method is used to ensure that the short link allocation can be successfully completed in the end, thereby realizing the local persistence and fault tolerance of the private short link pool.

[0061] This application provides a distributed short-chain resource management and fault-tolerant system based on a two-layer pool architecture, including:

[0062] A creation module is used to create a public short chain pool table, set an auto-incrementing primary key and a unique index for short chains in the public short chain pool table, and construct a public short chain pool corresponding to the public short chain pool table. The public short chain pool serves as a central storage repository for short chain resources.

[0063] The write module is used to obtain the distributed lock control single instance execution based on the capacity status of the public short link pool, randomly generate a batch of short link resources using a preset character set, and perform batch write operations using insert-ignore statements to replenish the resources of the public short link pool.

[0064] The database entry module is used to create a private short link pool based on a concurrent linked list queue in the memory of each service instance. The capacity of the private short link pool is monitored by a scheduled task, and short links are retrieved in batches from the public short link pool to the private short link pool to realize the database entry of short links. The private short link pool is used for local caching of short link resources.

[0065] The anomaly detection and self-repair module is used to obtain the cursor position of the public short chain pool, monitor the unique index conflict anomaly during the short chain entry process, and trigger the cursor anomaly detection and self-repair operation based on the unique index conflict anomaly to realize the dual storage cursor mechanism.

[0066] The local persistence and fault tolerance module is used to listen for service shutdown events and trigger the storage of resources of the private short link pool to local disk files. When the service starts, it loads the resources of the private short link pool to realize the local persistence and fault tolerance of the private short link pool.

[0067] The beneficial effects of this invention are:

[0068] 1) By establishing a two-tier pool architecture of public and private short chain pools, hierarchical management of short chain resources is achieved, reducing database access pressure and improving system response speed;

[0069] 2) By utilizing distributed locks and batch operations, efficient replenishment of public short-chain pool resources is achieved, ensuring the sufficiency of system resources in a high-concurrency environment;

[0070] 3) Through scheduled tasks and capacity monitoring, the private short chain pool is dynamically replenished, ensuring a stable supply of short chain resources for each service instance;

[0071] 4) A dual-storage cursor mechanism and anomaly self-healing logic were designed, which effectively solved the problems of cursor consistency and resource duplication in a distributed environment;

[0072] 5) It realizes local persistence and fault tolerance of private short chain pool, which improves the stability and resource utilization efficiency of the system under abnormal conditions. Attached Figure Description

[0073] Figure 1 This is a schematic flowchart of the method of the present invention;

[0074] Figure 2 This is a diagram showing the overall architecture of the system of the present invention;

[0075] Figure 3 This is a detailed flowchart of S1 of the present invention;

[0076] Figure 4 This is a detailed flowchart of S2 of the present invention. Detailed Implementation

[0077] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it.

[0078] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0079] Example 1:

[0080] like Figure 1 As shown, this embodiment provides a distributed short-chain resource management and fault tolerance method based on a two-layer pool architecture, including the following steps:

[0081] S1: Create a public short link pool table, set an auto-incrementing primary key and a unique index for short links in the public short link pool table, and construct a public short link pool corresponding to the public short link pool table. The public short link pool serves as a central storage repository for short link resources.

[0082] Specifically, such as Figure 3 As shown, S1 includes:

[0083] S1.1 Based on the MySQL database environment, create the public short link pool table (surl_pool), and set the identifier field (id) of the public short link pool table as an auto-incrementing primary key to establish the primary key constraint of the public short link pool table;

[0084] S1.2 Set the short link field (surl) in the public short link pool table to a six-character variable length type, create a unique index constraint for the short link field, and ensure the uniqueness of short links;

[0085] S1.3 Based on the primary key constraint and unique index constraint, the public short link pool table is configured as a transaction storage engine, and a public short link pool corresponding to the public short link pool table is constructed. The public short link pool serves as a central storage repository for short link resources.

[0086] In this embodiment, the structure of the public short-chain pool table (surl_pool) is as follows:

[0087] CREATE TABLE `surl_pool` (

[0088] `id` int(11) NOT NULL AUTO_INCREMENT,

[0089] `surl` varchar(6) NOT NULL COMMENT 'Short link (6-character string)',

[0090] PRIMARY KEY (`id`),

[0091] UNIQUE KEY `surl_pool_surl_IDX` (`surl`) COMMENT 'Guarantee the uniqueness of short links'

[0092] ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

[0093] First, the short link resource management system (hereinafter referred to as the system) creates a public short link pool table (surl_pool) in the MySQL database and carefully designs the table structure to meet the specific needs of short link resource management. In the table structure design, the identifier field (id) is set as an auto-incrementing primary key. This design has two key advantages: on the one hand, the auto-incrementing primary key ensures that each record has a unique identifier, facilitating precise location and efficient access; on the other hand, the auto-incrementing characteristic makes inserting new records simpler, as the system does not need to manually manage and allocate ID values. At the same time, the system strictly establishes primary key constraints to ensure that the value of the identifier field is unique and non-nullable throughout the table, preventing potential data consistency problems at the database level.

[0094] For the design of the short link field (surl), the system adopts a six-character variable-length type (VARCHAR(6)). This design not only meets the length requirements of short links but also allows for flexible storage of short link values ​​of different lengths. The six-character design is the result of careful calculation, using a combination of letters (az, AZ) and numbers (0-9). Six characters can generate approximately 56.8 billion (62) 6 A number of different short chains are sufficient to meet the needs of most application scenarios. In practical applications, the system usually uses the Base62 encoding algorithm (using 62 characters from 0-9, az, and AZ) to generate short chains, ensuring that the generated short chains are both short and easy to use, and have sufficient capacity to support large-scale applications.

[0095] To ensure the uniqueness of short links, the system creates a unique index constraint on the short link field (surl). This constraint allows the database to physically prevent any operations that could lead to duplicate short links, effectively preventing resource conflicts and data corruption. The unique index not only guarantees data integrity but also significantly improves the performance of querying records based on short links, as the database can quickly locate records using the index structure without performing a full table scan. This is particularly important for high-frequency query scenarios in short link services, significantly reducing response time and system load.

[0096] Considering the stringent requirements for transaction safety and concurrency control in short-chain resource management, the system configures the public short-chain pool table with a transaction-supporting storage engine, typically InnoDB. InnoDB provides full ACID transaction support, row-level locking, and foreign key constraints, making it particularly suitable for high-concurrency read / write applications. During short-chain resource scheduling, multiple service instances may simultaneously attempt to acquire or modify short-chain resources. Using a transaction-supporting storage engine ensures the atomicity, consistency, isolation, and durability of these concurrent operations, effectively preventing data races and inconsistent states.

[0097] Through the above design and configuration, the system successfully constructed a public short link pool, which serves as a central storage repository for short link resources, undertaking the crucial functions of centralized resource management and allocation. In actual operation, the public short link pool pre-stores a large number of generated short link resources, which are continuously replenished through background tasks to ensure the pool always maintains a sufficient number of available short links. When the system needs to allocate short links to users, the multi-path resource scheduling mechanism efficiently retrieves resources from the public short link pool, achieving rapid response and stable service.

[0098] It's worth noting that the design of the public short-link pool table also considers scalability and maintainability. In addition to the core `id` and `surl` fields, the table structure can include other auxiliary fields as needed, such as creation time (`create_time`), status identifier (`status`), and partition identifier (`partition_id`), to support more complex resource management strategies and system monitoring requirements. Furthermore, the system will periodically perform maintenance operations on the public short-link pool table, such as optimizing indexes and cleaning up expired data, to ensure long-term stable database performance.

[0099] Through this meticulously designed data storage structure, the multi-path short-link resource scheduling system provides a solid data foundation for upper-layer resource acquisition and scheduling algorithms, ensuring the stable operation and efficient response of short-link services in high-concurrency environments. Practical experience has proven that this design can support thousands of resource acquisition requests per second while maintaining extremely low conflict rates and millisecond-level response times, meeting the stringent requirements of modern internet applications for short-link services.

[0100] S2: Based on the capacity status of the public short link pool, obtain the distributed lock control single instance execution, randomly generate a batch of short link resources using a preset character set, and use the insert-ignore statement to perform batch write operations to replenish the resources of the public short link pool.

[0101] Specifically, such as Figure 4 As shown, S2 includes:

[0102] S2.1 Use Redis's atomic set command to create a distributed lock identifier with an expiration time, obtain the distributed lock to control the single-instance execution of supplementary tasks, and ensure concurrency safety;

[0103] First, the system uses Redis's atomic set command (SET command with NX and EX options) to create a distributed lock identifier with an expiration time. Specifically, the system executes a command like "SET short_link_replenish_lock unique_identifier NX EX 300", where "short_link_replenish_lock" is the lock key name, "unique_identifier" is typically a combination of the service instance's unique identifier and a timestamp, the "NX" option ensures success only if the key does not exist, and "EX 300" sets the lock's expiration time to 300 seconds. This mechanism ensures that in a distributed environment, only one service instance can successfully acquire the lock and execute the supplementary task at any given time, effectively preventing resource waste and potential conflicts caused by multiple instances executing simultaneously. Furthermore, setting a reasonable expiration time avoids the problem of locks being permanently unreleased due to instance crashes, enhancing the system's fault recovery capabilities.

[0104] S2.2 Define a preset character set containing 62 characters including numbers and uppercase and lowercase letters. Use a secure random number class to generate a random number and use a character acquisition method to randomly extract 6 characters from the preset character set to generate a single short chain.

[0105] After successfully acquiring the distributed lock, the system begins the short chain generation task. The system first defines a preset character set containing 62 characters: digits (0-9), lowercase letters (az), and uppercase letters (AZ). This character set design ensures both the conciseness and readability of the short chain and provides ample combinatorial space (6 characters can generate approximately 56.8 billion different combinations). To ensure the randomness and security of the generated short chain, the system uses a professional secure random number generator class (such as SecureRandom in Java) instead of a common pseudo-random number generator. The secure random number generator utilizes the entropy source provided by the operating system to generate truly random numbers, significantly improving the unpredictability of the generated results and preventing potential guessing attacks.

[0106] In this embodiment, for example, the preset character set is: "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" (a total of 62 characters). 200,000 short chains are generated in each batch, with a target replenishment quantity of 30,000,000. By monitoring the remaining available amount in the public short chain pool (total number of records minus the current cursor value), a replenishment operation is triggered when the remaining amount falls below a preset threshold (70% of the total capacity).

[0107] S2.3 Perform a batch generation operation on the single short chain, generating multiple short chains in each batch to obtain the batch short chain resources;

[0108] After generating random numbers, the system extracts characters from a preset character set using a character retrieval method. Specifically, the system first generates a random integer between 0 and 61, then uses this integer as an index to retrieve the corresponding character from the preset character set array. By repeating this process six times and concatenating the retrieved characters, the system successfully generates a 6-digit random short chain. It is worth noting that the system design specifically considers the uniformity of character distribution, ensuring that each character has an equal probability of being selected, thus avoiding wasted combination space due to some characters appearing too frequently.

[0109] To improve generation efficiency, the system implements a batch generation strategy. Each time a replenishment task is triggered, the system does not generate a single short link, but instead generates multiple short links consecutively (typically 500 to 1000), forming a batch of short link resources. This batch operation significantly reduces the number of database interactions, lowering network overhead and system load. Simultaneously, batch generation allows the system to quickly replenish large amounts of resources in a short time, effectively handling sudden high-demand scenarios. In practice, the system typically dynamically adjusts the number of links generated in a single batch based on resource consumption rates and database performance, finding the optimal balance between efficiency and load.

[0110] S2.4 Based on the batch short chain resources, the multiple short chains are inserted into the public short chain pool table in batches using an insert-ignore statement, and the number of non-duplicate short chains that were actually successfully inserted is obtained through a row count function, thereby replenishing the resources of the public short chain pool.

[0111] After generating a batch of short link resources, the system uses special SQL syntax—the INSERT IGNORE INTO statement (such as "INSERT IGNORE INTO" in MySQL)—to insert multiple short links into a shared short link pool table. The unique aspect of this syntax is that when an inserted short link conflicts with an existing record in the table (violating a unique index constraint), the database automatically ignores the conflicting record instead of reporting an error and interrupting the operation, ensuring successful completion. Through this mechanism, the system effectively handles potential short link duplication issues, improving the stability and fault tolerance of resource replenishment.

[0112] After the insertion operation is completed, the system uses a row count function (such as ROW_COUNT() in MySQL) to obtain the number of successfully inserted non-duplicate short links. This data is not only an important indicator of the effectiveness of the current supplementation task, but also the basis for the system to decide whether to continue supplementation. If the number of successfully inserted short links is significantly lower than expected (e.g., the success rate is below 70%), the system will determine that the current short link space may be close to saturation and consider adjusting the short link length or character set to expand the available space. Conversely, if the success rate is close to 100%, the system will consider that the current resource space is still sufficient and can continue to use the existing strategy for supplementation.

[0113] Once the entire resource replenishment process is complete, the system will proactively release the previously acquired distributed lock (by deleting the lock key using the DEL command), allowing other instances to execute replenishment tasks when needed. Simultaneously, the system will record detailed information about this replenishment, including execution time, number of instances generated, and success rate, serving as an important data source for system monitoring and performance optimization.

[0114] Through this meticulously designed resource replenishment mechanism, the multi-path short link resource scheduling system ensures that the public short link pool always maintains sufficient available resources, providing a solid guarantee for upper-layer resource scheduling and allocation. Practice has shown that this mechanism can maintain stable operation under high load environments, replenishing tens of thousands of unique short links per minute, meeting the resource requirements of large-scale short link services. More importantly, through distributed lock control and batch operation optimization, the system ensures replenishment efficiency while keeping the pressure on the database within a reasonable range, avoiding interference with normal business operations during the resource replenishment process.

[0115] S3: Create a private short link pool based on a concurrent linked list queue in the memory of each service instance. Monitor the capacity of the private short link pool through a scheduled task and retrieve short links from the public short link pool in batches to the private short link pool to realize the storage of short links. The private short link pool is used for local caching of short link resources.

[0116] The short link entry process involves more than just storing the short link itself; it involves first establishing the association before storing it. In other words, the short link obtained can be associated with business parameters, user_id, etc., and then stored in the business table for use by the requesting party or later.

[0117] Specifically, S3 includes:

[0118] S3.1 In the memory of each of the service instances, a concurrent linked list queue (ConcurrentLinkedQueue) is used as the data structure of the private short link pool, and a maximum capacity (2,000,000) and a replenishment threshold (90%, i.e. 1,800,000) are set for the private short link pool;

[0119] First, the system creates a private short-link pool in the memory of each service instance, choosing a concurrent linked list queue (ConcurrentLinkedQueue) as its underlying data structure. This choice reflects profound technical considerations: ConcurrentLinkedQueue is a lock-free concurrent queue implementation in the Java concurrency package, employing a non-blocking algorithm based on CAS (Compare-And-Swap) operations, which performs excellently in high-concurrency environments. Compared to traditional lock-based collections, ConcurrentLinkedQueue supports multi-threaded concurrent access without explicit synchronization control, significantly reducing thread contention and context switching overhead. In real-world scenarios for short-link services, a large number of concurrent threads may simultaneously acquire resources from the private short-link pool. The lock-free nature of ConcurrentLinkedQueue ensures that these operations can be completed efficiently, without performance degradation or thread blocking due to lock contention.

[0120] For capacity management of the private short link pool, the system employs carefully calculated parameters: a maximum capacity of 2,000,000 short links, with a replenishment threshold set at 90% of the maximum capacity, or 1,800,000 links. These parameters are based on in-depth analysis of system operating characteristics and resource consumption patterns. The maximum capacity of two million links ensures that a single service instance can continue to provide service for an extended period even when the public short link pool is temporarily unavailable, enhancing the system's fault tolerance and stability. Simultaneously, this capacity will not consume excessive memory resources under most server configurations, maintaining reasonable resource utilization. Based on the average size of short link objects (approximately 100 bytes per object), two million short links would consume approximately 200MB of memory, a perfectly acceptable memory overhead in modern servers.

[0121] Setting the replenishment threshold to 90% is a trade-off between resource utilization efficiency and replenishment frequency. A higher threshold (close to 100%) would lead to excessively frequent replenishment operations, increasing system overhead; conversely, a threshold that is too low would result in resources remaining in an overabundant state for extended periods, wasting memory. The 90% threshold means that replenishment is only triggered when the number of resources in the private short-chain pool falls below 1.8 million. This ensures that each replenishment operation replenishes a substantial number of resources (200,000), reducing replenishment frequency while maintaining a consistent resource sufficiency.

[0122] S3.2 The scheduled execution service implements a timed task that checks the current length of the private short chain pool every second. When the current length of the private short chain pool is lower than the replenishment threshold, a replenishment operation is triggered, and replenishment triggering conditions are generated.

[0123] The system implements timed monitoring of the private short-link pool's status through a scheduled execution service. Specifically, the system creates a scheduled task that checks the current length of the private short-link pool every second and compares it to a preset replenishment threshold. This high-frequency checking ensures the system can promptly detect resource shortages and quickly initiate a replenishment process, preventing service interruptions due to resource exhaustion. Simultaneously, the check frequency of once per second does not place a significant burden on the system, maintaining a good balance between monitoring and performance. When the length of the private short-link pool is detected to be below the replenishment threshold, the system generates replenishment trigger conditions, including information such as the current resource quantity and the quantity to be replenished, providing a basis for subsequent replenishment operations.

[0124] S3.3 Based on the supplementary triggering conditions, a distributed lock is acquired to ensure the serial execution of the retrieval operation in a multi-service instance environment. Multiple short chains (50,000) are retrieved from the public short chain pool each time, and the short chain resources retrieved in batches are output.

[0125] After triggering the replenishment operation, the system first attempts to acquire a distributed lock to ensure the serial execution of the retrieval operation in a multi-service instance environment. This step is necessary because multiple service instances simultaneously retrieving resources from the public short link pool may lead to uneven resource allocation or a surge in database pressure. Through distributed lock control, the system ensures that only one service instance can execute the retrieval operation at any given time, while other instances must wait for the lock to be released before proceeding. It is worth noting that the lock granularity here is limited to the retrieval operation and does not affect the parallelism of each instance retrieving short links from its own private short link pool, therefore it will not become a bottleneck for system performance.

[0126] After successfully acquiring the distributed lock, the system retrieves short link resources in batches from the public short link pool. Each retrieval operation obtains 50,000 short links; this quantity is the result of careful optimization, achieving a good balance between replenishment efficiency and database load. In the specific implementation, the system typically uses the "SELECT FOR UPDATE SKIP LOCKED" syntax (in supported databases such as MySQL 8.0+) or a similar mechanism to ensure that duplicate short links are not retrieved in a multi-instance environment. During the retrieval process, the system records the start and end IDs of each batch, forming a complete operation log for easy troubleshooting and performance analysis.

[0127] S3.4 Add the batch-received short chain resources to the concurrent linked list queue of the private short chain pool, and record the identifier of the last record as the new cursor value. The private short chain pool serves as a local cache for the short chain resources.

[0128] After retrieving short-chain resources from the public short-chain pool, the system adds these resources to the concurrent linked list queue of the private short-chain pool. The addition process is incremental; the system does not clear the existing queue but appends the newly retrieved resources to the tail of the queue, ensuring continuous resource availability. After addition, the system records the identifier of the last record as a new cursor value. This cursor value is used as the starting position for the next retrieval operation, achieving sequential consumption of resources and avoiding duplicate retrieval. Cursor management is a key mechanism for ensuring efficient resource utilization. By maintaining an independent cursor for each service instance, the system ensures that resources in the public short-chain pool are evenly distributed among the service instances, without omissions or duplications.

[0129] The private short link pool, acting as a local cache for short link resources, plays a crucial role in the entire short link service architecture. It not only significantly reduces the access pressure on the central database and improves the response speed of short link retrieval, but also enhances the system's resilience and fault tolerance. In the event of network fluctuations or temporary database unavailability, service instances can still rely on the private short link pool to continue providing services, ensuring business continuity. Practice has proven that systems employing this private short link pool design can easily handle tens of thousands of short link retrieval requests per second during peak periods while maintaining sub-millisecond response times, greatly improving user experience and system reliability.

[0130] Furthermore, the design of the private short link pool provides a solid foundation for the horizontal scaling of the system. When the system needs to add service instances to handle higher loads, the new instances only need to create their own private short link pool and obtain resources from the public short link pool to be quickly put into use, without complex coordination and configuration. This loosely coupled design makes system expansion simple and efficient, effectively supporting rapid business growth and fluctuations.

[0131] S4: Obtain the cursor position of the public short chain pool, monitor the unique index conflict anomaly during the short chain entry process, and trigger cursor anomaly detection and self-repair operation based on the unique index conflict anomaly to realize the dual storage cursor mechanism.

[0132] Specifically, S4 includes:

[0133] S4.1 uses a string type to store cursor values ​​in Redis, and creates a dedicated cursor table in MySQL to store the historical values ​​and update times of the cursors, establishing a dual storage structure for cursors;

[0134] First, the system employs a unique dual-storage cursor structure, maintaining cursor information in both Redis and MySQL. In Redis, the current cursor value is stored as a string, with keys typically in the format "short_link_cursor:{instance_id}", where {instance_id} is a unique identifier for the service instance, ensuring that different instances have independent cursors. Redis, as a high-performance in-memory database, provides millisecond-level read and write speeds, enabling cursor queries and updates to be completed quickly with minimal impact on overall system performance. Simultaneously, a dedicated cursor table (cursor_table) is created in MySQL to store historical cursor values ​​and update times. This table typically contains fields such as instance_id (instance ID), cursor_value (cursor value), and update_time (update time), and a unique index is created on instance_id to ensure query efficiency.

[0135] This dual-storage architecture design is based on profound technical considerations: Redis provides high-speed access, meeting the system's performance requirements for frequent cursor reads and writes; while MySQL provides persistent storage and transaction support, ensuring the recoverability of cursor information in the event of system restart or Redis data loss. Furthermore, the dual-storage architecture also achieves data redundancy; when one storage mechanism fails, the system can recover cursor information from the other, significantly improving the system's fault tolerance and reliability. In normal operation, the system first updates the cursor value in Redis, and then asynchronously updates the record in MySQL. This design ensures data consistency while avoiding the impact of MySQL write operations on system performance.

[0136] S4.2 Monitor the unique index conflict anomaly during the short link entry process. When a conflict occurs, obtain the identifier value A, the cursor value B in Redis, and the cursor value C in MySQL corresponding to the latest short link in the private short link pool.

[0137] The system pays special attention to unique index conflicts that may occur during the short link insertion process. When service instances retrieve resources from the public short link pool and attempt to record these short links into the system, they may encounter unique index conflicts—this usually means that some short links have already been retrieved and used by other instances or previous operations. Such conflicts are common in distributed systems and can be caused by various factors such as network latency, data inconsistency, or cursor out-of-sync. The system intelligently handles these anomalies by capturing them and triggering a cursor self-healing process.

[0138] When the system detects a unique index conflict anomaly, it immediately initiates a complex but efficient cursor verification and repair process. First, the system collects three key cursor values: the identifier value A corresponding to the latest short link in the private short link pool, the cursor value B stored in Redis, and the cursor value C recorded in MySQL. These three values ​​represent different dimensions of the cursor state in the system: A reflects the latest position of the actually acquired resource, B represents the theoretical cursor position recorded in the cache, and C is the cursor's historical record in persistent storage. Ideally, these three values ​​should be similar or identical; significant differences indicate a problem with cursor synchronization.

[0139] S4.3 Calculate the maximum value of the three values ​​among the latest short link in the private short link pool (identifier A), Redis (cursor value B), and MySQL (cursor value C) to determine the current correct cursor position, and synchronize the correct cursor value to Redis and MySQL to achieve cursor self-repair;

[0140] The system employs a simple yet effective maximum value strategy to determine the correct cursor position. Specifically, the system calculates the maximum value among three values: A, B, and C, and considers this maximum value as the correct cursor position. This strategy is based on the reasonable assumption that the cursor should only move forward and not backward; therefore, the largest cursor value is most likely to reflect the true resource consumption status. Once the correct cursor value is determined, the system immediately synchronizes this value to Redis and MySQL, ensuring consistency among the three data parties. This proactive synchronization mechanism not only corrects the current inconsistency but also prevents potential resource duplication issues.

[0141] S4.4 Calculate the difference between the correct cursor value and the minimum of the three values ​​to determine the number of short chains that are repeatedly claimed, and delete the corresponding number of short chains from the private short chain pool to realize the dual-storage cursor mechanism.

[0142] After updating the cursor, the system further calculates the difference between the correct cursor value and the minimum of the three collection values. This difference represents the number of short links that may have been repeatedly claimed due to cursor inconsistency. For example, if the correct cursor value is 10000 and the minimum collection value is 9500, a difference of 500 indicates that the system may have claimed 500 short links repeatedly. To prevent these duplicate short links from entering the service process and causing chaos, the system removes the corresponding number of short links (500 in this example) from the private short link pool. This precise resource correction ensures that the system maintains the correctness and uniqueness of resource allocation even in the event of cursor anomalies.

[0143] The implementation of the dual-storage cursor mechanism also includes a series of detailed optimizations. For example, the system uses atomic operations (such as the GETSET command) when updating Redis cursors to ensure consistency of cursor updates in high-concurrency environments; in the MySQL cursor table, the system not only records the current cursor value but also retains historical update records, facilitating problem tracking and system auditing; when deleting duplicate short links, the system adopts batch operations instead of deleting them one by one, significantly improving processing efficiency. Furthermore, the system implements transaction protection for cursor updates, ensuring that cursor updates and resource retrieval are atomic operations, preventing resource duplication or omission caused by intermediate states.

[0144] This dual-storage cursor mechanism and its self-healing function have demonstrated exceptional reliability and fault tolerance in practical operation. Even in the event of network fluctuations, service restarts, or temporary database unavailability, the system can quickly restore the correct cursor state through the self-healing mechanism, ensuring the continuity and correctness of resource allocation. Statistical data shows that after deploying this mechanism, the system's resource duplication rate was reduced by 99.9%, service interruptions caused by cursor inconsistency were completely eliminated, and the stability and reliability of the entire short-chain service were significantly improved.

[0145] Furthermore, the dual-storage cursor mechanism provides the system with rich monitoring and diagnostic capabilities. By comparing the differences in cursor values ​​across different storage locations, system administrators can quickly identify potential performance bottlenecks or data synchronization issues; by analyzing the frequency and magnitude of cursor repairs, they can assess the system's health and resource allocation efficiency. This data is not only used for routine maintenance but also provides valuable decision-making support for system optimization and architecture evolution.

[0146] Overall, the dual-storage cursor mechanism is a key innovation in multi-path short-chain resource scheduling systems. Through multi-layered data storage and intelligent self-healing algorithms, it successfully solves the challenges of cursor synchronization and resource duplication in distributed environments, providing a solid resource management foundation for the entire short-chain service. This mechanism is not only applicable to short-chain systems but can also be extended to other distributed application scenarios requiring precise resource allocation, demonstrating broad technical value and application prospects.

[0147] S5: Listen for service shutdown events and trigger the storage of resources from the private short link pool to local disk files. Load the resources from the private short link pool when the service starts to achieve local persistence and fault tolerance of the private short link pool.

[0148] Specifically, S5 includes:

[0149] S5.1 uses Spring framework event listener annotations to listen for context shutdown events, capture signals of normal service shutdown, and generate service shutdown event triggering conditions;

[0150] First, the system employs the event listener mechanism provided by the Spring framework to capture service shutdown signals. Specifically, the system uses the `@EventListener` annotation to listen for the `ContextClosedEvent` event, a standard event automatically triggered when the Spring context is closed. This event is triggered when the system receives a shutdown signal (such as a graceful shutdown command or a termination request from the container orchestration system), subsequently invoking the registered listener methods. This framework-based event listener approach has several advantages: firstly, it is independent of specific deployment environments and platforms, ensuring the generality of the solution; secondly, it can be reliably triggered during the normal application shutdown process without missing shutdown events. In actual code, this typically manifests as an implementation similar to the following:

[0151] / / java

[0152] @EventListener(ContextClosedEvent.class)

[0153] public void handleContextShutdown(ContextClosedEvent event) {

[0154] / / Handling service shutdown logic

[0155] persistShortLinkPool();

[0156] }

[0157] S5.2 Based on the service shutdown event triggering condition, obtain the remaining short links in the private short link pool and convert them into a list set. Use file input / output operations to serialize the list set and write it to a local disk file to realize the persistence of the private short link pool resources.

[0158] Once the service shutdown event is captured, the system immediately initiates the persistence process for the private short-link pool. First, the system retrieves all remaining short-link resources from the ConcurrentLinkedQueue. Due to the concurrent nature of the queue, this process requires special care to avoid data inconsistency issues. The system typically uses atomic operations or explicit locking mechanisms to ensure that no new short links are added or removed during persistence. After retrieving the queue contents, the system converts these short links into a regular list collection (such as an ArrayList). This conversion simplifies the subsequent serialization process and avoids the complexity that might arise from attempting to serialize concurrent data structures.

[0159] The core step of persistence is serializing the short linked list and writing it to a local disk file. The system uses Java's object serialization mechanism or a more efficient third-party serialization library (such as Kryo, Protobuf, etc.) to convert the short linked list into a binary data stream. When choosing a serialization scheme, the system balances multiple factors such as serialization speed, data size, and compatibility. Considering the potentially limited time during service shutdown, the system tends to choose a faster serialization scheme, even if this may result in a slightly larger file size. After serialization, the system uses Java file input / output operations (such as FileOutputStream or BufferedOutputStream) to write the data to a predefined local path. The filename typically includes the service instance ID and a timestamp, ensuring that files do not overwrite each other in a multi-instance environment. To prevent data loss due to file system failures, the system also implements atomicity guarantees for file writes, such as writing to a temporary file first and then renaming it, or using fsync to ensure that the data is actually written to disk.

[0160] The entire persistence process is designed to be efficient and reliable. Even with a private short-chain pool containing millions of short chains, persistence operations typically complete within seconds, without significantly extending service downtime. The system also implements logging and exception handling for the persistence process, ensuring that even if problems occur during persistence, detailed information is recorded for subsequent analysis and remediation.

[0161] S5.3 checks whether there are persistent files on the local disk when the service starts, reads the file content and deserializes it into a short link list, adds the short links in the short link list to the private short link pool, and obtains the resource loading result.

[0162] During service startup, the system implements an automatic recovery mechanism for private short link pool resources. First, the system checks if persistent files exist in a predefined path. If a file matching the naming rules is found, the system reads the file content and converts the binary data back into a short link list using the appropriate deserialization mechanism. To ensure data integrity, the system implements strict verification logic during deserialization, including checksum verification and type safety checks, to prevent recovery failures due to file corruption or format changes. The recovered short link list is then added to the newly created private short link pool, quickly restoring the resource state of the service instance.

[0163] S5.4 Based on the resource loading result, delete the persistent file to avoid repeated loading. During the short link entry process, capture the unique key conflict exception and re-obtain a new short link from the private short link pool to replace the conflict item. Use recursive retry to ensure that the short link allocation can be successfully completed in the end, thereby realizing the local persistence and fault tolerance of the private short link pool.

[0164] It's worth noting that the system immediately deletes persistent files after resource recovery is complete, preventing the same batch of resources from being reloaded on the next startup. This "one-time recovery" strategy prevents resource duplication and simplifies resource management logic. Before deleting files, the system typically records the number of recovered short links and related statistics. This data serves as an important indicator for operation and maintenance monitoring and as a basis for verifying the successful completion of the recovery process.

[0165] The system also incorporates a dedicated fault-tolerance mechanism to address potential resource conflicts. When allocating a short link from the recovered private short link pool and attempting to write it to the database, a unique key conflict exception may occur—this typically happens during service restarts, when previously persisted short links have already been allocated and used by other instances. The system catches these exceptions and implements an intelligent retry strategy: first, it retrieves a new short link from the private short link pool to replace the conflicting item, and then retryes the database operation. This process employs a recursive retry pattern to ensure successful short link allocation and prevent request failures due to resource conflicts. The system also sets a maximum retry limit to prevent getting stuck in an infinite retry loop in extreme cases, and records detailed conflict and retry logs for subsequent problem analysis.

[0166] The local persistence and fault-tolerance mechanisms of the entire private short link pool provide robust resource protection for the multi-path short link resource scheduling system. In actual deployments, this mechanism significantly reduces resource waste caused by service restarts or cluster adjustments. Statistics show that it can retain and restore an average of approximately 1.2 million short link resources during each normal service restart, equivalent to saving the system several hours of resource generation time. More importantly, this mechanism ensures that even with frequent deployments and service updates, the system maintains high resource utilization and stable service quality.

[0167] Furthermore, this mechanism enhances the system's resilience and disaster recovery capabilities. In extreme cases, if the public short link pool becomes temporarily unavailable or the database connection is interrupted, the recovered private short link pool can serve as a temporary buffer, continuing to provide short link resources to the system and preventing service interruption. This "offline working" capability is a crucial component of the system's high availability design, significantly improving service continuity and user experience.

[0168] From a technical implementation perspective, the local persistence mechanism of the private short-chain pool fully leverages the event model of the Spring framework and Java's serialization mechanism, while incorporating best practices in file system operations and exception handling, making it an elegant and efficient solution. This design not only solves resource management problems in specific scenarios but also provides a valuable reference pattern for similar distributed resource scheduling systems.

[0169] Example 2:

[0170] Based on Example 1, this example provides a specific implementation method for retrieving multiple short chains from the public short chain pool each time and outputting the short chain resources retrieved in batches, including:

[0171] A1. Divide the public short chain pool into N logical partitions according to the identifier range. Each logical partition forms an independent resource path through different cursor ranges. Construct a multi-path short chain resource acquisition model, where N is a positive integer greater than 1.

[0172] In practical applications of distributed short-chain services, the single-path resource acquisition method often struggles to cope with resource contention issues in high-concurrency environments. To address this challenge, this invention employs an innovative multi-path short-chain resource scheduling mechanism, specifically implemented as follows:

[0173] First, the system divides the public short link pool into N logical partitions based on the identifier range. For example, the `surl_pool` table can be divided into 8 intervals based on the range of `id` values. Each logical partition is defined by a different cursor range, forming an independent resource acquisition path. This partitioning strategy breaks the limitation of sequential acquisition using a traditional single cursor, enabling the system to acquire resources from multiple intervals in parallel, similar to the multipath transmission mechanism in network communication, significantly improving the flexibility and efficiency of resource acquisition.

[0174] A2. Monitor the four key indicators of resource density, competition level, response time and conflict rate in real time for each resource path, record the four key indicators of the past multiple visits using the path status table, and calculate the weighted average of the four key indicators as the path health score.

[0175] For each resource path, the system monitors four key performance indicators in real time: resource density, contention level, response time, and conflict rate. Resource density is measured by calculating the number of available short links within a unit interval; contention level is determined by statistically analyzing the access frequency of multiple service instances to that path. Simultaneously, the system measures the average time spent acquiring resources from each path as the response time indicator, and calculates the proportion of unique index conflicts occurring during the acquisition process as the conflict rate indicator. These indicators are recorded in the path status table, storing historical data from multiple past accesses to provide a data foundation for subsequent path evaluation.

[0176] Based on collected historical monitoring data, the system uses a weighted average algorithm to calculate the health score for each path. During the calculation, different weights are assigned to resource density, the reciprocal of competition level, the reciprocal of response time, and the reciprocal of conflict rate, resulting in a comprehensive health score for each path. This multi-dimensional evaluation mechanism avoids the biased judgments that may result from relying on a single indicator, making path status assessment more comprehensive and accurate.

[0177] Furthermore, specifically, A2 includes:

[0178] A2.1 For each resource path, calculate the number of available short links within a unit interval as a resource density index, and count the access frequency of multiple service instances to each resource path as a competition level index to obtain a path basic status index, which includes the resource density index and the competition level index.

[0179] In a multi-path short-chain resource scheduling system, this invention implements a comprehensive path status monitoring and evaluation mechanism to accurately assess the health status of each resource path. This mechanism, through multi-dimensional indicator collection, historical data recording, and intelligent scoring algorithms, creates a precise profile of the resource paths, providing a solid data foundation for resource scheduling decisions.

[0180] First, the system calculates the number of available short chains within a unit interval for each resource path, serving as a resource density metric to measure the abundance of path resources. Specifically, this is calculated by retrieving the number of unallocated short chains in each logical partition using an SQL query, dividing it by the partition's total capacity, and obtaining the resource density value. For example, if a partition has a total capacity of 4 million short chains and currently has 3 million unallocated chains, its resource density is 0.75. Higher resource density indicates more abundant resources on the path, making it suitable for prioritizing allocation to more acquisition needs. Simultaneously, the system also tracks the access frequency of multiple service instances to each resource path, serving as a competition metric to measure the intensity of path contention. This is achieved by maintaining an access counter for each path in Redis, recording the number of times each service instance accesses the path per unit time (typically 1 minute), thus obtaining access frequency data. These two metrics together constitute the basic path status indicators, reflecting the path's resource status and usage intensity.

[0181] A2.2 The average time taken to obtain resources from each of the resource paths is measured as a response time indicator, and the proportion of unique index conflicts that occur when obtaining resources from each of the resource paths is calculated as a conflict rate indicator to obtain a path performance indicator, which includes the response time indicator and the conflict rate indicator.

[0182] Secondly, the system measures the average time spent acquiring resources from each resource path, serving as a response time metric to measure path response speed. Specifically, this is achieved by recording timestamps before and after each resource acquisition operation, calculating the time difference, and maintaining a sliding window to calculate the average time spent on the most recent N operations. Simultaneously, the system also calculates the proportion of unique index conflicts occurring when acquiring resources from each resource path, serving as a conflict rate metric to measure path reliability. The conflict rate is calculated as: the number of unique index conflicts divided by the total number of acquisition attempts. These two metrics constitute the path performance indicators, reflecting the path's operational efficiency and stability.

[0183] A2.3 Based on the path basic status indicators and path performance indicators, record the resource density, contention level, response time and conflict rate values ​​of the past N accesses in the path status table to obtain historical monitoring data, where N is a positive integer greater than 1;

[0184] Based on the aforementioned path status and performance metrics, the system records resource density, contention level, response time, and conflict rate values ​​for the past N visits in a specially designed path status table, forming complete historical monitoring data. The value of N is typically set between 10 and 30, providing sufficient historical samples without overloading processing due to excessive data. This historical data is used not only for current status assessment but also for analyzing path performance trends and predicting potential problems.

[0185] A2.4 Apply a weighted average algorithm to the historical monitoring data, set weights corresponding to resource density, reciprocal of competition level, reciprocal of response time and reciprocal of conflict rate, calculate the comprehensive health score, and obtain the path health score;

[0186] Based on the collected historical monitoring data, the system applies a carefully designed weighted average algorithm for comprehensive scoring. Notably, during the calculation process, the system uses the reciprocals of competition level, response time, and conflict rate as calculation factors. This ensures that the smaller these indicator values, the higher the corresponding contribution score, aligning with the optimal path principle of "low competition, low latency, and low conflict." The specific calculation formula is: Health Score = w1 × Resource Density + w2 × (1 / Competition Level) + w3 × (1 / Response Time) + w4 × (1 / Conflict Rate), where w1, w2, w3, and w4 are the corresponding weights, and their sum is 1. Through multiple experimental optimizations, the system typically sets w1=0.3, w2=0.2, w3=0.2, and w4=0.3. This weight allocation emphasizes both resource abundance and stability while also fully considering efficiency factors.

[0187] A2.5 Based on the path health score, the performance trend of each resource path is periodically evaluated to identify paths with degraded or abnormal performance, thereby achieving comprehensive monitoring and evaluation of path status.

[0188] Finally, based on the calculated path health score, the system periodically evaluates the performance trend of each resource path. Every certain time interval (usually 5 minutes), the system compares and analyzes the health scores for three consecutive evaluation periods to identify paths with continuously declining performance or abnormal fluctuations. When a path's health score drops by more than 15% twice consecutively, or drops by more than 30% in a single instance, the system marks it as a "performance-degraded path" and triggers special monitoring. When the score falls below a preset safety threshold (usually 0.4), it is marked as an "abnormal path" and may be temporarily excluded from resource scheduling.

[0189] This path status monitoring and evaluation mechanism is highly flexible and scalable. System administrators can adjust the weight configuration, evaluation cycle, and alarm thresholds of various indicators based on actual operating conditions, enabling the system to better adapt to different business scenarios and load characteristics. Furthermore, the system supports adding new monitoring indicators, such as path resource consumption rate and error rate, to further improve the comprehensiveness and accuracy of the evaluation.

[0190] Through this comprehensive path status monitoring and evaluation mechanism, the system can monitor the health status of each resource path in real time, providing accurate data support for multi-path resource scheduling and ensuring that the acquisition of short-chain resources is both efficient and stable. Practical application has proven that this mechanism enables the system to identify and avoid approximately 85% of potential path problems in advance, improving resource allocation efficiency by about 20% and providing a strong guarantee for the stable operation of the entire short-chain service.

[0191] A3. Based on the path health score, the weight of each path is dynamically adjusted using the exponential smoothing algorithm, and the amount of resources to be acquired is allocated proportionally according to the weight of each path to achieve multi-path resource scheduling;

[0192] After obtaining the path health score, the system dynamically adjusts the weights of each path using an exponential smoothing algorithm. This algorithm balances the influence of historical and recent data, ensuring both stability and timely response to changes in weight adjustments. Based on the adjusted path weights, the system proportionally allocates the required resources for the current task, achieving intelligent multi-path resource scheduling. For example, if 500,000 short links need to be acquired, the system will obtain the corresponding amount of resources from each of the eight paths according to their respective weight ratios.

[0193] A4. Obtain the short chain resources from each path in parallel, add the obtained short chain resources to the private short chain pool, update the status information and cursor position of each path, and obtain the short chain resources obtained in batches based on multi-path scheduling.

[0194] During resource acquisition, the system acquires short-chain resources from various paths in parallel, significantly improving acquisition efficiency. The acquired short-chain resources are added to a private short-chain pool, and the system simultaneously updates the status information and cursor position of each path, recording performance data for the current operation to provide a reference for future resource scheduling. This parallel acquisition and real-time feedback mechanism enables the system to continuously optimize its resource acquisition strategy.

[0195] A5. When congestion is detected on a certain path, a congestion control mechanism is activated to temporarily reduce the weight of the congested path and increase the acquisition ratio of other paths, or to add the congested path to a cooling queue, thereby achieving adaptive multi-path resource scheduling optimization.

[0196] When the system detects congestion on a path, it immediately activates a congestion control mechanism. The system uses an exponentially weighted moving average algorithm to calculate the congestion index for each resource path. When the real-time congestion index exceeds a preset threshold, the weight of the congested path is temporarily reduced, decreasing the amount of resources acquired from that path, while simultaneously increasing the acquisition ratio of other healthy paths. In cases of severe congestion, the system may even add the path to a cooling-off queue, setting a cooldown period during which access to the path is suspended. Its status is then reassessed after the cooldown period ends. This adaptive multi-path resource scheduling optimization mechanism effectively avoids resource contention hotspots and improves the overall system stability and resource acquisition efficiency.

[0197] Through the above multi-path short-chain resource scheduling mechanism, this invention significantly improves the efficiency of short-chain resource acquisition and system stability. Compared with the traditional single-path scheme, the resource utilization rate is increased by about 25%, the conflict rate is reduced by about 40%, and the system performance in a distributed high-concurrency environment is greatly enhanced.

[0198] Furthermore, when congestion is detected on a path, a congestion control mechanism is activated to temporarily reduce the weight of the congested path and increase the acquisition ratio of other paths, or to add the congested path to a cooling queue, thereby achieving adaptive multi-path resource scheduling optimization, specifically including:

[0199] The congestion index for each resource path is calculated using an exponentially weighted moving average algorithm. A smoothing factor is applied to the calculation based on the current conflict rate and the historical congestion index to obtain the real-time congestion index.

[0200] When the real-time congestion index exceeds a preset threshold, congestion detection is triggered. The weight of the congested path is temporarily reduced to decrease the amount of resources acquired, while the acquisition ratio of other healthy paths is increased. The access interval of the congested path is adjusted to avoid frequent competition. Alternatively, the congested path is added to a cooling queue and a cooling time is set. During the cooling period, the congested path is no longer accessed. After the cooling period ends, the status of the congested path is reassessed.

[0201] In multi-path short-chain resource scheduling mechanisms, this invention designs a sophisticated congestion control mechanism to promptly identify and address resource path congestion. This mechanism, based on an exponentially weighted moving average algorithm, can sensitively capture congestion signals and intelligently adjust resource acquisition strategies, ensuring stable and efficient system operation even under highly competitive environments.

[0202] The system employs the Exponentially Weighted Moving Average (EWMA) algorithm to calculate the congestion index for each resource path. The core of this algorithm lies in using a smoothing factor β to reasonably balance the weights of current and historical values, effectively filtering out short-term fluctuations while retaining long-term trend information. Specifically, the system applies a smoothing factor to calculate the real-time congestion index based on the current conflict rate and historical congestion index. The calculation formula is: Current Congestion Index = β × Current Conflict Rate + (1-β) × Historical Congestion Index, where β typically ranges from 0.1 to 0.3. This calculation method ensures that the congestion index reflects the current path status while maintaining a certain degree of stability, avoiding frequent strategy switching due to temporary fluctuations.

[0203] When the real-time congestion index exceeds a preset threshold, the system immediately triggers a congestion detection process. The preset threshold is typically set between 0.6 and 0.8 and can be dynamically adjusted based on system load characteristics. Once congestion is confirmed, the system immediately takes a series of measures for congestion control. First, the weight of the congested path is temporarily reduced, decreasing the amount of resources acquired from that path. The weight adjustment is proportional to the degree of congestion; the more severe the congestion, the greater the weight reduction, typically decreasing by 30% to 70%. Simultaneously, the system increases the acquisition ratio of other healthy paths to ensure that the overall resource acquisition is not affected.

[0204] In addition to adjusting weights, the system also adjusts the access interval of congested paths to avoid frequent contention exacerbating congestion. Specifically, this involves increasing the access interval of the path, for example, from once per second to once every 3-5 seconds, giving the path "breathing time" and helping to alleviate congestion. This dynamic adjustment of access frequency can significantly reduce contention pressure on the path without completely abandoning it.

[0205] For paths with particularly severe congestion, the system takes more aggressive measures, adding the path to a cooling-off queue and setting a cooling-off period. The length of the cooling-off period is dynamically determined based on the degree of congestion, generally ranging from tens of seconds to several minutes. During the cooling-off period, the system completely stops accessing the path, allocating all its resource acquisition needs to other healthy paths. This "forced rest" mechanism effectively breaks the vicious cycle, creating conditions for severely congested paths to recover. After the cooling-off period ends, the system reassesses the path's status. If the congestion index has dropped to a safe level, it is reinstated to normal scheduling; otherwise, the cooling-off period may be extended or other remedial measures may be taken.

[0206] In actual operation, the system dynamically adjusts the sensitivity of congestion control based on the overall load conditions. During low-load periods, the congestion detection threshold can be appropriately relaxed to reduce unnecessary path switching; while during peak high-load periods, the sensitivity is increased to respond promptly to congestion signals. This adaptive control strategy enables the system to maintain optimal performance under various load conditions.

[0207] Through this sophisticated congestion control mechanism, the system can intelligently detect and effectively respond to congestion on resource paths, ensuring a continuous and stable supply of short-chain resources. Actual testing shows that this mechanism can control system performance degradation caused by path congestion to within 10%, significantly improving the system's stability and reliability in highly competitive environments. Especially during periods of sudden traffic surges, the congestion control mechanism's adaptive adjustment capability significantly reduces the risk of system crashes, providing strong protection for short-chain services.

[0208] Example 3:

[0209] Based on Example 1, this example provides a cursor management mechanism, specifically including:

[0210] B1. Cursor management is divided into three basic strategies: greedy mode, cooperative mode and random mode. A weighted average is calculated based on the conflict rate index, resource acquisition delay and cursor update conflict to obtain a comprehensive competition index, which is used to evaluate the competition status.

[0211] In further optimization of the multi-path short-chain resource scheduling system, this embodiment innovatively establishes a cursor management mechanism. Through intelligent strategy selection and adaptive adjustment, it solves the complex challenges of cursor management in a distributed environment. This mechanism not only effectively reduces resource contention conflicts but also significantly improves the overall resource acquisition efficiency of the system.

[0212] The cursor management mechanism first defines three basic strategy modes: greedy mode, cooperative mode, and random mode. In greedy mode, service instances actively advance the cursor position to acquire as many contiguous resources as possible, suitable for low-contest environments. In cooperative mode, service instances proactively release the cursor lock and set an appropriate backoff distance after acquiring resources to avoid intense competition with other instances, suitable for high-contest environments. In random mode, service instances randomly select cursor positions, acquiring resources in leaps, increasing system exploratory capabilities and helping to discover new low-contest areas. Each of these three basic strategies has its advantages, and the system needs to intelligently select the most suitable strategy combination based on the real-time environment.

[0213] For example, the three basic strategies for cursor management are as follows:

[0214] Greedy mode: Prioritizes areas with high resource density and low competition as cursor positions to pursue maximum efficiency;

[0215] Cooperative mode: The cursor position is selected in turn according to a predefined order or rules to ensure fairness and system stability;

[0216] Random mode: Randomly selects the cursor position within the available range, increasing the unpredictability of the system and preventing periodic conflicts caused by multiple service instances forming a synchronization mode.

[0217] To accurately assess the current competitive state, the system calculates a comprehensive competition index based on three key indicators: conflict rate, resource acquisition latency, and cursor update conflict. The conflict rate reflects the frequency of unique index conflicts during resource acquisition; resource acquisition latency measures the time overhead from request initiation to resource acquisition completion; and cursor update conflict specifically tracks the number of conflicts during cursor position updates. The system applies a weighted average algorithm to these three indicators, typically assigning a weight of 0.4 to the conflict rate, 0.3 to the resource acquisition latency, and 0.3 to the cursor update conflict, resulting in a comprehensive competition index between 0 and 1, used to quantify the intensity of competition within the current system.

[0218] B2. Based on the comprehensive competition index, dynamically adjust the cursor management strategy to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, and output a hybrid strategy selection mechanism;

[0219] Based on the calculated comprehensive competition index, the system dynamically adjusts its cursor management strategy. When the comprehensive competition index is high (typically greater than 0.6), the system determines that it is currently in a first-level competitive environment (high-competition environment) and significantly increases the proportion of cooperative strategies, for example, raising the probability of selecting cooperative strategies to around 70%. Conversely, when the comprehensive competition index is low (typically less than 0.3), the system determines that it is currently in a second-level competitive environment (low-competition environment) and increases the proportion of greedy strategies, raising the probability of selecting greedy strategies to around 60%. For medium-competition environments (comprehensive competition index between 0.3 and 0.6), the system adopts a more balanced strategy combination. This dynamic adjustment mechanism allows the system to flexibly switch strategies according to environmental changes, maintaining optimal performance.

[0220] Specifically, B2 includes:

[0221] B2.1 Regularly analyze the performance of each strategy under different competitive environments, establish a strategy-environment-effect mapping relationship by statistically analyzing the strategy effects according to the competition index range, and obtain historical performance analysis results;

[0222] B2.2 Based on the historical performance analysis results, when the current competition index falls into a specific range, the optimal strategy within that specific range is determined. The strategy selection parameters are adjusted to favor the optimal strategy and the internal parameters of the strategy are optimized to achieve self-tuning of the strategy parameters. This is to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, and output a hybrid strategy selection mechanism. The degree of competition in the first competitive environment is greater than that in the second competitive environment.

[0223] B2.3 broadcasts the policy selection mode of the first instance to other instances, obtains the policy selection information of other instances, and adds randomness to the policy selection information to avoid policy selection synchronization, thereby achieving cluster policy coordination.

[0224] To continuously optimize strategy selection, the system periodically analyzes the performance of each strategy under different competitive environments. Specifically, the system statistically analyzes the effectiveness of each strategy according to a comprehensive competition index range (usually dividing the range from 0 to 1 into 10 intervals), establishing a mapping relationship between strategy, environment, and effect. For example, the system might find that in the competition index range of 0.7-0.8, the average success rate of the cooperative strategy is 85%, while that of the greedy strategy is only 40%; while in the competition index range of 0.1-0.2, the average success rate of the greedy strategy reaches 95%, and that of the cooperative strategy is 80%. These historical performance analysis results provide a data foundation for strategy optimization.

[0225] Based on historical performance analysis, the system implements self-tuning of strategy parameters. When the current competition index falls within a specific range, the system queries the best strategy within that range and adjusts the strategy selection parameters accordingly, favoring the strategy with the best historical performance. For example, if historical data shows that the cooperative strategy performs best in the competition index range of 0.7-0.8, when the current competition index is 0.75, the system will increase the probability of selecting the cooperative strategy to 80% or higher. Simultaneously, the system optimizes internal strategy parameters, such as adjusting the cursor backoff distance in cooperative mode and adjusting the amount of resources acquired per instance in greedy mode, making strategy execution more efficient. This self-tuning mechanism allows the system to continuously learn and adapt to different competitive environments, always maintaining near-optimal performance.

[0226] In a distributed environment, to avoid new points of contention caused by multiple service instances adopting the same strategy, the system implements a cluster strategy coordination mechanism. Each service instance broadcasts its strategy selection mode to other instances in the cluster and simultaneously obtains the strategy selection information of other instances. Upon receiving this information, the instance adds a certain degree of randomness to its own strategy selection to avoid complete synchronization with the strategy selections of other instances. For example, if it finds that multiple instances are using a cooperative strategy, the current instance may slightly increase the proportion of a greedy or random strategy to reduce potential strategy conflicts. This coordination mechanism maintains the rationality of the overall strategy while avoiding new problems caused by excessive homogenization.

[0227] Through this complex and sophisticated cursor management mechanism, the multi-path short-chain resource scheduling system can efficiently manage cursor positions under various competitive environments, significantly reducing resource acquisition conflicts. Actual testing shows that compared to a fixed strategy, this mechanism reduces the resource acquisition conflict rate by approximately 60%, the average resource acquisition latency by approximately 45%, and the overall system throughput by approximately 30%. Especially under high-load and fluctuating environments, the mechanism's adaptive capabilities enable the system to maintain stable high performance, providing a reliable resource supply guarantee for short-chain services.

[0228] Another significant advantage of this hybrid strategy cursor management mechanism is its strong scalability. System administrators can easily add new cursor management strategies or adjust the internal implementation of existing strategies according to specific business needs. As long as a standard performance evaluation interface is provided for the new strategy, the system can automatically incorporate it into the self-tuning framework, enabling continuous expansion and optimization of the strategy pool. This design philosophy allows the cursor management mechanism to continuously evolve and address potential new competitive models and resource allocation challenges in the future.

[0229] B3. Under the hybrid strategy selection mechanism, a choice is made between the cooperative strategy and the greedy strategy according to the strategy selection parameters, and a random strategy is used to increase the exploratory nature under the first probability. The strategy execution results are recorded and the strategy effect score is updated to realize hybrid strategy cursor management.

[0230] After determining the strategy combination, the system implements a hybrid strategy selection mechanism. Each time the cursor needs to be managed, the system probabilistically selects between a cooperative strategy and a greedy strategy based on the current strategy selection parameters. For example, if the current cooperative strategy has a 70% weighting and the greedy strategy has a 30% weighting, the system will have a 70% probability of selecting the cooperative strategy and a 30% probability of selecting the greedy strategy. In addition, the system will also select a random strategy with a certain probability (usually 10% to 20%) to increase exploration and discover potential high-quality resource areas. After each strategy execution, the system records the strategy execution results in detail, including resource acquisition success rate, time consumption, number of conflicts, and other indicators, and updates the strategy's effectiveness score in the current environment, providing a reference for subsequent strategy selection.

[0231] Example 4:

[0232] This embodiment provides a distributed short-chain resource management and fault-tolerant system based on a two-layer pool architecture, such as... Figure 2 As shown, it includes:

[0233] A creation module is used to create a public short chain pool table, set an auto-incrementing primary key and a unique index for short chains in the public short chain pool table, and construct a public short chain pool corresponding to the public short chain pool table. The public short chain pool serves as a central storage repository for short chain resources.

[0234] The write module is used to obtain the distributed lock control single instance execution based on the capacity status of the public short link pool, randomly generate a batch of short link resources using a preset character set, and perform batch write operations using insert-ignore statements to replenish the resources of the public short link pool.

[0235] The database entry module is used to create a private short link pool based on a concurrent linked list queue in the memory of each service instance. The capacity of the private short link pool is monitored by a scheduled task, and short links are retrieved in batches from the public short link pool to the private short link pool to realize the database entry of short links. The private short link pool is used for local caching of short link resources.

[0236] The anomaly detection and self-repair module is used to obtain the cursor position of the public short chain pool, monitor the unique index conflict anomaly during the short chain entry process, and trigger the cursor anomaly detection and self-repair operation based on the unique index conflict anomaly to realize the dual storage cursor mechanism.

[0237] The local persistence and fault tolerance module is used to listen for service shutdown events and trigger the storage of resources of the private short link pool to local disk files. When the service starts, it loads the resources of the private short link pool to realize the local persistence and fault tolerance of the private short link pool.

[0238] This invention effectively addresses several challenges in short-chain resource management in a distributed environment through a two-layer pool architecture design, achieving efficient short-chain resource management and fault tolerance mechanisms, and significantly improving system stability, response speed, and resource utilization.

[0239] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A distributed short-chain resource management and fault-tolerance method based on a two-layer pool architecture, characterized in that, Includes the following steps: Create a public short link pool table, set an auto-incrementing primary key and a unique index for short links in the public short link pool table, and construct a public short link pool corresponding to the public short link pool table. The public short link pool serves as a central storage repository for short link resources. Based on the capacity status of the public short link pool, the distributed lock control single instance is executed, a batch of short link resources are randomly generated using a preset character set, and a batch write operation is performed using the insert-ignore statement to replenish the resources of the public short link pool. A private short link pool based on a concurrent linked list queue is created in the memory of each service instance. A scheduled task monitors the capacity of the private short link pool and retrieves short links from the public short link pool in batches to the private short link pool for short link storage. The private short link pool serves as a local cache for short link resources. This includes: using a concurrent linked list queue as the data structure for the private short link pool in the memory of each service instance; setting a maximum capacity and a replenishment threshold for the private short link pool; implementing a scheduled task every second to check the current length of the private short link pool through a scheduled execution service; triggering a replenishment operation when the current length of the private short link pool is lower than the replenishment threshold and generating a replenishment trigger condition; acquiring a distributed lock based on the replenishment trigger condition to ensure the serial execution of the retrieval operation in a multi-service instance environment; retrieving multiple short links from the public short link pool each time and outputting the batched short link resources; adding the batched short link resources to the concurrent linked list queue of the private short link pool, and recording the identifier of the last record as a new cursor value; the private short link pool serves as a local cache for short link resources. The process involves obtaining the cursor position of the public short link pool, monitoring for unique index conflicts during the short link insertion process, and triggering cursor anomaly detection and self-repair operations based on these unique index conflicts to achieve a dual-storage cursor mechanism. This includes: storing cursor values ​​as strings in Redis and creating a dedicated cursor table in MySQL to store historical cursor values ​​and update times, thus establishing a dual-storage cursor structure; monitoring for unique index conflicts during the short link insertion process, and when a conflict occurs, obtaining the identifier value A of the latest short link in the private short link pool, the cursor value B in Redis, and the cursor value C in MySQL; calculating the maximum value among these three values ​​to determine the current correct cursor position, and synchronously updating the correct cursor value to Redis and MySQL to achieve cursor self-repair; calculating the difference between the correct cursor value and the minimum of the three values ​​to determine the number of duplicate short links, and deleting the corresponding number of short links from the private short link pool to achieve the dual-storage cursor mechanism. The system listens for service shutdown events and triggers the storage of resources from the private short link pool to local disk files. When the service starts, it loads the resources from the private short link pool to achieve local persistence and fault tolerance of the private short link pool.

2. The method according to claim 1, characterized in that, Create a public short link pool table, set an auto-incrementing primary key and a unique index for short links in the public short link pool table, and construct a public short link pool corresponding to the public short link pool table. The public short link pool serves as a central storage repository for short link resources, including: Based on the MySQL database environment, the public short link pool table is created, and the identifier field of the public short link pool table is set as an auto-incrementing primary key to establish the primary key constraint of the public short link pool table. Set the short link field in the public short link pool table to a six-character variable-length type and create a unique index constraint for the short link field to ensure the uniqueness of short links; Based on the primary key constraint and unique index constraint, the public short link pool table is configured as a transaction storage engine, and a public short link pool corresponding to the public short link pool table is constructed. The public short link pool serves as a central storage repository for short link resources.

3. The method according to claim 1, characterized in that, Based on the capacity status of the public short link pool, the distributed lock control single instance is executed. A batch of short link resources are randomly generated using a preset character set, and batch write operations are performed using insert-ignore statements to replenish the resources of the public short link pool, including: Use Redis's atomic set command to create a distributed lock identifier with an expiration time, obtain the distributed lock to control the single-instance execution of supplementary tasks, and ensure concurrency safety; Define a preset character set containing 62 characters including numbers and uppercase and lowercase letters. Use a secure random number class to generate a random number and randomly extract 6 characters from the preset character set using a character acquisition method to generate a single short chain. The single short chain is batch generated, and multiple short chains are generated in each batch to obtain the batch short chain resources; Based on the batch short link resources, the multiple short links are inserted into the public short link pool table in batches using an insert-ignore statement, and the number of non-duplicate short links that were successfully inserted is obtained through a row count function, thereby replenishing the resources of the public short link pool.

4. The method according to claim 1, characterized in that, Multiple short chains are retrieved from the public short chain pool each time, and the short chain resources retrieved in batches are output, including: The public short chain pool is divided into N logical partitions according to the identifier range. Each logical partition forms an independent resource path through different cursor ranges, thus constructing a multi-path short chain resource acquisition model, where N is a positive integer greater than 1. For each resource path, four key indicators are monitored in real time: resource density, competition level, response time, and conflict rate. The path status table records the four key indicators accessed multiple times in the past, and the weighted average of the four key indicators is calculated as the path health score. Based on the path health score, the weight of each path is dynamically adjusted using an exponential smoothing algorithm, and the amount of resources to be acquired is allocated proportionally according to the weight of each path to achieve multi-path resource scheduling. The short chain resources are obtained in parallel from each path, and the obtained short chain resources are added to the private short chain pool. The status information and cursor position of each path are updated to obtain the short chain resources in batches based on multi-path scheduling. When congestion is detected on a certain path, a congestion control mechanism is activated to temporarily reduce the weight of the congested path and increase the acquisition ratio of other paths, or to add the congested path to a cooling queue, thereby achieving adaptive multi-path resource scheduling optimization.

5. The method according to claim 4, characterized in that, For each resource path, four key indicators are monitored in real time: resource density, competition level, response time, and conflict rate. The path status table records these four key indicators from past visits, and a weighted average of these four key indicators is calculated as the path health score, including: The number of available short links within a unit interval is calculated for each resource path as a resource density index, and the access frequency of multiple service instances to each resource path is counted as a competition level index to obtain the basic path status index, which includes the resource density index and the competition level index. The average time taken to obtain resources from each of the resource paths is measured as a response time indicator, and the proportion of unique index conflicts that occur when obtaining resources from each of the resource paths is calculated as a conflict rate indicator to obtain a path performance indicator, which includes the response time indicator and the conflict rate indicator. Based on the path basic status indicators and path performance indicators, the resource density, contention level, response time and conflict rate values ​​of the past N accesses are recorded in the path status table to obtain historical monitoring data, where N is a positive integer greater than 1. A weighted average algorithm is applied to the historical monitoring data, with weights corresponding to resource density, the reciprocal of competition level, the reciprocal of response time, and the reciprocal of conflict rate, respectively, to calculate the comprehensive health score and obtain the path health score; Based on the path health score, the performance trend of each resource path is periodically evaluated to identify paths with degraded or abnormal performance, thereby achieving comprehensive monitoring and evaluation of path status.

6. The method according to claim 5, characterized in that, When congestion is detected on a path, a congestion control mechanism is activated to temporarily reduce the weight of the congested path and increase the acquisition ratio of other paths, or to add the congested path to a cooling queue, thereby achieving adaptive multi-path resource scheduling optimization, including: The congestion index for each resource path is calculated using an exponentially weighted moving average algorithm. A smoothing factor is applied to the calculation based on the current conflict rate and the historical congestion index to obtain the real-time congestion index. When the real-time congestion index exceeds a preset threshold, congestion detection is triggered. The weight of the congested path is temporarily reduced to decrease the amount of resources acquired, while the acquisition ratio of other healthy paths is increased. The access interval of the congested path is adjusted to avoid frequent competition. Alternatively, the congested path is added to a cooling queue and a cooling time is set. During the cooling period, the congested path is no longer accessed. After the cooling period ends, the status of the congested path is reassessed.

7. The method according to claim 1, characterized in that, The method further includes: A cursor management mechanism is established, which divides cursor management into three basic strategies: greedy mode, cooperative mode and random mode. In the greedy mode, the region with resource density higher than a first preset threshold and competition level lower than a second preset threshold is selected as the cursor position. In the cooperative mode, the cursor positions are selected in turn according to a predefined order. In the random mode, the cursor position is selected randomly within the available range. A comprehensive competition index is obtained by calculating a weighted average of conflict rate, resource acquisition delay, and cursor update conflict, which is used to assess the competition status. Based on the comprehensive competition index, the cursor management strategy is dynamically adjusted to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, thus outputting a hybrid strategy selection mechanism. Under the hybrid strategy selection mechanism, a choice is made between the cooperative strategy and the greedy strategy according to the strategy selection parameters, and a random strategy is used to increase the exploratory nature under the first probability. The strategy execution results are recorded and the strategy effect score is updated to realize hybrid strategy cursor management.

8. The method according to claim 7, characterized in that, Based on the comprehensive competition index, the cursor management strategy is dynamically adjusted to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, outputting a hybrid strategy selection mechanism, including: Regularly analyze the performance of each strategy under different competitive environments, establish a strategy-environment-effect mapping relationship by statistically analyzing the strategy effects according to the competition index range, and obtain historical performance analysis results; Based on the historical performance analysis results, when the current competition index falls into a specific range, the optimal strategy within that specific range is determined. The strategy selection parameters are adjusted to favor the optimal strategy and the internal parameters of the strategy are optimized to achieve self-tuning of the strategy parameters. This is to increase the proportion of cooperative strategies in the first competitive environment and increase the proportion of greedy strategies in the second competitive environment, outputting a hybrid strategy selection mechanism. The degree of competition in the first competitive environment is greater than that in the second competitive environment. The strategy selection mode of the first instance is broadcast to other instances to obtain the strategy selection information of other instances. Randomness is added to the strategy selection information to avoid strategy selection synchronization and achieve cluster strategy coordination.

9. The method according to claim 1, characterized in that, The system listens for service shutdown events and triggers the storage of resources from the private short link pool to local disk files. When the service starts, it loads the resources from the private short link pool, achieving local persistence and fault tolerance of the private short link pool, including: Use Spring framework's event listener annotations to listen for context shutdown events, capture signals of normal service shutdown, and generate service shutdown event triggering conditions; Based on the service shutdown event triggering condition, the remaining short links in the private short link pool are obtained and converted into a list collection. The list collection is serialized and written to a local disk file using file input / output operations to achieve resource persistence of the private short link pool. When the service starts, check if there are persistent files on the local disk, read the file content and deserialize it into a short link list, add the short links in the short link list to the private short link pool, and obtain the resource loading result. Based on the resource loading result, the persistent file is deleted to avoid repeated loading. During the short link entry process, a unique key conflict exception is captured and a new short link is retrieved from the private short link pool to replace the conflicting item. A recursive retry method is used to ensure that the short link allocation can be successfully completed in the end, thereby realizing the local persistence and fault tolerance of the private short link pool.

10. A distributed short-chain resource management and fault-tolerant system based on a two-layer pool architecture, characterized in that, include: A creation module is used to create a public short chain pool table, set an auto-incrementing primary key and a unique index for short chains in the public short chain pool table, and construct a public short chain pool corresponding to the public short chain pool table. The public short chain pool serves as a central storage repository for short chain resources. The write module is used to obtain the distributed lock control single instance execution based on the capacity status of the public short link pool, randomly generate a batch of short link resources using a preset character set, and perform batch write operations using insert-ignore statements to replenish the resources of the public short link pool. The database entry module is used to create a private short link pool based on a concurrent linked list queue in the memory of each service instance. It monitors the capacity of the private short link pool through a scheduled task and retrieves short links from the public short link pool in batches to the private short link pool to achieve short link entry. The private short link pool serves as a local cache for short link resources. This includes: using a concurrent linked list queue as the data structure for the private short link pool in the memory of each service instance; setting a maximum capacity and a replenishment threshold for the private short link pool; implementing a scheduled task every second to check the current length of the private short link pool through a scheduled execution service; triggering a replenishment operation when the current length of the private short link pool is lower than the replenishment threshold and generating a replenishment trigger condition; acquiring a distributed lock based on the replenishment trigger condition to ensure the serial execution of the retrieval operation in a multi-service instance environment; retrieving multiple short links from the public short link pool each time and outputting the batched short link resources; adding the batched short link resources to the concurrent linked list queue of the private short link pool, and recording the identifier of the last record as a new cursor value. The private short link pool serves as a local cache for short link resources. An anomaly detection and self-repair module is used to obtain the cursor position of the public short link pool, monitor unique index conflict anomalies during the short link insertion process, and trigger cursor anomaly detection and self-repair operations based on the unique index conflict anomalies to achieve a dual-storage cursor mechanism. This includes: storing cursor values ​​as strings in Redis and creating a dedicated cursor table in MySQL to store historical cursor values ​​and update times, establishing a dual-storage structure for the cursor; monitoring unique index conflict anomalies during the short link insertion process; when a conflict occurs, obtaining the identifier value A corresponding to the latest short link in the private short link pool, the cursor value B in Redis, and the cursor value C in MySQL; calculating the maximum value among these three values ​​to determine the current correct cursor position, and synchronously updating the correct cursor value to Redis and MySQL to achieve cursor self-repair; calculating the difference between the correct cursor value and the minimum of the three values ​​to determine the number of duplicate short links, and deleting the corresponding number of short links from the private short link pool to achieve the dual-storage cursor mechanism. The local persistence and fault tolerance module is used to listen for service shutdown events and trigger the storage of resources of the private short link pool to local disk files. When the service starts, it loads the resources of the private short link pool to realize the local persistence and fault tolerance of the private short link pool.

Images