Cloud storage service quality management method based on burst I / O perception

Abstract

The invention discloses a cloud storage service quality management method based on burst I / O perception, and aims to solve the problems of low resource allocation efficiency and insufficient fairness when an existing scheduling algorithm processes periodic and burst I / O loads. The method comprises the steps of user registration and initialization, scheduling label calculation, scheduler requesting and staged scheduling. During user registration, users are divided into internal users and external users of the system according to access sources and characteristics, the external users are further divided into common users and sudden users, and priorities and scheduling parameters are configured. The system extracts a scheduling template based on user information and generates a scheduling label. The scheduler adopts a two-layer architecture of a small top heap and a polling chain table, and executes scheduling according to a descending order of priorities. A sudden user manages and controls the activation duration through a time window, and a dynamic compensation mechanism is started when the system load is insufficient. According to the invention, burst load sensitive identification and differential response are realized, and the accuracy of service quality, the fairness of resource scheduling and the expandability of the system are improved.

Description

Technical Field

[0001] This invention relates to the fields of network communication and computer science, and in particular to a cloud storage service quality management method based on bursty I / O awareness. Background Technology

[0002] In current distributed storage systems, existing scheduling algorithms can effectively support resource allocation for stable requests, providing fair and consistent I / O bandwidth. However, with the rapid growth of data analytics and machine learning / deep learning applications in cloud storage services, these applications exhibit periodic and bursty I / O load characteristics, posing new challenges to Quality of Service (QoS) management. Existing methods often struggle to meet performance requirements when handling such dynamic loads, particularly in terms of resource allocation efficiency and fairness.

[0003] First, traditional scheduling methods are coarse-grained in their request classification. Taking Ceph as an example, its dmClock algorithm only categorizes requests into three main types: Client, Background recovery, and Background best-effort. dmClock creates a scheduling client for each type of request, and each time a request arrives, it is added to the corresponding client's request queue. Each scheduling client contains three key scheduling parameters: Reservation, representing the minimum resource guarantee the system reserves for it; Limit, representing the maximum resource limit it can use; and Weight, used to determine how to proportionally allocate remaining resources after the reserved resources are satisfied. However, this coarse-grained classification results in external user requests being scheduled in a first-in, first-out manner, lacking fine-grained management at the user level. In high-load scenarios, the system struggles to dynamically allocate resources based on user priority or task importance. For example, machine learning / deep learning training tasks, when generating intensive I / O requests due to data loading, can significantly interfere with the performance of ordinary users.

[0004] Second, existing scheduling mechanisms have significant limitations in dealing with periodic bursts of I / O load. Applications such as machine learning / deep learning often generate high-density I / O requests in a short period of time, followed by a relatively long idle period. However, algorithms such as dmClock lack burst detection and cannot utilize the resources relinquished by users during idle periods to provide additional bandwidth during peak times.

[0005] Third, the bottleneck of scheduling efficiency under large-scale loads is becoming increasingly prominent. Taking dmClock as an example, its scheduling algorithm based on a heap data structure has a time complexity of O(log n) when managing request tags. In large-scale distributed storage systems, the number of users may reach tens of thousands, and each scheduling requires multiple adjustments to the heap structure, leading to a significant increase in computational overhead. Tests show that when the number of users increases to 8191, the system performance is only 25% of that when there is only one user. Frequent heap adjustments exacerbate scheduling latency, thereby severely reducing the overall system efficiency.

[0006] To address the aforementioned challenges, this paper proposes a burst-I / O-aware cloud storage service quality management method to improve the efficiency and fairness of resource allocation in cloud storage systems under diverse user demands. To resolve the performance bottlenecks and resource contention issues of traditional methods under burst traffic, a burst-aware hierarchical scheduling mechanism is designed, and a high-performance scheduler architecture is introduced. These improvements solve the efficiency and fairness challenges in dynamic loads, thereby shortening the completion time of bursty tasks, increasing system throughput, and ensuring balanced resource allocation. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of the prior art by proposing a cloud storage service quality management method based on burst I / O awareness. By proposing request classification and priority management, a burst-aware hierarchical scheduling mechanism and a high-performance scheduler architecture, this invention achieves sensitive identification of burst loads, differentiated response to multi-user QoS requirements, and low-latency support in large-scale scheduling scenarios.

[0008] The technical solution to achieve the purpose of this invention is:

[0009] A cloud storage service quality management method based on bursty I / O awareness, characterized by the following steps:

[0010] Step S1, User Registration and System Initialization Configuration: During the system initialization phase, all users are divided into internal system users and external system users based on their access sources and request characteristics. External system users include, but are not limited to, access requests initiated by end users, and are further divided into ordinary users and bursty users based on their I / O modes and business characteristics.

[0011] Step S2: Calculate the scheduling label for the request: After a user sends a request, the system extracts a scheduling template matching the user from the internal parameter template library based on the information provided during the user registration phase. The information provided during the user registration phase includes, but is not limited to, user type, whether it is a burst request, and priority level. The scheduling template defines resource control parameters used to allocate a reservation label (Reservation), a limit label (Limit), and a weight label (Weight) to the request, i.e., {R, L, W}. All labels use the same basic allocation principle. Specifically, the R scheduling label reflects the system's basic service guarantee for the user; the L scheduling label defines the upper limit of resources that a single user can use per unit of time; the W scheduling label plays a leading role in the resource surplus phase and participates in competitive resource allocation according to the weight ratio set by each user.

[0012] Step S3: Place the request in the scheduler to wait for scheduling: After completing the scheduling label calculation, the system places the request in the scheduler; the scheduler adopts a two-layer architecture, manages client categories through a min-heap, each element corresponds to a client category, and its priority is determined by the label of the next request to be scheduled in that category, and manages clients of the same category through a polling linked list;

[0013] Step S4, Scheduling Phase: The scheduling phase is divided into three phases with decreasing priority: reserved scheduling phase, burst scheduling phase, and weighted scheduling phase; the scheduler proceeds to the next phase only if there are no schedulable requests in a higher priority scheduling phase.

[0014] Step S5, Reservation Scheduling Phase: The scheduler checks the reservation tags of all requests and defines the set of schedulable requests E as: E = {r|R≤t_now}, where r represents a request, R is the reservation tag of request r, t_now is the current system time, and E represents the set of all requests whose R tags are less than the current system time. Only requests that meet this condition will be scheduled. If E is a non-empty set, the scheduler selects the request with the smallest reservation tag in E: Next Request = arg min_{r∈E}R, where Next Request represents the scheduler selecting a request with the smallest reservation tag R from the set of requests E that meet the scheduling condition. This request will be processed when memory is idle. If set E is empty, meaning there are no schedulable requests in the reservation scheduling phase, the scheduler will enter the burst scheduling phase.

[0015] Step S6, Burst Scheduling Phase: When there are no scheduling requests in the reserved scheduling phase, i.e., when set E in step S5 is empty, the scheduler enters the burst scheduling phase; clients are divided into four states by tags: ready without time slice limit, ready with time slice limit, not ready without time slice limit, and not ready with time slice limit. Only clients in the ready without time slice limit state can participate in scheduling in this phase.

[0016] The scheduler first marks requests that meet the constraint label condition as ready, whereby the constraint label L is less than or equal to the current system time t_now, where the constraint label L is the resource usage limit identifier of request r, and the current system time t_now is the real-time system time at the time of scheduling; a schedulable burst request set B is defined as the set of all requests in the ready state and not in the time-slice-limited state; if set B is not empty, the scheduler selects the request with the smallest weight label W in set B as the next scheduled request, where the weight label W is the resource contention priority identifier of request r; if set B is empty, the scheduler enters the weighted scheduling phase;

[0017] Step S7, Weighted Scheduling Phase: The scheduler marks requests that meet the following condition as ready: L≤t_now. The set of schedulable requests E' is defined as: E'={r|L≤t_now}, representing the set of all requests with L tags less than the current system time. Only requests meeting this condition will be scheduled. If set E' is non-empty, Next Request=arg min_{r∈E'}W, where Next Request represents the scheduler selecting the request with the smallest weight tag W from the set of requests E' that meet the scheduling condition. This request will be processed when memory is idle. If E' is empty, the scheduler returns an empty request, indicating that the system currently has no requests that can be processed.

[0018] Furthermore, in step S1, the internal users of the system mainly refer to tasks automatically generated by the system, including but not limited to copy repair, data rebalancing, and background maintenance operations; the ordinary users exhibit a stable, continuous, and small-batch data access mode, including but not limited to file read / write, synchronization, and backup operations; the sudden users have high concurrency, concentrated instantaneous load, and obvious periodic I / O characteristics, and the scenarios in which they occur are business scenarios that put great pressure on storage system resources, including but not limited to AI training and big data analysis.

[0019] Furthermore, the users within the system in step S1 are further divided into two categories: mandatory tasks and degradeable tasks, corresponding to high-priority and low-priority scheduling strategies, respectively, and a dedicated scheduling template parameter is configured for each type of task.

[0020] Furthermore, in step S1, the system assigns scheduling priorities to users based on their account level, service type, and historical activity level when they register, and divides ordinary users into three categories: high priority HP, medium priority MP, and low priority LP. Each priority category corresponds to a set of statically configured scheduling templates, which include three parameters: minimum resource guarantee, maximum resource limit, and resource contention weight.

[0021] Furthermore, in step S1, during the registration phase, the system allows users experiencing bursts to actively configure their scheduling parameters, including the total data budget I / O quota budget for each burst cycle and the user-defined maximum bandwidth expectation b0; the system uses these scheduling parameters to derive the maximum burst scheduling time window for each user's scheduling cycle to control the activation duration of burst tasks, calculated using the following formula:

[0022]

[0023] In the formula, IOQB stands for I / O quota budget; during each burst cycle, the system will prioritize allocating up to b0 bandwidth to the user until the system can no longer guarantee the maximum expected bandwidth for each user as the system load changes.

[0024] When the system cannot meet sudden requests, i.e., the bandwidth currently provided by the system is less than the maximum expected bandwidth set by the user, a dynamic compensation mechanism is used to compensate for the service by extending the subsequent scheduling cycle. The calculation formula is as follows:

[0025]

[0026] Where b0 is the maximum bandwidth expectation set by the user, b represents the actual bandwidth currently provided by the system, t is the time that the current time window has been running, and Δt is the amount of time that the system needs to compensate in future scheduling cycles.

[0027] Furthermore, in step S2, when client i of type t sends the r-th request, the W tag assigned to the request is the request arrival time and the previous request W tag plus 1 / w. t Maximum value:

[0028]

[0029] This formula is used to calculate the weighted scheduling label. The scheduler prioritizes processing requests with the smallest weight label without violating the constraints. The scheduling constraint is that the current system time must be greater than the limit label value of the request.

[0030] The meanings of each symbol are as follows:

[0031] t: Indicates the type of client, such as high priority, medium priority, low priority;

[0032] i: Represents the client number, used to distinguish different clients of the same type;

[0033] r: indicates that this request is the r-th request sent by client i;

[0034] This represents the weighted scheduling label for the r-th request from client i.

[0035] This represents the weighted scheduling label of the previous request of client i, i.e., the (r-1)th request;

[0036] w t This is a weighted scheduling parameter, shared by all clients of the same type; the larger the value, the larger the label step size (1 / w). t The smaller the value, the higher the request scheduling frequency for this type of client;

[0037] Current time: Represents the current system time.

[0038] Furthermore, in step S3, the polling list has a Next_Process pointer, which points to the next client to be scheduled in the current category. When a new request arrives, if the user is not active and is not in the category list, it is inserted into the position pointed to by the Next_Process pointer and marked as active. If the user is already active, the request is appended to the end of the user's queue. After the scheduling cycle ends, the user is moved to the end of the list, and the Next_Process pointer is updated to point to the subsequent client.

[0039] Furthermore, in step S6, the ready state means that scheduling the request will not cause the client to exceed the speed limit, and the client will be re-marked as non-ready after each request is scheduled; the time-slice limit state means that the client has reached the traffic usage limit for the current period, and the client is not allowed to schedule requests within this period; at the beginning of each period, the system will reinitialize the time window and allocate it to the user, and at the same time set the user state to non-time-slice limit state, where the time window is the maximum duration for which the user is allowed to use resources within a single burst period.

[0040] Furthermore, in step S6, the time window management for sudden client events follows these rules:

[0041] If the client's timer is paused, then start the timer;

[0042] If the client has only one request remaining and that request is not in a time-slice-limited state, then pause the timer;

[0043] If the time window has expired, the client is marked as being in a time-slice-limited state.

[0044] Furthermore, in step S6, when the scheduler schedules the selected request, it prefetches multiple requests and puts them into the cache.

[0045] Compared with the prior art, the present invention, employing the above technical solution, has the following beneficial effects:

[0046] (1) More precise service quality assurance: Supports user-level priority division and emergency identification to meet the service timeliness requirements of critical tasks.

[0047] (2) Fairer system resource scheduling: Supports fair scheduling among multiple users of the same type, avoiding long-term resource starvation for a certain user.

[0048] (3) More efficient response to emergencies: Through the emergency identification and timed scheduling mechanism, the response and completion time of emergency requests is significantly shortened.

[0049] (4) The scheduling system is more scalable: the innovative data structure design greatly reduces the scheduling complexity and adapts to concurrent access by thousands of tenants.

[0050] (5) Superior performance in actual deployment: Under high load, high concurrency and sudden mixed access scenarios, the IOPS variance of the system is significantly reduced and the service performance fluctuation convergence effect is better than the existing solution. Attached Figure Description

[0051] Figure 1 This application presents an overall architecture design for a cloud storage service quality management method based on bursty I / O awareness.

[0052] Figure 2 The scheduling model for burst requests provided in the embodiments of this application;

[0053] Figure 3 The high-performance scheduler architecture provided in the embodiments of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] The following section provides a detailed description of a cloud storage service quality management method based on bursty I / O awareness, combining process steps and relevant formulas. Its overall architecture is as follows: Figure 1 As shown, its core process includes key steps such as user request classification, priority allocation, scheduling label calculation, and phased scheduling execution. The specific implementation process is as follows:

[0056] Step S1: User Registration and Initialization Configuration. During the system initialization phase, the scheduler first categorizes all users into internal and external users based on their access source and request characteristics. Internal users mainly refer to tasks automatically generated by the system, including but not limited to replica repair, data rebalancing, and background maintenance operations. These users are highly correlated with the overall system health; therefore, this invention further divides them into two categories: mandatory tasks and degradeable tasks, corresponding to high-priority and low-priority scheduling strategies, respectively. A dedicated scheduling template parameter is configured for each type of task to minimize the impact on the service performance of external users.

[0057] External users primarily consist of access requests initiated by end users, which are further categorized into regular users and bursty users based on their I / O patterns and business characteristics. Regular users typically exhibit stable, continuous, and small-batch data access patterns, such as file read / write, synchronization, and backup operations. During user registration, the system assigns scheduling priorities based on account level, service type, and historical activity, classifying regular users into three categories: High Priority (HP), Medium Priority (MP), and Low Priority (LP). Each priority category corresponds to a statically configured scheduling template, containing three core parameters: Reservation (minimum resource guarantee), Limit (maximum resource limit), and Weight (resource contention weight), used to guide subsequent request queuing and bandwidth allocation.

[0058] Sudden-burst users are the primary target of this invention's scheduling optimization. They typically exhibit high concurrency, concentrated instantaneous loads, and clearly periodic I / O characteristics, commonly found in business scenarios that place significant pressure on storage system resources, such as AI training and big data analytics. To improve the service controllability and scheduling efficiency of these users, the system allows sudden-burst users to actively configure their scheduling parameters during the registration phase. These parameters mainly include: the total data budget IOQB (I / O Quota Budget) for each burst cycle, and the user-defined maximum bandwidth expectation b0. Based on these two parameters, the system derives the maximum burst scheduling time window for each user's scheduling cycle, calculated using the following formula:

[0059]

[0060] During each burst cycle, the system will prioritize allocating up to b0 of bandwidth to the user until the system can no longer guarantee the maximum expected bandwidth for each user as the system load changes.

[0061] This time window controls the activation duration of burst tasks. Within each burst cycle, the system will prioritize allocating up to b0 of bandwidth to the user. However, as system load changes, the system cannot always guarantee the maximum expected bandwidth for each user. Therefore, this invention further proposes a dynamic compensation mechanism to compensate for service disruptions when the system is currently unable to meet burst requests by extending subsequent scheduling cycles. The specific formula for calculating the compensation time is as follows:

[0062]

[0063] Where b0 is the user's maximum bandwidth expectation, b represents the actual bandwidth currently provided by the system, t is the time the time window has been running, and Δt is the amount of time the system needs to compensate in future scheduling cycles. That is, if the bandwidth currently provided by the system is less than the user's maximum bandwidth, the system will automatically extend the user's scheduling window in subsequent cycles to ensure that the user receives service quality equivalent to the original plan. This mechanism can alleviate the problem of insufficient scheduling resources while maintaining the user's perceived service continuity and performance stability.

[0064] Step S2: Calculate the scheduling label for the request. After a user sends a request, the system quickly extracts a scheduling template matching the user from the internal parameter template library based on the information provided during the registration phase (e.g., user type, whether it's a burst request, priority level, etc.). The scheduling template defines three key resource control parameters: Reservation, Limit, and Weight, which are used to assign three types of scheduling labels to the request, namely {R, L, W}. All labels use the same basic allocation principle. This paper takes the r-th request from client i of type t as an example to analyze its W label characteristics. The W label assigned to the request is the request arrival time and the previous request's W label plus 1 / w. t Maximum value:

[0065]

[0066] This formula is used to calculate the weighted scheduling label. The scheduler prioritizes processing requests with the smallest weight label without violating the constraints. The scheduling constraint is that the current system time must be greater than the limit label value of the request.

[0067] The meanings of each symbol are as follows:

[0068] t: Indicates the type of client, such as high priority, medium priority, low priority;

[0069] i: Represents the client number, used to distinguish different clients of the same type;

[0070] r: indicates that this request is the r-th request sent by client i;

[0071] This represents the weighted scheduling label for the r-th request from client i.

[0072] This represents the weighted scheduling label of the previous request of client i, i.e., the (r-1)th request;

[0073] w t This is a weighted scheduling parameter, shared by all clients of the same type; the larger the value, the larger the label step size (1 / w). t The smaller the value, the higher the request scheduling frequency for this type of client;

[0074] Current time: Represents the current system time.

[0075] Specifically, the R tag reflects the system's basic service guarantee for users, ensuring that no user is starved under any circumstances; the L tag defines the upper limit of resources that a single user can use per unit of time, used to curb resource abuse and prevent a single user from consuming excessive bandwidth during sudden traffic surges; the W tag plays a leading role during periods of resource surplus, participating in competitive resource allocation according to the weight ratio set by each user, ensuring that high-priority users can obtain more elastic resources after meeting basic guarantees. These three types of tags together constitute a three-dimensional resource management network. Working together, they not only ensure the stability of critical business operations but also improve the overall utilization rate of system resources.

[0076] Step S3: Place the request into the efficient scheduler to await scheduling. For example... Figure 3 As shown, after calculating the scheduling label, the system places the current request in the High-Efficiency Scheduler (HPSA) to await scheduling. The HPSA is implemented using a heap and round-robin structure. In the min-heap, each element represents a client category (TypeNode), whose priority is determined by the request label of the next client to be scheduled within that category. The heap structure allows the scheduler to quickly select the next client category to be scheduled. Inside each TypeNode, a round-robin list manages the clients belonging to that template. The round-robin list uses a Next_Process pointer to point to the next client to be scheduled within the current category, thus achieving fair scheduling of clients within the category.

[0077] When a new request arrives, the system first checks if the user to whom the request belongs is active, i.e., whether they are already in their category list. If the user is not yet marked as active, the system "wakes them up" and inserts them into the position pointed to by the Next_Process pointer, ensuring they can be scheduled first. The request is then inserted at the end of the user's queue, waiting for scheduling. If the user is already active, the system does not need to modify their list position; it simply appends the request to the end of their existing request queue, maintaining the original scheduling order.

[0078] Step S4: Scheduling Phase. The scheduler is the core component of the BAQoS algorithm, responsible for selecting the next request to be processed from multiple clients. The scheduling process is divided into three phases with decreasing priority: the reservation scheduling phase, the burst scheduling phase, and the weighted scheduling phase. The scheduler will only proceed to the next phase when there are no more schedulable requests in a higher-priority scheduling phase. This hierarchical design ensures that the system prioritizes scheduling burst requests after meeting the basic service level objectives of each client, thus shortening the processing time for burst request clients while maintaining fairness and efficiency.

[0079] Step S5: Reservation Scheduling Phase. In this phase, the scheduler checks the reservation tags R of all requests and defines the set of schedulable requests E as: E = {r|R ≤ t_now}, where r represents a request, R is the reservation tag of request r, t_now is the current system time, and E represents the set of all requests whose R tags are less than the current system time. Only requests that meet this condition will be scheduled. If E is a non-empty set, the scheduler selects the request with the smallest reservation tag in E: Next Request = argmin_{r∈E}R, where Next Request represents the scheduler selecting a request with the smallest reservation tag R from the set of requests E that meet the scheduling condition. This request will be processed when memory is idle. If set E is empty, meaning there are no schedulable requests in the reservation scheduling phase, the scheduler will enter the burst scheduling phase.

[0080] Step S6, the burst scheduling phase, is the core of BAQoS. Its main goal is to efficiently handle burst requests while ensuring fairness. In this phase, clients are categorized into four states using the "ready" and "limit" labels: ready without time-slice limit, ready with time-slice limit, not ready without time-slice limit, and not ready with time-slice limit. Only clients in the ready and not time-slice limit states can be scheduled during the burst scheduling phase. The ready state indicates that the client's request is ready, and scheduling this request will not cause the client to exceed its speed limit. To ensure that each scheduling does not exceed the client's speed limit, the client is re-marked as not ready after each scheduled request. The time-slice limit state indicates that the client has reached its traffic limit for the current period, and the client is not allowed to schedule requests within this period. At the beginning of each period, the system reinitializes the time window T. burst Assign it to the user and set its state to non-time-slice-limited state. This is done by assigning a time window T to each burst client. burst This prevents clients from posing as bursty clients to gain maximum system bandwidth.

[0081] The scheduler first marks requests that satisfy the following condition as ready: L ≤ t_now, where L is the restriction label of request r. Next, it defines a set of schedulable burst requests B as the set of ready and non-time-slice-limited requests: B = {r | r is ready and unlimited}. If The scheduler selects the request with the lowest weight label W from B: Next Request = arg min_{r∈B}W. For burst clients, the scheduler manages its time window T_burst and follows these rules:

[0082] If the client's timer is paused, then start the timer.

[0083] If the client has only one request remaining and it is not time-slice limited, then pause the timer.

[0084] If T_burst has been exhausted, the client is marked as time-slice-limited.

[0085] After processing this scheduling information, the scheduler schedules the selected requests. Simultaneously, each scheduling operation prefetches multiple requests and places them in the cache to improve overall system bandwidth. If If there are no schedulable requests during the burst scheduling phase, the scheduler enters the weighted scheduling phase.

[0086] Step S7, Weighted Scheduling Phase. In this phase, the scheduling process is similar to the burst scheduling phase, but only applies to ordinary clients; time slice limits and other related information do not need to be considered. The scheduler marks requests as ready if the constraint label satisfies the following condition: L ≤ t_now. Define the set of schedulable requests E' as: E' = {r | r is ready}. If The scheduler selects the request with the smallest weight label W in E': Next Request = arg min_{r∈E'}W. If The scheduler returns an empty request, indicating that the system currently has no requests to process. At this stage, the remaining system resources are fairly allocated to ready clients according to the weight parameter W.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cloud storage service quality management method based on bursty I / O awareness, characterized in that, Includes the following steps: Step S1, User Registration and System Initialization Configuration: During the system initialization phase, all users are divided into internal system users and external system users based on their access sources and request characteristics. External system users include, but are not limited to, access requests initiated by end users, and are further divided into ordinary users and bursty users based on their I / O modes and business characteristics. Step S2: Calculate the scheduling label for the request: After a user sends a request, the system extracts a scheduling template matching the user from its internal parameter template library based on the information provided during the user registration phase. The information provided during the user registration phase includes, but is not limited to, user type, whether it is a burst request, and priority level. The scheduling template defines resource control parameters used to allocate a reservation label (Reservation), a limit label (Limit), and a weight label (Weight) to the request, i.e., {R, L, W}. All labels use the same basic allocation principle. Specifically, the R scheduling label reflects the system's basic service guarantee for the user. The L scheduling label defines the maximum amount of resources that a single user can use per unit of time; The W scheduling tag plays a leading role during periods of resource abundance, participating in competitive resource allocation according to the weight ratios set by each user. Step S3: Place the request in the scheduler to wait for scheduling: After completing the scheduling label calculation, the system places the request in the scheduler; the scheduler adopts a two-layer architecture, manages client categories through a min-heap, each element corresponds to a client category, and its priority is determined by the label of the next request to be scheduled in that category, and manages clients of the same category through a polling linked list; Step S4, Scheduling Phase: The scheduling phase is divided into three phases with decreasing priority: reserved scheduling phase, burst scheduling phase, and weighted scheduling phase; the scheduler proceeds to the next phase only if there are no schedulable requests in a higher priority scheduling phase. Step S5, Reservation Scheduling Phase: The scheduler checks the reservation tags of all requests and defines the set of schedulable requests E as: E = {r|R≤t_now}, where r represents a request, R is the reservation tag of request r, t_now is the current system time, and E represents the set of all requests whose R tags are less than the current system time. Only requests that meet this condition will be scheduled. If E is a non-empty set, the scheduler selects the request with the smallest reservation tag in E: Next Request = arg min_{r∈E}R, where Next Request represents the scheduler selecting a request with the smallest reservation tag R from the set of requests E that meet the scheduling condition. This request will be processed when memory is idle. If set E is empty, meaning there are no schedulable requests in the reservation scheduling phase, the scheduler will enter the burst scheduling phase. Step S6, Burst Scheduling Phase: When there are no scheduling requests in the reserved scheduling phase, i.e., when set E in step S5 is empty, the scheduler enters the burst scheduling phase; clients are divided into four states by tags: ready without time slice limit, ready with time slice limit, not ready without time slice limit, and not ready with time slice limit. Only clients in the ready without time slice limit state can participate in scheduling in this phase. The scheduler first marks requests that meet the constraint label condition as ready, whereby the constraint label L is less than or equal to the current system time t_now, where the constraint label L is the resource usage limit identifier of request r, and the current system time t_now is the real-time system time at the time of scheduling; a schedulable burst request set B is defined as the set of all requests in the ready state and not in the time-slice-limited state; if set B is not empty, the scheduler selects the request with the smallest weight label W in set B as the next scheduled request, where the weight label W is the resource contention priority identifier of request r; if set B is empty, the scheduler enters the weighted scheduling phase; Step S7, Weighted Scheduling Phase: The scheduler marks requests that meet the following condition as ready: L≤t_now. The set of schedulable requests E' is defined as: E'={r|L≤t_now}, representing the set of all requests with L tags less than the current system time. Only requests meeting this condition will be scheduled. If set E' is non-empty, Next Request=arg min_{r∈E'}W, where Next Request represents the scheduler selecting the request with the smallest weight tag W from the set of requests E' that meet the scheduling condition. This request will be processed when memory is idle. If E' is empty, the scheduler returns an empty request, indicating that the system currently has no requests that can be processed.

2. A cloud storage service quality management method based on bursty I / O awareness, characterized in that, In step S1, the internal users of the system mainly refer to tasks automatically generated by the system, including but not limited to copy repair, data rebalancing, and background maintenance operations; the ordinary users exhibit a stable, continuous, and small-batch data access mode, including but not limited to file read / write, synchronization, and backup operations; the sudden users have high concurrency, concentrated instantaneous load, and obvious periodic I / O characteristics, and the scenarios in which they occur are business scenarios that put great pressure on storage system resources, including but not limited to AI training and big data analysis.

3. The cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S1, the internal users of the system are further divided into two categories: mandatory tasks and degradeable tasks, which correspond to high-priority and low-priority scheduling strategies, respectively, and a dedicated scheduling template parameter is configured for each type of task.

4. The cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S1, the system assigns scheduling priorities to users based on their account level, service type, and historical activity level when they register, and divides ordinary users into three categories: high priority HP, medium priority MP, and low priority LP. Each priority category corresponds to a set of statically configured scheduling templates, which include three parameters: minimum resource guarantee, maximum resource limit, and resource contention weight.

5. The cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S1, during the registration phase, the system allows users experiencing bursts to actively configure their scheduling parameters, including the total data budget I / O quota budget for each burst cycle and the user-defined maximum bandwidth expectation b0. These scheduling parameters are used to derive the maximum burst scheduling time window for each user's scheduling cycle to control the activation duration of burst tasks. The calculation formula is as follows: In the formula, IOQB stands for I / O quota budget; during each burst cycle, the system will prioritize allocating up to b0 bandwidth to the user until the system can no longer guarantee the maximum expected bandwidth for each user as the system load changes. When the system cannot meet sudden requests, i.e., the bandwidth currently provided by the system is less than the maximum expected bandwidth set by the user, a dynamic compensation mechanism is used to compensate for the service by extending the subsequent scheduling cycle. The calculation formula is as follows: Where b0 is the maximum bandwidth expectation set by the user, b represents the actual bandwidth currently provided by the system, t is the time that the current time window has been running, and Δt is the amount of time that the system needs to compensate in future scheduling cycles.

6. A cloud storage service quality management method based on bursty I / O awareness, characterized in that, In step S2, when client i of type t sends the r-th request, the W tag assigned to the request is the request arrival time and the previous request W tag plus 1 / w. t Maximum value: This formula is used to calculate the weighted scheduling label. The scheduler prioritizes processing requests with the smallest weight label without violating the constraints. The scheduling constraint is that the current system time must be greater than the limit label value of the request. The meanings of each symbol are as follows: t: Indicates the type of client, such as high priority, medium priority, low priority; i: Represents the client number, used to distinguish different clients of the same type; r: indicates that this request is the r-th request sent by client i; This represents the weighted scheduling label for the r-th request from client i. This represents the weighted scheduling label of the previous request of client i, i.e., the (r-1)th request; w t This is a weighted scheduling parameter, shared by all clients of the same type; the larger the value, the larger the label step size (1 / w). t The smaller the value, the higher the request scheduling frequency for this type of client; Current time: Represents the current system time.

7. The cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S3, the polling list has a Next_Process pointer, which points to the next client to be scheduled in the current category. When a new request arrives, if the user is not active and is not in the category list, it is inserted into the position pointed to by the Next_Process pointer and marked as active. If the user is already active, the request is appended to the end of the user's queue. After the scheduling cycle ends, the user is moved to the end of the list, and the Next_Process pointer is updated to point to the next client.

8. A cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S6, the ready state means that scheduling the request will not cause the client to exceed the speed limit, and the client will be re-marked as non-ready after each request is scheduled; the time-slice limit state means that the client has reached the traffic usage limit for the current period, and the client is not allowed to schedule requests during this period; at the beginning of each period, the system will reinitialize the time window and allocate it to the user, and at the same time set the user state to non-time-slice limit state, where the time window is the maximum duration for which the user is allowed to use resources within a single burst period.

9. A cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S6, the time window management for sudden clients follows these rules: If the client's timer is paused, then start the timer; If the client has only one request remaining and that request is not in a time-slice-limited state, then pause the timer; If the time window has expired, the client is marked as being in a time-slice-limited state.

10. A cloud storage service quality management method based on bursty I / O awareness according to claim 1, characterized in that, In step S6, when the scheduler schedules the selected request, it prefetches multiple requests and puts them into the cache.

Images