Request scheduling method and device, equipment, storage medium and program product

Abstract

The invention relates to a request scheduling method and device, equipment, a storage medium and a program product. The method comprises the following steps: in response to a plurality of requests currently transmitted by a storage system, dividing the plurality of requests into schedulable requests and to-be-scheduled requests according to the token filling number of the storage system, the concurrent scheduling upper limit of the storage system and the number of requests in a cache scheduling queue of the storage system; storing the to-be-scheduled request in a cache scheduling queue; and scheduling the schedulable request. By adopting the method, the scheduling rationality of scheduling the storage system request can be improved, so that the lowest service quality performance of the storage system is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a request scheduling method, apparatus, device, storage medium and program product. Background Art

[0002] With the rapid development of computer technology, the number of requests received by storage systems is increasing, and the requirements for request processing are also becoming higher and higher.

[0003] In the related art, when processing multiple requests, each request is usually queued and dequeued based on a request scheduling model, and each request is scheduled based on the dequeuing order of the requests.

[0004] However, in a high-concurrency environment, the scheduling of requests in related technologies is irrational and cannot guarantee the minimum performance of the service quality of the storage system. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, equipment, storage medium and program product that can request scheduling to address the above technical problems, so as to improve the scheduling rationality of scheduling storage system requests and thereby ensure the minimum service quality performance of the storage system.

[0006] In a first aspect, the present application provides a request scheduling method, the method comprising:

[0007] In response to multiple requests currently incoming to the storage system, the multiple requests are divided into schedulable requests and to-be-scheduled requests according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system, and the number of requests in the cache scheduling queue of the storage system;

[0008] The requests to be scheduled are stored in the cache scheduling queue;

[0009] Schedules schedulable requests.

[0010] In the technical solution provided by the embodiment of the present application, in response to multiple requests currently incoming to the storage system, the multiple requests are divided into schedulable requests and to-be-scheduled requests according to the number of token fills in the storage system, the upper limit of concurrent scheduling of the storage system, and the number of requests in the cache scheduling queue of the storage system, and then the to-be-scheduled requests are stored in the cache scheduling queue to schedule the schedulable requests. In this method, the multiple requests currently incoming and the requests in the cache scheduling queue are classified according to the number of token fills in the storage system, the upper limit of concurrent scheduling of the storage system, and the number of requests in the cache scheduling queue of the storage system, which is equivalent to comprehensively considering the multi-dimensional factors including the performance guarantee lower limit of the storage system, the upper limit of concurrent processing, and the key requests in the cache scheduling queue, determining the scheduling priority of each request, and flexibly and accurately allocating more objective and accurate scheduling priorities to the multiple requests currently incoming and the requests in the cache scheduling queue, ensuring that the schedulable requests can be smoothly scheduled and guaranteeing the minimum performance of the service quality. Next, the determined schedulable requests are scheduled in a timely manner to improve scheduling efficiency while meeting the minimum service quality performance of the storage system; the determined requests to be scheduled are stored in the cache scheduling queue so that in the next round of scheduling, the minimum service quality performance of the next scheduling cycle can be met, thereby continuously rotating to ensure the minimum service quality performance of the storage system.

[0011] In one embodiment, according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system, and the number of requests in the cache scheduling queue of the storage system, multiple requests are divided into schedulable requests and to-be-scheduled requests, including:

[0012] Distribute tokens to target requests among multiple requests according to a reserved token bucket corresponding to a service quality policy of the storage system; the target request represents a request that matches the token fill quantity;

[0013] Determine the schedulable requests and requests to be scheduled based on the concurrent scheduling upper limit, requests in the cache scheduling queue, and target requests.

[0014] In the technical solution provided in the embodiment of the present application, according to the reserved token bucket corresponding to the service quality policy of the storage system, the target request is screened out from multiple requests, and tokens are distributed to the target request among the multiple requests. According to the concurrent scheduling upper limit, the schedulable requests that match the concurrent scheduling upper limit are determined from the cache scheduling queue and the target request to ensure that the schedulable requests can be scheduled and processed in time, thereby improving the effectiveness of request scheduling.

[0015] In one embodiment, the cache scheduling queue includes a lower limit guarantee queue, and determines the schedulable requests and the requests to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request, including:

[0016] If the concurrent scheduling upper limit is greater than the number of requests in the lower limit guarantee queue, obtaining a first request difference between the concurrent scheduling upper limit and the number of requests in the lower limit guarantee queue;

[0017] When the number of target requests is greater than or equal to the first request difference, the requests in the lower limit guarantee queue and the first request in the target request that matches the first request difference are determined as schedulable requests; and the second request in the target request except the first request and other requests among the multiple requests except the target request are determined as requests to be scheduled.

[0018] In the technical solution provided by the embodiment of the present application, when the concurrent scheduling upper limit is greater than the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue are determined as schedulable requests, and based on the first request difference between the concurrent scheduling upper limit and the number of requests in the lower limit guarantee queue, the first request matching the first request difference is again determined from the target requests as the schedulable request, so that the number of schedulable requests is consistent with the concurrent scheduling upper limit, so as to maximize the utilization of the system's concurrent resources and give full play to the concurrent processing capabilities.

[0019] In one embodiment, the cache scheduling queue also includes a common queue; determining the schedulable requests and the requests to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request, including:

[0020] When the number of target requests is less than the first request difference, obtaining a second request difference between the number of target requests and the first request difference;

[0021] Distribute tokens for requests in the common queue that match the second request difference according to the remaining tokens in the reserved token bucket;

[0022] The requests in the lower limit guarantee queue, the target request, and the requests in the common queue that match the difference with the second request are all determined as schedulable requests; and the remaining requests in the common queue are determined as requests to be scheduled.

[0023] In the technical solution provided in the embodiment of the present application, when the number of target requests is less than the first request difference, the requests in the lower limit guarantee queue and the target requests are determined as schedulable requests, and based on the second request difference between the number of target requests and the first request difference, the requests matching the second request difference are again determined from the ordinary queue as schedulable requests, so that the number of schedulable requests is as close as possible to the concurrent scheduling upper limit, so as to maximize the utilization of the system's concurrent resources and give full play to the concurrent processing capabilities.

[0024] In one embodiment, the cache scheduling queue includes a lower limit guarantee queue; determining the schedulable requests and the requests to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request, including:

[0025] If the concurrent scheduling upper limit is less than or equal to the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue that match the concurrent scheduling upper limit are determined as schedulable requests; and,

[0026] The target request and other requests except the target request among the multiple requests are all determined as requests to be scheduled.

[0027] In the technical solution provided by the embodiment of the present application, when the upper limit of concurrent scheduling is less than or equal to the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue that match the upper limit of concurrent scheduling are determined as schedulable requests. Since the requests in the lower limit guarantee queue are requests that have been allocated tokens but not successfully scheduled in the previous round, there is no need to perform repeated queue retrieval and verification operations during this scheduling, which speeds up the dequeue efficiency of the requests in the lower limit guarantee queue and effectively reduces the scheduling delay overhead.

[0028] In one embodiment, the cache scheduling queue includes a lower limit guarantee queue and a candidate queue; storing the to-be-scheduled request in the cache scheduling queue includes:

[0029] For any pending scheduling request, if the pending scheduling request has been assigned a token, the pending scheduling request is stored in the lower limit guarantee queue;

[0030] If the request to be scheduled is not assigned a token, the request to be scheduled is stored in the candidate queue.

[0031] In the technical solution provided in the embodiment of the present application, the requests to be scheduled are subdivided into requests that can be scheduled directly and requests that need to be allocated tokens before scheduling, according to whether tokens are allocated to the requests to be scheduled. This allows the requests to be scheduled in different queues to be processed differently in the next round of request scheduling, thereby improving the request scheduling efficiency.

[0032] In one embodiment, the candidate queue includes a common queue and a waiting queue, and storing the to-be-scheduled request in the candidate queue includes:

[0033] Get the global token bucket of the storage system;

[0034] If the number of tokens in the global token bucket meets the preset conditions, the scheduling request is stored in the general queue;

[0035] If the number of tokens in the global token bucket does not meet the preset condition, the request to be scheduled is stored in the waiting queue.

[0036] In the technical solution provided in the embodiment of the present application, the queue to which the unassigned token request belongs is determined according to the number of tokens in the global token bucket. When there are sufficient tokens in the global token bucket, the request to be scheduled is placed in the ordinary queue to be scheduled out of the queue later; when there are insufficient tokens in the global token bucket, the request to be scheduled is placed in the waiting queue to restrict its scheduling, thereby providing a more reliable and reasonable scheduling basis for the next round of scheduling and ensuring that the scheduling process can be carried out smoothly and reasonably.

[0037] In one embodiment, scheduling the schedulable request includes:

[0038] Determine scheduling resources for schedulable requests in the storage system according to a global token bucket of the storage system;

[0039] Scheduling requests based on scheduling resources.

[0040] In the technical solution provided in the embodiment of the present application, based on the global token bucket of the storage system, the scheduling resources for processing non-lower limit guarantee requests and the scheduling resources for processing lower limit guarantee requests in the storage system are accurately determined from the resources of the entire storage system, so as to realize the scheduling processing of schedulable requests for guaranteeing lower limit guarantee requests.

[0041] In one embodiment, the method further comprises:

[0042] Monitor the performance loss value of the storage system;

[0043] If the performance loss value is less than the loss threshold, the number of tokens in the global token bucket is reduced according to the preset token reduction sequence;

[0044] If the performance loss value is greater than the loss upper limit, the number of tokens in the global token bucket is increased according to the preset token increment sequence.

[0045] In the technical solution provided in the embodiment of the present application, the token filling strategy of the global token bucket is determined based on the comparison results of the performance loss value with the loss critical value and the loss upper limit value. When the performance loss value is less than the loss critical value, the number of tokens in the global token bucket is increased to achieve the lower limit protection target to the greatest extent. When the performance loss value is greater than the loss critical value, the number of tokens in the global token bucket is reduced to avoid the overall performance out of control caused by the performance of the lower limit protection strategy exceeding the lower limit value, thereby improving the stability and reliability of the system.

[0046] In a second aspect, the present application further provides a request scheduling device, including:

[0047] A request response module, used to respond to multiple requests currently incoming to the storage system, and divide the multiple requests into schedulable requests and to-be-scheduled requests according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system, and the number of requests in the cache scheduling queue of the storage system;

[0048] The request enqueue module is used to store the requests to be scheduled into the cache scheduling queue;

[0049] The request scheduling module is used to schedule schedulable requests.

[0050] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any one of the embodiments of the first aspect when executing the computer program.

[0051] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in any one of the embodiments of the first aspect above are implemented.

[0052] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 An application environment diagram of a request scheduling method in one embodiment;

[0055] Figure 2 A flowchart of a request scheduling method in one embodiment;

[0056] Figure 3 A flowchart of a request classification step in one embodiment;

[0057] Figure 4 A flowchart of a request classification step in another embodiment;

[0058] Figure 5 A flowchart of a request classification step in another embodiment;

[0059] Figure 6A schematic diagram of a flow chart of a scheduling request storage step in one embodiment;

[0060] Figure 7 A schematic flow chart of a scheduling request storage step in another embodiment;

[0061] Figure 8 A flowchart of a request scheduling method in another embodiment;

[0062] Fig. 9 A schematic diagram of a flow chart of an L token adjustment method in an embodiment;

[0063] Fig.10 A schematic diagram of the scheduling principle of a lower limit guarantee scheduling model in an embodiment;

[0064] Fig.11 A flowchart of a request scheduling step in one embodiment;

[0065] Fig.12 is a schematic diagram of an L token feedback mechanism in one embodiment;

[0066] Fig.13 A structural block diagram of a request scheduling device in one embodiment;

[0067] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] The request scheduling method provided in the embodiment of the present application can be applied to Figure 1 In the computer system shown, the computer system includes a storage system 102 and a server 104. The storage system 102 is a system composed of various storage devices, control components, and management information scheduling devices (hardware) and algorithms (software) used to store programs and data in a computer, and provides data for the server 104. The server 104 processes and provides various applications and services of the computer system, and provides access and management functions for the storage system.

[0070] In the field of storage systems, storage systems support more and more concurrent access requests (I / O requests), requiring servers to use limited system resources to schedule multiple I / O requests to ensure that I / O requests can be processed in a timely manner. Then, in the scenario where I / O is issued to multiple logical volumes in the storage system at the same time, it will cause competition for system resources among various I / O requests, which may cause key I / O requests to be unable to be scheduled and processed in a timely manner, thereby affecting the stability and reliability of the entire computer system. Therefore, how to ensure that key I / O requests can be scheduled in a timely manner and that the overall performance loss of the computer system is controlled is crucial to the high availability and reliability of the computer system.

[0071] Taking the guarantee of minimum service quality performance as an example, in the related technology, when managing and scheduling requests, it is usually based on the Quality of Service (QoS) scheduling model with queue priority dequeueing, by setting up multiple levels of queues (each application in the computer system has a corresponding lower limit guarantee queue), and the capacity of each lower limit guarantee queue is greater than the number of requests with the minimum guarantee lower limit. In this way, whenever an I / O request arrives at the system, it will be inserted into the corresponding lower limit guarantee queue, and will be dequeued from the lower limit guarantee queue first during processing, ensuring the lower limit of the number of read and write operations per second (IOPS). By default, priority dequeueing means completing an I / O target.

[0072] However, in actual applications, the system may contain multiple storage levels, such as memory, solid-state disks (SSDs), and hard disk drives (HDDs). Insufficient coordination between different storage levels will lead to bottlenecks when data is transmitted between different levels. In this case, even if the I / O requests in the lower limit guarantee queue are dequeued first, these requests may be blocked on the storage array side. For example, the cache cannot process a large number of concurrent I / O requests in time, resulting in I / O requests being blocked by the cache; the storage array has limited processing power, and too many I / O requests will cause the storage array to process slowly, etc., resulting in the computer system's overall I / O lower limit target per unit time still not being achieved.

[0073] In addition, the I / O scheduling model that relies too much on queue priority and dequeue order requires enqueue and dequeue operations for each I / O request. In order to maintain multiple priority queues, the system needs to continuously perform queue management operations, such as search, insertion, and deletion. These operations also increase the burden on the system and increase the scheduling overhead. Especially in a high-concurrency environment, the frequency of these operations is very high, resulting in increased scheduling latency. In addition, as mentioned above, due to the limited system resources (such as cache, storage array, etc.), I / O may still be blocked in resource competition, resulting in the inability of the existing scheduling model to meet the latency guarantee requirement scenario.

[0074] In summary, the scheduling methods in the existing technologies lack dynamic adjustment capabilities, require long scheduling delays, cannot achieve fine bandwidth control, and cause resource contention and unfairness, resulting in unreasonable scheduling methods and thus being unable to guarantee the system's minimum service quality performance.

[0075] Based on this, the embodiments of the present application provide a request scheduling method, apparatus, device, storage medium and program product, which respond to multiple requests currently incoming to the storage system, reasonably and objectively divide the multiple requests into schedulable requests and requests to be scheduled according to the token fill number of the storage system, the concurrent scheduling upper limit of the storage system and the number of requests in the cache scheduling queue of the storage system, store the requests to be scheduled in the cache scheduling queue, schedule the schedulable requests, improve the rationality of the scheduling method, and ensure the minimum performance of the system's service quality.

[0076] The following uses a server as the execution subject to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems through specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0077] In an exemplary embodiment, Figure 2 As shown, a request scheduling method is provided, comprising the following steps:

[0078] S201, in response to multiple requests currently incoming to the storage system, divide the multiple requests into schedulable requests and to-be-scheduled requests according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system and the number of requests in the cache scheduling queue of the storage system.

[0079] The storage system refers to a system composed of various storage devices, control components, and management information scheduling devices and algorithms (software) used to store programs and data in a computer. The storage system in this embodiment can be a centralized storage system or a distributed storage system. In practical applications, the storage system usually continuously and concurrently receives multiple requests from the same client or different clients.

[0080] It should be noted that for each request passed in by the storage system, the scheduling process is based on the premise that the request is allocated with a token. Based on this, before scheduling the request without a token, it is necessary to allocate a token to the request according to the token filling quantity of the storage system. The token filling quantity in this embodiment can be a quantitative indicator of the lower limit of performance guarantee, which is manually set by the user for a certain logical volume or multiple logical volumes based on actual needs.

[0081] The concurrent scheduling upper limit refers to the upper limit of the requests that the storage system can process in parallel, which can be expressed as the maximum processing request upper limit or the maximum processing request amount. Generally, in order to fully utilize the concurrent processing capability of the storage system, the closer the concurrent scheduling upper limit is to the number of schedulable requests, the better.

[0082] The cache scheduling queue stores requests to be scheduled, including requests that cannot be successfully scheduled due to the limitation of the number of tokens and / or requests that cannot be successfully scheduled due to the limitation of the concurrent scheduling upper limit.

[0083] In an embodiment of the present application, in the face of multiple requests, based on the token fill number of the storage system, the concurrent scheduling upper limit and the number of requests in the cache scheduling queue, the schedulable requests that need to be scheduled this time are determined from the multiple requests currently incoming and the requests in the cache scheduling queue, and the remaining requests except the schedulable requests in the multiple requests currently incoming and the requests in the cache scheduling queue are determined as requests to be scheduled.

[0084] Among them, schedulable requests refer to requests that are allocated tokens, and the number of schedulable requests is less than or equal to the concurrent scheduling limit; pending scheduling requests refer to requests that cannot be successfully scheduled this time, which can be requests that are not allocated tokens, or requests that are allocated tokens but exceed the concurrent scheduling limit.

[0085] Optionally, the cache scheduling queue includes a lower limit guarantee queue and a general queue, wherein the requests in the lower limit guarantee queue are requests for allocated tokens, and the requests in the general queue are requests for unallocated tokens. Then, when the number of tokens filled is sufficient, requests matching the upper limit of concurrent scheduling can be determined from the requests in the lower limit guarantee queue, the multiple incoming requests, and the requests in the general queue in accordance with the preset scheduling priority order as schedulable requests, and the requests in the lower limit guarantee queue, the multiple incoming requests, and the requests in the general queue except for the schedulable requests are regarded as requests to be scheduled.

[0086] S202: Store the request to be scheduled in a cache scheduling queue.

[0087] When the schedulable requests and the requests to be scheduled are determined, the requests to be scheduled can be stored in the cache scheduling queue according to the time when the requests to be scheduled are input. The cache scheduling queue is a first-in-first-out queue, that is, the earlier the time when the requests to be scheduled are input, the earlier they are out of the queue and the higher the scheduling priority.

[0088] In another implementation scenario, each request to be scheduled may be classified according to the type of the request to be scheduled and divided into cache scheduling queues of different priorities.

[0089] Exemplarily, for any request to be scheduled, if the request to be scheduled is a request for allocating a token, the scheduling request is classified into a cache scheduling queue of the first priority; if the request to be scheduled is a request for not allocating a token, the scheduling request is classified into a cache scheduling queue of the second priority.

[0090] S203, scheduling the schedulable request.

[0091] The schedulable requests may include requests that have been allocated tokens and / or requests that have not been allocated tokens.

[0092] For schedulable requests that have been allocated tokens, they can be directly scheduled; for requests that have not been allocated tokens, tokens need to be allocated to the requests based on the token filling quantity of the storage system before scheduling.

[0093] In addition, it should be noted that after determining the schedulable requests and the requests to be scheduled, the schedulable requests can be scheduled first and then the requests to be scheduled can be stored, or the requests to be scheduled can be stored first and then the schedulable requests can be scheduled, or the requests to be scheduled can be stored and then the schedulable requests can be scheduled at the same time, and this embodiment does not impose any restrictions on this.

[0094] In an embodiment of the present application, in response to multiple requests currently incoming to the storage system, the multiple requests are divided into schedulable requests and to-be-scheduled requests according to the number of tokens filled in the storage system, the upper limit of concurrent scheduling of the storage system, and the number of requests in the cache scheduling queue of the storage system, and then the to-be-scheduled requests are stored in the cache scheduling queue to schedule the schedulable requests. In this method, the multiple requests currently incoming and the requests in the cache scheduling queue are classified according to the number of tokens filled in the storage system, the upper limit of concurrent scheduling of the storage system, and the number of requests in the cache scheduling queue of the storage system, which is equivalent to comprehensively considering the multi-dimensional factors including the performance guarantee lower limit of the storage system, the upper limit of concurrent processing, and the key requests in the cache scheduling queue, determining the scheduling priority of each request, and flexibly and accurately allocating more objective and accurate scheduling priorities to the multiple requests currently incoming and the requests in the cache scheduling queue, ensuring that the schedulable requests can be smoothly scheduled and guaranteeing the minimum performance of the service quality. Next, the determined schedulable requests are scheduled in a timely manner to improve scheduling efficiency while meeting the minimum service quality performance of the storage system; the determined requests to be scheduled are stored in the cache scheduling queue so that in the next round of scheduling, the minimum service quality performance of the next scheduling cycle can be met, thereby continuously rotating to ensure the minimum service quality performance of the storage system.

[0095] In the above-mentioned embodiment, the basis for request division is described, namely, the number of tokens filled, the upper limit of concurrent scheduling, and the number of requests in the cache scheduling queue of the storage system. On this basis, there can be multiple division methods, for example, when the upper limit of concurrent scheduling is met, the requests in the cache scheduling queue are preferentially determined as schedulable requests, or tokens are preferentially allocated to the currently incoming requests, and the requests to which tokens are allocated are determined as schedulable requests, etc. Based on this, an achievable method of request division is described below through an embodiment.

[0096] In an exemplary embodiment, Figure 3 As shown, according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system and the number of requests in the cache scheduling queue of the storage system, multiple requests are divided into schedulable requests and to-be-scheduled requests, including the following steps:

[0097] S301, distributing tokens to a target request among multiple requests according to a reserved token bucket corresponding to a service quality policy of a storage system; the target request represents a request that matches a token filling quantity.

[0098] Among them, the service quality policy refers to the minimum guarantee policy set manually, which can be the minimum guarantee policy for a certain logical volume of the computer system, such as the performance lower limit target of the D drive is 500M / S, or 500 IO requests / S. It can also be the minimum guarantee policy for multiple logical volumes, such as the performance lower limit target of the C drive and the D drive as a whole is 5000M / S, or 5000 IO requests / S.

[0099] In actual applications, the token filling quantity and filling window are determined based on the service quality policy, and then within the filling window, the reserved token bucket is filled with tokens (Reservation tokens) that match the token filling quantity. For example, if the service quality policy is that the performance lower limit target of the C drive is 1000 requests / S, then 1000 tokens are filled into the reserved token bucket per second.

[0100] The tokens in the reserved token bucket are used to ensure the lower limit of the request scheduling of the logical volume corresponding to the quality of service policy.

[0101] For the multiple requests currently transmitted to the storage system, determine whether the logical volume corresponding to each request belongs to the logical volume corresponding to the service quality policy, and then determine the target request that matches the token filling quantity from the multiple requests in the logical volume corresponding to the service quality policy, and allocate tokens to the target request according to the tokens in the reserved token bucket.

[0102] Continuing with the service quality policy with the performance lower limit target of 1000 requests / S for drive C (that is, the number of token fills is 1000, and there are 1000 R tokens in the reserved token bucket) as an example, if 2000 of the current 5000 incoming requests are for drive D and 3000 are for drive C, then 1000 target requests are determined from the 3000 requests for drive C, and the 1000 tokens in the reserved token bucket are distributed to the 1000 target requests for drive C.

[0103] In this embodiment, other requests except the target request among the multiple requests are determined as requests to be scheduled.

[0104] S302, determining schedulable requests and requests to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request.

[0105] When the number of schedulable requests is less than or equal to the concurrent scheduling upper limit, the schedulable requests are determined from the cache scheduling queue and the target request in turn according to the priority of the request in the cache scheduling queue and the scheduling priority of the target request.

[0106] Exemplarily, taking the case where the priority of the target request is greater than the priority of the requests in the cache scheduling queue, the number of target requests is compared with the concurrent scheduling upper limit. If the number of target requests is greater than or equal to the concurrent scheduling upper limit, the requests in the target requests that match the concurrent scheduling upper limit are determined as schedulable requests, and other requests except the target request among the multiple requests, the remaining requests in the target requests except the schedulable requests, and the requests in the cache scheduling queue are determined as requests to be scheduled.

[0107] In an embodiment of the present application, according to the reserved token bucket corresponding to the service quality policy of the storage system, the target request is screened out from multiple requests, and tokens are distributed to the target request among the multiple requests. According to the concurrent scheduling upper limit, the schedulable requests that match the concurrent scheduling upper limit are determined from the cache scheduling queue and the target request to ensure that the schedulable requests can be scheduled and processed in a timely manner, thereby improving the effectiveness of request scheduling.

[0108] In actual application scenarios, limited by the previous round of front-end concurrent scheduling resources, the cache scheduling queue may include requests that have been allocated tokens but have not been successfully scheduled due to insufficient front-end concurrent scheduling resources in the previous round. In the embodiment of the present application, such requests (requests that have been allocated tokens but have not been successfully scheduled due to insufficient front-end concurrent scheduling resources in the previous round) are set as the highest priority requests to be scheduled and stored in the lower limit guarantee queue. When the request scheduling is executed next time, there is no need to do too many queue retrieval and verification operations, and the requests to be scheduled in the lower limit guarantee queue can be quickly dequeued directly, reducing the scheduling delay overhead.

[0109] In an exemplary embodiment, Figure 4 As shown, the cache scheduling queue includes a lower limit guarantee queue, which determines the schedulable requests and the requests to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request, including:

[0110] S401: If the upper limit of concurrent scheduling is greater than the number of requests in the lower limit guarantee queue, a first request difference between the upper limit of concurrent scheduling and the number of requests in the lower limit guarantee queue is obtained.

[0111] The upper limit of concurrent scheduling is greater than the number of requests in the lower limit guarantee queue, indicating that all requests in the lower limit guarantee queue can be scheduled. At this time, in order to give full play to the concurrent processing capability of the storage system, the first request difference between the upper limit of concurrent scheduling and the number of requests in the lower limit guarantee queue can also be calculated, and the request matching the first request difference is again determined from the target request as the schedulable request.

[0112] S402, when the number of target requests is greater than or equal to the first request difference, the requests in the lower limit guarantee queue and the first request in the target request that matches the first request difference are determined as schedulable requests; and the second request in the target request except the first request and other requests in the multiple requests except the target request are determined as requests to be scheduled.

[0113] While determining the requests in the lower limit guarantee queue as schedulable requests, the difference between the number of target requests and the first request is determined. If the number of target requests is greater than or equal to the first request difference, the first request that matches the first request difference is determined from the target requests and is also used as a schedulable request, so that the number of schedulable requests matches the upper limit of concurrent scheduling. At the same time, for the requests that failed to be successfully scheduled this time, that is, the second request in the target request except the first request, and the other requests in the multiple requests except the target request are all determined as pending scheduling requests for the next scheduling.

[0114] Taking 3000 incoming requests as an example, if the concurrent scheduling upper limit is 2000, the number of requests in the lower limit guarantee queue is 1500, the number of target requests is 600, and the number of non-target requests (other requests) is 2400, then the 1500 requests in the lower limit guarantee queue and the 500 target requests can be determined as schedulable requests, and the 100 second requests in the target requests except the schedulable requests and the 2400 non-target requests can be determined as requests to be scheduled.

[0115] In an embodiment of the present application, when the concurrent scheduling upper limit is greater than the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue are determined as schedulable requests, and based on the first request difference between the concurrent scheduling upper limit and the number of requests in the lower limit guarantee queue, the first request matching the first request difference is again determined from the target requests as the schedulable request, so that the number of schedulable requests is consistent with the concurrent scheduling upper limit, so as to maximize the utilization of the system's concurrent resources and give full play to the concurrent processing capabilities.

[0116] In actual application scenarios, limited by the number of token fills, the cache scheduling queue may also include requests that were not allocated tokens in the previous round of scheduling. In an embodiment of the present application, requests that were not allocated tokens in the previous round of scheduling are stored in a common queue, and the scheduling priority in the common queue is less than the scheduling priority of the currently incoming request, and the scheduling priority of the currently incoming request is less than the priority of the request in the lower limit guarantee queue. In this way, when request scheduling is performed next time, if the number of token fills and the upper limit of concurrent scheduling are sufficient, after scheduling the lower limit guarantee queue and the currently incoming request, the request in the common queue can be further determined as a schedulable request and scheduled.

[0117] In an exemplary embodiment, Figure 5 As shown, the cache scheduling queue also includes a common queue; according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request, the schedulable requests and the requests to be scheduled are determined, including:

[0118] S501: When the number of target requests is less than the first request difference, obtain a second request difference between the number of target requests and the first request difference.

[0119] If the number of target requests is less than the first request difference, it means that all requests in the lower limit guarantee queue and the target requests can be scheduled. At this time, in order to give full play to the concurrent processing capability of the storage system, the second request difference between the number of target requests and the first request difference can also be calculated, and the requests matching the second request difference can be determined from the common queue as schedulable requests.

[0120] S502: Distribute tokens to requests in the common queue that match the second request difference according to the remaining tokens in the reserved token bucket.

[0121] The remaining tokens in the reserved token bucket refer to the remaining tokens in the reserved token bucket after allocating tokens to multiple currently incoming requests when the currently incoming requests are less than the token fill quantity.

[0122] Since the requests in the common queue are requests to which tokens are not allocated, before scheduling the requests in the common queue, tokens may be distributed to the requests in the common queue that match the second request difference according to the remaining tokens in the reserved token bucket.

[0123] S503, determine the requests in the lower limit guarantee queue, the target request, and the requests in the common queue that match the second request difference as schedulable requests; and determine the remaining requests in the common queue as requests to be scheduled.

[0124] The requests in the lower limit guarantee queue, the target request, and the requests in the common queue that match the difference with the second request are all determined as schedulable requests, so that the number of schedulable requests is as close as possible to the concurrent scheduling upper limit.

[0125] For the remaining requests in the general queue except the schedulable requests, if there are tokens in the reserved token bucket, tokens are allocated to the requests, and the requests with allocated tokens are stored in the lower limit guarantee queue, and the requests without allocated tokens are stored in the general queue.

[0126] In an embodiment of the present application, when the number of target requests is less than the first request difference, the requests in the lower limit guarantee queue and the target requests are both determined as schedulable requests, and based on the second request difference between the number of target requests and the first request difference, requests matching the second request difference are again determined from the ordinary queue as schedulable requests, so that the number of schedulable requests is as close as possible to the concurrent scheduling upper limit, so as to maximize the utilization of the system's concurrent resources and give full play to the concurrent processing capabilities.

[0127] The foregoing Figure 4-Figure 5 The embodiment of the embodiment is an embodiment of determining the scheduling request and the request to be scheduled in the scenario where the upper limit of concurrent scheduling is greater than the number of requests in the lower limit guarantee queue. Next, in the scenario where the upper limit of concurrent scheduling is less than or equal to the number of requests in the lower limit guarantee queue, the method of determining the schedulable request and the request to be scheduled is described.

[0128] In an exemplary embodiment, the cache scheduling queue includes a lower limit guarantee queue; according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request, determining the schedulable requests and the requests to be scheduled includes:

[0129] If the concurrent scheduling upper limit is less than or equal to the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue that match the concurrent scheduling upper limit are determined as schedulable requests; and the target request and other requests among the multiple requests except the target request are determined as requests to be scheduled.

[0130] The upper limit of concurrent scheduling is less than or equal to the number of requests in the lower limit guarantee queue, and the requests in the lower limit guarantee queue have the highest priority. In this case, the requests that match the upper limit of concurrent scheduling are preferentially determined from the lower limit guarantee queue as schedulable requests, and the scheduling is completed this time. In addition, the multiple requests currently incoming, that is, the target request for allocating tokens and other requests that are not allocated tokens, are determined as requests to be scheduled. Furthermore, the target request for allocating tokens is stored in the lower limit guarantee queue, and the other requests that are not allocated tokens are stored in the ordinary queue in the cache scheduling queue for the next round of scheduling.

[0131] In an embodiment of the present application, when the upper limit of concurrent scheduling is less than or equal to the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue that match the upper limit of concurrent scheduling are determined as schedulable requests. Since the requests in the lower limit guarantee queue are requests that have been allocated tokens but not successfully scheduled in the previous round, there is no need to perform repeated queue retrieval and verification operations during this scheduling, which speeds up the efficiency of dequeuing requests in the lower limit guarantee queue and effectively reduces the scheduling delay overhead.

[0132] The above embodiment has described in detail the division method of the schedulable requests and the requests to be scheduled. Next, the storage queue of the divided requests to be scheduled will be described.

[0133] In an exemplary embodiment, Figure 6 As shown, the cache scheduling queue includes a lower limit guarantee queue and a candidate queue; storing the request to be scheduled in the cache scheduling queue includes:

[0134] S601: For any request to be scheduled, if a token has been allocated to the request to be scheduled, the request to be scheduled is stored in a lower limit guarantee queue.

[0135] Among them, the request to be scheduled can be a request that has been allocated a token but has not been successfully scheduled due to insufficient front-end concurrent scheduling resources in the previous round, or a request that has not been allocated a token because the token filling quantity is less than the request quantity.

[0136] For any request to be scheduled, determine whether a token is assigned to the scheduling request. If a token has been assigned, it means that the scheduling request is a request that was not successfully scheduled due to insufficient front-end concurrent scheduling resources in the previous round, and the scheduling priority is higher than the newly incoming request in the next round. The request to be scheduled is stored in the lower limit guarantee queue so that the request to be scheduled in the lower limit guarantee queue can be directly scheduled in the next round of scheduling.

[0137] S602: If the request to be scheduled is not allocated a token, the request to be scheduled is stored in a candidate queue.

[0138] If no token is allocated, it means that the scheduling request is a request that has not been allocated a token in the previous round, and the scheduling priority is lower than the newly incoming request in the next round, then the request to be scheduled is stored in the candidate queue.

[0139] For the requests to be scheduled in the candidate queue, tokens need to be distributed before the request scheduling is performed. In addition, the scheduling of the requests in the candidate queue is performed after the requests in the lower limit guarantee queue and the multiple requests incoming in the next round have been scheduled.

[0140] In an embodiment of the present application, based on whether a token is allocated to the request to be scheduled, the request to be scheduled is subdivided into requests that can be scheduled directly and requests that require allocation of tokens before scheduling, so that in the next round of request scheduling processing, different processing can be performed on the requests to be scheduled in different queues, thereby improving the request scheduling efficiency.

[0141] Next, the storage method of the pending scheduling requests to which no token is allocated in the above embodiment is further described.

[0142] In an exemplary embodiment, Figure 7 As shown, the candidate queue includes a common queue and a waiting queue, and the requests to be scheduled are stored in the candidate queue, including:

[0143] S701, obtain a global token bucket of the storage system.

[0144] The tokens in the global token bucket are limit tokens (Limit tokens, L tokens), which are used to support the scheduling of non-lower limit guaranteed I / O requests. The tokens in the global token bucket can be filled according to a preset filling rate and filling quantity.

[0145] S702: If the number of tokens in the global token bucket meets a preset condition, the request to be scheduled is stored in a common queue.

[0146] S703: If the number of tokens in the global token bucket does not meet a preset condition, the request to be scheduled is stored in a waiting queue.

[0147] The preset condition is a condition for subdividing the unallocated tokens into a common queue or a waiting queue, and may be a condition for limiting the number of tokens in a global token bucket.

[0148] For any round of scheduling, the scheduling priority of requests in the common queue is lower than the priority of multiple requests passed in during the current scheduling process, and is higher than the scheduling priority of requests in the waiting queue.

[0149] Taking the preset condition that the number of global token buckets is greater than the upper limit threshold as an example, if the number of tokens is greater than the upper limit threshold, it is determined that the number of tokens in the global token bucket is sufficient, and the request to be scheduled is stored in the normal queue; if the number of tokens is less than the upper limit threshold, it is determined that the number of tokens in the global token bucket is sufficient, and the request to be scheduled is stored in the waiting queue.

[0150] In an embodiment of the present application, the queue to which the unassigned token request belongs is determined based on the number of tokens in the global token bucket. When there are sufficient tokens in the global token bucket, the request to be scheduled is placed in the ordinary queue to be dequeued for subsequent scheduling; when there are insufficient tokens in the global token bucket, the request to be scheduled is placed in the waiting queue to restrict its scheduling, thereby providing a more reliable and reasonable scheduling basis for the next round of scheduling and ensuring that the scheduling process is carried out smoothly and reasonably.

[0151] In actual applications, the process of request scheduling based on L tokens and the process of request scheduling based on reserved token buckets are carried out simultaneously, which will cause competition for the entire system resources. Therefore, the number of L tokens can be adjusted to adjust the request scheduling efficiency of the lower limit guarantee target.

[0152] In an exemplary embodiment, Figure 8 As shown, scheduling the schedulable request includes:

[0153] S801, determining scheduling resources for schedulable requests in the storage system according to a global token bucket of the storage system.

[0154] According to the global token bucket of the storage system, the scheduling resources for scheduling the requests with non-minimum performance of the service quality in the storage system are determined, and the difference between the overall scheduling resources of the system and the scheduling resources is used as the scheduling resources for the schedulable requests.

[0155] In an embodiment of the present application, the global token bucket of the storage system is negatively correlated with the scheduling resources for schedulable requests. If the number of tokens in the global token bucket is more, the scheduling resources for schedulable requests are fewer. If the number of tokens in the global token bucket is fewer, the scheduling resources for schedulable requests are more.

[0156] S802: Schedule the schedulable request based on the scheduling resources.

[0157] The scheduling resources refer to the system resources in the entire system that schedule the schedulable requests corresponding to the service quality policy, including but not limited to coordinating the system resources and cache resources between different storage levels. In the embodiment of the present application, based on the scheduling resources, the schedulable requests are smoothly transmitted to the application queue to complete the full cycle processing of the schedulable requests.

[0158] In an embodiment of the present application, based on the global token bucket of the storage system, the scheduling resources for processing non-lower limit guarantee requests and the scheduling resources for processing lower limit guarantee requests in the storage system are accurately determined from the entire storage system resources to implement the scheduling processing of schedulable requests that guarantee lower limit guarantee requests.

[0159] It should be noted that the difference between the global token bucket and the reserved token bucket is that the R tokens in the reserved token bucket are used to support the scheduling of requests for the lower limit guarantee target, and the L tokens in the global token bucket are used to support the scheduling of requests for non-lower limit guarantee targets. However, scheduling based on the global token bucket and scheduling based on the reserved token bucket both need to share other scheduling resources of the system. Based on this, the process of scheduling based on the reserved token bucket can be reversely regulated by controlling the number of tokens in the global token bucket.

[0160] In an exemplary embodiment, Fig. 9 As shown, the method also includes:

[0161] S901, monitoring the performance loss value of the storage system.

[0162] The performance loss value of the storage system can reflect the rationality of resource scheduling in the process of ensuring the lower limit service request of the storage system, and further reflect the reliability of the entire system.

[0163] S902: If the performance loss value is less than the loss threshold, the number of tokens in the global token bucket is reduced according to a preset token decreasing sequence.

[0164] If the performance loss value is less than the loss critical value, it means that the overall performance loss is controllable at this time. In this case, we can further judge whether the service quality policy is achieved. If the service quality policy is not achieved, reduce the number of tokens in the global token bucket according to the preset token decrement sequence, and allocate more system resources for the schedulable requests for the service quality policy; if the service quality policy can be guaranteed, we can continue to fill the tokens in the global token bucket with the existing filling strategy, or reduce the number of tokens in the global token bucket in sequence according to the preset token decrement sequence, and allocate more system resources for the schedulable requests for the service quality policy, so as to further guarantee the service quality policy.

[0165] S903: If the performance loss value is greater than the loss upper limit value, the number of tokens in the global token bucket is increased according to a preset token increment sequence.

[0166] If the performance loss value is greater than the upper limit of the loss, it means that the service quality strategy (lower limit guarantee strategy) is unreasonable, resulting in a large number of non-lower limit guarantee requests cannot be scheduled. At this time, regardless of whether the service quality strategy is achieved, the number of tokens in the global token bucket needs to be increased according to the preset token increment sequence to avoid excessive overall system performance loss and ensure system reliability.

[0167] If the performance loss value is between the critical loss value and the upper limit loss value, in order to ensure the overall performance of the system, the filling quantity of L tokens will be stopped from being adjusted regardless of whether the lower limit guarantee target is achieved.

[0168] In an embodiment of the present application, the token filling strategy of the global token bucket is determined based on the comparison results of the performance loss value with the loss critical value and the loss upper limit value. When the performance loss value is less than the loss critical value, the number of tokens in the global token bucket is increased to achieve the lower limit protection target to the greatest extent. When the performance loss value is greater than the loss critical value, the number of tokens in the global token bucket is reduced to avoid the overall performance out of control caused by the performance of the lower limit protection strategy exceeding the lower limit value, thereby improving the stability and reliability of the system.

[0169] In an exemplary embodiment, the lower limit guarantee scheduling model can be used to execute the request scheduling process. Taking IO request as an example, Fig.10 This is a schematic diagram of the scheduling principle of the lower limit guarantee scheduling model. Fig.10 As shown, the following steps are included:

[0170] S1001, responding to a scheduling instruction of an IO request.

[0171] S1002, scheduling IO requests in the lower limit guarantee queue.

[0172] S1003, scheduling the currently incoming IO request.

[0173] S1004, scheduling IO requests in the common queue.

[0174] S1005, IO request scheduling ends.

[0175] In the embodiment of the present application, the lower limit guarantee scheduling model divides a scheduling process into three steps, first scheduling the IO requests in the lower limit guarantee queue, then scheduling the currently incoming IO, and finally scheduling the IO of the ordinary queue. Unlike the traditional lower limit guarantee queue, the IO requests in the lower limit guarantee queue in the embodiment of the present application are IO requests that have obtained the R token in the last scheduling but cannot be successfully scheduled due to insufficient upper limit of front-end concurrent scheduling. Therefore, when scheduling the IO requests in the lower limit guarantee queue, the embodiment of the present application does not need to do too much queue retrieval and verification operations, and directly and quickly dequeues, effectively reducing the scheduling delay overhead.

[0176] In an exemplary embodiment, the implementation steps of scheduling the currently incoming IO request in the above embodiment are described. Fig.11 As shown, the following steps are included:

[0177] S1101, scheduling the currently incoming IO request.

[0178] Get the reserved token bucket of the service quality policy to which the currently scheduled IO request belongs.

[0179] S1102, determine whether the R token is sufficient.

[0180] The number of R tokens in the reserved token bucket is compared with the token fill number corresponding to the quality of service policy to determine whether the R tokens are sufficient.

[0181] Optionally, if the token quantity of the R token is greater than or equal to the token fill quantity corresponding to the service quality policy, it is determined that the R tokens are sufficient, and tokens are allocated to the target request that matches the token fill quantity; if the token quantity of the R token is less than the token quantity corresponding to the service quality policy, it is determined that the R tokens are insufficient.

[0182] S1103: When there are sufficient R tokens, determine whether the target number of requests for allocating R tokens exceeds the upper limit of concurrent scheduling.

[0183] If there are sufficient R tokens, tokens are allocated to target requests that match the token fill quantity.

[0184] S1104: If the number of target requests does not exceed the upper limit of concurrent scheduling, directly schedule the target requests that are allocated R tokens.

[0185] In this case, there is no need to enqueue the target request, it can be scheduled directly.

[0186] S1105, if the number of target requests exceeds the upper limit of concurrent scheduling, the target requests exceeding the upper limit of concurrent scheduling are placed in the lower limit guarantee queue.

[0187] Among them, the target requests that exceed the upper limit of concurrent scheduling will be placed in the lower limit guarantee queue and have the highest priority scheduling in the next scheduling process.

[0188] S1106: When the R tokens are insufficient, determine whether the L tokens in the global token bucket are sufficient.

[0189] Optionally, if the token quantity of L tokens is greater than or equal to a preset quantity threshold, it is determined that the L tokens are sufficient; if the token quantity of L tokens is less than the preset quantity threshold, it is determined that the L tokens are insufficient.

[0190] S1107, when there are sufficient L tokens, the currently incoming requests that have not been allocated R tokens are placed in the general queue.

[0191] Among them, the requests in the common queue are waiting for subsequent scheduling and dequeuing, and their scheduling priority is lower than the scheduling priority of the requests in the lower limit guarantee queue.

[0192] S1108: When L tokens are insufficient, currently incoming requests that have not been allocated R tokens are placed in a waiting queue.

[0193] In the embodiment of the present application, when the R tokens and the upper limit of concurrent scheduling are sufficient, the lower limit guarantee I / O requests are directly scheduled without entering the queue, thereby reducing the scheduling delay. When the R tokens are sufficient but the upper limit of concurrent scheduling is insufficient, the requests allocated with R tokens are stored in the lower limit guarantee queue, so that they have the highest scheduling priority in the next round of scheduling. In the next round of scheduling, there is no need to check the token resources again and they can be directly dequeued, thereby reducing the scheduling delay. When the R tokens and the upper limit of concurrent scheduling are insufficient, depending on whether the L tokens are sufficient, the requests that are not allocated with R tokens are mounted to the ordinary queue or the waiting queue, thereby providing a reliable basis for the next scheduling.

[0194] In an exemplary embodiment, a method for adjusting L tokens in a global token bucket is provided, such as Fig.12 As shown, Fig.12 The schematic diagram of the L token feedback mechanism includes the following steps:

[0195] S1201, start executing the feedback mechanism.

[0196] S1202, determine whether the lower limit guarantee target is achieved.

[0197] S1203: If not reached, determine whether the performance loss value exceeds the loss critical value.

[0198] S1204: If the performance loss value does not exceed the loss critical value, reduce the L token.

[0199] S1205: If the performance loss value exceeds the loss critical value, determine whether the performance loss value exceeds the loss upper limit value.

[0200] S1206: If the performance loss value exceeds the upper limit of the loss, an L token is added.

[0201] S1207: If the performance loss value does not exceed the upper limit of the loss, stop adjusting the L token.

[0202] By adopting the token feedback mechanism in the embodiment of the present application, through the combination of R tokens and L tokens, the number of L tokens can be dynamically adjusted, the system resources can be flexibly adjusted, the occupation of system resources by non-lower limit guaranteed IO requests can be limited, and the system performance loss can be controlled within a reasonable range.

[0203] In one embodiment, a complete request scheduling method is provided, comprising the following steps:

[0204] (1) Distribute tokens to target requests among multiple requests according to the reserved token bucket corresponding to the service quality policy of the storage system.

[0205] (2) If the concurrent scheduling upper limit is greater than the number of requests in the lower limit guarantee queue, obtain the first request difference between the concurrent scheduling upper limit and the number of requests in the lower limit guarantee queue.

[0206] (3) When the number of target requests is greater than or equal to the first request difference, the requests in the lower limit guarantee queue and the first request in the target request that matches the first request difference are determined as schedulable requests; and the second request in the target request except the first request and the other requests in the multiple requests except the target request are determined as requests to be scheduled.

[0207] (4) When the number of target requests is less than the first request difference, obtain a second request difference between the number of target requests and the first request difference; distribute tokens to requests in the ordinary queue that match the second request difference based on the remaining tokens in the reserved token bucket; determine the requests in the lower limit guarantee queue, the target requests, and the requests in the ordinary queue that match the second request difference as schedulable requests; and determine the remaining requests in the ordinary queue as requests to be scheduled.

[0208] (5) If the concurrent scheduling upper limit is less than or equal to the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue that match the concurrent scheduling upper limit are determined as schedulable requests; and the target request and other requests among the multiple requests except the target request are determined as requests to be scheduled.

[0209] (6) If the request to be scheduled has been assigned a token, the request to be scheduled is stored in the lower limit guarantee queue.

[0210] (7) If the number of tokens in the global token bucket meets the preset conditions, the scheduling request is stored in the normal queue.

[0211] (8) If the number of tokens in the global token bucket does not meet the preset conditions, the scheduling request will be stored in the waiting queue.

[0212] (9) Scheduling the schedulable request based on the scheduling resources for the schedulable request.

[0213] (10) Monitor the performance loss value of the storage system.

[0214] (11) If the performance loss value is less than the loss threshold and the lower limit guarantee strategy is not achieved, the tokens in the global token bucket are reduced according to the preset token reduction sequence.

[0215] (12) If the performance loss value is between the critical loss value and the upper loss value, regardless of whether the lower limit guarantee target is achieved, notify to adjust the tokens in the global token bucket.

[0216] (13) If the performance loss value is greater than the upper limit of the loss, regardless of whether the lower limit guarantee target is achieved, the tokens in the global token bucket are increased according to the preset token increment sequence.

[0217] In the embodiment of the present application, the I / O requests that need to guarantee the lower limit are scheduled by combining the lower limit guarantee queue with the token bucket, ensuring that the I / O requests with the lower limit guarantee can be scheduled out of the queue first, while reducing the delay consumption caused by the queue operation. When processing the current incoming IO request, the I / O request that has obtained the R token but cannot be successfully scheduled due to insufficient front-end concurrency is stored in the lower limit guarantee queue. When scheduling the requests in the lower limit guarantee queue, there is no need to do too much queue retrieval and verification operations, and the queue is directly and quickly dequeued, which effectively reduces the scheduling delay overhead. In addition, when adjusting the tokens in the global token bucket, for scenarios where the current performance loss is already between the critical value and the maximum value of the performance loss, in order to ensure the overall performance of the system, the L tokens will be stopped from being adjusted regardless of whether the lower limit guarantee target is achieved; for scenarios where the current lower limit guarantee is not achieved and the system performance loss is less than the critical value, the L tokens are reduced to limit the non-lower limit guarantee I / O performance, and the occupation of system resources by I / O requests in the non-lower limit guarantee queue is limited by reducing the number of L tokens, thereby facilitating the achievement of the lower limit guarantee target; for scenarios where the current system performance loss exceeds the maximum value, the L tokens are increased to avoid the situation where the unreasonable lower limit guarantee policy configuration leads to an excessive reduction in L tokens, and a large number of non-lower limit guarantee I / Os are hung in the waiting queue due to the inability to obtain L tokens, resulting in a serious decline in the overall system performance.

[0218] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0219] Based on the same inventive concept, the embodiment of the present application also provides a request scheduling device for implementing the request scheduling method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more request scheduling device embodiments provided below can refer to the limitations of the request scheduling method above, and will not be repeated here.

[0220] In an exemplary embodiment, Fig.13 As shown, a request scheduling device is provided, including: a request response module 1301, a request queue module 1302 and a request scheduling module 1303, wherein:

[0221] The request response module 1301 is used to respond to multiple requests currently incoming to the storage system and divide the multiple requests into schedulable requests and to-be-scheduled requests according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system and the number of requests in the cache scheduling queue of the storage system;

[0222] The request queue module 1302 is used to store the request to be scheduled in the cache scheduling queue;

[0223] The request scheduling module 1303 is used to schedule the schedulable requests.

[0224] In an exemplary embodiment, the request response module 1301 includes: a token distribution unit and a request classification unit, wherein:

[0225] A token distribution unit, configured to distribute tokens to a target request among multiple requests according to a reserved token bucket corresponding to a service quality policy of the storage system; the target request represents a request matching a token fill quantity;

[0226] The request classification unit is used to determine schedulable requests and requests to be scheduled according to the concurrent scheduling upper limit, requests in the cache scheduling queue and target requests.

[0227] In an exemplary embodiment, the cache scheduling queue includes a lower limit guarantee queue, a request classification unit, including: a first difference acquisition subunit and a first classification subunit, wherein:

[0228] A first difference acquisition subunit is used to acquire a first request difference between the concurrent scheduling upper limit and the number of requests in the lower limit guarantee queue if the concurrent scheduling upper limit is greater than the number of requests in the lower limit guarantee queue;

[0229] The first classification subunit is used to determine the requests in the lower limit guarantee queue and the first request in the target request that matches the first request difference as schedulable requests when the number of target requests is greater than or equal to the first request difference; and to determine the second request in the target request except the first request and other requests among the multiple requests except the target request as requests to be scheduled.

[0230] In an exemplary embodiment, the cache scheduling queue also includes a common queue; a request classification unit includes: a second difference acquisition subunit, a remaining token distribution subunit and a second classification subunit, wherein:

[0231] A second difference acquisition subunit is used to acquire a second request difference between the number of target requests and the first request difference when the number of target requests is less than the first request difference;

[0232] The remaining token distribution subunit is used to distribute tokens to the requests in the common queue that match the second request difference according to the remaining tokens in the reserved token bucket;

[0233] The second classification subunit is used to determine the requests in the lower limit guarantee queue, the target request, and the requests in the general queue that match the second request difference as schedulable requests; and determine the remaining requests in the general queue as requests to be scheduled.

[0234] In an exemplary embodiment, the cache scheduling queue includes a lower limit guarantee queue; the request classification unit is also used to: when the concurrent scheduling upper limit is less than or equal to the number of requests in the lower limit guarantee queue, determine the requests in the lower limit guarantee queue that match the upper limit of concurrent scheduling as schedulable requests; and determine the target request and other requests among multiple requests except the target request as requests to be scheduled.

[0235] In an exemplary embodiment, the cache scheduling queue includes a lower limit guarantee queue and a candidate queue; the request enqueue module 1302 includes: a lower limit guarantee enqueue unit and a candidate enqueue unit, wherein:

[0236] A lower limit guarantee queue entry unit is used for storing any to-be-scheduled request into the lower limit guarantee queue if the to-be-scheduled request has been assigned a token;

[0237] The candidate enqueue unit is used to store the request to be scheduled in the candidate queue if the request to be scheduled has not been allocated a token.

[0238] In an exemplary embodiment, the candidate queue includes a common queue and a waiting queue, and the candidate enqueue unit includes: a global token bucket acquisition subunit, a common enqueue subunit and a waiting enqueue subunit, wherein:

[0239] A global token bucket acquisition subunit is used to acquire a global token bucket of a storage system;

[0240] The general enqueue subunit is used to store the request to be scheduled into the general queue if the number of tokens in the global token bucket meets the preset conditions;

[0241] The waiting queue subunit is used to store the scheduling request in the waiting queue if the number of tokens in the global token bucket does not meet the preset condition.

[0242] In an exemplary embodiment, the scheduling request module 1303 includes: a scheduling resource determination unit and a scheduling resource application unit, wherein:

[0243] A scheduling resource determination unit, used to determine scheduling resources for schedulable requests in the storage system according to a global token bucket of the storage system;

[0244] The scheduling resource application unit is used to schedule the schedulable requests based on the scheduling resources.

[0245] In an exemplary embodiment, the request scheduling device further includes: a loss value monitoring unit, a token reduction unit and a token promotion unit, wherein:

[0246] A loss value monitoring unit, used to monitor the performance loss value of the storage system;

[0247] A token reduction unit, configured to reduce the number of tokens in the global token bucket according to a preset token reduction sequence if the performance loss value is less than the loss threshold value;

[0248] The token increasing unit is used to increase the number of tokens in the global token bucket according to a preset token increasing sequence if the performance loss value is greater than the loss upper limit value.

[0249] Each module in the above request scheduling device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0250] In an exemplary embodiment, a computer device is provided, wherein the computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.14As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store request scheduling data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a request scheduling method is implemented.

[0251] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0252] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0253] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0254] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0255] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0256] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0257] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0258] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A request scheduling method, characterized in that: The method comprises: In response to multiple requests currently incoming to the storage system, the multiple requests are divided into schedulable requests and to-be-scheduled requests according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system, and the number of requests in the cache scheduling queue of the storage system; Storing the request to be scheduled in the cache scheduling queue; The schedulable request is scheduled.

2. The method according to claim 1, characterized in that The dividing the multiple requests into schedulable requests and to-be-scheduled requests according to the token filling quantity of the storage system, the concurrent scheduling upper limit of the storage system, and the number of requests in the cache scheduling queue of the storage system includes: Distribute tokens to target requests among the multiple requests according to a reserved token bucket corresponding to a quality of service policy of the storage system; the target request represents a request that matches the token fill quantity; The schedulable requests and the to-be-scheduled requests are determined according to the concurrent scheduling upper limit, the requests in the cache scheduling queue, and the target request.

3. The method according to claim 2, characterized in that The cache scheduling queue includes a lower limit guarantee queue, and determining the schedulable request and the request to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request includes: If the concurrent scheduling upper limit is greater than the number of requests in the lower limit guarantee queue, obtaining a first request difference between the concurrent scheduling upper limit and the number of requests in the lower limit guarantee queue; When the number of target requests is greater than or equal to the first request difference, the requests in the lower limit guarantee queue and the first request in the target requests that matches the first request difference are determined as the schedulable requests; and the second request in the target requests except the first request and other requests in the multiple requests except the target request are determined as the requests to be scheduled.

4. The method according to claim 3, characterized in that The cache scheduling queue also includes a common queue; the determining the schedulable request and the request to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request includes: When the number of target requests is less than the first request difference, obtaining a second request difference between the number of target requests and the first request difference; Distribute tokens for requests in the common queue that match the second request difference according to remaining tokens in the reserved token bucket; The requests in the lower limit guarantee queue, the target request, and the requests in the common queue that match the difference with the second request are all determined as the schedulable requests; and the remaining requests in the common queue are determined as the requests to be scheduled.

5. The method according to claim 2, characterized in that: The cache scheduling queue includes a lower limit guarantee queue; the determining the schedulable request and the request to be scheduled according to the concurrent scheduling upper limit, the requests in the cache scheduling queue and the target request includes: If the concurrent scheduling upper limit is less than or equal to the number of requests in the lower limit guarantee queue, the requests in the lower limit guarantee queue that match the concurrent scheduling upper limit are determined as the schedulable requests; and The target request and other requests among the multiple requests except the target request are all determined as the requests to be scheduled.

6. The method according to any one of claims 1 to 5, characterized in that: The cache scheduling queue includes a lower limit guarantee queue and a candidate queue; storing the to-be-scheduled request in the cache scheduling queue includes: For any request to be scheduled, if the request to be scheduled has been allocated a token, the request to be scheduled is stored in the lower limit guarantee queue; If the request to be scheduled is not allocated a token, the request to be scheduled is stored in the candidate queue.

7. The method according to claim 6, characterized in that The candidate queue includes a common queue and a waiting queue, and storing the to-be-scheduled request in the candidate queue includes: Obtain a global token bucket of the storage system; If the number of tokens in the global token bucket meets a preset condition, the request to be scheduled is stored in the common queue; If the number of tokens in the global token bucket does not satisfy the preset condition, the request to be scheduled is stored in the waiting queue.

8. The method according to any one of claims 1 to 5, characterized in that: The scheduling of the schedulable request includes: Determining, according to a global token bucket of the storage system, a scheduling resource in the storage system for the schedulable request; The schedulable request is scheduled based on the scheduling resource.

9. The method according to claim 8, characterized in that The method further comprises: Monitoring the performance loss value of the storage system; If the performance loss value is less than the loss threshold, reducing the number of tokens in the global token bucket according to a preset token decrement sequence; If the performance loss value is greater than the loss upper limit value, the number of tokens in the global token bucket is increased according to a preset token increment sequence.

10. A request scheduling device, characterized in that: The device comprises: A request response module, configured to respond to multiple requests currently incoming to the storage system and divide the multiple requests into schedulable requests and to-be-scheduled requests according to the number of tokens filled in the storage system, the concurrent scheduling upper limit of the storage system, and the number of requests in the cache scheduling queue of the storage system; A request enqueue module, used for storing the request to be scheduled in the cache scheduling queue; A request scheduling module is used to schedule the schedulable requests.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images