Scheduling method and device based on quality of service resources
By using reserved binary heaps, weighted binary heaps, and upper bound binary heaps in a distributed storage system to generate and update clock tags for tenant request queues, the problems of high computational overhead and unfair scheduling in the mclock algorithm are solved, achieving efficient and stable QoS scheduling.
Patent Information
- Application Number
- CN202211449853.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-11-18
AI Technical Summary
Existing technologies for QoS scheduling based on the mclock algorithm in distributed storage systems suffer from high computational overhead, are prone to blocking scheduling, and struggle to guarantee scheduling fairness and response latency, especially when the number of tenants is large or the state changes.
The system employs a reserved binary heap, a weighted binary heap, and an upper bound binary heap structure to generate and update the clock tag of the tenant request queue, reducing computational overhead. The system also uses binary heap adjustments to ensure scheduling fairness and response latency.
It significantly reduces the computational overhead of clock tag generation and updating, improves scheduling efficiency, ensures scheduling fairness and response latency stability, and adapts to the service quality requirements of different tenants.
Smart Images

Figure CN115776471B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of Internet technology, and in particular to a scheduling method and device based on quality of service resources. Background Art
[0002] With the development of mobile Internet cloud computing, distributed storage technology, as the underlying technology cornerstone, is crucial to meet the storage quality requirements of different tenants in a multi-tenant mode. Distributed storage systems need to solve the scheduling and on-demand allocation of limited storage space and input and output resources under the premise of ensuring performance and reliability, so as to meet the resource service requirements of different tenants, so that the system as a whole can provide more user-friendly storage services. Therefore, it is extremely important for distributed storage systems to schedule resources based on Quality of Service (QoS) and meet the scheduling fairness of different tenants' needs, ensure the response delay corresponding to requests from different tenants and different types, and balance the scheduling overhead and overall throughput of the system.
[0003] Currently, scheduling is often performed based on mclock. A clock tag is generated for each request received, and scheduling is performed based on the clock tag corresponding to each request. After a request is scheduled out of the queue, the clock tags of other related requests need to be updated. The computational overhead required for scheduling, generating and updating clock tags in this way is large, and it is easy to block scheduling. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides a scheduling method and device based on quality of service resources.
[0005] In a first aspect, the present disclosure provides a scheduling method based on quality of service resources, comprising:
[0006] Obtain the request queues corresponding to each tenant, and generate a clock label for the head request of the request queue corresponding to each tenant; the clock label includes a reserved label, a weight label and an upper limit label; wherein the client is used to carry one or more tenants, and the server is used to maintain the request queues of different tenants carried by the client;
[0007] A reserved binary heap, a weighted binary heap and an upper limit binary heap are created based on the clock labels of the head requests in the request queues corresponding to the tenants; the clock label value corresponding to each tree node in the reserved binary heap is the reserved label value of the corresponding head request, the clock label value corresponding to each tree node in the weighted binary heap is the weighted label value of the corresponding head request, and the clock label value corresponding to each tree node in the upper limit binary heap is the upper limit label value of the corresponding head request;
[0008] Based on the clock label values corresponding to each tree node in the reserved binary heap, the weighted binary heap and the upper limit binary heap, the request queue corresponding to the target tenant is scheduled to dequeue the head request, and the next request is updated to be the head request and a clock label is generated for the updated head request.
[0009] In some embodiments, it also includes: adjusting the positions of the target tenant in the tree nodes corresponding to the reserved binary heap, the weighted binary heap and the upper limit binary heap respectively based on the clock label of the updated head-of-queue request.
[0010] In some embodiments, it also includes:
[0011] Receive new requests sent by tenants;
[0012] If the request queue corresponding to the tenant is empty, inserting the newly added request into the request queue corresponding to the tenant, and generating a clock tag for the newly added request based on the arrival time of the newly added request;
[0013] If the request queue corresponding to the tenant is not empty, the newly added request is inserted into the tail of the request queue corresponding to the tenant.
[0014] In some embodiments, it also includes:
[0015] Based on the clock label corresponding to the newly added request, the positions of the tree nodes corresponding to the tenant in the reserved binary heap, the weighted binary heap and the upper limit binary heap are adjusted respectively.
[0016] In some embodiments, it also includes:
[0017] Obtaining the weight label value of the root node of the weighted binary heap;
[0018] According to the time difference between the weight label value of the root node and the current time, the position of the tree node corresponding to the tenant to which the newly added request belongs in the weighted binary heap is adjusted.
[0019] In some embodiments, generating a clock tag includes:
[0020] Based on the data block size requested by the corresponding head-of-team request, a normalized number of read / write operations per second (IOPS) corresponding to the head-of-team request is obtained;
[0021] A clock tag of the head request is generated according to the normalized IOPS, the target IOPS and the current timestamp corresponding to the head request.
[0022] In some embodiments, generating a clock tag includes:
[0023] A clock tag of the head-of-team request is generated according to the data block size, target bandwidth and current timestamp requested by the head-of-team request.
[0024] In some embodiments, generating the clock tag includes: generating the clock tag based on a physical clock timestamp.
[0025] In some embodiments, the clock tag is represented using a signed integer data type.
[0026] In a second aspect, the present disclosure provides a scheduling device based on quality of service resources, including:
[0027] An acquisition module is used to acquire request queues corresponding to each tenant; wherein the client is used to carry one or more tenants, and the server is used to maintain request queues of different tenants carried by the client;
[0028] A clock label processing module, configured to generate a clock label for the head request of the request queue corresponding to each of the tenants; the clock label includes a reserved label, a weight label and an upper limit label;
[0029] A binary heap processing module is used to create a reserved binary heap, a weighted binary heap and an upper limit binary heap based on the clock labels of the head requests in the request queues corresponding to the tenants; the clock label value corresponding to each tree node in the reserved binary heap is the reserved label value of the corresponding head request, the clock label value corresponding to each tree node in the weighted binary heap is the weighted label value of the corresponding head request, and the clock label value corresponding to each tree node in the upper limit binary heap is the upper limit label value of the corresponding head request;
[0030] A scheduling module, configured to schedule the request queue corresponding to the target tenant to dequeue the head request based on the clock label values corresponding to each tree node in the reserved binary heap, the weighted binary heap, and the upper limit binary heap, and update the next request to be the head request;
[0031] The clock tag processing module is also used to generate a clock tag for the updated head-of-queue request.
[0032] In a third aspect, the present disclosure provides an electronic device, including: a memory and a processor;
[0033] The memory is configured to store computer program instructions;
[0034] The processor is configured to execute the computer program instructions so that the electronic device implements the scheduling method based on quality of service resources as described in the first aspect and any one of the first aspects.
[0035] In a fourth aspect, the present disclosure provides a readable storage medium, comprising: computer program instructions; at least one processor of an electronic device executes the computer program instructions, so that the electronic device implements the scheduling method based on service quality resources as described in the first aspect and any one of the first aspect.
[0036] In a fifth aspect, the present disclosure provides a computer program product, and an electronic device runs the computer program product so that the electronic device implements the scheduling method based on quality of service resources as described in the first aspect and any one of the first aspects.
[0037] The present disclosure provides a scheduling method and device based on quality of service resources, wherein the method obtains the request queues corresponding to each tenant, generates a clock label for the head request of each request queue, and then creates a reserved binary heap, a weighted binary heap, and an upper limit binary heap based on the clock label generated for the head request of each request queue; then schedules the head request of the target tenant to be dequeued based on the reserved binary heap, the weighted binary heap, and the upper limit binary heap, and updates the next request to be the head request and generates a clock label for the updated head request. Using the method of the present disclosure, a clock label is generated for the head request of the request queue corresponding to each tenant, which greatly reduces the computational overhead of generating the clock label compared to the traditional method of generating a clock label for each request; in addition, after scheduling a request to be dequeued, a clock label is generated for the updated head request, without the need to modify the clock labels of all requests in the tenant's request queue as in the traditional method, which greatly reduces the computational overhead of updating the clock label. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0040] Figure 1 A flowchart of a scheduling method based on quality of service resources provided in an embodiment of the present disclosure;
[0041] Figure 2 A schematic diagram of the structure of the secondary mapping queue provided by the present disclosure;
[0042] Figure 3 Based on Figure 2 The schematic diagram of the structure of the reserved binary heap, weighted binary heap and upper limit binary heap constructed by the secondary mapping queue shown;
[0043] Figure 4 A flowchart of a scheduling method based on quality of service resources provided by another embodiment of the present disclosure;
[0044] Figure 5 A flowchart of a scheduling method based on quality of service resources provided by another embodiment of the present disclosure;
[0045] Figure 6 A schematic diagram of the structure of a scheduling device based on quality of service resources provided in one embodiment of the present disclosure. DETAILED DESCRIPTION
[0046] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0048] At present, the mclock algorithm is often used for QoS scheduling. In order to make it clearer, here we first introduce the QoS scheduling based on the mclock algorithm in detail:
[0049] From a logical structure perspective, mclock is divided into two parts: client and server.
[0050] Client: refers to a certain client or a certain type of client, which resides on the actual client or server. The client is responsible for issuing tenant QoS parameters, collecting request completion information, etc. QoS parameters include the ID, reserved quota, weight quota, upper quota, etc. corresponding to the tenant. Based on the three quota parameters, high limit, low guarantee and weight scheduling are achieved.
[0051] Server: Complete specific input and output scheduling, and the server will maintain a request queue for each tenant. The server will generate a clock label for the received request, including: reserved label (R label), weight label (P label) and upper limit label (L label), and generate the clock label through formula (1), which is as follows:
[0052]
[0053] In formula (1), the superscript r represents the tenant's rth request, and currentTime represents the time when the request arrives. Based on formula (1), it can be seen that the reservation label, weight label, and upper limit label corresponding to each rth request of a tenant are generated based on the reservation label, weight label, and upper limit label corresponding to the r-1th request. Among them, the clock label for the first request of the tenant is initially set to the arrival time of the request.
[0054] Referring to the above formula (1), it can be seen that the reservation tag is a time tag assigned to the request based on the reserved quota of the request, the weight tag is a time tag assigned to the request based on the weight quota of the request, and the upper limit tag is a time tag assigned to the request based on the upper limit quota of the request; wherein the reservation quota indicates the minimum resource that the server needs to provide; the upper limit quota indicates the maximum resource upper limit that the server can provide for the request; the weight quota indicates the proportion of resources occupied by the request when processing a request that does not reach the upper limit quota when the server meets the reserved quota of all requests. The above formula (1) can represent the reservation quota, weight quota and upper limit quota by time. Therefore, the reservation tag is equivalent to indicating the length of time occupied by the reserved quota corresponding to the request; the weight tag is equivalent to indicating the length of time occupied by the weight quota corresponding to the request; and the upper limit tag is equivalent to indicating the length of time occupied by the upper limit quota corresponding to the request. Taking IOPS as an example, the reserved quota of a request can be 1000IOPS, the upper limit quota can be 2000IOPS, and the weight quota can be 3, wherein the larger the value corresponding to the weight quota, the higher the proportion of resources occupied. The reserved quota and upper limit quota can also be expressed in terms of bandwidth.
[0055] In a distributed scenario, the clock label of the request can be generated by formula (2), which is as follows:
[0056]
[0057] In a distributed scenario, the client records the number of requests completed by each server. When sending a request corresponding to a tenant to the server, it carries the increment of the number of completed requests received since the last request was sent, and is the sum of the number of requests completed by other servers except the target server, where ρ represents the increment from the last request reservation stage, and δ represents all increments from the last request completion.
[0058] The scheduling based on QoS resources using the mclock algorithm is divided into two stages: reservation stage and weight stage.
[0059] Reservation phase: Query requests with R tag values less than or equal to currentTime for scheduling.
[0060] Weight phase: After all requests that meet the constraints in the reservation phase are scheduled, they enter the weight phase. In the request set whose L label value is less than or equal to currentTime, the request with the smallest P label value is selected for scheduling. When a tenant's request is scheduled in the weight phase, the reservation label value of all requests of the tenant in the current queue needs to be subtracted by 1 / R i , so that the tenant's reserved quota will not be affected by the weight part, ensuring scheduling fairness.
[0061] Among them, when scheduling in the reservation phase and weight, the currentTime here refers to the scheduling time, which is the timestamp when the scheduling is actually performed and has nothing to do with the time when the request arrives.
[0062] It should be emphasized that since the weight label value is relative, when a new tenant joins or an existing client recovers from idle to active, every time a new request arrives, the P label values of all tenants' requests need to be calibrated and adjusted. Specifically, a difference value needs to be added to the P label value of each request. This difference value is the difference between the current time and the minimum P label value to ensure the scheduling fairness of all tenants' requests and avoid starvation of the active tenant's request scheduling.
[0063] Using the mclock algorithm for QoS resource-based scheduling has at least the following problems:
[0064] 1. In the weighted phase, whenever a request is scheduled, the reserved tag values of all requests of the tenant need to be adjusted. When the number of tenant requests is large, adjusting the reserved tag value will inevitably be very expensive and may block scheduling, thereby blocking all requests, and thus causing all tenants' request delays and overall throughput to be unable to be guaranteed.
[0065] 2. When new tenants are added or existing tenants are restored from idle to active, the weight label values of all requests need to be calibrated and adjusted, which will inevitably result in high overhead.
[0066] 3. The workload of each tenant is relatively complex, and the size of the requested data blocks often fluctuates greatly. How to measure the consumption of input and output resources based on this? Unified processing will lead to poor scheduling fairness.
[0067] 4. The scheduler's overhead and throughput are the most important basic guarantees for evaluating the scheduling effect. Scheduling based on the traditional mclock algorithm cannot conveniently and intuitively evaluate the scheduling effect of the scheduler. The scheduler can be understood as a functional module in the server that specifically executes scheduling based on QoS resources.
[0068] Based on the above problems, the present disclosure provides a scheduling method and device based on service quality resources, wherein, using the method of the present disclosure, the server generates a clock label for the head request of the request queue corresponding to each tenant, which greatly reduces the computational overhead of generating the clock label compared to the traditional method of generating a clock label for each request; in addition, after scheduling a request out of the queue, the server generates a clock label for the updated head request, without modifying the clock labels of all requests in the tenant's request queue as in the traditional method, which greatly reduces the computational overhead caused by updating the clock label.
[0069] It should be noted that in the traditional mclock algorithm and dmclock algorithm, a client can be understood as a tenant, while in the method provided in the present invention, the tenant is a logical tenant, and a client can carry one or more tenants, wherein the requests can be classified in advance, and a unified class of requests sent by the client corresponds to a tenant. The client will carry the tenant's identifier when sending the request, and the server will schedule in units of tenants.
[0070] Exemplarily, the scheduling method based on quality of service resources provided in the present disclosure can be performed by the scheduling device based on quality of service resources provided in the present disclosure, and the scheduling device based on quality of service resources can be implemented in any software and / or hardware manner. For example, it can be but not limited to: electronic devices such as servers, Internet of Things servers, cloud servers, server clusters, etc., and the present disclosure does not limit the type of electronic devices.
[0071] Next, the scheduling method based on quality of service resources provided by the present disclosure will be described in detail through several specific embodiments. In the following embodiments, a scheduling device based on quality of service resources (hereinafter referred to as a scheduler) is used as an example for illustration.
[0072] Figure 1 A flowchart of a scheduling method based on quality of service resources provided by an embodiment of the present disclosure. Figure 1 As shown, the method of this embodiment includes:
[0073] S101. Obtain request queues corresponding to each tenant, and generate clock labels for head requests of the request queues corresponding to each tenant, wherein the clock labels include a reserved label, a weight label, and an upper limit label.
[0074] The implementation of the scheduler includes two levels of mapping queues, wherein the first level is a client queue created based on the client, each client can carry one or more tenants, therefore, the first level can also be understood as a tenant queue established based on the tenants carried by the client, and the second level is a real request queue on the server side, wherein the request queue of a single tenant is used as an identifier in this disclosure. The scheduler can generate three clock labels for the head request of each tenant's request queue, namely, a reserved label Ri, a weight label Wi, and an upper limit label Li, wherein the subscript i represents the number of the tenant to which the head request belongs, and is used to identify the tenant.
[0075] For example, Figure 2 As shown in the figure, suppose there are 5 tenants, namely tenant 1 to tenant 5. The first-level queue will be created according to the tenants hosted by the client; suppose tenant 1 and tenant 2 each send 4 requests, and the request queues of tenants 3 to 5 are similar. Figure 2 The request queues of tenants 3 to 5 are not shown. In addition, the first request in the request queues corresponding to tenants 1 to 5 contains the corresponding clock tag, and the other requests after the first request do not have a clock tag. Please continue to refer to Figure 2 As shown in the figure, the three clock labels corresponding to tenant 1 are (R1, W1, L1), and the three clock labels corresponding to tenant 2 are (R2, W2, L2). Figure 2 The clock labels of the request queues corresponding to tenants 3 to 5 are not shown.
[0076] When the scheduler receives a head-of-line request from a tenant, it can substitute the QoS parameter carried in the head-of-line request into the formula (1) or formula (2) mentioned above to obtain the clock label corresponding to the head-of-line request.
[0077] S102: Create a reserved binary heap, a weighted binary heap, and an upper limit binary heap based on the clock label of the first request in the request queue corresponding to each tenant.
[0078] The scheduling queue is organized and managed by a priority queue implemented by a binary heap, and the bottom layer of the binary heap is based on a cache-friendly linear structure storage implementation. The heap construction method builds a reserved binary heap, a weighted binary heap, and an upper limit binary heap for the top of the small heap. Among them, the tree nodes in the reserved binary heap, the weighted binary heap, and the upper limit binary heap all point to the request queue corresponding to a tenant, and the position of the tree node in the corresponding binary heap is determined based on the clock label value of the head request.
[0079] 1. Reserved binary heap: When creating a reserved binary heap, the position of the tree node corresponding to the tenant in the binary heap is determined based on the size of the reserved label value of the head request of the request queue corresponding to the tenant. The smaller the reserved label value, the closer the tree node corresponding to the tenant is to the root node, and the larger the reserved label value, the closer the tree node corresponding to the tenant is to the leaf node. The reserved label value corresponding to the root node of the reserved binary heap is the smallest.
[0080] 2. Weighted binary heap: When creating a weighted binary heap, the position of the tree node corresponding to the tenant in the weighted binary heap is determined by the upper limit label value and the weight label value. Among them, all tree nodes in the weighted binary heap contain a ready label representing whether the dequeue condition is met. The ready label is determined based on the upper limit label value requested by the head of the queue. If the upper limit label value is less than or equal to the current time, the ready label is true. If the upper limit label value is greater than the current time, the ready label value is false. The smaller the weight label value and the tenant whose ready label is true, the closer the tree node corresponding to the tenant in the weighted binary heap is to the root node. The larger the weight label value and the tenant whose ready label is false, the closer the tree node corresponding to the tenant in the weighted binary heap is to the leaf node. The root node of the weighted binary heap has the smallest weight label value.
[0081] 3. Upper-bound binary heap: When creating an upper-bound binary heap, the position of the tree node corresponding to the tenant in the upper-bound binary heap is determined by the upper-bound label value. All tree nodes in the upper-bound binary heap contain a ready label representing whether the dequeue condition is met (consistent with the ready label on the node of the weighted binary heap). The tree node corresponding to the tenant with a smaller upper-bound label value is closer to the root node in the upper-bound binary heap, and the tree node corresponding to the tenant with a larger upper-bound label value is closer to the leaf node in the upper-bound binary heap. The root node of the upper-bound binary heap has the smallest upper-bound label value.
[0082] For example, Figure 3 Shown based on Figure 2 The structure diagram of the reserved binary heap, weighted binary heap, and upper limit binary heap created by the secondary mapping queue shown in the figure is shown in the figure. Figure 3 The binary heap on the far left is the reserved binary heap, the binary heap in the middle is the weighted binary heap, and the binary heap on the far right is the upper limit binary heap. The clock label value of each tree node in the reserved binary heap is the reserved label value requested by the corresponding team leader, the clock label value of each tree node in the weighted binary heap is the weighted label value requested by the corresponding team leader, and the clock label value of each tree node in the upper limit binary heap is the upper limit label value requested by the corresponding team leader. Figure 3From the three binary heaps shown, it can be seen that since the reserved label value, weight label value and upper limit label value corresponding to the first request in the tenant's request queue are different, the positions of the tree nodes corresponding to the same tenant in different binary heaps are different.
[0083] S103, based on the clock label values corresponding to each tree node in the reserved binary heap, the weighted binary heap and the upper limit binary heap, schedule the head request of the request queue corresponding to the target tenant to be dequeued, update the next request to be the head request, and generate a clock label for the updated head request.
[0084] Scheduling request dequeue is the process of selecting a reasonable tenant and taking the first element from its request queue. Combined with the introduction of the mclock algorithm in the previous article, we can see that scheduling is divided into two stages: reservation stage and weight stage.
[0085] Reservation phase: traverse the reservation binary heap, starting from the root node, and determine whether the reservation tag value of the head request of each tree node in the reservation binary heap is less than or equal to the current time (scheduling time). This is used as the basis for whether the dequeue condition is met. If it is met, the tenant corresponding to the corresponding node is determined as the target tenant, and the head request in the request queue corresponding to the target tenant is scheduled to be dequeued.
[0086] Weight phase: Starting from the root node of the upper limit binary heap, traverse the upper limit label value of the head request on each tree node to see if it is less than or equal to the current time (scheduling time), that is, judge whether the dequeue condition is met based on the upper limit label value, and set the ready label that meets the dequeue condition to true to determine whether it can participate in the subsequent weight competition. The traversal continues until the root node does not meet the dequeue condition. Among them, weight competition refers to the process of adjusting the position of the tree node corresponding to the tenant in the weighted binary heap according to the weight label value of its head request for all tenants that meet the upper limit condition (ready label is true). Finally, the tenant at the root node of the weighted binary heap wins and becomes the target tenant, and finally the head request of the target tenant's request queue is scheduled to be dequeued.
[0087] It should be noted that the position adjustment of the tree nodes of the winning tenant in the upper limit binary heap and the position adjustment of the tree nodes in the weighted binary heap are carried out synchronously, that is, the position of the tree nodes corresponding to the tenant in the weighted binary heap is adjusted synchronously when the tree nodes corresponding to the tenant are judged one by one based on the upper limit label value. Finally, the tree nodes in the weighted binary heap that meet the dequeue conditions are moved to the upper layer, and finally the node corresponding to the target tenant that meets the dequeue conditions (i.e., the root node) is selected from the weighted binary heap, and the head request of the node corresponding to the target tenant is dequeued.
[0088] After the head request of the target tenant is dequeued, the request queue of the target tenant will change. After the head request is dequeued, if the request queue is not empty, the next request in the request queue adjacent to the dequeued head request will become the new head request of the request queue. Therefore, it is necessary to generate a clock label for the updated head request. The scheduler can use the above formula (1) or formula (2) to substitute the relevant QoS parameters into the formula to generate the clock label of the updated head request.
[0089] The method of this embodiment generates a clock tag for the head request of the request queue corresponding to each tenant, which greatly reduces the computational overhead of generating clock tags compared to the traditional method of generating a clock tag for each request. In addition, after scheduling a request out of the queue, a clock tag is generated for the updated head request, and there is no need to modify the clock tags of all requests in the request queue of the tenant as in the traditional method, which greatly reduces the computational overhead caused by updating the clock tags.
[0090] exist Figure 1 In the illustrated embodiment, both step S101 and step S103 involve generating a clock tag. The method provided by the present disclosure uses a physical clock timestamp for calculation to generate a clock tag for the head-of-queue request, and the timestamp can use a nanosecond-level timestamp with higher accuracy to adapt to distributed storage systems oriented to microsecond-level delays.
[0091] The real-world time corresponding to the physical clock can usually be provided by the operating system. The physical clock timestamp is used to calculate the clock label of the request, which has better observability and higher readability. It is also very friendly and convenient for evaluating the scheduling effect of the scheduler and troubleshooting.
[0092] In addition, the generated clock label can be represented by an integer (INT) data type, for example, INT 64. Compared with floating-point number identification, the integer data type is more CPU-friendly and has a lower overhead, which can improve the efficiency of clock label generation.
[0093] Figure 4 A flowchart of a scheduling method based on quality of service resources provided by another embodiment of the present disclosure. Figure 4 As shown, the method provided in this embodiment is Figure 1 Based on the embodiment shown, after step S103, the following further includes:
[0094] S104. Adjust the tree node positions corresponding to the target tenant in the reserved binary heap, the weighted binary heap, and the upper limit binary heap based on the updated clock label of the head-of-queue request.
[0095] The tree nodes in the reserved binary heap, weighted binary heap, and upper-bounded binary heap are not isolated, but point to the request queue of the same tenant. After a head request is dequeued from the request queue corresponding to any tenant, the clock label of the tree node corresponding to the corresponding tenant changes due to the change of the head request, which will affect the position of the tree node corresponding to the tenant in the three binary heaps. Therefore, after dequeuing a head request, it is necessary to adjust the node position of the tree node (root node) corresponding to the target tenant in the reserved binary heap, weighted binary heap, and upper-bounded binary heap.
[0096] The specific adjustments are as follows:
[0097] When dequeuing a request in the reservation phase, it is necessary to adjust the position of the tree node corresponding to the tenant in the reserved binary heap and make a small top heap adjustment. It is also necessary to make a small top heap adjustment for the tree node of the tenant in the weighted binary heap and the upper limit binary heap. That is, after the reservation node dequeues the head request of the target tenant, it is necessary to adjust the position of the tree node corresponding to the target tenant in the reserved binary heap, weighted binary heap, and upper limit binary heap respectively.
[0098] When a request is dequeued in the weighted phase, the position of the tree node corresponding to the tenant in the weighted binary heap needs to be adjusted, and the impact on the reserved binary heap also needs to be considered. In the traditional way, the reserved label value of all requests held in the request queue corresponding to the dequeued tenant needs to be subtracted by 1 / R i The time interval is adjusted and the tree node corresponding to the tenant is adjusted to the appropriate position. By subtracting 1 / R i The time interval ensures that the reserved quota is not affected by the weighted quota, ensures the scheduling fairness of the reserved quota, and prevents the tenant from starving during the reservation phase.
[0099] In this embodiment, however, there is no need to adjust all requests held in the request queue corresponding to the tenant. Instead, the reserved label value of the tenant's head request is updated first, which results in minimal overhead. After the reserved label value of the head request is updated, the position of the node corresponding to the tenant's head request in the reserved binary heap is adjusted. That is, in this embodiment, after the head request of the target tenant is dequeued in the weighted phase, the positions of the tree nodes corresponding to the target tenant in the weighted binary heap and the upper limit binary heap are adjusted based on the updated weight label value and upper limit label value of the head request, and the updated reserved label value of the head request is subtracted by 1 / R. i The time interval is adjusted, and then the position of the tree node corresponding to the target tenant in the reserved binary heap is adjusted.
[0100] In comparison, the traditional method requires updating the reserved tag values of all requests held in the request queue corresponding to the tenant. With the method disclosed in the present invention, it is only necessary to update the reserved tag value of the first request in the queue, with extremely low overhead. The effect is more obvious when the number of requests of the tenant is large.
[0101] In this embodiment, after the scheduling tenant's head-of-queue request is dequeued, a clock label is generated for the updated head-of-queue request, and the positions of the corresponding tree nodes of the target tenant in the reserved binary heap, the weighted binary heap and the upper limit binary heap are timely updated based on the newly generated clock label. Since the clock label value of the tenant's head-of-queue request is updated first to adjust its position in the reserved binary heap, the weighted binary heap and the upper limit binary heap, the overhead is extremely small.
[0102] Figures 1 to 4 The embodiment shown mainly introduces how to schedule request dequeue and how to adjust and update the three binary heaps after dequeuing a request. Figure 5 The illustrated embodiment describes how to enqueue received requests.
[0103] Figure 5 A flowchart of a scheduling method based on quality of service resources provided by another embodiment of the present disclosure. Figure 5 As shown, the method of this embodiment includes:
[0104] S501: Receive a new addition request sent by a tenant.
[0105] The new request carries the tenant's ID and QoS parameters. After receiving the new request, it is necessary to add the new request to the request queue corresponding to the tenant based on the tenant's ID. Therefore, first determine whether the request queue corresponding to the tenant is empty. If the request queue corresponding to the tenant is empty, execute step S502; if the request queue corresponding to the tenant is not empty, execute step S503.
[0106] S502: Insert the newly added request into the request queue corresponding to the tenant, and generate a clock tag for the request based on the arrival time of the newly added request.
[0107] Among them, if the tenant sending the new request is a new tenant, it is necessary to add the new tenant to the first-level queue of the secondary mapping queue, generate a second-level request queue, and insert the new request into the tenant's request queue. The new request is the first request in the tenant's request queue. If all requests of the tenant sending the new request have been dispatched out of the queue, the first-level queue of the secondary mapping queue contains the tenant, but the second-level request queue corresponding to the tenant is empty, the new request can be inserted into the corresponding request queue. The new request is the first request in the tenant's request queue.
[0108] Afterwards, the scheduler may generate a clock tag for the newly added request based on the above formula (1) or formula (2). Similarly, the clock tag may be generated using a physical clock timestamp, and the clock tag is represented by an integer data type.
[0109] S503: Insert the newly added request into the tail of the request queue corresponding to the tenant.
[0110] Since there are unscheduled requests in the request queue corresponding to the tenant, the new request can be inserted at the end of the corresponding request queue, and the scheduler does not need to generate a clock label for the new request, but only needs to ensure that the clock label of the head request of the request queue is ready.
[0111] When a new request is received, the computational overhead of generating the clock tag can be greatly reduced by enqueuing the new request and generating a clock tag for the new request as the head request when the request queue of the tenant corresponding to the new request is empty.
[0112] exist Figure 5 Based on the illustrated embodiment, optionally, after S503, the following steps may also be included:
[0113] S504. Adjust the positions of the tree nodes corresponding to the tenant in the reserved binary heap, the weighted binary heap, and the upper limit binary heap respectively based on the clock label corresponding to the newly added request.
[0114] If the tenant who sends the new request is a new tenant, the new tenant needs to be added as a new node to the reserved binary heap, weighted binary heap and upper limit binary heap. The new node is first inserted at the end of the binary heap, and then the new node is adjusted to the appropriate position based on the clock label of the head request (i.e., the new request) corresponding to the tenant. For the reserved binary heap, when adjusting, the small top heap adjustment is based on the reserved label value of the head request of the new node, and the reserved label value of the root node is finally the smallest; for the weighted binary heap, the tree node in the binary heap has two states, which can be distinguished based on the ready label. When adjusting the position of the new node, first determine the state of the ready label of the head request, and make small top heap adjustments based on the priority of meeting the dequeue condition (ready label is true) and the size of the weight label value; for the upper limit binary heap, make small top heap adjustments based on the priority of not meeting the dequeue condition (ready label is false) and the size of the upper limit label value.
[0115] If the tenant sending the new request is an existing tenant and the request queue corresponding to the tenant is empty, there will be a tree node corresponding to the tenant in the three binary heaps. The position of the tree node corresponding to the tenant in the three binary heaps can be adjusted based on the value of the clock label corresponding to the new request. The adjustment method is similar to adjusting the position of the tree node in the binary heap based on the clock label of the new tenant. For the sake of brevity, it will not be repeated here.
[0116] If the request queue of the tenant that sends the new request is not empty, the values of the three clock labels of the first request in the request queue will not change. Therefore, no adjustment is required in this case.
[0117] Optionally, it may also include:
[0118] S505 , obtaining a weight label value of a root node of a weighted binary heap, and obtaining a weight difference value according to the weight label value of the root node and the current time.
[0119] S506. Calibrate the weight label value corresponding to the tenant sending the new addition request based on the weight difference, and adjust the position of the tree node corresponding to the tenant sending the new addition request in the weighted binary heap based on the calibrated weight label value.
[0120] In the present disclosure, if the tenant sending the new request is a new tenant, or the tenant sending the new request is an existing tenant and the tenant is restored from an idle state to an active state, in order to ensure scheduling fairness, it is necessary to calibrate and adjust the weight tags of the relevant requests. Exemplarily, whether a request arrives within a preset time period can be used as a condition for judging whether the tenant is idle or active. The preset time period can be configured, such as a preset time period of 5 minutes, 6 minutes, etc. The present disclosure does not limit the preset time period. Alternatively, the tenant's status can also be determined to be idle or active based on other configuration parameters.
[0121] When it is determined that a tenant meets any of the above conditions, the weighted binary heap corresponding to the tenant is adjusted. First, the weight tag value of the head request at the top of the weighted binary heap (i.e., the head request of the root node) is obtained, recorded as min_weight_tag. When the tenant corresponding to the root node is not the tenant who sends the new request, the time difference between min_weight_tag and currentTime is calculated. The time difference is the weight difference. The time difference is added to the weight tag value of the head request of the tenant corresponding to the new request. The tenant who sends the new request is adjusted to align the tenant who sends the new request with other tenants to avoid request starvation.
[0122] In other embodiments, it is also possible to obtain the weight label values of multiple tree nodes in the upper layer of the weighted binary heap, calculate the average of the weight label values of the multiple tree nodes, and then calculate the time difference between the average and the current time. The time difference is the weight difference. The time difference is added to the weight label value of the head request of the tenant corresponding to the new request, and adjust the tenant who sends the new request to align the tenant who sends the new request with other tenants to avoid request starvation.
[0123] The present invention does not need to adjust the weight labels of all requests immediately as in the traditional method, but adjusts the weight binary heap during the scheduling process. Specifically, during the adjustment process, since the clock label generation of the updated head of the queue request depends on the previous request that was dequeued, and the clock label of the previous request has been added with the previously calculated time difference, generating the clock label for the updated head of the queue request is equivalent to adding the previously calculated time difference, so no additional overhead is generated.
[0124] exist Figures 1 to 5 In the illustrated embodiment, since the sizes of IO data blocks (referred to as data blocks for short) requested by different requests are different, under bandwidth limitation conditions, large IO data blocks require greater transmission delay than small IO data blocks. In other words, small IO data blocks are more sensitive to delay than large IO data blocks. Therefore, it is necessary to prioritize the service quality of small IO data blocks to avoid small IO data blocks being blocked by large IO data blocks. Therefore, the present disclosure can optimize and adjust the clock label based on the normalized IOPS corresponding to the request or the bandwidth required by the request to ensure that the scheduling order of small IO data blocks and large IO data blocks is more reasonable, thereby ensuring the service quality under different IO sizes.
[0125] 1. IOPS: Based on the set IO data block size, the requested data block size is normalized to obtain the normalized IOPS, and the clock label is calculated based on the normalized IOPS and the Qos parameter. The clock label optimized based on IOPS can be obtained by the following formula (3):
[0126]
[0127] In formula (3), n represents normalized IOPS; Rs r 、Ps r , Ls r They represent the reservation label, weight label, and upper limit label after IOPS optimization; r s 、w s , l s They represent the target IOPS corresponding to the reservation label, the target IOPS corresponding to the weight label, and the target IOPS corresponding to the upper limit label respectively; ρ represents the increment from the last request reservation stage, and δ represents the increment of requests completed by other servers. In formula (3), the subscript i represents the tenant number.
[0128] In formula (3), r s 、w s , l s The values of can be the same or different, and this disclosure is not limited to this.
[0129] For example, assuming that a small IO data block is defined as an IOsize less than or equal to 8KB, and based on this, the large IO data block is normalized. Then, a large IO data block of 1Mb has 128 times the IOPS of the small IO data block, and the value of n is 128.
[0130] 2. Bandwidth: The bandwidth is processed based on the requested data block size, and the clock label is calculated based on the requested data block size and QoS parameters. The clock label optimized based on bandwidth can be obtained by the following formula (4):
[0131]
[0132] Among them, size indicates the requested data block size, Rb r , Pb r , Lb r Respectively represent the reserved label, weight label and upper limit label based on bandwidth optimization; r b 、w b , l b They represent the target bandwidth corresponding to the reservation label, the target bandwidth corresponding to the weight label, and the target bandwidth corresponding to the upper limit label respectively; ρ represents the increment from the last request reservation stage, and δ represents the increment of requests completed by other servers. The subscript i in formula (4) represents the tenant number.
[0133] In formula (4), r b 、w b , l b The values of can be the same or different, and this disclosure is not limited to this.
[0134] By normalizing the IOPS required for large and small data blocks and optimizing the clock labels based on the normalized IOPS or optimizing the clock labels based on the bandwidth, workload adaptation can be achieved to ensure scheduling fairness and service quality between different tenants and within the tenant itself.
[0135] Above Figures 1 to 5 In the illustrated embodiment, when generating clock tags for the head request, updated head request, and newly added request of each request queue, it can be implemented by using either formula (4) or formula (5).
[0136] In a specific embodiment, when the scheduler runs in polling mode, the performance overhead obtained by testing is shown in the following Table 1:
[0137] Table 1
[0138]
[0139] Among them, effective dequeue means that the current dispatch gets the request. P50 means the 50th percentile delay, P99 and P999 mean the 99th and 99.9th percentile delays respectively.
[0140] Combined with the data shown in Table 1, it can be seen that the scheduling method based on service quality resources provided by the present invention can achieve a better effect of limiting the height and guaranteeing the bottom line in a multi-tenant mode according to the configurations such as reserved quotas, weight quotas, and upper limit quotas set for different tenants. It can also achieve scheduling by weight, and requests of different sizes can also be adaptive, taking into account response delay and scheduling fairness.
[0141] Figure 6 This is a schematic diagram of the structure of a scheduling device based on quality of service resources provided by an embodiment of the present disclosure. Figure 6 As shown, the service quality resource based 600 provided in this embodiment includes:
[0142] The acquisition module 601 is used to acquire the request queues corresponding to each tenant respectively; wherein the client is used to carry one or more tenants, and the server is used to maintain the request queues of different tenants carried by the client.
[0143] A clock label processing module 602 is used to generate a clock label for the head request of the request queue corresponding to each tenant; the clock label includes a reserved label, a weight label and an upper limit label;
[0144] The binary heap processing module 603 is used to create a reserved binary heap, a weighted binary heap and an upper limit binary heap based on the clock labels of the head requests in the request queues corresponding to each of the tenants; the clock label value corresponding to each tree node in the reserved binary heap is the reserved label value of the corresponding head request, the clock label value corresponding to each tree node in the weighted binary heap is the weighted label value of the corresponding head request, and the clock label value corresponding to each tree node in the upper limit binary heap is the upper limit label value of the corresponding head request.
[0145] The scheduling module 604 is used to schedule the request queue corresponding to the target tenant to dequeue the head request based on the clock label values corresponding to each tree node in the reserved binary heap, the weighted binary heap and the upper limit binary heap, and update the next request to be the head request.
[0146] The clock tag processing module 602 is further configured to generate a clock tag for the updated head-of-queue request.
[0147] In some embodiments, the binary heap processing module 603 is further used to adjust the positions of the tree nodes corresponding to the target tenant in the reserved binary heap, the weighted binary heap and the upper limit binary heap respectively based on the clock label of the updated head-of-queue request.
[0148] In some embodiments, the acquisition module 601 is also used to receive a new request sent by a tenant, and if the request queue corresponding to the tenant is empty, the new request is inserted into the request queue corresponding to the tenant, and a clock label is generated for the new request based on the arrival time of the new request; if the request queue corresponding to the tenant is not empty, the new request is inserted at the end of the request queue corresponding to the tenant.
[0149] In some embodiments, the binary heap processing module 603 is further used to adjust the positions of the tree nodes corresponding to the tenant in the reserved binary heap, the weighted binary heap and the upper limit binary heap respectively based on the clock label corresponding to the new request.
[0150] In some embodiments, the binary heap processing module 603 is also used to obtain the weight label value of the root node of the weighted binary heap; and adjust the position of the tree node corresponding to the tenant to which the new request belongs in the weighted binary heap according to the time difference between the weight label value of the root node and the current time.
[0151] In some embodiments, the clock tag processing module 602 is specifically used to perform normalization processing based on the data block size requested by the corresponding head-of-team request to obtain the normalized read and write times per second corresponding to the head-of-team request; and to generate a clock tag for the head-of-team request based on the normalized IOPS, target IOPS and current timestamp corresponding to the head-of-team request.
[0152] In some embodiments, the clock tag processing module 602 is specifically configured to generate a clock tag of the head-of-line request according to the data block size, target bandwidth, and current timestamp requested by the head-of-line request.
[0153] In some embodiments, the clock tag processing module 602 is specifically configured to generate a clock tag based on a physical clock timestamp.
[0154] In some embodiments, the clock tag is represented using a signed integer data type.
[0155] The scheduling device based on quality of service resources provided in this embodiment can be used to implement the technical solution of any of the aforementioned method embodiments. Its implementation principles and technical effects are similar. Please refer to the detailed description of the aforementioned method embodiments. For the sake of brevity, they will not be repeated here.
[0156] Exemplarily, the present disclosure provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; when the one or more processors execute the one or more computer programs, the electronic device implements the scheduling method based on quality of service resources of the foregoing embodiments.
[0157] Exemplarily, the present disclosure provides a chip system, which is applied to an electronic device including a memory and a sensor; the chip system includes: a processor; when the processor executes the scheduling method based on service quality resources of the above embodiment.
[0158] Exemplarily, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor in an electronic device, the scheduling method based on quality of service resources of the above embodiment is implemented.
[0159] Exemplarily, the present disclosure provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the scheduling method based on quality of service resources of the above embodiment.
[0160] In the above embodiments, all or part of the functions can be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. Available media can be magnetic media, (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid state disks (SSDs)), etc.
[0161] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0162] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A scheduling method based on quality of service resources, It is characterized in that include: Obtain the request queues corresponding to the tenants respectively, and generate a clock tag for the head request of the request queues corresponding to the tenants respectively; The clock label includes a reservation label, a weight label and an upper limit label; wherein the client is used to carry one or more tenants, and the server is used to maintain the request queues of different tenants carried by the client; Based on the clock labels of the first requests in the request queues corresponding to the tenants, a reserved binary heap, a weighted binary heap and an upper limit binary heap are created respectively; The clock label value corresponding to each tree node in the reserved binary heap is the reserved label value requested by the corresponding team leader, the clock label value corresponding to each tree node in the weighted binary heap is the weighted label value requested by the corresponding team leader, and the clock label value corresponding to each tree node in the upper limit binary heap is the upper limit label value requested by the corresponding team leader. Based on the clock label values corresponding to each tree node in the reserved binary heap, the weighted binary heap and the upper limit binary heap, the request queue corresponding to the target tenant is scheduled to dequeue the head request, and the next request is updated to be the head request and a clock label is generated for the updated head request.
2. The method according to claim 1, It is characterized in that Also includes: Based on the updated clock tag of the head-of-queue request, the positions of the target tenant in the tree nodes corresponding to the reserved binary heap, the weighted binary heap and the upper limit binary heap are adjusted respectively.
3. The method according to claim 1, It is characterized in that Also includes: Receive new requests sent by tenants; If the request queue corresponding to the tenant is empty, inserting the newly added request into the request queue corresponding to the tenant, and generating a clock tag for the newly added request based on the arrival time of the newly added request; If the request queue corresponding to the tenant is not empty, the newly added request is inserted into the tail of the request queue corresponding to the tenant.
4. The method according to claim 3, It is characterized in that Also includes: Based on the clock label corresponding to the newly added request, the positions of the tree nodes corresponding to the tenant in the reserved binary heap, the weighted binary heap and the upper limit binary heap are adjusted respectively.
5. The method according to claim 4, It is characterized in that Also includes: Obtaining the weight label value of the root node of the weighted binary heap; According to the time difference between the weight label value of the root node and the current time, the position of the tree node corresponding to the tenant to which the newly added request belongs in the weighted binary heap is adjusted.
6. The method according to claim 1, It is characterized in that The generating clock label includes: Performing normalization processing based on the data block size requested by the corresponding head of team request to obtain the normalized number of reads and writes per second corresponding to the head of team request; A clock tag for the head-of-team request is generated according to the normalized number of reads and writes per second, the target number of reads and writes per second, and the current timestamp corresponding to the head-of-team request.
7. The method according to claim 1, It is characterized in that The generating clock label includes: A clock tag of the head-of-team request is generated according to the data block size, target bandwidth and current timestamp requested by the head-of-team request.
8. The method according to claim 1, It is characterized in that The generating the clock tag includes: generating the clock tag based on the physical clock timestamp.
9. The method according to claim 1, It is characterized in that The clock label is represented by a signed integer data type.
10. A scheduling device based on quality of service resources, It is characterized in that include: An acquisition module is used to acquire request queues corresponding to each tenant; wherein the client is used to carry one or more tenants, and the server is used to maintain request queues of different tenants carried by the client; A clock label processing module, configured to generate a clock label for the head request of the request queue corresponding to each of the tenants; the clock label includes a reserved label, a weight label and an upper limit label; A binary heap processing module is used to create a reserved binary heap, a weighted binary heap and an upper limit binary heap based on the clock labels of the head requests in the request queues corresponding to the tenants; the clock label value corresponding to each tree node in the reserved binary heap is the reserved label value of the corresponding head request, the clock label value corresponding to each tree node in the weighted binary heap is the weighted label value of the corresponding head request, and the clock label value corresponding to each tree node in the upper limit binary heap is the upper limit label value of the corresponding head request; A scheduling module, configured to schedule the request queue corresponding to the target tenant to dequeue the head request based on the clock label values corresponding to each tree node in the reserved binary heap, the weighted binary heap, and the upper limit binary heap, and update the next request to be the head request; The clock tag processing module is also used to generate a clock tag for the updated head-of-queue request.
11. An electronic device, It is characterized in that include: Memory and processor; The memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions so that the electronic device implements the scheduling method based on quality of service resources as described in any one of claims 1 to 9.
12. A readable storage medium, It is characterized in that include: Computer program instructions; At least one processor of the electronic device executes the computer program instructions, so that the electronic device implements the scheduling method based on quality of service resources as described in any one of claims 1 to 9.
13. A computer program product, It is characterized in that The electronic device runs the computer program product so that the electronic device implements the scheduling method based on quality of service resources as described in any one of claims 1 to 9.
