I / O request scheduling method and device, electronic equipment and storage medium

Abstract

The invention discloses an I / O request scheduling method and device, electronic equipment and a storage medium, and belongs to the technical field of computers. The I / O request scheduling method comprises the following steps: acquiring state information of all hard disk pools, and determining an I / O type priority mapping table corresponding to each hard disk pool according to the state information; wherein the I / O type priority mapping table stores a corresponding relationship between the type and the priority of the I / O request; determining the priority of the to-be-processed I / O request corresponding to each hard disk pool according to the I / O type priority mapping table; and enqueueing the I / O requests to be processed to the corresponding priority queues according to the priorities. The resources of the storage system can be balanced, and the scheduling requirements of different types of I / O requests can be met.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an I / O request scheduling method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of cloud computing and virtualization technology, traditional computer hardware resources are incorporated into resource pools for pool management to provide customized computing and storage services. In the append-write distributed storage system, user data is written to the hard disk pool in an appended manner and cannot be modified. When the user modifies the data, the old data becomes invalid data and requires background tasks to perform garbage collection. At the same time, distributed storage systems generally use erasure coding technology to ensure data security, allowing hard disk damage, storage node downtime and local network failure within a certain range. When a failure occurs, the system automatically uses erasure coding technology to repair the data to ensure data security. In addition, after operation and maintenance operations such as capacity expansion, the system needs to migrate the data in the old hard disk to the newly expanded hard disk to achieve data rebalancing in the storage cluster. In addition, since bad sectors will inevitably be generated during the long-term use of hard disk media, the system usually performs bad sector scanning periodically to timely discover and repair bad sectors. The above operations will generate a large number of I / O requests, which are usually processed according to the reception time of the I / O requests in related technologies, but the above method cannot meet the load requirements of critical tasks.

[0003] Therefore, how to balance the resources of the storage system and meet the scheduling requirements of different types of I / O requests is a technical problem that those skilled in the art currently need to solve. Summary of the invention

[0004] The purpose of this application is to provide an I / O request scheduling method, device, electronic device and storage medium, which can balance the resources of the storage system and meet the scheduling requirements of different types of I / O requests.

[0005] In order to solve the above technical problems, the present application provides an I / O request scheduling method, which is applied to a distributed storage system including multiple hard disk pools. The I / O request scheduling method includes:

[0006] Obtaining status information of all the hard disk pools, and determining an I / O type priority mapping table corresponding to each hard disk pool according to the status information; wherein the I / O type priority mapping table stores a correspondence between the type and priority of an I / O request;

[0007] Determine the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table;

[0008] The pending I / O requests are queued into the corresponding priority queue according to the priority.

[0009] Optionally, obtaining status information of all the hard disk pools, and determining an I / O type priority mapping table corresponding to each hard disk pool according to the status information, includes:

[0010] Obtaining status information of all the hard disk pools;

[0011] The hard disk pool status mapping table is queried according to the status information of each hard disk pool to determine the corresponding I / O type priority mapping table.

[0012] Optionally, determining the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table includes:

[0013] If a priority query request from the client is received, the I / O type priority mapping tables corresponding to all the hard disk pools are sent to the client, so that the client can determine the priority of the pending I / O request corresponding to each hard disk pool according to the I / O type priority mapping table.

[0014] Optionally, also include:

[0015] A corresponding resource allocation quota is set for each priority queue; wherein the resource allocation quota is used to describe the resource scheduling upper limit of the priority queue.

[0016] Optionally, also include:

[0017] Setting a corresponding maximum number of parallel queues for each priority queue;

[0018] Selecting one of the priority queues as a target priority queue and determining a queue set; wherein the queue set includes the target priority queue and all priority queues with a priority higher than that of the target priority queue;

[0019] Determine whether the sum of the number of pending I / O requests of all priority queues in the queue set is greater than the sum of the maximum parallel numbers; wherein the sum of the maximum parallel numbers is the sum of the maximum parallel numbers of all priority queues in the queue set;

[0020] If so, a busy signal is sent to the upper layer application, so that the upper layer application adjusts the sending strategy of the pending I / O request.

[0021] Optionally, also include:

[0022] The dequeue ratio information of each priority queue is set according to the priority of the pending I / O request, so as to schedule the pending I / O request according to the dequeue ratio information; wherein the dequeue ratio information is used to describe the dequeue ratio of the pending I / O request of each priority level in the priority queue.

[0023] Optionally, also include:

[0024] Setting a maximum number of tokens for a token bucket corresponding to each priority queue, and setting a token issuance cycle for the token bucket; wherein the tokens in the token bucket are credentials for allowing pending I / O requests to be dequeued;

[0025] Based on the token bucket algorithm, the pending I / O requests in each priority queue are controlled to be dequeued according to the dequeuing ratio information.

[0026] The present application also provides an I / O request scheduling device, which is applied to a distributed storage system including multiple hard disk pools, and the I / O request scheduling device includes:

[0027] A priority query module, used to obtain status information of all the hard disk pools, and determine an I / O type priority mapping table corresponding to each hard disk pool according to the status information; wherein the I / O type priority mapping table stores the correspondence between the type and priority of the I / O request;

[0028] A priority setting module, used to determine the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table;

[0029] The queue control module is used to queue the pending I / O requests into the corresponding priority queue according to the priority.

[0030] The present application also provides a storage medium on which a computer program is stored. When the computer program is executed, the steps of the above-mentioned I / O request scheduling method are implemented.

[0031] The present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned I / O request scheduling method when calling the computer program in the memory.

[0032] The present application provides an I / O request scheduling method, which obtains the status information of the hard disk pool, and determines the I / O type priority mapping table corresponding to each hard disk pool according to the status information of the hard disk pool. The present application determines the priority of the pending I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table, and classifies the I / O requests by priority and queues them into the corresponding priority queue. The above process can adjust the priority of the I / O request according to the status information of the hard disk pool, thereby balancing the resource utilization of the distributed storage system, so that high-priority I / O requests can be processed in a timely manner, and low-priority requests can also receive appropriate attention when resources permit. Therefore, the present application can balance the resources of the storage system and meet the scheduling requirements of different types of I / O requests. The present application also provides an I / O request scheduling device, a storage medium and an electronic device, which have the above-mentioned beneficial effects and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 A flowchart of an I / O request scheduling method provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a hard disk pool management service provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a coroutine concurrency control model provided in an embodiment of the present application;

[0037] Figure 4 A flowchart for implementing an overload back pressure mechanism provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of a front-end and back-end I / O request scheduling architecture based on an append-write distributed storage system provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of the structure of an I / O request scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] See below Figure 1 , Figure 1 A flowchart of an I / O request scheduling method provided in an embodiment of the present application.

[0042] Specific steps may include:

[0043] S101: Acquire status information of all the hard disk pools, and determine an I / O type priority mapping table corresponding to each hard disk pool according to the status information;

[0044] The present embodiment can be applied to a management node of a distributed storage system, where there are multiple hard disk pools. The present embodiment can collect status information of each hard disk pool through a monitoring tool or an API, such as the usage rate, remaining space, read / write speed, fault status, etc. of the hard disk pool.

[0045] In this embodiment, the corresponding relationship between the status information and the I / O (Input / Output) type priority mapping table can be pre-set, and the I / O type priority mapping table corresponding to each hard disk pool can be determined based on the above corresponding relationship. Specifically, in this embodiment, the I / O type priority mapping table of the hard disk pool can be set according to the status information of the hard disk pool itself.

[0046] The I / O type priority mapping table stores the correspondence between the type and priority of I / O requests. The types of I / O requests can include foreground I / O requests and background I / O requests. Foreground I / O requests include read data I / O requests and write data I / O requests. Background I / O requests include background repair I / O requests, migration I / O requests, and bad sector scanning I / O requests. Foreground I / O requests refer to: the distributed storage cluster receives read and write operations from businesses or users. Background I / O requests refer to: various read and write operations generated within the distributed storage cluster. Bad sector scanning is a technology that detects and repairs damaged areas on hard disks to improve data security and hard disk reliability.

[0047] S102: Determine the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table;

[0048] Each hard disk pool may have its corresponding pending I / O request. After the I / O type priority mapping table is determined, the priority of the pending I / O request corresponding to the hard disk pool may be determined.

[0049] S103: Enqueue the pending I / O request into a corresponding priority queue according to the priority;

[0050] Among them, according to the priority of the I / O request, this embodiment can queue the pending I / O request into the corresponding priority queue so as to process these requests in order of priority. The "queue" in this embodiment means storing the pending I / O request into the corresponding priority queue. This embodiment can create multiple priority queues in advance, and set the corresponding relationship between each priority queue and the priority of the I / O request, for example, priority queue A is used to store pending I / O requests with priorities 1 to 3, priority queue B is used to store pending I / O requests with priorities 4 to 6, and priority queue C is used to store pending I / O requests with priorities 7 to 9. The above method can effectively manage pending I / O requests of different priorities, ensure that key tasks are processed in a timely manner, and take into account the processing of other tasks.

[0051] The present embodiment provides an I / O request scheduling method, which obtains the status information of the hard disk pool, and determines the I / O type priority mapping table corresponding to each hard disk pool according to the status information of the hard disk pool. The present embodiment determines the priority of the pending I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table, and classifies the I / O requests by priority and queues them into the corresponding priority queue. The above process can adjust the priority of the I / O request according to the status information of the hard disk pool, thereby balancing the resource utilization of the distributed storage system, so that high-priority I / O requests are processed in a timely manner, and low-priority requests can also receive appropriate attention when resources permit. Therefore, the present embodiment can balance the resources of the storage system and meet the scheduling requirements of different types of I / O requests.

[0052] As for Figure 1 Further introduction to the corresponding embodiment, after executing S101~S103, the dequeue ratio information of each priority queue can also be set according to the priority of the pending I / O request, so as to schedule the pending I / O request according to the dequeue ratio information.

[0053] Among them, this embodiment can set the dequeue ratio information of the priority queue according to the priority of the pending I / O requests already existing in the priority queue. The above-mentioned dequeue ratio information is used to describe the dequeue ratio of the pending I / O requests of each priority level in the priority queue. For example, it can be defined that the dequeue ratio of high-priority requests is higher than that of low-priority requests to ensure that more resources are allocated to high-priority requests. "Dequeue" in this embodiment means: submitting the pending I / O requests in the priority queue to the corresponding hard disk pool for processing.

[0054] This embodiment can better balance resources by setting the dequeue ratio, ensuring fairness and service quality between requests of different priorities. In the above manner, pending I / O requests of different priorities can be processed in order according to their importance and urgency.

[0055] This embodiment classifies I / O requests by priority and stores them in corresponding priority queues, and sets the dequeue ratio information of each priority queue. The dequeue ratio information includes the dequeue ratio of each priority queue for the pending I / O requests. When scheduling the pending I / O requests according to the dequeue ratio information, this embodiment can make the processing I / O requests of each priority in the priority queue dequeue according to the corresponding ratio, so as to avoid important tasks being blocked by low-priority tasks, thereby improving the responsiveness and throughput of the system. By setting the dequeue ratio information, this embodiment can achieve fairness and service quality assurance between pending I / O requests of different priorities. Therefore, this embodiment can balance the resources of the storage system and meet the scheduling requirements of different types of I / O requests.

[0056] As for Figure 1 For further introduction to the corresponding embodiment, the I / O type priority mapping table corresponding to each of the hard disk pools can be set in the following manner: obtaining status information of all the hard disk pools; querying the hard disk pool status mapping table according to the status information of each of the hard disk pools, and determining the corresponding I / O type priority mapping table.

[0057] The hard disk pool status mapping table stores a mapping relationship between status information and an I / O type priority mapping table. Based on the mapping relationship in the hard disk pool status mapping table, the corresponding I / O type priority mapping table can be determined.

[0058] The above disk pool status mapping table can record and store the mapping relationship between the status information and the mapping table number (that is, the number of the I / O type priority mapping table) in the form of functions, links, linked lists, etc. The process of determining the I / O type priority mapping table based on the mapping relationship is as follows:

[0059] Step A1: Obtaining status information of all the hard disk pools;

[0060] Step A2: querying the hard disk pool status mapping table according to the status information of each hard disk pool to obtain a mapping table number;

[0061] Among them, this embodiment can pre-set a hard disk pool status mapping table, which stores the correspondence between status information and mapping table numbers, and the mapping table number is the number of the I / O type priority mapping table; the mapping table number is the unique identifier of the I / O type priority mapping table.

[0062] Step A3: Determine the corresponding I / O type priority mapping table according to the mapping table number corresponding to each hard disk pool.

[0063] In this embodiment, all I / O type priority mapping tables can be pre-configured, and the mapping table numbers of the positions of the I / O type priority mapping tables can be assigned to the I / O type priority mapping tables. In this way, the corresponding I / O type priority mapping table can be directly queried according to the mapping table number, so as to improve the efficiency of setting the I / O type priority mapping table for the hard disk pool.

[0064] As for Figure 1 Further introduction to the corresponding embodiment: the priority of the pending I / O request corresponding to each of the hard disk pools can be set in the following manner: if a priority query request from the client is received, the I / O type priority mapping table corresponding to all the hard disk pools is sent to the client, so that the client can determine the priority of the pending I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table.

[0065] Furthermore, this embodiment can also obtain the operation and maintenance configuration of the distributed storage system, and send the priority of the background I / O request to all storage nodes of each hard disk pool according to the operation and maintenance configuration. In a distributed storage system, multiple hard disks form a hard disk pool, and multiple hard disk pools can form a large storage pool; the operation and maintenance configuration of the above distributed storage system includes the hard disk pool and the storage pool operation and maintenance configuration.

[0066] Current technologies usually control garbage collection and data repair / migration I / O requests through fixed concurrency, which cannot flexibly respond to dynamically changing garbage volumes and storage system loads, resulting in the inability to maximize performance. When foreground I / O and background I / O coexist, the existing system cannot complete unified scheduling and independent control, cannot accurately control various types of I / O loads in the foreground and background, and cannot fully utilize cluster resources. In actual scenarios, I / O intensive or frequent tasks will occupy more I / O resources, and the existing system cannot effectively guarantee the performance requirements of users' critical task loads. It can be seen that the relevant technologies have problems such as insufficient flexibility, low scheduling accuracy, and inability to meet critical task load requirements in terms of I / O control. Based on the above problems, this embodiment can solve the limitations of fixed concurrency, inaccurate scheduling of foreground and background tasks, and the problems of inability to guarantee critical task loads and insufficient utilization of cluster resources by designing a new hard disk pool management service MGR and a new I / O queue scheduling algorithm.

[0067] Specifically, this embodiment designs a hard disk pool management service MGR, and defines multiple states to describe the status of the hard disk pool. By monitoring the status changes of the hard disk pool, the system can adopt corresponding scheduling strategies according to different states to optimize the allocation and utilization of I / O. This embodiment can also design a hard disk pool status mapping table to map the actual hard disk pool status to the I / O type priority mapping table. This embodiment can determine the scheduling priority of different types of I / O requests according to the status of the hard disk pool. In the above manner, I / O requests for critical tasks can be given priority to ensure the performance requirements of critical tasks. This embodiment can also design a corresponding I / O type priority mapping table to convert various types of I / O requests into loads for I / O queue scheduling, and accurately control the dequeue of various types of I / O in proportion. This can better balance the utilization of system resources and meet the needs of different types of tasks.

[0068] See also Figure 2 , Figure 2A schematic diagram of a hard disk pool management service provided in an embodiment of the present application shows a client, a hard disk pool management service MGR, hard disk pool 1, hard disk pool 2, and hard disk pool 3. MGR can regularly detect hard disk pools 1 to 3; there is a damaged data shard OSD in hard disk pool 1; hard disk pool 2 is in a full hard disk pool state and is expanding new disks; there are two damaged data shard OSDs in hard disk pool 3. The client can obtain the I / O request priority of each hard disk pool from MGR regularly. MGR (Manager) is a monitoring and management service for distributed storage clusters, responsible for cluster configuration, status monitoring and fault handling. Client CLIENT is an interface used by upper-level applications or services of the distributed storage cluster to access and operate stored data. OSD (Object Storage Device) is a storage node in a distributed storage cluster, responsible for receiving, processing and persisting data.

[0069] MGR determines the state of the hard disk pool by periodically traversing the control mode, OSD status, and capacity of each hard disk pool. These states include whether the hard disk pool is at risk of being full, whether the hard disk pool is full, whether there is a hard disk failure or host failure in the hard disk pool, etc. According to different states, the MGR module obtains the priority table corresponding to various types of I / O in the current system through the hard disk pool status mapping table. In this way, the scheduling order of different types of I / O requests can be determined according to the state of the hard disk pool to optimize the allocation and utilization of I / O.

[0070] The client CLIENT periodically queries the priority of GC (Garbage Collection) / Flush / user read / write requests from the MGR through commands, and sets different priorities for the requests sent to the P layer according to the I / O type and the disk pool number corresponding to the request. This can meet the needs of different types of tasks. The P layer (Persistence layer) refers to: the module responsible for data persistence in the distributed storage cluster, which is used to write data to persistent storage media.

[0071] Flush: A background service in a distributed storage cluster that is used to refresh data from the cache layer to the persistence layer to ensure data persistence and consistency.

[0072] GC (Garbage Collection) refers to a background service unique to the append-write distributed storage cluster, which is used to clean up invalid or expired data and release storage space.

[0073] MGR regularly sends the priority of background repair / migration / bad track scanning to all OSDs corresponding to each hard disk pool. By monitoring the status changes of the hard disk pool, the system can adopt corresponding scheduling strategies according to different statuses, centrally manage the hard disk pool status, and improve scalability and maintainability.

[0074] See Table 1:

[0075] Table 1 Disk pool status mapping table

[0076]

[0077] The following describes the disk pool status mapping principle, assuming that the disk pool has no faults:

[0078] If the disk pool has no risk of being full, the priority order in the I / O type priority mapping table is: user read / write I / O requests > repair / migration I / O requests > GC I / O requests = flush I / O requests.

[0079] If the disk pool is at risk of being full or is already full, the priority order in the I / O type priority mapping table is: foreground read / write I / O request = repair / migration I / O request = GC I / O request > flush I / O request. Pool full means that the used capacity of the disk pool reaches or exceeds the total capacity of the disk pool, and new data cannot be written at this time.

[0080] If the hard disk pool fails:

[0081] If there is a hard disk failure (single-shard failure) or a host failure, and the hard disk pool is not at risk of being full, the priority order in the I / O type priority mapping table is: user read / write I / O requests > repair / migration I / O requests > GC I / O requests = flush I / O requests.

[0082] If there is a hard disk failure (single-shard failure) or a host failure, or the hard disk pool is at risk of being full or is full, the priority order in the I / O type priority mapping table is: GC I / O request = read / write I / O request > repair / migration I / O request > flush I / O request.

[0083] If there is a hard disk failure (2-shard failure) and the hard disk pool is not at risk of being full, the priority order in the I / O type priority mapping table is: repair / migration I / O requests > user read / write I / O requests > GC I / O requests = flush I / O requests.

[0084] If a hard disk fails (2 slices fail), the hard disk pool is at risk of being full or is already full, the priority order in the I / O type priority mapping table is: repair / migration I / O requests > user read / write I / O requests = GC I / O requests > flush I / O requests.

[0085] By defining different states, this embodiment can determine the scheduling order of different types of I / O requests according to the state of the hard disk pool. In this way, I / O requests of key tasks can be processed first to ensure the performance requirements of key tasks. For example, for a hard disk pool with multiple hard disk failures, the repair priority can be set to the highest so that data repair can be completed as soon as possible to ensure data security.

[0086] See Table 2:

[0087] Table 2 I / O type priority mapping table

[0088]

[0089] In this embodiment, the I / O type priority mapping table can be designed with reference to Table 2 to convert various I / O requests into the load basis for I / O queue scheduling. Convert various I / O requests into the load for I / O queue scheduling, and accurately control the dequeue of various IOs in proportion. By defining the priorities of different types of I / O requests, the system can schedule I / O queues according to the priorities of different types of I / O requests. This can better balance the utilization of system resources and meet the needs of different types of tasks. For example, in EC4+2. When the hard disk fails (2-shard failure), there is a risk of data loss. The repair I / O request is set to a high priority to quickly complete the repair task and ensure data security. EC (Erasure Code): It is an efficient data redundancy encoding technology that achieves data reliability and fault tolerance by adding redundant information. The design of the I / O type priority mapping table can dynamically adjust the priority of foreground and background tasks. Different priorities have different concurrent scheduling controls in the hard disk service, and ultimately achieve the effect of dynamically adjusting the concurrency of foreground and background tasks.

[0090] This embodiment defines multiple states to describe the status of the hard disk pool, so that the status and health of the hard disk pool can be more comprehensively understood. In addition to the mentioned states such as pool full risk, whether the hard disk pool is full, whether there is a hard disk failure or host failure in the hard disk pool, other states can also be considered, such as the load of the hard disk pool, the available capacity of the hard disk pool, etc. In this way, the performance and reliability of the hard disk pool can be evaluated more accurately, and corresponding scheduling strategies can be adopted according to different states.

[0091] When designing the I / O type priority mapping table, this embodiment can determine the priority according to the importance and urgency of different types of I / O requests. For example, for the I / O request of a critical task, its priority can be set to the highest to ensure the performance requirements of the critical task. For the I / O request of a common task, its priority can be set to medium or low to better balance the utilization of system resources. By accurately scheduling different types of I / O requests, the system can better meet the needs of different tasks.

[0092] As for Figure 1 As a further introduction to the corresponding embodiment, it is also possible to: set a corresponding resource allocation quota for each priority queue; wherein the resource allocation quota is used to describe the resource scheduling upper limit of the priority queue.

[0093] The resources allocated to the priority queues mentioned above may be coroutine resources, and a coroutine is an abstract resource block allocated by the user on a thread. The number of coroutines is fixed, and the scheduler can determine the upper limit of the number of coroutines for each priority queue, but it does not limit certain coroutines to be fixedly allocated to a certain priority queue; all coroutines respond to task scheduling according to task requirements. The purpose of introducing coroutines is to control the number of concurrent low- and medium-priority I / Os, and to ensure that more than 50% of high-priority I / Os are always available for coroutines. This embodiment can achieve coroutine concurrency control in the following ways:

[0094] Step B1: Create multiple coroutines;

[0095] Step B2: setting a corresponding coroutine resource allocation quota for each priority queue; wherein the coroutine resource allocation quota is used to describe the coroutine resource scheduling upper limit of the priority queue;

[0096] Step B3: Allocate a corresponding number of coroutines to each of the priority queues based on the coroutine resource allocation quota.

[0097] Specifically, in this embodiment, the priority queue can be divided into three levels: high, medium, and low, and the coroutine resources can be divided into three resource pools: an exclusive resource pool, a shared resource pool, and a minimum resource pool.

[0098] This embodiment may provide that: the high-priority queue can schedule the exclusive resource pool and the shared resource pool, the medium-priority queue can schedule the shared resource pool, and the low-priority queue can schedule the minimum resource pool. Therefore, the resource scheduling upper limit of the low-priority queue is equal to the proportion of the minimum resource pool to all coroutine resources (for example, 25%), the resource scheduling upper limit of the medium-priority queue is equal to the proportion of the shared resource pool to all coroutine resources (for example, 50%), and the resource scheduling upper limit of the high-priority queue is equal to the proportion of the exclusive resource pool plus the shared resource pool to all coroutine resources (for example, 100%).

[0099] On this basis, this embodiment can implement the following exclusive mechanism: when IO task A occupies coroutine 1, until IO task A is executed and the above coroutine 1 is released, other IO tasks can no longer use the above coroutine, even if the priority of other IO tasks is higher than IO task A. The exclusive mechanism can be implemented by adding an exclusive lock to coroutine 1. Based on the exclusive mechanism, it may happen that the low-priority queue occupies 25% of the resources, and the medium-priority queue can only occupy a maximum of 25% of the resources. Through resource pool configuration, the high-priority queue can always ensure that more than 50% of the resources are available.

[0100] Furthermore, this embodiment can also set the maximum number of tokens for the token bucket corresponding to each priority queue, and set the token issuance cycle of the token bucket; based on the token bucket algorithm, control the dequeueing of pending I / O requests in each priority queue according to the dequeue ratio information.

[0101] See also Figure 3 , Figure 3 A schematic diagram of a coroutine concurrency control model provided in an embodiment of the present application, showing coroutines 1 to 3 and priority queues 1 to 3. Each priority queue can periodically replenish the token corresponding to the priority, and tasks can be taken and suspended from the priority queue. Coroutine CO (Coroutine): is a lightweight concurrent programming model for efficient task scheduling and collaboration.

[0102] In this embodiment, priority queues can be created according to the priorities of all I / O requests. When different IO requests are queued, they enter corresponding queues according to the priorities carried by the CLIENT. A fixed number of coroutines are created and numbered, and all priority queues are divided into three intervals within the scheduler.

[0103] The first interval (high priority 1, high priority 2, high priority 3) is allocated M coroutines (accounting for 100%). The second interval (medium priority 4, medium priority 5, medium priority 6) is allocated M / 2 coroutines (accounting for 50%, when there are already two low priority tasks being executed, only M / 4 medium priority tasks can be executed). The third interval (low priority 7, low priority 8, low priority 9) is allocated M / 4 coroutines (accounting for 25%).

[0104] The corresponding queue ratios for priorities 1 to 9 (from high to low) are: 100:80:60:50:30:20:10:5:1.

[0105] Priority queues have to meet the dequeue ratio while also having a maximum token number limit. Different priorities have different numbers of tokens, and tokens are issued every 100ms. If the number of tokens cannot meet the requirement, they cannot be dequeued and need to wait for the next round of token replenishment. In summary, the hard disk can accurately measure the load every 100ms. When the persistence layer receives an I / O request, it will put the I / O request into the corresponding priority queue. Different priority queues have token and task number controls, and they are dequeued in proportion to the priority. Before dequeuing, it is determined whether the token is satisfied. Only when it is satisfied can the I / O request be executed.

[0106] As for Figure 1 For further description of the corresponding embodiment, overload back pressure can also be achieved in the following ways:

[0107] Step C1: setting a corresponding maximum parallel quantity for each priority queue;

[0108] Step C2: selecting one of the priority queues as a target priority queue, and determining a queue set; wherein the queue set includes the target priority queue and all priority queues having a higher priority than the target priority queue;

[0109] Step C3: Determine whether the sum of the number of pending I / O requests of all priority queues in the queue set is greater than the sum of the maximum number of parallel requests; if so, proceed to step C4 to perform operations corresponding to the back pressure mechanism; if not, proceed to step C3 again after a certain delay;

[0110] The sum of the maximum parallel quantities is the sum of the maximum parallel quantities of all priority queues in the queue set.

[0111] Step C4: Sending a busy signal to the upper layer application so that the upper layer application adjusts the sending strategy of the pending I / O request;

[0112] Wherein, step C4 sends a busy signal BUSY to the upper layer application (also called upper layer logic), indicating that the current distributed storage system is in a busy state and cannot process more pending I / O requests.

[0113] After receiving the busy signal, the upper-layer application needs to adjust the delivery strategy of the pending I / O requests. The upper-layer application adjusts the delivery strategy of the pending I / O requests in the following ways, but is not limited to:

[0114] Method 1: The upper-layer application tries to send the pending I / O request again after a delay.

[0115] Method 2: The upper-layer application reduces the frequency of sending pending I / O requests.

[0116] See also Figure 4 , Figure 4 A flowchart for implementing an overload back pressure mechanism provided by an embodiment of the present application, after receiving an I / O request, obtain the priority of the I / O request, I / O requests of priority 1 to 3 are stored in a high priority queue, I / O requests of priority 4 to 6 are stored in a medium priority queue, and I / O requests of priority 7 to 9 are stored in a low priority queue. Determine whether the number of queues in the high priority queue exceeds 64. If so, feedback a BUSY message to allow the client to retry sending the I / O request. If not, determine whether the memory exceeds 128M. Determine whether the total number of queues in the high priority queue and the medium priority queue exceeds 92. If so, feedback a BUSY message to allow the client to retry sending the I / O request. If not, determine whether the memory exceeds 128M. Determine whether the total number of queues in the high priority queue, the medium priority queue, and the low priority queue exceeds 172. If so, feedback a BUSY message to allow the client to retry sending the I / O request. If not, determine whether the memory exceeds 128M. If the memory exceeds 128M, a BUSY message is fed back to enable the client to retry sending the I / O request; if the memory does not exceed 128M, an enqueue operation is performed to obtain a coroutine task queue for I / O request scheduling. This embodiment provides a new coroutine concurrency control model and back pressure mechanism to achieve accurate scheduling of foreground and background tasks and efficient use of hard disk I / O capabilities. The system can dynamically adjust the concurrency of foreground and background tasks according to the status of the hard disk pool and the priority of the I / O type, avoid resource competition and starvation, and improve the response speed and throughput of the system. At the same time, the back pressure mechanism introduced in this embodiment can protect the hard disk service from overload.

[0117] For high-priority I / O requests (priority 1~3), a maximum of 64 can be processed in parallel. When there are more than 64 high-priority tasks in the queue, a BUSY message is fed back to the upper-level logic to enable the client to retry.

[0118] For medium-priority I / O requests (priority 4 to 6), when there are more than 92 tasks in the queue of high-priority tasks + medium-priority tasks, a BUSY message is fed back to the upper-level logic to enable the client to retry;

[0119] For low-priority I / O requests (priority 7 to 9), when there are more than 172 high + medium + low priority tasks queued, a BUSY message is fed back to the upper-level logic to enable the client to retry.

[0120] This embodiment can implement memory back pressure in the following way: for read / write IO (OSD service), a maximum of 128 MB of memory is allocated, and repair / migration can use a maximum of 1 / 2 of it and a minimum of 1 / 8 of it. When the foreground IO occupies more than 128 MB, repair / migration can apply for a maximum of 16 MB of memory.

[0121] This embodiment uses a coroutine concurrency control model to improve the concurrent processing capability and resource utilization of the system. By dynamically adjusting the concurrency of foreground and background tasks, resource competition and starvation can be avoided, and the response speed and throughput of the system can be improved. At the same time, the introduction of a back pressure mechanism can protect the hard disk service from overload. When the load of a single hard disk service is too high, limiting the submission rate of tasks can effectively control the load and avoid performance degradation or failure of the hard disk service. This can improve the stability and reliability of the system.

[0122] See below Figure 5 , Figure 5 A schematic diagram of a front-end and back-end I / O request scheduling architecture based on an append-write distributed storage system provided in an embodiment of the present application; the cache layer CACHE-TIER includes multiple OCSs. OCS (Object Cache Storage) is a global data cache service for a distributed storage cluster, used to improve data access speed and performance. The cache can receive user read I / O requests and user write I / O requests, and kv represents a distributed database. libappend-only indicates that the index layer provides a dynamic library. libkv-client represents a client dynamic library provided to the outside by a distributed database. INDEX (Index Service) is a global metadata index layer of a distributed storage cluster, used to quickly locate and retrieve storage objects. INDEX includes read I / O requests, write I / O requests, garbage collection I / O requests and libpclient, and libpclient represents a client provided to the outside by a computer. The hard disk pool management service MGR includes a hard disk status management service, a bad track scanning service, and an I / O request priority mapping table. The hard disk pool management service MGR can obtain the priority of an I / O request. PERSISTENCE represents the P layer (Persistence layer), including storage nodes OSD in multiple distributed storage clusters. EC 2+2 indicates an erasure code configuration, that is, there are 2 data blocks and 2 check blocks. INDEX can transmit I / O requests to and from the P layer.

[0123] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an I / O request scheduling device provided in an embodiment of the present application. The system can be applied to a distributed storage system including multiple hard disk pools. The I / O request scheduling device includes:

[0124] The priority query module 601 is used to obtain the status information of all the hard disk pools, and determine the I / O type priority mapping table corresponding to each hard disk pool according to the status information; wherein the I / O type priority mapping table stores the correspondence between the type and priority of the I / O request;

[0125] A priority setting module 602, configured to determine the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table;

[0126] The queue control module 603 is used to queue the pending I / O requests into the corresponding priority queue according to the priority.

[0127] The present embodiment provides an I / O request scheduling method, which obtains the status information of the hard disk pool, and determines the I / O type priority mapping table corresponding to each hard disk pool according to the status information of the hard disk pool. The present embodiment determines the priority of the pending I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table, and classifies the I / O requests by priority and queues them into the corresponding priority queue. The above process can adjust the priority of the I / O request according to the status information of the hard disk pool, thereby balancing the resource utilization of the distributed storage system, so that high-priority I / O requests are processed in a timely manner, and low-priority requests can also receive appropriate attention when resources permit. Therefore, the present embodiment can balance the resources of the storage system and meet the scheduling requirements of different types of I / O requests.

[0128] Furthermore, the priority query module 601 obtains the status information of all the hard disk pools, and determines the I / O type priority mapping table corresponding to each hard disk pool based on the status information. The process includes: obtaining the status information of all the hard disk pools; querying the hard disk pool status mapping table based on the status information of each hard disk pool to determine the corresponding I / O type priority mapping table.

[0129] Furthermore, the process by which the priority setting module 602 determines the priority of each pending I / O request corresponding to the hard disk pool according to the I / O type priority mapping table includes: if a priority query request from the client is received, the I / O type priority mapping tables corresponding to all the hard disk pools are sent to the client, so that the client determines the priority of each pending I / O request corresponding to the hard disk pool according to the I / O type priority mapping table.

[0130] Furthermore, it also includes:

[0131] The resource allocation module is used to set a corresponding resource allocation quota for each priority queue; wherein the resource allocation quota is used to describe the resource scheduling upper limit of the priority queue.

[0132] Furthermore, it also includes:

[0133] A concurrency control module is used to set a corresponding maximum parallel number for each priority queue; it is also used to select one of the priority queues as a target priority queue and determine a queue set; wherein the queue set includes the target priority queue and all priority queues with a priority higher than the target priority queue; it is also used to determine whether the sum of the number of pending I / O requests of all priority queues in the queue set is greater than the sum of the maximum parallel numbers; wherein the sum of the maximum parallel numbers is the sum of the maximum parallel numbers of all priority queues in the queue set; if so, a busy signal is sent to an upper-layer application to enable the upper-layer application to adjust the sending strategy of the pending I / O requests.

[0134] Furthermore, it also includes:

[0135] A dequeue control module is used to set the dequeue ratio information of each priority queue according to the priority of the pending I / O request, so as to schedule the pending I / O request according to the dequeue ratio information; wherein the dequeue ratio information is used to describe the dequeue ratio of the pending I / O request of each priority in the priority queue.

[0136] Furthermore, it also includes:

[0137] A token control module is used to set the maximum number of tokens for the token bucket corresponding to each priority queue and to set the token issuance cycle of the token bucket; wherein the tokens in the token bucket are credentials for allowing pending I / O requests to be dequeued; and is also used to control the dequeuing of pending I / O requests in each priority queue according to the dequeuing ratio information based on the token bucket algorithm.

[0138] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0139] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0140] The present application also provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.

[0141] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of this application.

[0142] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. An I / O request scheduling method, characterized in that: Applied to a distributed storage system including multiple hard disk pools, the I / O request scheduling method includes: Obtaining status information of all the hard disk pools, and determining an I / O type priority mapping table corresponding to each hard disk pool according to the status information; wherein the I / O type priority mapping table stores a correspondence between the type and priority of an I / O request; Determine the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table; The pending I / O requests are queued into the corresponding priority queue according to the priority.

2. The I / O request scheduling method according to claim 1, characterized in that: Acquiring status information of all the hard disk pools, and determining an I / O type priority mapping table corresponding to each hard disk pool according to the status information, including: Obtaining status information of all the hard disk pools; The hard disk pool status mapping table is queried according to the status information of each hard disk pool to determine the corresponding I / O type priority mapping table.

3. The I / O request scheduling method according to claim 1, characterized in that: Determining the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table includes: If a priority query request from the client is received, the I / O type priority mapping tables corresponding to all the hard disk pools are sent to the client, so that the client can determine the priority of the pending I / O request corresponding to each hard disk pool according to the I / O type priority mapping table.

4. The I / O request scheduling method according to claim 1, characterized in that: Also includes: A corresponding resource allocation quota is set for each priority queue; wherein the resource allocation quota is used to describe the resource scheduling upper limit of the priority queue.

5. The I / O request scheduling method according to claim 1, characterized in that: Also includes: Setting a corresponding maximum number of parallel queues for each priority queue; Selecting one of the priority queues as a target priority queue and determining a queue set; wherein the queue set includes the target priority queue and all priority queues with a priority higher than that of the target priority queue; Determine whether the sum of the number of pending I / O requests of all priority queues in the queue set is greater than the sum of the maximum parallel numbers; wherein the sum of the maximum parallel numbers is the sum of the maximum parallel numbers of all priority queues in the queue set; If so, a busy signal is sent to the upper layer application, so that the upper layer application adjusts the sending strategy of the pending I / O request.

6. The I / O request scheduling method according to any one of claims 1 to 5, characterized in that: Also includes: The dequeue ratio information of each priority queue is set according to the priority of the pending I / O request, so as to schedule the pending I / O request according to the dequeue ratio information; wherein the dequeue ratio information is used to describe the dequeue ratio of the pending I / O request of each priority level in the priority queue.

7. The I / O request scheduling method according to any one of claim 6, characterized in that: Also includes: Setting a maximum number of tokens for a token bucket corresponding to each priority queue, and setting a token issuance cycle for the token bucket; wherein the tokens in the token bucket are credentials for allowing pending I / O requests to be dequeued; Based on the token bucket algorithm, the pending I / O requests in each priority queue are controlled to be dequeued according to the dequeuing ratio information.

8. An I / O request scheduling device, characterized in that: Applied to a distributed storage system including multiple hard disk pools, the I / O request scheduling device includes: A priority query module, used to obtain status information of all the hard disk pools, and determine an I / O type priority mapping table corresponding to each hard disk pool according to the status information; wherein the I / O type priority mapping table stores the correspondence between the type and priority of the I / O request; A priority setting module, used to determine the priority of the to-be-processed I / O request corresponding to each of the hard disk pools according to the I / O type priority mapping table; The queue control module is used to queue the pending I / O requests into the corresponding priority queue according to the priority.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the I / O request scheduling method according to any one of claims 1 to 7 when calling the computer program in the memory.

10. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the I / O request scheduling method according to any one of claims 1 to 7 are implemented.

Images