RAID-IO scheduling method and device, computer equipment and storage medium

Abstract

The invention discloses an RAID-IO scheduling method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the timing scheduling of the garbage collection process of each storage node in a redundant array of independent disks, controlling the garbage collection periods of each storage node to be staggered, and obtaining the garbage collection state of each storage node; receiving an IO request and acquiring the priority of the IO request; and according to the garbage collection state of each storage node and the priority of the IO request, allocating the IO request to one or more storage nodes in a non-garbage collection state for processing. According to the method, the garbage collection process of each storage node is scheduled in an off-peak manner, the storage nodes in a non-garbage collection state are dynamically allocated in combination with IO priorities, IO paths and access sequences are optimized, the problems of path conflicts and addressing delay are relieved, the tail delay under mixed read-write loads is remarkably reduced, the throughput of the storage system is improved, and then the QoS performance is comprehensively improved.

Description

Technical Field

[0001] This invention relates to the field of NVME data transmission technology, and in particular to RAID-IO scheduling methods, apparatus, computer equipment, and storage media. Background Technology

[0002] In NVMe storage systems, RAID (Redundant Array of Independent Disks) improves performance and fault tolerance by combining multiple disks, and is widely used in data centers, databases, and other scenarios with high storage performance requirements. As SSDs (Solid State Drives) gradually replace HDDs (Hard Disk Drives) as the mainstream storage medium, the demand for high throughput and low latency in SSD-based RAID systems is becoming more urgent.

[0003] However, the storage characteristics of SSDs bring key challenges. For example, they need to be written in "pages" and erased in "blocks". The garbage collection (GC) mechanism needs to clean up invalid data blocks to free up storage space. In traditional RAID scheduling, the GC of each node lacks coordination and is prone to competing for resources with host IO requests, resulting in system lag. At the same time, traditional RAID scheduling algorithms have inherent defects. There is no dynamic optimization of IO path allocation and disk access sequence, which can easily cause path conflicts and addressing delays. Existing fixed priority scheduling, static striping and other solutions are difficult to balance bursty IOPS (I / O operations per second) and low latency stability. Under mixed read and write loads, the tail latency increases sharply and the QoS (Quality of Service) of high-priority IO cannot be guaranteed.

[0004] Therefore, optimizing RAID-IO scheduling strategies, resolving resource contention between GC and IO, alleviating the performance bottleneck of traditional algorithms, and achieving a synergistic improvement in high throughput and low latency have become urgent technical problems to be solved in RAID-based systems. Summary of the Invention

[0005] This invention provides a RAID-IO scheduling method, apparatus, computer equipment, and storage medium to solve the technical problem that resource contention between GC and IO causes a sharp increase in throughput at the beginning and end of latency.

[0006] Firstly, a RAID-IO scheduling method is provided, including: The garbage collection process of each storage node in the independent disk redundant array is scheduled on a timer to control the garbage collection cycle of each storage node to be staggered, and the garbage collection status of each storage node is obtained. Receive IO requests and obtain the priority of the IO requests; Based on the garbage collection status of each storage node and the priority of the IO request, the IO request is assigned to one or more storage nodes that are not in a garbage collection state for processing.

[0007] Secondly, a RAID-IO scheduling device is provided, including: The timing module is used to schedule the garbage collection process of each storage node in the independent disk redundant array, control the garbage collection cycle of each storage node to be staggered, and obtain the garbage collection status of each storage node. A receiving module is used to receive IO requests and obtain the priority of the IO requests; The processing module is used to allocate the IO request to one or more storage nodes that are not in a garbage collection state for processing, based on the garbage collection status of each storage node and the priority of the IO request.

[0008] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the RAID-IO scheduling method described above.

[0009] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the RAID-IO scheduling method described above.

[0010] The beneficial effects of this invention compared with the prior art are as follows: This invention avoids resource contention between storage nodes and IO requests by staggering the garbage collection process of each storage node, thus effectively reducing system lag; and by dynamically allocating non-garbage-collected storage nodes according to IO priority, it optimizes IO paths and access sequences, alleviates path conflicts and addressing latency issues, balances bursty IOPS with low latency stability, significantly reduces tail latency under mixed read and write loads, improves storage system throughput, and thus comprehensively improves QoS performance, adapting to the high-performance requirements of data centers, databases and other scenarios.

[0011] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are described in detail below. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a RAID-IO scheduling method in one embodiment of the present invention; Figure 2 yes Figure 1 A schematic diagram of a specific implementation method for step S10; Figure 3 yes Figure 1 A schematic diagram of a specific implementation method for step S20; Figure 4 yes Figure 1 A schematic diagram of a specific implementation method for step S30; Figure 5 yes Figure 4 A flowchart illustrating a specific implementation of step S32; Figure 6 This is a schematic diagram of the garbage collection execution cycle of the RAID-IO scheduling method in one embodiment of the present invention; Figure 7 This is a schematic diagram of the periodic checking process of the RAID-IO scheduling method in one embodiment of the present invention; Figure 8 This is a schematic diagram of the parity information distribution of the RAID-IO scheduling method in one embodiment of the present invention; Figure 9 This is a schematic diagram of a RAID-IO scheduling device in one embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 11 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0014] It should be understood that, when used in this specification and the appended claims, the terms “comprising” and “including” indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0016] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0017] Please see Figure 1 As shown, Figure 1 This is a schematic flowchart illustrating a RAID-IO scheduling method provided in an embodiment of the present invention. The RAID-IO scheduling method includes the following steps: S10: Schedule the garbage collection process of each storage node in the independent disk redundant array at regular intervals, control the garbage collection cycle of each storage node to be staggered, and obtain the garbage collection status of each storage node.

[0018] Step S10 is the core foundation for garbage collection and IO request resource contention in the independent disk redundant array scheduling. By scheduling the garbage collection of each storage node to stagger the peak execution, it ensures that most nodes are in a non-garbage collection state at the same time, reserving sufficient response resources for IO requests, reducing system lag from the root, and providing a guarantee for high throughput and low latency. At the same time, obtaining the garbage collection status provides a basis for decision-making on subsequent IO path allocation.

[0019] Among them, Redundant Array of Independent Disks (RAID) is a logical storage unit composed of multiple storage nodes, used to improve storage performance and fault tolerance; Garbage collection (GC) is the mechanism in SSDs to reorganize invalid data blocks and merge valid data pages to release writable space; Storage node, in this embodiment, is the SSD device in the RAID system, which is the core unit for data storage and IO response.

[0020] In some embodiments of the present invention, such as Figure 2 As shown, a specific timed scheduling scheme is provided. In S10, the garbage collection process of each storage node in the independent disk redundant array is scheduled in a timed manner, the garbage collection cycle of each storage node is staggered, and the garbage collection status of each storage node is obtained. Specifically, it includes the following steps S11-S14.

[0021] S11: By using the timer in the independent disk redundant array controller, a fixed garbage collection execution cycle and a cycle time window are set for each storage node; wherein, the duration of the cycle time window is T, the length of the garbage collection execution cycle is N×T, and N is the total number of storage nodes.

[0022] Step S11: By clarifying the quantitative relationship between the garbage collection execution cycle and the cycle time window, rigid time rules are established for the staggered execution of GC on each node to avoid time overlap caused by inconsistent cycles. At the same time, the N×T cycle design ensures that all nodes can complete GC within a complete cycle, thus ensuring the long-term stable operation of the storage system.

[0023] It is understood that, in this embodiment, the RAID controller is the core component for managing the RAID system, and its timer is used to trigger GC checks and execution; the cycle time window is a fixed time segment of duration T, which in this embodiment is the basic unit for dividing the garbage collection execution period of each storage node; the garbage collection execution cycle in this embodiment is the total time for all storage nodes to complete one round of GC, ensuring that the GC of each storage node is cyclically ordered.

[0024] See Figure 6 As shown in the example of a RAID system with disk0-disk3 (N=4), if the cycle time window duration T=TW, then the garbage collection execution cycle is 4×TW. This setting provides a time planning basis for subsequent staggered GC execution, such as disk0 in [t,t+TW] and disk1 in [t+TW,t+2TW].

[0025] Understandable, Figure 6 SSD#0, SSD#1, SSD#2, and SSD#3 represent the four independent solid-state drives that make up a RAID (such as RAID-5) array, corresponding to disk0-disk3 devices respectively.

[0026] "Predictable" indicates that the SSD is in a normal service state. In this state, the SSD primarily handles read and write I / O requests from the host, with low and stable response latency and predictable performance. Garbage collection within the SSD is not activated or is at a very low priority, causing no significant interference to user I / O. For the entire RAID array, disks in this state can provide high-quality parallel I / O services or be used for rapid data reconstruction.

[0027] "Busy" indicates that the SSD is in an internal maintenance state, specifically, it is performing garbage collection (GC). GC operations require data movement and block erasure within the SSD, which consumes significant bandwidth and computing resources. This leads to a sharp increase and large fluctuations in latency for host I / O responses. "Busy" accurately reflects the characteristics of disk performance degradation and the SSD being busy with internal tasks at this time.

[0028] TW is the most basic unit of time interval. The timer is triggered in TW cycles. Within each TW, the system guarantees that at most one SSD is in Busy state.

[0029] t is the starting point of the timeline. As can be seen from the state distribution, the first Busy period for SSD#0 begins at t, indicating the point in time when the system completes initialization and begins the first GC cycle time window. After t, in each TW interval, a different SSD will take turns entering the Busy state.

[0030] S12: The timer periodically checks the storage status of each storage node according to the set garbage collection execution cycle, determines whether each storage node has reached the garbage collection trigger condition, and forms a list of nodes that need to perform garbage collection.

[0031] Step S12 involves periodically checking to avoid invalid GC (not meeting the triggering conditions) or missed GC (meeting the conditions but not executing). The formation of the node list provides clear scheduling objects for subsequent garbage collection period planning, ensuring the targeted and efficient scheduling of GC.

[0032] In this embodiment, the garbage collection trigger condition refers to the criteria for determining that GC needs to be started, such as the proportion of invalid data blocks in the storage node reaching a preset threshold; the node list records the list of storage nodes that need to perform garbage collection, which is the basis for allocating garbage collection execution time periods in this embodiment.

[0033] See Figure 7 As shown, Figure 7 This embodiment provides an example of a periodic check, and the check logic of this example is as follows: Initialization Phase: The system assigns an initial value to the garbage collection execution time Tgc[i] of each device in the RAID group (e.g., disk0, disk1, disk2, and disk3 correspond to dev0, dev1, dev2, and dev3, respectively). These values ​​are set to an arithmetic sequence (0, t, 2t, 3t), thus forcibly staggering the GC trigger times of each device at the starting point. This means that dev0 will prepare for GC immediately, dev1 after time t, dev2 after time 2t, and so on.

[0034] Start Timer: After initialization, start a global periodic timer. The trigger interval of this timer is the basic time unit in the process.

[0035] Periodic check loop: Each time the timer is triggered, the system enters a loop, traversing each device.

[0036] Countdown and Judgment: For the current device i, if its GC time Tgc[i] > 0, then execute Tgc[i]-- (i.e., decrement the value by 1), and then check the next device. If Tgc[i] is already equal to 0 (indicating that the countdown has ended after several decrementing operations), then execute the following two actions: Triggering GC means instructing device i to begin garbage collection.

[0037] Reset the timer, that is, reset Tgc[i] to a full period t. In this way, device i will trigger GC again after t time units.

[0038] Loop End: Once all devices have been checked and processed, the current timer task ends. The system waits for the next timer trigger and repeats the periodic check loop, countdown and judgment, and loop end steps.

[0039] By periodically checking the status of each disk using a timer to determine whether GC needs to be triggered and recording the garbage collection status of each disk, a list of nodes is formed, providing accurate data support for subsequent planning of GC execution periods.

[0040] S13: Based on the node list, plan the garbage collection execution period for each storage node according to the cyclic time window.

[0041] For step S13, by dynamically planning the garbage collection execution period, it is ensured that even if there are multiple storage nodes that need garbage collection, the rule of only one storage node being GCd in the same cycle time window can be strictly followed to avoid resource contention, while flexibly adapting to the dynamic changes in the GC requirements of storage nodes.

[0042] In some embodiments of the present invention, a specific garbage collection execution period planning scheme is provided. In S13, that is, based on the node list, the garbage collection execution period of each storage node is planned according to the cyclic time window, which specifically includes the following steps: If a storage node is already performing garbage collection in the current cycle time window, the remaining storage nodes that need to perform garbage collection will be allocated to subsequent consecutive cycle time windows in order of first detection and first execution. If no storage node needs to perform garbage collection within the cycle time window, the cycle time window remains in a state of no garbage collection execution. Subsequently, the storage nodes that need to perform garbage collection are reordered and the node list is updated, and they are assigned to the consecutive cycle time windows following the current cycle time window with no garbage collection execution.

[0043] This garbage collection execution time planning scheme clarifies the scheduling rules when multiple nodes require garbage collection. It ensures scheduling fairness by detecting and executing first, and avoids wasting time by allocating subsequent consecutive windows, thus guaranteeing the orderly and efficient execution of GC. It strictly avoids GC conflicts between multiple storage nodes in the same cycle time window. It is also adapted to scenarios without garbage collection needs, avoiding meaningless scheduling that occupies system resources. At the same time, by reordering and updating the node list, it ensures that storage nodes with newly added garbage collection needs can obtain execution time in a timely manner, ensuring scheduling flexibility and resource utilization.

[0044] See Figure 6 As shown, taking the RAID system of disk0-disk3 as an example, if disk0 has already performed GC in the [t,t+TW] window, the disk1 and disk2 subsequently detected in the node list will be allocated to the [t+TW,t+2TW] and [t+2TW,t+3TW] windows in sequence according to the rules, to ensure that the GC of each storage node is performed in a staggered manner.

[0045] In practice, for example, if no disk reaches the garbage collection trigger condition within a certain cycle time window, the window remains idle. When the next disk reaches the trigger condition, the node list is updated and the disk is assigned to the consecutive window following the idle window, ensuring that the scheduling adapts to the actual needs.

[0046] S14: Within any cyclic time window of duration T, control one storage node that has been allocated a garbage collection execution period to perform garbage collection, while the remaining N-1 storage nodes remain in a non-garbage collection state.

[0047] For step S14, rigid constraints ensure that only one storage node is GC within the same T, maximizing the number of available storage nodes and providing sufficient response resources for IO requests, fundamentally solving the problem of GC and IO competing for resources in traditional RAID.

[0048] See Figure 6 As shown, taking a RAID system consisting of N=4 disks 0-3 as an example, only one disk performs GC within TW at the same time, while the other N-1 disks remain in a non-garbage collection state, reserving sufficient resources for IO response.

[0049] S20: Receive an IO request and obtain the priority of the IO request.

[0050] Step S20 provides a foundation for differentiated scheduling of IO requests. By distinguishing priorities, it ensures that high-priority IO requests receive better resource allocation and response mechanisms, thus solving the problem of QoS not being guaranteed for high-priority requests caused by the one-size-fits-all approach in traditional scheduling.

[0051] Among them, QoS (Quality of Service) refers to performance indicators such as response latency and throughput of IO requests, and its core is to ensure the stable performance of high-priority requests.

[0052] In some embodiments of the present invention, such as Figure 3 As shown, a specific IO request receiving scheme is provided. In S20, that is, receiving the IO request and obtaining the priority of the IO request, the specific steps include the following steps S21-S22.

[0053] S21: Receive an IO request sent by the host, wherein the IO request carries a priority flag bit pre-specified by the host.

[0054] For step S21, the method of carrying priority information is clarified. By embedding a flag bit in the IO request, the accuracy and integrity of priority information transmission are ensured, providing a reliable basis for subsequent priority identification and avoiding priority judgment errors.

[0055] It is understood that the priority flag, in this embodiment, is a binary flag pre-set by the host to identify the importance of IO requests and is the basis for distinguishing between high / low priority IO requests.

[0056] S22: Identify the priority flag in the IO request and determine the priority level of the IO request based on the flag.

[0057] For step S22, the abstract flag is transformed into a clear priority classification (high priority / low priority), providing a clear decision-making standard for subsequent differentiated scheduling, ensuring that high-priority requests can trigger a dedicated fast response mechanism, and low-priority requests can reasonably occupy resources.

[0058] S30: Based on the garbage collection status of each storage node and the priority of the IO request, the IO request is assigned to one or more storage nodes that are not in a garbage collection state for processing.

[0059] For step S30, storage nodes are allocated by combining the garbage collection status and the priority of IO requests. This avoids the delay caused by IO requests accessing storage nodes that are executing garbage collection. At the same time, the disk access sequence is optimized to alleviate the path conflict and addressing delay problems of traditional algorithms, thereby achieving a synergistic improvement in throughput and low latency.

[0060] In some embodiments of the present invention, such as Figure 4 As shown, a specific IO request processing scheme is provided. In S30, the IO request is assigned to one or more storage nodes in non-garbage collection state for processing according to the garbage collection status of each storage node and the priority of the IO request. Specifically, it includes the following steps S31-S32.

[0061] S31: If the IO request is a low-priority request, then the IO request will be assigned to a storage node that is not in a garbage collection state for processing.

[0062] Step S31: Formulate an appropriate scheduling strategy for low-priority IO requests. Without occupying the dedicated resources of high-priority requests, make reasonable use of the idle resources of storage nodes in non-garbage collection state to complete the request processing, balance the resource occupation of different priority requests, and ensure the overall operating efficiency of the system.

[0063] Understandably, storage nodes that are not in a garbage collection state have stable response capabilities. When low-priority IO requests are assigned to these nodes, they can complete data reading and writing normally without affecting the fast response channel for high-priority IO requests, thus achieving efficient utilization of resources in a tiered manner.

[0064] S32: If the IO request is a high-priority request, it is allocated to the currently available storage node and processed using a parity-based data reconstruction algorithm.

[0065] Step S32 formulates an adapted scheduling strategy for high-priority IO requests. By using a parity check reconstruction algorithm, it bypasses the access bottleneck of storage nodes in the garbage collection state, ensuring that even if the target access node is in the garbage collection state, the data can be quickly reconstructed through other storage nodes in the non-garbage collection state, thus guaranteeing low-latency response for high-priority IO requests and improving QoS.

[0066] In this embodiment, the parity check data reconstruction algorithm is an algorithm that uses XOR operation to reconstruct the target access node data using the sub-data blocks of other storage nodes and parity check information, in order to solve the problem of high-priority IO access during target access node GC.

[0067] In some embodiments of the present invention, such as Figure 5 As shown, a specific high-priority IO request processing scheme is provided. In S32, if the IO request is a high-priority request, it is allocated to the currently available storage node and processed using a data reconstruction algorithm based on parity check. Specifically, it includes the following steps S321-S323.

[0068] S321: Obtain the target access node for the high-priority IO request and the garbage collection status of the target access node.

[0069] In step S321, the high-priority I / O access objects and their states are identified, providing crucial information for subsequent decisions. Specifically, if the target access node is in a non-garbage-collection state, it is accessed directly; if it is in a garbage-collection state, data reconstruction is initiated. This avoids response delays caused by blindly allocating storage nodes and ensures the accuracy of scheduling decisions.

[0070] The target access node is the storage node where the data to be read or written by the IO request is located; it is the direct target unit of the IO access.

[0071] S322: When the target access node of a high-priority request is in a garbage collection state, the target access node returns an access error response.

[0072] For step S322, by returning an error response, high-priority IO requests are prevented from waiting on storage nodes in the garbage collection state, and subsequent data reconstruction mechanisms are triggered in a timely manner, reducing invalid waiting time and ensuring the timeliness of request responses.

[0073] Understandably, an access error response is an inaccessible signal returned to the host when the target access node is in a garbage collection state, which is used to trigger the data reconstruction mechanism.

[0074] S323: Upon receiving the error response, a data reconstruction mechanism is triggered, and the remaining storage nodes in non-garbage collection state are invoked to reconstruct the data of the target access node based on parity calculation.

[0075] For step S323, by calling the resource and parity check algorithm of the storage node in non-garbage collection state, the data reconstruction is completed by bypassing the storage node in garbage collection state, ensuring that high-priority IO is not affected by GC, achieving low-latency response, and significantly improving QoS.

[0076] Among them, the data reconstruction mechanism is a mechanism to recover the target access node data based on the sub-data blocks and parity information of the storage node in the non-garbage collection state; the XOR operation (⊕) is the logical operation used for data reconstruction in this embodiment, which can deduce unknown data from known data.

[0077] In some embodiments of the present invention, a specific data reconstruction scheme is provided. In S323, that is, after receiving the error response, a data reconstruction mechanism is triggered, and the remaining storage nodes in the non-garbage collection state are called to perform data reconstruction of the target access node based on parity calculation. Specifically, it includes the following steps: The stored data of each storage node is pre-divided into several sub-data blocks, and the parity information corresponding to each sub-data block is calculated. The parity information is then distributed and stored in each storage node. Based on the distribution of sub-data blocks in the target access node that returned the access error response, obtain the corresponding sub-data blocks and parity information from the storage nodes that are not in the garbage collection state; By using XOR operations, the storage data of the target access node is reconstructed using the obtained sub-data blocks and parity information, thus completing the high-priority IO request response.

[0078] Understandably, the distributed storage of sub-data block splitting and verification information ensures that the required data can be obtained from multiple storage nodes that are not in a garbage collection state during reconstruction, avoiding single point of dependence and improving reconstruction efficiency.

[0079] Sub-data blocks are smaller blocks of data that are split from the original stored data according to preset rules. See [link to relevant documentation]. Figure 8 As shown, A1-A3, B1-B3, and C1-C3 are sub-blocks of data blocks A, B, and C, respectively. Ap, Bp, and Cp are the parity check information corresponding to data blocks A, B, and C, respectively, which satisfy the following relationships: A1=A2⊕A3⊕Ap, B1=B2⊕Bp⊕B3, C1=C2⊕Cp⊕C3, where ⊕ is the XAND operator.

[0080] The parity check information is redundant information calculated based on the XOR operation of sub-data blocks. See also Figure 8 As shown, Ap represents the parity information of data block A, used for data reconstruction. Data and parity information are distributed across disks 0-3. Sub-blocks A1-A3, Ap, and their parity information are stored on different disks. This design provides ample data for rapid reconstruction during subsequent target access node GC. For example, within [t + TW, t+2TW], disk1 performs GC, and the stored sub-block A2 experiences high latency. Data access to A2 is achieved through parity checking, calculated as A2=A1⊕A3⊕Ap, B2=B1⊕Bp⊕B3, and so on.

[0081] It is understandable that the time taken to calculate the data of a high-priority IO request through parity check is Tparity, while the time taken to directly access the target access node of a high-priority request that is in the garbage collection state is Tgc. In the GC scenario, the latency will increase significantly, resulting in Tparity < Tgc. Therefore, using parity check to access the data of the GC target access node can achieve the goal of improving the QoS of IO access.

[0082] As can be seen, the above solution effectively reduces system lag by staggering the garbage collection process of each storage node to avoid it competing for resources with IO requests. Furthermore, by dynamically allocating non-garbage-collected storage nodes based on IO priority, the IO path and access sequence are optimized, alleviating path conflicts and addressing latency issues. This balances bursty IOPS with low-latency stability, significantly reduces tail latency under mixed read / write loads, improves storage system throughput, and thus comprehensively enhances QoS performance, adapting to the high-performance requirements of data centers, databases, and other scenarios.

[0083] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0084] In one embodiment, the present invention provides a RAID-IO scheduling device 100, which corresponds one-to-one with the RAID-IO scheduling method in the above embodiments. For example... Figure 9 As shown, the RAID-IO scheduling device 100 includes a timing module 101, a receiving module 102, and a processing module 103. Detailed descriptions of each functional module are as follows: The timing module 101 is used to schedule the garbage collection process of each storage node in the independent disk redundant array, control the garbage collection cycle of each storage node to be staggered, and obtain the garbage collection status of each storage node.

[0085] The receiving module 102 is used to receive IO requests and obtain the priority of the IO requests.

[0086] The processing module 103 is used to allocate the IO request to one or more storage nodes that are not in a garbage collection state for processing, based on the garbage collection status of each storage node and the priority of the IO request.

[0087] In one embodiment, the timing module 101 is specifically used for: A fixed garbage collection execution cycle and a cycle time window are set for each storage node through a timer in the independent disk redundant array controller; wherein, the duration of the cycle time window is T, the length of the garbage collection execution cycle is N×T, and N is the total number of storage nodes; The timer periodically checks the storage status of each storage node according to the set garbage collection execution cycle, determines whether each storage node has met the garbage collection triggering conditions, and forms a list of nodes that need to be garbage collected. Based on the node list, the garbage collection execution period for each storage node is planned according to the cyclic time window; Within any cyclic time window of duration T, one storage node with an allocated garbage collection execution period is controlled to perform garbage collection, while the remaining N-1 storage nodes remain in a non-garbage collection state.

[0088] The step of planning the garbage collection execution period for each storage node based on the node list and the cyclic time window includes: If a storage node is already performing garbage collection in the current cycle time window, the remaining storage nodes that need to perform garbage collection will be allocated to subsequent consecutive cycle time windows in order of first detection and first execution. If no storage node needs to perform garbage collection within the cycle time window, the cycle time window remains in a state of no garbage collection execution. Subsequently, the storage nodes that need to perform garbage collection are reordered and the node list is updated, and they are assigned to the consecutive cycle time windows following the current cycle time window with no garbage collection execution.

[0089] In one embodiment, the receiving module 102 is specifically used for: Receive an I / O request from the host, the I / O request carrying a priority flag bit pre-specified by the host; Identify the priority flag in the IO request and determine the priority level of the IO request based on the flag.

[0090] In one embodiment, the processing module 103 is specifically used for: If the IO request is a low-priority request, then the IO request will be assigned to a storage node that is not in a garbage collection state for processing; If the IO request is a high-priority request, it will be allocated to a currently available storage node and processed using a parity-based data reconstruction algorithm.

[0091] Wherein, if the IO request is a high-priority request, it is allocated to a currently available storage node and processed using a parity-based data reconstruction algorithm, including: Obtain the target access node for high-priority IO requests and the garbage collection status of the target access node; When the target access node of a high-priority request is in a garbage collection state, the target access node returns an access error response; Upon receiving the error response, a data reconstruction mechanism is triggered, which calls on the remaining storage nodes that are not in a garbage collection state to reconstruct the data of the target access node based on parity calculation.

[0092] Upon receiving the error response, a data reconstruction mechanism is triggered, invoking the remaining storage nodes that are not in a garbage collection state to reconstruct the data of the target access node based on parity calculation, including: The stored data of each storage node is pre-divided into several sub-data blocks, and the parity information corresponding to each sub-data block is calculated. The parity information is then distributed and stored in each storage node. Based on the distribution of sub-data blocks in the target access node that returned the access error response, obtain the corresponding sub-data blocks and parity information from the storage nodes that are not in the garbage collection state; By using XOR operations, the storage data of the target access node is reconstructed using the obtained sub-data blocks and parity information, thus completing the high-priority IO request response.

[0093] Specific limitations regarding the RAID-IO scheduling device 100 can be found in the limitations of the RAID-IO scheduling method described above, and will not be repeated here. Each module in the RAID-IO scheduling device 100 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0094] In one embodiment, a computer device 200 is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device 200 includes a processor 220, memory, and a network interface 250 connected via a system bus 210. The processor 220 provides computing and control capabilities. The memory of the computer device 200 includes non-volatile and / or volatile storage media and internal memory 240. The non-volatile storage media 230 stores an operating system 231, computer programs 232, and a database 233. The internal memory 240 provides an environment for the operation of the operating system and computer programs in the non-volatile storage media 230. The network interface 250 of the computer device 200 is used to communicate with external clients via a network connection. When the computer program is executed by the processor 220, it implements the functions or steps of a RAID-IO scheduling method server. That is, when the processor 220 executes the computer program, it implements the following steps: The garbage collection process of each storage node in the independent disk redundant array is scheduled on a timer to control the garbage collection cycle of each storage node to be staggered, and the garbage collection status of each storage node is obtained. Receive IO requests and obtain the priority of the IO requests; Based on the garbage collection status of each storage node and the priority of the IO request, the IO request is assigned to one or more storage nodes that are not in a garbage collection state for processing.

[0095] In one embodiment, a computer device 300 is provided, which may be a client, and its internal structure diagram may be as follows: Figure 11As shown. The computer device includes a processor 320, memory, network interface 350, display screen 370, and input device 360 ​​connected via a system bus 310. The processor 320 provides computing and control capabilities. The memory includes a non-volatile storage medium 330 and internal memory 340. The non-volatile storage medium 330 stores an operating system 331 and a computer program 332. The internal memory provides an environment for the operation of the operating system 331 and the computer program 332 in the non-volatile storage medium 330. The network interface 350 of the computer device 300 is used for communication with an external server via a network connection. When the computer program is executed by the processor 320, it implements the functions or steps of a RAID-IO scheduling method on the client side. That is, when the processor 320 executes the computer program 332, it performs the following steps: The garbage collection process of each storage node in the independent disk redundant array is scheduled on a timer to control the garbage collection cycle of each storage node to be staggered, and the garbage collection status of each storage node is obtained. Receive IO requests and obtain the priority of the IO requests; Based on the garbage collection status of each storage node and the priority of the IO request, the IO request is assigned to one or more storage nodes that are not in a garbage collection state for processing.

[0096] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The garbage collection process of each storage node in the independent disk redundant array is scheduled on a timer to control the garbage collection cycle of each storage node to be staggered, and the garbage collection status of each storage node is obtained. Receive IO requests and obtain the priority of the IO requests; Based on the garbage collection status of each storage node and the priority of the IO request, the IO request is assigned to one or more storage nodes that are not in a garbage collection state for processing.

[0097] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0100] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A RAID-IO scheduling method, characterized in that, include: The garbage collection process of each storage node in the independent disk redundant array is scheduled on a timer to control the garbage collection cycle of each storage node to be staggered, and the garbage collection status of each storage node is obtained. Receive IO requests and obtain the priority of the IO requests; Based on the garbage collection status of each storage node and the priority of the IO request, the IO request is assigned to one or more storage nodes that are not in a garbage collection state for processing.

2. The RAID-IO scheduling method according to claim 1, characterized in that, The method of timed scheduling of the garbage collection process for each storage node in the independent disk redundancy array, controlling the garbage collection cycles of each storage node to be staggered, and obtaining the garbage collection status of each storage node includes: A fixed garbage collection execution cycle and a cycle time window are set for each storage node through a timer in the independent disk redundant array controller; wherein, the duration of the cycle time window is T, the length of the garbage collection execution cycle is N×T, and N is the total number of storage nodes; The timer periodically checks the storage status of each storage node according to the set garbage collection execution cycle, determines whether each storage node has met the garbage collection triggering conditions, and forms a list of nodes that need to be garbage collected. Based on the node list, the garbage collection execution period for each storage node is planned according to the cyclic time window; Within any cyclic time window of duration T, one storage node with an allocated garbage collection execution period is controlled to perform garbage collection, while the remaining N-1 storage nodes remain in a non-garbage collection state.

3. The RAID-IO scheduling method according to claim 2, characterized in that, The step of planning the garbage collection execution period for each storage node based on the node list and the cyclic time window includes: If a storage node is already performing garbage collection in the current cycle time window, the remaining storage nodes that need to perform garbage collection will be allocated to subsequent consecutive cycle time windows in order of first detection and first execution. If no storage node needs to perform garbage collection within the cycle time window, the cycle time window remains in a state of no garbage collection execution. The storage nodes that need to perform garbage collection later are reordered and the node list is updated, and then assigned to the consecutive cycle time windows following the current cycle time window with no garbage collection execution.

4. The RAID-IO scheduling method according to claim 3, characterized in that, The step of receiving an I / O request and obtaining the priority of the I / O request includes: Receive an I / O request from the host, the I / O request carrying a priority flag bit pre-specified by the host; Identify the priority flag in the IO request and determine the priority level of the IO request based on the flag.

5. The RAID-IO scheduling method according to claim 4, characterized in that, The step of allocating the IO request to one or more storage nodes that are not in a garbage collection state for processing, based on the garbage collection status of each storage node and the priority of the IO request, includes: If the IO request is a low-priority request, then the IO request will be assigned to a storage node that is not in a garbage collection state for processing; If the IO request is a high-priority request, it will be allocated to a currently available storage node and processed using a parity-based data reconstruction algorithm.

6. The RAID-IO scheduling method according to claim 5, characterized in that, If the IO request is a high-priority request, it is allocated to a currently available storage node and processed using a parity-based data reconstruction algorithm, including: Obtain the target access node for high-priority IO requests and the garbage collection status of the target access node; When the target access node of a high-priority request is in a garbage collection state, the target access node returns an access error response; Upon receiving the error response, a data reconstruction mechanism is triggered, which calls on the remaining storage nodes that are not in a garbage collection state to reconstruct the data of the target access node based on parity calculation.

7. The RAID-IO scheduling method according to claim 6, characterized in that, Upon receiving the error response, a data reconstruction mechanism is triggered, invoking the remaining storage nodes that are not in a garbage collection state to reconstruct the data of the target access node based on parity calculation, including: The stored data of each storage node is pre-divided into several sub-data blocks, and the parity information corresponding to each sub-data block is calculated. The parity information is then distributed and stored in each storage node. Based on the distribution of sub-data blocks in the target access node that returned the access error response, obtain the corresponding sub-data blocks and parity information from the storage nodes that are not in the garbage collection state; By using XOR operations, the storage data of the target access node is reconstructed using the obtained sub-data blocks and parity information, thus completing the high-priority IO request response.

8. A RAID-IO scheduling device, characterized in that, include: The timing module is used to schedule the garbage collection process of each storage node in the independent disk redundant array, control the garbage collection cycle of each storage node to be staggered, and obtain the garbage collection status of each storage node. A receiving module is used to receive IO requests and obtain the priority of the IO requests; The processing module is used to allocate the IO request to one or more storage nodes that are not in a garbage collection state for processing, based on the garbage collection status of each storage node and the priority of the IO request.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the RAID-IO scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the RAID-IO scheduling method as described in any one of claims 1 to 7.

Images