A RAID resource allocation method, apparatus, device, and medium

By allocating and releasing APU resources in RAID, the problem of excessive resource consumption during write I/O requests is solved, thus improving storage performance.

CN116401063BActive Publication Date: 2026-11-17INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310397484.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-14
Publication Date
2026-11-17
Estimated Expiration
2043-04-14

AI Technical Summary

Technical Problem

In RAID, write I/O requests rapidly consume limited APU resources, causing other write I/O requests to fail due to the APU resources being reserved, thus impacting storage performance.

Method used

When a write IO request is received, the first APU resource is allocated to the data block, and the data is transferred to the hardpage and APU context. An intermediate checksum is generated and the second APU resource is allocated to the checksum. Resources are released according to the disk write status of the data block and checksum block to avoid the reuse of multiple APU resources.

Benefits of technology

This reduces the consumption of APU resources when executing write I/O requests, avoids resource reservation failures, and improves RAID storage performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116401063B_ABST
    Figure CN116401063B_ABST
Patent Text Reader

Abstract

This application discloses a RAID resource allocation method, apparatus, device, and medium, relating to the field of data transmission technology. Based on a write I / O request, a first APU resource is allocated to a data block; data is transmitted to the hardpage, and write I / O information is stored in the APU context; data from the previous write I / O request is retrieved, and while allocating a second APU resource based on the generated intermediate checksum, it is transmitted to the hardpage, and write I / O information is stored in the APU context; based on the disk write status of the data block and / or checksum block, the first APU resource and / or the second APU resource are released. In this way, a checksum block only requires one APU resource, eliminating the need to call multiple APU resources, thus avoiding the situation where limited APU resources are rapidly consumed when executing a write I / O request, causing the failure of reserved APU resources for most other write I / O requests, further improving RAID storage performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a RAID resource allocation method, apparatus, device and medium. Background Technology

[0002] In this era of rapid technological advancement, storage technology has also undergone continuous innovation. Data security, reliability, and storage performance have become crucial metrics for large enterprises when evaluating storage devices. Redundant Arrays of Independent Disks (RAID) are commonly categorized into DRAID5 and DRAID6. In DRAID5, a stripe (with a stripe number, which can be represented as a stripe number) consists of a data block (d) and a parity block (p), allowing for the loss of one data block. In DRAID6, a stripe consists of a data block (d), a parity block (p), and a parity block (q), allowing for the simultaneous loss of data on two data blocks. When a write I / O request is made to the disk, APU resources (non-volatile memory: hardpage, APU context, etc.) are utilized to ensure data security and prevent data loss in the event of power failure. After the server issues a write I / O request, it first writes the data to be written to the hardpage, then uses the apucontext resource to record the information of this write request (write request information generally includes: stripe number, block number, memory information, etc.), and then writes the data to disk. After successful disk write, the apucontext resource is released. In the event of a power outage or other failure, the information stored in the apucontext and hardpage will not be lost. It is only necessary to retrieve the information corresponding to the write I / O request from the apucontext and initiate a new write I / O request. However, APU resources are limited, and multiple apucontexts need to be allocated during the write I / O request process. The `ext` resource, where the intermediate parity delta data shares an `apucontext` resource with data block `d`, requires an `apucontext` resource containing parity delta information when calculating new parity blocks `p` and `q`. Therefore, calculating new parity blocks also requires copying parity delta information to the `apucontext` resources of multiple parity blocks. Specifically, when there is one parity block, the parity delta information is copied to the `apucontext` resource of that block; when there are two parity blocks, the parity delta information is copied to the `apucontext` resources of both blocks. At this point, it's clear that executing a write I / O request rapidly consumes limited APU resources, causing most other write I / O requests to fail due to APU resource reservations and wait, which also impacts storage performance.

[0003] Given the aforementioned problems, finding ways to improve the storage performance of RAID is a problem that those skilled in the art are striving to solve. Summary of the Invention

[0004] The purpose of this application is to provide a RAID resource allocation method, apparatus, device, and medium for improving RAID storage performance.

[0005] To address the aforementioned technical problems, this application provides a resource allocation method for RAID, comprising: When a write IO request is received, the first APU resource is allocated to the data block corresponding to the write IO request. The APU resource includes at least a hardpage and an APU context. Transmit the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, and store the write IO information corresponding to the write IO request in the apucontext corresponding to the first APU resource; Retrieve the data corresponding to the previous write IO request and generate an intermediate verification value based on the data corresponding to the write IO request. The intermediate verification value is stored in the verification block. Allocate the second APU resource to the intermediate check value, and transmit the intermediate check value to the hardpage corresponding to the second APU resource. Store the write IO information corresponding to the check block in the apucontext corresponding to the second APU resource. Release the first APU resources and / or the second APU resources based on the disk write status of data blocks and / or parity blocks.

[0006] Preferably, it further includes: The sio_flages flag is used to mark data blocks and check blocks.

[0007] Preferably, when the data block is written to disk, releasing the first APU resources and / or the second APU resources includes: Cancel the sio_flages flag corresponding to the data block and release the first APU resource; Query and determine if the sio_flages flag corresponding to the check block exists; If not, then release the second APU resource; If so, the second APU resource will not be released.

[0008] Preferably, when the parity block is written to disk, releasing the first APU resources and / or the second APU resources includes: Cancel the sio_flages flag corresponding to the check block and release the second APU resources.

[0009] Preferably, after transmitting the intermediate checksum to the hardpage corresponding to the second APU resource, the method further includes: Generate the page table corresponding to the hardpage. The page table is stored in the apucontext corresponding to the second APU resource and is used to manage the hardpage.

[0010] Preferably, after transmitting the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, the method further includes: Generate the page table corresponding to the hardpage. The page table is stored in the apucontext corresponding to the first apu resource and is used to manage the hardpage.

[0011] Preferably, before receiving a write IO request, the method further includes: Monitor idle APU resources in real time to allocate APU resources to the data blocks corresponding to write I / O requests.

[0012] To address the aforementioned technical problems, this application also provides a RAID resource allocation device, comprising: The first allocation module is used to allocate the first APU resource to the data block corresponding to the write IO request when a write IO request is received. The APU resource includes at least a hardpage and an APU context. The transmission module is used to transmit the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, and store the write IO information corresponding to the write IO request in the apucontext corresponding to the first APU resource; The acquisition and generation module is used to acquire the data corresponding to the previous write IO request and generate an intermediate verification value based on the data corresponding to the write IO request. The intermediate verification value is stored in the verification block. The second allocation module is used to allocate the second APU resource to the intermediate check value, transmit the intermediate check value to the hardpage corresponding to the second APU resource, and store the write IO information corresponding to the check block in the apucontext corresponding to the second APU resource. The release module is used to release the first APU resources and / or the second APU resources based on the disk write status of data blocks and / or parity blocks.

[0013] In addition, the device is equipped with the following modules, specifically: Preferably, it further includes: The tagging module is used to tag data blocks and check blocks using the sio_flages identifier.

[0014] Preferably, when the data block is written to disk, releasing the first APU resources and / or the second APU resources includes: The first cancellation and release module is used to cancel the sio_flages flag corresponding to the data block and release the first APU resource; The query and judgment module is used to query and determine whether the sio_flages identifier corresponding to the verification block exists; If not, then release the second APU resource; If so, the second APU resource will not be released.

[0015] Preferably, when the parity block is written to disk, releasing the first APU resources and / or the second APU resources includes: The second cancellation and release module is used to cancel the sio_flages flag corresponding to the verification block and release the second APU resources.

[0016] Preferably, after transmitting the intermediate checksum to the hardpage corresponding to the second APU resource, the method further includes: The first generation module is used to generate the page table corresponding to the hardpage. The page table is stored in the apucontext corresponding to the second apu resource and is used to manage the hardpage.

[0017] Preferably, after transmitting the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, the method further includes: The second generation module is used to generate the page table corresponding to the hardpage. The page table is stored in the apucontext corresponding to the first apu resource and is used to manage the hardpage.

[0018] Preferably, before receiving a write IO request, the method further includes: The real-time monitoring module is used to monitor idle APU resources in real time, so as to allocate APU resources to the data blocks corresponding to write IO requests.

[0019] To address the aforementioned technical problems, this application also provides a RAID resource allocation device, comprising: Memory, used to store computer programs; A processor is used to direct computer programs to implement the resource allocation methods of RAID.

[0020] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of all the above-described RAID resource allocation methods.

[0021] This application provides a RAID resource allocation method, comprising: when a write IO request is received, allocating a first APU resource to the data block corresponding to the write IO request, wherein the APU resource includes at least a hardpage and an APU context; transmitting the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, and storing the write IO information corresponding to the write IO request in the APU context corresponding to the first APU resource; obtaining the data corresponding to the previous write IO request and generating an intermediate check value based on the data corresponding to the write IO request, wherein the intermediate check value is stored in a check block; allocating a second APU resource to the intermediate check value, transmitting the intermediate check value to the hardpage corresponding to the second APU resource, and storing the write IO information corresponding to the check block in the APU context corresponding to the second APU resource; and releasing the first APU resource and / or the second APU resource according to the disk write status of the data block and / or the check block. At this point, only one APU resource is needed for a parity block to store the copy data of the intermediate parity value and the write IO information of the parity block. Multiple APU resources are not required. This avoids the situation where the limited APU resources are quickly consumed when executing a write IO request, causing the reserved APU resources for most other write IO requests to fail, thus further improving the storage performance of RAID.

[0022] This application also provides a RAID resource allocation device, equipment, and medium, with the same effect as above. Attached Figure Description

[0023] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a RAID resource allocation method provided in an embodiment of this application; Figure 2 This is a structural diagram of a RAID resource allocation device provided in an embodiment of this application; Figure 3 This is a structural diagram of a RAID resource allocation device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0026] The core of this application is to provide a RAID resource allocation method, apparatus, device, and medium that can improve the storage performance of RAID.

[0027] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] RAID is generally a storage system composed of multiple disks, and the data storage relationship between the disks is defined by an algorithm. Common RAID algorithms include 0, 1, 5, 6, 10, 50, and 60, each storing data differently on the disks. It's important to note that RAID 5 and / or RAID 6 use user-defined "stripes" as the basic unit for scheduling. A stripe is a data length unit, user-defined, used in the RAID algorithm to split addresses and rotate data among different array members. RAID is a method of storing the same data in different locations on multiple hard drives. By placing data on multiple hard drives, input / output operations can overlap in a balanced manner, improving performance. Because multiple hard drives increase the mean time between failures (MTBF), storing redundant data also increases fault tolerance.

[0029] RAID 5 is a redundant disk array consisting of at least three identical hard drives, referred to as "physical disks." The data stored on each physical disk is divided into stripes of equal size, arranged from front to back according to a pre-defined length. Each stripe is numbered sequentially on each physical disk using a non-negative integer, and stripes with the same number are managed as a single "stripe line." Within each stripe line, the number of stripes matches the number of physical disks. According to RAID 5 requirements, one stripe within each stripe line is selected as the "parity disk" (P disk) using a specific algorithm, while the remaining stripes serve as "data disks" (D0, D1, D2, etc.). Data is stored on the data disks, while the parity disk stores a "checksum" calculated from the data disk's contents. In RAID 5, the parity value is calculated by removing data stored in the same location on all data disks within the same stripe and then performing a bitwise XOR operation on them. It's important to note that the write load on the parity disk is typically higher than that on the data disks. To balance the load, different physical disks are usually selected as the parity disk in different stripes; this process is called "stepping." The most common stepping method is to decrement by one to the left. If the p disk position in the current stripe is physical disk n, then physical disk n-1 is selected as p disk in the next stripe; if n=0, the stepping returns to the last physical disk.

[0030] RAID 6 is a redundant disk array consisting of at least four disks of the same size. Its space is divided into stripes of the same size, similar to RAID 5. The difference is that in each stripe row, the RAID 6 algorithm selects two physical disks as parity disks. One parity disk is still the p disk, using the same algorithm as RAID 5; the additional parity disk uses the Reed-Solomon algorithm to calculate the parity value, and this disk is simply called the q disk.

[0031] When one physical disk in a RAID 5 array fails, the RAID 5 algorithm can read data from the remaining disks and recalculate the correct data on the failed disk, thus completing "data recovery." If the recovered data is copied and stored to a new hard drive to replace the failed disk, and eventually the entire failed disk is replaced, this process is called "data rebuild." A RAID 5 array cannot function when more than one physical disk fails.

[0032] In RAID 6, each stripe row has two parity disks, p and q. The Reed-Solomon algorithm can be used to correctly recover data even if up to two physical disks fail. Therefore, RAID 6 is more secure than RAID 5, but it wastes more storage space on the parity disks. RAID 5 arrays fail when more than two physical disks fail. It should be noted that Reed-Solomon, mentioned in this embodiment, can also be abbreviated as RS. Reed-Solomon codes (RS codes) are a type of forward error correction channel coding. For the valid polynomial generated from the corrected data, the encoding process first calculates redundancy for these polynomials at multiple points, then transmits or stores them. This sampling beyond the necessary value of the polynomial causes it to overdeterminate (over-limited). When the receiver correctly receives enough points, it can recover the original polynomial, even if many points on the received polynomial are damaged by noise. In this embodiment, a "sector" is a unit of measurement equal to 512 bytes, inherited from ordinary hard drive systems. Although some new hard drives use "4KB sectors," the sector in this embodiment specifically refers to a 512-byte sector as defined in the traditional sense, and is used only as a unit of quantity. Since RAID 5 and / or RAID 6 data is managed in units of equal-sized "stripes," stripes are generally managed in a tabular format. In this case, each small cell represents a stripe, each row represents a stripe row, and each column represents a physical disk. When calculating parity values ​​p and q or performing data recovery, the data on the physical disk needs to be copied to the RAID system's Random Access Memory (RAM).

[0033] RAID technology has three main functions: First, by striping data on disks, it enables block-based data access, reducing disk seek time and improving data access speed; second, by reading several disks in an array simultaneously, it reduces disk seek time and improves data access speed; and third, by mirroring or storing parity information, it achieves data redundancy protection.

[0034] There are three types of disk arrays: external disk array enclosures, internal disk array cards, and software-based emulation. External disk array enclosures are most commonly used in large servers and feature hot-swap capability; however, these products are generally expensive. Internal disk array cards are cheaper but require more technical installation skills, making them suitable for technicians. Hardware arrays offer features such as online expansion, dynamic modification of RAID levels, automatic data recovery, drive roaming, and high-speed caching. They provide solutions for performance, data protection, reliability, availability, and manageability. The RAID card uses a dedicated processing unit for operation. Software emulation refers to configuring multiple hard drives connected to a Small Computer System Interface (SCSI) interface as logical drives to form an array using the disk management functions provided by the network operating system. Software arrays can provide data redundancy, but the performance of the disk subsystem will be reduced, sometimes significantly, by around 30%. Therefore, it can slow down the machine and is unsuitable for servers with high data traffic.

[0035] Disk arrays function as independent systems, either directly connected to the host or connected via a network. A disk array has multiple ports that can be connected to different hosts or different ports. Connecting a host to different ports of the array can improve transfer speed. Similar to the integrated caching on a single disk, disk arrays also incorporate a certain amount of buffer memory to accelerate interaction with the host. The host interacts with the disk array's cache, and the cache interacts with the specific disks. In applications, some frequently accessed data needs to be read often. The disk array uses its internal algorithm to locate this frequently accessed data and store it in the cache, speeding up the host's access to this data. For other data not in the cache, the array reads it directly from the disk and transfers it to the host. Data written by the host is only written to the cache, allowing the host to complete the write operation immediately. The cache then gradually writes the data to the disk.

[0036] RAID is generally used to improve data transfer rates. RAID significantly increases the throughput of a storage system by simultaneously storing and retrieving data across multiple disks. In RAID, many disk drives can transfer data concurrently, and these drives are logically considered as a single drive, allowing speeds several times, tens of times, or even hundreds of times faster than a single drive. RAID is used when CPU speeds increase rapidly while disk drive data transfer rates cannot be significantly increased. Additionally, it provides fault tolerance through data verification. Ordinary disk drives cannot provide fault tolerance. Without the cyclic redundancy check (CRC) code written to the disks, RAID fault tolerance relies on the hardware fault tolerance of each disk drive, thus providing higher security. Many RAID modes have comprehensive mutual verification / recovery measures, or even direct mirror backups, greatly improving the fault tolerance and stability of the RAID system. However, RAID also has the following drawbacks: RAID 0 lacks redundancy; if one disk (physical) fails, all data becomes unusable. RAID 1 disk utilization is capped at 50% (when using two disks), the lowest of all RAID levels. RAID 0+1 can be considered a compromise between RAID 0 and RAID 1. RAID 0+1 provides data security, but with a lower level of protection than Mirror, while offering higher disk space utilization.

[0037] Figure 1 A flowchart illustrating a RAID resource allocation method provided in this application embodiment is shown below. Figure 1 As shown, the resource allocation method of this RAID includes: S10: When a write IO request is received, the first APU resource is allocated to the data block corresponding to the write IO request; APU resources include at least hardpage and apucontext; S11: Transmit the data corresponding to the write IO request to the hardpage corresponding to the first apu resource, and store the write IO information corresponding to the write IO request in the apucontext corresponding to the first apu resource; S12: Obtain the data corresponding to the previous write IO request and generate an intermediate verification value based on the data corresponding to the write IO request; Intermediate check values ​​are stored in the check block; S13: Allocate the second APU resource to the intermediate check value, and transmit the intermediate check value to the hardpage corresponding to the second APU resource. Store the write IO information corresponding to the check block in the apucontext corresponding to the second APU resource. S14: Release the first APU resources and / or the second APU resources based on the disk write status of data blocks and / or parity blocks.

[0038] It should be noted that the division of data blocks and parity blocks is based on the smallest block unit. `Stridenumber` is the stripe number of the corresponding block, and `stripnumber` is the corresponding block number. In apucontext, data blocks are denoted as `d_buffer`, parity blocks as `p_buffer` and `q_buffer`, and the block storing intermediate parity values ​​is denoted as `paritydelta_buffer`, with the corresponding intermediate parity values ​​denoted as `parity_delta`. For RAID, which is generally divided into RAID5 and RAID6, when the RAID is RAID5, there is one and only one parity block, denoted as `p_buffer`; when the RAID is RAID6, there are two parity blocks, denoted as `p_buffer` and `q_buffer`. Therefore, the above-mentioned parts in apucontext can be represented as: Struct Apucontext{ uint 64 stridenumber; uint 16 stripnumber; uint 32 d_buffer; uint 32 p_buffer; uint 32 q_buffer; uint 32 paritydelta_buffer; } Before receiving a write IO request, the process also includes: real-time monitoring of idle APU resources to allocate APU resources to the data block corresponding to the write IO request.

[0039] The process includes, after transmitting the intermediate checksum to the hardpage corresponding to the second APU resource, generating a page table for the hardpage, which is stored in the APU context corresponding to the second APU resource for managing the hardpage. Similarly, after transmitting the data corresponding to the write I / O request to the hardpage corresponding to the first APU resource, the process includes, again, generating a page table for the hardpage, which is stored in the APU context corresponding to the first APU resource for managing the hardpage. The page table manages all information within the hardpage.

[0040] It should be noted that when allocating apucontext and hardpage resources to a data block and retrieving new data (the data transmitted corresponding to the current write IO request) from the host, the data is stored in the hardpage memory. Page tables are allocated to manage the page tables corresponding to the data blocks in the hardpage and apucontext. IO information (including stripe number, block number, page table, etc.) is recorded in the apucontext. Old data (the data transmitted corresponding to the previous write IO request) is read. The new data and old data are XORed to obtain intermediate verification data. Hardpage and page tables are allocated, the calculation result is stored in the hardpage, and the paritydelta page table information is stored in the paritydelta_buffer in the data block apucontext.

[0041] While preparing to write new data to disk, the verification process is initiated. At this point, an APUContext is allocated to each verification block. Then, the process of reading old verification data is performed, and the old verification data is XORed with intermediate verification data to obtain the new verification data. Hard pages and page tables are allocated, and the calculated new verification data is written to the hard pages. Verification block I / O information is recorded in the corresponding APUContexts, preparing for the data write-to-disk process. It should be noted that this involves multi-threading, so the order in which data blocks and verification blocks are written to disk is uncertain. In this case, only one APUContext is allocated per verification block, replacing the existing write I / O process where two APUContexts are allocated per verification block (one for storing new verification block information, and one for copying the APUContext information corresponding to the data block and paritydelta). This reduces the required APUContext resource consumption, avoids waiting due to insufficient APU resources, and improves storage performance.

[0042] This application provides a RAID resource allocation method, comprising: when a write IO request is received, allocating a first APU resource to the data block corresponding to the write IO request, wherein the APU resource includes at least a hardpage and an APU context; transmitting the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, and storing the write IO information corresponding to the write IO request in the APU context corresponding to the first APU resource; obtaining the data corresponding to the previous write IO request and generating an intermediate check value based on the data corresponding to the write IO request, wherein the intermediate check value is stored in a check block; allocating a second APU resource to the intermediate check value, transmitting the intermediate check value to the hardpage corresponding to the second APU resource, and storing the write IO information corresponding to the check block in the APU context corresponding to the second APU resource; and releasing the first APU resource and / or the second APU resource according to the disk write status of the data block and / or the check block. At this point, only one APU resource is needed for a parity block to store the copy data of the intermediate parity value and the write IO information of the parity block. Multiple APU resources are not required. This avoids the situation where the limited APU resources are quickly consumed when executing a write IO request, causing the reserved APU resources for most other write IO requests to fail, thus further improving the storage performance of RAID.

[0043] Based on the above embodiments, as a more preferred implementation, it further includes: The `sio_flages` flag is used to mark data blocks and parity blocks. This `sio_flages` flag records the usage of intermediate parity data. It should be noted that the `sio_flages` flag is included as a member in the `sio` flag structure. Additionally, `paritysio` is a variable of type `sio` structure, and each parity block corresponds to one `paritysio` variable.

[0044] Then, the sio_flages identifier is divided according to the following representation: sio struct{ uint 32 sio_flages; } Since calculating the new parity block p and the new parity block q both require intermediate parity data, and the intermediate parity data and the data blocks share the same apucontext resource, when releasing the apucontext resource corresponding to paritydelta, it is necessary to determine whether there are other blocks that need to be used based on the sio_flages flag. Only when the last data block and parity block corresponding to an I / O within a stripe is written to disk can the apucontext resource be released.

[0045] Furthermore, since RAID is generally divided into RAID5 and RAID6, releasing the first APU resources and / or the second APU resources when the data blocks are written to disk includes: Cancel the sio_flages flag corresponding to the data block and release the first APU resource; Query and determine if the sio_flages flag corresponding to the check block exists; If not, then release the second APU resource; If so, the second APU resource will not be released.

[0046] If the RAID is RAID5 at this time, find the paritysio variable corresponding to the data block and mark sio_flages as: sio_flages&=~DataNeedApuContext. This means that the flag representing DataNeedApuContext (the data block needs apucontext resources) is removed, indicating that the data block does not need the apucontext resources corresponding to paritydelta.

[0047] If the RAID is DRAID6 at this time, the sio_flages variable corresponding to parity block p and parity block q of the data block will be marked as: sio_flages&=~DataNeedApuContext. This means that the flag representing DataNeedApuContext (the data block needs apucontext resources) will be removed, indicating that the data block does not need the apucontext resources corresponding to paritydelta.

[0048] If the RAID is RAID5 at this time, find the sio_flages flag of the corresponding parity block. If the sio_flages flag does not have the SingleParityNeedDelta flag, it means that the parity block has been written to disk and no longer needs the apucontext resource corresponding to paritydelta. Since the data block also does not need the apucontext resource corresponding to paritydelta at this time, the apucontext resource is released. Otherwise, if the SingleParityNeedDelta flag is present, it means that the parity block data has not yet completed the disk writing process, and the apucontext resource is not released.

[0049] If the RAID is DRAID6 at this time, and neither parity block needs the apucontext resource corresponding to paritydelta (neither of the two sio_flages flags has the OtherParityNeedDelta flag), then the apucontext resource will be released since the data block also does not need paritydelta. Otherwise, it means that at least one parity block needs the apucontext resource, so the apucontext resource will not be released.

[0050] Simultaneously, when the parity block is written to disk, releasing the first APU resources and / or the second APU resources includes: Cancel the sio_flages flag corresponding to the check block and release the second APU resources.

[0051] If the RAID is RAID5 at this time, the sio_flages flag corresponding to the parity block can be represented as: sio_flages&=~SingleParityNeedDelta. If there is no DataNeedApuContext flag at this time, it means that the data block has completed the write IO request, that is, the apucontext resource is no longer needed, so the apucontext resource is released; otherwise, the apucontext resource is not released.

[0052] If the RAID is DRAID6 at this point, find the sio_flages flag corresponding to the parity block that has been written to disk. If there is no OtherParityNeedDelta flag and no DataNeedApuContext flag, it means that the other parity block and the data block have been written to disk, so release the apucontext resource; otherwise, do not release the apucontext resource. It can be understood that the sio_flages corresponding to the other parity block that has been written to disk can be represented as: sio_flages&=~OtherParityNeedDelta.

[0053] It should also be noted that this embodiment uses a programming language for algorithm implementation, designing a write I / O process algorithm that consumes minimal APU resources and is applicable to the write I / O process of distributed DRAID array configurations, in order to improve storage device performance. Only one APU context is allocated per parity block, replacing the existing write I / O process which allocates two APU contexts per parity block (one for storing new parity block information, and one for copying the APU context information corresponding to the data block and paritydelta). This reduces the required APU context resource consumption, avoids situations where APU resources are insufficient and waiting occurs, lowers the probability of asynchronous resource requests and waiting, and improves storage performance.

[0054] In addition, RAID can be implemented through software and / or hardware, as follows: Using host-based software to provide RAID functionality, implemented at the operating system level, offers advantages over hardware RAID in terms of lower cost and intuitiveness. However, software RAID has the following drawbacks: First, performance: Software RAID can impact overall system performance because it requires the CPU to perform RAID calculations. Second, functionality: Software RAID supports a limited number of RAID levels. Third, compatibility: Software RAID is tied to the host operating system; therefore, compatibility verification is required for both software RAID and operating system upgrades. Upgrades can only be performed when the RAID software and operating system are compatible, which reduces the flexibility of the data processing environment.

[0055] Hardware RAID includes host-based hardware RAID and array-based hardware RAID. Host-based hardware RAID typically installs a dedicated RAID controller on the host, and all disk drives are connected to the host; the RAID controller can also be integrated onto the motherboard. However, host-based hardware RAID controllers are not an efficient solution in data center environments with a large number of hosts. Array-based hardware RAID, on the other hand, uses an external hardware RAID controller, which acts as the interface between the host and the disks, presenting storage volumes to the host, which manages these volumes as physical drives. Hardware RAID controllers have the following main characteristics: first, managing and controlling disk aggregation; second, translating I / O requests between logical and physical disks; and third, regenerating data in the event of disk failure.

[0056] In the above embodiments, the resource allocation method for RAID has been described in detail. This application also provides embodiments corresponding to the resource allocation device for RAID. It should be noted that this application describes the embodiments of the device part from two perspectives: one is based on the functional module, and the other is based on the hardware.

[0057] Figure 2 This application provides a structural diagram of a RAID resource allocation device according to an embodiment of the present application. Figure 2 As shown, this application also provides a RAID resource allocation device, including: The first allocation module 20 is used to allocate the first APU resource to the data block corresponding to the write IO request when a write IO request is received, wherein the APU resource includes at least hardpage and APUcontext; The transmission module 21 is used to transmit the data corresponding to the write IO request to the hardpage corresponding to the first apu resource, and store the write IO information corresponding to the write IO request in the apucontext corresponding to the first apu resource. The acquisition and generation module 22 is used to acquire the data corresponding to the previous write IO request and generate an intermediate verification value based on the data corresponding to the write IO request. The intermediate verification value is stored in the verification block. The second allocation module 23 is used to allocate the second APU resource to the intermediate check value, transmit the intermediate check value to the hardpage corresponding to the second APU resource, and store the write IO information corresponding to the check block in the apucontext corresponding to the second APU resource. Release module 24 is used to release the first APU resources and / or the second APU resources according to the disk write status of data blocks and / or parity blocks.

[0058] It should be noted that the division of data blocks and parity blocks is based on the smallest block unit. `Stridenumber` is the stripe number of the corresponding block, and `stripnumber` is the corresponding block number. In apucontext, data blocks are denoted as `d_buffer`, parity blocks as `p_buffer` and `q_buffer`, and the block storing intermediate parity values ​​is denoted as `paritydelta_buffer`, with the corresponding intermediate parity values ​​denoted as `parity_delta`. For RAID, which is generally divided into RAID5 and RAID6, when the RAID is RAID5, there is one and only one parity block, denoted as `p_buffer`; when the RAID is RAID6, there are two parity blocks, denoted as `p_buffer` and `q_buffer`. Therefore, the above-mentioned parts in apucontext can be represented as: Struct Apucontext{ uint 64 stridenumber; uint 16 stripnumber; uint 32 d_buffer; uint 32 p_buffer; uint 32 q_buffer; uint 32 paritydelta_buffer; } Preferably, after transmitting the intermediate checksum to the hardpage corresponding to the second APU resource, the method further includes: a first generation module, used to generate a page table corresponding to the hardpage, the page table being stored in the APU context corresponding to the second APU resource, and used to manage the hardpage. Similarly, after transmitting the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, the method further includes: a second generation module, used to generate a page table corresponding to the hardpage, the page table being stored in the APU context corresponding to the first APU resource, and used to manage the hardpage. The page table is used to manage all information within the hardpage.

[0059] It should be noted that when allocating apucontext and hardpage resources to a data block and retrieving new data (the data transmitted corresponding to the current write IO request) from the host, the data is stored in the hardpage memory. Page tables are allocated to manage the page tables corresponding to the data blocks in the hardpage and apucontext. IO information (including stripe number, block number, page table, etc.) is recorded in the apucontext. Old data (the data transmitted corresponding to the previous write IO request) is read. The new data and old data are XORed to obtain intermediate verification data. Hardpage and page tables are allocated, the calculation result is stored in the hardpage, and the paritydelta page table information is stored in the paritydelta_buffer in the data block apucontext.

[0060] While preparing to write new data to disk, the verification process is initiated. At this point, an APUContext is allocated to each verification block. Then, the process of reading old verification data is performed, and the old verification data is XORed with intermediate verification data to obtain the new verification data. Hard pages and page tables are allocated, and the calculated new verification data is written to the hard pages. Verification block I / O information is recorded in the corresponding APUContexts, preparing for the data write-to-disk process. It should be noted that this involves multi-threading, so the order in which data blocks and verification blocks are written to disk is uncertain. In this case, only one APUContext is allocated per verification block, replacing the existing write I / O process where two APUContexts are allocated per verification block (one for storing new verification block information, and one for copying the APUContext information corresponding to the data block and paritydelta). This reduces the required APUContext resource consumption, avoids waiting due to insufficient APU resources, and improves storage performance.

[0061] Before receiving a write IO request, the process also includes: real-time monitoring of idle APU resources to allocate APU resources to the data block corresponding to the write IO request.

[0062] Preferably, before receiving a write IO request, the method further includes: The real-time monitoring module is used to monitor idle APU resources in real time, so as to allocate APU resources to the data blocks corresponding to write IO requests.

[0063] Preferably, it further includes: The tagging module is used to mark data blocks and parity blocks using the `sio_flages` flag. The `sio_flages` flag records the usage of intermediate parity data. It should be noted that the `sio_flages` flag is included as a member in the `sio` flag structure. Additionally, `paritysio` is a variable of type `sio` structure, and each parity block corresponds to one `paritysio` variable.

[0064] Then, the sio_flages identifier is divided according to the following representation: sio struct{ uint 32 sio_flages; } Since calculating the new parity block p and the new parity block q both require intermediate parity data, and the intermediate parity data and the data blocks share the same apucontext resource, when releasing the apucontext resource corresponding to paritydelta, it is necessary to determine whether there are other blocks that need to be used based on the sio_flages flag. Only when the last data block and parity block corresponding to an I / O within a stripe is written to disk can the apucontext resource be released.

[0065] Furthermore, since RAID is generally divided into RAID5 and RAID6, releasing the first APU resources and / or the second APU resources when the data blocks are written to disk includes: The first cancellation and release module is used to cancel the sio_flages flag corresponding to the data block and release the first APU resource; The query and judgment module is used to query and determine whether the sio_flages identifier corresponding to the verification block exists; If not, then release the second APU resource; If so, the second APU resource will not be released.

[0066] If the RAID is RAID5 at this time, find the paritysio variable corresponding to the data block and mark sio_flages as: sio_flages&=~DataNeedApuContext. This means that the flag representing DataNeedApuContext (the data block needs apucontext resources) is removed, indicating that the data block does not need the apucontext resources corresponding to paritydelta.

[0067] If the RAID is DRAID6 at this time, the sio_flages variable corresponding to parity block p and parity block q of the data block will be marked as: sio_flages&=~DataNeedApuContext. This means that the flag representing DataNeedApuContext (the data block needs apucontext resources) will be removed, indicating that the data block does not need the apucontext resources corresponding to paritydelta.

[0068] If the RAID is RAID5 at this time, find the sio_flages flag of the corresponding parity block. If the sio_flages flag does not have the SingleParityNeedDelta flag, it means that the parity block has been written to disk and no longer needs the apucontext resource corresponding to paritydelta. Since the data block also does not need the apucontext resource corresponding to paritydelta at this time, the apucontext resource is released. Otherwise, if the SingleParityNeedDelta flag is present, it means that the parity block data has not yet completed the disk writing process, and the apucontext resource is not released.

[0069] If the RAID is DRAID6 at this time, and neither parity block needs the apucontext resource corresponding to paritydelta (neither of the two sio_flages flags has the OtherParityNeedDelta flag), then the apucontext resource will be released since the data block also does not need paritydelta. Otherwise, it means that at least one parity block needs the apucontext resource, so the apucontext resource will not be released.

[0070] Preferably, when the parity block is written to disk, releasing the first APU resources and / or the second APU resources includes: The second cancellation and release module is used to cancel the sio_flages flag corresponding to the verification block and release the second APU resources.

[0071] If the RAID is RAID5 at this time, the sio_flages flag corresponding to the parity block can be represented as: sio_flages&=~SingleParityNeedDelta. If there is no DataNeedApuContext flag at this time, it means that the data block has completed the write IO request, that is, the apucontext resource is no longer needed, so the apucontext resource is released; otherwise, the apucontext resource is not released.

[0072] If the RAID is DRAID6 at this point, find the sio_flages flag corresponding to the parity block that has been written to disk. If there is no OtherParityNeedDelta flag and no DataNeedApuContext flag, it means that the other parity block and the data block have been written to disk, so release the apucontext resource; otherwise, do not release the apucontext resource. It can be understood that the sio_flages corresponding to the other parity block that has been written to disk can be represented as: sio_flages&=~OtherParityNeedDelta.

[0073] It should also be noted that this embodiment uses a programming language for algorithm implementation, designing a write I / O process algorithm that consumes minimal APU resources and is applicable to the write I / O process of distributed DRAID array configurations, in order to improve storage device performance. Only one APU context is allocated per parity block, replacing the existing write I / O process which allocates two APU contexts per parity block (one for storing new parity block information, and one for copying the APU context information corresponding to the data block and paritydelta). This reduces the required APU context resource consumption, avoids situations where APU resources are insufficient and waiting occurs, lowers the probability of asynchronous resource requests and waiting, and improves storage performance.

[0074] The apparatus and method provided in this application have the same beneficial effects. Specifically, when a write I / O request is received, a first APU resource is allocated to the data block corresponding to the write I / O request, wherein the APU resource includes at least a hardpage and an APU context; the data corresponding to the write I / O request is transmitted to the hardpage corresponding to the first APU resource, and the write I / O information corresponding to the write I / O request is stored in the APU context corresponding to the first APU resource; the data corresponding to the previous write I / O request is obtained and an intermediate check value is generated based on the data corresponding to the write I / O request, wherein the intermediate check value is stored in a check block; a second APU resource is allocated to the intermediate check value, and the intermediate check value is transmitted to the hardpage corresponding to the second APU resource, and the write I / O information corresponding to the check block is stored in the APU context corresponding to the second APU resource; the first APU resource and / or the second APU resource are released according to the disk write status of the data block and / or the check block. At this point, only one APU resource is needed for a parity block to store the copy data of the intermediate parity value and the write IO information of the parity block. Multiple APU resources are not required. This avoids the situation where the limited APU resources are quickly consumed when executing a write IO request, causing the reserved APU resources for most other write IO requests to fail, thus further improving the storage performance of RAID.

[0075] Figure 3 This application provides a structural diagram of a RAID resource allocation device, as shown in the embodiment. Figure 3 As shown, a RAID resource allocation device includes: Memory 30 is used to store computer programs; The processor 31 is used to implement the steps of the RAID resource allocation method mentioned in the above embodiments when executing a computer program.

[0076] The RAID resource allocation device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0077] The processor 31 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 31 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 31 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 31 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 31 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0078] The memory 30 may include one or more computer-readable storage media, which may be non-transitory. The memory 30 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 30 is used to store at least the following computer program, which, after being loaded and executed by the processor 31, is capable of implementing the relevant steps of the RAID resource allocation method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 30 may also include an operating system and data, and the storage method may be temporary or permanent storage. The operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, the RAID resource allocation method.

[0079] In some embodiments, the RAID resource allocation device may further include a display screen, input / output interfaces, communication interfaces, a power supply, and a communication bus.

[0080] Those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the resource allocation device for RAID and may include more or fewer components than illustrated.

[0081] The RAID resource allocation device provided in this application embodiment includes a memory 30 and a processor 31. When the processor 31 executes the program stored in the memory 30, it can implement the RAID resource allocation method.

[0082] Finally, this application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0083] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] The resource allocation method, apparatus, device, and medium for RAID provided in this application have been described in detail above. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0085] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A resource allocation method for RAID, characterized in that, include: When a write IO request is received, the first APU resource is allocated to the data block corresponding to the write IO request. The APU resource includes at least a hardpage and an APU context. The hardpage is a non-volatile memory that stores the transmission data corresponding to the write IO request, and the APU context is a non-volatile memory that records IO information. The IO information includes the stripe number, the block number, and the page table. Transmit the data corresponding to the write IO request to the hardpage corresponding to the first APU resource, and store the write IO information corresponding to the write IO request in the apucontext corresponding to the first APU resource; generate a page table corresponding to the hardpage corresponding to the first APU resource, and store the page table in the apucontext corresponding to the first APU resource for managing the hardpage; Obtain the data corresponding to the previous write IO request and generate an intermediate verification value based on the data corresponding to the write IO request, wherein the intermediate verification value is stored in a verification block; Allocate a second APU resource to the intermediate check value and transmit the intermediate check value to the hardpage corresponding to the second APU resource. Store the write I / O information corresponding to the check block in the apucontext corresponding to the second APU resource. Generate a page table corresponding to the hardpage corresponding to the second APU resource. The page table is stored in the apucontext corresponding to the second APU resource for managing the hardpage. Based on the disk write status of the data block and / or the check block, release the first APU resource and / or the second APU resource; a check block only requires one APU resource to store the copy data of the intermediate check value and the write IO information of the check block, without the need for multiple APU resources; The data corresponding to the previous write IO request is used as the old data. The new data and the old data are XORed to obtain an intermediate checksum. A hardpage and page table are allocated, and the calculation result is stored in the hardpage. The page table information of the intermediate checksum is stored in the paritydelta_buffer of the data block apucontext. While preparing to write the new data to disk, the verification process is started. An apucontext is allocated to each verification. The old verification data and the intermediate checksum are XORed to obtain the new verification data. A hardpage and page table are allocated, and the calculated new verification data is written to the hardpage. The verification block IO information is recorded in the corresponding apucontext, and the process of writing the verification data to disk is prepared.

2. The RAID resource allocation method according to claim 1, characterized in that, Also includes: The data block and the check block are marked using the sio_flages identifier.

3. The RAID resource allocation method according to claim 2, characterized in that, When the data block is written to disk, releasing the first APU resource and / or the second APU resource includes: Cancel the sio_flages identifier corresponding to the data block and release the first APU resource; Query and determine whether the sio_flages identifier corresponding to the verification block exists; If not, release the second APU resource; If so, the second APU resource will not be released.

4. The RAID resource allocation method according to claim 1, characterized in that, When the verification block is written to disk, releasing the first APU resource and / or the second APU resource includes: Cancel the sio_flages flag corresponding to the verification block and release the second APU resource.

5. The RAID resource allocation method according to claim 1, characterized in that, Before receiving the write I / O request, the following is also included: The idle APU resources are monitored in real time to allocate the APU resources to the data blocks corresponding to the write IO requests.

6. A resource allocation device for RAID, characterized in that, include: The first allocation module is used to allocate a first APU resource to the data block corresponding to the write IO request when a write IO request is received. The APU resource includes at least a hardpage and an apucontext. The hardpage is a non-volatile memory that stores the transmission data corresponding to the write IO request, and the apucontext is a non-volatile memory that records IO information. The IO information includes a stripe number, a block number, and a page table. The transmission module is used to transmit the data corresponding to the write IO request to the hardpage corresponding to the first apu resource, and store the write IO information corresponding to the write IO request in the apucontext corresponding to the first apu resource; An acquisition and generation module is used to acquire the data corresponding to the previous write IO request and generate an intermediate verification value based on the data corresponding to the write IO request, wherein the intermediate verification value is stored in a verification block; The second allocation module is used to allocate the second APU resource to the intermediate verification value, transmit the intermediate verification value to the hardpage corresponding to the second APU resource, and store the write IO information corresponding to the verification block in the apucontext corresponding to the second APU resource. The release module is used to release the first APU resource and / or the second APU resource according to the disk write status of the data block and / or the check block; The first generation module is used to generate the page table corresponding to the hardpage of the second APU resource. The page table is stored in the APUContext corresponding to the second APU resource and is used to manage the hardpage. The second generation module is used to generate the page table corresponding to the hardpage of the first APU resource. The page table is stored in the APU context corresponding to the first APU resource and is used to manage the hardpage. A single check block requires only one APU resource to store the copy data of the intermediate check value and the write I / O information of the check block, eliminating the need for multiple APU resources. The data corresponding to the previous write IO request is used as the old data. The new data and the old data are XORed to obtain an intermediate checksum. A hardpage and page table are allocated, and the calculation result is stored in the hardpage. The page table information of the intermediate checksum is stored in the paritydelta_buffer of the data block apucontext. While preparing to write the new data to disk, the verification process is started. An apucontext is allocated to each verification. The old verification data and the intermediate checksum are XORed to obtain the new verification data. A hardpage and page table are allocated, and the calculated new verification data is written to the hardpage. The verification block IO information is recorded in the corresponding apucontext, and the process of writing the verification data to disk is prepared.

7. A RAID resource allocation device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the resource allocation method of RAID as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the RAID resource allocation method as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • IO data processing method based on RAID5 and related device

    CN112162697A

  • Disk array initialization method, device and equipment and readable storage medium

    CN115657960A