Data request processing method and device, equipment and storage medium

Abstract

The invention provides a data request processing method and device, equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps that an RAID controller constructs a response delay queue; receiving a data input / output request of a host, and decomposing the data input / output request into a plurality of sub-requests; determining a first sub-request and a second sub-request from the plurality of sub-requests, the first sub-request being associated with the second sub-request; comparing the response delay time of the hard disk device corresponding to the first sub-request and the response delay time of the hard disk device corresponding to the second sub-request; according to the comparison result and the response delay queue, delay adjustment operation is executed on the first sub-request and the second sub-request, so that the completion time for processing the first sub-request and the second sub-request is consistent; and after the first sub-request and the second sub-request are completed, calculating new verification data and releasing the corresponding buffer area. According to the scheme, by introducing the delay queues and regulating and controlling the distribution opportunity of the I / O requests, the problem of buffer area resource waste caused by response delay differences of different hard disk devices is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for processing data requests. Background Art

[0002] With the development of information technology, the demand for data storage and transmission is growing. The NVMe protocol, a high-performance storage protocol designed specifically for solid-state drives (SSDs), utilizes the PCIe interface to connect directly to the CPU, eliminating the protocol conversion layer of traditional SATA or SAS interfaces. This significantly reduces I / O latency and increases data transfer rates, fully utilizing the high-speed read and write capabilities of SSDs. Consequently, it is widely used in modern computing systems.

[0003] RAID technology combines multiple physical hard drives into a single logical storage unit, improving the storage system's data read and write speeds while enhancing data redundancy and fault tolerance. A typical RAID system consists of a host server, a RAID controller, and multiple NVMe drives. The RAID controller manages all NVMe drives in the downstream array, establishes and allocates management queues and I / O queues, and schedules drives and performs data access operations based on different RAID levels. For RAID levels that require data parity, the controller also handles data parity calculation, data splitting and reassembly, error management, and data recovery.

[0004] However, I / O requests within a RAID group are significantly coupled and dependent. When a host initiates a data access request to a hard drive in a RAID group, it typically triggers additional requests to other hard drives in the same group. This complex request correlation makes I / O operations within a RAID group highly dynamic and uncertain. In addition, RAID systems need to be compatible with hard drives of different models and brands, which have significant differences in performance and I / O access latency. For example, the latency of some hard drives may be only 4t (t represents a unit of time), while the latency of other hard drives may be as high as 10t. This difference results in inconsistent response and processing speeds for I / O requests from different hard drives in the same RAID group, which in turn affects resource allocation within the RAID controller and the performance of the entire RAID group. Summary of the Invention

[0005] The present application provides a data request processing method, apparatus, device and storage medium to at least solve the above technical problems existing in the prior art.

[0006] According to a first aspect of the present application, a method for processing a data request is provided, the method comprising: The RAID controller builds a response delay queue; The RAID controller receives a data input / output request from a host and decomposes the data input / output request into a plurality of sub-requests; The RAID controller determines a first sub-request and a second sub-request from the plurality of sub-requests, the first sub-request being associated with the second sub-request; Comparing the response delay time of the hard disk device corresponding to the first sub-request and the hard disk device corresponding to the second sub-request; performing a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue, so that completion times of processing the first sub-request and the second sub-request are consistent; After the first sub-request and the second sub-request are processed, new verification data is calculated and the corresponding buffer is released.

[0007] In one embodiment, the RAID controller constructs a response delay queue, including: At the initialization moment of the RAID controller, obtaining the response delay time of each hard disk device in the RAID group; A response delay queue is constructed according to the response delay time.

[0008] In one embodiment, performing a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue includes: Putting the sub-request corresponding to the hard disk device with the shorter response delay time into the response delay queue, and setting the delay time to the absolute value of the difference between the two response delay times; Allocate another sub-request to the corresponding buffer and dispatch it to the hard disk device corresponding to the sub-request for data reading; When the delay time arrives, the sub-requests in the response delay queue are allocated to the second buffer zone and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0009] In one embodiment, when the hard disk device in the RAID group includes N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; decomposing the data input / output request into multiple sub-requests includes: The data input and output request is decomposed into at least a first read request for reading old data from the target data disk, a first write request for writing new data to the target data disk, a second read request for reading old verification data from the verification disk, and a second write request for writing new verification data to the verification disk.

[0010] In one embodiment, the RAID controller determines the first sub-request and the second sub-request from the multiple sub-requests, including: The RAID controller determines that the first read request and the second read request are associated according to the data dependency relationship calculated by verification, and obtains the first sub-request and the second sub-request.

[0011] In one embodiment, calculating new verification data and releasing the corresponding buffer includes: Performing an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; Performing a second XOR operation on the XOR operation result and the new data to be written into the target data disk to generate the new verification data; The first buffer storing the old data and the second buffer storing the old verification data are released.

[0012] In one embodiment, after releasing the first buffer storing the old data and the second buffer storing the old verification data, the method further includes: In response to the first write request, writing the new data to the target data disk; In response to the second write request, the new parity data is written to the parity disk.

[0013] According to a second aspect of the present application, a device for processing a data request is provided, the device comprising: A building module for a RAID controller to build a response delay queue; A decomposition module, configured for the RAID controller to receive a data input / output request from a host and decompose the data input / output request into multiple sub-requests; a determination module, configured for the RAID controller to determine a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request; A comparison module, configured to compare a response delay time of a hard disk device corresponding to the first sub-request and a response delay time of a hard disk device corresponding to the second sub-request; an execution module, configured to perform a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue, so that completion times of processing the first sub-request and the second sub-request are consistent; The processing module is used to calculate new verification data and release the corresponding buffer after the first sub-request and the second sub-request are processed.

[0014] In one embodiment, the building block is specifically configured to: At the initialization moment of the RAID controller, obtaining the response delay time of each hard disk device in the RAID group; A response delay queue is constructed according to the response delay time.

[0015] In one embodiment, the execution module is specifically configured to: Putting the sub-request corresponding to the hard disk device with the shorter response delay time into the response delay queue, and setting the delay time to the absolute value of the difference between the two response delay times; Allocate another sub-request to the first buffer, and dispatch it to the hard disk device corresponding to the sub-request for data reading; When the delay time arrives, the sub-requests in the response delay queue are allocated to the second buffer zone and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0016] In one embodiment, when the hard disk device in the RAID group includes N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the decomposition module is specifically configured to: The data input and output request is decomposed into at least a first read request for reading old data from the target data disk, a first write request for writing new data to the target data disk, a second read request for reading old verification data from the verification disk, and a second write request for writing new verification data to the verification disk.

[0017] In one embodiment, the determination module is specifically configured to: The RAID controller determines that the first read request and the second read request are associated according to the data dependency relationship calculated by verification, and obtains the first sub-request and the second sub-request.

[0018] In one embodiment, when calculating new verification data and releasing the corresponding buffer, the processing module is specifically configured to: Performing an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; Performing a second XOR operation on the XOR operation result and the new data to be written into the target data disk to generate the new verification data; The first buffer storing the old data and the second buffer storing the old verification data are released.

[0019] In one embodiment, the device further includes a writing module; after releasing the first buffer storing the old data and the second buffer storing the old verification data, the writing module is configured to: In response to the first write request, writing the new data to the target data disk; In response to the second write request, the new parity data is written to the parity disk.

[0020] According to a third aspect of the present application, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0021] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0022] The data request processing method, device, equipment and storage medium of the present application include: the RAID controller builds a response delay queue; the RAID controller receives the data input and output request of the host, and decomposes the data input and output request into multiple sub-requests; the RAID controller determines the first sub-request and the second sub-request from the multiple sub-requests, and the first sub-request is associated with the second sub-request; compares the response delay time of the hard disk device corresponding to the first sub-request and the hard disk device corresponding to the second sub-request; according to the comparison result and the response delay queue, performs a delay adjustment operation on the first sub-request and the second sub-request to make the completion time of processing the first sub-request and the second sub-request consistent; after the first sub-request and the second sub-request are processed, calculates new verification data and releases the corresponding buffer. The technical solution of the present application effectively solves the problem of buffer resource waste caused by the difference in response delay of different NVMe devices by introducing a delay queue and accurately controlling the dispatch timing of I / O requests, optimizes the resource utilization of the data buffer in the RAID controller, and improves the data transmission efficiency and throughput of the RAID system.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which: In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0025] Figure 1 The following is a schematic diagram of the implementation process of the data request processing method of the embodiment of the present application Figure 1 ; Figure 2 The following is a schematic diagram of the implementation process of the data request processing method of the embodiment of the present application Figure 2 ; Figure 3 The following is a schematic diagram of the implementation process of the data request processing method of the embodiment of the present application Figure 3 ; Figure 4 The following is a schematic diagram of the implementation process of the data request processing method of the embodiment of the present application Figure 4 ; Figure 5 The following is a schematic diagram of the implementation process of the data request processing method of the embodiment of the present application Figure 5 ; Figure 6 A schematic diagram showing the structure of a data request processing device according to an embodiment of the present application is shown; Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0027] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0028] PCIe: A high-speed serial computer expansion bus standard, also known as Peripheral Component Interconnect Express, uses an efficient point-to-point connection protocol and features high-speed, low-latency, and high-bandwidth transmission. It supports hot plugging and is widely used in a variety of high-performance components and peripherals in modern computer systems. It is the core technology for connecting modern high-performance hardware.

[0029] NVMe: NVMe (Non-Volatile Memory Express) is a communication protocol based on the PCIe (PCI-Express) interface, designed specifically for SSDs (solid-state drives) to provide high-speed, low-latency storage access.

[0030] RAID (Redundant Array of Independent Disks) is a data storage technology that combines multiple independent physical hard drives into a single logical storage unit. This improves the storage system's data read and write speeds, enhances data redundancy, and improves fault tolerance. Different RAID levels can achieve different performance and security goals.

[0031] This application provides a method for processing data requests, such as Figure 1 As shown, the method includes: Step 101: The RAID controller builds a response delay queue.

[0032] In this example, the RAID controller acts as a bridge between the host and NVMe hard drive devices, managing all hard drive devices in the downstream array. A response delay queue is a data structure created within the RAID controller (at the software level) to temporarily store specific I / O subrequests. In a RAID system, different hard drive devices have varying response delays due to factors such as model and brand. Therefore, by constructing a response delay queue and delaying processing for a specific period of time, we ensure that I / O requests with varying response delays are processed at similar or even simultaneous times.

[0033] Step 102: The RAID controller receives a data input / output request from a host, and decomposes the data input / output request into multiple sub-requests.

[0034] In this example, a data I / O request refers to a read or write operation command issued by the host to a hard disk device. Data I / O requests are broken down into multiple sub-requests because, in a RAID system, a single host request may involve operations on multiple hard disks. For example, a write operation in RAID 5 results in two read operations and two write operations.

[0035] When the RAID controller receives a data input / output request from the host, it breaks down the original data input / output request into a series of low-level read / write operations for specific hard disks according to the characteristics of the RAID level and data organization method, so that each sub-request can be processed and scheduled separately.

[0036] Step 103: The RAID controller determines a first sub-request and a second sub-request from the multiple sub-requests, where the first sub-request is associated with the second sub-request.

[0037] In this example, dependency refers to the data dependency between the processing results of two subrequests in subsequent operations; both must complete before the next step can proceed. Based on the RAID level's data organization and verification calculation rules, the RAID controller identifies the first and second subrequests as being related, preparing for their coordinated processing.

[0038] Step 104: Compare the response delay time of the hard disk device corresponding to the first sub-request and the response delay time of the hard disk device corresponding to the second sub-request.

[0039] In this example, response latency refers to the time it takes from the RAID controller sending an I / O request to a drive to the drive completing the request and returning the result. This time reflects the speed at which a specific drive processes requests. Response latency varies for different drive devices due to factors such as hardware performance and firmware algorithms.

[0040] After determining the associated first sub-request and second sub-request, the response delay times of their respective corresponding hard disk devices are compared, so that delay adjustment can be performed based on the delay time difference to make the processing completion time of the two sub-requests as consistent as possible.

[0041] Step 105: Perform a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue, so that the completion time of processing the first sub-request and the completion time of processing the second sub-request are consistent.

[0042] In this example, latency adjustment involves delaying the processing of the sub-request with the lower response latency based on a comparison between the response delay queue and the latency time, so that it completes at a similar or even identical time to the sub-request with the higher response latency. Specifically, based on the comparison, the sub-request targeting the faster-response drive is temporarily placed in the response delay queue, while the sub-request targeting the slower-response drive is immediately dispatched. This allows the results of both sub-requests to be returned simultaneously, significantly reducing idle buffer resources caused by inconsistent completion times for associated requests.

[0043] Step 106: After the first sub-request and the second sub-request are processed, new verification data is calculated and the corresponding buffer is released.

[0044] In this example, the buffer is a memory area within the RAID controller used to temporarily store data. The buffer is released after data processing is complete, improving memory resource utilization. After the first and second sub-requests are processed, the old data and checksum data read from them are used with the new data to be written to perform an exclusive-OR operation to generate new checksum data. The buffer storing the old data and checksum data is then released to make available buffer resources for subsequent requests.

[0045] The present application provides a method for processing data requests, including: a RAID controller constructs a response delay queue; the RAID controller receives a data input and output request from a host, and decomposes the data input and output request into multiple sub-requests; the RAID controller determines a first sub-request and a second sub-request from the multiple sub-requests, and the first sub-request is associated with the second sub-request; compares the response delay time of the hard disk device corresponding to the first sub-request and the hard disk device corresponding to the second sub-request; based on the comparison result and the response delay queue, performs a delay adjustment operation on the first sub-request and the second sub-request, so that the completion time of processing the first sub-request and the second sub-request is consistent; after the first sub-request and the second sub-request are processed, calculates new verification data and releases the corresponding buffer. The technical solution of the present application effectively solves the problem of buffer resource waste caused by the difference in response delays of different NVMe devices by introducing a delay queue and accurately controlling the dispatch timing of I / O requests, optimizes the resource utilization of the data buffer in the RAID controller, and improves the data transmission efficiency and throughput of the RAID system.

[0046] In one example, the construct response delay queue is as follows: Figure 2 Shown, including: Step 201: When the RAID controller is initialized, the response delay time of each hard disk device in the RAID group is obtained.

[0047] In this example, initialization refers to when the RAID controller starts up or when a RAID group is created or reorganized. During this phase, the RAID controller establishes communication with each drive in the RAID group and obtains relevant device information. Response latency refers to the time between when a drive receives an I / O request and when it begins processing it. It reflects the drive's response speed to I / O requests.

[0048] When the RAID controller is initialized, a specific test command or query command is sent to each hard disk device in the RAID group, and the time required for each hard disk device to return a response is recorded, thereby obtaining the response delay time of each hard disk device.

[0049] Step 202: Construct a response delay queue according to the response delay time.

[0050] In this example, the response delay time of each hard drive device is obtained and stored in a response delay queue according to a specific rule (such as sorting from smallest to largest). For example, a queue can be created to prioritize hard drives with shorter response delays over those with longer delays, or hard drives can be organized according to other specific scheduling algorithms. This constructed response delay queue provides a basis for subsequent delay adjustments to I / O requests, allowing the RAID controller to appropriately arrange the request processing order based on the delay characteristics of the hard drives.

[0051] In one example, the delay adjustment operation is performed on the first sub-request and the second sub-request according to the comparison result and the response delay queue, such as Figure 3 Shown, including: Step 301: Put the sub-request corresponding to the hard disk device with the shorter response delay time into the response delay queue, and set the delay time to the absolute value of the difference between the two response delay times.

[0052] In this example, the hard disk device with the shorter response delay time refers to the hard disk device with the shorter response delay time after comparison between the hard disk devices corresponding to two related sub-requests. The delay time is determined based on the difference in the response delay times of the two hard disk devices, specifically the absolute value of the difference between the two response delay times. For example, if the hard disk device corresponding to the first sub-request has a response delay time of 5ms, and the hard disk device corresponding to the second sub-request has a response delay time of 8ms, then the sub-request corresponding to the hard disk device with the shorter response delay time is the first sub-request, and the delay time is set to 3ms (the absolute value of 8ms - 5ms).

[0053] After determining the response delays of the hard disk devices corresponding to the first and second sub-requests, the sub-request with the shorter response delay is temporarily placed in the response delay queue. This delay adjustment allows the two sub-requests to complete at similar or even identical times. Setting the delay to the absolute value of the difference between the two response delays allows for precise control of the delays, thereby coordinating the completion times of the two sub-requests.

[0054] Step 302: Allocate another sub-request to a corresponding buffer, and dispatch it to the hard disk device corresponding to the sub-request for data reading.

[0055] In this example, the other sub-request is for the disk device with the longer response latency. The disk device for a sub-request with a longer response latency takes longer to process the I / O request. Therefore, to improve efficiency, this sub-request is first allocated to the corresponding buffer and then dispatched to the corresponding disk device for data reading.

[0056] Step 303: When the delay time arrives, the sub-requests in the response delay queue are allocated to corresponding buffers, and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0057] In this example, the arrival of the delay time refers to the time from when the sub-request is placed in the response delay queue and the timing begins, to when the set delay time has passed and the timing ends. The sub-request in the response delay queue refers to the sub-request corresponding to the hard disk device with the shorter response delay time in step 301. When the delay time is reached, the sub-request previously placed in the response delay queue is allocated to the corresponding buffer and dispatched to the corresponding hard disk device for data reading. The delay adjustment operation effectively solves the problem of buffer resource waste caused by the difference in response delays of different hard disk devices, improves the performance and throughput of the RAID system, and also enhances the resource utilization efficiency of the system.

[0058] In one example, when the hard disk devices in the RAID group include N data disks and n check disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; decomposing the data input and output request into multiple sub-requests includes: decomposing the data input and output request into at least a first read request to read old data from a target data disk, a first write request to write new data to the target data disk, a second read request to read old check data from the check disk, and a second write request to write new check data to the check disk.

[0059] In this example, N data disks is the number of physical hard disks in the RAID group that store user data (for example, three data disks), where N ≥ 3 (for example, RAID 5 requires at least three disks, and RAID 6 requires at least four disks). n parity disks is the number of physical hard disks that store redundant parity information, where 1 ≤ n ≤ 3 (for example, RAID 5 uses one parity disk, and RAID 6 uses two).

[0060] First read request to read old data from the target data disk: In a RAID system, data update operations often require reading old data first. This is because RAID's data verification mechanism requires reading old data and old verification data before writing new data to calculate the new verification data. The target data disk is the hard disk device that stores the data to be updated. The purpose of the first read request is to read the old data from the target data disk to prepare for the subsequent verification calculation and data update.

[0061] The first write request to write new data to the target data disk: After reading the old data, the system needs to write the new data to the target data disk. This step completes the data update, but the verification data has not been updated at this time.

[0062] A second read request to read old parity data from the parity disk: The parity disk is a hard disk device that stores parity data. This second read request is used to read old parity data associated with the target data disk from the parity disk. This step works in conjunction with the first read request to provide the necessary raw data for subsequent calculation of new parity data.

[0063] The second write request to write the new verification data to the verification disk: After calculating the new verification data, the system needs to write the new verification data to the verification disk. This step completes the update of the verification data and ensures the integrity and consistency of the data.

[0064] This decomposition ensures that the RAID system correctly maintains the relationship between data and parity data during data update operations. By reading the old data and old parity data first, and then writing the new data and new parity data, the system can ensure that if any operation fails, the data can be restored through the redundant parity data, thus ensuring data reliability and availability.

[0065] In one example, the RAID controller determines the first sub-request and the second sub-request from the multiple sub-requests, including: the RAID controller determines that the first read request and the second read request are associated based on the data dependency relationship calculated by verification, and obtains the first sub-request and the second sub-request.

[0066] In this example, data dependency in parity calculation means that in a RAID system, the generation of new parity data depends on both the old data and the old parity data. Specifically, the new parity data is obtained by performing a specific operation (such as an exclusive-OR operation) on the old data, the new data, and the old parity data. Therefore, a data dependency exists between the first read request for reading the old data and the second read request for reading the old parity data. This means that their read operations must be completed before the related write operations to ensure the correct calculation of the new parity data. The first sub-request and the second sub-request are two related requests determined from multiple sub-requests based on this data dependency.

[0067] After receiving the decomposed sub-requests, the RAID controller analyzes the relationships between them. For data update operations, the calculation of new parity data depends on both the old data and the old parity data. Therefore, based on this logical relationship in the parity calculation, the RAID controller determines that the first read request (reading the old data) and the second read request (reading the old parity data) are interrelated. The execution order and results of these two requests directly impact the subsequent generation and writing of new parity data. By identifying this relationship, the RAID controller can uniformly schedule and manage these two sub-requests, optimizing the overall data processing flow.

[0068] In one example, the new checksum data is calculated and the corresponding buffer is released, such as Figure 4 As shown, including: Step 401: Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request.

[0069] In this example, the XOR operation is a bitwise operation that compares each bit of two binary numbers and returns a 0 if the two bits are the same and a 1 if the two bits are different.

[0070] In a RAID system, the calculation of new parity data requires combining old data with the old parity data. By performing an XOR operation on the old data and the old parity data, an intermediate result can be obtained. This result reflects the difference between the old data and the old parity data.

[0071] Step 402: Perform a second XOR operation on the XOR operation result and the new data to be written into the target data disk to generate the new verification data.

[0072] In this example, the second XOR operation refers to performing a second XOR operation on the result of the first XOR operation with another piece of data. The new data to be written to the target data disk refers to the data that the host wishes to update to the target data disk. The new parity data refers to the parity data calculated based on the old data, the old parity data, and the new data, used to update the parity disk.

[0073] After obtaining the XOR result of the old data and the old checksum data, it is XORed again with the new data. This step aims to reflect the changes in the new data in the checksum data, thereby generating new checksum data corresponding to the new data. This ensures that after the data disk is updated with the new data, the checksum data on the checksum disk can still be used correctly for data verification and recovery.

[0074] Step 403: Release the first buffer storing the old data and the second buffer storing the old verification data.

[0075] In this example, the first buffer refers to a memory area used to store old data read by the first sub-request, and the second buffer refers to a memory area used to store old verification data read by the second sub-request.

[0076] After the calculation of the new verification data is completed, the old data and the old verification data are no longer needed. At this time, the first buffer storing the old data and the second buffer storing the old verification data are released, and these buffer resources can be returned to the system so that other data read and write operations or verification calculations can use these buffers.

[0077] In one example, after releasing the first buffer storing the old data and the second buffer storing the old verification data, Figure 5 As shown, the method further includes: Step 501: In response to the first write request, write the new data to the target data disk.

[0078] In this example, after calculating the new parity data and releasing the associated buffer, the system needs to actually write the new data to the target data disk. In response to the first write request, the RAID controller transfers the new data from the corresponding buffer to the specified location on the target data disk, completing the data update operation.

[0079] Step 502: In response to the second write request, write the new verification data to the verification disk.

[0080] In this example, after the new parity data is calculated and the associated buffer is released, the new parity data needs to be written to the parity disk. In response to the second write request, the RAID transfers the new parity data from the corresponding buffer to the designated location on the parity disk, completing the parity data update operation.

[0081] In an example, the following example uses an NVMe hard disk device as a data disk: 、 、 and check disk Taking the RAID 5 as an example, the processing method of the above data request is described in detail: Step 601: On the RAID controller At the initialization moment, the RAID controller obtains the response delay time of each downstream hard disk device processing I / O request response through NVMe management commands, which are: 、 、 and , and based on 、 、 and Create a delay queue for the current RAID group .

[0082] Step 602: Receives a data disk I / O request from the host According to the data organization and verification mechanism of RAID5, Decompose into 、 、 、 Four I / O sub-requests. Among them, For NVMe hard disk devices Read old data; Used to write new data ; For use from the check disk Read out the old calibration data; Used to write the calculated new verification data into .

[0083] Step 603: RAID Controller determination and This is a dependent RAID request. This is because in RAID5, writing new data requires reading the old data and old parity data for verification calculations. Therefore, there is a close data dependency between the two read requests. The goal of this solution is to enable these two requests to be processed at similar or even the same time.

[0084] Step 604: Get the disk and The response delay times of I / O requests are and , compare the sizes of the two. > , then execute step 605; otherwise, execute step 606.

[0085] Step 605: Sub-request Put in delay queue Implement delayed scheduling, and set the delay time to At the same time, the request Allocate the required buffer , and Distribute to Perform data processing. The response delay is large, so the request to read the old verification data will be sent to Delay, allowing requests to read old data Process in advance to avoid data processing delays caused by waiting.

[0086] Step 606: Sub-request Put in delay queue Implement delayed scheduling, and set the delay time to At the same time, the request Allocate the required buffer , and Distribute to Perform data processing. The response delay is large, so the request to read the old data will be sent first. Delay, allowing requests to read old verification data Process in advance to avoid data processing delays caused by waiting.

[0087] Step 607: Wait for the set delay time arrive.

[0088] Step 608: When the delay time Upon arrival, from the delay queue The sub-request in the delay queue is taken out and assigned to the corresponding buffer, and the sub-request is dispatched to the corresponding hard disk device.

[0089] For example, if step 605 is executed and the delay time Upon arrival, from the delay queue Remove the subrequest , will request Allocate to buffer , and Distribute to Perform data processing.

[0090] If step 606 is executed and the delay time Upon arrival, from the delay queue Remove the subrequest , will request Allocate to buffer , and Distribute to Perform data processing.

[0091] Step 609: Hard disk device and Complete their respective I / O sub-requests at similar or identical times and After that, the old data read is XORed with the old verification data, and then the result is XORed with the new data to be written again to generate new verification data. The buffer is then released and , completing the data processing at this stage.

[0092] Step 610: Respond to the sub-request , write new data to the target data disk , complete the data update operation. In response to the sub-request , write the new verification data to the verification disk , complete the update operation of the verification data to ensure the integrity and reliability of the data.

[0093] Through the above steps, efficient processing of data requests is achieved in the RAID 5 system. While ensuring data security, the use of the buffer zone is optimized, and the performance and throughput of the system are improved.

[0094] In an example, the following example uses an NVMe hard disk device as a data disk: 、 、 and check disk 、 Taking the RAID 6 structure as an example, the processing flow of the above data request is described in detail from another perspective: Step 701: On the RAID controller At the initialization moment, the RAID controller obtains the response delay time of each downstream hard disk device processing I / O request response through NVMe management commands, which are: 、 and (data disk) and 、 (two parity disks), and at the same time establish a delay queue for the current RAID group based on these delay times .

[0095] Step 702: Receives a data disk I / O request from the host According to the data organization and verification mechanism of RAID6, Decomposed into six I / O sub-requests: (from Read old data), (Towards Write new data), (from Read old check P), (from Read old verification Q), (Towards Write new checksum P), (Towards Write new checksum Q).

[0096] Step 703: RAID controller determination 、 and These are associated RAID requests. This is because in RAID 6, writing new data requires first reading the old data, the old parity P, and the old parity Q to calculate the new parity P and Q. Therefore, there is a close data dependency between these three read requests. The goal of this solution is to enable these three requests to be processed at similar or even identical times.

[0097] Step 704: Obtain the target data disk , check disk and Response delay time 、 and , and calculate the maximum value among the three .

[0098] Step 705: For the three read requests, perform the following operations respectively: If the request delay is less than , then put the request into the delay queue , and set its delay time to The delay time corresponding to the request (for example, The delay time is ).

[0099] If the request delay is equal to , then immediately allocate a buffer for the request (such as distribute , distribute , distribute ), and distribute it to the corresponding hard disk device.

[0100] Note: We no longer separate steps 705 and 706 here, but instead process them together. In fact, RAID 5 requires two requests, so we use branching; RAID 6 requires three requests, so we use a loop or branching process.

[0101] Step 706: Wait for the delay time set for each request in the delay queue to arrive.

[0102] Step 707: When the delay time of a request in the delay queue arrives, the request is taken out from the delay queue, a buffer is allocated for it, and the request is dispatched to the corresponding hard disk device.

[0103] Step 708: Hard disk device 、 and Complete their respective I / O sub-requests at similar or identical times ( 、 、 ) After that, the new check data P and Q are calculated using the old data, old check P and old check Q, and the new data to be written. Then the buffer is released 、 and .

[0104] Step 709: Respond to the sub-request , write new data to the target data disk ; In response to a subrequest , write the new verification data P into the verification disk ; In response to a subrequest , write the new verification data Q to the verification disk .

[0105] This application also provides a data request processing device, such as Figure 6 As shown, the device includes: A construction module 801 is used for the RAID controller to construct a response delay queue; A decomposition module 802 is configured for the RAID controller to receive a data input / output request from a host and decompose the data input / output request into multiple sub-requests; A determination module 803 is configured for the RAID controller to determine a first sub-request and a second sub-request from the multiple sub-requests, wherein the first sub-request is associated with the second sub-request; Comparison module 804, configured to compare the response delay time of the hard disk device corresponding to the first sub-request with the response delay time of the hard disk device corresponding to the second sub-request; An execution module 805 is configured to perform a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue, so that the completion time of processing the first sub-request and the completion time of processing the second sub-request are consistent; The processing module 806 is configured to calculate new verification data and release corresponding buffers after the first sub-request and the second sub-request are processed.

[0106] In one example, the building module 801 is specifically configured to: At the initialization moment of the RAID controller, obtaining the response delay time of each hard disk device in the RAID group; A response delay queue is constructed according to the response delay time.

[0107] In one example, the execution module 805 is specifically configured to: Putting the sub-request corresponding to the hard disk device with the shorter response delay time into the response delay queue, and setting the delay time to the absolute value of the difference between the two response delay times; Allocate another sub-request to the first buffer, and dispatch it to the hard disk device corresponding to the sub-request for data reading; When the delay time arrives, the sub-requests in the response delay queue are allocated to the second buffer zone and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0108] In one example, when the hard disk devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the decomposition module 802 is specifically configured to: The data input and output request is decomposed into at least a first read request for reading old data from the target data disk, a first write request for writing new data to the target data disk, a second read request for reading old verification data from the verification disk, and a second write request for writing new verification data to the verification disk.

[0109] In one example, the determination module 803 is specifically configured to: The RAID controller determines that the first read request and the second read request are associated according to the data dependency relationship calculated by verification, and obtains the first sub-request and the second sub-request.

[0110] In one example, when calculating new verification data and releasing the corresponding buffer, the processing module 806 is specifically configured to: Performing an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; Performing a second XOR operation on the XOR operation result and the new data to be written into the target data disk to generate the new verification data; The first buffer storing the old data and the second buffer storing the old verification data are released.

[0111] In one example, the apparatus further includes a writing module 807; after releasing the first buffer storing the old data and the second buffer storing the old verification data, the writing module 807 is configured to: In response to the first write request, writing the new data to the target data disk; In response to the second write request, the new parity data is written to the parity disk.

[0112] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0113] Figure 7 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0114] like Figure 7As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. Computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to bus 904.

[0115] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0116] The computing unit 901 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the data request processing method. For example, in some embodiments, the data request processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the data request processing method described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the data request processing method by any other suitable means (e.g., via firmware).

[0117] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0119] In the context of this application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0121] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0122] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0123] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0124] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0125] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for processing a data request, characterized in that: The method comprises: The RAID controller builds a response delay queue; The RAID controller receives a data input / output request from a host and decomposes the data input / output request into a plurality of sub-requests; The RAID controller determines a first sub-request and a second sub-request from the plurality of sub-requests, the first sub-request being associated with the second sub-request; Comparing the response delay time of the hard disk device corresponding to the first sub-request and the hard disk device corresponding to the second sub-request; performing a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue, so that completion times of processing the first sub-request and the second sub-request are consistent; After the first sub-request and the second sub-request are processed, new verification data is calculated and the corresponding buffer is released.

2. The method according to claim 1, characterized in that The RAID controller constructs a response delay queue, including: At the initialization moment of the RAID controller, obtaining the response delay time of each hard disk device in the RAID group; A response delay queue is constructed according to the response delay time.

3. The method according to claim 1, characterized in that The performing a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue includes: Putting the sub-request corresponding to the hard disk device with the shorter response delay time into the response delay queue, and setting the delay time to the absolute value of the difference between the two response delay times; Allocate another sub-request to the corresponding buffer and dispatch it to the hard disk device corresponding to the sub-request for data reading; When the delay time arrives, the sub-requests in the response delay queue are allocated to the second buffer zone and dispatched to the hard disk device corresponding to the sub-request for data reading.

4. The method according to claim 3, characterized in that When the hard disk device in the RAID group includes N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; decomposing the data input / output request into multiple sub-requests includes: The data input and output request is decomposed into at least a first read request for reading old data from the target data disk, a first write request for writing new data to the target data disk, a second read request for reading old verification data from the verification disk, and a second write request for writing new verification data to the verification disk.

5. The method according to claim 4, characterized in that The RAID controller determines a first sub-request and a second sub-request from the plurality of sub-requests, comprising: The RAID controller determines that the first read request and the second read request are associated according to the data dependency relationship calculated by verification, and obtains the first sub-request and the second sub-request.

6. The method according to claim 5, characterized in that The calculating of new verification data and releasing the corresponding buffer zone includes: Performing an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; Performing a second XOR operation on the XOR operation result and the new data to be written into the target data disk to generate the new verification data; The first buffer storing the old data and the second buffer storing the old verification data are released.

7. The method according to claim 6, characterized in that After releasing the first buffer storing the old data and the second buffer storing the old verification data, the method further includes: In response to the first write request, writing the new data to the target data disk; In response to the second write request, the new parity data is written to the parity disk.

8. A data request processing device, characterized in that: The device comprises: A building module for a RAID controller to build a response delay queue; A decomposition module, configured for the RAID controller to receive a data input / output request from a host and decompose the data input / output request into multiple sub-requests; a determination module, configured for the RAID controller to determine a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request; A comparison module, configured to compare a response delay time of a hard disk device corresponding to the first sub-request and a response delay time of a hard disk device corresponding to the second sub-request; an execution module, configured to perform a delay adjustment operation on the first sub-request and the second sub-request according to the comparison result and the response delay queue, so that completion times of processing the first sub-request and the second sub-request are consistent; The processing module is used to calculate new verification data and release the corresponding buffer after the first sub-request and the second sub-request are processed.

9. The device according to claim 8, characterized in that The building blocks are specifically used for: At the initialization moment of the RAID controller, obtaining the response delay time of each hard disk device in the RAID group; A response delay queue is constructed according to the response delay time.

10. The device according to claim 8, characterized in that The execution module is specifically used for: Putting the sub-request corresponding to the hard disk device with the shorter response delay time into the response delay queue, and setting the delay time to the absolute value of the difference between the two response delay times; Allocate another sub-request to the first buffer, and dispatch it to the hard disk device corresponding to the sub-request for data reading; When the delay time arrives, the sub-requests in the response delay queue are allocated to the second buffer zone and dispatched to the hard disk device corresponding to the sub-request for data reading.

11. The device according to claim 10, characterized in that When the hard disk device in the RAID group includes N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the decomposition module is specifically configured to: The data input and output request is decomposed into at least a first read request for reading old data from the target data disk, a first write request for writing new data to the target data disk, a second read request for reading old verification data from the verification disk, and a second write request for writing new verification data to the verification disk.

12. The device according to claim 11, characterized in that The determination module is specifically used for: The RAID controller determines that the first read request and the second read request are associated according to the data dependency relationship calculated by verification, and obtains the first sub-request and the second sub-request.

13. The device according to claim 12, characterized in that When calculating new verification data and releasing the corresponding buffer, the processing module is specifically used to: Performing an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; Performing a second XOR operation on the XOR operation result and the new data to be written into the target data disk to generate the new verification data; The first buffer storing the old data and the second buffer storing the old verification data are released.

14. The device according to claim 13, characterized in that The device further includes a writing module; after releasing the first buffer storing the old data and the second buffer storing the old verification data, the writing module is configured to: In response to the first write request, writing the new data to the target data disk; In response to the second write request, the new parity data is written to the parity disk.

15. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images