A data request processing method, device, apparatus, and storage medium

Abstract

The application provides a data request processing method and device, equipment and storage medium, relates to the computer technical field, and includes that the RAID controller constructs a response delay queue; a data input / output request of a host is received, and the data input / output request is decomposed into multiple sub-requests; a first sub-request and a second sub-request are determined from the multiple sub-requests, the first sub-request is associated with the second sub-request; the response delay time of the hard disk equipment corresponding to the first sub-request and the hard disk equipment corresponding to the second sub-request is compared; according to the comparison result and the response delay queue, a delay adjustment operation is performed on the first sub-request and the second sub-request, so that the completion time of processing the first sub-request and the second sub-request is consistent; after the first sub-request and the second sub-request are completed, new check data is calculated and the corresponding buffer area is released. The scheme introduces the delay queue and regulates the dispatching time of the I / O request, and solves the buffer area resource waste problem caused by the response delay difference of different hard disk equipment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for processing data requests. Background Technology

[0002] With the development of information technology, the demand for data storage and transmission is increasing. The NVMe protocol, a high-performance storage protocol designed specifically for solid-state drives (SSDs), directly connects to the CPU via the PCIe interface, eliminating the protocol conversion layer of traditional SATA or SAS interfaces. This significantly reduces I / O operation latency, improves data transfer rates, and fully utilizes the high-speed read and write capabilities of SSDs, leading to their widespread application in modern computing systems.

[0003] RAID technology improves data read / write speeds and enhances data redundancy and fault tolerance by combining multiple physical hard drives into a single logical storage unit. A typical RAID system consists of a host server, a RAID controller, and multiple NVMe hard drives. The RAID controller manages all NVMe hard drives in the downstream array, establishes and allocates management queues and I / O queues, and schedules hard drive access operations according to different RAID levels. For RAID levels with parity checks, the controller also handles parity calculations, data splitting and reassembly, error management, and data recovery.

[0004] However, I / O requests within a RAID group exhibit significant coupling and dependency. When a host initiates a data access request to one hard drive in the RAID group, it typically triggers additional requests to other hard drives within the same group. This complex request correlation makes I / O operations within a RAID group highly dynamic and uncertain. Furthermore, RAID systems need to be compatible with hard drives of different models and brands, which vary significantly in performance and I / O access latency. For example, some hard drives may have a latency of only 4t (t represents a unit of time), while others may have a latency as high as 10t. This difference leads to inconsistent response and processing speeds of different hard drives within the same RAID group to I / O requests, thus affecting the resource allocation within the RAID controller and the overall performance of the RAID group. Summary of the Invention

[0005] This application provides a data request processing method, apparatus, device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0006] According to a first aspect of this application, a method for processing a data request is provided, the method comprising:

[0007] RAID controllers build response latency queues;

[0008] The RAID controller receives data input / output requests from the host and decomposes the data input / output requests into multiple sub-requests;

[0009] The RAID controller determines a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request.

[0010] Compare the response latency of the hard drive device corresponding to the first sub-request with that of the hard drive device corresponding to the second sub-request;

[0011] Based on the comparison results and the response delay queue, a delay adjustment operation is performed on the first sub-request and the second sub-request so that the completion time of processing the first sub-request and the second sub-request is consistent.

[0012] After the first and second sub-requests are processed, new verification data is calculated and the corresponding buffer is released.

[0013] In one possible implementation, the RAID controller constructs a response latency queue, including:

[0014] At the initialization time of the RAID controller, the response latency of each hard drive in the RAID group is obtained;

[0015] Construct a response delay queue based on the response delay time.

[0016] In one possible implementation, performing a delay adjustment operation on the first sub-request and the second sub-request based on the comparison result and the response delay queue includes:

[0017] Sub-requests corresponding to hard disk devices with shorter response latency are placed in the response latency queue, and the latency is set to the absolute value of the difference between the response latency of the two devices.

[0018] The other sub-request is allocated to the corresponding buffer and dispatched to the hard disk device corresponding to the sub-request for data reading;

[0019] When the delay time is reached, the sub-request in the response delay queue is allocated to the second buffer and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0020] In one possible implementation, when the hard disk devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the step of decomposing the data input / output request into multiple sub-requests includes:

[0021] The data input / output request is decomposed into at least a first read request to read old data from the target data disk, a first write request to write new data to the target data disk, a second read request to read old verification data from the verification disk, and a second write request to write new verification data to the verification disk.

[0022] In one possible implementation, the RAID controller determines the first sub-request and the second sub-request from the plurality of sub-requests, including:

[0023] The RAID controller determines that the first read request and the second read request are related based on the data dependency relationship calculated by the verification, and obtains the first sub-request and the second sub-request.

[0024] In one possible implementation, calculating new verification data and releasing the corresponding buffer includes:

[0025] Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request;

[0026] The result of the XOR operation is then XORed with the new data to be written to the target data disk to generate the new verification data.

[0027] Release the first buffer storing the old data and the second buffer storing the old verification data.

[0028] In one possible implementation, after releasing the first buffer storing the old data and the second buffer storing the old check data, the method further includes:

[0029] In response to the first write request, the new data is written to the target data disk;

[0030] In response to the second write request, the new verification data is written to the verification disk.

[0031] According to a second aspect of this application, a data request processing apparatus is provided, the apparatus comprising:

[0032] Modules for building response latency queues for RAID controllers;

[0033] The decomposition module is used by the RAID controller to receive data input / output requests from the host and decompose the data input / output requests into multiple sub-requests.

[0034] The determination module is used by the RAID controller to determine a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request.

[0035] The comparison module is used to compare the response latency of the hard disk device corresponding to the first sub-request with that of the hard disk device corresponding to the second sub-request.

[0036] An execution module is configured to perform a delay adjustment operation on the first sub-request and the second sub-request based on the comparison result and the response delay queue, so that the completion time of processing the first sub-request and the second sub-request is consistent.

[0037] The processing module is used to calculate new verification data and release the corresponding buffer after the first and second sub-requests have been processed.

[0038] In one possible implementation, the building module is specifically used for:

[0039] At the initialization time of the RAID controller, the response latency of each hard drive in the RAID group is obtained;

[0040] Construct a response delay queue based on the response delay time.

[0041] In one possible implementation, the execution module is specifically used for:

[0042] Sub-requests corresponding to hard disk devices with shorter response latency are placed in the response latency queue, and the latency is set to the absolute value of the difference between the response latency of the two devices.

[0043] The other sub-request is allocated to the first buffer and dispatched to the corresponding hard disk device for data reading;

[0044] When the delay time is reached, the sub-request in the response delay queue is allocated to the second buffer and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0045] In one possible implementation, when the hard disk devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the decomposition module is specifically used for:

[0046] The data input / output request is decomposed into at least a first read request to read old data from the target data disk, a first write request to write new data to the target data disk, a second read request to read old verification data from the verification disk, and a second write request to write new verification data to the verification disk.

[0047] In one possible implementation, the determination module is specifically used for:

[0048] The RAID controller determines that the first read request and the second read request are related based on the data dependency relationship calculated by the verification, and obtains the first sub-request and the second sub-request.

[0049] In one possible implementation, when the processing module calculates new verification data and releases the corresponding buffer, it is specifically used for:

[0050] Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request;

[0051] The result of the XOR operation is then XORed with the new data to be written to the target data disk to generate the new verification data.

[0052] Release the first buffer storing the old data and the second buffer storing the old verification data.

[0053] In one possible embodiment, the device further includes a writing module; after releasing the first buffer storing the old data and the second buffer storing the old check data, the writing module is used to:

[0054] In response to the first write request, the new data is written to the target data disk;

[0055] In response to the second write request, the new verification data is written to the verification disk.

[0056] According to a third aspect of this application, an electronic device is provided, comprising:

[0057] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0058] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0059] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0060] This application discloses a data request processing method, apparatus, device, and storage medium. The method includes: a RAID controller constructing a response latency queue; the RAID controller receiving data input / output requests from the host and decomposing the data input / output requests into multiple sub-requests; the RAID controller determining a first sub-request and a second sub-request from the multiple sub-requests, associating the first sub-request with the second sub-request; comparing the response latency times of the hard disk devices corresponding to the first sub-request and the second sub-request; performing latency adjustment operations on the first and second sub-requests based on the comparison result and the response latency queue, so that the completion times of processing the first and second sub-requests are consistent; and calculating new parity data and releasing the corresponding buffers after the first and second sub-requests are processed. The technical solution of this application effectively solves the problem of buffer resource waste caused by the difference in response latency between different NVMe devices by introducing a latency queue and precisely controlling the dispatching timing of I / O requests, optimizes the resource utilization of the data buffer within the RAID controller, and improves the data transmission efficiency and throughput of the RAID system.

[0061] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0062] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein:

[0063] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0064] Figure 1 This illustration shows the implementation flow of the data request processing method according to an embodiment of this application. Figure 1 ;

[0065] Figure 2 This illustration shows the implementation flow of the data request processing method according to an embodiment of this application. Figure 2 ;

[0066] Figure 3 This illustration shows the implementation flow of the data request processing method according to an embodiment of this application. Figure 3 ;

[0067] Figure 4 This illustration shows the implementation flow of the data request processing method according to an embodiment of this application. Figure 4 ;

[0068] Figure 5This illustration shows the implementation flow of the data request processing method according to an embodiment of this application. Figure 5 ;

[0069] Figure 6 A schematic diagram of the structure of a data request processing apparatus according to an embodiment of this application is shown;

[0070] Figure 7 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0071] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0072] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0073] PCIe: A high-speed serial computer expansion bus standard, short for Peripheral Component Interconnect Express. It adopts an efficient point-to-point connection protocol and features high speed, low latency, and high bandwidth. It supports hot-swapping and is widely used in various high-performance components and peripherals in modern computer systems. It is the core technology for connecting modern high-performance hardware.

[0074] NVMe: NVMe (Non-Volatile Memory Express) is a communication protocol based on the PCIe (PCI-Express) interface, designed specifically for SSDs (Solid State Drives) to provide high-speed, low-latency storage access.

[0075] RAID stands for Redundant Array of Independent Disks, a data storage technology that combines multiple independent physical hard drives in a specific way to form a logical storage unit. This improves the data read and write speed of the storage system, enhances data redundancy, and improves fault tolerance. Different RAID levels can achieve different performance and security goals.

[0076] This application provides a method for processing data requests, such as... Figure 1 As shown, the method includes:

[0077] Step 101: The RAID controller builds a response latency queue.

[0078] In this example, the RAID controller acts as a bridge between the host and the NVMe hard drives, managing all the hard drives in the downstream array. The response latency queue is a data structure created internally (at the software level) within the RAID controller to temporarily store specific I / O sub-requests. In a RAID system, different hard drives have varying response latency due to factors such as model and brand. Therefore, by building a response latency queue and delaying processing for a specific time, it ensures that I / O requests with different response latencies can be processed at similar or even identical times.

[0079] Step 102: The RAID controller receives the host's data input / output request and decomposes the data input / output request into multiple sub-requests.

[0080] In this example, a data input / output request refers to a read / write operation command issued by the host to the hard drive device. The data input / output request is broken down into multiple sub-requests because in a RAID system, a single host request may involve operations on multiple hard drives; for example, a write operation in RAID 5 will trigger two read operations and two write operations.

[0081] When the RAID controller receives a data input / output request from the host, it breaks down the original data input / output request into a series of low-level read / write operations for specific hard drives according to the characteristics of the RAID level and the data organization method, so that each sub-request can be processed and scheduled separately.

[0082] Step 103: The RAID controller determines the first sub-request and the second sub-request from the plurality of sub-requests, and the first sub-request is associated with the second sub-request.

[0083] In this example, correlation refers to the data dependency between the processing results of two sub-requests in subsequent operations; both must be completed before the next step can proceed. Based on the data organization and verification calculation rules of the RAID level, the RAID controller identifies the first and second correlated sub-requests from multiple sub-requests, preparing for their subsequent collaborative processing.

[0084] Step 104: Compare the response latency of the hard disk device corresponding to the first sub-request with that of the hard disk device corresponding to the second sub-request.

[0085] In this example, response latency refers to the time elapsed from when the RAID controller dispatches an I / O request to a hard drive until the hard drive completes the request and returns the result. This time reflects the speed at which a specific hard drive processes requests. Response latency varies between different hard drives due to factors such as hardware performance and firmware algorithms.

[0086] After determining the associated first and second sub-requests, compare their respective response latency times on the hard disk devices so that subsequent latency adjustments can be made based on the difference in latency times, making the processing completion times of the two sub-requests as consistent as possible.

[0087] Step 105: Based on the comparison result and the response delay queue, perform a delay adjustment operation on the first sub-request and the second sub-request so that the completion time of processing the first sub-request and the second sub-request is consistent.

[0088] In this example, the delay adjustment operation compares the response delay queue with the delay time, and then delays the processing of sub-requests with lower response delays, ensuring they are processed around the same time as or even at the same time as sub-requests with higher response delays. Specifically, based on the comparison results, sub-requests targeting faster-responding hard drives are temporarily placed in the response delay queue for a period of time, while sub-requests targeting slower-responding hard drives are immediately dispatched. This allows the results of both sub-requests to be returned simultaneously, significantly reducing buffer resource idleness caused by inconsistent completion times of related requests.

[0089] Step 106: After the first and second sub-requests are processed, calculate the new verification data and release the corresponding buffer.

[0090] In this example, the buffer is a memory area in the RAID controller used to temporarily store data. Releasing the buffer after data processing is complete improves memory resource utilization. After the first and second sub-requests are processed, the old data and old parity data read from them, along with the new data to be written, are XORed to generate new parity data. Then, the buffer storing the old data and old parity data is released to provide available buffer resources for subsequent request processing.

[0091] This application provides a data request processing method, including: a RAID controller constructing a response latency queue; the RAID controller receiving data input / output requests from the host and decomposing the data input / output requests into multiple sub-requests; the RAID controller determining a first sub-request and a second sub-request from the multiple sub-requests, associating the first sub-request with the second sub-request; comparing the response latency times of the hard disk devices corresponding to the first sub-request and the second sub-request; performing latency adjustment operations on the first and second sub-requests based on the comparison result and the response latency queue, so that the completion times of processing the first and second sub-requests are consistent; and calculating new parity data and releasing the corresponding buffers after the first and second sub-requests are processed. The technical solution of this application effectively solves the problem of buffer resource waste caused by the difference in response latency between different NVMe devices by introducing a latency queue and precisely controlling the dispatching timing of I / O requests, optimizes the resource utilization of the data buffer within the RAID controller, and improves the data transmission efficiency and throughput of the RAID system.

[0092] In one example, the construction of a response latency queue, such as Figure 2 As shown, it includes:

[0093] Step 201: At the initialization time of the RAID controller, obtain the response latency time of each hard disk device in the RAID group.

[0094] In this example, initialization refers to the time when the RAID controller starts up or when the RAID group is created / reassembled. During this stage, the RAID controller establishes communication connections with each hard drive in the RAID group and obtains relevant device information. Response latency refers to the time interval between when a hard drive receives an I / O request and when it begins processing that request; it reflects the hard drive's response speed to I / O requests.

[0095] During RAID controller initialization, specific test or query commands are sent to each hard drive in the RAID group, and the time required for each hard drive to return a response is recorded, thereby obtaining the response latency of each hard drive.

[0096] Step 202: Construct a response delay queue based on the response delay time.

[0097] In this example, based on the obtained response latency of each hard drive device, the response latency times are stored in a response latency queue according to certain rules (such as sorting from smallest to largest). For example, a queue can be created to prioritize hard drives with shorter response latency times and prioritize those with longer response latency times, or the hard drives can be organized according to other specific scheduling algorithms. The constructed response latency queue will provide a basis for subsequent latency adjustment operations on I / O requests, enabling the RAID controller to reasonably arrange the processing order of requests based on the latency characteristics of the hard drive devices.

[0098] In one example, the delay adjustment operation is performed on the first sub-request and the second sub-request based on the comparison result and the response delay queue, such as... Figure 3 As shown, it includes:

[0099] Step 301: Place the sub-request corresponding to the hard disk device with the shorter response latency into the response latency queue, and set the latency to the absolute value of the difference between the two response latency times.

[0100] In this example, the hard drive with the shorter response latency refers to the one with the smaller response latency among the hard drives corresponding to two related sub-requests. The latency is set based on the difference between the response latencies of the two hard drives, specifically the absolute value of the difference between their response latencies. For example, if the hard drive with the first sub-request has a response latency of 5ms and the hard drive with the second sub-request has a response latency of 8ms, then the sub-request corresponding to the hard drive with the shorter response latency is the first sub-request, and its latency is set to 3ms (the absolute value of 8ms - 5ms).

[0101] After determining the response latency of the hard disk devices corresponding to the first and second sub-requests, the sub-request with the shorter response latency is temporarily placed in the response latency queue. Through latency adjustment, the two sub-requests can be processed at similar or even identical times. Setting the latency to the absolute value of the difference between their response latency is to precisely control the latency and coordinate the completion times of the two sub-requests.

[0102] Step 302: Allocate the other sub-request to the corresponding buffer and dispatch it to the hard disk device corresponding to the sub-request for data reading.

[0103] In this example, the other sub-request refers to the sub-request corresponding to the hard drive device with a longer response latency. The hard drive device corresponding to the sub-request with a longer response latency requires more time to process the I / O request. Therefore, to improve efficiency, this sub-request is first allocated to the corresponding buffer, and then dispatched to the hard drive device corresponding to the sub-request for data reading.

[0104] Step 303: When the delay time is reached, the sub-requests in the response delay queue are allocated to the corresponding buffers and dispatched to the hard disk devices corresponding to the sub-requests for data reading.

[0105] In this example, the delay time is defined as the time elapsed after the sub-request is placed in the response delay queue and the timing begins. The sub-requests in the response delay queue refer to the sub-requests corresponding to the hard drive devices with shorter response delays as described in step 301. When the delay time is reached, the sub-requests previously placed in the response delay queue are allocated to their corresponding buffers and dispatched to the corresponding hard drive devices for data reading. This delay adjustment effectively solves the problem of wasted buffer resources caused by differences in response delays among different hard drive devices, improves the performance and throughput of the RAID system, and enhances the system's resource utilization efficiency.

[0106] In one example, when the hard drive devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the step of decomposing the data input / output request into multiple sub-requests includes: decomposing the data input / output request into at least a first read request to read old data from the target data disk, a first write request to write new data to the target data disk, a second read request to read old parity data from the parity disk, and a second write request to write new parity data to the parity disk.

[0107] In this example, N data disks are the number of physical hard drives in the RAID group that store user data (e.g., 3 data disks), satisfying N≥3 (e.g., RAID5 requires at least 3 disks, and RAID6 requires at least 4 disks). n parity disks are the number of physical hard drives that store redundancy check information, satisfying 1≤n≤3 (e.g., RAID5 uses 1 parity disk, and RAID6 uses 2).

[0108] The first read request, which retrieves old data from the target data disk, is crucial in RAID systems. Data update operations typically require reading old data first. This is because RAID's data verification mechanism demands that old data and parity data be read before writing new data to calculate the new parity data. The target data disk is the hard drive that stores the data to be updated; the purpose of the first read request is to retrieve old data from the target data disk to prepare for subsequent parity calculations and data updates.

[0109] The first write request involves writing new data to the target data disk: After reading the old data, the system needs to write the new data to the target data disk. This step completes the data update, but the verification data has not yet been updated.

[0110] A second read request is made to retrieve old check data from the check disk: the check disk is a hard disk device that stores check data. The second read request is used to retrieve old check data related to the target data disk from the check disk. This step works in conjunction with the first read request to provide the necessary raw data for subsequent calculation of new check data.

[0111] The second write request involves writing the new checksum data to the checksum disk. After calculating the new checksum data, the system needs to write it to the checksum disk. This step completes the update of the checksum data, ensuring data integrity and consistency.

[0112] This decomposition method ensures that the RAID system can correctly maintain the relationship between data and parity data during data update operations. By first reading the old data and old parity data, and then writing the new data and new parity data, the system can guarantee that in the event of any operation failure, the data can be recovered through redundant parity data, thereby ensuring data reliability and availability.

[0113] In one example, the RAID controller determines the first sub-request and the second sub-request from the plurality of sub-requests, including: the RAID controller determines that the first read request and the second read request are related based on the data dependency relationship calculated by the verification, and obtains the first sub-request and the second sub-request.

[0114] In this example, the data dependency in parity calculation refers to the fact that in a RAID system, the generation of new parity data depends on the old data and the old parity data. Specifically, the new parity data is obtained by performing specific operations (such as XOR operations) on the old data, the new data, and the old parity data. Therefore, there is a data dependency between the first read request that reads the old data and the second read request that reads the old parity data; that is, their read operations must be completed before the relevant write operations to ensure the correct calculation of the new parity data. The first sub-request and the second sub-request are two related requests determined from multiple sub-requests based on this data dependency.

[0115] After receiving multiple sub-requests, the RAID controller analyzes the relationships between them. For data update operations, the calculation of new parity data depends on the old data and the old parity data. Therefore, based on this logical relationship of parity calculation, the RAID controller determines that the first read request (reading old data) and the second read request (reading old parity data) are interrelated. The execution order and results of these two requests directly affect the subsequent generation and writing of new parity data. By identifying this relationship, the RAID controller can uniformly schedule and manage these two sub-requests to optimize the entire data processing flow.

[0116] In one example, the calculation of new verification data and the release of the corresponding buffer, such as... Figure 4 As shown, it includes:

[0117] Step 401: Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request.

[0118] In this example, the XOR operation is a bitwise operation that compares each bit of two binary numbers. If the two bits are the same, the result is 0; if the two bits are different, the result is 1.

[0119] In a RAID system, calculating new parity data requires combining old data and old parity data. By performing an XOR operation on the old data and old parity data, an intermediate result can be obtained, which reflects the difference between the old data and the old parity data.

[0120] Step 402: Perform a second XOR operation on the result of the XOR operation and the new data to be written to the target data disk to generate the new verification data.

[0121] In this example, a double XOR operation means performing a second XOR operation with the result of the first XOR operation on another piece of data. The new data to be written to the target data disk refers to the data that the host wants to update to the target data disk. The new checksum data refers to the checksum data calculated based on the old data, the old checksum data, and the new data, used to update the checksum disk.

[0122] After obtaining the XOR result of the old data and the old checksum data, a second XOR operation is performed with the new data. The purpose of this step is to reflect the changes in the new data in the checksum data, thereby generating new checksum data corresponding to the new data. In this way, it can be ensured that after the data disk is updated to the new data, the checksum data in the checksum disk can still be correctly used for data verification and recovery.

[0123] Step 403: Release the first buffer storing the old data and the second buffer storing the old verification data.

[0124] In this example, the first buffer refers to the memory area used to store the old data read by the first sub-request, and the second buffer refers to the memory area used to store the old check data read by the second sub-request.

[0125] After the calculation of the new checksum is completed, the old data and old checksum are no longer needed. At this point, releasing the first buffer storing the old data and the second buffer storing the old checksum returns these buffer resources to the system so that other data read / write operations or checksum calculations can use these buffers.

[0126] In one example, after releasing the first buffer storing the old data and the second buffer storing the old checksum data, as follows: Figure 5 As shown, the method further includes:

[0127] Step 501: In response to the first write request, write the new data to the target data disk.

[0128] In this example, after calculating the new parity data and releasing the relevant buffers, the system needs to actually write the new data to the target data disk. In response to the first write request, the RAID controller transfers the new data from the corresponding buffer to the specified location on the target data disk, completing the data update operation.

[0129] Step 502: In response to the second write request, write the new verification data to the verification disk.

[0130] In this example, after calculating the new parity data and releasing the relevant buffers, the new parity data needs to be written to the parity disk. In response to the second write request, the RAID transfers the new parity data from the corresponding buffer to the specified location on the parity disk, completing the parity data update operation.

[0131] In one example, using an NVMe hard drive as the data disk... , , and verification disk Taking RAID 5 as an example, the method for processing the above data requests will be described in detail:

[0132] Step 601: On the RAID controller At initialization, the RAID controller obtains the response latency of each downstream hard drive device in processing I / O requests via NVMe management commands, in the following order: , , and At the same time, based on , , and Establish a delay queue for the current RAID group .

[0133] Step 602: Received a data write-to-disk I / O request from the host Following the RAID5 data organization and verification mechanism, Decomposed into , , , Four I / O sub-requests. Among them, Used from NVMe hard drive devices Read old data; Used to write new data ; Used from the verification disk Read out the old verification data; Used to write the calculated new verification data .

[0134] Step 603: RAID Controller determination and This is a dependent RAID request. This is because in RAID5, writing new data requires first reading the old data and old parity data for parity calculation. Therefore, there is a close data dependency between these two read requests. The goal of this solution is to enable these two requests to be processed at similar or even the same time.

[0135] Step 604: Obtain the disk and The response latency for the I / O requests are respectively and Compare the sizes of the two. If > If the condition is met, proceed to step 605; otherwise, proceed to step 606.

[0136] Step 605: Sub-request Add to the delay queue Implement delayed scheduling, with the delay time set to... At the same time, a request will be made. Allocate the required buffer and will Distributed to Perform data processing. Because... The response delay is relatively large, so the request to read the old verification data will be processed first. Delay, allowing requests to read old data Process data in advance to avoid delays caused by waiting.

[0137] Step 606: Sub-request Add to the delay queue Implement delayed scheduling, with the delay time set to... At the same time, a request will be made. Allocate the required buffer and will Distributed to Perform data processing. Because... The response delay is relatively large, so requests to read old data will be processed first. Delaying requests that read old verification data Process data in advance to avoid delays caused by waiting.

[0138] Step 607: Wait for the set delay time. arrive.

[0139] Step 608: When the delay time Upon arrival, from the delay queue The sub-requests are retrieved from the delay queue, allocated to the corresponding buffers, and then dispatched to the corresponding hard disk devices.

[0140] For example, if step 605 is executed and a delay time is applied... Upon arrival, from the delay queue Extracting child requests , will request Allocate to buffer and will Distributed to Perform data processing.

[0141] If step 606 is executed and a delay time is applied... Upon arrival, from the delay queue Extracting child requests , will request Allocate to buffer and will Distributed to Perform data processing.

[0142] Step 609: Hard Disk Device and Each I / O sub-request is completed at approximately the same time or simultaneously. and Next, the old data and the old checksum are XORed together, and then the result is XORed again with the new data to be written to generate new checksum. The buffer is then released. and This completes the data processing for this stage.

[0143] Step 610: Respond to the sub-request Write the new data to the target data disk. This completes the data update operation. It responds to the sub-request. Write the new verification data to the verification disk. This completes the data update operation to ensure data integrity and reliability.

[0144] Through the above steps, efficient processing of data requests is achieved in the RAID5 system. While ensuring data security, the use of the buffer is optimized, and the system performance and throughput are improved.

[0145] In one example, using an NVMe hard drive as the data disk... , , and verification disk , Taking RAID6 as an example, let's describe the data request processing flow in detail from another perspective:

[0146] Step 701: On the RAID controller At initialization, the RAID controller obtains the response latency of each downstream hard drive device in processing I / O requests via NVMe management commands, in the following order: , and (Data disk) and , (Two parity disks), and simultaneously establish a delay queue for the current RAID group based on these latency times. .

[0147] Step 702: Received a data write-to-disk I / O request from the host Following the RAID6 data organization and verification mechanism, Decomposed into six I / O sub-requests: (from Read old data) (Towards Write new data) (from Read old verification P), (from Read old verification (Q) (Towards Write a new checksum P). (Towards Write a new checksum Q).

[0148] Step 703: RAID Controller determination , and This is a correlated RAID request. This is because in RAID6, writing new data requires first reading the old data, the old parity P, and the old parity Q to calculate the new parity P and Q. Therefore, there is a close data dependency between these three read requests. The goal of this solution is to enable these three requests to be processed at similar or even the same time.

[0149] Step 704: Obtain the target data disk Verification disk and Response latency , and And calculate the maximum value among the three. .

[0150] Step 705: For the three read requests, perform the following operations respectively:

[0151] If the delay time of the request is less than Then the request will be placed in the delayed queue. And set its delay time to The corresponding delay time for this request (e.g., The delay time is ).

[0152] If the request delay time equals Then immediately allocate a buffer for the request (e.g. distribute , distribute , distribute (and distribute them to the corresponding hard drive devices).

[0153] Note: Here we no longer separate steps 705 and 706, but process them uniformly. In fact, RAID5 involves two requests, so we use branching; RAID6 involves three requests, so we use a loop or branching to process them.

[0154] Step 706: Wait for the set delay time for each request in the delay queue to arrive.

[0155] Step 707: When the delay time of a request in the delay queue arrives, retrieve the request from the delay queue, allocate a buffer for it, and dispatch it to the corresponding hard disk device.

[0156] Step 708: Hard Disk Device , and Each I / O sub-request is completed at approximately the same time or simultaneously. , , After that, using the old data, old checksum P and old checksum Q, and the new data to be written, calculate the new checksum data P and Q. Then release the buffer. , and .

[0157] Step 709: Respond to the subrequest Write the new data to the target data disk. ; Responding to sub-request Write the new check data P to the check disk. ; Responding to sub-request Write the new check data Q to the check disk. .

[0158] This application also provides a data request processing apparatus, such as... Figure 6 As shown, the device includes:

[0159] Module 801 is used by the RAID controller to build response latency queues;

[0160] The decomposition module 802 is used for the RAID controller to receive data input / output requests from the host and decompose the data input / output requests into multiple sub-requests.

[0161] The determination module 803 is used by the RAID controller to determine a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request.

[0162] Comparison module 804 is used to compare the response latency of the hard disk device corresponding to the first sub-request with that of the hard disk device corresponding to the second sub-request.

[0163] The execution module 805 is configured to perform a delay adjustment operation on the first sub-request and the second sub-request based on the comparison result and the response delay queue, so that the completion time of processing the first sub-request and the second sub-request is consistent.

[0164] The processing module 806 is used to calculate new verification data and release the corresponding buffer after the first sub-request and the second sub-request have been processed.

[0165] In one example, the building module 801 is specifically used for:

[0166] At the initialization time of the RAID controller, the response latency of each hard drive in the RAID group is obtained;

[0167] Construct a response delay queue based on the response delay time.

[0168] In one example, the execution module 805 is specifically used for:

[0169] Sub-requests corresponding to hard disk devices with shorter response latency are placed in the response latency queue, and the latency is set to the absolute value of the difference between the response latency of the two devices.

[0170] The other sub-request is allocated to the first buffer and dispatched to the corresponding hard disk device for data reading;

[0171] When the delay time is reached, the sub-request in the response delay queue is allocated to the second buffer and dispatched to the hard disk device corresponding to the sub-request for data reading.

[0172] In one example, when the hard drive devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the decomposition module 802 is specifically used for:

[0173] The data input / output request is decomposed into at least a first read request to read old data from the target data disk, a first write request to write new data to the target data disk, a second read request to read old verification data from the verification disk, and a second write request to write new verification data to the verification disk.

[0174] In one example, the determination module 803 is specifically used for:

[0175] The RAID controller determines that the first read request and the second read request are related based on the data dependency relationship calculated by the verification, and obtains the first sub-request and the second sub-request.

[0176] In one example, when the processing module 806 calculates new checksum data and releases the corresponding buffer, it is specifically used for:

[0177] Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request;

[0178] The result of the XOR operation is then XORed with the new data to be written to the target data disk to generate the new verification data.

[0179] Release the first buffer storing the old data and the second buffer storing the old verification data.

[0180] In one example, the device further includes a write module 807; after releasing the first buffer storing the old data and the second buffer storing the old check data, the write module 807 is configured to:

[0181] In response to the first write request, the new data is written to the target data disk;

[0182] In response to the second write request, the new verification data is written to the verification disk.

[0183] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0184] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0185] like Figure 7 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0186] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0187] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the data request processing method. For example, in some embodiments, the data request processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the data request processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the data request processing method by any other suitable means (e.g., by means of firmware).

[0188] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0189] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0190] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0191] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0192] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0193] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0194] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0195] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0196] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing data requests, characterized in that, The method includes: RAID controllers build response latency queues; The RAID controller receives data input / output requests from the host and decomposes the data input / output requests into multiple sub-requests; The RAID controller determines a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request. Compare the response latency of the hard drive device corresponding to the first sub-request with that of the hard drive device corresponding to the second sub-request; Based on the comparison results and the response delay queue, a delay adjustment operation is performed on the first sub-request and the second sub-request so that the completion time of processing the first sub-request and the second sub-request is consistent. The step of performing a delay adjustment operation on the first sub-request and the second sub-request based on the comparison result and the response delay queue includes: Sub-requests corresponding to hard disk devices with shorter response latency are placed in the response latency queue, and the latency is set to the absolute value of the difference between the response latency of the two devices. The other sub-request is allocated to the first buffer and dispatched to the corresponding hard disk device for data reading; When the delay time is reached, the sub-requests in the response delay queue are allocated to the second buffer and dispatched to the hard disk device corresponding to the sub-request for data reading; After the first and second sub-requests are processed, new verification data is calculated and the corresponding buffer is released.

2. The method according to claim 1, characterized in that, The RAID controller constructs a response latency queue, including: At the initialization time of the RAID controller, the response latency of each hard drive in the RAID group is obtained; Construct a response delay queue based on the response delay time.

3. The method according to claim 1, characterized in that, When the hard disk devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the step of decomposing the data input / output request into multiple sub-requests includes: The data input / output request is decomposed into at least a first read request to read old data from the target data disk, a first write request to write new data to the target data disk, a second read request to read old verification data from the verification disk, and a second write request to write new verification data to the verification disk.

4. The method according to claim 3, characterized in that, The RAID controller determines the first sub-request and the second sub-request from the plurality of sub-requests, including: The RAID controller determines that the first read request and the second read request are related based on the data dependency relationship calculated by the verification, and obtains the first sub-request and the second sub-request.

5. The method according to claim 4, characterized in that, The calculation of new verification data and release of the corresponding buffer includes: Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; The result of the XOR operation is then XORed with the new data to be written to the target data disk to generate the new verification data. Release the first buffer storing the old data and the second buffer storing the old verification data.

6. The method according to claim 5, characterized in that, After releasing the first buffer storing the old data and the second buffer storing the old check data, the method further includes: In response to the first write request, the new data is written to the target data disk; In response to the second write request, the new verification data is written to the verification disk.

7. A data request processing apparatus, characterized in that, The device includes: Modules for building response latency queues for RAID controllers; The decomposition module is used by the RAID controller to receive data input / output requests from the host and decompose the data input / output requests into multiple sub-requests. The determination module is used by the RAID controller to determine a first sub-request and a second sub-request from the plurality of sub-requests, wherein the first sub-request is associated with the second sub-request. The comparison module is used to compare the response latency of the hard disk device corresponding to the first sub-request with that of the hard disk device corresponding to the second sub-request. An execution module is configured to perform a delay adjustment operation on the first sub-request and the second sub-request based on the comparison result and the response delay queue, so that the completion time of processing the first sub-request and the second sub-request is consistent. The execution module is specifically used to: place the sub-request corresponding to the hard disk device with the shorter response latency into the response latency queue, and set the latency time to the absolute value of the difference between the two response latency times; allocate the other sub-request to the first buffer, and dispatch it to the hard disk device corresponding to the sub-request for data reading; when the latency time is reached, allocate the sub-request in the response latency queue to the second buffer, and dispatch it to the hard disk device corresponding to the sub-request for data reading; The processing module is used to calculate new verification data and release the corresponding buffer after the first and second sub-requests have been processed.

8. The apparatus according to claim 7, characterized in that, The building module is specifically used for: At the initialization time of the RAID controller, the response latency of each hard drive in the RAID group is obtained; Construct a response delay queue based on the response delay time.

9. The apparatus according to claim 8, characterized in that, When the hard drive devices in the RAID group include N data disks and n parity disks, N is an integer greater than or equal to 3, and n is an integer greater than or equal to 1 and less than or equal to 3; the decomposition module is specifically used for: The data input / output request is decomposed into at least a first read request to read old data from the target data disk, a first write request to write new data to the target data disk, a second read request to read old verification data from the verification disk, and a second write request to write new verification data to the verification disk.

10. The apparatus according to claim 9, characterized in that, The determination module is specifically used for: The RAID controller determines that the first read request and the second read request are related based on the data dependency relationship calculated by the verification, and obtains the first sub-request and the second sub-request.

11. The apparatus according to claim 10, characterized in that, When the processing module calculates new verification data and releases the corresponding buffer, it is specifically used for: Perform an XOR operation on the old data read by the first sub-request and the old verification data read by the second sub-request; The result of the XOR operation is then XORed with the new data to be written to the target data disk to generate the new verification data. Release the first buffer storing the old data and the second buffer storing the old verification data.

12. The apparatus according to claim 11, characterized in that, The device further includes a writing module; after releasing the first buffer storing the old data and the second buffer storing the old check data, the writing module is used to: In response to the first write request, the new data is written to the target data disk; In response to the second write request, the new verification data is written to the verification disk.

13. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

Images