Storage device IO request aggregation method, terminal and storage medium
By aggregating multiple IO requests in memory blocks and generating aggregate submission instructions, the problem of high consumption of IO interaction resources in the NVMe protocol is solved, and system performance and throughput is improved, especially when handling a large number of small IO requests.
Patent Information
- Application Number
- CN202411122918.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-15
AI Technical Summary
The existing NVMe protocol requires a large amount of resource consumption in the IO read/write interaction between the host and the storage device, resulting in reduced system performance, especially when handling a large number of small IO requests.
By storing information of multiple IO requests in a memory block, an aggregate submission instruction is generated and sent to a storage device, and aggregating multiple IO requests into one request, reducing the number of system calls and the interaction frequency between the CPU and the IO device.
It improves the overall efficiency and throughput of the system, reduces IO waiting time, improves system performance, and achieves high concurrency processing capabilities.
Smart Images

Figure CN118963672A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of servers, and in particular relates to a method for aggregating storage device IO requests, a terminal and a storage medium. Background Art
[0002] With the rapid development of technologies such as cloud computing and big data, the amount of data that needs to be stored is also growing explosively, and storage efficiency has become a more important factor affecting the development of various technologies. At present, the communication between the host and the storage device based on the NVME protocol must be completed through the SQ submission queue and the CQ completion queue. The host needs to put the SQE instruction executed by the storage device into the SQ queue, and then notify the storage device to read the SQE instruction from the SQ queue. The controller inside the storage device reads the SQE instruction from the SQ queue of the host memory through the PCIe message, and executes the instruction inside the storage device. Finally, the completed instruction result CQE is placed in the host's CQ queue through the PCIe message, and the host is notified to complete an IO interaction.
[0003] In the current NVMe protocol, one SQE and one CQE are required to complete an IO read / write interaction. The starting logical block and size of the host IO read / write operation are transmitted to the storage device through the SQE instruction. When there are a large number of read / write IOs between the host and the storage device, a large number of interactions between the host and the storage device are required, which consumes a lot of resources and reduces the performance of the system. Summary of the invention
[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method, a terminal and a storage medium for aggregating IO requests of a storage device to solve the above-mentioned technical problems.
[0005] In a first aspect, the present invention provides a method for aggregating storage device IO requests, comprising: Applying for a memory block, storing information of multiple IO requests in the memory block; Configure a commit instruction based on the address of the memory block and information of multiple IO requests stored in the memory block, and generate an aggregate commit instruction; generating an aggregation identifier for the aggregation submission instruction, and sending the aggregation submission instruction to a storage device; An aggregate completion instruction returned by the storage device is received, where the aggregate completion instruction includes response data to multiple IO requests.
[0006] In an optional implementation, applying for a memory block, storing the logical block start addresses of multiple IO requests in the memory block, includes: Confirm that the storage device supports IO aggregation and the maximum number of IOs supported by the storage device at a time; Set the memory block size based on the maximum IO number; Applying for a memory block of a corresponding size based on the memory block size; A new IO request is cached, a number is assigned to the IO request in a memory block, and information of the corresponding IO request is added after the number.
[0007] In an optional implementation, a new IO request is cached, a number is assigned to the IO request in a memory block, and information of the corresponding IO request is added after the number, including: Generate an IO request information linked list in the memory block; The number of the IO request is stored in the linked list, and the IO request information is added to the table item corresponding to the corresponding number, wherein the IO request information includes the starting address of the logic block and the number of the logic blocks.
[0008] In an optional implementation, the linked list includes: The first table entry is used to store the logical block start address of the IO request; The second table entry is used to store the status when an IO request error occurs; The third table item is used to store the number of the IO request; The fourth table item is used to store the number of logical blocks requested by the IO; The fifth table item is used to store the response data of the IO request; The sixth table entry is used to store the data address of the first IO request.
[0009] In an optional implementation, configuring a commit instruction based on the address of a memory block and information of multiple IO requests stored in the memory block to generate an aggregate commit instruction includes: Retrieve a commit instruction, and change the logic block start address in the commit instruction to the address of the memory block; Update the number of logical blocks in the commit instruction to the sum of the number of logical blocks of multiple IO requests; Changing the data address in the submit instruction to the data addresses of multiple IO requests, wherein the data addresses of the multiple IO requests are stored in sequence according to the IO request number sequence; The commit instruction after parameter update is cached in the commit queue as an aggregate commit instruction.
[0010] In an optional embodiment, the method further comprises: The storage device confirms receipt of the aggregation submission instruction based on the aggregation identifier; Read the memory block pointed to by the logical start address item of the aggregate commit instruction, and read the information of each IO request from the memory block one by one; Calculate the sub-data address of the IO request according to the order of the IO request, the data address in the aggregate submission instruction and the number of logical blocks of each IO request; Based on the information of the read IO request, data is read from the corresponding sub-data address and processed accordingly; If the IO request processing fails, a failure status flag is generated for the corresponding IO request.
[0011] In an optional implementation, receiving an aggregation completion instruction returned by a storage device, wherein the aggregation completion instruction includes response data to multiple IO requests, including: Response data of each IO request is parsed from the aggregation completion instruction.
[0012] In a second aspect, the present invention provides a storage device IO request aggregation system, comprising: A memory application module, used for applying for a memory block, storing information of multiple IO requests in the memory block; An instruction configuration module, used to configure a commit instruction based on the address of a memory block and information of multiple IO requests stored in the memory block, and generate an aggregate commit instruction; An aggregation marking module, used to generate an aggregation identifier for the aggregation submission instruction and send the aggregation submission instruction to a storage device; The data receiving module is used to receive an aggregation completion instruction returned by the storage device, where the aggregation completion instruction includes response data to multiple IO requests.
[0013] In a third aspect, a terminal is provided, including: processor, memory, wherein: The memory is used to store computer programs. The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.
[0014] According to a fourth aspect, a computer storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the methods described in the above aspects.
[0015] The beneficial effect of the present invention lies in that the storage device IO request aggregation method, terminal and storage medium provided by the present invention reduce the number of system calls by aggregating IO requests, thereby reducing the interaction frequency between the CPU and IO devices, and improving the overall efficiency and throughput of the system. In particular, when processing a large number of small IO requests, aggregation can reduce IO waiting time and effectively improve system performance; aggregating multiple IO requests into one IO request can allow a single thread or process to process multiple IO requests simultaneously, achieve high concurrency, and improve the response speed and concurrency performance of the system.
[0016] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.
[0019] Figure 2 It is a schematic diagram of the memory block structure of a method according to an embodiment of the present invention.
[0020] Figure 3 It is a schematic diagram of storing data addresses of a method according to an embodiment of the present invention.
[0021] Figure 4 is a schematic block diagram of a system according to an embodiment of the present invention.
[0022] Figure 5 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0023] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0025] The key terms appearing in the present invention are explained below.
[0026] IO: Input / Output, input and output; NVMe: Non Volatile Memory Express, NVMe storage protocol; SQ: Submission Queue, the queue for NVMe protocol to submit instructions; CQ: Completion Queue, the queue for NVMe protocol completion instructions; SQE: Submission Queue Entry, NVMe protocol submission instruction; CQE: Completion Queue Entry, NVMe protocol completion instruction; PCIe: peripheral component interconnect express, a high-speed serial computer expansion bus standard; Dword: Double Word means four bytes, the first four bytes of the SQE instruction; Doorbell: used to notify the NVMe device or host that a new command or data has been added to the queue; Data Pointer: includes PRP1 and PRP2, both of which are used to indicate the location of data in the host memory; Storage device: A device used to store information, in this article it refers to a solid-state drive.
[0027] The storage device IO request aggregation method provided in the embodiment of the present invention is executed by a computer device. Accordingly, the storage device IO request aggregation system runs in the computer device.
[0028] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a storage device IO request aggregation system. According to different requirements, the order of the steps in the flow chart may be changed, and some may be omitted.
[0029] like Figure 1 As shown, the method includes: Step 110, applying for a memory block, storing information of multiple IO requests in the memory block; Step 120, configuring a commit instruction based on the address of the memory block and information of multiple IO requests stored in the memory block, and generating an aggregate commit instruction; Step 130, generating an aggregation identifier for the aggregation submission instruction, and sending the aggregation submission instruction to a storage device; Step 140: Receive an aggregation completion instruction returned by the storage device, where the aggregation completion instruction includes response data to multiple IO requests.
[0030] In an efficient data processing system, in order to optimize the performance of IO (input / output) operations, a technology called "IO aggregation" is usually used. This technology allows the system to merge multiple small, scattered IO requests into a larger request, thereby reducing the number of interactions with storage devices (such as hard disks, SSDs, etc.), improving overall throughput and reducing latency.
[0031] (1) Memory block application and IO request storage: Memory block application: The system first applies for a large enough memory block in the memory management module based on the current load and expected IO request volume. This memory block is specifically used to store the IO request information to be aggregated.
[0032] IO request storage: Whenever an IO request arrives (whether from an application or other parts of the operating system), the system stores the information of the request (such as target address, data size, operation type, etc.) in the previously requested memory block. This information is organized into a format that is convenient for subsequent processing and aggregation.
[0033] (2) Generation of aggregate submission instructions: Configuring commit instructions: The system configures each individual commit instruction based on the information of multiple IO requests stored in the memory block and the association between these requests (such as whether they access the same storage device, whether they can be processed in parallel, etc.).
[0034] Generate aggregate commit instructions: Then, the system aggregates these individual commit instructions to generate an aggregate commit instruction containing multiple IO requests. This aggregate instruction contains all the necessary information that needs to be sent to the storage device at one time, as well as the execution order or parallelism requirements between these requests.
[0035] Aggregation ID generation: In order to accurately identify and process the response to this aggregation instruction later, the system will generate a unique aggregation identifier (such as a serial number or UUID) for this aggregation submission instruction.
[0036] (3) Sending commands and receiving responses: Sending an aggregate commit instruction: The system sends an aggregate commit instruction with an aggregate identifier to a target storage device through an appropriate communication protocol (such as SCSI, NVMe, etc.).
[0037] Receive aggregate completion instructions: After processing these aggregated IO requests, the storage device will generate an aggregate completion instruction and return it to the system. This instruction contains the response data for all original IO requests, and may also include execution status, error code and other information.
[0038] (4) Response processing: Parsing the aggregation completion instruction: After receiving the aggregation completion instruction, the system first identifies which aggregation submission instruction this instruction corresponds to based on the aggregation identifier. Then, it parses the response data in the instruction and extracts the response results of each IO request separately.
[0039] Update status and notification: The system updates the internal status (such as cache, file system metadata, etc.) based on the parsed response data, and may notify the original requester (such as application) of the result of the IO operation through a callback function or event notification mechanism.
[0040] To facilitate understanding of the present invention, the method for aggregating storage device IO requests provided by the present invention is further described below based on the principle of the method for aggregating storage device IO requests of the present invention and in combination with the process of aggregating storage device IO requests in the embodiment.
[0041] Specifically, the method for aggregating storage device IO requests includes: S1. Apply for a memory block, and store information of multiple IO requests in the memory block.
[0042] S101. Confirm that the storage device supports IO aggregation and the maximum number of IOs supported by the storage device at a time.
[0043] After initialization, the host uses the identify command to query whether the storage device supports the IO aggregation feature and the maximum IO request supported by the storage device at a time.
[0044] S102: Setting a memory block size based on a maximum IO quantity, and applying for a memory block of a corresponding size based on the memory block size.
[0045] In another implementation, the size of the memory block may also be directly specified. For example, if the host needs to send multiple IOs to the same storage device, a 4K continuous memory block is requested.
[0046] The memory block structure is as follows Figure 2 As shown, Sub_SLBA represents the SLBA (logical block starting address) of the aggregate IO, Sub_IO Status represents the status when an error occurs in the aggregate IO, IO_SN represents the sequence number of the aggregate IO, counting from 0, and Sub_NLB is the NLB (logical block number) requested by the aggregate IO.
[0047] In addition, in another embodiment of the present invention, an IO request information linked list is generated in a memory block; the IO request number is stored in the linked list, and the IO request information is added to the table entry corresponding to the corresponding number, and the IO request information includes a logical block starting address and a logical block quantity.
[0048] The linked list includes: a first table item for storing the logical block starting address of the IO request; a second table item for storing the state when an IO request error occurs; a third table item for storing the number of the IO request; a fourth table item for storing the number of logical blocks of the IO request; a fifth table item for storing the response data of the IO request; and a sixth table item for storing the data address of the first IO request.
[0049] S2. Configure a commit instruction based on the address of the memory block and information of multiple IO requests stored in the memory block, and generate an aggregate commit instruction.
[0050] Fill in the information of each sub-IO and set AIO to 1. The maximum number of single aggregated IOs is 32 or the total size of aggregated IOs exceeds the maximum IO supported by the storage device at a time. For each aggregated IO, the value of NLB is the sum of the Sub_NLB of the aggregated IO.
[0051] If the host IO is aggregated, the data of the first IO is stored in the LIST pointed to by PRP1 and PRP2 (assuming LIST is needed), and the subsequent IO only needs to store all the data in sequence after the first IO data in the LIST.
[0052] The host puts the SQE into the SQ and writes Doorbell to notify the storage device.
[0053] Specifically, the process includes the following: S201, calling a commit instruction, and changing the logic block start address in the commit instruction to the address of the memory block.
[0054] If the current SQE is for aggregated IO requests, the SLBA in the SQE in the NVMe protocol is changed from pointing to the starting position of the logical block to pointing to 4k continuous memory, and named as the starting logical block of aggregated IO (SLBA_AIO). The explanation of SLBA_AIO is shown in Table 1.
[0055] Table 1
[0056] S202: Update the number of logical blocks in the commit instruction to the sum of the numbers of logical blocks of the multiple IO requests.
[0057] The NLB of SQE in the NVMe protocol is changed from the original number of logical blocks operated to the number of logical blocks of all aggregated IOs. That is, when IO is aggregated, NLB is the sum of all Sub_NLBs.
[0058] S203: Change the data address in the submit instruction to the data addresses of multiple IO requests, and store the data addresses of the multiple IO requests in sequence according to the order of IO request numbers.
[0059] When aggregated IO reads and writes, the Data Pointer field in the SQE in the NVMe protocol is changed from the original data address that only points to one IO to the data addresses of multiple IOs, and stored in sequence according to the order of IO_SN. For example, Figure 3 As shown. Assume that 3 IOs are aggregated, IO_0 data is 12K, in the first two of the LIST pointed to by PRP1 and PRP2, IO_1 data is 4K, in the third of the LIST pointed to by PRP2, and IO_2 data is 4K, in the fourth of the LIST pointed to by PRP2. When the storage device reads data, only the first IO needs to read PRP1, and subsequent IOs only need to be read according to Sub_NLB calculation.
[0060] S204: Cache the commit instruction after parameter update into a commit queue as an aggregate commit instruction.
[0061] Use the 16th bit of Dword 12 to mark whether the SQE is an aggregated IO SQE. If the AIO bit is set to 1, it means that the SQE is an IO aggregated SQE. If AIO is set to 0, it means it is a normal SQE. The definition of SQE Dword 12 is shown in Table 2.
[0062] Table 2
[0063] S3, storage device processes aggregate IO requests.
[0064] The storage device confirms receipt of the aggregate commit instruction based on the aggregate identifier; reads the memory block pointed to by the logical start address item of the aggregate commit instruction, and reads the information of each IO request from the memory block one by one; calculates the sub-data address of the IO request according to the order of the IO requests, the data address in the aggregate commit instruction and the number of logical blocks of each IO request; reads data from the corresponding sub-data address based on the read IO request information and performs corresponding processing; if the IO request processing fails, generates a failure status identifier for the corresponding IO request.
[0065] After receiving the aggregate commit instruction, the storage device will perform a series of sophisticated and efficient operations to ensure the correctness of the data and the stability of the system. The following is a more detailed and expanded description of the process: (1) Confirmation and analysis based on aggregate identification: The storage device first parses the received data packet and verifies the validity of the aggregation identifier through the built-in verification mechanism. This identifier is the key to distinguishing tasks submitted by different aggregations, ensuring that each task can be correctly identified and processed.
[0066] Once the aggregate identifier is confirmed to be valid, the storage device enters the process of processing the aggregate commit instruction.
[0067] (2) Read the memory block pointed to by the logical start address: The storage device locates the corresponding memory block according to the logical start address specified in the aggregate commit instruction. This memory block stores basic information about all IO requests in the aggregate commit task, such as request type, size, target logical block address, etc.
[0068] Through an efficient memory access mechanism, the device reads the information of these IO requests one by one to prepare for subsequent data processing.
[0069] (3) Calculate the sub-data address of the IO request: For each IO request, the storage device calculates the actual data storage location, i.e., the sub-data address, based on its order in the aggregate commit instruction, the data address (usually the logical block address), and the number of logical blocks specified for each request. This calculation process takes into account the physical layout of the storage device, the data block size, and possible advanced storage features such as striping and mirroring to ensure the accuracy and efficiency of data access.
[0070] (4) Data reading and processing: Based on the calculated sub-data address, the storage device reads the data from the corresponding physical location. This may involve parallel access to multiple storage media (such as hard disks, SSDs, etc.) to speed up data reading.
[0071] The read data will be processed accordingly according to the type of IO request (such as read, write, modify, etc.). For example, for a write request, the data will be written to the specified storage location; for a read request, the data will be returned directly to the requester.
[0072] (5) Error handling and status identification: If an error (such as data corruption, storage medium failure, etc.) is encountered when processing an IO request, the storage device will immediately stop processing the request and generate a failure status indicator for it.
[0073] This failure status indicator will be recorded in the internal log.
[0074] At the same time, the storage device will decide whether to continue processing subsequent IO requests based on the configured policies (such as retry mechanism, error isolation, etc.) to ensure the overall stability and reliability of the system.
[0075] (6) Completion and feedback: When all IO requests are processed (whether successful or failed), the storage device updates the status of the aggregate commit command and sends a completion notification to the host.
[0076] This notification contains the processing result of each IO request, as well as any related error information and status identification, so that the requester can perform subsequent processing or recovery operations.
[0077] In a specific embodiment of the present invention, the storage device takes out SQE from SQ, and determines whether it is an aggregated IO according to the AIO field of SQE. If not, it is processed normally. If it is an aggregated IO, the 4k continuous memory pointed to by SLBA_AIO is read, and 128 bits are read each time to read data from the Data Pointer according to Sub_SLBA, Sub_NLB, and IO_SN for processing. When reading data, only the first IO needs to read PRP1, and other IOs only need to read in PRP2 according to Sub_NLB. If there is an error, the Status is written into the 128-bit Sub_IOStatus field corresponding to this sub-IO, and the bit corresponding to CQE AIO_Result is written to 1 according to IO_SN. If there is no error, the bit corresponding to CQE AIO_Result is written to 0. The process is looped until the total size of the processing is equal to the size of NLB. If there is no error in all aggregated IOs, the Status Field of CQE is 0, otherwise it is 1.
[0078] The storage device puts the CQE into the CQ queue and notifies the host.
[0079] When processing storage queue entries (SQEs) in a storage queue (SQ), the storage device pays special attention to the situation of aggregated IO (AIO) and performs corresponding optimization to improve efficiency. The following is a detailed expanded description of the process: (1) SQE withdrawal and AIO judgment: The storage device takes out SQEs (storage queue entries) from the SQ (storage queue) one by one for processing.
[0080] For each SQE, first check its AIO (aggregate IO) field. If the field indicates that this is not an aggregate IO request, the SQE is processed according to the normal process, that is, data is read or written directly according to the information in the SQE, and the corresponding completion queue (CQ) entry (CQE) is updated.
[0081] (2) Processing flow of aggregated IO: If the AIO field of the SQE indicates that this is an aggregate IO request, the storage device will first read the 4KB continuous memory area pointed to by the SLBA_AIO (Starting Logical Block Address_AIO) field in the SQE. This memory area contains detailed information about all sub-IO requests in the aggregate IO.
[0082] Each time, a 128-bit (16-byte) data block is read from this memory. These data blocks contain information such as Sub_SLBA (sub-starting logical block address), Sub_NLB (number of sub-logical blocks) and IO_SN (IO sequence number) of the sub-IO request.
[0083] (3) Data reading and processing: For the first sub-IO request in the aggregate IO, in addition to reading the above 128-bit information, it is also necessary to read PRP1 (physical region page table 1) to obtain the information of the physical location of the data. This is because the first sub-IO may need to start reading from a non-continuous physical address.
[0084] For subsequent sub-IO requests, since they usually follow the previous sub-IO, they can be read continuously in PRP2 (physical region page table 2) directly according to Sub_NLB (sub-logical block number) without reading PRP1 again.
[0085] Based on the data and information read, the storage device performs corresponding data processing operations (such as reading, writing, modifying, etc.).
[0086] (4) Error handling and status update: If an error (such as data corruption, storage medium failure, etc.) is encountered when processing a sub-IO request, the storage device will write the error status to the Sub_IO Status field in the 128-bit information corresponding to this sub-IO.
[0087] At the same time, according to the IO_SN (IO sequence number), the corresponding bit in the AIO_Result field of CQE is written as 1 to mark the failure of the sub-IO request processing.
[0088] If no error is encountered, the corresponding bit in the AIO_Result field of CQE is written as 0, indicating that the sub-IO request is processed successfully.
[0089] (5) Cycle processing and completion check: The storage device processes all sub-IO requests in the aggregate IO in a loop until the total size processed is equal to the size of the NLB (number of logical blocks) specified in the SQE.
[0090] If there are no errors in all the sub-requests of the aggregate IO, the Status Field of CQE is set to 0, indicating that the entire aggregate IO processing is successful.
[0091] If any subrequest has an error, the Status Field of CQE is set to 1, indicating that there is an error in the aggregate IO processing.
[0092] (6) Placement and notification of CQE: After completing the processing of all sub-IO requests, the storage device puts the updated CQE into the CQ (completion queue).
[0093] Then, the storage device notifies the host to check the CQ queue through an appropriate mechanism (such as interruption, polling, etc.) so that the host can obtain the processing results of the aggregated IO and perform subsequent operations.
[0094] S4. Receive an aggregation completion instruction returned by the storage device, where the aggregation completion instruction includes response data to multiple IO requests.
[0095] The host takes out the CQE from the CQ. If it is a CQE for an aggregated IO request, it determines whether the value of the Status Field of the CQE is 0. If it is 0, everything is correct. If it is not 0, the host reads the error cause of the corresponding sub-IO from the Sub_IO Status of the continuous 4k memory pointed to by SLBA_AIO according to the AIO_Result field of the CQE. If it is not an aggregated IO, it is processed normally. In this way, the execution status of each IO request can be traced.
[0096] In one embodiment of the present invention, the response data of each IO request may be extracted from the CQE, and the response data may be stored in a corresponding position of the memory block.
[0097] The CQE structure of the aggregated IO request is shown in Table 3.
[0098] Table 3
[0099] In some embodiments, the storage device IO request aggregation system may include multiple functional modules composed of computer program segments. The computer program of each program segment in the storage device IO request aggregation system may be stored in a memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) Function that aggregates storage device IO requests.
[0100] In this embodiment, the storage device IO request aggregation system can be divided into multiple functional modules according to the functions it performs, such as Figure 4 As shown. The functional modules of the system 400 may include: a memory application module 410, an instruction configuration module 420, an aggregation marking module 430 and a data receiving module 440. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can perform fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.
[0101] A memory application module, used for applying for a memory block, storing information of multiple IO requests in the memory block; An instruction configuration module, used to configure a commit instruction based on the address of a memory block and information of multiple IO requests stored in the memory block, and generate an aggregate commit instruction; An aggregation marking module, used to generate an aggregation identifier for the aggregation submission instruction and send the aggregation submission instruction to a storage device; The data receiving module is used to receive an aggregation completion instruction returned by the storage device, where the aggregation completion instruction includes response data to multiple IO requests.
[0102] Optionally, as an embodiment of the present invention, applying for a memory block, storing the logical block start addresses of multiple IO requests in the memory block, includes: Confirm that the storage device supports IO aggregation and the maximum number of IOs supported by the storage device at a time; Set the memory block size based on the maximum IO number; Applying for a memory block of a corresponding size based on the memory block size; A new IO request is cached, a number is assigned to the IO request in a memory block, and information of the corresponding IO request is added after the number.
[0103] Optionally, as an embodiment of the present invention, a new IO request is cached, a number is assigned to the IO request in a memory block, and information of the corresponding IO request is added after the number, including: Generate an IO request information linked list in the memory block; The number of the IO request is stored in the linked list, and the IO request information is added to the table item corresponding to the corresponding number, wherein the IO request information includes the starting address of the logic block and the number of the logic blocks.
[0104] Optionally, as an embodiment of the present invention, the linked list includes: The first table entry is used to store the logical block start address of the IO request; The second table entry is used to store the status when an IO request error occurs; The third table item is used to store the number of the IO request; The fourth table item is used to store the number of logical blocks requested by the IO; The fifth table item is used to store the response data of the IO request; The sixth table entry is used to store the data address of the first IO request.
[0105] Optionally, as an embodiment of the present invention, configuring a commit instruction based on the address of a memory block and information of multiple IO requests stored in the memory block to generate an aggregate commit instruction includes: Retrieve a commit instruction, and change the logic block start address in the commit instruction to the address of the memory block; Update the number of logical blocks in the commit instruction to the sum of the number of logical blocks of multiple IO requests; Changing the data address in the submit instruction to the data addresses of multiple IO requests, wherein the data addresses of the multiple IO requests are stored in sequence according to the IO request number sequence; The commit instruction after parameter update is cached in the commit queue as an aggregate commit instruction.
[0106] Optionally, as an embodiment of the present invention, the method further includes: The storage device confirms receipt of the aggregation submission instruction based on the aggregation identifier; Read the memory block pointed to by the logical start address item of the aggregate commit instruction, and read the information of each IO request from the memory block one by one; Calculate the sub-data address of the IO request according to the order of the IO request, the data address in the aggregate submission instruction and the number of logical blocks of each IO request; Based on the information of the read IO request, data is read from the corresponding sub-data address and processed accordingly; If the IO request processing fails, a failure status flag is generated for the corresponding IO request.
[0107] Optionally, as an embodiment of the present invention, receiving an aggregation completion instruction returned by a storage device, wherein the aggregation completion instruction includes response data to multiple IO requests, including: Response data of each IO request is parsed from the aggregation completion instruction.
[0108] Figure 5 The present invention is a schematic diagram of the structure of a terminal 500 provided in an embodiment of the present invention. The terminal 500 can be used to execute the storage device IO request aggregation method provided in an embodiment of the present invention.
[0109] The terminal 500 may include: a processor 510, a memory 520 and a communication unit 530. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0110] The memory 520 can be used to store the execution instructions of the processor 510, and the memory 520 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 520 are executed by the processor 510, the terminal 500 can execute some or all of the steps in the following method embodiments.
[0111] The processor 510 is the control center of the storage terminal, and uses various interfaces and lines to connect various parts of the entire electronic terminal. It runs or executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 510 can only include a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.
[0112] The communication unit 530 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals or send user data to other terminals.
[0113] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).
[0114] Therefore, the present invention reduces the number of system calls by aggregating IO requests, thereby reducing the interaction frequency between the CPU and IO devices, thereby improving the overall efficiency and throughput of the system. In particular, when processing a large number of small IO requests, aggregation can reduce IO waiting time and effectively improve system performance; aggregating multiple IO requests into one IO request allows a single thread or process to process multiple IO requests simultaneously, achieving high concurrency and improving the response speed and concurrency performance of the system. The technical effects that can be achieved by this embodiment can be found in the description above and will not be repeated here.
[0115] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.
[0116] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.
[0117] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.
[0118] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0119] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0120] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.
Claims
1. A method for aggregating storage device IO requests, characterized in that: include: Applying for a memory block, storing information of multiple IO requests in the memory block; Configure a commit instruction based on the address of the memory block and information of multiple IO requests stored in the memory block, and generate an aggregate commit instruction; generating an aggregation identifier for the aggregation submission instruction, and sending the aggregation submission instruction to a storage device; An aggregate completion instruction returned by the storage device is received, where the aggregate completion instruction includes response data to multiple IO requests.
2. The method according to claim 1, characterized in that Applying for a memory block, storing the logical block start addresses of multiple IO requests in the memory block, including: Confirm that the storage device supports IO aggregation and the maximum number of IOs supported by the storage device at a time; Set the memory block size based on the maximum IO number; Applying for a memory block of a corresponding size based on the memory block size; A new IO request is cached, a number is assigned to the IO request in a memory block, and information of the corresponding IO request is added after the number.
3. The method according to claim 2, characterized in that Cache new IO requests, assign numbers to the IO requests in the memory block, and add information about the corresponding IO requests after the numbers, including: Generate an IO request information linked list in the memory block; The numbers of the IO requests are stored in the linked list, and the IO request information is added to the table entry corresponding to the corresponding number.
4. The method according to claim 3, characterized in that The IO request information includes a logic block start address and a logic block quantity.
5. The method according to claim 3, characterized in that: The linked list includes: The first table entry is used to store the logical block start address of the IO request; The second table entry is used to store the status when an IO request error occurs; The third table item is used to store the number of the IO request; The fourth table item is used to store the number of logical blocks requested by the IO; The fifth table item is used to store the response data of the IO request; The sixth table entry is used to store the data address of the first IO request.
6. The method according to claim 1, characterized in that Based on the address of the memory block and the information of multiple IO requests stored in the memory block, a commit instruction is configured to generate an aggregate commit instruction, including: Retrieve a commit instruction, and change the logic block start address in the commit instruction to the address of the memory block; Update the number of logical blocks in the commit instruction to the sum of the number of logical blocks of multiple IO requests; Changing the data address in the submit instruction to the data addresses of multiple IO requests, wherein the data addresses of the multiple IO requests are stored in sequence according to the IO request number sequence; The commit instruction after parameter update is cached in the commit queue as an aggregate commit instruction.
7. The method according to claim 6, characterized in that The method further comprises: The storage device confirms receipt of the aggregation submission instruction based on the aggregation identifier; Read the memory block pointed to by the logical start address item of the aggregate commit instruction, and read the information of each IO request from the memory block one by one; Calculate the sub-data address of the IO request according to the order of the IO request, the data address in the aggregate submission instruction and the number of logical blocks of each IO request; Based on the information of the read IO request, data is read from the corresponding sub-data address and processed accordingly; If the IO request processing fails, a failure status flag is generated for the corresponding IO request.
8. The method according to claim 7, characterized in that Receive an aggregation completion instruction returned by the storage device, where the aggregation completion instruction includes response data to multiple IO requests, including: Response data of each IO request is parsed from the aggregation completion instruction.
9. A terminal, characterized in that: include: A memory, for storing an aggregation program of storage device IO requests; A processor is used to implement the steps of the storage device IO request aggregation method as described in any one of claims 1 to 8 when executing the storage device IO request aggregation program.
10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a storage device IO request aggregation program, which, when executed by a processor, implements the steps of the storage device IO request aggregation method as described in any one of claims 1 to 8.
Citation Information
Cited By
Request submission method and device, computer equipment and storage medium
CN119806403A
Data processing method and device, electronic equipment and readable storage medium
CN120872254A
