Input and output request processing method and device and computer readable storage medium

By splitting the input and output requests into smaller sub-requests and performing aggregation processing, the problem of cache resource waste caused by small-scale IO requests in traditional IO processing methods is solved, and more efficient memory utilization and granular management is achieved.

CN119045743AActive Publication Date: 2024-11-29WUXI STARS MICRO SYSTEM TECHNOLOGIES CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411217299.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-29
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

In traditional IO processing methods, the cache system adopts a large fixed block size, which leads to the inability to effectively utilize the cache space when processing small-scale IO requests, resulting in wasting resources.

Method used

Split the input and output requests into smaller sub-requests, and aggregate them according to the address information of the sub-requests to form data strips corresponding to the data blocks or strips in the disk array, thereby performing vertical space aggregation to reduce memory fragmentation. Improve memory utilization.

Benefits of technology

It realizes granular management of host memory space, reduces memory fragmentation, improves memory utilization, and adapts to the needs of various business scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045743A_ABST
    Figure CN119045743A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computers, and discloses an input and output request processing method and device and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of input and output requests, and splitting the plurality of input and output requests into a plurality of sub-requests; based on the address information corresponding to the plurality of sub-requests, aggregating the plurality of sub-requests into corresponding data bars respectively; and performing longitudinal space aggregation on the plurality of data bars to obtain a plurality of aggregation requests, and issuing the plurality of aggregation requests to the disk array. In this way, the input and output requests are split into smaller sub-requests, the granularity of the input and output requests is reduced, aggregation is carried out according to the corresponding address information, the data bars corresponding to the data blocks or the disk strips in the disk array are obtained, and the aggregation requests are aggregated on the basis of the data bars. Therefore, granular management of the memory space of the host can be realized, memory fragments are reduced, and the memory utilization rate is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to an input / output request processing method, device and computer-readable storage medium. Background Art

[0002] In traditional IO processing methods, the cache system usually uses a larger fixed block size, such as 64KB. This setting is more efficient when processing large-scale IO requests. However, when facing small-scale IO requests, the setting of large cache blocks will result in the cache space not being effectively utilized, resulting in a waste of cache resources. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention provide an input / output request processing method, device and computer-readable storage medium, which are used to solve the problem of cache resource waste in the processing of small-scale IO requests in the prior art.

[0004] According to one aspect of an embodiment of the present invention, a method for processing an input / output request is provided, the method comprising:

[0005] Get multiple input and output requests, and split the multiple input and output requests into multiple sub-requests;

[0006] Based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data stripes; the data stripes correspond to the data blocks or disk stripes in the disk array corresponding to the sub-requests;

[0007] Multiple data strips are aggregated in the vertical space to obtain multiple aggregation requests, and the multiple aggregation requests are sent to the disk array.

[0008] In an optional implementation, multiple input and output requests are split into multiple sub-requests, including:

[0009] Get the minimum processing unit; the minimum processing unit is used to represent the address width corresponding to the sub-request;

[0010] Taking the boundary of the minimum processing unit as the splitting boundary, multiple input and output requests are split in sequence to obtain multiple sub-requests.

[0011] In an optional implementation, based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data strips, including:

[0012] If the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request, creating a first data stripe based on the type of the disk array corresponding to the first sub-request, and adding the first sub-request to the first data stripe;

[0013] If the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request, and the first data stripe is not in a locked state, the second sub-request is added to the first data stripe.

[0014] In an optional implementation, creating a first data stripe based on the type of the disk array corresponding to the first sub-request includes:

[0015] If the disk array corresponding to the first sub-request is a first-type disk array, a data stripe distributed in the longitudinal space is constructed, and the depth of the data stripe is set to a preset depth to obtain a first data stripe; the first-type disk array is a disk array that manages data blocks therein longitudinally; the preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data block in the disk array;

[0016] If the disk array corresponding to the first sub-request is a second-type disk array, a data stripe distributed in the horizontal space is constructed, and the data stripe width is set to a preset width to obtain a first data stripe; the second-type disk array is a disk array that manages data blocks therein horizontally; the preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array.

[0017] In an optional implementation, based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data strips, further comprising:

[0018] If the address range corresponding to the first data stripe includes the address range corresponding to the third sub-request, and the first data stripe is in a locked state, a shadow data stripe corresponding to the first data stripe is created, and the third sub-request is added to the shadow data stripe;

[0019] If the aggregation request corresponding to the first data stripe is executed completely, the first data stripe is released, and the shadow data stripe is used as the new first data stripe.

[0020] In an optional implementation, based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data strips, further comprising:

[0021] Determining a life cycle of the first data stripe based on an attribute of the first sub-request and a size of a data block or stripe in the disk array corresponding to the first sub-request;

[0022] If the life cycle of the first data strip has not ended, determining that the first data strip is not in a locked state;

[0023] If the life cycle of the first data strip ends, it is determined that the first data strip is in a locked state.

[0024] In an optional implementation, multiple data strips are aggregated in a longitudinal space to obtain multiple aggregation requests, including:

[0025] If the disk array corresponding to the data stripe is a first-type disk array, the data stripes located in the same longitudinal space are aggregated to obtain an aggregation request;

[0026] If the disk array corresponding to the data stripe is a second-type disk array, sub-requests in the same longitudinal space in multiple data stripes are aggregated to obtain an aggregate request.

[0027] According to another aspect of an embodiment of the present invention, there is provided an input / output request processing device, the device comprising:

[0028] A request splitting module is used to obtain multiple input and output requests and split the multiple input and output requests into multiple sub-requests;

[0029] A data stripe determination module is used to aggregate the multiple sub-requests into corresponding data stripes based on the address information corresponding to the multiple sub-requests; the data stripe corresponds to the data block or disk stripe in the disk array corresponding to the sub-request;

[0030] The aggregation request determination module is used to aggregate multiple data strips in the longitudinal space, obtain multiple aggregation requests, and send the multiple aggregation requests to the disk array.

[0031] According to another aspect of an embodiment of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0032] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute any operation of the input and output request processing method described above.

[0033] According to another aspect of an embodiment of the present invention, there is provided a computer-readable storage medium, in which at least one executable instruction is stored. When the executable instruction is executed on an input / output request processing apparatus or a computer device, the input / output request processing apparatus or the computer device executes the operation of any of the input / output request processing methods described above.

[0034] The embodiment of the present invention reduces the granularity of input and output requests by splitting them into smaller sub-requests, and aggregates them according to their corresponding address information to obtain data stripes corresponding to data blocks or disk stripes in the disk array. Aggregate requests corresponding to data blocks or disk stripes in the disk array are split and aggregated based on the data strips, thereby achieving granular management of the host memory space, reducing memory fragmentation, and improving memory utilization.

[0035] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0037] Figure 1 It is a flowchart of an input and output request processing method provided by an embodiment of the present invention;

[0038] Figure 2 It is a schematic diagram of splitting and aggregating input and output requests in an input and output request processing method provided by an embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram of a process of splitting an input / output request into sub-requests in an input / output request method provided by an embodiment of the present invention;

[0040] Figure 4 It is a schematic diagram of splitting an input / output request into sub-requests in an input / output request processing method provided by an embodiment of the present invention;

[0041] Figure 5 It is a schematic diagram of a process of aggregating sub-requests into data strips in an input / output request processing method provided by an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of an aggregated request obtained by aggregating when the disk array corresponding to the data stripe is a first type of disk array in an input / output request processing method provided by an embodiment of the present invention;

[0043] Figure 7 It is a schematic diagram of an aggregate request obtained by aggregating when the disk array corresponding to the data stripe is a second type of disk array in an input / output request processing method provided by an embodiment of the present invention;

[0044] Figure 8 It is an application diagram of an input and output request processing method provided by an embodiment of the present invention;

[0045] Fig. 9 is a schematic diagram of the structure of an embodiment of an input and output request processing device provided by an embodiment of the present invention;

[0046] Fig.10 It is a schematic diagram of the structure of an embodiment of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0048] In traditional IO processing methods, cache systems usually use a large fixed block size, such as 64KB. This setting is more efficient when processing large-scale IO requests. However, when facing small-scale IO requests, the setting of large cache blocks will result in the ineffective use of cache space, resulting in a waste of cache resources. In addition, cache applications are usually consistent with the disk direction. In disk arrays with parity checks such as RAID5 or RAID6, data verification calculations need to be performed based on the positions of different member disks. This means that when performing verification calculations, the system needs to query multiple cache blocks and determine their positions in the striped layout, which not only increases the computational complexity but also reduces query efficiency.

[0049] Based on this, an embodiment of the present invention provides an input / output request processing method, the method comprising: obtaining multiple input / output requests, and splitting the multiple input / output requests into multiple sub-requests respectively; based on the address information corresponding to the multiple sub-requests, aggregating the multiple sub-requests into corresponding data stripes respectively; the data stripes correspond to the data blocks or disk stripes in the disk array corresponding to the sub-requests; aggregating the multiple data stripes in the vertical space to obtain multiple aggregation requests, and sending the multiple aggregation requests to the disk array. In this way, by splitting the input / output requests into smaller sub-requests, the granularity of the input / output requests is reduced, and the data stripes corresponding to the data blocks or disk stripes in the disk array are obtained according to the corresponding address information, and the aggregation requests corresponding to the data blocks or disk stripes in the disk array are split and aggregated on the basis of the data stripes, thereby realizing the granular management of the host memory space, reducing memory fragmentation, and improving memory utilization.

[0050] The following describes a specific embodiment of an input and output request processing method of the present invention. Figure 1 It is a flowchart of an input and output request processing method provided by an embodiment of the present invention. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 1 As shown, the method may include:

[0051] Step 110: Acquire multiple input / output requests, and split the multiple input / output requests into multiple sub-requests respectively.

[0052] In the embodiment of the present invention, the obtained input and output requests, that is, user IO, are split into sub-requests to obtain multiple sub-requests, thereby reducing the granularity of the obtained input and output requests so that they can be processed by cache blocks with smaller granularity.

[0053] Step 120: Based on the address information corresponding to the multiple sub-requests, aggregate the multiple sub-requests into corresponding data strips respectively.

[0054] In an embodiment of the present invention, a data stripe corresponds to a data block or a disk stripe in a disk array corresponding to a sub-request, which is specifically reflected in that the structure of the data stripe is consistent with the data block or disk stripe in the disk array corresponding to the sub-request. For example, if the disk array corresponding to the sub-request is a horizontally managed disk array, then the data stripe is also a horizontally distributed structure, and at the same time, the address range corresponding to each data stripe is consistent with the address range of the data block or disk stripe in the disk array; that is, the data stripe is equivalent to a mapping space corresponding to the data block or disk stripe in the disk array.

[0055] In an embodiment of the present invention, based on the address information corresponding to multiple sub-requests, the sub-requests are filled into the data strip containing the corresponding address information, so that the various sub-requests are aggregated into the data strip. Therefore, through the filling status of the data strip, it is possible to know the specific spatial range in which the acquired input and output requests will operate in the disk array.

[0056] Step 130: Aggregate multiple data stripes in the longitudinal space to obtain multiple aggregation requests, and send the multiple aggregation requests to the disk array.

[0057] In the embodiment of the present invention, since the response and processing of input and output requests are performed according to the longitudinal space within the disk array, it is necessary to aggregate the multiple data strips obtained by aggregation in the longitudinal space to obtain an aggregation request corresponding to the longitudinal space, and then send these aggregation requests to the disk array so that the disk array processes the processing within the relevant address range of the longitudinal space corresponding to the aggregation request based on the aggregation request, thereby completing the response to the obtained input and output requests.

[0058] In an optional embodiment, Figure 2 is a schematic diagram of splitting and aggregating input and output requests in an input and output request processing method provided by an embodiment of the present invention, Figure 3 FIG. 1 is a flow chart of splitting an input / output request into sub-requests in an input / output request method provided by an embodiment of the present invention. Figure 2and Figure 3 As shown, in the above step 110, splitting the multiple input and output requests into multiple sub-requests respectively may include the following steps:

[0059] Step 310: Obtain the minimum processing unit.

[0060] In the embodiment of the present invention, the minimum processing unit (Grain) is used to characterize the address width corresponding to the sub-request. The size of the minimum processing unit, that is, the sector size whose address width is an integer multiple of a power of 2, cannot exceed the size of a data block of a disk array in the system. Therefore, when splitting according to the minimum processing unit, the input and output requests can be split according to a granularity smaller than the data block of the disk array, so that the input and output requests can be responded to and processed at a finer granularity, thereby realizing granular management of the memory.

[0061] Step 320: Using the boundary of the minimum processing unit as the splitting boundary, the multiple input and output requests are split in sequence to obtain multiple sub-requests.

[0062] In the embodiment of the present invention, Figure 2 As shown, the boundary of the minimum processing unit is used as the splitting boundary, and the multiple input and output requests are split in sequence, and the multiple input and output requests are respectively split into corresponding sub-requests (PreIO).

[0063] Figure 4 is a schematic diagram of splitting an input / output request into sub-requests in an input / output request processing method provided by an embodiment of the present invention, such as Figure 4 As shown, IOA and IOB are user IOs, that is, the system obtains two input and output requests, and then splits IOA and IOB with the boundary of the minimum processing unit Grain as the splitting boundary, splitting IOA into 5 sub-requests, among which the last sub-request did not reach the boundary of the minimum processing unit when splitting, so its size after splitting is smaller than the size of the minimum processing unit; at the same time, when splitting IOB, since the last sub-request of the previously split IOA did not reach the boundary of the minimum processing unit when splitting, the splitting of IOB is not to re-divide the splitting boundary starting from IOB, but to fill IOB with the boundary of the minimum processing unit that was not reached when IOA was last split, so that the last sub-request of IOA and the first sub-request of IOB are just spliced ​​into one minimum processing unit. In other words, the splitting of input and output requests is based on the minimum processing unit, rather than the address range of the obtained input and output requests. The minimum processing unit is filled and split when the minimum processing unit is reached. If the input and output requests filled when the minimum processing unit boundary is reached come from different input and output requests, the filled input and output requests are then divided into multiple sub-requests according to their sources. Figure 4For example, if the address range of IOA is 0000-0009, the address range of IOB is 000B-0013, and the size of the minimum processing unit is 4 bits, then IOA is split into sub-requests with address ranges of 0000-0004, 0004-0007, and 0008-0009, and IOB is split into sub-requests with address ranges of 000B-000C, 000D-0010, and 00011-0013.

[0064] In an optional embodiment, if Figure 2 As shown in the figure, after splitting multiple input and output requests into multiple sub-requests (PreIO), each sub-request is aggregated into a data line (Data Line, DL). Figure 5 FIG. 1 is a flow chart of aggregating sub-requests into data strips in an input / output request processing method provided by an embodiment of the present invention. Figure 5 As shown, in the above step 120, based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data strips, which may include the following steps:

[0065] Step 510: If the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request, create a first data stripe based on the type of disk array corresponding to the first sub-request, and add the first sub-request to the first data stripe.

[0066] In an embodiment of the present invention, if the address ranges corresponding to the currently existing data stripes do not include the address range corresponding to the first sub-request, it indicates that the first sub-request does not hit any currently existing data stripes, and therefore a new data stripe needs to be registered for the first sub-request. Specifically, based on the type of disk array corresponding to the first sub-request, the first data stripe is created, and after the creation is completed, the first sub-request is added to the first data stripe, and the buffer resources corresponding to the first sub-request are registered, thereby completing the aggregation of the first sub-request.

[0067] In an optional implementation, since the disk arrays have different specifications or attributes, and their management methods are also different, it is necessary to create the first data stripe according to the type of the disk array corresponding to the first data stripe, that is, the type of the disk array accessed by the first sub-request. Specifically, it can be divided into the following two situations:

[0068] In the first case, if the disk array corresponding to the first sub-request is a first-type disk array, a data stripe distributed in the vertical space is constructed, and the data stripe depth is set to a preset depth to obtain a first data stripe. The first-type disk array is a disk array that manages the data blocks therein vertically, specifically, a disk array of the type Jbod, Raid0 or Raid1, etc. The attribute of the data stripe is correspondingly set to column so that it is managed vertically, thereby constructing a data stripe distributed in the vertical space. The preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data block in the disk array; that is, the space size corresponding to the first data stripe does not exceed the size of a data block in the disk array.

[0069] In the second case, if the disk array corresponding to the first sub-request is a second-type disk array, a data stripe distributed in the horizontal space is constructed, and the width of the data stripe is set to a preset width to obtain a first data stripe. The second-type disk array is a disk array that manages the data blocks therein horizontally, specifically, a disk array of the type RAID5 or RAID6. Accordingly, the attribute of the data stripe is set to row so that it is managed horizontally, thereby constructing a data stripe distributed in the horizontal space. The preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array, that is, the space size corresponding to the first data stripe is consistent with the space size of the disk stripe, and one data stripe corresponds to one granular stripe.

[0070] Therefore, different data strips are created for different types of disk arrays, so as to perform targeted optimization according to the type and specification of the disk array, thereby improving the universality of the input and output request processing method and enabling it to adapt to the needs of various business scenarios.

[0071] Step 520: If the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request, and the first data stripe is not in a locked state, then the second sub-request is added to the first data stripe.

[0072] In an embodiment of the present invention, if the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request, it indicates that the second sub-request hits the first data stripe; if the first data stripe is not in a locked state, it indicates that the first data stripe is still within its life cycle and the sub-requests can still be aggregated into the first data stripe. Therefore, the second sub-request can be added to the first data stripe at this time, and the buffer resources corresponding to the second sub-request can be registered at the same time to complete the aggregation of the second sub-request. If the buffer resources corresponding to the second sub-request have been registered with other sub-requests, it is no longer necessary to register the buffer resources corresponding to the second sub-request.

[0073] In an optional embodiment, the life cycle of a data stripe is determined by the attributes of the sub-request corresponding to it when it is created and the size of the data block or stripe in the disk array to which it corresponds. Taking the first data stripe created in the above step 510 as an example, after the first data stripe is created, the life cycle of the first data stripe is determined based on the attributes of the first sub-request and the size of the data block or stripe in the disk array corresponding to the first sub-request; wherein the attributes of the first sub-request indicate whether the first sub-request is a random input and output request or a continuous input and output request. If the first sub-request is a random input and output request, then the probability that the first data stripe will not aggregate other sub-requests after aggregating the first sub-request is high, and the life cycle of the first data stripe can be set to be relatively short. If the first sub-request is a continuous input and output request, then the probability that the first data stripe will not aggregate other sub-requests after aggregating the first sub-request is high, and the life cycle of the first data stripe can be set to be relatively short. After the sub-request, other sub-requests will be received and other sub-requests will be aggregated. Therefore, the life cycle of the first data stripe is set to be longer so that subsequent continuous sub-requests can be aggregated into the first data stripe. The larger the data block or stripe in the disk array corresponding to the first data stripe, the wider the corresponding address range, and the more sub-requests can be aggregated. Therefore, the life cycle of the first data stripe can be set to be longer. Correspondingly, the smaller the data block or stripe in the disk array corresponding to the first data stripe, the narrower the corresponding address range, and the fewer sub-requests can be aggregated. Therefore, the life cycle of the first data stripe can be set to be shorter. Through the above two dimensions, the life cycle of the data stripe is adjusted to meet the requirements of input and output requests and disk arrays.

[0074] In an optional implementation, after a data strip is created, its state is determined in real time according to its life cycle. Taking the first data strip created in step 510 as an example, if the life cycle of the first data strip has not ended, it is determined that the first data strip is not in a locked state; if the life cycle of the first data strip has ended, it is determined that the first data strip is in a locked state.

[0075] In an optional implementation, if the address range corresponding to the first data stripe includes the address range corresponding to the third sub-request, and the first data stripe is in a locked state, it indicates that the life cycle of the first data stripe has ended, and the sub-requests can no longer be aggregated into the first data stripe. Therefore, a shadow data stripe corresponding to the first data stripe is created, and the third sub-request is added to the shadow data stripe. For the same storage space, there can only be two executable data stripes at the same time, that is, the foreground data stripe, and the shadow data stripe corresponding to the foreground data stripe. When the executable foreground data stripe is in a locked state, the corresponding shadow data stripe continues to aggregate the sub-requests to ensure the smoothness of the input and output request processing. At the same time, if the aggregation request corresponding to the first data stripe is executed, the first data stripe is released, and the shadow data stripe corresponding to the first data stripe is used as the new first data stripe, thereby changing the shadow data stripe to the foreground data stripe, so that the sub-requests aggregated therein can be further aggregated into an aggregation request.

[0076] In an optional embodiment, as described above, since there are two types of disk arrays, namely, a first type of disk array that manages data blocks vertically, and a second type of disk array that manages data blocks horizontally, when data strips are aggregated in the vertical space to obtain an aggregation request, the corresponding aggregation operation is different depending on the type of disk array. Figure 6 is a schematic diagram of an aggregated request obtained by aggregating when the disk array corresponding to the data stripe is a first type of disk array in an input / output request processing method provided by an embodiment of the present invention, such as Figure 6 As shown, if the disk array corresponding to the data stripe is a first-type disk array, since the first-type disk array itself is also managed vertically, its corresponding data stripes are also distributed vertically, so the data stripes located in the same vertical space can be directly aggregated to obtain an aggregation request. Figure 7 1 is a schematic diagram of an aggregated request obtained by aggregating when the disk array corresponding to the data stripe is a second type of disk array in an input / output request processing method provided by an embodiment of the present invention, such as Figure 7 As shown in the figure, if the disk array corresponding to the data stripe is the second type disk array, since the second type disk array itself is managed horizontally, its corresponding data stripes are also distributed horizontally, so it is necessary to first split the multiple data stripes and aggregate the sub-requests in the same vertical space to obtain an aggregate request. Figure 2 , Figure 6 and Figure 7It can be seen that when aggregating data strips (DL) into aggregation requests (PstIO), the data strips (DL) corresponding to the same longitudinal space can be aggregated first to obtain aggregated data strips (MDL), thereby directly performing longitudinal spatial aggregation inside the aggregated data strip to obtain the aggregation request (PstIO), and for data strips that are not located in the same longitudinal space, longitudinal spatial aggregation is also directly performed inside them to obtain the aggregation request (PstIO); therefore, when performing longitudinal spatial aggregation, it can be performed directly inside the data strip or the aggregated data strip without involving other data strips, thereby improving aggregation efficiency.

[0077] In an optional embodiment, if Figure 7 As shown in the figure, since the second type of disk array is a RAID5 or RAID6 disk array, it also needs to perform checksum calculation. Therefore, when sending the aggregation request to the second type of disk array, the relevant algorithm will be selected according to the stripe data situation. When the stripe is full, the checksum is directly calculated by the new data; when the stripe is not full, the read IO of the penalty data is generated first, and the new checksum is calculated by the penalty data and the new data. Finally, the new data and the new checksum are sent to the disk array together. Among them, the number of disks hit in different disk stripes is different, and the corresponding algorithms are also different, and the corresponding penalty data obtained is also different. Figure 7 For example, for the first data strip, the read other algorithm is used to generate the checksum, which is calculated by reading the old data of other disks and the new data of the hit disk, and the penalty IO corresponding Figure 7 The read IO of the two old data blocks on the right is read; for the second, third and fourth data strips, the read old algorithm is used to generate the checksum, and the calculation check is performed by reading the old data and new data of the hit disk, and the penalty IO corresponds to Figure 7 Read IO that reads the two old data blocks on the left.

[0078] In an optional implementation, since there are multiple splits and aggregations from obtaining input and output requests to sending the input and output requests to the disk array, in order to avoid errors in the correspondence between requests during the processing process which may lead to errors in the final processing and execution, each split or aggregation will save the association or binding information between the requests before and after the split or aggregation.

[0079] In an optional embodiment, Figure 8 is an application diagram of an input / output request processing method provided by an embodiment of the present invention. The input / output request processing method provided by an embodiment of the present invention can be applied to Figure 8 The system architecture is shown in Figure 1. Figure 8As shown, the system includes a request splitting module (IO PreProc), a data stripe management module (DLController), a request aggregation module (IO PstProc), a data cache module (Memory), an associated information storage module (MapTable) and a calculation module (Calculator). Among them, the request splitting module (IO PreProc), the data stripe management module (DL Controller) and the request aggregation module (IO PstProc) are the main functional modules for input and output request splitting and aggregation. The request splitting module (IO PreProc) splits the received user IO, that is, the input and output request into sub-requests (PreIO) according to the minimum processing unit, and the data stripe management module (DL Controller) aggregates each sub-request into a data stripe (DL). At the same time, if there are data stripes corresponding to the same longitudinal space, these data stripes are further aggregated into aggregated data stripes (MDL). The request aggregation module (IO PstProc) aggregates the data stripes or aggregated data stripes in the longitudinal space to obtain an aggregated request (PstIO) and sends it to the disk array. The data cache module (Memory) serves as a data buffer. When the data stripe management module (DL Controller) aggregates sub-requests, the buffer resources corresponding to the sub-requests are registered through the data cache module (Memory); the association information storage module (MapTable) is used to store the association or binding information when the request is split and aggregated; the calculation module (Calculator) is responsible for data calculation, which can be data encryption calculation. If the disk array corresponding to the input and output request is a RAID5 or RAID6 type disk array, it can also be a verification calculation.

[0080] The input / output request processing method provided by the embodiment of the present invention includes: obtaining multiple input / output requests, and splitting the multiple input / output requests into multiple sub-requests respectively; based on the address information corresponding to the multiple sub-requests, aggregating the multiple sub-requests into corresponding data stripes respectively; the data stripes correspond to the data blocks or disk stripes in the disk array corresponding to the sub-requests; aggregating the multiple data stripes in the vertical space to obtain multiple aggregated requests, and sending the multiple aggregated requests to the disk array. In this way, by splitting the input / output requests into smaller sub-requests, the granularity of the input / output requests is reduced, and the data stripes corresponding to the data blocks or disk stripes in the disk array are aggregated according to the corresponding address information, and the aggregated requests corresponding to the data blocks or disk stripes in the disk array are split and aggregated on the basis of the data stripes, thereby realizing granular management of the host memory space, reducing memory fragmentation, and improving memory utilization.

[0081] The embodiment of the present invention also provides an input and output request processing device, Fig. 9is a schematic diagram of the structure of an embodiment of an input and output request processing device provided by an embodiment of the present invention, such as Fig. 9 As shown, the device 900 includes:

[0082] A request splitting module 910 is used to obtain multiple input and output requests and split the multiple input and output requests into multiple sub-requests respectively;

[0083] A data stripe determination module 920 is used to aggregate the multiple sub-requests into corresponding data stripes based on the address information corresponding to the multiple sub-requests; the data stripe corresponds to the data block or disk stripe in the disk array corresponding to the sub-request;

[0084] The aggregation request determination module 930 is used to aggregate multiple data stripes in a longitudinal space to obtain multiple aggregation requests, and send the multiple aggregation requests to the disk array.

[0085] In an optional implementation, the request splitting module 910 includes:

[0086] A minimum processing unit acquisition unit is used to acquire a minimum processing unit; the minimum processing unit is used to characterize the address width corresponding to the sub-request;

[0087] The input / output request splitting unit is used to split multiple input / output requests in sequence by taking the boundary of the minimum processing unit as the splitting boundary to obtain multiple sub-requests.

[0088] In an optional implementation, the data strip determination module 920 includes:

[0089] a data stripe creation unit, configured to create a first data stripe based on the type of the disk array corresponding to the first sub-request and add the first sub-request to the first data stripe if the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request;

[0090] The request adding unit is used to add the second sub-request to the first data stripe if the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request and the first data stripe is not in a locked state.

[0091] In an optional implementation manner, the data stripe creating unit is used to:

[0092] If the disk array corresponding to the first sub-request is a first-type disk array, a data stripe distributed in the longitudinal space is constructed, and the depth of the data stripe is set to a preset depth to obtain a first data stripe; the first-type disk array is a disk array that manages data blocks therein longitudinally; the preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data block in the disk array;

[0093] If the disk array corresponding to the first sub-request is a second-type disk array, a data stripe distributed in the horizontal space is constructed, and the data stripe width is set to a preset width to obtain a first data stripe; the second-type disk array is a disk array that manages data blocks therein horizontally; the preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array.

[0094] In an optional implementation, the data strip determination module 920 further includes:

[0095] a shadow data stripe creation unit, configured to create a shadow data stripe corresponding to the first data stripe and add the third sub-request to the shadow data stripe if the address range corresponding to the first data stripe includes the address range corresponding to the third sub-request and the first data stripe is in a locked state;

[0096] The data stripe updating unit is used to release the first data stripe if the aggregation request corresponding to the first data stripe is executed, and use the shadow data stripe as the new first data stripe.

[0097] In an optional implementation, the data strip determination module 920 further includes:

[0098] A life cycle determination unit, configured to determine a life cycle of the first data stripe based on an attribute of the first sub-request and a size of a data block or stripe in a disk array corresponding to the first sub-request;

[0099] a state determination unit, configured to determine that the first data strip is not in a locked state if the life cycle of the first data strip has not ended;

[0100] The state determination unit is further configured to determine that the first data strip is in a locked state if the life cycle of the first data strip ends.

[0101] In an optional implementation, the aggregation request determination module 930 includes:

[0102] If the disk array corresponding to the data stripe is a first-type disk array, the data stripes located in the same longitudinal space are aggregated to obtain an aggregation request;

[0103] If the disk array corresponding to the data stripe is a second-type disk array, sub-requests in the same longitudinal space in multiple data stripes are aggregated to obtain an aggregate request.

[0104] The device and method embodiments in the embodiments of this application are based on the same application concept.

[0105] Fig.10 The schematic diagram of the structure of the embodiment of the computer device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0106] like Fig.10 As shown, the computer device may include: a processor (processor) 1002 , a communication interface (Communications Interface) 1004 , a memory (memory) 1006 , and a communication bus 1008 .

[0107] The processor 1002, the communication interface 1004, and the memory 1006 communicate with each other via the communication bus 1008. The communication interface 1004 is used to communicate with other devices such as a client or other server network elements. The processor 1002 is used to execute the program 1010, which can specifically execute the relevant steps in the above-mentioned embodiment of the method for processing input and output requests.

[0108] Specifically, the program 1010 may include program code, which includes computer-executable instructions.

[0109] The processor 1002 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0110] The memory 1006 is used to store the program 1010. The memory 1006 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0111] The program 1010 can be specifically called by the processor 1002 to enable the computer device to execute the relevant steps in the above-mentioned embodiment of the method for processing input and output requests.

[0112] It can be understood by those skilled in the art that Fig.10 The structure shown is only for illustration and does not limit the structure of the above-mentioned device. Fig.10 More or fewer components as shown, or with Fig.10 Different configurations are shown.

[0113] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on an input / output request processing apparatus or a computer device, the input / output request processing apparatus or the computer device executes the input / output request processing method in any of the above method embodiments.

[0114] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0115] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0116] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subrequests or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0117] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of protection of the present invention. The word "comprising" does not exclude the existence of elements or steps not listed in the present invention. The word "one" or "an" preceding an element does not exclude the existence of multiple such elements. The present invention can be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In the embodiments in which several devices are listed, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A method for processing input and output requests, characterized in that: The method comprises: Acquire multiple input and output requests, and split the multiple input and output requests into multiple sub-requests respectively; Based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data stripes; the data stripes correspond to the data blocks or disk stripes in the disk array corresponding to the sub-requests; The plurality of data stripes are aggregated in a longitudinal space to obtain a plurality of aggregation requests, and the plurality of aggregation requests are sent to the disk array.

2. The method according to claim 1, characterized in that The step of splitting the plurality of input and output requests into a plurality of sub-requests comprises: Obtaining a minimum processing unit; the minimum processing unit is used to represent the address width corresponding to the sub-request; The plurality of input and output requests are sequentially split with the boundary of the minimum processing unit as the splitting boundary to obtain a plurality of sub-requests.

3. The method according to claim 1, characterized in that The step of respectively aggregating the plurality of sub-requests into corresponding data strips based on the address information corresponding to the plurality of sub-requests includes: If the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request, creating a first data stripe based on the type of the disk array corresponding to the first sub-request, and adding the first sub-request to the first data stripe; If the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request, and the first data stripe is not in a locked state, the second sub-request is added to the first data stripe.

4. The method according to claim 3, characterized in that The creating a first data stripe based on the type of the disk array corresponding to the first sub-request includes: If the disk array corresponding to the first sub-request is a first-type disk array, a data stripe distributed in the longitudinal space is constructed, and the data stripe depth is set to a preset depth to obtain the first data stripe; the first-type disk array is a disk array that manages data blocks therein longitudinally; the preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data block in the disk array; If the disk array corresponding to the first sub-request is a second-type disk array, a data stripe distributed in the horizontal space is constructed, and the data stripe width is set to a preset width to obtain the first data stripe; the second-type disk array is a disk array that manages data blocks therein horizontally; the preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array.

5. The method according to claim 3, characterized in that: The step of respectively aggregating the plurality of sub-requests into corresponding data strips based on the address information corresponding to the plurality of sub-requests further includes: If the address range corresponding to the first data stripe includes the address range corresponding to the third sub-request, and the first data stripe is in a locked state, creating a shadow data stripe corresponding to the first data stripe, and adding the third sub-request to the shadow data stripe; If the aggregation request corresponding to the first data stripe is executed completely, the first data stripe is released, and the shadow data stripe is used as the new first data stripe.

6. The method according to claim 3 or 5, characterized in that: The step of respectively aggregating the plurality of sub-requests into corresponding data strips based on the address information corresponding to the plurality of sub-requests further includes: Determining a life cycle of the first data stripe based on an attribute of the first sub-request and a size of a data block or stripe in a disk array corresponding to the first sub-request; If the life cycle of the first data stripe has not ended, determining that the first data stripe is not in a locked state; If the life cycle of the first data stripe ends, it is determined that the first data stripe is in a locked state.

7. The method according to claim 1, characterized in that The aggregating the plurality of data strips in a longitudinal spatial manner to obtain a plurality of aggregation requests includes: If the disk array corresponding to the data stripe is a first type disk array, the data stripes located in the same longitudinal space are aggregated to obtain the aggregation request; If the disk array corresponding to the data stripe is a second type of disk array, sub-requests in a same longitudinal space in a plurality of the data stripes are aggregated to obtain the aggregate request.

8. An input / output request processing device, characterized in that: The device comprises: A request splitting module, used for obtaining a plurality of input and output requests, and splitting the plurality of input and output requests into a plurality of sub-requests respectively; A data stripe determination module, configured to aggregate the plurality of sub-requests into corresponding data stripes based on address information corresponding to the plurality of sub-requests; the data stripes correspond to data blocks or disk stripes in the disk array corresponding to the sub-requests; The aggregation request determination module is used to aggregate the plurality of data stripes in a longitudinal space to obtain a plurality of aggregation requests, and send the plurality of aggregation requests to the disk array.

9. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the input and output request processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the input / output request processing apparatus or computer device, the input / output request processing apparatus or computer device executes the operation of the input / output request processing method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Data writing method and device

    CN110471615A

  • Data storage method and data storage device

    CN113986604A

  • RAID card, data access method and system thereof and storage medium

    CN116027990A

  • Data request processing method and device, computer equipment and storage medium

    CN116737070A

  • Column storage format-based database parallel aggregation method, computer equipment and storage medium

    CN117349293A