Input / Output Request Processing Method, Device, and Computer-Readable Storage Medium

Abstract

The present invention relates to the field of computer technologies, and discloses an input / output request processing method, apparatus, and computer-readable storage medium. The method includes: obtaining a plurality of input / output requests, and separately splitting the plurality of input / output requests into a plurality of sub-requests; based on the address information corresponding to the plurality of sub-requests, separately aggregating the plurality of sub-requests into corresponding data strips; performing vertical space aggregation on the plurality of data strips to obtain a plurality of aggregation requests, and sending the plurality of aggregation requests to a disk array. In this way, by splitting the input / output requests into smaller sub-requests, the granularity of the input / output requests is reduced, and aggregation is performed according to the corresponding address information to obtain data strips corresponding to data blocks or disk stripes in the disk array, and aggregation requests are aggregated on the basis of the data strips, so as to realize granular management of the memory space of the host, reduce memory fragmentation, and improve memory utilization rate.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technologies, and particularly to an input / output request processing method, apparatus, and computer-readable storage medium. Background Art

[0002] In traditional IO processing methods, cache systems usually adopt a relatively large fixed block size, such as 64KB. This setting has high efficiency when processing large-scale IO requests. However, in the face of small-scale IO requests, the setting of large cache blocks will result in ineffective utilization of cache space and waste of cache resources. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention provide an input / output request processing method, apparatus, and computer-readable storage medium, which are used to solve the problem of waste of cache resources in the processing of small-scale IO requests in the prior art.

[0004] According to one aspect of the embodiments of the present invention, an input / output request processing method is provided, and the method includes:

[0005] Obtain a plurality of input / output requests, and respectively split the plurality of input / output requests into a plurality of sub-requests;

[0006] Based on the address information corresponding to the plurality of sub-requests, respectively aggregate the plurality of sub-requests into corresponding data stripes; the data stripes correspond to data blocks or disk stripes in the disk array corresponding to the sub-requests;

[0007] Perform vertical space aggregation on the plurality of data stripes to obtain a plurality of aggregation requests, and send the plurality of aggregation requests to the disk array.

[0008] In an optional implementation manner, splitting the plurality of input / output requests into a plurality of sub-requests respectively includes:

[0009] Obtain the minimum processing unit; the minimum processing unit is used to represent the address width corresponding to the sub-request;

[0010] Taking the boundary of the minimum processing unit as the splitting boundary, sequentially split the plurality of input / output requests to obtain a plurality of sub-requests.

[0011] In an optional implementation manner, based on the address information corresponding to the plurality of sub-requests, respectively aggregating the plurality of sub-requests into corresponding data stripes includes:

[0012] If the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request, then create a first data stripe based on the type of the disk array corresponding to the first sub-request, and add the first sub-request to the first data stripe;

[0013] If the address range corresponding to the first data stripe contains the address range corresponding to the second sub-request and the first data stripe is not in a locked state, add the second sub-request to the first data stripe.

[0014] In an alternative embodiment, creating a first data stripe based on the type of disk array corresponding to the first sub-request includes:

[0015] If the disk array corresponding to the first sub-request is a first type of disk array, construct data stripes distributed in the longitudinal space and set the data stripe depth to a preset depth to obtain the first data stripe; the first type of disk array is a disk array that manages data blocks therein longitudinally; the preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data blocks in the disk array;

[0016] If the disk array corresponding to the first sub-request is a second type of disk array, construct data stripes distributed in the transverse space and set the data stripe width to a preset width to obtain the first data stripe; the second type of disk array is a disk array that manages data blocks therein transversely; the preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array.

[0017] In an alternative embodiment, aggregating multiple sub-requests into corresponding data stripes respectively based on the address information corresponding to the multiple sub-requests further includes:

[0018] If the address range corresponding to the first data stripe contains the address range corresponding to the third sub-request and the first data stripe is in a locked state, create a shadow data stripe corresponding to the first data stripe and add the third sub-request to the shadow data stripe;

[0019] If the aggregation request corresponding to the first data stripe is completed, release the first data stripe and use the shadow data stripe as the new first data stripe.

[0020] In an alternative embodiment, aggregating multiple sub-requests into corresponding data stripes respectively based on the address information corresponding to the multiple sub-requests further includes:

[0021] Determine the lifecycle of the first data stripe based on the attribute of the first sub-request and the size of the data blocks or stripes in the disk array corresponding to the first sub-request;

[0022] If the lifecycle of the first data stripe has not ended, determine that the first data stripe is not in a locked state;

[0023] If the lifecycle of the first data stripe has ended, determine that the first data stripe is in a locked state.

[0024] In an alternative embodiment, aggregating multiple data strips in the vertical space to obtain multiple aggregation requests, including:

[0025] If the disk array corresponding to the data strip is a first type of disk array, aggregating the data strips located in the same vertical space to obtain an aggregation request;

[0026] If the disk array corresponding to the data strip is a second type of disk array, aggregating the sub-requests located in the same vertical space among the multiple data strips to obtain an aggregation request.

[0027] According to another aspect of the embodiments of the present invention, there is provided an input / output request processing apparatus, which includes:

[0028] A request splitting module, configured to obtain multiple input / output requests and split the multiple input / output requests into multiple sub-requests respectively;

[0029] A data strip determination module, configured to aggregate the multiple sub-requests into corresponding data strips respectively based on the address information corresponding to the multiple sub-requests; the data strip corresponds to a data block or a disk stripe in the disk array corresponding to the sub-request;

[0030] An aggregation request determination module, configured to aggregate the multiple data strips in the vertical space to obtain multiple aggregation requests, and send the multiple aggregation requests to the disk array.

[0031] According to another aspect of the embodiments of the present invention, there is provided a computer device, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0032] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations of the input / output request processing method described above arbitrarily.

[0033] According to yet another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which at least one executable instruction is stored, and when the executable instruction runs on the input / output request processing apparatus or the computer device, it causes the input / output request processing apparatus or the computer device to execute the operations of the input / output request processing method described above arbitrarily.

[0034] In the embodiments of the present invention, by splitting the input / output request into smaller sub-requests, reducing the granularity of the input / output request, and aggregating them according to the corresponding address information to obtain data strips corresponding to the data blocks or disk stripes in the disk array, and on the basis of the data strips, splitting and aggregating to obtain aggregation requests corresponding to the data blocks or disk stripes in the disk array, so as to realize the granular management of the host memory space, reduce memory fragmentation, and improve memory utilization.

[0035] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to understand the technical means of the embodiment of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and understandable, the following specifically illustrates the specific implementation manners of the present invention. Description of the Drawings

[0036] The drawings are only used to illustrate the embodiments and are not considered as a limitation to the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0037] Figure 1 is a schematic flowchart of a method for processing input / output requests provided by an embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of splitting and aggregating input / output requests in a method for processing input / output requests provided by an embodiment of the present invention;

[0039] Figure 3 is a schematic flowchart of splitting an input / output request into sub-requests in a method for processing input / output requests provided by an embodiment of the present invention;

[0040] Figure 4 is a schematic diagram of splitting an input / output request into sub-requests in a method for processing input / output requests provided by an embodiment of the present invention;

[0041] Figure 5 is a schematic flowchart of aggregating sub-request data items in a method for processing input / output requests provided by an embodiment of the present invention;

[0042] Figure 6 is a schematic diagram of aggregating to obtain an aggregation request when the disk array corresponding to the data item is a first type of disk array in a method for processing input / output requests provided by an embodiment of the present invention;

[0043] Figure 7 is a schematic diagram of aggregating to obtain an aggregation request when the disk array corresponding to the data item is a second type of disk array in a method for processing input / output requests provided by an embodiment of the present invention;

[0044] Figure 8 is a schematic application diagram of a method for processing input / output requests provided by an embodiment of the present invention;

[0045] Figure 9 is a schematic structural diagram of an embodiment of an input / output request processing device provided by an embodiment of the present invention;

[0046] Figure 10 is a schematic structural diagram of an embodiment of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0047] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0048] In traditional IO processing methods, the cache system usually adopts a relatively large fixed block size, such as 64 KB. This setting will have relatively high efficiency when processing large-scale IO requests. However, in the face of small-scale IO requests, the setting of large cache blocks will result in ineffective utilization of the cache space, leading to waste of cache resources. Moreover, the application of the cache usually keeps consistent with the disk direction. In disk arrays with parity check such as RAID5 or RAID6, the parity calculation of data needs to be performed according to the positions of different member disks. This leads to the system needing to query multiple cache blocks and determine their positions in the striped layout when performing parity calculation, which not only increases the computational complexity but also reduces the query efficiency.

[0049] Based on this, an embodiment of the present invention provides an input / output request processing method, which includes: obtaining a plurality of input / output requests, and respectively splitting the plurality of input / output requests into a plurality of sub-requests; aggregating the plurality of sub-requests into corresponding data stripes based on the address information corresponding to the plurality of sub-requests; the data stripes corresponding to data blocks or disk stripes in the disk array corresponding to the sub-requests; aggregating the plurality of data stripes in the vertical space to obtain a plurality of aggregated requests, and sending the plurality of aggregated requests to the disk array. In this way, by splitting the input / output requests into smaller sub-requests, the granularity of the input / output requests is reduced, and they are aggregated according to their corresponding address information to obtain data stripes corresponding to data blocks or disk stripes in the disk array. On the basis of the data stripes, aggregated requests corresponding to data blocks or disk stripes in the disk array are split and aggregated, so as to realize the granular management of the host memory space, reduce memory fragmentation, and improve memory utilization.

[0050] The following introduces a specific embodiment of an input / output request processing method of the present invention. Figure 1 is a schematic flowchart of an input / output request processing method provided by an embodiment of the present invention. This specification provides method operation steps such as in the embodiment or flowchart, but based on routine or non-creative labor, there can be more or fewer operation steps. The step order listed in the embodiment is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it can be executed in the order shown in the embodiment or the drawings or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing). Specifically, as Figure 1 shown, the method may include:

[0051] Step 110: Obtain multiple input / output requests, and split each of the multiple input / output requests into multiple sub-requests.

[0052] In an embodiment of the present invention, the obtained input / output requests are user IOs. Each of the obtained input / output requests is split into sub-requests to obtain multiple sub-requests, thereby reducing the granularity of the obtained input / output requests so that they can be processed by cache blocks with a smaller granularity.

[0053] Step 120: Based on the address information corresponding to the multiple sub-requests, aggregate the multiple sub-requests into corresponding data stripes respectively.

[0054] In an embodiment of the present invention, a data stripe corresponds to a data block or a disk stripe in a disk array corresponding to a sub-request. Specifically, the structure of the data stripe is the same as that of the data block or the disk stripe in the disk array corresponding to the sub-request. For example, if the disk array corresponding to the sub-request is a horizontally managed disk array, the data stripe is also horizontally distributed, and at the same time, the address range corresponding to each data stripe is the same as the address range of the data block or the disk stripe in the disk array; that is to say, the data stripe is equivalent to a mapped space corresponding to the data block or the disk stripe in the disk array.

[0055] In an embodiment of the present invention, based on the address information corresponding to the multiple sub-requests, the sub-requests are filled into the data stripes containing the corresponding address information, thereby aggregating each sub-request into the data stripe. Thus, through the filling situation of the data stripe, it can be known the specific spatial range of the obtained input / output requests that will be operated on in the disk array.

[0056] Step 130: Aggregate the multiple data stripes in the vertical space to obtain multiple aggregated requests, and send the multiple aggregated requests to the disk array.

[0057] In an embodiment of the present invention, since in the disk array, the response and processing of input / output requests are performed in the vertical space, it is necessary to aggregate the multiple obtained data stripes in the vertical space to obtain aggregated requests corresponding to the vertical space, and then send these aggregated requests to the disk array so that the disk array processes the processing within the relevant address range of the vertical space corresponding to the aggregated requests, thereby completing the response to the obtained input / output requests.

[0058] In an alternative embodiment, Figure 2 is a schematic diagram of splitting and aggregating input / output requests in an input / output request processing method provided by an embodiment of the present invention. Figure 3 is a schematic flowchart of splitting an input / output request into sub-requests in an input / output request method provided by an embodiment of the present invention, as Figure 2And Figure 3 As shown in Figure 3 , in the above step 110, splitting multiple input / output requests into multiple sub-requests respectively may include the following steps:

[0059] Step 310: Obtain the minimum processing unit.

[0060] In an embodiment of the present invention, the minimum processing unit (Grain) is used to represent the address width corresponding to the sub-request. The size of the minimum processing unit, that is, the sector size where the address width is an integer multiple of the power of 2, and its maximum cannot exceed the size of the data block of the disk array in the system. Therefore, when splitting according to the minimum processing unit, the input / output requests can be split according to a granularity smaller than the data block of the disk array, so as to respond to and process the input / output requests from a finer granularity, thereby realizing the granularity management of the memory.

[0061] Step 320: Using the boundary of the minimum processing unit as the splitting boundary, split multiple input / output requests in sequence to obtain multiple sub-requests.

[0062] In an embodiment of the present invention, as Figure 2 shown, using the boundary of the minimum processing unit as the splitting boundary, split multiple input / output requests in sequence, and split multiple input / output requests into corresponding sub-requests (PreIO) respectively.

[0063] Figure 4 FIG. is a schematic diagram of splitting an input / output request into sub-requests in an input / output request processing method provided by an embodiment of the present invention. As Figure 4 shown, IOA and IOB are user IOs, that is, the system obtains two input / output requests, and then uses the boundary of the minimum processing unit Grain as the splitting boundary to split IOA and IOB. IOA is split into 5 sub-requests, and the last sub-request does not reach the boundary of the minimum processing unit during splitting, so its size after splitting is smaller than the size of the minimum processing unit. At the same time, when splitting IOB, because the last sub-request of the previously split IOA does not reach the boundary of the minimum processing unit during splitting, the splitting of IOB does not start from IOB to re-divide the splitting boundary, but IOB is used to fill the boundary of the minimum processing unit that was not reached during the last splitting of IOA. Thus, the last sub-request of IOA and the first sub-request of IOB exactly form a minimum processing unit. That is to say, the splitting of the input / output request is based on the minimum processing unit, rather than the address range of the obtained input / output request, and splitting is performed to fill the minimum processing unit and reach the boundary of the minimum processing unit. If the input / output requests filled when reaching the boundary of the minimum processing unit come from different input / output requests, then the filled input / output requests are further divided into multiple sub-requests according to their sources. To Figure 4For example, if the address range of IOA is 0000 - 0009, the address range of IOB is 000B - 0013, and the size of the minimum processing unit is 4 bits, then IOA is split into sub - requests with address ranges 0000 - 0004, 0004 - 0007, and 0008 - 0009 respectively, and IOB is split into sub - requests with address ranges 000B - 000C, 000D - 0010, and 0011 - 0013 respectively.

[0064] In an alternative embodiment, as Figure 2 shown, after splitting multiple input / output requests into multiple sub - requests (PreIO), each sub - request is aggregated into a data line (DL). Figure 5 FIG. is a schematic flowchart of aggregating sub - requests into data lines in an input / output request processing method provided by an embodiment of the present invention. As Figure 5 shown, in step 120 above, based on the address information corresponding to multiple sub - requests, aggregating multiple sub - requests into corresponding data lines respectively may include the following steps:

[0065] Step 510: If the address range corresponding to the current data line does not include the address range corresponding to the first sub - request, then create a first data line based on the type of the disk array corresponding to the first sub - request, and add the first sub - request to the first data line.

[0066] In an embodiment of the present invention, if the address ranges corresponding to the currently existing data lines do not include the address range corresponding to the first sub - request, it indicates that the first sub - request does not hit any of the currently existing data lines. Therefore, a new data line needs to be registered for the first sub - request. Specifically, create a first data line based on the type of the disk array corresponding to the first sub - request. After creation, add the first sub - request to the first data line, and register the buffer resources corresponding to the first sub - request, thereby completing the aggregation of the first sub - request.

[0067] In an alternative embodiment, since disk arrays have different specifications or attributes and their management methods are also different, it is necessary to create the first data line according to the type of the disk array corresponding to the first data line, that is, the type of the disk array accessed by the first sub - request. Specifically, it can be divided into the following two cases:

[0068] In the first case, if the disk array corresponding to the first sub-request is a first type of disk array, data stripes distributed in the vertical space are constructed, and the depth of the data stripe is set to a preset depth to obtain a first data stripe. The first type of disk array is a disk array that manages the data blocks therein vertically. Specifically, it is a disk array of types such as Jbod, Raid0, or Raid1. Correspondingly, the attribute of the data stripe is set to column so that it can be managed vertically, thereby constructing data stripes distributed in the vertical space. The preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data block in the disk array; that is, the spatial size corresponding to the first data stripe does not exceed the size of one data block in the disk array.

[0069] In the second case, if the disk array corresponding to the first sub-request is a second type of disk array, data stripes distributed in the horizontal space are constructed, and the width of the data stripe is set to a preset width to obtain a first data stripe. The second type of disk array is a disk array that manages the data blocks therein horizontally. Specifically, it is a disk array of types such as RAID5 or RAID6. Correspondingly, the attribute of the data stripe is set to row so that it can be managed horizontally, thereby constructing data stripes distributed in the horizontal space. The preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array, that is, the spatial size corresponding to the first data stripe is the same as the spatial size of the disk stripe, and one data stripe corresponds to one granular stripe.

[0070] Thus, different data stripes are created for different types of disk arrays, so as to perform directional optimization according to the type and specification of the disk array, improve the universality of the input / output request processing method, and enable it to adapt to the requirements of various service scenarios.

[0071] Step 520: If the address range corresponding to the first data stripe contains the address range corresponding to the second sub-request, and the first data stripe is not in the locked state, then add the second sub-request to the first data stripe.

[0072] In the embodiment of the present invention, if the address range corresponding to the first data stripe contains the address range corresponding to the second sub-request, it indicates that the second sub-request hits the first data stripe; if the first data stripe is not in the locked state, it indicates that the first data stripe is still within its life cycle and sub-requests can still be aggregated into the first data stripe. Therefore, at this time, the second sub-request can be added to the first data stripe, and the buffer resource corresponding to the second sub-request is registered at the same time to complete the aggregation of the second sub-request. If there are already other sub-requests registered for the buffer resource corresponding to the second sub-request, then there is no need to register the buffer resource corresponding to the second sub-request at this time.

[0073] In an alternative embodiment, the life cycle of a data stripe is determined by the attributes of the sub-request corresponding to it at the time of creation and the size of the data block or stripe in the corresponding disk array. Taking the first data stripe created in step 510 above as an example, after the first data stripe is created, based on the attributes of the first sub-request and the size of the data block or stripe in the disk array corresponding to the first sub-request, the life cycle of the first data stripe is determined; among them, the attributes of the first sub-request indicate whether the first sub-request is a random input / output request or a sequential input / output request. If the first sub-request is a random input / output request, the probability that the first data stripe will not aggregate other sub-requests after aggregating the first sub-request is relatively high, so the life cycle of the first data stripe can be set relatively short. If the first sub-request is a sequential input / output request, after the first data stripe aggregates the first sub-request, subsequent accesses to other sub-requests will still be received and other sub-requests will be aggregated. Therefore, the life cycle of the first data stripe is set to be longer so that subsequent sequential sub-requests can be aggregated into the first data stripe; the larger the data block or stripe in the disk array corresponding to the first data stripe, the wider the corresponding address range and the more sub-requests that can be aggregated. Therefore, the life cycle of the first data stripe can be set to be longer. Correspondingly, the smaller the data block or stripe in the disk array corresponding to the first data stripe, the narrower the corresponding address range and the fewer sub-requests that can be aggregated. Therefore, the life cycle of the first data stripe can be set to be shorter. Through the above two dimensions, the life cycle of the data stripe is adjusted to meet the requirements of the input / output request and the disk array.

[0074] In an alternative embodiment, after a data stripe is created, its current state is determined in real time according to its life cycle. Taking the first data stripe created in step 510 above as an example, if the life cycle of the first data stripe has not ended, it is determined that the first data stripe is not in a locked state; if the life cycle of the first data stripe has ended, it is determined that the first data stripe is in a locked state.

[0075] In an alternative embodiment, if the address range corresponding to the first data strip includes the address range corresponding to the third sub-request and the first data strip is in a locked state, it indicates that the life cycle of the first data strip has ended. At this time, it is no longer possible to aggregate the sub-requests into the first data strip. Therefore, a shadow data strip corresponding to the first data strip is created, and the third sub-request is added to the shadow data strip. For the same storage space, only two executable data strips can exist simultaneously, namely the foreground data strip and the shadow data strip corresponding to the foreground data strip. When the executable foreground data strip is in a locked state, its corresponding shadow data strip continues to aggregate sub-requests to ensure the smooth processing of input / output requests. At the same time, if the aggregation request corresponding to the first data strip is executed, the first data strip is released, and the shadow data strip corresponding to the first data strip is used as the new first data strip, thereby changing the shadow data strip to the foreground data strip so that the sub-requests aggregated therein can be further aggregated into an aggregation request.

[0076] In an alternative embodiment, as described above, since there are two types of disk arrays, namely the first type of disk array that manages the data blocks longitudinally and the second type of disk array that manages the data blocks horizontally, when aggregating data strips in the longitudinal space to obtain an aggregation request, the corresponding aggregation operations are different according to the type of disk array. Figure 6 is a schematic diagram of obtaining an aggregation request when the disk array corresponding to the data strip in an input / output request processing method provided by an embodiment of the present invention is the first type of disk array. As Figure 6 shown, if the disk array corresponding to the data strip is the first type of disk array, since the first type of disk array itself is also managed longitudinally and its corresponding data strips are also distributed longitudinally, the data strips in the same longitudinal space can be directly aggregated to obtain an aggregation request. Figure 7 is a schematic diagram of obtaining an aggregation request when the disk array corresponding to the data strip in an input / output request processing method provided by an embodiment of the present invention is the second type of disk array. As Figure 7 shown, if the disk array corresponding to the data strip is the second type of disk array, since the second type of disk array itself is managed horizontally and its corresponding data strips are also distributed horizontally, it is necessary to first split multiple data strips and aggregate the sub-requests in the same longitudinal space to obtain an aggregation request. Further combined with Figure 2 、 Figure 6 and Figure 7It can be seen that when aggregating data stripes (DL) into an aggregation request (PstIO), the data stripes (DL) corresponding to the same vertical space can be aggregated first to obtain an aggregated data stripe (MDL). Thus, the aggregation in the vertical space can be directly performed inside the aggregated data stripe to obtain the aggregation request (PstIO). For the data stripes that are not in the same vertical space as it, the aggregation in the vertical space is also directly performed inside them to obtain the aggregation request (PstIO). Therefore, when performing the aggregation in the vertical space, it can be directly performed inside the data stripe or the aggregated data stripe without involving other data stripes, thereby improving the aggregation efficiency.

[0077] In an alternative embodiment, as Figure 7 shown, since the second type of disk array is a RAID5 or RAID6 disk array and it also needs to perform parity calculations, when sending the aggregation request to the second type of disk array, relevant algorithms are also selected according to the stripe data situation. When the stripe is full, the parity is directly calculated through the new data; when the stripe is not full, a read IO for penalty data is first generated, and the new parity is calculated through the penalty data and the new data. Finally, the new data and the new parity are sent to the disk array together. Among them, the number of disks hit in different disk stripes is different, and the corresponding algorithms are also different, and the corresponding penalty data obtained is also different. Taking Figure 7 as an example, for the first data stripe, it uses the read other algorithm to generate parity, and calculates the parity by reading the old data of other disks and the new data of the hit disk. The penalty IO corresponds to Figure 7 the read IO for reading the two old data blocks on the right in Figure 7 ; for the second, third, and fourth data stripes, it uses the read old algorithm to generate parity, and calculates the parity by reading the old data and the new data of the hit disk. The penalty IO corresponds to

[0078] In an alternative embodiment, since there are multiple splits and aggregations in the process from obtaining the input / output request to sending the input / output request to the disk array, in order to avoid errors in the corresponding relationship between requests during the processing and resulting in errors in the final processing and execution, the association or binding information between the requests before and after each split or aggregation is saved.

[0079] In an alternative embodiment, Figure 8 is an application schematic diagram of an input / output request processing method provided by an embodiment of the present invention. The input / output request processing method provided by an embodiment of the present invention can be applied to a system architecture such as Figure 8 shown. As Figure 8As shown in the figure, the system includes a request splitting module (IO PreProc), a data strip management module (DLController), a request aggregation module (IO PstProc), a data cache module (Memory), an association information storage module (MapTable), and a calculation module (Calculator). Among them, the request splitting module (IO PreProc), the data strip management module (DL Controller), and the request aggregation module (IO PstProc) are the main functional modules for input / output request splitting and aggregation. The request splitting module (IO PreProc) splits the received user IO, that is, the input / output request, into sub-requests (PreIO) according to the minimum processing unit. The data strip management module (DL Controller) aggregates each sub-request into a data strip (DL). At the same time, if there are data strips corresponding to the same vertical space, these data strips are further aggregated into an aggregated data strip (MDL). The request aggregation module (IO PstProc) aggregates the data strip or the aggregated data strip in the vertical space to obtain an aggregated request (PstIO) and sends it to the disk array. The data cache module (Memory) serves as a data buffer. When the data strip management module (DL Controller) aggregates sub-requests, buffer resources corresponding to the sub-requests are registered through the data cache module (Memory). The association information storage module (MapTable) is used to store the association or binding information during request splitting and aggregation. The calculation module (Calculator) is responsible for data calculation. This data calculation can be data encryption calculation. If the disk array corresponding to the input / output request is of the RAID5 or RAID6 type, it can also be parity calculation.

[0080] The input / output request processing method provided by the embodiment of the present invention includes: obtaining a plurality of input / output requests, and splitting the plurality of input / output requests into a plurality of sub-requests respectively; based on the address information corresponding to the plurality of sub-requests, aggregating the plurality of sub-requests into corresponding data strips respectively; the data strip corresponds to a data block or a disk stripe in the disk array corresponding to the sub-request; aggregating the plurality of data strips in the vertical space to obtain a plurality of aggregated requests, and sending the plurality of aggregated requests to the disk array. In this way, by splitting the input / output request into smaller sub-requests, the granularity of the input / output request is reduced, and the aggregation is performed according to the corresponding address information to obtain a data strip corresponding to the data block or disk stripe in the disk array. Based on the data strip, an aggregated request corresponding to the data block or disk stripe in the disk array is split and aggregated, so as to realize the granular management of the host memory space, reduce memory fragmentation, and improve memory utilization rate.

[0081] The embodiment of the present invention also provides an input / output request processing device Figure 9It is a schematic structural diagram of an embodiment of an input / output request processing device provided by an embodiment of the present invention. As Figure 9 shown, the device 900 includes:

[0082] A request splitting module 910, configured to obtain a plurality of input / output requests, and split the plurality of input / output requests into a plurality of sub-requests respectively;

[0083] A data stripe determination module 920, configured to aggregate the plurality of sub-requests into corresponding data stripes respectively based on the address information corresponding to the plurality of sub-requests; the data stripe corresponds to a data block or a disk stripe in the disk array corresponding to the sub-request;

[0084] An aggregated request determination module 930, configured to aggregate the plurality of data stripes in the vertical space to obtain a plurality of aggregated requests, and send the plurality of aggregated requests to the disk array.

[0085] In an alternative embodiment, the request splitting module 910 includes:

[0086] A minimum processing unit obtaining unit, configured to obtain a minimum processing unit; the minimum processing unit is used to represent the address width corresponding to the sub-request;

[0087] An input / output request splitting unit, configured to sequentially split the plurality of input / output requests with the boundary of the minimum processing unit as the splitting boundary to obtain a plurality of sub-requests.

[0088] In an alternative embodiment, the data stripe determination module 920 includes:

[0089] A data stripe creation unit, configured to create a first data stripe based on the type of the disk array corresponding to the first sub-request if the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request, and add the first sub-request to the first data stripe;

[0090] A request adding unit, configured to add the second sub-request to the first data stripe if the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request and the first data stripe is not in a locked state.

[0091] In an alternative embodiment, the data stripe creation unit is configured to:

[0092] If the disk array corresponding to the first sub-request is a first type of disk array, construct data stripes distributed in the vertical space, and set the depth of the data stripe to a preset depth to obtain a first data stripe; the first type of disk array is a disk array that manages the data blocks therein longitudinally; the preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data blocks in the disk array;

[0093] If the disk array corresponding to the first sub-request is a second type of disk array, construct data stripes distributed in the horizontal space, and set the width of the data stripes to a preset width to obtain the first data stripe; the second type of disk array is a disk array that manages data blocks therein horizontally; the preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array.

[0094] In an alternative embodiment, the data stripe determination module 920 further includes:

[0095] A shadow data stripe creation unit, configured to create a shadow data stripe corresponding to the first data stripe and add the third sub-request to the shadow data stripe if the address range corresponding to the first data stripe includes the address range corresponding to the third sub-request and the first data stripe is in a locked state;

[0096] A data stripe update unit, configured to release the first data stripe and use the shadow data stripe as the new first data stripe if the aggregation request corresponding to the first data stripe is completed.

[0097] In an alternative embodiment, the data stripe determination module 920 further includes:

[0098] A lifecycle determination unit, configured to determine the lifecycle of the first data stripe based on the attributes of the first sub-request and the size of the data blocks or stripes in the disk array corresponding to the first sub-request;

[0099] A status determination unit, configured to determine that the first data stripe is not in a locked state if the lifecycle of the first data stripe has not ended;

[0100] The status determination unit is further configured to determine that the first data stripe is in a locked state if the lifecycle of the first data stripe has ended.

[0101] In an alternative embodiment, the aggregation request determination module 930 includes:

[0102] If the disk array corresponding to the data stripe is a first type of disk array, aggregate the data stripes located in the same vertical space to obtain an aggregation request;

[0103] If the disk array corresponding to the data stripe is a second type of disk array, aggregate the sub-requests located in the same vertical space among multiple data stripes to obtain an aggregation request.

[0104] The device and method embodiments in this application are based on the same application concept.

[0105] Figure 10 The structural schematic diagram of the embodiment of the computer device provided by the embodiment of the present invention is shown. The specific implementation of the computer device is not limited in the specific embodiments of the present invention.

[0106] As shown Figure 10 in the figure, the computer device may include: a processor 1002, a communications interface 1004, a memory 1006, and a communication bus 1008.

[0107] Among them: the processor 1002, the communications interface 1004, and the memory 1006 communicate with each other through the communication bus 1008. The communications interface 1004 is used to communicate with network elements of other devices such as clients or other servers. The processor 1002 is used to execute the program 1010, and specifically can execute the relevant steps in the above-mentioned embodiments of the input / output request processing method.

[0108] Specifically, the program 1010 may include program code, and the program code includes computer-executable instructions.

[0109] The processor 1002 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computer device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0110] The memory 1006 is used to store the program 1010. The memory 1006 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0111] The program 1010 can specifically be called by the processor 1002 to enable the computer device to execute the relevant steps in the above-mentioned embodiments of the input / output request processing method.

[0112] Those of ordinary skill in the art can understand that Figure 10 the structure shown is only schematic and does not limit the structure of the above-mentioned device. For example, the computer device may also include more or fewer components than Figure 10 shown in the figure, or have a different configuration from Figure 10 shown in the figure.

[0113] An embodiment of the present invention provides a computer-readable storage medium. The storage medium stores at least one executable instruction. When the executable instruction runs on an input / output request processing device or a computer device, the input / output request processing device or the computer device is caused to execute the input / output request processing method in any of the above method embodiments.

[0114] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Additionally, embodiments of the present invention are not directed to any particular programming language.

[0115] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that embodiments of the present invention may be practiced without these specific details. Similarly, in order to streamline the present invention and assist in understanding one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention above, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0116] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-requests or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive.

[0117] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the protection scope of the present invention. The word "comprising" does not exclude the presence of elements or steps not listed in the present invention. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In an embodiment listing several devices, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for processing input and output requests, characterized in that: The method comprises: Acquire multiple input / output requests, and split the multiple input / output requests into multiple sub-requests respectively; splitting the multiple input / output requests into multiple sub-requests respectively includes: acquiring a minimum processing unit; the minimum processing unit is used to represent the address width corresponding to the sub-request, and the size of the minimum processing unit does not exceed the size of the data block of the disk array; using the boundary of the minimum processing unit as the splitting boundary, sequentially splitting the multiple input / output requests to obtain the multiple sub-requests; Based on the address information corresponding to the multiple sub-requests, the multiple sub-requests are respectively aggregated into corresponding data stripes; the data stripes correspond to the data blocks or disk stripes in the disk array corresponding to the sub-requests; The plurality of data stripes are aggregated in a longitudinal space to obtain a plurality of aggregation requests, and the plurality of aggregation requests are sent to the disk array.

2. The method according to claim 1, characterized in that: The step of respectively aggregating the plurality of sub-requests into corresponding data strips based on the address information corresponding to the plurality of sub-requests includes: If the address range corresponding to the current data stripe does not include the address range corresponding to the first sub-request, creating a first data stripe based on the type of the disk array corresponding to the first sub-request, and adding the first sub-request to the first data stripe; If the address range corresponding to the first data stripe includes the address range corresponding to the second sub-request, and the first data stripe is not in a locked state, the second sub-request is added to the first data stripe.

3. The method according to claim 2, characterized in that The creating a first data stripe based on the type of the disk array corresponding to the first sub-request includes: If the disk array corresponding to the first sub-request is a first-type disk array, a data stripe distributed in the longitudinal space is constructed, and the data stripe depth is set to a preset depth to obtain the first data stripe; the first-type disk array is a disk array that manages data blocks therein longitudinally; the preset depth is an integer multiple of the address width of the minimum processing unit, and the preset depth is less than or equal to the size of the data block in the disk array; If the disk array corresponding to the first sub-request is a second-type disk array, a data stripe distributed in the horizontal space is constructed, and the data stripe width is set to a preset width to obtain the first data stripe; the second-type disk array is a disk array that manages data blocks therein horizontally; the preset width is equal to the product of the address width of the minimum processing unit and the number of disks in the disk array.

4. The method according to claim 2, characterized in that: The step of respectively aggregating the plurality of sub-requests into corresponding data strips based on the address information corresponding to the plurality of sub-requests further includes: If the address range corresponding to the first data stripe includes the address range corresponding to the third sub-request, and the first data stripe is in a locked state, creating a shadow data stripe corresponding to the first data stripe, and adding the third sub-request to the shadow data stripe; If the aggregation request corresponding to the first data stripe is executed completely, the first data stripe is released, and the shadow data stripe is used as the new first data stripe.

5. The method according to claim 2 or 4, characterized in that: The step of respectively aggregating the plurality of sub-requests into corresponding data strips based on the address information corresponding to the plurality of sub-requests further includes: Determining a life cycle of the first data stripe based on an attribute of the first sub-request and a size of a data block or stripe in a disk array corresponding to the first sub-request; If the life cycle of the first data stripe has not ended, determining that the first data stripe is not in a locked state; If the life cycle of the first data stripe ends, it is determined that the first data stripe is in a locked state.

6. The method according to claim 1, characterized in that The aggregating the plurality of data strips in a longitudinal spatial manner to obtain a plurality of aggregation requests includes: If the disk array corresponding to the data stripe is a first type disk array, the data stripes located in the same longitudinal space are aggregated to obtain the aggregation request; If the disk array corresponding to the data stripe is a second type of disk array, sub-requests in a same longitudinal space in a plurality of the data stripes are aggregated to obtain the aggregate request.

7. An input / output request processing device, characterized in that: The device comprises: The request splitting module is used to obtain multiple input and output requests and split the multiple input and output requests into multiple sub-requests respectively; the splitting of the multiple input and output requests into multiple sub-requests respectively includes: obtaining a minimum processing unit; the minimum processing unit is used to represent the address width corresponding to the sub-request, and the size of the minimum processing unit does not exceed the size of the data block of the disk array; using the boundary of the minimum processing unit as the splitting boundary, the multiple input and output requests are split in sequence to obtain the multiple sub-requests; A data stripe determination module, configured to aggregate the plurality of sub-requests into corresponding data stripes based on address information corresponding to the plurality of sub-requests; the data stripes correspond to data blocks or disk stripes in the disk array corresponding to the sub-requests; The aggregation request determination module is used to aggregate the plurality of data stripes in a longitudinal space to obtain a plurality of aggregation requests, and send the plurality of aggregation requests to the disk array.

8. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the input and output request processing method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the input / output request processing apparatus or computer device, the input / output request processing apparatus or computer device executes the operation of the input / output request processing method as described in any one of claims 1 to 6.

Images