Method, device, computer device and storage medium for submitting a request

Abstract

The application relates to the technical field of storage, and discloses a method and device for submitting a request, computer equipment and a storage medium, which comprise the following steps: obtaining multiple requests, analyzing the requests to obtain identification information of data objects of the requests, operation types and operation position information; determining identification information of a disk corresponding to a data object according to the identification information of the data object; obtaining a stress index value of a target disk according to the identification information of the target disk; when it is determined that the stress index value is greater than or equal to a preset stress threshold value and multiple target requests corresponding to the target disk are selected from the multiple requests, grouping and merging processing is performed on the multiple target requests according to the identification information of data objects of the target requests and the operation types; the target number of requests finally existing in all groups after the grouping and merging processing is summarized; and all groups of the finally existing requests are submitted to a kernel of a target equipment in batches according to the target number and a preset number threshold value. The application can improve the data read-write speed.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a method, device, computer equipment and storage medium for submitting a request. Background Art

[0002] In cloud storage platforms, storage systems typically manage data through data objects (OBJs). Physical disks are typically divided into multiple areas, including metadata, journal, and data. The metadata area typically stores metadata about data objects, while the journal and data areas store the data themselves. Each data object is composed of data blocks in the metadata area, journal, and data areas.

[0003] During the data write process, when updating a data block within a data object, metadata information must first be retrieved from the metadata area to determine whether the last updated data for that data block was in the data area or the log area. If the last updated data was in the data area, the current update must be written to the corresponding data block in the log area; otherwise, it must be written to the corresponding data block in the data area. This way, some of the latest data for the same data object is in the data area, and some is in the log area. When updating multiple data blocks within the same data object, if the blocks to be updated are not contiguous, multiple requests will be received. The storage system submits only one request to the kernel at a time to perform the corresponding read or write operation.

[0004] In this way, each time a request is submitted, frequent system calls are required, resulting in a waste of resources and slower reading and writing speeds. Summary of the Invention

[0005] In view of this, the present invention provides a method, apparatus, computer device, storage medium and program product for submitting a request to solve the problems of slow reading and writing speeds and long request processing delays.

[0006] In a first aspect, the present invention provides a method for submitting a request, the method comprising:

[0007] Obtain multiple requests, parse each request, and obtain identification information, operation type, and operation location information of a data object corresponding to each request;

[0008] Determining identification information of a disk corresponding to each data object according to identification information of the data object corresponding to each request;

[0009] Acquire a pressure index value of the target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one of the requested data objects;

[0010] When it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold, and a plurality of target requests corresponding to the target disk are selected from the plurality of requests, the plurality of target requests are grouped according to identification information and an operation type of a data object corresponding to each of the plurality of target requests, wherein each group includes one or more target requests;

[0011] When the first group of requests includes multiple target requests, determining whether to merge at least two target requests in the first group to generate one request based on the operation position information corresponding to each target request in the first group of requests, and determining the request that ultimately exists in the first group, wherein the first group of requests is any group of requests after the multiple target requests are grouped;

[0012] Sum up the target number of requests that eventually exist for all groups;

[0013] According to the target quantity and a preset quantity threshold, the finally existing requests of all the groups are submitted to the kernel of the target device in batches, so as to perform data reading and writing operations on the target disk according to the finally existing requests.

[0014] The method for submitting a request provided by the present invention has the following advantages:

[0015] By parsing the request, the identification information, operation type, and operation location information of the data object are obtained. Furthermore, after the target requests belonging to the target disk are determined based on the identification information of the data object, the target requests can be grouped and merged according to the identification information, operation type, and operation location information of the data object corresponding to the target request. The merged requests are submitted to the kernel in batches according to the number of requests after grouping and merging (i.e., the target number). In this way, multiple requests can be submitted when each batch is submitted, which can reduce the number of kernel calls, thereby reducing resource waste and improving read and write speeds. In addition, adjusting the submission strategy according to the actual load of the target disk can ensure the stability of the target disk, which can also improve the service stability of the cloud storage platform.

[0016] In an optional embodiment, when determining that the pressure indicator value is greater than or equal to a preset pressure threshold and selecting multiple target requests corresponding to the target disk from the multiple requests, grouping the multiple target requests according to identification information and operation type of the data object corresponding to each of the multiple target requests includes:

[0017] The target requests with the same identification information of the data objects and the same operation type are grouped together.

[0018] Specifically, by grouping target requests with the same data object identification information and the same operation type, when merging, requests belonging to the same data object, with the same operation type and consecutive locations can be merged into one request, and the merged request can be read or written in one go. This reduces disk seek time and data transfer time, improving disk I / O performance.

[0019] In an optional implementation, submitting the finally existing requests of all groups to the kernel of the target device in batches according to the target number and a preset number threshold includes:

[0020] Determining whether the target quantity is greater than the preset quantity threshold;

[0021] When it is determined that the target number is greater than the preset number threshold, in a first submission batch, submitting a number of requests equal to the preset number threshold from all the requests finally existing in the groups to the kernel as a batch;

[0022] In a current submission batch that is not the first submission batch, obtaining a processing delay of each of the requests submitted in a previous submission batch corresponding to the current submission batch;

[0023] determining whether to adjust a quantity threshold corresponding to the last submission batch based on a processing delay of each of the requests submitted in the last submission batch and a preset delay threshold, wherein when the last submission batch is the first submission batch, the quantity threshold corresponding to the last submission batch is the preset quantity threshold;

[0024] When it is determined that the quantity threshold corresponding to the last submitted batch is to be adjusted, the quantity threshold corresponding to the last submitted batch is adjusted according to a preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch;

[0025] Determine the number of currently unsubmitted requests;

[0026] When it is determined that the number of the currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all the currently unsubmitted requests are submitted to the kernel as a batch.

[0027] Specifically, the batch size threshold is dynamically adjusted based on the processing latency of the previous submitted batch, ensuring efficient operation even under high load and improving disk stability. Submitting requests in batches reduces the overhead of system calls and context switches, improving request response speed and processing efficiency. Furthermore, the number of requests submitted in each batch is determined based on actual load conditions, ensuring more efficient utilization of disk and other resources and avoiding resource waste and overload.

[0028] In an optional embodiment, the method further includes:

[0029] When it is determined that the target number is less than or equal to the preset number threshold, all the requests that finally exist in the groups are submitted to the kernel as a batch.

[0030] Specifically, when the number of requests is small, submitting all requests to the kernel at the same time can reduce the number of system calls.

[0031] In an optional embodiment, the method further includes:

[0032] When it is determined that the pressure indicator value is less than the preset pressure threshold and a plurality of target requests corresponding to the target disk are selected from the plurality of requests, obtaining interface information corresponding to each of the plurality of target requests;

[0033] Acquire metadata information of the data object corresponding to each target request according to identification information of the data object corresponding to each target request;

[0034] determining a type of the first target request according to interface information of the first target request and / or metadata information of a data object corresponding to the first target request, wherein the first target request is any one of a plurality of target requests, the type includes a first type or a second type, and a response level of requests of the first type is higher than a response level of requests of the second type;

[0035] Submitting the first type of target requests to the kernel in sequence;

[0036] Furthermore, when there are multiple target requests of the second type, requests obtained by merging the target requests of the second type are submitted to the kernel in batches.

[0037] Specifically, high-response-level requests are prioritized to ensure timely completion of critical tasks. In addition, low-response-level requests are merged to reduce the overhead of system calls and context switches, thereby improving resource utilization.

[0038] In an optional implementation manner, determining the type of the first target request according to the interface information of the first target request and / or metadata information of the data object corresponding to the first target request includes:

[0039] When it is determined that the interface information of the first target request matches the preset interface information, determining the type of the first target request as the first type;

[0040] or,

[0041] When it is determined that the interface information of the first target request does not match the preset interface information, determining whether the metadata information corresponding to the first target request includes preset marking information;

[0042] When it is determined that the metadata information corresponding to the first target request includes the preset tag information, determining the type of the first target request as the first type;

[0043] or,

[0044] When it is determined that the metadata information corresponding to the first target request does not include the preset tag information, the type of the first target request is determined to be the second type.

[0045] Specifically, through the dual checking of the interface information and the metadata information, the requests can be classified more accurately, which ensures the correct identification of high priority requests (first type) and low priority requests (second type).

[0046] In an optional embodiment, the method further includes:

[0047] When it is determined that the pressure indicator value is less than the preset pressure threshold, or only one target request corresponding to the target disk is selected from the multiple requests, each of the at least one target request corresponding to the target disk is submitted to the kernel in sequence.

[0048] Specifically, when resources are sufficient or the number of requests is small, submitting one request at a time can improve the response speed of the request and reduce the processing delay.

[0049] In a second aspect, the present invention provides an apparatus for submitting a request, the apparatus comprising:

[0050] An acquisition module, configured to acquire multiple requests, parse each request, and obtain identification information, operation type, and operation location information of a data object corresponding to each request;

[0051] a determination module, configured to determine identification information of a disk corresponding to each data object according to identification information of the data object corresponding to each request;

[0052] The acquisition module is further configured to acquire a pressure index value of the target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object;

[0053] a grouping module, configured to, when determining that the pressure indicator value is greater than or equal to a preset pressure threshold and selecting multiple target requests corresponding to the target disk from the multiple requests, group the multiple target requests according to identification information and operation type of the data object corresponding to each of the multiple target requests, wherein each group includes one or more target requests;

[0054] The determining module is further configured to, when the first group of requests includes a plurality of the target requests, determine, based on the operation location information corresponding to each of the target requests in the first group of requests, whether to merge at least two of the target requests in the first group to generate one request, and determine the request that ultimately exists in the first group, wherein the first group of requests is any group of requests formed by grouping the plurality of the target requests;

[0055] A summary module is used to summarize the target number of requests that eventually exist in all groups;

[0056] The submission module is used to submit the final existing requests of all groups to the kernel of the target device in batches according to the target number and a preset number threshold, so as to perform data reading and writing operations on the target disk according to the final existing requests.

[0057] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the method for submitting a request according to the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0058] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for submitting a request according to the first aspect or any corresponding embodiment thereof.

[0059] In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the method for submitting a request according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 is a schematic diagram of a disk layout according to an embodiment of the present invention;

[0062] Figure 2 is a schematic diagram of storage distribution of data objects according to an embodiment of the present invention;

[0063] Figure 3 is a flowchart of a method for submitting a request according to an embodiment of the present invention;

[0064] Figure 4 is a flowchart of another method for submitting a request according to an embodiment of the present invention;

[0065] Figure 5 is a structural block diagram of an apparatus for submitting a request according to an embodiment of the present invention;

[0066] Figure 6 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0068] Embodiments of the present invention can be applied to a server, which may be equipped with a storage system and an operating system. The storage system is used to manage multiple disks mounted on the server. The operating system may include a kernel, which is the core component of the operating system and is responsible for managing system resources and providing services to other software. For example, the other software may be the aforementioned storage system. During data reading and writing, the storage system processes requests received from clients, or requests generated by itself, and submits these requests to the operating system kernel after processing, which then performs read and write operations on the disks.

[0069] like Figure 1 As shown, a disk can be divided into multiple areas, including the metadata area, the journal area, and the data area. Each area is further divided into multiple data blocks. The metadata area is typically used to store metadata information about data objects, while the journal and data areas are used to store the data itself. Each data object can be composed of data blocks in the metadata area, data blocks in the journal area, and data blocks in the data area. In this way, each disk can contain data blocks corresponding to multiple data objects.

[0070] like Figure 2 As shown, the data writing process is described below taking any data object as an example.

[0071] When the data object is written for the first time, the data is first written to the data blocks of the data object in the data area, for example, the 1st, 2nd, 4th, 6th and 7th data blocks. In order to avoid abnormal situations such as data writing errors during the writing process, when writing data for the second time, it can be written to the corresponding position in the log area first, for example, the 2nd, 4th, 6th and 7th data blocks. In this way, when an abnormality occurs in the second writing process, it can also be guaranteed that the data written for the first time still exists. When writing data for the third time, it can be written to the corresponding position in the data area again, for example, the 4th and 7th data blocks. When writing data for the fourth time, it is written to the corresponding position in the log area, for example, the 7th data block. Under such data writing rules, the latest data of the data object will be scattered across the log area and the data area, and the location of the stored data is discontinuous. For example, in Figure 2 The latest data of the first data block is in the data area, the latest data of the second data block is in the log area, the third data block does not store data, and the latest data of the fourth data block is in the data area.

[0072] When reading or updating these data blocks, the operation locations are discontinuous. If a request is received for an operation on the first, second, and third data blocks, the latest data in these three blocks is discontinuously stored. This splits the request into three sub-requests, multiplying the request by one, which in turn increases the number of requests. Furthermore, due to the limited storage space in the log area, a data flush operation is performed when the log area runs out of space. During this data flush, the discontinuous data blocks storing the latest data generate a large number of read and write requests.

[0073] After generating a request, the storage system needs to call the operating system kernel, and the operating system will continue to perform subsequent request processing operations, that is, read or write operations on the disk. Each time a request is processed, the operating system needs to be called once, resulting in a large amount of resources being wasted on system calls and reducing request throughput.

[0074] An embodiment of the present invention provides a method for submitting requests, which determines the submission strategy according to the actual load conditions, merges various requests generated by the storage system when the load is high, and submits the merged requests in batches to the operating system for processing, thereby reducing the number of system calls and improving request throughput.

[0075] According to an embodiment of the present invention, an embodiment of a method for submitting a request is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0076] In this embodiment, a method for submitting a request is provided, which can be executed by a target device, which can be the server mentioned above. Figure 3 is a flowchart of a method for submitting a request according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0077] Step S301: Acquire multiple requests, parse each request, and obtain identification information, operation type, and operation location information of the data object corresponding to each request.

[0078] Among them, the multiple requests may include the requests obtained by splitting the total requests received from the client, and / or the requests generated by the storage system when flushing data from the log area to the data area. For the former, the requests can be divided into requests for updating disk information and requests for updating business data on the disk, or the requests can be divided into ordinary requests or critical requests, and the processing delay level of critical requests is higher than the processing delay level of ordinary requests. The operation type can be a read operation or a write operation. The operation position information may include the starting position and data length of the data operated by the request on the disk, and the data length can be represented by the number of data blocks.

[0079] Step S302: Determine identification information of a disk corresponding to each data object according to identification information of the data object corresponding to each request.

[0080] Specifically, for any request, the target device can determine the disk identification information corresponding to the identification information of the data object corresponding to the request in the pre-built correspondence table between the identification information of the data object and the identification information of the disk based on the identification information of the data object corresponding to the request.

[0081] Alternatively, the target device may obtain metadata information included in the data object corresponding to the request based on the identification information of the data object corresponding to the request, and extract the identification information of the disk from the metadata information.

[0082] Step S303: Obtain the pressure index value of the target disk according to the identification information of the target disk.

[0083] The target disk is the disk corresponding to the data object of any of the multiple requests, and the target disk corresponds to at least one of the requested data objects. The stress indicator value can be one of the input / output operations per second (IOPS), throughput, and utilization (the percentage of time the disk is busy over a period of time). It can also be calculated based on a preset algorithm to combine IOPS, throughput, and utilization.

[0084] Specifically, the target device can monitor the load of all mounted disks in real time and update the pressure index values ​​obtained in real time or calculated in real time to the target data table. The target data table can include the identification information of each disk and its pressure index value. After receiving multiple requests, the corresponding pressure index value can be obtained from the target data table based on the identification information of the disk corresponding to any request.

[0085] Alternatively, the target device may obtain the identification information corresponding to the request after parsing, and then collect the pressure index value of the target disk according to the identification of the target disk.

[0086] Step S304, when it is determined that the pressure index value is greater than or equal to the preset pressure threshold, and multiple target requests corresponding to the target disk are selected from multiple requests, the multiple target requests are grouped according to the identification information and operation type of the data object corresponding to each of the multiple target requests.

[0087] Each group may include one or more target requests.

[0088] Specifically, the target device may determine whether a pressure index value of the target disk is greater than or equal to a preset pressure threshold, and, based on the identification information of the target disk, select, from the plurality of requests, a request whose disk identification information is the identification information of the target disk. When it is determined that the pressure index value is greater than or equal to the preset pressure threshold, and when a plurality of target requests corresponding to the target disk are selected from the plurality of requests, the plurality of target requests are grouped according to the identification information and operation type of the data object corresponding to each of the plurality of target requests.

[0089] When it is determined that the pressure indicator value is less than a preset pressure threshold, or when a target request corresponding to the target disk is selected from the multiple requests, each target request may be submitted to the kernel in sequence.

[0090] The specific process for grouping multiple target requests based on the operation type and data object identification information described above can be as follows: first, target requests with the same data object identification information are grouped together, and then, within each group, target requests with the same operation type are grouped together. Alternatively, target requests with corresponding operation types can be grouped together, and then, within each group, target requests with the same data object identification information are grouped together. In this way, target requests with the same data object identification information and the same operation type can be grouped together, resulting in one or more groups.

[0091] In some optional implementations, after selecting the target request corresponding to the target disk, any one request can be selected from the remaining requests, and the above-mentioned steps S303 and S304 can be executed according to the identification information of the disk corresponding to the request, and so on, to complete the group processing of all obtained requests.

[0092] Step S305, when the first group of requests includes multiple target requests, determine whether to merge at least two target requests in the first group according to the operation location information corresponding to each target request in the first group of requests, generate a request, and determine the request that finally exists in the first group.

[0093] The first group of requests is any group of requests obtained by grouping multiple target requests.

[0094] Specifically, for the first group of requests, when the first group of requests includes one target request, merging is not performed. When the first group of requests includes multiple target requests, the target device may first sort each target request in the first group of requests based on the operation position information corresponding to each target request in the first group of requests. Then, the target device may determine whether the operation positions of the first target request and the second target request are continuous based on the operation position information of the first target request and the operation position information of the second target request. If the operation positions are continuous, it is determined that the first target request and the second target request can be merged. If the operation positions are discontinuous, it is determined that the first target request and the second target request cannot be merged. The first target request and the second target request are any two target requests that are sequentially sorted in the first group of requests, and the first target request is ranked before the second target request. After determining that multiple target requests can be merged into one request, these multiple target requests are merged to generate a new request. In this way, the requests ultimately remaining in the first group may include one or more generated requests and / or target requests that do not have a merge relationship with any target request.

[0095] The above-mentioned determination of whether the operation positions of the first target request and the second target request are continuous based on the operation position information of the first target request and the operation position information of the second target request can be processed in the following steps:

[0096] The first end position is determined based on the starting position and data length in the operation position information of the first target request. Furthermore, it can be determined whether the first end position and the starting position in the operation position information of the second target request are adjacent. If so, it is determined that the operation positions of the first target request and the second target request are continuous; if not, it is determined that the operation positions of the first target request and the second target request are discontinuous. When the data length is represented by the number of data blocks, the first end position can be calculated based on the starting position and the number of data blocks in the operation position information of the first target request, as well as a preset data block size.

[0097] Step S306: Summarize the target number of requests that ultimately exist in all groups.

[0098] Step S307 : Submitting all groups of final existing requests to the kernel of the target device in batches according to the target number and the preset number threshold, so as to perform data read and write operations on the target disk according to the final existing requests.

[0099] Specifically, step S307 may include the following steps:

[0100] Step 1: Determine whether the target quantity is greater than a preset quantity threshold.

[0101] In step 2, when it is determined that the target number is less than or equal to the preset number threshold, all the groups' final existing requests are submitted to the kernel as a batch.

[0102] In step three, when it is determined that the target quantity is greater than the preset quantity threshold, in the first submission batch, all requests that ultimately exist in all groups and have a quantity equal to the preset quantity threshold are submitted to the kernel as a batch.

[0103] Step 4: In a current submission batch that is not the first submission batch, obtain the processing delay of each request submitted in a previous submission batch corresponding to the current submission batch.

[0104] Step 5: Determine whether to adjust the quantity threshold corresponding to the last submitted batch based on the processing delay of each request submitted in the last submitted batch and the preset delay threshold.

[0105] Among them, when the last submitted batch is the first submitted batch, the quantity threshold corresponding to the last submitted batch is the preset quantity threshold.

[0106] Specifically, when the target device determines that the processing delay of any request in the last submitted batch is greater than or less than a preset delay threshold, it determines to adjust the quantity threshold corresponding to the last submitted batch. When the target device determines that the processing delay of any request in the last submitted batch is equal to the preset delay threshold, it determines not to adjust the quantity threshold corresponding to the last submitted batch.

[0107] In some optional embodiments, the target device may be set with a preset ratio. When the processing delay of requests greater than or equal to the preset ratio in the last submitted batch is greater than or less than the preset delay threshold, it is determined that the quantity threshold corresponding to the last submitted batch is adjusted. When the processing delay of only requests less than the preset ratio in the last submitted batch is greater than or less than the preset delay threshold, it is determined that the quantity threshold corresponding to the last submitted batch is not adjusted. In this way, by determining whether to adjust the quantity threshold based on the processing delays of multiple requests, the adjusted quantity threshold can be made more accurate.

[0108] Step 6: When it is determined to adjust the quantity threshold corresponding to the last submitted batch, the quantity threshold corresponding to the last submitted batch is adjusted according to the preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch.

[0109] Specifically, when the target device determines that the processing time of any request of the last submitted batch is less than the preset delay threshold, the preset quantity adjustment rule may be to increase the quantity threshold corresponding to the last submitted batch by a preset percentage. For example, the quantity threshold corresponding to the last submitted batch is 50, the preset percentage is 50%, and the quantity threshold of the current submitted batch may be 75. When the target device determines that the processing delay of any request of the last submitted batch is greater than the preset delay threshold, the preset quantity adjustment rule may be to decrease the quantity threshold corresponding to the last submitted batch by a preset percentage. For example, the quantity threshold corresponding to the last submitted batch is 50, the preset percentage is 50%, and the quantity threshold of the current submitted batch may be 25.

[0110] Step 7: Determine the number of currently unsubmitted requests.

[0111] Step eight: when it is determined that the number of currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all currently unsubmitted requests are submitted to the kernel as a batch.

[0112] Step nine, when it is determined that the number of currently unsubmitted requests is greater than the number threshold corresponding to the currently submitted batch, the number of requests equal to the number threshold corresponding to the currently submitted batch among the unsubmitted requests in all groups that ultimately exist is submitted to the kernel as a batch, and enters the next submission round corresponding to the current submission round.

[0113] In some optional implementations, all requests that ultimately exist in the group corresponding to the target disk can be placed in the same context. Each time a request is submitted, a corresponding number of requests are extracted from the context and submitted to the kernel. The context refers to processing multiple requests as a whole. Reinitializing the context each time a request is processed would incur additional overhead. Therefore, allowing multiple requests to share the same context avoids repeated initialization and cleanup work. Furthermore, multiple requests can share already open resources, reducing the frequent opening and closing of resources and thereby increasing processing speed.

[0114] In some optional embodiments, when multiple requests are obtained by splitting the total requests received from the client, the above steps S301 to S307 can be used for processing and submission. When the disk identifiers corresponding to the multiple requests are all identification information of the target disk, and are all generated by the storage system when flushing data from the log area to the data area, after completing the grouping operation of step S304, the target device can be set with a data object quantity threshold. The target device can select at least one target group from at least one group obtained by grouping based on the data object quantity threshold (for example, if the data object quantity threshold is 5, all target groups corresponding to 5 data objects are selected from at least one group). Then, based on the number of requests in all selected target groups and the preset quantity threshold, the number of requests required to be submitted for the current submission batch is determined (specifically, the minimum value of the two can be selected). Finally, based on the determined number of requests required to be submitted for the current submission batch, the corresponding number of requests in all target groups can be submitted to the kernel as a batch. If there are still remaining requests in all target groups after the processing of the current submitted batch is completed, they can be processed in the same way, and the requests in all selected target groups can be submitted to the kernel for data read and write operations.

[0115] Because flushing N discontinuous data blocks from the log area to the data area generates N read requests and N write requests, if a large number of data blocks are flushed simultaneously, the flushing process will be blocked and a large amount of submission queue space will be occupied, affecting the request processing performance of the target device. Therefore, by limiting the number of data objects flushed and the number of requests per flush, the number of data objects and requests flushed at the same time can be reduced, avoiding the problem of flushing process blocking and large submission queue space occupation, thereby improving the request processing performance of the target device.

[0116] The method for submitting a request provided by this embodiment obtains the identification information, operation type and operation location information of the data object by parsing the request. Furthermore, after determining the target request belonging to the target disk based on the identification information of the data object, the target request can be grouped and merged according to the identification information, operation type and operation location information of the data object corresponding to the target request, and the merged requests are submitted to the kernel in batches according to the number of requests after grouping and merging (that is, the target number). In this way, multiple requests can be submitted when each batch is submitted, which can reduce the number of times the kernel is called, thereby reducing resource waste and improving read and write speeds. In addition, adjusting the submission strategy according to the actual load of the target disk can ensure the stability of the target disk, that is, it can improve the service stability of the cloud storage platform.

[0117] Since different types of requests have different requirements for processing latency, for example, the above-mentioned requests for updating disk information and key requests have higher requirements for processing latency, and require the use of most resources to process them in order to speed up the processing speed, these requests can adopt a synchronous submission strategy, that is, submitting each request to the kernel one by one. Therefore, when the resources of the target disk are sufficient, that is, when it is determined that the pressure index value is less than the preset pressure threshold, and multiple target requests corresponding to the target disk are selected from multiple requests, a method for submitting requests provided in this embodiment can be executed, which can be executed by the target device, which can be the above-mentioned target device. Figure 4 is a flowchart of a method for submitting a request according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0118] Step S401: Acquire interface information corresponding to each of a plurality of target requests.

[0119] The interface information may be identification information of the interface, for example, the number and name of the interface.

[0120] Step S402: acquiring metadata information of the data object corresponding to each target request according to identification information of the data object corresponding to each target request.

[0121] Specifically, the target device may determine the data object corresponding to each target request based on the identification information of the data object corresponding to each target request, and extract metadata information of the data object corresponding to each target request.

[0122] Step S403: Determine the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request.

[0123] The first target request is any one of the multiple target requests, and the type includes either the first type or the second type. The response level for the first type request is higher than the response level for the second type request. The first type request may include the aforementioned request to update disk information and / or a critical request, and the second type request may include the aforementioned request to update business data on the disk and / or a normal request.

[0124] Specifically, the request to update the disk information may be obtained through a preset interface. The metadata information corresponding to the key request may include preset tag information. Accordingly, step S403 may be implemented in the following ways:

[0125] Method 1: When it is determined that the interface information of the first target request matches the preset interface information, the type of the first target request is determined to be the first type. Alternatively, when it is determined that the interface information of the first target request does not match the preset interface information, it is determined whether the metadata information corresponding to the first target request includes preset tag information. When it is determined that the metadata information corresponding to the first target request includes preset tag information, the type of the first target request is determined to be the first type. Alternatively, when it is determined that the metadata information corresponding to the first target request does not include preset tag information, the type of the first target request is determined to be the second type.

[0126] Method 2: When it is determined that the metadata information corresponding to the first target request includes preset tag information, the type of the first target request is determined to be the first type. Alternatively, when it is determined that the metadata information corresponding to the first target request does not include preset tag information, it is determined whether the interface information of the first target request matches the preset interface information. When it is determined that the interface information of the first target request matches the preset interface information, the type of the first target request is determined to be the first type. When it is determined that the interface information of the first target request does not match the preset interface information, the type of the first target request is determined to be the second type.

[0127] Method three: Determine whether the interface information of the first target request matches the preset interface information, and determine whether the metadata information corresponding to the first target request includes preset tag information. When it is determined that the interface information of the first target request matches the preset interface information, or when it is determined that the metadata information corresponding to the first target request includes preset tag information, the type of the first target request is determined to be the first type. When it is determined that the interface information of the first target request does not match the preset interface information, and when it is determined that the metadata information corresponding to the first target request does not include preset tag information, the type of the first target request is determined to be the second type.

[0128] Step S404: Submit the first type of target requests to the kernel in sequence.

[0129] Specifically, for the first type of target request, since it has a relatively high requirement on processing delay, the first type of target request can be submitted to the kernel one by one.

[0130] Step S405 : When there are multiple target requests of the second type, the requests obtained by merging the target requests of the second type are submitted to the kernel in batches.

[0131] The specific processing of step S405 may be similar to the specific processing of step S307 and will not be repeated here.

[0132] The request submission method provided in this embodiment dynamically adjusts the number threshold for the current batch based on the processing delay of the previously submitted batch, ensuring efficient operation even under high load conditions and improving disk stability. Submitting requests in batches reduces the overhead of system calls and context switches, improving request response speed and processing efficiency. Furthermore, the number of requests submitted in each batch is determined based on the actual load, resulting in more efficient utilization of disk and other resources and avoiding resource waste and overload.

[0133] This embodiment also provides a device for submitting a request, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0134] This embodiment provides a device for submitting a request, such as Figure 5 Shown, including:

[0135] The acquisition module 501 is used to acquire multiple requests, parse each request, and obtain the identification information, operation type, and operation location information of the data object corresponding to each request;

[0136] A determination module 502 is configured to determine identification information of a disk corresponding to each data object based on identification information of the data object corresponding to each request;

[0137] The acquisition module 501 is further configured to acquire a pressure index value of a target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object;

[0138] a grouping module 503 configured to, when determining that the pressure indicator value is greater than or equal to a preset pressure threshold and selecting multiple target requests corresponding to the target disk from the multiple requests, group the multiple target requests according to identification information and operation type of the data object corresponding to each of the multiple target requests, wherein each group includes one or more target requests;

[0139] The determination module 502 is further configured to, when the first group of requests includes multiple target requests, determine, based on the operation location information corresponding to each target request in the first group of requests, whether to merge at least two target requests in the first group to generate one request, and determine the request that ultimately exists in the first group, wherein the first group of requests is any group of requests after the multiple target requests are grouped together;

[0140] A summarizing module 504 is used to summarize the target number of requests that ultimately exist in all groups;

[0141] The submission module 505 is used to submit all groups of final existing requests to the kernel of the target device in batches according to the target number and the preset number threshold, so as to perform data read and write operations on the target disk according to the final existing requests.

[0142] In an optional implementation, the grouping module 503 is specifically configured to:

[0143] Target requests with the same data object identification information and the same operation type are grouped together.

[0144] In an optional implementation, the submission module 505 is specifically configured to:

[0145] Determining whether the target quantity is greater than a preset quantity threshold;

[0146] When it is determined that the target quantity is greater than the preset quantity threshold, in the first submission batch, all requests that ultimately exist in all groups and have a quantity equal to the preset quantity threshold are submitted to the kernel as a batch;

[0147] In a current submission batch that is not the first submission batch, obtain the processing latency of each request submitted in the previous submission batch corresponding to the current submission batch;

[0148] Determining whether to adjust the quantity threshold corresponding to the last submission batch based on the processing delay of each request submitted in the last submission batch and the preset delay threshold, wherein when the last submission batch is the first submission batch, the quantity threshold corresponding to the last submission batch is the preset quantity threshold;

[0149] When it is determined that the quantity threshold corresponding to the last submitted batch is to be adjusted, the quantity threshold corresponding to the last submitted batch is adjusted according to the preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch;

[0150] Determine the number of currently unsubmitted requests;

[0151] When it is determined that the number of currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batches, all currently unsubmitted requests are submitted to the kernel as a batch.

[0152] In an optional implementation, the submission module 505 is further configured to:

[0153] When it is determined that the target number is less than or equal to the preset number threshold, all requests that eventually exist in the group are submitted to the kernel as a batch.

[0154] In an optional implementation, the submission module 505 is further configured to:

[0155] When it is determined that the pressure indicator value is less than a preset pressure threshold and multiple target requests corresponding to the target disk are selected from the multiple requests, interface information corresponding to each of the multiple target requests is obtained;

[0156] Obtaining metadata information of the data object corresponding to each target request based on identification information of the data object corresponding to each target request;

[0157] determining, according to the interface information of the first target request and / or metadata information of the data object corresponding to the first target request, a type of the first target request, wherein the first target request is any one of a plurality of target requests, the type includes a first type or a second type, and a response level of the first type request is higher than a response level of the second type request;

[0158] Submitting the first type of target requests to the kernel in sequence;

[0159] Furthermore, when there are multiple target requests of the second type, requests obtained after merging the target requests of the second type are submitted to the kernel in batches.

[0160] In an optional implementation, the submission module 505 is specifically configured to:

[0161] When it is determined that the interface information of the first target request matches the preset interface information, determining the type of the first target request as the first type;

[0162] or,

[0163] When it is determined that the interface information of the first target request does not match the preset interface information, determining whether the metadata information corresponding to the first target request includes preset marking information;

[0164] When it is determined that the metadata information corresponding to the first target request includes preset tag information, determining the type of the first target request as a first type;

[0165] or,

[0166] When it is determined that the metadata information corresponding to the first target request does not include the preset tag information, the type of the first target request is determined to be the second type.

[0167] In an optional implementation, the submission module 505 is further configured to:

[0168] When it is determined that the pressure indicator value is less than the preset pressure threshold, or only one target request corresponding to the target disk is selected from the multiple requests, each at least one target request corresponding to the target disk is sequentially submitted to the kernel.

[0169] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0170] The device for submitting a request in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0171] The embodiment of the present invention also provides a computer device having the above Figure 6 The device for submitting the request is shown.

[0172] See also Figure 6 , Figure 6 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0173] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0174] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0175] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0177] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0178] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0179] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0180] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for submitting a request, characterized in that: The method comprises: Obtain multiple requests, parse each request, and obtain identification information, operation type, and operation location information of a data object corresponding to each request; Determining identification information of a disk corresponding to each data object according to identification information of the data object corresponding to each request; Acquire a pressure index value of the target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one of the requested data objects; When it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold, and a plurality of target requests corresponding to the target disk are selected from the plurality of requests, the plurality of target requests are grouped according to identification information and an operation type of a data object corresponding to each of the plurality of target requests, wherein each group includes one or more target requests; When the first group of requests includes multiple target requests, determining whether to merge at least two target requests in the first group to generate one request based on the operation position information corresponding to each target request in the first group of requests, and determining the request that ultimately exists in the first group, wherein the first group of requests is any group of requests after the multiple target requests are grouped; Sum up the target number of requests that eventually exist for all groups; According to the target quantity and a preset quantity threshold, the finally existing requests of all the groups are submitted to the kernel of the target device in batches, so as to perform data reading and writing operations on the target disk according to the finally existing requests.

2. The method according to claim 1, characterized in that When it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold, and multiple target requests corresponding to the target disk are selected from the multiple requests, grouping the multiple target requests according to identification information and operation type of the data object corresponding to each of the multiple target requests includes: The target requests with the same identification information of the data objects and the same operation type are grouped together.

3. The method according to claim 1 or 2, characterized in that Submitting the finally existing requests of all groups to the kernel of the target device in batches according to the target number and a preset number threshold includes: Determining whether the target quantity is greater than the preset quantity threshold; When it is determined that the target number is greater than the preset number threshold, in a first submission batch, submitting a number of requests equal to the preset number threshold from all the requests finally existing in the groups to the kernel as a batch; In a current submission batch that is not the first submission batch, obtaining a processing delay of each of the requests submitted in a previous submission batch corresponding to the current submission batch; determining whether to adjust a quantity threshold corresponding to the last submission batch based on a processing delay of each of the requests submitted in the last submission batch and a preset delay threshold, wherein when the last submission batch is the first submission batch, the quantity threshold corresponding to the last submission batch is the preset quantity threshold; When it is determined that the quantity threshold corresponding to the last submitted batch is to be adjusted, the quantity threshold corresponding to the last submitted batch is adjusted according to a preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch; Determine the number of currently unsubmitted requests; When it is determined that the number of the currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all the currently unsubmitted requests are submitted to the kernel as a batch.

4. The method according to claim 3, characterized in that The method further comprises: When it is determined that the target number is less than or equal to the preset number threshold, all the requests that finally exist in the groups are submitted to the kernel as a batch.

5. The method according to claim 1 or 2, characterized in that The method further comprises: When it is determined that the pressure indicator value is less than the preset pressure threshold and a plurality of target requests corresponding to the target disk are selected from the plurality of requests, obtaining interface information corresponding to each of the plurality of target requests; Acquire metadata information of the data object corresponding to each target request according to identification information of the data object corresponding to each target request; determining a type of the first target request according to interface information of the first target request and / or metadata information of a data object corresponding to the first target request, wherein the first target request is any one of a plurality of target requests, the type includes a first type or a second type, and a response level of requests of the first type is higher than a response level of requests of the second type; Submitting the first type of target requests to the kernel in sequence; Furthermore, when there are multiple target requests of the second type, requests obtained by merging the target requests of the second type are submitted to the kernel in batches.

6. The method according to claim 5, characterized in that The determining the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request includes: When it is determined that the interface information of the first target request matches the preset interface information, determining the type of the first target request as the first type; or, When it is determined that the interface information of the first target request does not match the preset interface information, determining whether the metadata information corresponding to the first target request includes preset marking information; When it is determined that the metadata information corresponding to the first target request includes the preset tag information, determining the type of the first target request as the first type; or, When it is determined that the metadata information corresponding to the first target request does not include the preset tag information, the type of the first target request is determined to be the second type.

7. The method according to claim 1 or 2, characterized in that The method further comprises: When it is determined that the pressure indicator value is less than the preset pressure threshold, or only one target request corresponding to the target disk is selected from the multiple requests, each of the at least one target request corresponding to the target disk is submitted to the kernel in sequence.

8. A device for submitting a request, characterized in that: The device comprises: An acquisition module, configured to acquire multiple requests, parse each request, and obtain identification information, operation type, and operation location information of a data object corresponding to each request; a determination module, configured to determine identification information of a disk corresponding to each data object according to identification information of the data object corresponding to each request; The acquisition module is further configured to acquire a pressure index value of the target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object; a grouping module, configured to, when determining that the pressure indicator value is greater than or equal to a preset pressure threshold and selecting multiple target requests corresponding to the target disk from the multiple requests, group the multiple target requests according to identification information and operation type of the data object corresponding to each of the multiple target requests, wherein each group includes one or more target requests; The determining module is further configured to, when the first group of requests includes a plurality of the target requests, determine, based on the operation location information corresponding to each of the target requests in the first group of requests, whether to merge at least two of the target requests in the first group to generate one request, and determine the request that ultimately exists in the first group, wherein the first group of requests is any group of requests formed by grouping the plurality of the target requests; A summary module is used to summarize the target number of requests that eventually exist in all groups; The submission module is used to submit the final existing requests of all groups to the kernel of the target device in batches according to the target number and a preset number threshold, so as to perform data reading and writing operations on the target disk according to the final existing requests.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for submitting a request according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for submitting a request according to any one of claims 1 to 7.

Images