Request submission method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of storage, and discloses a request submission method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a plurality of requests, and analyzing the requests to obtain identification information, operation types and operation position information of data objects of the requests; determining identification information of a disk corresponding to the data object according to the identification information of the requested data object; obtaining a pressure index value of the target disk according to the identification information of the target disk; when it is determined that the pressure index value is larger than or equal to a preset pressure threshold value and multiple target requests corresponding to the target disk are selected from the multiple requests, the multiple target requests are grouped and merged according to identification information and operation types of data objects of the target requests; summarizing the target number of the requests finally existing in all groups after grouping and merging processing; and according to the target number and a preset number threshold value, submitting the requests finally existing in all the groups to the kernel of the target equipment in batches. According to the invention, the data read-write speed can be improved.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a method, device, computer equipment and storage medium for submitting a request. Background Art

[0002] In cloud storage platforms, storage systems generally manage data through data objects (OBJ). Physical disks are generally divided into multiple areas, such as metadata area, journal area, and data area. The metadata area is usually used to store metadata information of data objects, and the journal area and data area are used to store the data itself. Each data object can be composed of data blocks in the metadata area, data blocks in the journal area, and data blocks in the data area.

[0003] In the process of writing data, when updating the data block of the stored data in the data object, it is necessary to first obtain metadata information from the metadata area to determine whether the last updated data of the data block is in the data area or the log area. If the last updated data is in the data area, then the data needs to be written to the corresponding data block in the log area for this update, otherwise it needs to be written to the corresponding data block in the data area. In this way, part of the latest data of the same data object is in the data area, and part of the latest data is in the log area. When updating data in multiple data blocks in the data object, if the multiple data blocks to be updated are not continuous, multiple requests will be obtained. The storage system only submits one request to the kernel at a time to perform the read and write operations of the corresponding request.

[0004] In this way, each time a request is submitted, frequent system calls are required, resulting in a waste of resources and slower reading and writing speeds. Summary of the invention

[0005] In view of this, the present invention provides a method, apparatus, computer device, storage medium and program product for submitting a request to solve the problems of slow reading and writing speed and long request processing delay.

[0006] In a first aspect, the present invention provides a method for submitting a request, the method comprising:

[0007] Acquire multiple requests, parse each of the requests, and obtain identification information, operation type, and operation location information of a data object corresponding to each of the requests;

[0008] Determine identification information of a disk corresponding to each of the data objects according to identification information of the data object corresponding to each of the requests;

[0009] Acquire a pressure index value of the target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one of the requested data objects;

[0010] When it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold, and a plurality of target requests corresponding to the target disk are selected from the plurality of requests, the plurality of target requests are grouped according to identification information and operation type of a data object respectively corresponding to each of the plurality of target requests, wherein each group includes one or more of the target requests;

[0011] When the first group of requests includes multiple target requests, determining whether to merge at least two target requests in the first group to generate one request according to the operation position information corresponding to each target request in the first group of requests, and determining the request that finally exists in the first group, wherein the first group of requests is any group of requests after the multiple target requests are grouped;

[0012] Sum up the target number of requests that eventually exist for all groups;

[0013] According to the target quantity and a preset quantity threshold, the finally existing requests of all the groups are submitted to the kernel of the target device in batches, so as to perform data reading and writing operations on the target disk according to the finally existing requests.

[0014] The method for submitting a request provided by the present invention has the following advantages:

[0015] By parsing the request, the identification information, operation type and operation location information of the data object are obtained. Further, after the target request belonging to the target disk is determined according to the identification information of the data object, the target request can be grouped and merged according to the identification information, operation type and operation location information of the data object corresponding to the target request, and the merged requests are submitted to the kernel in batches according to the number of requests after grouping and merging the requests (that is, the target number). In this way, multiple requests can be submitted when each batch is submitted, which can reduce the number of kernel calls, thereby reducing resource waste and improving read and write speeds. In addition, adjusting the submission strategy according to the actual load of the target disk can ensure the stability of the target disk, that is, the service stability of the cloud storage platform can be improved.

[0016] In an optional implementation, when it is determined that the pressure index value is greater than or equal to a preset pressure threshold, and multiple target requests corresponding to the target disk are selected from the multiple requests, the multiple target requests are grouped according to identification information and operation type of the data object respectively corresponding to each of the multiple target requests, including:

[0017] The target requests with the same identification information of the data object and the same operation type are grouped together.

[0018] Specifically, by grouping target requests with the same identification information of the data object and the same operation type into one group, when merging, requests belonging to the same data object, with the same operation type and continuous positions can be merged into one request, and the merged request can be read or written at one time. In this way, the disk seek time and data transmission time can be reduced, and the disk I / O performance can be improved.

[0019] In an optional implementation, submitting the requests that finally exist in all groups to the kernel of the target device in batches according to the target number and the preset number threshold includes:

[0020] Determining whether the target quantity is greater than the preset quantity threshold;

[0021] When it is determined that the target quantity is greater than the preset quantity threshold, in a first submission batch, submitting the requests of all the groups that ultimately exist with a quantity equal to the preset quantity threshold to the kernel as a batch;

[0022] In a current submission batch that is not the first submission batch, obtaining a processing delay of each of the requests submitted in a previous submission batch corresponding to the current submission batch;

[0023] Determining whether to adjust the quantity threshold corresponding to the last submitted batch according to the processing delay of each of the requests submitted in the last submitted batch and the preset delay threshold, wherein when the last submitted batch is the first submitted batch, the quantity threshold corresponding to the last submitted batch is the preset quantity threshold;

[0024] When it is determined to adjust the quantity threshold corresponding to the last submitted batch, the quantity threshold corresponding to the last submitted batch is adjusted according to a preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch;

[0025] Determine the number of currently unsubmitted requests;

[0026] When it is determined that the number of currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all currently unsubmitted requests are submitted to the kernel as a batch.

[0027] Specifically, the number threshold of the current batch is dynamically adjusted according to the processing delay of the last submitted batch to ensure efficient operation under high load conditions and improve the stability of the disk. By submitting requests in batches, the overhead of system calls and context switches is reduced, and the request response speed and processing efficiency are improved. In addition, the number of requests submitted in each batch is determined based on the actual load situation, making the utilization of disks and other resources more reasonable and avoiding resource waste and overload.

[0028] In an optional embodiment, the method further includes:

[0029] When it is determined that the target number is less than or equal to the preset number threshold, all the requests that finally exist in the groups are submitted to the kernel as a batch.

[0030] Specifically, when the number of requests is small, submitting all requests to the kernel at the same time can reduce the number of system calls.

[0031] In an optional embodiment, the method further includes:

[0032] When it is determined that the pressure indicator value is less than the preset pressure threshold and multiple target requests corresponding to the target disk are selected from the multiple requests, obtaining interface information corresponding to each of the multiple target requests;

[0033] Acquire metadata information of the data object corresponding to each of the target requests according to identification information of the data object corresponding to each of the target requests;

[0034] Determine the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request, wherein the first target request is any one of the plurality of target requests, the type includes a first type or a second type, and a response level of the request of the first type is higher than a response level of the request of the second type;

[0035] Submitting the first type of target requests to the kernel in sequence;

[0036] And, when there are multiple target requests of the second type, requests obtained after merging the target requests of the second type are submitted to the kernel in batches.

[0037] Specifically, high-response-level requests are prioritized to ensure timely completion of critical tasks. In addition, low-response-level requests are merged to reduce the overhead of system calls and context switches, thus improving resource utilization.

[0038] In an optional implementation manner, determining the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request includes:

[0039] When it is determined that the interface information of the first target request matches the preset interface information, determining the type of the first target request as the first type;

[0040] or,

[0041] When it is determined that the interface information of the first target request does not match the preset interface information, determining whether the metadata information corresponding to the first target request includes preset marking information;

[0042] When it is determined that the metadata information corresponding to the first target request includes the preset tag information, determining the type of the first target request as the first type;

[0043] or,

[0044] When it is determined that the metadata information corresponding to the first target request does not include the preset marking information, the type of the first target request is determined to be the second type.

[0045] Specifically, through double checking of the interface information and the metadata information, the requests can be classified more accurately, which ensures the correct identification of high priority requests (first type) and low priority requests (second type).

[0046] In an optional embodiment, the method further includes:

[0047] When it is determined that the pressure indicator value is less than the preset pressure threshold, or only one target request corresponding to the target disk is selected from the multiple requests, each at least one target request corresponding to the target disk is submitted to the kernel in sequence.

[0048] Specifically, when resources are sufficient or the number of requests is small, submitting one request at a time can improve the response speed of the request and reduce the processing delay.

[0049] In a second aspect, the present invention provides a device for submitting a request, the device comprising:

[0050] An acquisition module, used to acquire multiple requests, parse each of the requests, and obtain identification information, operation type, and operation location information of a data object corresponding to each of the requests;

[0051] A determination module, configured to determine identification information of a disk corresponding to each of the data objects according to identification information of the data object corresponding to each of the requests;

[0052] The acquisition module is further used to acquire the pressure index value of the target disk according to the identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object;

[0053] a grouping module, configured to group the multiple target requests according to identification information and operation type of the data object respectively corresponding to each of the multiple target requests, when it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold value and multiple target requests corresponding to the target disk are selected from the multiple requests, wherein each group includes one or more target requests;

[0054] The determination module is further configured to determine whether to merge at least two of the target requests in the first group to generate one request, and determine the request that finally exists in the first group, when the first group of requests includes multiple target requests, according to the operation position information corresponding to each of the target requests in the first group of requests, wherein the first group of requests is any group of requests after the multiple target requests are grouped;

[0055] A summary module is used to summarize the target number of requests that finally exist in all groups;

[0056] The submission module is used to submit the final existing requests of all groups to the kernel of the target device in batches according to the target number and a preset number threshold, so as to perform data reading and writing operations on the target disk according to the final existing requests.

[0057] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for submitting a request according to the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0058] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method for submitting a request according to the first aspect or any corresponding embodiment thereof.

[0059] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for submitting a request according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 is a schematic diagram of a disk layout according to an embodiment of the present invention;

[0062] Figure 2 is a schematic diagram of storage distribution of data objects according to an embodiment of the present invention;

[0063] Figure 3 is a flowchart of a method for submitting a request according to an embodiment of the present invention;

[0064] Figure 4 is a flowchart of another method for submitting a request according to an embodiment of the present invention;

[0065] Figure 5 is a structural block diagram of an apparatus for submitting a request according to an embodiment of the present invention;

[0066] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0068] The embodiment of the present invention can be applied to a server, and a storage system and an operating system can be installed on the server. The storage system is used to manage multiple disks mounted on the server. The operating system may include a kernel, which is the core part of the operating system and is responsible for managing system resources and providing services to other software. For example, the other software may be the above-mentioned storage system. During the data reading and writing process, the storage system is used to process the request sent by the client or the request generated by itself, and submit it to the kernel of the operating system after processing, and the kernel performs read and write operations on the disk.

[0069] like Figure 1 As shown in the figure, a disk can be divided into multiple areas such as metadata area, journal area and data area, and each area is divided into multiple data blocks. Among them, the metadata area is usually used to store metadata information of data objects, and the journal area and data area are used to store the data itself. Each data object can be composed of data blocks in the metadata area, data blocks in the journal area and data blocks in the data area. In this way, each disk can include data blocks corresponding to multiple data objects.

[0070] like Figure 2 As shown, the data writing process is described below by taking any data object as an example.

[0071] When the data object is written for the first time, the data is first written to the data block of the data object in the data area, for example, the 1st, 2nd, 4th, 6th and 7th data blocks. In order to avoid abnormal situations such as data writing errors during the writing process, when the data is written for the second time, it can be written to the corresponding position in the log area first, for example, the 2nd, 4th, 6th and 7th data blocks. In this way, when an abnormality occurs in the second writing process, it can also be guaranteed that the data written for the first time still exists. When writing the data for the third time, it can be written to the corresponding position in the data area again, for example, the 4th and 7th data blocks. When writing the data for the fourth time, it is written to the corresponding position in the log area, for example, the 7th data block. Under such data writing rules, the latest data of the data object will be scattered in the log area and the data area, and the location of the stored data is not continuous. For example, in Figure 2 The latest data of the first data block is in the data area, the latest data of the second data block is in the log area, the third data block does not store data, and the latest data of the fourth data block is in the data area.

[0072] When reading or updating these data blocks, the operation position is discontinuous. If a request is received to operate on the first, second, and third data blocks, since the storage positions of the latest data of these three data blocks are discontinuous, this request will be split into three sub-requests, resulting in one request being magnified into three, that is, the number of requests increases. In addition, since the storage space of the log area is small, when the storage space of the log area is insufficient, a data flush operation will be performed. During the data flush operation, since the data blocks storing the latest data are discontinuous, a large number of read and write requests will be generated.

[0073] After generating a request, the storage system needs to call the kernel of the operating system, and the operating system continues to perform subsequent request processing operations, that is, read or write operations on the disk. Each time a request is processed, the operating system needs to be called once, resulting in a large amount of resources being wasted on system calls, reducing request throughput.

[0074] An embodiment of the present invention provides a method for submitting requests, which determines a submission strategy according to actual load conditions, merges various requests generated by a storage system when the load is high, and submits the merged requests in batches to an operating system for processing, thereby reducing the number of system calls and improving request throughput.

[0075] According to an embodiment of the present invention, an embodiment of a method for submitting a request is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0076] In this embodiment, a method for submitting a request is provided, which can be executed by a target device, and the target device can be the server mentioned above. Figure 3 is a flowchart of a method for submitting a request according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0077] Step S301, obtain multiple requests, parse each request, and obtain identification information, operation type and operation location information of the data object corresponding to each request.

[0078] Among them, the multiple requests may include the above-mentioned requests obtained by splitting the total requests received from the client, and / or, generated by the storage system when flushing data from the log area to the data area. For the former, the requests can be divided into requests for updating disk information and requests for updating business data on the disk, or, the requests can be divided into ordinary requests or critical requests, and the processing delay level of critical requests is higher than the processing delay level of ordinary requests. The operation type can be a read operation or a write operation. The operation position information may include the starting position and data length of the data operated by the request on the disk, and the data length can be represented by the number of data blocks.

[0079] Step S302: Determine identification information of a disk corresponding to each data object according to identification information of the data object corresponding to each request.

[0080] Specifically, for any request, the target device can determine the identification information of the disk corresponding to the identification information of the data object corresponding to the request in the pre-built correspondence table between the identification information of the data object and the identification information of the disk based on the identification information of the data object corresponding to the request.

[0081] Alternatively, the target device may obtain metadata information included in the data object corresponding to the request according to the identification information of the data object corresponding to the request, and extract the identification information of the disk from the metadata information.

[0082] Step S303: acquiring the pressure index value of the target disk according to the identification information of the target disk.

[0083] The target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object. The pressure index value can be one of the input / output operations per second (IOPS), throughput, and utilization (the percentage of time the disk is busy in a period of time), etc., or it can be obtained by comprehensively calculating IOPS, throughput, and utilization according to a preset algorithm.

[0084] Specifically, the target device can monitor the load of all mounted disks in real time, and update the pressure index value obtained in real time or calculated to the target data table, which can include the identification information of each disk and its pressure index value. After obtaining multiple requests, the corresponding pressure index value can be obtained in the target data table according to the identification information of the disk corresponding to any request.

[0085] Alternatively, the target device may obtain the identification information corresponding to the request after parsing, and then collect the pressure index value of the target disk according to the identification of the target disk.

[0086] Step S304, when it is determined that the pressure index value is greater than or equal to the preset pressure threshold, and multiple target requests corresponding to the target disk are selected from multiple requests, the multiple target requests are grouped according to the identification information and operation type of the data object corresponding to each of the multiple target requests.

[0087] Each group may include one or more target requests.

[0088] Specifically, the target device can determine whether the pressure index value of the target disk is greater than or equal to a preset pressure threshold, and select, from multiple requests, a request whose disk identification information is the identification information of the target disk, based on the identification information of the target disk. When it is determined that the pressure index value is greater than or equal to the preset pressure threshold, and multiple target requests corresponding to the target disk are selected from multiple requests, the multiple target requests are grouped according to the identification information and operation type of the data object corresponding to each of the multiple target requests.

[0089] When it is determined that the pressure indicator value is less than a preset pressure threshold, or when a target request corresponding to the target disk is selected from multiple requests, each target request may be submitted to the kernel in sequence.

[0090] The specific process of grouping multiple target requests according to the operation type and the identification information of the data object can be as follows: first, the target requests with the same identification information of the data object are grouped into one group, and then the target requests with the same operation type in each group are grouped into another group. Alternatively, the target requests with the same operation type can be grouped into one group, and then the target requests with the same identification information of the data object in each group are grouped into another group. In this way, the target requests with the same identification information of the data object and the same operation type can be grouped into one group, and one or more groups can be obtained.

[0091] In some optional implementations, after selecting the target request corresponding to the target disk, any one request can be selected from the remaining requests, and the above-mentioned steps S303 and S304 can be executed according to the identification information of the disk corresponding to the request, and so on, to complete the grouping processing of all the obtained requests.

[0092] Step S305, when the first group of requests includes multiple target requests, determine whether to merge at least two target requests in the first group according to the operation location information corresponding to each target request in the first group of requests, generate a request, and determine the request that finally exists in the first group.

[0093] The first group of requests is any group of requests after a plurality of target requests are grouped.

[0094] Specifically, for the first group of requests, when the first group of requests includes one target request, no merging process is performed. When the first group of requests includes multiple target requests, the target device can first sort each target request in the first group of requests according to the operation position information corresponding to each target request in the first group of requests. Then, the target device can determine whether the operation positions of the first target request and the second target request are continuous according to the operation position information of the first target request and the operation position information of the second target request. When it is determined that the operation positions are continuous, it is determined that the first target request and the second target request can be merged. When it is determined that the operation positions are not continuous, it is determined that the first target request and the second target request cannot be merged. Among them, the first target request and the second target request are any two target requests that are continuously sorted in the first group of requests, and the sorting of the first target request is before the sorting of the second target request. After determining that multiple target requests can be merged into one request, these multiple target requests are merged to generate a new request. In this way, the requests that finally exist in the first group may include one or more generated requests, and / or target requests that do not have a merging relationship with any target request.

[0095] The above-mentioned determination of whether the operation positions of the first target request and the second target request are continuous according to the operation position information of the first target request and the operation position information of the second target request can be processed in the following steps:

[0096] The first end position is determined according to the starting position and data length in the operation position information of the first target request, and then it can be determined whether the first end position is adjacent to the starting position in the operation position information of the second target request. If so, it is determined that the operation positions of the first target request and the second target request are continuous. If not, it is determined that the operation positions of the first target request and the second target request are discontinuous. When the data length is represented by the number of data blocks, the first end position can be calculated according to the starting position and the number of data blocks in the operation position information of the first target request, and the preset data block size.

[0097] Step S306, summarizing the target number of requests that ultimately exist in all groups.

[0098] Step S307 , submitting the final existing requests of all groups to the kernel of the target device in batches according to the target number and the preset number threshold, so as to perform data read and write operations on the target disk according to the final existing requests.

[0099] Specifically, step S307 may include the following steps:

[0100] Step 1: Determine whether the target quantity is greater than a preset quantity threshold.

[0101] Step 2: When it is determined that the target number is less than or equal to a preset number threshold, all the requests that eventually exist in the group are submitted to the kernel as a batch.

[0102] Step 3: When it is determined that the target quantity is greater than a preset quantity threshold, in the first submission batch, all requests that ultimately exist in all groups and have a quantity equal to the preset quantity threshold are submitted to the kernel as a batch.

[0103] Step 4: In a current submission batch that is not the first submission batch, obtain the processing delay of each request submitted in a previous submission batch corresponding to the current submission batch.

[0104] Step 5: Determine whether to adjust the quantity threshold corresponding to the last submitted batch based on the processing delay of each request submitted in the last submitted batch and the preset delay threshold.

[0105] Among them, when the last submitted batch is the first submitted batch, the quantity threshold corresponding to the last submitted batch is the preset quantity threshold.

[0106] Specifically, when the target device determines that the processing delay of any request of the last submitted batch is greater than or less than the preset delay threshold, it is determined that the quantity threshold corresponding to the last submitted batch is adjusted. When the target device determines that the processing delay of any request of the last submitted batch is equal to the preset delay threshold, it is determined that the quantity threshold corresponding to the last submitted batch is not adjusted.

[0107] In some optional implementations, the target device may be set with a preset ratio. When the processing delay of requests greater than or equal to the preset ratio in the last submitted batch is greater than or less than the preset delay threshold, it is determined that the quantity threshold corresponding to the last submitted batch is adjusted. When the processing delay of only requests less than the preset ratio in the last submitted batch is greater than or less than the preset delay threshold, it is determined not to adjust the quantity threshold corresponding to the last submitted batch. In this way, by determining whether to adjust the quantity threshold based on the processing delays of multiple requests, the adjusted quantity threshold can be made more accurate.

[0108] Step six, when it is determined to adjust the quantity threshold corresponding to the last submitted batch, adjust the quantity threshold corresponding to the last submitted batch according to the preset quantity adjustment rule to obtain the quantity threshold of the current submitted batch.

[0109] Specifically, when the target device determines that the processing time of any request of the last submitted batch is less than the preset delay threshold, the preset quantity adjustment rule may be to increase the quantity threshold corresponding to the last submitted batch by a preset percentage, for example, the quantity threshold corresponding to the last submitted batch is 50, the preset percentage is 50%, and the quantity threshold of the current submitted batch may be 75. When the target device determines that the processing delay of any request of the last submitted batch is greater than the preset delay threshold, the preset quantity adjustment rule may be to reduce the quantity threshold corresponding to the last submitted batch by a preset percentage, for example, the quantity threshold corresponding to the last submitted batch is 50, the preset percentage is 50%, and the quantity threshold of the current submitted batch may be 25.

[0110] Step 7: Determine the number of currently unsubmitted requests.

[0111] Step eight, when it is determined that the number of currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all currently unsubmitted requests are submitted to the kernel as a batch.

[0112] Step nine, when it is determined that the number of currently unsubmitted requests is greater than the number threshold corresponding to the currently submitted batch, submit the number of requests equal to the number threshold corresponding to the currently submitted batch among the unsubmitted requests that ultimately exist in all groups to the kernel as a batch, and enter the next submission round corresponding to the current submission round.

[0113] In some optional implementations, all the requests that eventually exist in the above-mentioned groups corresponding to the target disk can be placed in the same context, and each time they are submitted, the corresponding number of requests can be extracted from the context and sent to the kernel. The context means that when processing multiple requests, they are processed as a whole. If the context is reinitialized each time a request is processed, additional overhead will be incurred. Therefore, allowing multiple requests to share the same context can avoid repeated initialization and cleanup work. In addition, multiple requests can share already opened resources, reducing the frequent opening and closing of resources, thereby improving processing speed.

[0114] In some optional implementations, when multiple requests are obtained by splitting the total requests received from the client, the above steps S301 to S307 can be used for processing and submission. When the disk identifiers corresponding to the multiple requests are all identification information of the target disk, and are all generated by the storage system when flushing data from the log area to the data area, after completing the grouping operation of step S304, the target device can be set with a data object quantity threshold. The target device can select at least one target group from at least one group obtained by grouping according to the data object quantity threshold (for example, the data object quantity threshold is 5, and all target groups corresponding to 5 data objects are selected from at least one group). Then, according to the number of requests in all selected target groups and the preset quantity threshold, the number of requests that need to be submitted for the current submission batch is determined (specifically, the minimum value of the two can be selected). Finally, according to the number of requests that need to be submitted for the current submission batch determined, the corresponding number of requests in all target groups can be submitted to the kernel as a batch. If there are still remaining requests in all target groups after the processing of the current submitted batch is completed, they can be processed in the same way, and the requests in all selected target groups can be submitted to the kernel for data reading and writing operations.

[0115] Since N read requests and N write requests will be generated in the process of flushing N discontinuous data blocks from the log area to the data area, if a large number of data blocks are flushed at the same time, the flushing process will be blocked and a large amount of submission queue space will be occupied, affecting the request processing performance of the target device. Therefore, by limiting the number of data objects flushed and the number of requests for each flush, the number of data objects and the number of requests flushed at the same time can be reduced, the problem of flushing process blockage and occupying a large amount of submission queue space can be avoided, and the request processing performance of the target device can be improved.

[0116] The method for submitting a request provided in this embodiment obtains the identification information, operation type and operation location information of the data object by parsing the request. Further, after determining the target request belonging to the target disk according to the identification information of the data object, the target request can be grouped and merged according to the identification information, operation type and operation location information of the data object corresponding to the target request, and the merged requests are submitted to the kernel in batches according to the number of requests after grouping and merging the requests (that is, the target number). In this way, multiple requests can be submitted when each batch is submitted, which can reduce the number of kernel calls, thereby reducing resource waste and improving read and write speeds. In addition, adjusting the submission strategy according to the actual load of the target disk can ensure the stability of the target disk, that is, the service stability of the cloud storage platform can be improved.

[0117] Since different types of requests have different requirements for processing delay, for example, the above-mentioned requests for updating disk information and key requests have higher requirements for processing delay, and most of the resources need to be used for processing to speed up the processing. These requests can adopt a synchronous submission strategy, that is, submitting one request to the kernel one by one. Therefore, when the resources of the target disk are sufficient, that is, when it is determined that the pressure index value is less than the preset pressure threshold, and multiple target requests corresponding to the target disk are selected from multiple requests, a method for submitting requests provided in this embodiment can be executed, which can be executed by the target device, and the target device can be the above-mentioned target device. Figure 4 is a flowchart of a method for submitting a request according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0118] Step S401, obtaining interface information corresponding to each of a plurality of target requests.

[0119] The interface information may be identification information of the interface, for example, the number and name of the interface.

[0120] Step S402: acquiring metadata information of the data object corresponding to each target request according to the identification information of the data object corresponding to each target request.

[0121] Specifically, the target device may determine the data object corresponding to each target request according to the identification information of the data object corresponding to each target request, and extract metadata information of the data object corresponding to each target request.

[0122] Step S403: Determine the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request.

[0123] The first target request is any one of the multiple target requests, and the type includes the first type or the second type, and the response level of the first type request is higher than the response level of the second type request. The first type request may include the above-mentioned request to update the disk information and / or the key request, and the second type request may be the above-mentioned request to update the business data in the disk and / or the ordinary request.

[0124] Specifically, the request to update the disk information may be obtained through a preset interface, and the metadata information corresponding to the key request may include preset tag information. Accordingly, step S403 may be implemented in the following ways:

[0125] Method 1: When it is determined that the interface information of the first target request matches the preset interface information, the type of the first target request is determined to be the first type. Alternatively, when it is determined that the interface information of the first target request does not match the preset interface information, it is determined whether the metadata information corresponding to the first target request includes preset tag information. When it is determined that the metadata information corresponding to the first target request includes preset tag information, the type of the first target request is determined to be the first type. Alternatively, when it is determined that the metadata information corresponding to the first target request does not include preset tag information, the type of the first target request is determined to be the second type.

[0126] Method 2: When it is determined that the metadata information corresponding to the first target request includes preset tag information, the type of the first target request is determined to be the first type. Alternatively, when it is determined that the metadata information corresponding to the first target request does not include preset tag information, it is determined whether the interface information of the first target request matches the preset interface information. When it is determined that the interface information of the first target request matches the preset interface information, the type of the first target request is determined to be the first type. When it is determined that the interface information of the first target request does not match the preset interface information, the type of the first target request is determined to be the second type.

[0127] Method three, determining whether the interface information of the first target request matches the preset interface information, and determining whether the metadata information corresponding to the first target request includes preset tag information. When it is determined that the interface information of the first target request matches the preset interface information, or when it is determined that the metadata information corresponding to the first target request includes preset tag information, the type of the first target request is determined to be the first type. When it is determined that the interface information of the first target request does not match the preset interface information, and when it is determined that the metadata information corresponding to the first target request does not include preset tag information, the type of the first target request is determined to be the second type.

[0128] Step S404: Submit the first type of target requests to the kernel in sequence.

[0129] Specifically, for the first type of target request, since it has a relatively high requirement on the processing delay, the first type of target requests can be submitted to the kernel one by one.

[0130] Step S405: when there are multiple target requests of the second type, the requests obtained after merging the target requests of the second type are submitted to the kernel in batches.

[0131] The specific processing of step S405 may be similar to the specific processing of step S307, and will not be repeated here.

[0132] The method for submitting requests provided in this embodiment dynamically adjusts the quantity threshold of the current batch according to the processing delay of the last submitted batch, ensuring efficient operation under high load conditions and improving the stability of the disk. By submitting requests in batches, the overhead of system calls and context switches is reduced, and the request response speed and processing efficiency are improved. In addition, the number of requests submitted in each batch is determined based on the actual load conditions, making the utilization of the disk and other resources more reasonable and avoiding resource waste and overload.

[0133] In this embodiment, a device for submitting a request is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0134] This embodiment provides a device for submitting a request, such as Figure 5 As shown, including:

[0135] The acquisition module 501 is used to acquire multiple requests, parse each request, and obtain the identification information, operation type and operation location information of the data object corresponding to each request;

[0136] A determination module 502, configured to determine identification information of a disk corresponding to each data object according to identification information of a data object corresponding to each request;

[0137] The acquisition module 501 is further used to acquire the pressure index value of the target disk according to the identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object;

[0138] A grouping module 503 is used for grouping the multiple target requests according to the identification information and operation type of the data object respectively corresponding to each of the multiple target requests, when it is determined that the pressure index value is greater than or equal to the preset pressure threshold value and multiple target requests corresponding to the target disk are selected from the multiple requests, wherein each group includes one or more target requests;

[0139] The determination module 502 is further used to determine whether to merge at least two target requests in the first group to generate one request, and determine the request that finally exists in the first group, when the first group of requests includes multiple target requests, according to the operation position information corresponding to each target request in the first group of requests, wherein the first group of requests is any group of requests after the multiple target requests are grouped;

[0140] A summarizing module 504 is used to summarize the target number of requests that finally exist in all groups;

[0141] The submission module 505 is used to submit all groups of final existing requests to the kernel of the target device in batches according to the target quantity and the preset quantity threshold, so as to perform data read and write operations on the target disk according to the final existing requests.

[0142] In an optional implementation, the grouping module 503 is specifically configured to:

[0143] Target requests with the same data object identification information and the same operation type are grouped together.

[0144] In an optional implementation, the submission module 505 is specifically configured to:

[0145] Determine whether the target quantity is greater than a preset quantity threshold;

[0146] When it is determined that the target quantity is greater than the preset quantity threshold, in the first submission batch, all requests that eventually exist in all groups and have a quantity equal to the preset quantity threshold are submitted to the kernel as a batch;

[0147] In a current submission batch that is not the first submission batch, obtain the processing delay of each request submitted in a previous submission batch corresponding to the current submission batch;

[0148] Determining whether to adjust the quantity threshold corresponding to the last submitted batch according to the processing delay of each request submitted in the last submitted batch and the preset delay threshold, wherein when the last submitted batch is the first submitted batch, the quantity threshold corresponding to the last submitted batch is the preset quantity threshold;

[0149] When it is determined that the quantity threshold corresponding to the last submitted batch is to be adjusted, the quantity threshold corresponding to the last submitted batch is adjusted according to the preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch;

[0150] Determine the number of currently unsubmitted requests;

[0151] When it is determined that the number of currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all currently unsubmitted requests are submitted to the kernel as a batch.

[0152] In an optional implementation, the submission module 505 is further configured to:

[0153] When it is determined that the target number is less than or equal to the preset number threshold, all the requests that eventually exist in the group are submitted to the kernel as a batch.

[0154] In an optional implementation, the submission module 505 is further configured to:

[0155] When it is determined that the pressure indicator value is less than a preset pressure threshold and multiple target requests corresponding to the target disk are selected from the multiple requests, interface information corresponding to each of the multiple target requests is obtained;

[0156] Acquire metadata information of the data object corresponding to each target request according to identification information of the data object corresponding to each target request;

[0157] Determine the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request, wherein the first target request is any one of the multiple target requests, the type includes a first type or a second type, and the response level of the first type of request is higher than the response level of the second type of request;

[0158] Submitting the first type of target requests to the kernel in sequence;

[0159] Furthermore, when there are multiple target requests of the second type, requests obtained after merging the target requests of the second type are submitted to the kernel in batches.

[0160] In an optional implementation, the submission module 505 is specifically configured to:

[0161] When it is determined that the interface information of the first target request matches the preset interface information, determining the type of the first target request as the first type;

[0162] or,

[0163] When it is determined that the interface information of the first target request does not match the preset interface information, determining whether the metadata information corresponding to the first target request includes preset marking information;

[0164] When it is determined that the metadata information corresponding to the first target request includes preset tag information, determining the type of the first target request as a first type;

[0165] or,

[0166] When it is determined that the metadata information corresponding to the first target request does not include the preset marking information, the type of the first target request is determined to be the second type.

[0167] In an optional implementation, the submission module 505 is further configured to:

[0168] When it is determined that the pressure indicator value is less than a preset pressure threshold, or only one target request corresponding to the target disk is selected from multiple requests, each at least one target request corresponding to the target disk is submitted to the kernel in sequence.

[0169] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0170] The device for submitting a request in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0171] The embodiment of the present invention also provides a computer device having the above Figure 6 The means for submitting the request is shown.

[0172] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0173] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0174] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0175] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0177] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0178] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0179] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0180] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for submitting a request, characterized in that: The method comprises: Acquire multiple requests, parse each of the requests, and obtain identification information, operation type, and operation location information of a data object corresponding to each of the requests; Determine identification information of a disk corresponding to each of the data objects according to identification information of the data object corresponding to each of the requests; Acquire a pressure index value of the target disk according to identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one of the requested data objects; When it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold, and a plurality of target requests corresponding to the target disk are selected from the plurality of requests, the plurality of target requests are grouped according to identification information and operation type of a data object respectively corresponding to each of the plurality of target requests, wherein each group includes one or more of the target requests; When the first group of requests includes multiple target requests, determining whether to merge at least two target requests in the first group to generate one request according to the operation position information corresponding to each target request in the first group of requests, and determining the request that finally exists in the first group, wherein the first group of requests is any group of requests after the multiple target requests are grouped; Sum up the target number of requests that eventually exist for all groups; According to the target quantity and a preset quantity threshold, the finally existing requests of all the groups are submitted to the kernel of the target device in batches, so as to perform data reading and writing operations on the target disk according to the finally existing requests.

2. The method according to claim 1, characterized in that When it is determined that the pressure indicator value is greater than or equal to the preset pressure threshold, and multiple target requests corresponding to the target disk are selected from the multiple requests, the multiple target requests are grouped according to identification information and operation type of the data object respectively corresponding to each of the multiple target requests, including: The target requests with the same identification information of the data object and the same operation type are grouped together.

3. The method according to claim 1 or 2, characterized in that: Submitting the requests that finally exist in all groups to the kernel of the target device in batches according to the target number and the preset number threshold includes: Determining whether the target quantity is greater than the preset quantity threshold; When it is determined that the target quantity is greater than the preset quantity threshold, in a first submission batch, submitting the requests of all the groups that ultimately exist with a quantity equal to the preset quantity threshold to the kernel as a batch; In a current submission batch that is not the first submission batch, obtaining a processing delay of each of the requests submitted in a previous submission batch corresponding to the current submission batch; Determining whether to adjust the quantity threshold corresponding to the last submitted batch according to the processing delay of each of the requests submitted in the last submitted batch and the preset delay threshold, wherein when the last submitted batch is the first submitted batch, the quantity threshold corresponding to the last submitted batch is the preset quantity threshold; When it is determined to adjust the quantity threshold corresponding to the last submitted batch, the quantity threshold corresponding to the last submitted batch is adjusted according to a preset quantity adjustment rule to obtain the quantity threshold corresponding to the current submitted batch; Determine the number of currently unsubmitted requests; When it is determined that the number of currently unsubmitted requests is less than or equal to the number threshold of the currently submitted batch, all currently unsubmitted requests are submitted to the kernel as a batch.

4. The method according to claim 3, characterized in that The method further comprises: When it is determined that the target number is less than or equal to the preset number threshold, all the requests that finally exist in the groups are submitted to the kernel as a batch.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: When it is determined that the pressure indicator value is less than the preset pressure threshold and multiple target requests corresponding to the target disk are selected from the multiple requests, obtaining interface information corresponding to each of the multiple target requests; Acquire metadata information of the data object corresponding to each of the target requests according to identification information of the data object corresponding to each of the target requests; Determine the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request, wherein the first target request is any one of the plurality of target requests, the type includes a first type or a second type, and a response level of the request of the first type is higher than a response level of the request of the second type; Submitting the first type of target requests to the kernel in sequence; And, when there are multiple target requests of the second type, requests obtained after merging the target requests of the second type are submitted to the kernel in batches.

6. The method according to claim 5, characterized in that The determining the type of the first target request according to the interface information of the first target request and / or the metadata information of the data object corresponding to the first target request includes: When it is determined that the interface information of the first target request matches the preset interface information, determining the type of the first target request as the first type; or, When it is determined that the interface information of the first target request does not match the preset interface information, determining whether the metadata information corresponding to the first target request includes preset marking information; When it is determined that the metadata information corresponding to the first target request includes the preset tag information, determining the type of the first target request as the first type; or, When it is determined that the metadata information corresponding to the first target request does not include the preset marking information, the type of the first target request is determined to be the second type.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: When it is determined that the pressure indicator value is less than the preset pressure threshold, or only one target request corresponding to the target disk is selected from the multiple requests, each at least one target request corresponding to the target disk is submitted to the kernel in sequence.

8. A device for submitting a request, characterized in that: The device comprises: An acquisition module, used to acquire multiple requests, parse each of the requests, and obtain identification information, operation type, and operation location information of a data object corresponding to each of the requests; A determination module, configured to determine identification information of a disk corresponding to each of the data objects according to identification information of the data object corresponding to each of the requests; The acquisition module is further used to acquire the pressure index value of the target disk according to the identification information of the target disk, wherein the target disk is a disk corresponding to a data object of any one of the multiple requests, and the target disk corresponds to at least one requested data object; a grouping module, configured to group the multiple target requests according to identification information and operation type of the data object respectively corresponding to each of the multiple target requests, when it is determined that the pressure indicator value is greater than or equal to a preset pressure threshold value and multiple target requests corresponding to the target disk are selected from the multiple requests, wherein each group includes one or more target requests; The determination module is further configured to determine whether to merge at least two of the target requests in the first group to generate one request, and determine the request that finally exists in the first group, when the first group of requests includes multiple target requests, according to the operation position information corresponding to each of the target requests in the first group of requests, wherein the first group of requests is any group of requests after the multiple target requests are grouped; A summary module is used to summarize the target number of requests that finally exist in all groups; The submission module is used to submit the final existing requests of all groups to the kernel of the target device in batches according to the target number and a preset number threshold, so as to perform data reading and writing operations on the target disk according to the final existing requests.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for submitting a request as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method for submitting a request according to any one of claims 1 to 7.

Images