Cloud storage read-write speed limiting method, device, equipment, storage medium and program product

Abstract

This disclosure relates to a cloud storage read / write rate limiting method, apparatus, device, storage medium, and program product. The method is applicable to a target storage node in a cloud storage cluster. The method includes: acquiring read / write requests to be executed, including a first read / write request waiting to be executed by the target storage node and a second read / write request waiting to be executed by other storage nodes; within each unit of time, selecting target read / write requests from the pending read / write requests based on the data length, receiving order, and read / write rate limiting requirements of the cloud storage cluster for each pending read / write request; executing the first read / write request from the target read / write requests, and removing the target read / write request from the pending read / write requests. This disclosure, by acquiring all read / write requests that the cloud storage cluster needs to process on the target storage node and then performing rate limiting operations, enables overall cluster read / write rate limiting in multi-mount mode, maintaining the performance stability of cloud storage resources.

Description

Technical Field

[0001] This disclosure relates to the field of cloud storage technology, and in particular to a cloud storage read / write rate limiting method, apparatus, device, storage medium, and program product. Background Technology

[0002] Cloud storage is a data storage service based on cloud computing technology. It aggregates a large number of storage devices through virtualization technology to form a unified storage resource pool, providing data read and write services to users. It is widely used in various business scenarios. Cloud storage access modes include single-mount mode and multi-mount mode. In single-mount mode, cloud storage resources are mounted on a single compute node, which has exclusive access to them. Since only one compute node accesses the cloud storage resource, rate limiting can be performed on the compute node to ensure reasonable utilization of storage resources. In multi-mount mode, multiple compute nodes concurrently access the same cloud storage resource. Rate limiting cannot be performed on the compute nodes, especially when the cloud storage resource is provided by a cloud storage cluster containing multiple storage nodes. A common approach to rate limiting is to send read and write requests initiated by compute nodes to a central node. The central node determines whether the read and write request meets the rate limiting requirements and, if so, forwards the request to the cloud storage node for processing. However, in high-concurrency scenarios, if the performance requirements for cloud storage resources are high, the central node can easily become a bottleneck, causing the performance of cloud storage resources to fail to meet business needs. Therefore, how to limit the read and write speeds of cloud storage resources in multi-mount mode is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] To address the aforementioned technical issues, this disclosure provides a cloud storage read / write rate limiting method, apparatus, device, storage medium, and program product.

[0004] A first aspect of this disclosure provides a cloud storage read / write rate limiting method, the method comprising:

[0005] Obtain the read and write requests to be executed sent by each client, including a first read and write request waiting to be executed by the target storage node and a second read and write request waiting to be executed by other storage nodes in the cloud storage cluster;

[0006] Within each unit of time, based on the data length, receiving order, and read / write rate limiting requirements of the cloud storage cluster corresponding to each read / write request to be executed, target read / write requests are selected from the read / write requests to be executed. The sum of the data lengths corresponding to the target read / write requests and the first number of the target read / write requests satisfy the read / write rate limiting requirements.

[0007] Execute the first read / write request in the target read / write request, and remove the target read / write request from the list of read / write requests to be executed.

[0008] A second aspect of this disclosure provides a cloud storage read / write rate limiting device, the device comprising:

[0009] The acquisition module is used to acquire read and write requests to be executed sent by each client. The read and write requests to be executed include a first read and write request waiting to be executed by the target storage node and a second read and write request waiting to be executed by other storage nodes in the cloud storage cluster.

[0010] The filtering module is used to filter target read and write requests from the read and write requests to be executed within each unit time period, based on the data length, receiving order and read and write rate limiting requirements of the cloud storage cluster for each read and write request to be executed. The sum of the data lengths corresponding to the target read and write requests and the first number of the target read and write requests satisfy the read and write rate limiting requirements.

[0011] An execution module is used to execute the first read / write request in the target read / write request and remove the target read / write request from the read / write requests to be executed.

[0012] A third aspect of this disclosure provides a computer device including a memory and a processor, and a computer program, wherein the memory stores the computer program, and when the computer program is executed by the processor, it implements the cloud storage read / write rate limiting method of the first aspect described above.

[0013] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the cloud storage read / write rate limiting method of the first aspect described above.

[0014] A fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the cloud storage read / write rate limiting method of the first aspect described above.

[0015] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0016] In the cloud storage read / write rate limiting method, apparatus, device, storage medium, and program product provided in this disclosure, by acquiring read / write requests to be executed sent by various clients, including a first read / write request waiting to be executed by a target storage node and a second read / write request waiting to be executed by other storage nodes in the cloud storage cluster, within each unit of time, based on the data length, receiving order, and read / write rate limiting requirements of each read / write request to be executed, a target read / write request is selected from the read / write requests to be executed. The sum of the data lengths corresponding to the target read / write requests and the target... If the first number of read / write requests meets the read / write rate limiting requirements, the first read / write request in the target read / write request is executed, and the target read / write request is removed from the list of pending read / write requests. This allows the system to obtain all read / write requests that each storage node in the cloud storage cluster needs to process, and then perform overall rate limiting based on the information of all read / write requests. This ensures that even if multiple compute nodes concurrently access cloud storage resources in multi-mount mode, the read / write rate of cloud storage resources will not exceed the upper limit, ensuring reasonable utilization of storage resources and meeting the performance requirements in high-concurrency scenarios. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a cloud storage read / write rate limiting method provided in an embodiment of this disclosure;

[0020] Figure 2 This is a schematic diagram illustrating an interaction of a read / write request provided in an embodiment of this disclosure;

[0021] Figure 3 This is a flowchart of a read / write request processing method provided in an embodiment of this disclosure;

[0022] Figure 4 This is a flowchart of a method for filtering target read / write requests provided in an embodiment of this disclosure;

[0023] Figure 5 This is a schematic diagram of the structure of a cloud storage read / write speed limiting device provided in an embodiment of this disclosure;

[0024] Figure 6This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0025] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0026] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0027] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0028] Figure 1 This is a flowchart of a cloud storage read / write rate limiting method provided in an embodiment of this disclosure. This method can be executed by a cloud storage read / write rate limiting device, which can be implemented in software and / or hardware. The cloud storage read / write rate limiting device can be configured in a target storage node within a cloud storage cluster. Figure 1 As shown, the cloud storage read / write rate limiting method provided in this embodiment includes the following steps:

[0029] S101. Obtain the read / write requests to be executed sent by each client. The read / write requests to be executed include the first read / write request waiting to be executed by the target storage node and the second read / write request waiting to be executed by other storage nodes in the cloud storage cluster.

[0030] In this embodiment of the disclosure, the cloud storage cluster may contain one or more storage nodes, and the target storage node may be any storage node in the cloud storage cluster. When the cloud storage cluster contains multiple storage nodes, the client may specify any storage node in the cloud storage cluster to process the read / write request sent by the client. Specifically, the client may send a first read / write request to the specified target storage node, and simultaneously send a second read / write request to other storage nodes in the cloud storage cluster. The first read / write request indicates that the current storage node needs to perform the read / write operation corresponding to the read / write request, and the second read / write request indicates that other nodes will perform the read / write operation corresponding to the read / write request. The storage node receiving the second read / write request does not need to actually process the read / write request.

[0031] The read / write requests to be executed in this embodiment may include read / write requests received in real time by the target storage node, or read / write requests that were previously received but for which no corresponding read / write operation was performed.

[0032] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can obtain the read / write requests to be executed sent by each client. Specifically, it can obtain the first read / write request sent by each client that is waiting to be executed by the target storage node and the second read / write request that is waiting to be executed by other storage nodes.

[0033] In one exemplary embodiment of this disclosure, a flag bit may be set in the read / write request. When the value of the flag bit is a first value (e.g., 1), it indicates that the read / write request is a first read / write request. When the value of the flag bit is a second value (e.g., 0), it indicates that the read / write request is a second read / write request. The target storage node cloud storage read / write rate limiting device can parse the read / write request after receiving it and determine whether the currently received read / write request is a first read / write request or a second read / write request based on the value of the flag bit.

[0034] In another exemplary embodiment of the present disclosure, the first read / write request and / or the second read / write request can also be sent and / or received through a preset interface. The target storage node cloud storage read / write rate limiting device can parse the read / write request after receiving it, and determine whether the currently received read / write request is the first read / write request or the second read / write request based on whether the source port and / or destination port of the read / write request is a preset interface and the correspondence between the preset interface and the first read / write request and / or the second read / write request.

[0035] Optionally, when the first read / write request is a write request, the first read / write request includes the data to be written and the data length; when the second read / write request is a write request, the second read / write request includes the data length.

[0036] Specifically, for write requests, the storage node receiving the first read / write request needs to perform a data write operation. Therefore, in addition to basic information such as data length, the first read / write request also needs to contain the data to be written. However, the storage node receiving the second read / write request only needs to understand the overall resource usage of the cloud storage cluster through the second read / write request and does not need to perform an actual data write operation. Therefore, the second read / write request can only contain data length and / or other basic information, without containing the data to be written. For read requests, both the first and second read / write requests can contain data length. The first read / write request also contains other basic information, while the second read / write request can be configured to contain other basic information according to actual needs, thereby ensuring that the additional overhead of sending the second read / write request is acceptable.

[0037] S102. Within each unit of time, based on the data length, receiving order, and read / write rate limit of each pending read / write request, target read / write requests are selected from the pending read / write requests. The sum of the data lengths of the target read / write requests and the first number of target read / write requests satisfy the read / write rate limit.

[0038] The data length in this embodiment can be understood as the length of the data to be read / written, which can be obtained by parsing the read / write request.

[0039] The read / write rate limiting requirements in this embodiment can be understood as requirements to limit the overall data input / output performance of the cloud storage cluster. Optionally, the read / write rate limiting requirements may include the bandwidth of the cloud storage cluster and the query volume per unit time. The bandwidth requirement is used to limit the amount of data read and written per unit time, and the query volume per unit time requirement is used to limit the number of requests processed per unit time.

[0040] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can, after obtaining the currently pending read / write requests, select target read / write requests that meet the read / write rate limiting requirements from the pending read / write requests within each unit time period, based on the data length, receiving order, and read / write rate limiting requirements of each pending read / write request. Specifically, after each unit time period, according to the receiving order of each pending read / write request, it can select one or more read / write requests with the earliest receiving time, and verify whether the sum of the data lengths corresponding to the selected read / write requests and the first number of selected read / write requests meet the read / write rate limiting requirements. If they do, the selected read / write requests are determined as target read / write requests; otherwise, the selection is repeated.

[0041] In an exemplary embodiment of this disclosure, when verifying whether the sum of the data lengths corresponding to the filtered read and write requests and the first number of filtered read and write requests meet the read and write rate limiting requirements, the cloud storage read and write rate limiting device can determine whether the sum of the data lengths is less than or equal to the data length threshold corresponding to the bandwidth, and whether the first number is less than or equal to the query volume per unit time. If both are satisfied, the verification is determined to be successful.

[0042] S103. Execute the first read / write request in the target read / write request and remove the target read / write request from the list of read / write requests to be executed.

[0043] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can, after filtering out the target read / write requests from the pending read / write requests, perform the corresponding data read / write operation for the first read / write request included in the target read / write request that is waiting to be executed by the target storage node. At the same time, since each storage node in the cloud storage cluster can filter out the same target read / write requests based on all the pending read / write requests and the same read / write rate limiting requirements, for example, the target read / write requests filtered by storage node A include the first read / write requests 1A, 2A and the second read / write request 3B, and the target read / write requests filtered by storage node B include the second read / write requests 1A, 2A and the first read / write request 3B, when storage node A executes 1A and 2A, storage node B will also execute 3B. The target read / write requests will be executed on each storage node of the cloud storage cluster, and each storage node can remove the target read / write requests from the pending read / write requests.

[0044] This embodiment of the disclosure obtains read / write requests to be executed sent by various clients. These requests include a first read / write request waiting to be executed by the target storage node and a second read / write request waiting to be executed by other storage nodes in the cloud storage cluster. Within each unit of time, based on the data length, receiving order, and read / write rate limiting requirements of each read / write request to be executed, a target read / write request is selected from the pending read / write requests. The sum of the data lengths corresponding to the target read / write requests and the first number of target read / write requests satisfy the read / write rate limiting requirements. The first read / write request is then executed, and the target read / write request is removed from the pending read / write requests. This allows the acquisition of all read / write requests that need to be processed by each storage node in the cloud storage cluster on the target storage node in the cloud storage cluster. Then, based on the information of all read / write requests, an overall rate limiting operation is performed. This ensures that even if multiple computing nodes concurrently access cloud storage resources in multi-mount mode, the read / write rate of cloud storage resources does not exceed the upper limit, ensuring reasonable utilization of storage resources and meeting the performance requirements in high-concurrency scenarios.

[0045] In some embodiments, when the cloud storage read / write rate limiting device receives read / write requests sent by each client, it determines the received read / write request as a read / write request to be executed and stores it in a pre-established waiting queue. When the target read / write request is removed from the read / write requests to be executed, the target read / write request is removed from the waiting queue.

[0046] Specifically, each storage node in the cloud storage cluster corresponds to a pre-established waiting queue, which stores the read and write requests received by that storage node that are yet to be executed. The cloud storage read / write rate limiting device can identify each received read / write request as a pending request and store it at the tail of the corresponding waiting queue. When filtering target read / write requests based on the receiving order, filtering can start from the head of the queue. After the first read / write request in the target queue is executed, the selected target read / write request is removed from the waiting queue, thus ensuring the orderly processing of read and write requests.

[0047] Figure 2 This is a schematic diagram illustrating an interaction of a read / write request provided in an embodiment of this disclosure, such as... Figure 2 As shown, the cloud storage cluster contains three storage nodes A, B, and C, providing storage services for clients a and b. Each storage node corresponds to a waiting queue, currently containing six read / write requests awaiting execution. In chronological order, these are: read / write request 1A initiated by client a and actually executed by storage node A; read / write request 2C initiated by client a and actually executed by storage node C; read / write request 3A initiated by client b and actually executed by storage node A; read / write request 4B initiated by client a and actually executed by storage node B; read / write request 5B initiated by client b and actually executed by storage node B; and read / write request 6C initiated by client b and actually executed by storage node C. Taking read / write request 1A initiated by client a and actually executed by storage node A as an example, client a will send the first read / write request corresponding to 1A to storage node A, which will then store it in the waiting queue corresponding to storage node A. Figure 2 (Highlighting is used in the middle), and sends the second read / write request corresponding to 1A to other storage nodes B and C and stores it in the waiting queues corresponding to storage nodes B and C. Figure 2 (Using the conventional display method), thus, the three waiting queues corresponding to storage nodes A, B, and C all contain all read and write requests to be executed.

[0048] Figure 3 This is a flowchart of a read / write request processing method provided in an embodiment of this disclosure, such as... Figure 3 As shown, based on the above embodiments, read and write requests can be processed using the following method.

[0049] S301. Compare the second number of read / write requests to be executed stored in the waiting queue with the preset queue depth.

[0050] The preset queue depth in this embodiment can be understood as the maximum number of read and write requests that the waiting queue can accommodate, and the specific value can be set according to actual needs.

[0051] In this embodiment of the disclosure, after receiving read and write requests sent by various clients, the cloud storage read and write rate limiting device can first obtain the second number of read and write requests currently stored in the waiting queue, and compare the second number with the value of the preset queue depth.

[0052] S302. If the second quantity is less than the preset queue depth, the received read / write request is determined as a read / write request to be executed.

[0053] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can determine that the waiting queue is not yet full when the second quantity is less than the preset queue depth, and can continue to store read / write requests in it. Therefore, the received read / write requests are determined as read / write requests to be executed, so that the received read / write requests can be stored in the waiting queue in the future.

[0054] S303. If the second quantity is greater than or equal to the preset queue depth, a read / write failure reminder is returned to the client that sent the read / write request so that the client can initiate a retry.

[0055] The read / write failure notification in this embodiment can be understood as a notification message used to remind the client that the read / write request currently sent has not entered the processing flow waiting to be executed, and that the read / write has failed.

[0056] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can determine that the waiting queue is currently full when the second quantity is greater than or equal to the preset queue depth, and cannot store the currently received read / write request into the waiting queue. Therefore, it returns a read / write failure reminder to the client that sent the read / write request, so that the client can initiate a retry after receiving the read / write failure reminder.

[0057] This embodiment compares a second number of read / write requests to be executed stored in the waiting queue with a preset queue depth. If the second number is less than the preset queue depth, the received read / write request is determined as a read / write request to be executed. If the second number is greater than or equal to the preset queue depth, a read / write failure reminder is returned to the client that sent the read / write request so that the client can initiate a retry. By controlling the number of read / write requests in the waiting queue, excessive consumption of computing resources on the storage node can be avoided, thus ensuring the service quality of the storage node.

[0058] Figure 4 This is a flowchart of a method for filtering target read / write requests provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, based on the above embodiments, target read / write requests can be filtered using the following method.

[0059] S401. Compare the time difference between the current time and the recorded time with the unit time.

[0060] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can obtain information about the pre-stored record time and the current time when the waiting queue is not empty, calculate the time difference between the current time and the record time, and then compare the time difference with the unit time.

[0061] S402. If the time difference is less than the unit time, the read and write requests to be executed are traversed sequentially according to the receiving order of the read and write requests to be executed. When the third number of the target read and write requests determined after the recording time is less than the query volume per unit time, and the sum of the data length of the currently traversed read and write requests to be executed and the total data length is less than or equal to the data length threshold corresponding to the bandwidth, the currently traversed read and write requests to be executed are determined as the target read and write requests, and the third number and the total data length are updated. The total data length is the sum of the data lengths of the target read and write requests determined after the recording time.

[0062] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can, when it determines that the time difference between the current time and the recording time is less than a unit time, sequentially traverse the read / write requests to be executed according to the receiving order of the read / write requests to be executed, and determine whether the currently traversed read / write request can be identified as a target read / write request. Specifically, it can obtain a third number of target read / write requests identified from the recording time to the current time, and when the third number is less than the query volume per unit time, further obtain the sum of the data lengths of the target read / write requests identified from the recording time to the current time, determine it as the total data length, and sum the total data length with the data length corresponding to the currently traversed read / write request. The algorithm checks whether the summation result is less than or equal to the data length threshold corresponding to the bandwidth. If the third quantity is less than the query volume per unit time and the summation result is also less than or equal to the data length threshold, then the currently traversed read / write request to be executed is determined as the target read / write request, and the third quantity and the total data length are updated. Specifically, the third quantity can be incremented by one, and the total data length can be updated to the sum of the current total data length and the data length of the newly determined target read / write request. The algorithm then continues to traverse the next read / write request to be executed, repeating the above judgment on the time difference, the third quantity, and the data length to determine whether the next read / write request to be executed can be selected as the target read / write request.

[0063] In one exemplary embodiment of this disclosure, if the time difference between the current time and the recording time is less than a unit time, and the third quantity is less than the query volume per unit time, and the sum of the data length of the currently traversed read / write request to be executed and the total data length is greater than the data length threshold, the cloud storage read / write rate limiting device can stop traversing other read / write requests within the current unit time and determine that all target read / write requests within the current unit time have been determined. Alternatively, it can skip the current read / write request and traverse the next read / write request, continuing to determine whether the sum of the data length of the next read / write request and the total data length is less than or equal to the data length threshold. If it satisfies this, the next read / write request is determined as the target read / write request, and the third quantity and the total data length are updated. If it does not satisfy this, the traversal continues to the next read / write request, and so on, until the time difference between the current time and the recording time is greater than or equal to a unit time, or the third quantity reaches the query volume per unit time, at which point the traversal of other read / write requests within the current unit time is stopped.

[0064] In another exemplary embodiment of this disclosure, after the pending read / write requests are stored in a pre-established waiting queue, a timer can be set in the cloud storage read / write rate limiting device. The timer can continuously scan the read / write requests in the waiting queue, maintaining a rate that does not exceed the read / write rate limiting requirement for processing requests at the head of the queue. Specifically, the timer can maintain the following three values: the timestamp of the recording time, the third number of target read / write requests determined in the current unit of time, and the total data length of the target read / write requests determined in the current unit of time. When the timer executes, if the timestamp of the current time matches the timestamp of the recording time in the timer... If the time difference of the timestamps is within a unit of time, then it is determined whether the third number of target read / write requests and the total data length determined within the unit of time are less than the read / write rate limit. If they are less, it means that the read / write request being traversed can be executed in this round, and it is determined as the target read / write request. The third number of target read / write requests determined in the current unit of time is incremented by one, and the total data length of the target read / write requests determined in the current unit of time is added to the data length of the read / write requests being traversed. If they are greater than or equal to, it means that the target read / write requests processed in the current unit of time have exceeded or reached the rate limit, and the timer ends the current round of filtering.

[0065] S403. If the time difference is greater than or equal to the unit time, the recorded time is updated, and the third quantity and total data length are reset.

[0066] In this embodiment of the disclosure, the cloud storage read / write rate limiting device can update the recording time to the current time and reset the third quantity and total data length to zero when it determines that the time difference between the current time and the recording time is greater than or equal to the unit time.

[0067] This embodiment compares the time difference between the current time and the recording time with a unit time. If the time difference is less than the unit time, the read and write requests to be executed are traversed sequentially according to the receiving order. When the third number of target read and write requests determined after the recording time is less than the query volume per unit time, and the sum of the data length of the currently traversed read and write requests and the total data length is less than or equal to the data length threshold corresponding to the bandwidth, the currently traversed read and write requests to be executed are determined as target read and write requests, and the third number and the total data length are updated. The total data length is the sum of the data lengths of the target read and write requests determined after the recording time. If the time difference is greater than or equal to the unit time, the recording time is updated, and the third number and the total data length are reset. This can ensure that the read and write rate of cloud storage resources does not exceed the upper limit, while executing as many read and write requests as possible per unit time, shortening the read and write response time, and further improving the user experience.

[0068] Figure 5 This is a schematic diagram of a cloud storage read / write speed limiting device provided in an embodiment of this disclosure. Figure 5 As shown, the cloud storage read / write rate limiting device 500 is applicable to target storage nodes in a cloud storage cluster. The cloud storage read / write rate limiting device 500 includes: an acquisition module 510, a filtering module 520, and an execution module 530. The acquisition module 510 is used to acquire read / write requests to be executed sent by various clients. These read / write requests include a first read / write request waiting to be executed by the target storage node and a second read / write request waiting to be executed by other storage nodes in the cloud storage cluster. The filtering module 520 is used to filter target read / write requests from the pending read / write requests within each unit of time, based on the data length, receiving order, and read / write rate limiting requirements of the cloud storage cluster. The sum of the data lengths corresponding to the target read / write requests and the first number of target read / write requests satisfy the read / write rate limiting requirements. The execution module 530 is used to execute the first read / write request among the target read / write requests and remove the target read / write request from the pending read / write requests.

[0069] Optionally, the acquisition module 510 includes: a receiving unit, configured to, in response to receiving read / write requests sent by various clients, determine the received read / write requests as the read / write requests to be executed and store them in a pre-established waiting queue; the execution module 530 includes: a removal unit, configured to remove the target read / write request from the waiting queue.

[0070] Optionally, the receiving unit includes: a comparison subunit, configured to compare a second number of read / write requests to be executed stored in the waiting queue with a preset queue depth; a determination subunit, configured to determine the received read / write request as the read / write request to be executed if the second number is less than the preset queue depth; and a sending subunit, configured to return a read / write failure reminder to the client that sent the read / write request if the second number is greater than or equal to the preset queue depth, so that the client can initiate a retry.

[0071] Optionally, the read / write rate limiting requirements include the bandwidth of the cloud storage cluster and the number of queries per unit time.

[0072] Optionally, the filtering module 520 includes: a comparison unit, used to compare the time difference between the current time and the recording time with the unit time; a traversal unit, used to, if the time difference is less than the unit time, sequentially traverse the read / write requests to be executed according to the receiving order of the read / write requests to be executed, and when the third number of target read / write requests determined after the recording time is less than the query volume per unit time, and the sum of the data length of the currently traversed read / write requests to be executed and the total data length is less than or equal to the data length threshold corresponding to the bandwidth, determine the currently traversed read / write requests to be executed as the target read / write requests, and update the third number and the total data length, wherein the total data length is the sum of the data lengths of the target read / write requests determined after the recording time; and a change unit, used to, if the time difference is greater than or equal to the unit time, update the recording time and reset the third number and the total data length.

[0073] Optionally, when the first read / write request is a write request, the first read / write request includes the data to be written and the data length; when the second read / write request is a write request, the second read / write request includes the data length.

[0074] The cloud storage read / write speed limiting device provided in this embodiment can execute the method described in any of the above embodiments. Its execution method and beneficial effects are similar, and will not be repeated here.

[0075] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure.

[0076] like Figure 6 As shown, the computer device may include a processor 610 and a memory 620 storing computer program instructions.

[0077] Specifically, the processor 610 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0078] Memory 620 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 620 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 620 may include removable or non-removable (or fixed) media. Where appropriate, memory 620 may be internal or external to the integrated gateway device. In a particular embodiment, memory 620 is a non-volatile solid-state memory. In a particular embodiment, memory 620 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0079] The processor 610 reads and executes computer program instructions stored in the memory 620 to perform the steps of the cloud storage read / write rate limiting method provided in this embodiment of the disclosure.

[0080] In one example, the computer device may also include a transceiver 630 and a bus 640. Wherein, as... Figure 6 As shown, the processor 610, memory 620 and transceiver 630 are connected via bus 640 and communicate with each other.

[0081] Bus 640 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 640 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0082] This disclosure also provides a computer-readable storage medium that can store a computer program. When the computer program is executed by a processor, the processor enables the processor to implement the cloud storage read / write speed limiting method provided in this disclosure.

[0083] The aforementioned storage medium may, for example, include a memory 620 containing computer program instructions, which can be executed by the processor 610 of the cloud storage read / write rate limiting device to complete the cloud storage read / write rate limiting method provided in this embodiment. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), compact disc ROM (CD-ROM), magnetic tape, floppy disk, and optical data storage device. The aforementioned computer program may be written in any combination of one or more programming languages ​​to perform the operations of this embodiment. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0084] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, causes the processor to implement the cloud storage read / write speed limiting method provided in this disclosure.

[0085] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cloud storage read / write rate limiting method, characterized in that, The method is applicable to target storage nodes in a cloud storage cluster, and the method includes: Obtain the read and write requests to be executed sent by each client, including a first read and write request waiting to be executed by the target storage node and a second read and write request waiting to be executed by other storage nodes in the cloud storage cluster; Within each unit of time, based on the data length, receiving order, and read / write rate limiting requirements of the cloud storage cluster corresponding to each read / write request to be executed, target read / write requests are selected from the read / write requests to be executed. The sum of the data lengths corresponding to the target read / write requests and the first number of the target read / write requests satisfy the read / write rate limiting requirements. Execute the first read / write request in the target read / write request, and remove the target read / write request from the list of read / write requests to be executed.

2. The method according to claim 1, characterized in that, The step of obtaining the read / write requests to be executed sent by each client includes: In response to receiving read / write requests from various clients, the received read / write requests are identified as the read / write requests to be executed and stored in a pre-established waiting queue. Removing the target read / write request from the list of read / write requests to be executed includes: Remove the target read / write request from the waiting queue.

3. The method according to claim 2, characterized in that, Before determining the received read / write request as the read / write request to be executed, the method further includes: The second number of read / write requests to be executed stored in the waiting queue is compared with the preset queue depth; If the second quantity is less than the preset queue depth, the received read / write request will be determined as the read / write request to be executed. If the second quantity is greater than or equal to the preset queue depth, a read / write failure notification is returned to the client that sent the read / write request, so that the client can initiate a retry.

4. The method according to claim 1, characterized in that, The read / write rate limit requirements include the bandwidth of the cloud storage cluster and the number of queries per unit time.

5. The method according to claim 4, characterized in that, The step of filtering target read / write requests from the pending read / write requests within each unit of time, based on the data length, receiving order, and read / write rate limiting requirements of the cloud storage cluster for each pending read / write request, includes: Compare the time difference between the current time and the recorded time with the unit time; If the time difference is less than the unit time, the read and write requests to be executed are traversed sequentially according to the receiving order of the read and write requests to be executed. When the third number of target read and write requests determined after the recording time is less than the query volume per unit time, and the sum of the data length of the currently traversed read and write requests to be executed and the total data length is less than or equal to the data length threshold corresponding to the bandwidth, the currently traversed read and write requests to be executed are determined as the target read and write requests, and the third number and the total data length are updated. The total data length is the sum of the data lengths of the target read and write requests determined after the recording time. If the time difference is greater than or equal to the unit time, the recorded time is updated, and the third quantity and the total data length are reset.

6. The method according to claim 1, characterized in that, When the first read / write request is a write request, the first read / write request includes the data to be written and the data length; when the second read / write request is a write request, the second read / write request includes the data length.

7. A cloud storage read / write rate limiting device, characterized in that, The device is applicable to target storage nodes in a cloud storage cluster, and the device includes: The acquisition module is used to acquire read and write requests to be executed sent by each client. The read and write requests to be executed include a first read and write request waiting to be executed by the target storage node and a second read and write request waiting to be executed by other storage nodes in the cloud storage cluster. The filtering module is used to filter target read and write requests from the read and write requests to be executed within each unit time period, based on the data length, receiving order and read and write rate limiting requirements of the cloud storage cluster for each read and write request to be executed. The sum of the data lengths corresponding to the target read and write requests and the first number of the target read and write requests satisfy the read and write rate limiting requirements. An execution module is used to execute the first read / write request in the target read / write request and remove the target read / write request from the read / write requests to be executed.

8. A computer device, characterized in that, include: Memory; processor; And a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the cloud storage read / write rate limiting method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the cloud storage read / write rate limiting method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the cloud storage read / write rate limiting method as described in any one of claims 1-6.

Images