Back-end flow control method and device, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of computers, and discloses a back-end flow control method and device, computer equipment and a readable storage medium, and the method comprises the steps: obtaining current dirty cache line information and current request information; comparing the current dirty cache line information with a preset flow control range, and determining and executing a first operation; and comparing the current request information with a preset flow control threshold value, and determining and executing a second operation. Therefore, the processing operation on the first request is determined based on the size relationship between the use condition of the current dirty cache line and the preset flow control range, and the processing operation on the second request is determined based on the size relationship between the use condition of the current back-end IO and the preset flow control threshold, so that the use conditions of the current dirty cache line and the back-end IO are comprehensively considered; and IO execution of the foreground and the background is considered, so that the system performance is ensured while flow control is performed, and the conditions of reduction and fluctuation of the system performance are avoided.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and specifically to a back-end flow control method, apparatus, computer equipment, and readable storage medium. Background Art

[0002] As hard disks are used for a longer time, they will experience performance degradation due to fragmentation, so flow control is required to prevent poor-performance disks from slowing down disks with good performance and causing a decrease in the IOPS (Input / Output Operations Per Second) of the entire system. The current method of implementing flow control is usually to stop sending backend IO after exceeding the flow control line, and to hang the backend IO in a linked list, and then remove the link and deliver it after the number of backend IO drops below the flow control line. This will cause the resource number of the backend IO to be occupied by the linked list for a long time, resulting in other disks that are not under flow control being unable to generate backend IO because they have no resource number, causing system performance to degrade. Summary of the invention

[0003] In view of the above problems, the embodiments of the present invention provide a backend flow control method, apparatus, computer device and readable storage medium, which are used to solve the problem in the prior art that flow control of backend IO may cause system performance degradation.

[0004] According to one aspect of an embodiment of the present invention, a backend flow control method is provided, the method comprising:

[0005] Get the current dirty cache line information and the current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO;

[0006] Compare the current dirty cache line information with the preset flow control range, determine and execute the first operation; the first operation is used to represent the processing operation of the first request; the first request is an IO that generates a backend IO by applying for a dirty cache line after being issued;

[0007] The current request information is compared with the preset flow control threshold to determine and execute the second operation; the second operation is used to characterize the processing operation of the second request; the second request is an IO that directly generates a backend IO after being issued.

[0008] In an optional implementation, the preset flow control range includes a dirty cache line threshold; comparing the current dirty cache line information with the preset flow control range, and determining and executing the first operation, including:

[0009] If the current dirty cache line information is less than the dirty cache line threshold, determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and performing the first operation;

[0010] If the current dirty cache line information is greater than the dirty cache line threshold, it is determined that the first operation is to hang the first request in a waiting queue, and execute the first operation.

[0011] In an optional implementation, the preset flow control range further includes a refresh threshold; the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold; after determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and performing the first operation, the method further includes:

[0012] Update the current dirty cache line information;

[0013] If the current dirty cache line information is greater than the refresh threshold, a refresh command is initiated to refresh and release the dirty cache line.

[0014] In an optional implementation, comparing the current request information with a preset flow control threshold, determining and executing a second operation, includes:

[0015] If the current request information is less than the preset flow control threshold, determining the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and executing the second operation;

[0016] If the current request information is greater than the preset flow control threshold, the second operation is determined to be to hang the second request in the waiting queue and execute the second operation.

[0017] In an optional implementation, after determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and executing the second operation, the method further includes:

[0018] Update the current request information based on the issued backend IO and / or the returned response backend IO;

[0019] If the current request information is less than the preset flow control threshold, a refresh command is initiated to refresh and release the dirty cache line, and take the second request out of the waiting queue.

[0020] In an optional embodiment, the method further includes:

[0021] In response to taking out the first request from the waiting queue, if the current dirty cache line information corresponding to the first request is greater than the dirty cache line threshold, setting the waiting queue corresponding to the first request to prohibit unlinking;

[0022] In response to taking out the second request from the waiting queue, if the current request information corresponding to the second request is greater than a preset flow control threshold, the waiting queue corresponding to the second request is set to prohibit link disconnection.

[0023] In an optional implementation, obtaining current dirty cache line information and current request information includes:

[0024] Get the number of dirty cache lines currently applied, the current number of dirty cache lines, and the current number of dirty cache lines to be released;

[0025] Add the number of currently applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released to obtain current dirty cache line information;

[0026] Get the current backend IO quantity and current backend IO bandwidth;

[0027] Based on the current backend IO quantity and the current backend IO bandwidth, the current request information is determined.

[0028] According to another aspect of an embodiment of the present invention, a backend flow control device is provided, the device comprising:

[0029] An information acquisition module is used to acquire current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO;

[0030] A first operation determination module is used to compare the current dirty cache line information with the preset flow control range, determine and execute the first operation; the first operation is used to characterize the processing operation of the first request; the first request is an IO that generates a backend IO by applying for a dirty cache line after being issued;

[0031] The second operation determination module is used to compare the current request information with the preset flow control threshold, determine and execute the second operation; the second operation is used to characterize the processing operation of the second request; the second request is an IO that directly generates a backend IO after being issued.

[0032] According to another aspect of an embodiment of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0033] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute any operation of the back-end flow control method described above.

[0034] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which at least one executable instruction is stored. When the executable instruction is executed on a back-end flow control device or a computer device, the back-end flow control device or the computer device performs the operation of any of the back-end flow control methods described above.

[0035] The embodiments of the present invention respectively set a preset flow control range for the current usage of dirty cache lines, set a preset flow control threshold for the current usage of backend IO, determine the processing operation of the first request based on the size relationship between the current usage of dirty cache lines and the preset flow control range, and determine the processing operation of the second request based on the size relationship between the current usage of backend IO and the preset flow control threshold, thereby comprehensively considering the usage of the current dirty cache lines and backend IO, taking into account the execution of foreground and background IO, thereby ensuring system performance while controlling the flow and avoiding the decline and fluctuation of system performance.

[0036] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0038] Figure 1 It is a flow chart of a back-end flow control method provided by an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of a preset flow control range and a preset flow control threshold in a backend flow control method provided by an embodiment of the present invention;

[0040] Figure 3 It is a flowchart of determining and executing a first operation in a backend flow control method provided by an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of a flow chart of releasing dirty cache lines in a back-end flow control method provided by an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of a flow chart of determining and executing a second operation in a backend flow control method provided by an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of a flow chart of releasing dirty cache lines in a back-end flow control method provided by an embodiment of the present invention;

[0044] Figure 7 It is a timing diagram of flow control for dirty cache lines in a back-end flow control method provided by an embodiment of the present invention;

[0045] Figure 8It is a timing diagram of flow control of backend IO in a backend flow control method provided in an embodiment of the present invention;

[0046] Fig. 9 is a schematic structural diagram of an embodiment of a back-end flow control device provided by an embodiment of the present invention;

[0047] Fig.10 It is a schematic diagram of the structure of an embodiment of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0049] As hard disks are used for a longer time, they will experience performance degradation due to fragmentation, so flow control is required to prevent poor-performance disks from slowing down disks with good performance and causing a decrease in the IOPS (Input / Output Operations Per Second) of the entire system. The current method of implementing flow control is usually to stop sending backend IO after exceeding the flow control line, and to hang the backend IO in a linked list, and then remove the link and deliver it after the number of backend IO drops below the flow control line. This will cause the resource number of the backend IO to be occupied by the linked list for a long time, resulting in other disks that are not under flow control being unable to generate backend IO because they have no resource number, causing system performance to degrade.

[0050] In addition, due to the observation lag, when the module or unit responsible for flow control detects that the number of backend IOs has dropped below the flow control line, the system and disk have been idle for a period of time. At this time, only one backend IO is picked from the linked list, which will cause the system bandwidth to be unable to be fully utilized, resulting in inadequate use of system resources and reduced system performance. Furthermore, during the flow control process, dirty cache lines will also be flow controlled. The flow control of dirty cache lines is to control the number of dirty cache lines applied for IO of all disks. The total number of dirty cache lines cannot exceed the threshold, otherwise they will be flow controlled and hung in the linked list; if a disk has more WBIO (Write back IO), it will apply for more dirty cache lines, which will cause the risk of flow control for WB IO of other disks, resulting in the inability to execute WB IO of other disks, resulting in reduced system write performance. In addition, relying only on one linked list to control dirty cache lines will cause uneven flush IO of each disk during flush, causing system performance fluctuations.

[0051] Based on this, an embodiment of the present invention provides a back-end flow control method, which includes: obtaining current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current back-end IO; comparing the current dirty cache line information with a preset flow control range, determining and executing a first operation; the first operation is used to characterize the processing operation of the first request; the first request is the IO sent by the host and then written to the cache and returned in response; comparing the current request information with a preset flow control threshold, determining and executing a second operation; the second operation is used to characterize the processing operation of the second request; the second request is the IO sent by the host to the disk array or the cache flushed to the disk array. In this way, a preset flow control range is set for the current usage of the dirty cache line, and a preset flow control threshold is set for the current usage of the backend IO. The processing operation of the first request is determined based on the size relationship between the current usage of the dirty cache line and the preset flow control range, and the processing operation of the second request is determined based on the size relationship between the current usage of the backend IO and the preset flow control threshold. This comprehensively considers the usage of the current dirty cache line and the backend IO, and takes into account the execution of the foreground and background IO, so as to ensure the system performance while controlling the flow and avoid the decline and fluctuation of the system performance.

[0052] The following describes a specific embodiment of a backend flow control method of the present invention. Figure 1 It is a flowchart of a back-end flow control method provided by an embodiment of the present invention. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 1 As shown, the method may include:

[0053] Step 110: Obtain current dirty cache line information and current request information.

[0054] In the embodiment of the present invention, the dirty cache line is applied when WB IO (Write back IO) is executed, and the dirty cache line will generate backend IO when flushing, that is, backend IO, and WT IO (Write through IO) and read IO will also generate backend IO, which leads to two sources of demand for backend IO, but the processing capacity of the disk is limited. When the number of backend IO issued exceeds the processing capacity of the disk, the backend IO will be in a blocked state, resulting in other IOs unable to apply for resource numbers and unable to process. Based on this, in order to achieve flow control of backend IO, not only flow control should be performed according to the usage of backend IO, but also the usage of dirty cache lines should be comprehensively considered.

[0055] In an embodiment of the present invention, the current dirty cache line information is used to characterize the usage of the current dirty cache line, that is, to characterize the current occupancy and demand for the dirty cache line; the current request information is used to characterize the usage of the current backend IO, that is, to characterize the current backend IO, that is, to characterize the occupancy and demand for the backend IO.

[0056] In an optional implementation manner, the current dirty cache line information and the current request information may be obtained and determined as follows:

[0057] Step a1: Obtain the number of currently applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released.

[0058] In step a1, the current number of applied dirty cache lines wb_dirty_cl_cnt is the number of cache lines that have been applied but have not yet become dirty cache lines, that is, the cache lines have been applied and the data of WB IO is being moved from the main memory to the cache lines; the current number of dirty cache lines dirty_cl_cnt is the number of existing dirty cache lines, that is, the cache lines that have actually become dirty, and the data in the cache lines is inconsistent with the data in the disk; the current number of dirty cache lines to be released flush_dirty_cl_cnt is the number of dirty cache lines that have been flushed but the dirty data therein has not been cleared (cleaned), that is, a refresh request has been initiated and the data in the cache lines is being written back to the disk.

[0059] Step a2: Add the number of currently applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released to obtain current dirty cache line information.

[0060] In step a2, the current dirty cache line information curt_dirty_cl_cnt is obtained by adding the current number of dirty cache lines applied wb_dirty_cl_cnt, the current number of dirty cache lines dirty_cl_cnt and the current number of dirty cache lines to be released flush_dirty_cl_cnt; that is, as shown in the following formula:

[0061] curt_dirty_cl_cnt=dirty_cl_cnt+wb_dirty_cl_cnt+flush_dirty_cl_cnt.

[0062] Step a3: Get the current backend IO quantity and current backend IO bandwidth.

[0063] In step a3, the current backend IO quantity is the number of backend IOs currently obtained by statistics, and the current backend IO bandwidth is the data volume of the backend IO currently obtained by statistics.

[0064] Step a4: Based on the current backend IO quantity and the current backend IO bandwidth, determine the current request information.

[0065] In step a4, the current request information is related to both the current backend IO quantity and the current backend IO bandwidth. That is, the current request information comprehensively considers the current backend IO quantity and the current backend IO bandwidth to monitor and control both.

[0066] In an optional implementation, for each disk array in the system, the current dirty cache line information and current request information corresponding to it are obtained respectively, and flow control is performed according to the current dirty cache line information and current request information corresponding to it respectively. At the same time, for the current request information, the total link bandwidth of the system is additionally monitored, that is, the sum of the current backend IO bandwidth corresponding to all disk arrays is monitored to prevent the total bandwidth of the system from exceeding the performance range.

[0067] Step 120: Compare the current dirty cache line information with the preset flow control range, determine and execute the first operation.

[0068] In an embodiment of the present invention, a preset flow control range is used to limit the flow control range of the current dirty cache line. The current dirty cache line information is compared with the preset flow control range, and the first operation is determined according to the size relationship between the current dirty cache line information and the preset flow control range, and the first operation is executed. Among them, the first operation is used to characterize the processing operation of the first request, and the first request is an IO that generates a backend IO by applying for a dirty cache line after being issued. When processing the first request, it is necessary to first apply for a dirty cache line, and then flush the dirty cache line to generate a backend IO, so as to move the data in the first request from the main memory to the dirty cache line, and then write it from the dirty cache line to the disk. Therefore, in order to realize the flow control of the dirty cache line and the backend IO, when processing the first request, it is necessary to determine whether the dirty cache line is in flow control, and then determine the processing operation of the first request, that is, the first operation.

[0069] In the embodiment of the present invention, the first request is an IO sent by the host and written to the cache and then returned in response, that is, a WB IO.

[0070] Step 130: Compare the current request information with the preset flow control threshold, determine and execute the second operation.

[0071] In an embodiment of the present invention, a preset flow control threshold is used to limit the flow control threshold of the current request. The current request information is compared with the preset flow control threshold, and the second operation is determined based on the size relationship between the current request information and the preset flow control threshold, and the second operation is executed. Among them, the second operation is used to characterize the processing operation of the second request, and the second request is an IO that directly generates the backend IO after being issued. The second request can directly generate the backend IO when processing, and write the data in the second request from the main memory and cache to the disk. Therefore, in order to realize the flow control of the backend IO, when processing the second request, it is necessary to determine that the backend IO is in flow control, and then determine the processing operation of the second request, that is, the second operation.

[0072] In the embodiment of the present invention, the second request may include an IO sent by the host that is written to the cache and then written to the disk array before returning a response, that is, a WT IO, and a read-type IO, that is, a read IO.

[0073] In an optional implementation, flow control is performed on dirty cache lines and backend IO for each disk, respectively. Accordingly, the preset flow control range for flow control of dirty cache lines and the preset flow control threshold for flow control of backend IO may be different for different disks.

[0074] In an optional embodiment, Figure 2 Schematic diagram of a preset flow control range and a preset flow control threshold in a backend flow control method provided by an embodiment of the present invention, such as Figure 2As shown, the preset flow control range includes a dirty cache line threshold and a refresh threshold, wherein the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold. The dirty cache line threshold is used to characterize the threshold for flow control of dirty cache line applications. If the current dirty cache line information is greater than the dirty cache line threshold, flow control is performed on dirty cache line applications, and dirty cache line applications are no longer allowed. The refresh threshold is used to characterize the threshold for flow control of flushing dirty cache lines. If the current dirty cache line information is greater than the refresh threshold, a preset number of dirty cache lines are flushed to control the number of dirty cache lines to be below the number of dirty cache lines corresponding to the refresh threshold.

[0075] In an optional embodiment, Figure 3 This is a flow chart of determining and executing a first operation in a backend flow control method provided by an embodiment of the present invention. The above step 120: comparing the current dirty cache line information with the preset flow control range, determining and executing the first operation, may include the following steps:

[0076] Step 310: If the current dirty cache line information is less than the dirty cache line threshold, determine that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and perform the first operation.

[0077] In the embodiment of the present invention, Figure 2 As shown, if the current dirty cache line information is less than the dirty cache line threshold, it indicates that the flow control of the dirty cache line is released at this time, and the dirty cache line can be applied for at this time, that is, the first request can be processed. Accordingly, the first operation is determined to be taking out the first request from the waiting queue and / or applying for a dirty cache line for the first request, and executing the first operation.

[0078] In an embodiment of the present invention, if there is a first request in the waiting queue, the first request is taken out from the waiting queue, and a dirty cache line is applied for the first request to respond to and process the first request, that is, the first operation at this time is to take out the first request from the waiting queue and apply for a dirty cache line for the first request; if there is no first request in the waiting queue and the first request is received at this time, a dirty cache line can be directly applied for the received first request to respond to and process the first request, that is, the first operation at this time is to apply for a dirty cache line for the first request.

[0079] Step 320: If the current dirty cache line information is greater than the dirty cache line threshold, determine that the first operation is to hang the first request in the waiting queue, and execute the first operation.

[0080] In the embodiment of the present invention, Figure 2As shown, if the current dirty cache line information is greater than the dirty cache line threshold, it indicates that the dirty cache line is in flow control at this time, and the dirty cache line cannot be applied at this time, that is, the first request cannot be responded to and processed. Accordingly, the first operation is determined to hang the first request in the waiting queue and execute the first operation.

[0081] In an optional implementation, the waiting queue may be a linked list, i.e., a wait IO linked list. Then, in the above step 310, if the wait IO linked list is empty and the first request is received at this time, the first operation is to apply for a dirty cache line for the first request. If the wait IO linked list is not empty, a delinking operation is performed. At this time, the first operation is to take the first request out of the waiting queue, i.e., the wait IO linked list, and apply for a dirty cache line for the first request. In the above step 320, the first operation is to hang the first request into the waiting queue, i.e., the wait IO linked list.

[0082] In an optional implementation, the above steps 310 to 320 are steps of responding to the first request according to the dirty cache line threshold and applying for dirty cache lines, which will increase the number of dirty cache lines. Correspondingly, the method also includes steps of flushing and releasing dirty cache lines to reduce the number of dirty cache lines. Figure 4 is a flow chart of releasing dirty cache lines in a backend flow control method provided by an embodiment of the present invention, such as Figure 4 As shown, the above step 320: determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and after executing the first operation, further includes the steps of flushing the dirty cache line according to the refresh threshold, which are specifically as follows:

[0083] Step 410: Update current dirty cache line information.

[0084] In the embodiment of the present invention, after applying for dirty cache lines for the first request, the number of dirty cache lines will increase, and thus the current dirty cache line information is updated based on the number of applied dirty cache lines.

[0085] Step 420: If the current dirty cache line information is greater than the refresh threshold, a refresh command is initiated to refresh and release the dirty cache line.

[0086] In the embodiment of the present invention, Figure 2 As shown, if the current dirty cache line information is greater than the refresh threshold, it indicates that the number of dirty cache lines has exceeded the refresh threshold, and a certain number of dirty cache lines need to be flushed to maintain the number of dirty cache lines below or near the refresh threshold. At this time, a refresh command is initiated to flush the dirty cache lines, that is, to refresh and release the dirty cache lines.

[0087] In an optional implementation, after the flush of the dirty cache lines is completed and the dirty cache lines are released, the number of dirty cache lines will decrease, so the current dirty cache line information is updated again based on the number of released dirty cache lines.

[0088] In an optional implementation, a preset number of dirty cache lines may be acquired and flushed each time. If, after a flush of the dirty cache lines, the current dirty cache line information is still greater than the refresh threshold, the preset number of dirty cache lines may continue to be acquired and flushed until the current dirty cache line information is less than the refresh threshold, thereby maintaining the number of dirty cache lines below or near the refresh threshold.

[0089] In an optional embodiment, Figure 5 The flowchart of determining and executing a second operation in a backend flow control method provided by an embodiment of the present invention is as follows. The above step 130: comparing the current request information with the preset flow control threshold, determining and executing the second operation, may include the following steps:

[0090] Step 510: If the current request information is less than the preset flow control threshold, determine that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and execute the second operation.

[0091] In the embodiment of the present invention, Figure 2 As shown, if the current request information is less than the preset flow control threshold, it indicates that the flow control on the backend IO is released at this time, and the backend IO can be issued at this time, that is, the second request can be processed. Accordingly, the second operation is determined to be taking out the second request from the waiting queue and / or issuing the backend IO corresponding to the second request, and executing the second operation.

[0092] In an embodiment of the present invention, if there is a second request in the waiting queue, the second request is taken out from the waiting queue, and the backend IO corresponding to the second request is issued to respond to and process the second request, that is, the second operation at this time is to take out the second request from the waiting queue, and to issue the backend IO corresponding to the second request; if there is no second request in the waiting queue, and the second request is received at this time, the backend IO corresponding to the received second request can be directly issued to respond to and process the second request, that is, the second operation at this time is to issue the backend IO corresponding to the second request.

[0093] Step 520: If the current request information is greater than the preset flow control threshold, determine that the second operation is to hang the second request in the waiting queue and execute the second operation.

[0094] In the embodiment of the present invention, Figure 2As shown, if the current request information is greater than the preset flow control threshold, it indicates that the backend IO is in flow control at this time, and the backend IO cannot be issued at this time, that is, the second request cannot be responded to and processed. Accordingly, the second operation is determined to hang the second request in the waiting queue and execute the second operation.

[0095] In an optional implementation, the waiting queue can be a linked list, i.e., a wait IO linked list. Then, in the above step 510, if the wait IO linked list is empty and the second request is received at this time, the second operation is to issue the backend IO corresponding to the second request. If the wait IO linked list is not empty, the delinking operation is performed. At this time, the second operation is to take the second request out of the waiting queue, i.e., the wait IO linked list, and issue the backend IO corresponding to the second request. In the above step 520, the second operation is to hang the second request into the waiting queue, i.e., the wait IO linked list.

[0096] In an optional implementation, since flushing dirty cache lines will also generate backend IO, and only after the generated backend IO is processed can the clean operation on the dirty cache lines be completed, thereby releasing the dirty cache lines, and considering that when the current dirty cache line information is greater than the refresh threshold, the current request information is often also greater than the preset flow control threshold, at this time, since the backend IO cannot be sent down and processed, the dirty cache lines cannot be flushed normally, the method also includes the relevant steps of flushing the dirty cache lines to release the dirty cache lines. Figure 6 is a flow chart of releasing dirty cache lines in a backend flow control method provided by an embodiment of the present invention, such as Figure 6 As shown, in the above step 610: determining that the second operation is to take the second request out of the waiting queue and / or send the backend IO corresponding to the second request, and executing the second operation, it also includes the related steps of flushing the dirty cache line according to the preset flow control threshold, which are as follows:

[0097] Step 610: based on the issued backend IO and / or the returned response backend IO, update the current request information.

[0098] In the embodiment of the present invention, after the backend IO is sent, the number of backend IO and the bandwidth corresponding to the backend IO will increase, so the current request information is updated based on the sent backend IO. After the backend IO processing is completed and the response is returned, the number of backend IO will decrease and the originally occupied bandwidth will be released, so the current request information is updated based on the backend IO that returns the response.

[0099] Step 620: If the current request information is less than the preset flow control threshold, a refresh command is initiated to refresh and release the dirty cache line, and take out the second request from the waiting queue.

[0100] In the embodiment of the present invention, Figure 2 As shown, if the current request information is less than the preset flow control threshold, a certain number of dirty cache lines are released at this time to avoid the situation where the current dirty cache line information is greater than the refresh threshold and the current request information is also greater than the preset flow control threshold, which results in the inability to send and process the backend IO and flush the dirty cache lines normally. Therefore, when the current request information is less than the preset flow control threshold, a refresh command is initiated to flush the dirty cache lines, that is, refresh and release the dirty cache lines, and at the same time, the second request is taken out from the waiting queue, that is, the link is removed from the wait IO linked list corresponding to the second request.

[0101] In an optional embodiment, a preset number of dirty cache lines can be obtained and flushed each time. If after the refresh command is initiated, the number of backend IOs generated makes the current request information greater than the preset flow control threshold, the flushing of the dirty cache lines is stopped until the backend IO response returns, the number of backend IOs decreases, and the current request information is less than the preset flow control threshold, triggering the flushing of the dirty cache lines again.

[0102] In an optional implementation, if the current request information is less than a preset flow control threshold, when flushing a dirty cache line, the system is flushing other disks and has not yet scheduled a flush for this disk, then the flush will be retained until the dirty cache line corresponding to the flush is released.

[0103] In an optional embodiment, Figure 7 This is a timing diagram of flow control for dirty cache lines in a backend flow control method provided by an embodiment of the present invention. Figure 7 The flow control of dirty cache lines is further explained. Figure 7 As shown, at time T0, the flow control of the backend IO is released, and the number of released dirty cache lines is n (in units of CL cnt); at time T1, the flow control of the backend IO is released, and at time T2, the flow control of the backend IO is effective. During this period, since the released amount of dirty cache lines has not been consumed, it has been kept at n; at time T3, flush is started, n dirty cache lines are removed from the linked list for flushing, and all the retained dirty cache lines are released; at time T4, the flow control of the backend IO is effective, resulting in the failure of flush start at time T5; at time T6, the flow control of the backend IO is released, and the foreground wait IO queue, that is, the waiting queue corresponding to the first request is empty. In order to increase the number of flushes of dirty cache lines when the foreground IO is idle, the released amount of dirty cache lines is increased to n+delta, where the specific value of delta can be configured according to actual needs.

[0104] In an optional embodiment, Figure 8This is a timing diagram of backend IO flow control in a backend flow control method provided by an embodiment of the present invention. Figure 8 Further explanation of backend IO flow control. Figure 8 As shown, at time T0, the flow control of the backend IO is released. At this time, the second request waiting to be processed, that is, the WT / read wait IO release amount is m (in units of IOcnt), that is, the release amount of the waiting queue or linked list corresponding to the second request is m, that is, at this time, m second requests can be removed from the linked list corresponding to the second request for execution; at time T1, m WT / read IO executions are removed from the linked list, and the WT / readwait IO release amount becomes 0; at time T2, the BIO flow control is released, but the number of dirty cache lines is 0 at this time. In order to process more foreground wait IO when the number of dirty cache lines is 0, the WT / read wait IO release amount is increased to m+delta, where the specific value of delta can be configured according to actual needs; at time T3, m+delta WT / read IO executions are removed from the linked list, and the WT / read wait IO release amount is 0.

[0105] In an optional embodiment, in order to avoid repeated chain removal and flow control failure, when taking out the first request and the second request from the waiting queue and performing corresponding operations, the following steps are also included: in response to taking out the first request from the waiting queue, if the current dirty cache line information corresponding to the first request is greater than the dirty cache line threshold, the waiting queue corresponding to the first request is set to prohibit chain removal; in response to taking out the second request from the waiting queue, if the current request information corresponding to the second request is greater than the preset flow control threshold, the waiting queue corresponding to the second request is set to prohibit chain removal. In this way, repeated chain removal is avoided when the first request and the second request taken out have not been processed. Among them, the first request and the second request correspond to different waiting queues, that is, the first request and the second request are hung in different linked lists when hanging the chain.

[0106] The back-end flow control method provided by the embodiment of the present invention includes: obtaining current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current back-end IO; comparing the current dirty cache line information with a preset flow control range, determining and executing a first operation; the first operation is used to characterize the processing operation of the first request; the first request is the IO sent by the host and then written to the cache and then returned to the response; comparing the current request information with a preset flow control threshold, determining and executing a second operation; the second operation is used to characterize the processing operation of the second request; the second request is the IO sent by the host and then written to the cache and then written to the disk array and then returned to the response. In this way, a preset flow control range is set for the current usage of the dirty cache line, and a preset flow control threshold is set for the current usage of the backend IO. The processing operation of the first request is determined based on the size relationship between the current usage of the dirty cache line and the preset flow control range, and the processing operation of the second request is determined based on the size relationship between the current usage of the backend IO and the preset flow control threshold. This comprehensively considers the usage of the current dirty cache line and the backend IO, and takes into account the execution of the foreground and background IO, so as to ensure the system performance while controlling the flow and avoid the decline and fluctuation of the system performance.

[0107] The embodiment of the present invention further provides a back-end flow control device, Fig. 9 is a schematic diagram of the structure of an embodiment of a back-end flow control device provided by an embodiment of the present invention, such as Fig. 9 As shown, the device 900 includes:

[0108] The information acquisition module 910 is used to acquire the current dirty cache line information and the current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO;

[0109] The first operation determination module 920 is used to compare the current dirty cache line information with the preset flow control range, determine and execute the first operation; the first operation is used to represent the processing operation of the first request; the first request is an IO that generates a backend IO by applying for a dirty cache line after being issued;

[0110] The second operation determination module 930 is used to compare the current request information with the preset flow control threshold, determine and execute the second operation; the second operation is used to characterize the processing operation of the second request; the second request is an IO that directly generates a backend IO after being issued.

[0111] In an optional implementation, the preset flow control range includes a dirty cache line threshold; the first operation determination module 910 includes:

[0112] A first request execution unit, configured to determine that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and execute the first operation if the current dirty cache line information is less than the dirty cache line threshold;

[0113] The first request hanging unit is used to determine the first operation is to hang the first request into the waiting queue if the current dirty cache line information is greater than the dirty cache line threshold, and perform the first operation.

[0114] In an optional implementation, the preset flow control range further includes a refresh threshold; the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold; the device further includes:

[0115] A current dirty cache line information update module, used to update the current dirty cache line information;

[0116] The dirty cache line release module is used to initiate a refresh command to refresh and release the dirty cache line if the current dirty cache line information is greater than a refresh threshold.

[0117] In an optional implementation, the second operation determination module 920 includes:

[0118] A second request execution unit, configured to determine, if the current request information is less than a preset flow control threshold, that the second operation is to take out the second request from the waiting queue and / or issue a backend IO corresponding to the second request, and execute the second operation;

[0119] The second request hanging unit is used to determine the second operation as hanging the second request into the waiting queue and execute the second operation if the current request information is greater than the preset flow control threshold.

[0120] In an optional embodiment, the device further includes:

[0121] The current request information update module is used to update the current request information based on the issued backend IO and / or the returned response backend IO;

[0122] The dirty cache line release module is also used to initiate a refresh command to refresh and release the dirty cache line if the current request information is less than a preset flow control threshold, and take out the second request from the waiting queue.

[0123] In an optional embodiment, the device further comprises:

[0124] a chain-off prohibition setting module, configured to, in response to taking out a first request from a waiting queue, set the waiting queue corresponding to the first request to be chain-off prohibition if the current dirty cache line information corresponding to the first request is greater than a dirty cache line threshold;

[0125] The link-dropping prohibition setting module is also used to respond to taking out the second request from the waiting queue, and if the current request information corresponding to the second request is greater than the preset flow control threshold, then set the waiting queue corresponding to the second request to prohibit link-dropping.

[0126] In an optional implementation, the information acquisition module 910 includes:

[0127] Get the number of dirty cache lines currently applied, the current number of dirty cache lines, and the current number of dirty cache lines to be released;

[0128] Add the number of currently applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released to obtain current dirty cache line information;

[0129] Get the current backend IO quantity and current backend IO bandwidth;

[0130] Based on the current backend IO quantity and the current backend IO bandwidth, the current request information is determined.

[0131] The device and method embodiments in the embodiments of this application are based on the same application concept.

[0132] Fig.10 The schematic diagram of the structure of the embodiment of the computer device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0133] like Fig.10 As shown, the computer device may include: a processor (processor) 1002 , a communication interface (Communications Interface) 1004 , a memory (memory) 1006 , and a communication bus 1008 .

[0134] The processor 1002, the communication interface 1004, and the memory 1006 communicate with each other through the communication bus 1008. The communication interface 1004 is used to communicate with other devices such as a client or other server network elements. The processor 1002 is used to execute the program 1010, which can specifically execute the relevant steps in the above-mentioned embodiment of the back-end flow control method.

[0135] Specifically, the program 1010 may include program code, which includes computer-executable instructions.

[0136] The processor 1002 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0137] The memory 1006 is used to store the program 1010. The memory 1006 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0138] Program 1010 can be specifically called by processor 1002 to enable the computer device to execute the relevant steps in the above-mentioned embodiment of the back-end flow control method.

[0139] It can be understood by those skilled in the art that Fig.10 The structure shown is only for illustration and does not limit the structure of the above-mentioned device. Fig.10 More or fewer components as shown, or with Fig.10 Different configurations are shown.

[0140] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on a back-end flow control device or a computer device, the back-end flow control device or the computer device executes the back-end flow control method in any of the above method embodiments.

[0141] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0142] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0143] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0144] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of protection of the present invention. The word "comprising" does not exclude the existence of elements or steps not listed in the present invention. The word "one" or "an" preceding an element does not exclude the existence of multiple such elements. The present invention can be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In the embodiments in which several devices are listed, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A backend flow control method, characterized in that: The method comprises: Obtain current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO; Compare the current dirty cache line information with the preset flow control range, determine and execute a first operation; the first operation is used to characterize the processing operation of the first request; the first request is an IO that generates a backend IO by applying for a dirty cache line after being issued; The current request information is compared with a preset flow control threshold to determine and execute a second operation; the second operation is used to characterize a processing operation on a second request; the second request is an IO that directly generates a backend IO after being issued.

2. The method according to claim 1, characterized in that The preset flow control range includes a dirty cache line threshold; comparing the current dirty cache line information with the preset flow control range, determining and executing a first operation, includes: If the current dirty cache line information is less than the dirty cache line threshold, determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and performing the first operation; If the current dirty cache line information is greater than the dirty cache line threshold, determining that the first operation is to hang the first request in the waiting queue and execute the first operation.

3. The method according to claim 2, characterized in that The preset flow control range also includes a refresh threshold; the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold; after determining that the first operation is to take the first request out of the waiting queue and / or apply for a dirty cache line for the first request, and performing the first operation, it also includes: Updating the current dirty cache line information; If the current dirty cache line information is greater than the refresh threshold, a refresh command is initiated to refresh and release the dirty cache line.

4. The method according to claim 1, characterized in that: The comparing the current request information with the preset flow control threshold to determine and execute the second operation includes: If the current request information is less than the preset flow control threshold, determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and executing the second operation; If the current request information is greater than the preset flow control threshold, determining that the second operation is to hang the second request in the waiting queue and execute the second operation.

5. The method according to claim 4, characterized in that After determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and executing the second operation, the method further includes: Based on the issued backend IO and / or the returned response backend IO, update the current request information; If the current request information is less than the preset flow control threshold, a refresh command is initiated to refresh and release the dirty cache line, and the second request is taken out from the waiting queue.

6. The method according to claim 2 or 4, characterized in that: The method further comprises: In response to taking out a first request from the waiting queue, if the current dirty cache line information corresponding to the first request is greater than a dirty cache line threshold, setting the waiting queue corresponding to the first request to prohibit unlinking; In response to taking out a second request from the waiting queue, if the current request information corresponding to the second request is greater than the preset flow control threshold, the waiting queue corresponding to the second request is set to prohibit link disconnection.

7. The method according to any one of claims 1 to 6, characterized in that: The obtaining of current dirty cache line information and current request information includes: Get the number of dirty cache lines currently applied, the current number of dirty cache lines, and the current number of dirty cache lines to be released; Add the number of currently applied dirty cache lines, the current number of dirty cache lines and the current number of dirty cache lines to be released to obtain the current dirty cache line information; Get the current backend IO quantity and current backend IO bandwidth; The current request information is determined based on the current backend IO quantity and the current backend IO bandwidth.

8. A back-end flow control device, characterized in that: The device comprises: An information acquisition module, used to acquire current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO; A first operation determination module is used to compare the current dirty cache line information with a preset flow control range, determine and execute a first operation; the first operation is used to characterize a processing operation on a first request; the first request is an IO that generates a backend IO by applying for a dirty cache line after being issued; The second operation determination module is used to compare the current request information with the preset flow control threshold to determine and execute the second operation; the second operation is used to characterize the processing operation of the second request; the second request is an IO that directly generates a backend IO after being issued.

9. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the back-end flow control method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on a backend flow control device or a computer device, the backend flow control device or the computer device executes the operation of the backend flow control method as described in any one of claims 1 to 7.

Images