Backend Flow Control Method, Device, Computer Equipment and Readable Storage Medium

Abstract

The present invention relates to the field of computer technology, and discloses a back-end flow control method, device, computer device and readable storage medium. The method includes: obtaining current dirty cache line information and current request information; comparing the current dirty cache line information with a preset flow control range to determine and execute a first operation; comparing the current request information with a preset flow control threshold to determine and execute a second operation. In this way, based on the size relationship between the usage of the current dirty cache line and the preset flow control range, the processing operation for the first request is determined, and based on the size relationship between the usage of the current back-end IO and the preset flow control threshold, the processing operation for the second request is determined, so as to comprehensively consider the usage of the current dirty cache line and the back-end IO, and balance the execution of the foreground and background IOs, thereby ensuring the system performance while performing flow control and avoiding the situation of system performance degradation and fluctuations.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technologies, and in particular, to a method and apparatus for back-end flow control, a computer device, and a readable storage medium. Background Art

[0002] As the usage time of a hard disk becomes longer, the performance of the hard disk may decline due to fragmentation problems. Therefore, flow control processing is required to prevent a disk with poor performance from slowing down a disk with good performance and causing a decrease in the overall system's IOPS (Input / Output Operations Per Second). Currently, the method for implementing flow control is usually to stop issuing back-end IOs after exceeding the flow control line and hang the back-end IOs in a linked list. After the number of back-end IOs drops below the flow control line, the linked list is removed and the IOs are delivered. This will cause the resource numbers of back-end IOs to be occupied by the linked list for a long time, resulting in other disks that are not flow-controlled being unable to generate back-end IOs due to lack of resource numbers, causing a decline in system performance. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention provide a method and apparatus for back-end flow control, a computer device, and a readable storage medium, which are used to solve the problem in the prior art that flow control of back-end IOs will cause a decline in system performance.

[0004] According to one aspect of the embodiments of the present invention, a method for back-end flow control is provided. The method includes:

[0005] Obtain current dirty cache line information and current request information; the current dirty cache line information is used to represent the usage of the current dirty cache line; the current request information is used to represent the usage of the current back-end IO.

[0006] Compare the current dirty cache line information with a preset flow control range to determine and execute a first operation; the first operation is used to represent the processing operation of a first request; the first request is an IO that generates a back-end IO by applying for a dirty cache line after being issued.

[0007] Compare the current request information with a preset flow control threshold to determine and execute a second operation; the second operation is used to represent the processing operation of a second request; the second request is an IO that directly generates a back-end IO after being issued.

[0008] In an optional implementation, the preset flow control range includes a dirty cache line threshold. Comparing the current dirty cache line information with the preset flow control range to determine and execute the first operation includes:

[0009] If the current dirty cache line information is less than the dirty cache line threshold, determine that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and execute the first operation.

[0010] If the current dirty cache line information is greater than the dirty cache line threshold, determine that the first operation is to hang the first request into the waiting queue and execute the first operation.

[0011] In an alternative embodiment, the preset flow control range further includes a refresh threshold; the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold; after determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request and execute the first operation, it further includes:

[0012] Update the current dirty cache line information;

[0013] If the current dirty cache line information is greater than the refresh threshold, initiate a refresh command to refresh and release the dirty cache line.

[0014] In an alternative embodiment, comparing the current request information with the preset flow control threshold and determining and executing the second operation includes:

[0015] If the current request information is less than the preset flow control threshold, determine that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request and execute the second operation;

[0016] If the current request information is greater than the preset flow control threshold, determine that the second operation is to hang the second request into the waiting queue and execute the second operation.

[0017] In an alternative embodiment, after determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request and execute the second operation, it further includes:

[0018] Update the current request information based on the issued backend IO and / or the backend IO that returns the response;

[0019] If the current request information is less than the preset flow control threshold, initiate a refresh command to refresh and release the dirty cache line and take out the second request from the waiting queue.

[0020] In an alternative embodiment, the method further includes:

[0021] In response to taking out the first request from the waiting queue, if the current dirty cache line information corresponding to the first request is greater than the dirty cache line threshold, set the waiting queue corresponding to the first request to be prohibited from dequeuing;

[0022] In response to taking out the second request from the waiting queue, if the current request information corresponding to the second request is greater than the preset flow control threshold, set the waiting queue corresponding to the second request to be prohibited from dequeuing.

[0023] In an alternative embodiment, obtaining current dirty cache line information and current request information includes:

[0024] Obtaining the current number of applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released;

[0025] Adding the current number of applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released to obtain the current dirty cache line information;

[0026] Obtaining the current number of backend I / Os and the current backend I / O bandwidth;

[0027] Based on the current number of backend I / Os and the current backend I / O bandwidth, determining the current request information.

[0028] According to another aspect of the embodiments of the present invention, a backend flow control device is provided, and the device includes:

[0029] An information acquisition module, configured to acquire current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend I / O;

[0030] A first operation determination module, configured to compare the current dirty cache line information with a preset flow control range, determine and execute a first operation; the first operation is used to characterize the processing operation of a first request; the first request is an I / O that generates a backend I / O by applying for a dirty cache line after being issued;

[0031] A second operation determination module, configured to compare the current request information with a preset flow control threshold, determine and execute a second operation; the second operation is used to characterize the processing operation of a second request; the second request is an I / O that directly generates a backend I / O after being issued.

[0032] According to another aspect of the embodiments of the present invention, a computer device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0033] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations of the backend flow control method as described above.

[0034] According to yet another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, and at least one executable instruction is stored in the storage medium. When the executable instruction runs on the backend flow control device or the computer device, the backend flow control device or the computer device is caused to execute the operations of the backend flow control method as described above.

[0035] The embodiments of the present invention respectively set a preset flow control range for the current usage of the dirty cache line and a preset flow control threshold for the current usage of the backend IO, determine the processing operation for the first request based on the size relationship between the current usage of the dirty cache line and the preset flow control range, and determine the processing operation for the second request based on the size relationship between the current usage of the backend IO and the preset flow control threshold, thereby comprehensively considering the usage of the current dirty cache line and the backend IO, taking into account the execution of the foreground and backend IO, thereby ensuring system performance while controlling the flow and avoiding the decline and fluctuation of system performance.

[0036] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0038] Figure 1 This is a flow chart of a back-end flow control method provided by an embodiment of the present invention;

[0039] Figure 2 1 is a schematic diagram of a preset flow control range and a preset flow control threshold in a back-end flow control method provided by an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of a process for determining and executing a first operation in a backend flow control method provided by an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of a process for releasing dirty cache lines in a back-end flow control method provided by an embodiment of the present invention;

[0042] Figure 5 1 is a flow chart of determining and executing a second operation in a back-end flow control method provided by an embodiment of the present invention;

[0043] Figure 6 This is a schematic diagram of a process for releasing dirty cache lines in a back-end flow control method provided by an embodiment of the present invention;

[0044] Figure 7 This is a timing diagram of flow control for dirty cache lines in a back-end flow control method provided by an embodiment of the present invention;

[0045] Figure 8It is a timing diagram for flow control of backend I / O in a backend flow control method provided by an embodiment of the present invention;

[0046] Figure 9 It is a schematic structural diagram of an embodiment of a backend flow control device provided by an embodiment of the present invention;

[0047] Figure 10 It is a schematic structural diagram of an embodiment of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0048] Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0049] As the hard disk is used for a longer time, the hard disk will have a problem of performance degradation due to fragmentation. Therefore, flow control processing is required to avoid a good-performance disk being slowed down by a poor-performance disk, resulting in a decrease in the IOPS (Input / Output Operations Per Second) of the entire system. Currently, the method for implementing flow control is usually to stop issuing backend I / O after exceeding the flow control line and hang the backend I / O into a linked list. When the number of backend I / O decreases below the flow control line, the linked list is removed and the I / O is delivered. This will cause the resource numbers of the backend I / O to be occupied by the linked list for a long time, resulting in other disks that are not flow-controlled being unable to generate backend I / O due to lack of resource numbers, causing a decrease in system performance.

[0050] In addition, due to the existence of observation lag, when the module or unit responsible for flow control detects that the number of backend I / O has decreased below the flow control line, the system and the disk have been idle for some time. At this time, if only one backend I / O is removed from the linked list, it will cause the system bandwidth not to be fully utilized, resulting in the inability to fully utilize system resources and a decrease in system performance. Moreover, during the flow control process, flow control is also performed on dirty cache lines. The flow control of dirty cache lines is to control the quantity of dirty cache lines for the IO applications of all disks. The total quantity of dirty cache lines cannot exceed the threshold, otherwise they will be flow-controlled and hung into the linked list; if a disk has more WBIOs (Write back IOs), it will apply for more dirty cache lines, which will lead to the risk of flow control for the WB IOs of other disks, resulting in the inability to execute the WB IOs of other disks and a decrease in system write performance. And relying only on one linked list to flow-control dirty cache lines will cause uneven flush IOs for each disk during flush, resulting in system performance fluctuations.

[0051] Based on this, an embodiment of the present invention provides a backend flow control method, which includes: obtaining current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO; comparing the current dirty cache line information with a preset flow control range to determine and execute a first operation; the first operation is used to characterize the processing operation of a first request; the first request is an IO that the host issues, writes to the cache, and then returns a response; comparing the current request information with a preset flow control threshold to determine and execute a second operation; the second operation is used to characterize the processing operation of a second request; the second request is an IO that the host issues to the disk array or an IO where the cache is flushed to the disk array. In this way, a preset flow control range is set respectively according to the usage of the current dirty cache line, and a preset flow control threshold is set according to the usage of the current backend IO. The processing operation of the first request is determined based on the size relationship between the usage of the current dirty cache line and the preset flow control range, and the processing operation of the second request is determined based on the size relationship between the usage of the current backend IO and the preset flow control threshold. Thus, the usage of the current dirty cache line and the backend IO is comprehensively considered, and the execution of the foreground and background IOs is taken into account, so as to ensure the system performance while performing flow control and avoid the decline and fluctuation of the system performance.

[0052] The following introduces a specific embodiment of a backend flow control method of the present invention. Figure 1 FIG. is a flowchart of a backend flow control method provided by an embodiment of the present invention. This specification provides method operation steps such as in the embodiment or flowchart, but based on routine or non-creative labor, there may be more or fewer operation steps. The step order listed in the embodiment is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it can be executed in the order shown in the embodiment or the flowchart or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing). Specifically, as Figure 1 shown, the method may include:

[0053] Step 110: Obtain current dirty cache line information and current request information.

[0054] In the embodiment of the present invention, dirty cache lines are requested when WB IO (Write back IO) is executed, and when a dirty cache line is flushed, it will generate a backend IO, that is, the backend IO. Also, WT IO (Write through IO) and read IO will also generate backend IO. This leads to two sources of demand for backend IO. However, the processing capacity of the disk is limited. When the number of issued backend IOs exceeds the processing capacity of the disk, the backend IO will be blocked, causing other IOs to be unable to request resource numbers and thus unable to be processed. Based on this, in order to achieve flow control of the backend IO, not only the flow control should be carried out according to the usage of the backend IO, but also the usage of dirty cache lines should be considered comprehensively.

[0055] In the embodiment of the present invention, the current dirty cache line information is used to characterize the usage of the current dirty cache line, that is, to characterize the current occupation and demand of the dirty cache line; the current request information is used to characterize the usage of the current backend IO, that is, to characterize the current occupation and demand of the backend IO, that is, of the backend IO.

[0056] In an optional embodiment, the acquisition and determination methods of the current dirty cache line information and the current request information can be as follows:

[0057] Step a1: Obtain the number of currently requested dirty cache lines, the number of current dirty cache lines, and the number of currently to-be-released dirty cache lines.

[0058] In step a1, the number of currently requested dirty cache lines wb_dirty_cl_cnt is the number of cache lines that have been requested but have not yet become dirty cache lines, that is, cache lines have been requested currently, and the data of WB IO is being transferred from the main memory to the cache line; the number of current dirty cache lines dirty_cl_cnt is the number of currently existing dirty cache lines, that is, the cache lines have actually become dirty cache lines, and the data in the cache line is inconsistent with the data on the disk; the number of currently to-be-released dirty cache lines flush_dirty_cl_cnt is the number of dirty cache lines that have been flushed but the dirty data in them has not yet been cleared, that is, a flush request has been initiated, and the data in the cache line is being written back to the disk.

[0059] Step a2: Add the number of currently requested dirty cache lines, the number of current dirty cache lines, and the number of currently to-be-released dirty cache lines to obtain the current dirty cache line information.

[0060] In step a2, add the current number of dirty cache lines that have been applied for wb_dirty_cl_cnt, the current number of dirty cache lines dirty_cl_cnt, and the current number of dirty cache lines to be released flush_dirty_cl_cnt to obtain the current dirty cache line information curt_dirty_cl_cnt; that is, as shown in the following formula:

[0061] curt_dirty_cl_cnt = dirty_cl_cnt + wb_dirty_cl_cnt + flush_dirty_cl_cnt.

[0062] Step a3: Obtain the current number of backend I / Os and the current backend I / O bandwidth.

[0063] In step a3, the current number of backend I / Os is the number of backend I / Os obtained by current statistics, and the current backend I / O bandwidth is the data volume of the backend I / Os obtained by current statistics.

[0064] Step a4: Determine the current request information based on the current number of backend I / Os and the current backend I / O bandwidth.

[0065] In step a4, the current request information is related to both the current number of backend I / Os and the current backend I / O bandwidth. That is to say, the current request information will comprehensively consider the current number of backend I / Os and the current backend I / O bandwidth to monitor and control both.

[0066] In an optional implementation, for each disk array in the system, obtain its corresponding current dirty cache line information and current request information respectively, and perform flow control according to its corresponding current dirty cache line information and current request information respectively. At the same time, for the current request information, the total link bandwidth of the system will also be additionally monitored, that is, the sum of the current backend I / O bandwidths corresponding to all disk arrays will be monitored to avoid the total system bandwidth exceeding the performance range.

[0067] Step 120: Compare the current dirty cache line information with the preset flow control range, and determine and execute the first operation.

[0068] In an embodiment of the present invention, the preset flow control range is used to define the flow control range of the current dirty cache line. The current dirty cache line information is compared with the preset flow control range, and a first operation is determined and executed according to the size relationship between the current dirty cache line information and the preset flow control range. The first operation is used to represent the processing operation of the first request, and the first request is an IO that generates backend IO by applying for a dirty cache line after being issued. When processing the first request, it is necessary to first apply for a dirty cache line, and then flush the dirty cache line to generate backend IO, so as to transfer the data in the first request from the main memory to the dirty cache line first, and then write it from the dirty cache line to the disk. Therefore, in order to implement the flow control of the dirty cache line and the backend IO, when processing the first request, it is necessary to determine whether the dirty cache line is under flow control, and then determine the processing operation of the first request, that is, the first operation.

[0069] In an embodiment of the present invention, the first request is an IO that is issued by the host and returns a response after being written to the cache, that is, a WB IO.

[0070] Step 130: Compare the current request information with the preset flow control threshold, and determine and execute a second operation.

[0071] In an embodiment of the present invention, the preset flow control threshold is used to define the flow control threshold of the current request. The current request information is compared with the preset flow control threshold, and a second operation is determined and executed according to the size relationship between the current request information and the preset flow control threshold. The second operation is used to represent the processing operation of the second request, and the second request is an IO that directly generates backend IO after being issued. When processing the second request, it can directly generate backend IO, and write the data in the second request from the main memory and the cache to the disk. Therefore, in order to implement the flow control of the backend IO, when processing the second request, it is necessary to determine whether the backend IO is under flow control, and then determine the processing operation of the second request, that is, the second operation.

[0072] In an embodiment of the present invention, the second request may include an IO that is issued by the host, written to the cache, then written to the disk array, and then returns a response, that is, a WT IO, and a read type IO, that is, a read IO.

[0073] In an alternative embodiment, for each disk, the flow control of the dirty cache line and the backend IO is performed separately. Correspondingly, the preset flow control ranges for flow control of the dirty cache line and the preset flow control thresholds for flow control of the backend IO of different disks may be different.

[0074] In an alternative embodiment, Figure 2 is a schematic diagram of the preset flow control range and the preset flow control threshold in a backend flow control method provided by an embodiment of the present invention, as Figure 2As shown, the preset flow control range includes a dirty cache line threshold and a flush threshold. Among them, the number of dirty cache lines corresponding to the flush threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold. The dirty cache line threshold is used to represent the threshold for flow control of the application for dirty cache lines. If the current dirty cache line information is greater than the dirty cache line threshold, flow control is performed on the application for dirty cache lines, and no further application for dirty cache lines is allowed. The flush threshold is used to represent the threshold for flow control of the flush of dirty cache lines. If the current dirty cache line information is greater than the flush threshold, a preset number of dirty cache lines are flushed to control the number of dirty cache lines below the number of dirty cache lines corresponding to the flush threshold.

[0075] In an alternative embodiment, Figure 3 is a schematic flowchart of determining and executing the first operation in a backend flow control method provided by an embodiment of the present invention. The above step 120: comparing the current dirty cache line information with the preset flow control range to determine and execute the first operation may include the following steps:

[0076] Step 310: If the current dirty cache line information is less than the dirty cache line threshold, determine that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and execute the first operation.

[0077] In the embodiment of the present invention, as Figure 2 shown, if the current dirty cache line information is less than the dirty cache line threshold, it indicates that the flow control of the dirty cache line is released at this time, and a dirty cache line can be applied for at this time, that is, the first request can be processed. Correspondingly, determine that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and execute the first operation.

[0078] In the embodiment of the present invention, if there is a first request in the waiting queue, take out the first request from the waiting queue and apply for a dirty cache line for the first request to respond to and process the first request. That is, the first operation at this time is to take out the first request from the waiting queue and apply for a dirty cache line for the first request; if there is no first request in the waiting queue and a first request is received at this time, a dirty cache line can be directly applied for the received first request to respond to and process the first request. That is, the first operation at this time is to apply for a dirty cache line for the first request.

[0079] Step 320: If the current dirty cache line information is greater than the dirty cache line threshold, determine that the first operation is to hang the first request into the waiting queue, and execute the first operation.

[0080] In the embodiment of the present invention, as Figure 2As shown, if the current dirty cache line information is greater than the dirty cache line threshold, it indicates that the dirty cache line is under flow control at this time. At this time, a dirty cache line cannot be applied for, that is, the first request cannot be responded to and processed. Correspondingly, it is determined that the first operation is to hang the first request into the waiting queue and execute the first operation.

[0081] In an alternative embodiment, the waiting queue can be a linked list, namely the wait IO linked list. Then, in step 310 above, if the wait IO linked list is empty and the first request is received at this time, the first operation is to apply for a dirty cache line for the first request. If the wait IO linked list is not empty, the unlink operation is executed. At this time, the first operation is to take out the first request from the waiting queue, that is, the wait IO linked list, and apply for a dirty cache line for the first request. In step 320 above, the first operation is to hang the first request into the waiting queue, that is, the wait IO linked list.

[0082] In an alternative embodiment, the above steps 310 - 320 are related steps for responding to the first request according to the dirty cache line threshold and applying for a dirty cache line, which will cause the number of dirty cache lines to increase. Correspondingly, the method further includes related steps of flushing and releasing dirty cache lines to reduce the number of dirty cache lines. Figure 4 It is a schematic flowchart of releasing dirty cache lines in a backend flow control method provided by an embodiment of the present invention. As Figure 4 shown, after step 320: determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request and execute the first operation, it further includes related steps of flushing the dirty cache line according to the flush threshold, specifically as follows:

[0083] Step 410: Update the current dirty cache line information.

[0084] In an embodiment of the present invention, after applying for a dirty cache line for the first request, the number of dirty cache lines will increase. Therefore, based on the number of applied dirty cache lines, the current dirty cache line information is updated.

[0085] Step 420: If the current dirty cache line information is greater than the flush threshold, initiate a flush command to flush and release the dirty cache line.

[0086] In an embodiment of the present invention, as Figure 2 shown, if the current dirty cache line information is greater than the flush threshold, it indicates that the number of dirty cache lines has exceeded the flush threshold at this time, and a certain number of dirty cache lines need to be flushed to keep the number of dirty cache lines below or near the flush threshold. At this time, a flush command is initiated to flush the dirty cache line, that is, to flush and release the dirty cache line.

[0087] In an alternative embodiment, after flushing the dirty cache line and releasing the dirty cache line, the number of dirty cache lines will decrease. Therefore, based on the number of released dirty cache lines, the current dirty cache line information is updated again.

[0088] In an alternative embodiment, a preset number of dirty cache lines can be fetched and flushed each time. If the current dirty cache line information is still greater than the refresh threshold after flushing a batch of dirty cache lines, continue to fetch and flush the preset number of dirty cache lines until the current dirty cache line information is less than the refresh threshold, so as to maintain the number of dirty cache lines below or near the refresh threshold.

[0089] In an alternative embodiment, Figure 5 is a schematic flowchart of determining and executing the second operation in a backend flow control method provided by an embodiment of the present invention. The above step 130: comparing the current request information with a preset flow control threshold to determine and execute the second operation may include the following steps:

[0090] Step 510: If the current request information is less than the preset flow control threshold, determine that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and execute the second operation.

[0091] In the embodiment of the present invention, as Figure 2 shown, if the current request information is less than the preset flow control threshold, it indicates that the flow control of the backend IO is released at this time. At this time, the backend IO can be issued, that is, the second request can be processed. Correspondingly, determine that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and execute the second operation.

[0092] In the embodiment of the present invention, if there is a second request in the waiting queue, take out the second request from the waiting queue and issue the backend IO corresponding to the second request to respond to and process the second request. That is, the second operation at this time is to take out the second request from the waiting queue and issue the backend IO corresponding to the second request; if there is no second request in the waiting queue and a second request is received at this time, the backend IO corresponding to the received second request can be directly issued to respond to and process the second request. That is, the second operation at this time is to issue the backend IO corresponding to the second request.

[0093] Step 520: If the current request information is greater than the preset flow control threshold, determine that the second operation is to hang the second request into the waiting queue and execute the second operation.

[0094] In the embodiment of the present invention, as Figure 2As shown, if the current request information is greater than the preset flow control threshold, it indicates that the backend IO is currently under flow control. At this time, the backend IO cannot be issued, that is, the second request cannot be responded to and processed. Correspondingly, it is determined that the second operation is to hang the second request into the waiting queue and execute the second operation.

[0095] In an alternative embodiment, the waiting queue can be a linked list, namely the wait IO linked list. Then, in step 510 above, if the wait IO linked list is empty and the second request is received at this time, the second operation is to issue the backend IO corresponding to the second request. If the wait IO linked list is not empty, the unlink operation is executed. At this time, the second operation is to take out the second request from the waiting queue, that is, the wait IO linked list, and issue the backend IO corresponding to the second request. In step 520 above, the second operation is to hang the second request into the waiting queue, that is, the wait IO linked list.

[0096] In an alternative embodiment, since flushing the dirty cache line also generates backend IO, and only after the generated backend IO is processed can the clean operation of the dirty cache line be completed, thereby releasing the dirty cache line. And considering that when the current dirty cache line information is greater than the refresh threshold, the current request information often is also greater than the preset flow control threshold. At this time, since the backend IO cannot be issued and processed, the dirty cache line cannot be flushed normally. Therefore, this method further includes the relevant steps of flushing the dirty cache line to release the dirty cache line. Figure 6 It is a schematic flow diagram of releasing the dirty cache line in a backend flow control method provided by an embodiment of the present invention, as Figure 6 shown. After step 610: determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request and execute the second operation, it further includes the relevant steps of flushing the dirty cache line according to the preset flow control threshold, specifically as follows:

[0097] Step 610: Update the current request information based on the issued backend IO and / or the backend IO that returns a response.

[0098] In the embodiment of the present invention, after the backend IO is issued, the number of backend IOs and the bandwidth corresponding to the backend IO will increase. Therefore, the current request information is updated based on the issued backend IO. After the backend IO is processed and a response is returned, the number of backend IOs will decrease, and at the same time, the originally occupied bandwidth will be released. Therefore, the current request information is updated based on the backend IO that returns a response.

[0099] Step 620: If the current request information is less than the preset flow control threshold, initiate a refresh command to refresh and release the dirty cache line, and take out the second request from the waiting queue.

[0100] In an embodiment of the present invention, as Figure 2 shown, if the current request information is less than the preset flow control threshold, a certain number of dirty cache lines are released at this time to prevent the situation where when the current dirty cache line information is greater than the refresh threshold, the current request information is also greater than the preset flow control threshold, resulting in the inability to issue and process the backend IO and the inability to normally flush the dirty cache lines. Therefore, a refresh command is initiated when the current request information is less than the preset flow control threshold to flush the dirty cache lines, that is, to refresh and release the dirty cache lines. At the same time, the second request is taken out from the waiting queue, that is, de-linked from the wait IO linked list corresponding to the second request.

[0101] In an alternative embodiment, a preset number of dirty cache lines can be obtained and flushed each time. If the number of backend IOs generated after the refresh command is issued causes the current request information to be greater than the preset flow control threshold, the flush of the dirty cache lines is stopped until the backend IO response returns, the number of backend IOs decreases, and the current request information is less than the preset flow control threshold, and then the flush of the dirty cache lines is triggered again.

[0102] In an alternative embodiment, if the current request information is less than the preset flow control threshold and the system is flushing other disks and has not yet scheduled the flush of this disk when flushing the dirty cache lines, the flush is reserved until the dirty cache lines corresponding to the flush are released.

[0103] In an alternative embodiment, Figure 7 is a timing diagram for flow control of dirty cache lines in a backend flow control method provided by an embodiment of the present invention. The following further describes Figure 7 the flow control of dirty cache lines. As Figure 7 shown, at time T0, the flow control of the backend IO is released, and the number of dirty cache lines released at this time is n (unit: CL cnt); at time T1, the flow control of the backend IO is released, and at time T2, the flow control of the backend IO is effective. During this process, since the release amount of the dirty cache lines has not been consumed, it remains n all the time; at time T3, the flush is started, and n dirty cache lines are de-linked from the linked list for flushing, and all the reserved dirty cache lines are released; at time T4, the flow control of the backend IO is effective, resulting in the failure of the flush at time T5; at time T6, the flow control of the backend IO is released, and the foreground wait IO queue, that is, the waiting queue corresponding to the first request, is empty. In order to increase the flush amount of the dirty cache lines when the foreground IO is idle, the release amount of the dirty cache lines is increased to n + delta, where the specific value of delta can be configured according to actual requirements.

[0104] In an alternative embodiment, Figure 8It is a timing diagram for flow control of backend I / O in a backend flow control method provided by an embodiment of the present invention. The following further describes the flow control of backend I / O in conjunction with Figure 8 the flow control of backend I / O. As Figure 8 shown, at time T0, the flow control of backend I / O is lifted. At this time, the second requests waiting to be processed, that is, the release volume of WT / read wait I / O is m (unit: IOcnt), that is, the release volume of the waiting queue or linked list corresponding to the second requests is m. That is to say, at this time, m second requests can be removed from the linked list corresponding to the second requests for execution; at time T1, m WT / read I / Os are removed from the linked list for execution. At this time, the release volume of WT / read wait I / O becomes 0; at time T2, the BIO flow control is lifted, but the number of dirty cache lines is 0 at this time. In order to process more foreground wait I / Os when the number of dirty cache lines is 0, the release volume of WT / read wait I / O is increased to m + delta, where the specific value of delta can be configured according to actual requirements; at time T3, m + delta WT / read I / Os are removed from the linked list for execution, and the release volume of WT / read wait I / O is 0.

[0105] In an alternative embodiment, in order to avoid the flow control failure caused by repeated unlinking, when the first request and the second request are taken out from the waiting queue and corresponding operations are performed, the following steps are further included: in response to taking out the first request from the waiting queue, if the current dirty cache line information corresponding to the first request is greater than the dirty cache line threshold, the waiting queue corresponding to the first request is set to prohibit unlinking; in response to taking out the second request from the waiting queue, if the current request information corresponding to the second request is greater than the preset flow control threshold, the waiting queue corresponding to the second request is set to prohibit unlinking. Thus, repeated unlinking is avoided when the taken-out first request and second request are not processed. Among them, the first request and the second request correspond to different waiting queues, that is, the first request and the second request are hung into different linked lists when hanging the chain.

[0106] The back-end flow control method provided by the embodiment of the present invention includes: obtaining current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current back-end IO; comparing the current dirty cache line information with a preset flow control range to determine and execute a first operation; the first operation is used to characterize the processing operation of a first request; the first request is an IO that is sent by the host, returns a response after being written to the cache; comparing the current request information with a preset flow control threshold to determine and execute a second operation; the second operation is used to characterize the processing operation of a second request; the second request is an IO that is sent by the host, written to the cache and then written to the disk array and then returns a response. In this way, a preset flow control range is set respectively for the usage of the current dirty cache line, a preset flow control threshold is set for the usage of the current back-end IO, the processing operation of the first request is determined based on the size relationship between the usage of the current dirty cache line and the preset flow control range, and the processing operation of the second request is determined based on the size relationship between the usage of the current back-end IO and the preset flow control threshold, so as to comprehensively consider the usage of the current dirty cache line and the back-end IO, take into account the execution of the foreground and background IOs, and thus ensure the system performance while performing flow control, and avoid the situation of system performance degradation and fluctuation.

[0107] The embodiment of the present invention also provides a back-end flow control device Figure 9 is a schematic structural diagram of the embodiment of the back-end flow control device provided by the embodiment of the present invention, as Figure 9 shown, the device 900 includes:

[0108] An information acquisition module 910, configured to acquire current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current back-end IO;

[0109] A first operation determination module 920, configured to compare the current dirty cache line information with a preset flow control range to determine and execute a first operation; the first operation is used to characterize the processing operation of a first request; the first request is an IO that generates a back-end IO by applying for a dirty cache line after being sent;

[0110] A second operation determination module 930, configured to compare the current request information with a preset flow control threshold to determine and execute a second operation; the second operation is used to characterize the processing operation of a second request; the second request is an IO that directly generates a back-end IO after being sent.

[0111] In an optional implementation manner, the preset flow control range includes a dirty cache line threshold; the first operation determination module 910 includes:

[0112] A first request execution unit, configured to determine that the first operation is to take out a first request from a waiting queue and / or apply for a dirty cache line for the first request if the current dirty cache line information is less than a dirty cache line threshold, and execute the first operation;

[0113] A first request enqueueing unit, configured to determine that the first operation is to enqueue the first request into a waiting queue if the current dirty cache line information is greater than the dirty cache line threshold, and execute the first operation.

[0114] In an optional implementation, the preset flow control range further includes a refresh threshold; the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold; the apparatus further includes:

[0115] A current dirty cache line information updating module, configured to update the current dirty cache line information;

[0116] A dirty cache line releasing module, configured to initiate a refresh command to refresh and release the dirty cache line if the current dirty cache line information is greater than the refresh threshold.

[0117] In an optional implementation, the second operation determination module 920 includes:

[0118] A second request execution unit, configured to determine that the second operation is to take out a second request from a waiting queue and / or issue a backend IO corresponding to the second request if the current request information is less than a preset flow control threshold, and execute the second operation;

[0119] A second request enqueueing unit, configured to determine that the second operation is to enqueue the second request into a waiting queue if the current request information is greater than the preset flow control threshold, and execute the second operation.

[0120] In an optional implementation, the apparatus further includes:

[0121] A current request information updating module, configured to update the current request information based on the issued backend IO and / or the backend IO that returns a response;

[0122] The dirty cache line releasing module is further configured to initiate a refresh command to refresh and release the dirty cache line and take out the second request from the waiting queue if the current request information is less than the preset flow control threshold.

[0123] In an optional implementation, the apparatus further includes:

[0124] A dequeue prohibition setting module, configured to, in response to taking out the first request from the waiting queue, set the waiting queue corresponding to the first request to be prohibited from dequeueing if the current dirty cache line information corresponding to the first request is greater than the dirty cache line threshold;

[0125] The anti-chain-removal setting module is further configured to, in response to taking out the second request from the waiting queue, if the current request information corresponding to the second request is greater than the preset flow control threshold, set the waiting queue corresponding to the second request as anti-chain-removal.

[0126] In an alternative embodiment, the information acquisition module 910 includes:

[0127] Obtain the current number of applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released;

[0128] Add the current number of applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released to obtain the current dirty cache line information;

[0129] Obtain the current number of backend I / Os and the current backend I / O bandwidth;

[0130] Determine the current request information based on the current number of backend I / Os and the current backend I / O bandwidth.

[0131] The device and method embodiments in this application are based on the same application concept.

[0132] Figure 10 The structural schematic diagram of the computer device provided by the embodiment of the present invention is shown. The specific implementation of the computer device is not limited in the specific embodiments of the present invention.

[0133] As Figure 10 shown, the computer device may include: a processor 1002, a communication interface 1004, a memory 1006, and a communication bus 1008.

[0134] Among them: The processor 1002, the communication interface 1004, and the memory 1006 communicate with each other through the communication bus 1008. The communication interface 1004 is used to communicate with network elements of other devices such as clients or other servers. The processor 1002 is used to execute the program 1010, and specifically can execute the relevant steps in the above method embodiment for backend flow control.

[0135] Specifically, the program 1010 may include program code, and the program code includes computer-executable instructions.

[0136] The processor 1002 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computer device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0137] A memory 1006 for storing a program 1010. The memory 1006 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0138] Specifically, the program 1010 can be called by the processor 1002 to cause the computer device to execute the relevant steps in the above embodiments of the back-end flow control method.

[0139] Those of ordinary skill in the art can understand that Figure 10 The structure shown is only illustrative and does not limit the structure of the above device. For example, the computer device may further include more or fewer components than those shown Figure 10 in the figure, or may have a different configuration from that shown Figure 10 in the figure.

[0140] Embodiments of the present invention provide a computer-readable storage medium storing at least one executable instruction, which, when running on a back-end flow control device or a computer device, causes the back-end flow control device or the computer device to execute the back-end flow control method in any of the above method embodiments.

[0141] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. In addition, embodiments of the present invention are not directed to any particular programming language.

[0142] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that embodiments of the present invention may be practiced without these specific details. Similarly, in order to streamline the present invention and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0143] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive.

[0144] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the protection scope of the present invention. The word "comprising" does not exclude the existence of elements or steps not listed in the present invention. The word "a" or "an" preceding an element does not exclude the existence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a properly programmed computer. In an embodiment listing several devices, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the execution order.

Claims

1. A back-end flow control method, characterized in that, The method includes: Obtaining current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of the current dirty cache line; the current request information is used to characterize the usage of the current backend IO; Comparing the current dirty cache line information with a preset flow control range to determine and execute a first operation; the first operation is used to characterize the processing operation of a first request; the first request is an IO that generates a backend IO after being issued by applying for a dirty cache line; the preset flow control range includes a dirty cache line threshold; comparing the current dirty cache line information with the preset flow control range to determine and execute the first operation includes: if the current dirty cache line information is less than the dirty cache line threshold, determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and executing the first operation; if the current dirty cache line information is greater than the dirty cache line threshold, determining that the first operation is to hang the first request into the waiting queue, and executing the first operation; Comparing the current request information with a preset flow control threshold to determine and execute a second operation; the second operation is used to characterize the processing operation of a second request; the second request is an IO that directly generates a backend IO after being issued; comparing the current request information with the preset flow control threshold to determine and execute the second operation includes: if the current request information is less than the preset flow control threshold, determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request, and executing the second operation; if the current request information is greater than the preset flow control threshold, determining that the second operation is to hang the second request into the waiting queue, and executing the second operation.

2. The method according to claim 1, wherein The preset flow control range further includes a refresh threshold; the number of dirty cache lines corresponding to the refresh threshold is less than the number of dirty cache lines corresponding to the dirty cache line threshold; after determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request and executing the first operation, it further includes: Updating the current dirty cache line information; If the current dirty cache line information is greater than the refresh threshold, initiating a refresh command to refresh and release the dirty cache line.

3. The method according to claim 1, characterized in that After determining that the second operation is to take out the second request from the waiting queue and / or issue the backend IO corresponding to the second request and executing the second operation, it further includes: Updating the current request information based on the issued backend IO and / or the backend IO that returns a response; If the current request information is less than the preset flow control threshold, initiating a refresh command to refresh and release the dirty cache line, and taking out the second request from the waiting queue.

4. The method according to claim 1, characterized in that, The method further includes: In response to taking out the first request from the waiting queue, if the current dirty cache line information corresponding to the first request is greater than the dirty cache line threshold, setting the waiting queue corresponding to the first request to be prohibited from dequeueing. In response to taking out the second request from the waiting queue, if the current request information corresponding to the second request is greater than the preset flow control threshold, the waiting queue corresponding to the second request is set to be prohibited from dequeuing.

5. The method according to any one of claims 1-4, characterized in that, The obtaining of the current dirty cache line information and the current request information includes: Obtaining the current number of applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released; Adding the current number of applied dirty cache lines, the current number of dirty cache lines, and the current number of dirty cache lines to be released to obtain the current dirty cache line information; Obtaining the current number of backend I / Os and the current backend I / O bandwidth; Based on the current number of backend I / Os and the current backend I / O bandwidth, determining the current request information.

6. A back-end flow control device, characterized in that, The apparatus includes: An information acquisition module, configured to acquire current dirty cache line information and current request information; the current dirty cache line information is used to characterize the usage of current dirty cache lines; the current request information is used to characterize the usage of current backend I / Os; A first operation determination module, configured to compare the current dirty cache line information with a preset flow control range, determine and execute a first operation; the first operation is used to characterize the processing operation of a first request; the first request is an I / O that generates a backend I / O by applying for a dirty cache line after being issued; the preset flow control range includes a dirty cache line threshold; the comparing the current dirty cache line information with the preset flow control range, determining and executing the first operation includes: if the current dirty cache line information is less than the dirty cache line threshold, determining that the first operation is to take out the first request from the waiting queue and / or apply for a dirty cache line for the first request, and executing the first operation; if the current dirty cache line information is greater than the dirty cache line threshold, determining that the first operation is to enqueue the first request into the waiting queue, and executing the first operation; A second operation determination module, configured to compare the current request information with a preset flow control threshold, determine and execute a second operation; the second operation is used to characterize the processing operation of a second request; the second request is an I / O that directly generates a backend I / O after being issued; the comparing the current request information with the preset flow control threshold, determining and executing the second operation includes: if the current request information is less than the preset flow control threshold, determining that the second operation is to take out the second request from the waiting queue and / or issue the backend I / O corresponding to the second request, and executing the second operation; if the current request information is greater than the preset flow control threshold, determining that the second operation is to enqueue the second request into the waiting queue, and executing the second operation.

7. A computer device, characterized in that, It includes: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations of the backend flow control method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, At least one executable instruction is stored in the storage medium. When the executable instruction runs on the back-end flow control device or computer device, the back-end flow control device or computer device is caused to perform the operations of the back-end flow control method described in any one of claims 1-5.

Images