Cache tail delay optimization method based on NVMe solid state disk characteristics

Abstract

The invention discloses a cache tail delay optimization method based on NVMe solid state disk characteristics, and aims to solve the problem of cache long tail delay caused by a high fan-out request in a cache system. The method comprises a differential request processing strategy: based on weighted polling arbitration mechanism characteristics of the NVMe solid state disk, determining the number of sub-requests contained in different priorities according to fan-out sizes of requests, and reducing processing time differences among the sub-requests while giving higher priorities to high fan-out requests; according to the sub-request sorting algorithm, the influence of the sub-requests on the tail delay is evaluated according to the characteristics of the sub-requests, and sorting is performed according to the sub-requests, so that the priorities of the sub-requests are determined; according to the priority queue load balancing strategy, loads between NVMe priority queues are balanced, and queue congestion is avoided; and the feedback adjustment mechanism is used for monitoring and analyzing queue load and request delay to perform dynamic adjustment. Compared with an existing cache based on a solid state disk, the method has the advantage that the tail delay of the cache is optimized by fully utilizing the characteristics of the NVMe solid state disk.

Description

Technical Field

[0001] The present invention relates to the field of cache systems, and in particular to a cache tail delay optimization method, and in particular to a cache tail delay optimization method based on the characteristics of NVMe solid-state drives. Background Art

[0002] In the data-driven era, the scale and complexity of Internet data have increased significantly, and the demand for data storage and access has also continued to increase. The growth in data volume has caused traditional hard disk drives (HDDs) to become performance bottlenecks. Therefore, solid-state drives (SSDs) with excellent random read and write performance have been widely used, and SSD-based cache designs have also emerged. Compared with traditional memory caches, SSD-based caches provide larger cache capacity while ensuring reliability.

[0003] Modern Web service systems and cloud data storage systems use multi-layer architectures to manage increasingly complex Internet data, which leads to the emergence of high fan-out requests. High fan-out requests refer to complex client requests involving multiple internal services, which are usually decomposed into multiple sub-requests for parallel processing. Multi-layer systems usually absorb some requests by setting up caches, but high fan-out requests often cause long-tail latency problems in the cache, which in turn affects cache performance and system service quality. These requests must wait for all sub-requests to complete before returning a response, but there are significant differences in the processing time of sub-requests. On the one hand, the different characteristics of sub-requests (such as the amount of data involved and the popularity of the data involved) will make the processing time required for sub-requests difficult to predict; on the other hand, the existing SSD-based cache lacks an effective response strategy and can only process sub-requests at the same speed, which cannot guarantee the timely processing of high-priority sub-requests. The above two reasons will cause some sub-requests to take too long to process, thereby increasing the overall latency and causing long-tail latency problems.

[0004] SSDs based on the Non-Volatile Memory Express (NVMe) specification achieve higher throughput and lower latency with their multi-queue parallel mechanism. During parallel processing, NVMe SSDs use a multi-queue arbitration mechanism to alleviate resource contention issues. Among these arbitration mechanisms, the weighted round-robin (WRR) arbitration mechanism allows users to divide parallel queues into three priority levels: high, medium, and low, based on load characteristics, and allocate resources to these queues based on the set queue weights, thereby achieving differentiated request processing speeds. The multi-queue parallel and WRR features of NVMe SSDs provide an important technical foundation for solving the long-tail latency problem caused by high fan-out requests. Summary of the invention

[0005] In view of the defects of the prior art, the object of the present invention is to provide a cache tail delay optimization method based on the characteristics of NVMe solid state drives.

[0006] In order to achieve the above technical purpose, the technical solution adopted by the present invention is:

[0007] A cache tail latency optimization method based on NVMe solid-state drive characteristics, comprising:

[0008] Differentiated request processing strategy: Three priority levels are set based on the weighted polling arbitration mechanism of the NVMe SSD. The number of sub-requests in different priorities is determined according to the fan-out size of the client request. The sub-requests with high fan-out requests are given higher priority while reducing the processing time difference between sub-requests.

[0009] Subrequest sorting algorithm: evaluates the impact of subrequests on latency based on their characteristics, and sorts them according to the impact to determine the priority of subrequests, ensuring that all subrequests are processed in a timely manner;

[0010] Priority queue load balancing strategy: balances the load between NVMe priority queues by scheduling sub-requests in the overload queue to avoid congestion in the overload queue that affects the tail delay of the cache;

[0011] Feedback adjustment mechanism: monitors the load and request latency of the priority queue, analyzes high fan-out requests with higher latency, and dynamically adjusts the cache to ensure low tail latency.

[0012] Preferably, the differentiated request processing strategy includes the following components:

[0013] Component A1: Based on the weighted round-robin (WRR) mechanism characteristics of NVMe solid-state drives (SSDs), the NVMe queues are divided into three categories: high-priority queues, medium-priority queues, and low-priority queues, and the corresponding queue weights W are set. H , W M , W L , the queue weight satisfies W L <W M <W H At the same time, a high, medium, or low priority level is set for each sub-request, so that the sub-request enters a queue that matches its own priority level. Then, based on the queue weight, the NVMe controller takes out the same number of sub-requests as the weight from the priority queue in a round-robin manner for processing, resulting in differences in the processing speed of the requests.

[0014] Component A2: Record the fan-out size f requested by the client, and then compare it with W H , W M , W L To ensure the processing speed of high fan-out requests, the larger the fan-out of a request, the higher the priority of its sub-requests. LWhen W L <f≤W M When W M <f≤W H , the priority of all subrequests is set to high priority.

[0015] Component A3: Record the fan-out size f requested by the client, and then compare it with W H , W H +W M To avoid starvation and optimize the parallel processing of subrequests, the priority of some subrequests with high fan-out requests is appropriately reduced. H <f≤W H +W M When f>W, the priority of the subrequest includes high priority and medium priority; when f>W H +W M When , the priority of the sub-request includes high, medium and low priorities. The number of sub-requests with different priorities is set repeatedly by W H sub-requests are high priority, W M sub-requests are of medium priority, W L The sub-requests are determined as low priority until the priority assignment is completed.

[0016] Preferably, the subrequest sorting algorithm evaluates the impact of the subrequest on the cache tail latency from two dimensions: the impact of the subrequest on the latency of the current high fan-out request and the impact on the latency of other high fan-out requests, specifically including:

[0017] Step B1: Read and record all sub-requests contained in a high fan-out request.

[0018] Step B2: Obtain the data volume characteristics involved in the sub-requests, and evaluate the impact of the sub-requests on the latency of the current high fan-out request based on the characteristics, thereby completing the preliminary sorting. The larger the data volume involved in the sub-request, the higher the ranking.

[0019] Step B3: If the two sub-requests involve the same amount of data, obtain the popularity characteristics of the data involved in the sub-requests, evaluate the impact of the sub-requests on the delay of other high fan-out requests based on the characteristics, and complete the subsequent sorting. The higher the popularity of the data involved in the sub-requests, the higher the ranking.

[0020] Step B4: Determine the exact priority of each sub-request based on the sorting result and the number of sub-requests with different priorities. The higher the sub-request is in the sorting result, the higher its priority is.

[0021] Preferably, the priority queue load balancing strategy comprises the following steps:

[0022] Step C1: Obtain and record the load of each priority queue and the number of input / output operations per second (IOPS).

[0023] Step C2: Check the load of the queue to determine whether any queue is overloaded.

[0024] Step C3: If any queue is overloaded, further scheduling operations are performed on the sub-requests to balance the load. The overloaded queue is set as the source queue for scheduling, and other priority queues are set as the target queues for scheduling.

[0025] Step C4: Calculate the proportion of sub-requests to be scheduled IOPS source With W source Represents the number of input / output operations per second and queue weight of the dispatch source queue; IOPS destination With W destination Represents the number of input / output operations per second and the queue weight of the scheduling target queue respectively.

[0026] Step C5: When subsequent high fan-out requests arrive, sub-requests with a P ratio are scheduled from the overloaded queue to other queues by modifying the sub-request priority. The specific number of sub-requests scheduled is Where N represents the number of sub-requests that the high fan-out request was originally scheduled to enter the overload queue.

[0027] Step C6: Based on the latest number of sub-requests in each priority level obtained in C5 and the sorting result of B4, the priorities of the sub-requests are reset and then sent to each priority queue for processing.

[0028] Preferably, the feedback regulation mechanism includes the following components:

[0029] Component D1: Periodically collect and transmit the load information of the priority queues. At the end of each cycle, record the total number of sub-requests processed by each queue during the cycle. Record the ratio of the total number of sub-requests to the time window size corresponding to the cycle as the queue's IOPS. Record the ratio of each queue's IOPS to the queue weight as the queue's load ratio. If a queue's load ratio is much higher than the other two priority queues, the queue is overloaded.

[0030] Component D2: Monitor request latency. Monitor the overall latency of each high fan-out request as it completes and compare it with the 99th percentile (P99) latency of the past cycle. If the two are similar, it is determined that the latency of the current high fan-out request is abnormal, and the cache needs to be adjusted to reduce the completion latency of subsequent high fan-out requests.

[0031] Component D3: When there is an abnormal delay in a high fan-out request, identify the sub-request that causes the high delay, analyze the priority queue to which it belongs and the cause of the delay, and make dynamic adjustments. Specific adjustment measures include: when the overload is caused by the excessive load of the priority queue, pass the relevant information to C1 in advance for optimization; if it is due to improper priority setting, increase the priority of some sub-requests during the priority allocation process, and reduce the number of sub-requests allocated to the corresponding queue during the load balancing process. If the tail delay still does not reach the expected target after multiple adjustments, the weight of the priority queue will be further modified to reduce the overall cache tail delay.

[0032] A cache tail latency optimization method based on NVMe solid-state drive characteristics provided by the present invention includes:

[0033] Differentiated request processing strategy: Determine the number of sub-requests contained in different priorities based on the weighted polling arbitration mechanism of the NVMe SSD and the fan-out size of the client request.

[0034] Subrequest sorting algorithm: Based on the characteristics of subrequests, their impact on latency is evaluated and sorted according to the impact to determine the priority of subrequests.

[0035] Priority queue load balancing strategy: Load balancing between NVMe priority queues is achieved by scheduling sub-requests in the overload queue.

[0036] Feedback adjustment mechanism: monitors the load of the priority queue and request latency, analyzes requests with higher latency, and dynamically adjusts the cache to maintain low tail latency.

[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0038] 1. Make full use of the characteristics of NVMe solid-state drives: The present invention makes full use of the multi-queue parallel characteristics and WRR characteristics of NVMe SSD, making up for the defect that the existing SSD-based cache design cannot realize differentiated request processing.

[0039] 2. Low overhead: The sub-request features selected by the present invention can be directly obtained when parsing the sub-request, and no additional operation or calculation is required when evaluating the impact of the sub-request on the tail delay.

[0040] 3. Strong parallel processing capability. The priority queue load balancing strategy proposed in the present invention balances the load within the NVMe priority queue, enhances the parallel processing capability of requests within the cache, and ensures efficient operation of the cache.

[0041] 4. Effectively reduce tail latency. The present invention reduces the processing time difference between sub-requests and dynamically adjusts the cache through the proposed feedback adjustment mechanism, so that it can still maintain low tail latency when facing complex high fan-out requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall framework diagram of the present invention;

[0043] Figure 2 A schematic diagram of a differentiated request processing strategy according to a specific embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a sub-request sorting algorithm flow chart of a specific embodiment of the present invention;

[0045] Figure 4 A schematic diagram of a priority queue load balancing strategy flow chart of a specific embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of dynamic adjustment of feedback regulation according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0048] According to a cache tail delay optimization method based on NVMe solid state drive characteristics provided by the present invention, Figure 1 As shown, including:

[0049] Differentiated request processing strategy: Three priority levels are set based on the weighted polling arbitration mechanism of the NVMe SSD. The number of sub-requests in different priorities is determined according to the fan-out size of the client request. The sub-requests with high fan-out requests are given higher priority while reducing the processing time difference between sub-requests.

[0050] Subrequest sorting algorithm: evaluates the impact of subrequests on latency based on their characteristics, and sorts them according to the impact to determine the priority of subrequests, ensuring that all subrequests are processed in a timely manner;

[0051] Priority queue load balancing strategy: balances the load between NVMe priority queues by scheduling sub-requests in the overload queue to avoid congestion in the overload queue that affects the tail delay of the cache;

[0052] Feedback adjustment mechanism: monitors the load and request latency of the priority queue, analyzes high fan-out requests with higher latency, and dynamically adjusts the cache to ensure low tail latency.

[0053] Specifically, the differentiated request processing strategy is as follows: Figure 2 As shown, it includes the following components:

[0054] Component A1: Based on the weighted round-robin (WRR) mechanism characteristics of NVMe solid-state drives (SSDs), the NVMe queues are divided into three categories: high-priority queues, medium-priority queues, and low-priority queues, and the corresponding queue weights W are set. H , W M , W L , the queue weight satisfies W L <W M <W H At the same time, a high, medium, or low priority level is set for each sub-request, so that the sub-request enters a queue that matches its own priority level. Then, based on the queue weight, the NVMe controller takes out the same number of sub-requests as the weight from the priority queue in a round-robin manner for processing, resulting in differences in the processing speed of the requests.

[0055] Component A2: Record the fan-out size f requested by the client, and then compare it with W H , W M , W L To ensure the processing speed of high fan-out requests, the larger the fan-out of a request, the higher the priority of its sub-requests. L When W L <f≤W M When W M <f≤W H , the priority of all subrequests is set to high priority.

[0056] Component A3: Record the fan-out size f requested by the client, and then compare it with W H , W H +W M To avoid starvation and optimize the parallel processing of subrequests, the priority of some subrequests with high fan-out requests is appropriately reduced. H <f≤W H +W M When f>W, the priority of the subrequest includes high priority and medium priority; when f>W H +W M When , the priority of the sub-request includes high, medium and low priorities. The number of sub-requests with different priorities is set repeatedly by W H sub-requests are high priority, W M sub-requests are of medium priority, W LThe sub-requests are determined as low priority until the priority assignment is completed.

[0057] Specifically, component A1 is implemented based on SPDK (a high-performance storage development kit). For each priority category, a corresponding priority queue is set, and these priority queues are bound to different CPU cores. In addition, a corresponding background thread is set for each priority queue to realize the simulation of multi-queue parallel processing.

[0058] Specifically, for components A2 and A3, the weights of the three priority queues are set to 4, 2, and 1 respectively. At the same time, the number of sub-requests of different priorities is recorded in the initialization structure of the high fan-out request. The high fan-out request structure contains multiple fields such as fan-out size, client request ID, number of sub-request completions, and client request completion time.

[0059] Specifically, Figure 3 As shown, the sub-request sorting algorithm evaluates the impact of sub-requests on cache tail latency from two dimensions: the impact of sub-requests on the latency of the current high fan-out request and the impact on the latency of other high fan-out requests, specifically including:

[0060] Step B1: read and record all sub-requests contained in a high fan-out request;

[0061] Step B2: Obtain the data size characteristics of the sub-requests, and evaluate the impact of the sub-requests on the latency of the current high fan-out request based on the characteristics, so as to complete the preliminary sorting. The larger the data size of the sub-requests, the higher the ranking;

[0062] Step B3: If the two sub-requests involve the same amount of data, obtain the popularity characteristics of the data involved in the sub-requests, evaluate the impact of the sub-requests on the latency of other high fan-out requests based on the characteristics, and complete the subsequent sorting. The higher the popularity of the data involved in the sub-requests, the higher the ranking;

[0063] Step B4: Determine the exact priority of each sub-request based on the sorting result and the number of sub-requests with different priorities. The higher the sub-request is in the sorting result, the higher its priority is.

[0064] Specifically, for step B1, the present invention generates corresponding sub-request structures for all sub-requests and stores them in memory until the client request returns. The sub-request structure includes fields such as client request ID, sub-request data size, sub-request object popularity, sub-request priority, and sub-request ID.

[0065] Specifically, for step B2, the amount of data involved in the sub-request will directly affect the processing time of the sub-request in the NVMe SSD, so the larger the amount of data involved in the sub-request, the greater the impact on latency and the higher the ranking. For step B3, due to the lock contention problem in parallel processing, the higher the popularity of the data involved in the sub-request, the easier it is to cause lock contention and the greater the impact on latency.

[0066] For details, see Figure 4 , the priority queue load balancing strategy includes the following steps:

[0067] Step C1: Obtain and record the load of each priority queue and the number of input / output operations per second (IOPS);

[0068] Step C2: Check the load of the queue to determine whether any queue is overloaded;

[0069] Step C3: If any queue is overloaded, further scheduling operations are performed on the sub-requests to balance the load. The overloaded queue is set as the source queue for scheduling, and other priority queues are set as the target queues for scheduling;

[0070] Step C4: Calculate the proportion of sub-requests to be scheduled IOPS source With W source Represents the number of input / output operations per second and queue weight of the dispatch source queue; IOPS destination With W destination Respectively represent the number of input / output operations per second and queue weight of the scheduling target queue;

[0071] Step C5: When subsequent high fan-out requests arrive, sub-requests with a P ratio are scheduled from the overloaded queue to other queues by modifying the sub-request priority. The specific number of sub-requests scheduled is Where N represents the number of sub-requests that were originally scheduled to enter the overload queue for this high fan-out request;

[0072] Step C6: Based on the latest number of sub-requests in each priority level obtained in C5 and the sorting result of B4, the priorities of the sub-requests are reset and then sent to each priority queue for processing.

[0073] Specifically, for steps C4 and C5, the sub-requests will be scheduled to two priority queues at the same time, so the sub-request proportion P will be calculated twice, indicating the proportion of sub-requests that need to be scheduled to the corresponding priority queues. At the same time, in order to avoid the sub-requests affecting the load of other priority queues after scheduling, the specific scheduling quantity is calculated by rounding down, and further sub-request scheduling is performed for each high fan-out request. Sub-request scheduling is completed by modifying the priority of the sub-request to avoid performing scheduling operations after the sub-request has entered the NVMe queue.

[0074] Specifically, the feedback adjustment mechanism includes the following components:

[0075] Component D1: Periodically collect and transmit the load information of the priority queues. At the end of each cycle, record the total number of sub-requests processed by each queue during the cycle. Record the ratio of the total number of sub-requests to the time window size corresponding to the cycle as the queue's IOPS. Record the ratio of each queue's IOPS to the queue weight as the queue's load ratio. If a queue's load ratio is much higher than the other two priority queues, the queue is overloaded.

[0076] Component D2: Monitor request latency. Monitor the overall latency of each high fan-out request as it completes and compare it with the 99th percentile (P99) latency of the past cycle. If the two are similar, it is determined that the latency of the current high fan-out request is abnormal, and the cache needs to be adjusted to reduce the completion latency of subsequent high fan-out requests.

[0077] Component D3: When there is an abnormal delay in a high fan-out request, identify the sub-request that causes the high delay, analyze the priority queue to which it belongs and the cause of the delay, and make dynamic adjustments. Specific adjustment measures include: when the overload is caused by the excessive load of the priority queue, pass the relevant information to C1 in advance for optimization; if it is due to improper priority setting, increase the priority of some sub-requests during the priority allocation process, and reduce the number of sub-requests allocated to the corresponding queue during the load balancing process. If the tail delay still does not reach the expected target after multiple adjustments, the weight of the priority queue will be further modified to reduce the overall cache tail delay.

[0078] Specifically, the dynamic adjustment process included in component D3 is as follows: Figure 5 As shown. When the request delay is abnormal, the cause will be analyzed and recorded. First, the processing time of each sub-request is tracked to determine the source of the exception. According to the structure field of the sub-request, if the priority is too low, it is recorded as improper priority allocation; if the priority is reasonable, the queue load is checked. If the load is too heavy, it is recorded as queue overload, otherwise it is recorded as unreasonable weight. If the same reason causes delay abnormalities multiple times, corresponding dynamic adjustments will be made.

[0079] A cache tail latency optimization method based on NVMe solid-state drive characteristics provided by the present invention includes:

[0080] Differentiated request processing strategy: Determine the number of sub-requests contained in different priorities based on the weighted polling arbitration mechanism of the NVMe SSD and the fan-out size of the client request.

[0081] Subrequest sorting algorithm: Based on the characteristics of subrequests, their impact on latency is evaluated and sorted according to the impact to determine the priority of subrequests.

[0082] Priority queue load balancing strategy: Load balancing between NVMe priority queues is achieved by scheduling sub-requests in the overload queue.

[0083] Feedback adjustment mechanism: monitors the load of the priority queue and request latency, analyzes requests with higher latency, and dynamically adjusts the cache to maintain low tail latency.

[0084] In the description of the present application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0085] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A cache tail latency optimization method based on NVMe solid state drive characteristics, characterized in that: include: Differentiated request processing strategy: Three priority levels are set based on the weighted polling arbitration mechanism of the NVMe SSD. The number of sub-requests in different priorities is determined according to the fan-out size of the client request. While giving higher priority to sub-requests with high fan-out requests, the difference in processing time between sub-requests is reduced. Subrequest sorting algorithm: evaluates the impact of subrequests on latency based on their characteristics, and sorts them according to the impact to determine the priority of subrequests, ensuring that all subrequests are processed in a timely manner; Priority queue load balancing strategy: balances the load between NVMe priority queues by scheduling sub-requests in the overload queue to avoid congestion in the overload queue that affects the tail delay of the cache; Feedback adjustment mechanism: monitors the load and request latency of the priority queue, analyzes high fan-out requests with higher latency, and dynamically adjusts the cache to ensure low tail latency.

2. A cache tail delay optimization method based on NVMe solid state drive characteristics according to claim 1, characterized in that: The differentiated request processing strategy includes the following components: Component A1: Based on the weighted round-robin (WRR) mechanism characteristics of NVMe solid-state drives (SSDs), the NVMe queues are divided into three categories: high-priority queues, medium-priority queues, and low-priority queues, and the corresponding queue weights W are set. H , W m , W L , the queue weight satisfies W L <W M <W H At the same time, a high, medium, or low priority level is set for each sub-request, so that the sub-request enters a queue that matches its own priority level. Then, based on the queue weight, the NVMe controller takes out the same number of sub-requests as the weight from the priority queue in a round-robin manner for processing, resulting in differences in the processing speed of the requests. Component A2: Record the fan-out size f requested by the client, and then compare it with W H , W M , W L To ensure the processing speed of high fan-out requests, the larger the fan-out of a request, the higher the priority of its sub-requests. L When W L <f≤W M When W M <f≤W H , the priority of all subrequests is set to high priority. Component A3: Record the fan-out size f requested by the client, and then compare it with W H , W H +W M To avoid starvation and optimize the parallel processing of subrequests, the priority of some subrequests with high fan-out requests is appropriately reduced. H <f≤W H +W M When f>W, the priority of the subrequest includes high priority and medium priority; when f>W H +W M When , the priority of the sub-request includes high, medium and low priorities. The number of sub-requests with different priorities is set repeatedly by W H sub-requests are high priority, W M sub-requests are of medium priority, W L The sub-requests are determined as low priority until the priority assignment is completed.

3. A cache tail delay optimization method based on NVMe solid state drive characteristics according to claim 1, characterized in that: The subrequest sorting algorithm evaluates the impact of subrequests on cache tail latency from two dimensions: the impact of subrequests on the latency of the current high fan-out request and the impact on the latency of other high fan-out requests. Specifically, the following dimensions are included: Step B1: read and record all sub-requests contained in a high fan-out request; Step B2: Obtain the data size characteristics of the sub-requests, and evaluate the impact of the sub-requests on the latency of the current high fan-out request based on the characteristics, so as to complete the preliminary sorting. The larger the data size of the sub-requests, the higher the ranking; Step B3: If the two sub-requests involve the same amount of data, obtain the popularity characteristics of the data involved in the sub-requests, evaluate the impact of the sub-requests on the latency of other high fan-out requests based on the characteristics, and complete the subsequent sorting. The higher the popularity of the data involved in the sub-requests, the higher the ranking; Step B4: Determine the exact priority of each sub-request based on the sorting result and the number of sub-requests with different priorities. The higher the sub-request is in the sorting result, the higher its priority is.

4. A cache tail delay optimization method based on NVMe solid state drive characteristics according to claim 1, characterized in that: The priority queue load balancing strategy includes the following steps: Step C1: Obtain and record the load of each priority queue and the number of input / output operations per second (IOPS); Step C2: Check the load of the queue to determine whether any queue is overloaded; Step C3: If any queue is overloaded, further scheduling operations are performed on the sub-requests to balance the load. The overloaded queue is set as the source queue for scheduling, and other priority queues are set as the target queues for scheduling; Step C4: Calculate the proportion of sub-requests to be scheduled IOPS source With W source Represents the number of input / output operations per second and queue weight of the dispatch source queue; IOPS destination With W destination Respectively represent the number of input / output operations per second and queue weight of the scheduling target queue; Step C5: When subsequent high fan-out requests arrive, sub-requests with a P ratio are scheduled from the overloaded queue to other queues by modifying the sub-request priority. The specific number of sub-requests scheduled is Where N represents the number of sub-requests that were originally scheduled to enter the overload queue for this high fan-out request; Step C6: Based on the latest number of sub-requests in each priority level obtained in C5 and the sorting result of B4, the priorities of the sub-requests are reset and then sent to each priority queue for processing.

5. A cache tail delay optimization method based on NVMe solid state drive characteristics according to claim 1, characterized in that: The feedback regulation mechanism includes the following components: Component D1: Periodically collect and transmit the load information of the priority queues. At the end of each cycle, record the total number of sub-requests processed by each queue during the cycle. Record the ratio of the total number of sub-requests to the time window size corresponding to the cycle as the queue's IOPS. Record the ratio of each queue's IOPS to the queue weight as the queue's load ratio. If a queue's load ratio is much higher than the other two priority queues, the queue is overloaded. Component D2: Monitor request latency. Monitor the overall latency of each high fan-out request as it completes and compare it with the 99th percentile (P99) latency of the past cycle. If the two are similar, it is determined that the latency of the current high fan-out request is abnormal, and the cache needs to be adjusted to reduce the completion latency of subsequent high fan-out requests. Component D3: When there is an abnormal delay in a high fan-out request, identify the sub-request that causes the high delay, analyze the priority queue to which it belongs and the cause of the delay, and make dynamic adjustments. Specific adjustment measures include: when the overload is caused by the excessive load of the priority queue, pass the relevant information to C1 in advance for optimization; if it is due to improper priority setting, increase the priority of some sub-requests during the priority allocation process, and reduce the number of sub-requests allocated to the corresponding queue during the load balancing process. If the tail delay still does not reach the expected target after multiple adjustments, the weight of the priority queue will be further modified to reduce the overall cache tail delay. A cache tail latency optimization method based on NVMe solid-state drive characteristics, characterized in that: include: Differentiated request processing strategy: Determine the number of sub-requests contained in different priorities based on the weighted polling arbitration mechanism of the NVMe SSD and the fan-out size of the client request. Subrequest sorting algorithm: Based on the characteristics of subrequests, their impact on latency is evaluated and sorted according to the impact to determine the priority of subrequests. Priority queue load balancing strategy: Load balancing between NVMe priority queues is achieved by scheduling sub-requests in the overload queue. Feedback adjustment mechanism: monitors the load of the priority queue and request latency, analyzes requests with higher latency, and dynamically adjusts the cache to maintain low tail latency.

Images