A block device layer adaptive quality of service scheduling method and system

Abstract

The application provides a block device layer adaptive quality of service scheduling method and system, the method comprises the following steps: collecting the behavior characteristics of the IO request of the block device layer and forming a feature vector; inputting the feature vector into a state machine, identifying the current AI workload stage in combination with a hysteresis mechanism (a consistent determination of a plurality of time windows in the alternative stage); according to the identification result, atomically adjusting the quality of service control parameters of the token bucket of the block device layer to realize the scheduling of subsequent IO requests. The application does not need to rely on the notification of the upper AI framework, realizes the real-time stage inference with anti-jitter at low computational complexity, accurately matches the IO demand of each semantic stage of the AI task, optimizes the scheduling efficiency, stabilizes the IO delay, improves the GPU utilization, reduces the checkpoint writing time consumption and inference service delay.

Description

Technical Field

[0001] This invention belongs to the field of computer storage technology, and particularly relates to an adaptive quality of service scheduling method and system for block device layer. Background Technology

[0002] Artificial intelligence training and inference tasks exhibit drastically different input and output characteristics to the storage system at different stages: the data loading stage presents high-bandwidth sequential reads, the checkpointing stage is characterized by bursty, large-granular sequential writes, and the inference stage requires low-latency random reads. Currently, the quality of service control at the operating system kernel block device layer largely relies on statically configured token bucket parameters, which cannot perceive the semantic stage of the upper-layer AI workload. When faced with cross-stage switching, static strategies struggle to simultaneously meet the conflicting demands of high throughput and low latency, often resulting in GPUs being idle due to insufficient storage response, or burst writes being excessively throttled, lengthening the training cycle. Some existing technologies attempt to achieve adaptive scheduling at the distributed storage layer, virtualization layer, or application framework level, but their scheduling decision-making location is far from the physical block device, resulting in millisecond-level scheduling latency and an inability to timely regulate traffic to flash memory devices with microsecond-level response times. Furthermore, existing workload identification methods primarily classify workloads based on general, shallow I / O statistical characteristics such as sequential or random, read or write, lacking the ability to map these characteristics to AI semantic stages such as data loading and checkpointing, leading to a mismatch between scheduling strategies and actual task requirements. Summary of the Invention

[0003] This application provides a block device layer adaptive quality of service scheduling method and system. With extremely low computational overhead and without any modification to upper-layer applications, it can infer the semantic stage of AI workloads in real time, stably and accurately at the kernel block device layer, and drive the real-time adjustment of quality of service policies accordingly.

[0004] This application discloses an adaptive quality of service scheduling method for the block device layer, including: Collect at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector; The feature vector is input into the state machine, and the AI ​​workload stage corresponding to the current IO request is identified according to the hysteresis mechanism. The state machine refers to a computational model that maintains a set of states corresponding to the semantic stages of the AI ​​task and defines the conditions for transitions between states. The hysteresis mechanism refers to a mechanism that requires the alternative stages to be consistently determined within multiple consecutive time windows before triggering the state transition of the state machine. The AI ​​workload stage refers to the semantic stage of the upper-layer artificial intelligence task inferred from the IO behavior of the block device layer. Based on the identified current AI workload stage, the quality of service control parameters of the block device layer are adjusted to schedule subsequent IO requests.

[0005] Optionally, the step of inputting the feature vector into the state machine and identifying the AI ​​workload stage corresponding to the current IO request based on the hysteresis mechanism includes: The candidate AI workload stages are determined by matching the feature vectors with preset stage determination rules. Determine whether the same candidate AI workload phase is continuously identified within a preset number of consecutive time windows; When the judgment result is yes, the state machine is triggered to transition from the current state to a new state corresponding to the alternative AI workload stage, and the stage corresponding to the new state is taken as the current AI workload stage.

[0006] Optionally, the step of matching the feature vector with preset stage determination rules to determine candidate AI workload stages includes: Based on the sequential read ratio, IO size distribution ratio, and IO interval variance in the feature vector, the candidate AI workload stage is determined to be the data loading stage; Based on the sequential write ratio, IO size distribution ratio, and write traffic increase in the feature vector, the candidate AI workload stage is determined as the checkpoint stage; Based on the random read ratio, IO size distribution ratio, and read request interval variance in the feature vector, the candidate AI workload stage is determined to be the inference stage; Based on the comparison between the total IO traffic in the feature vector and the preset baseline threshold, the candidate AI workload stage is determined to be the computation stage.

[0007] Optionally, the feature vector is obtained by collecting at least one behavioral feature of the input / output I / O requests of the acquisition block device layer, including: When an IO request is completed, atomically update the counts related to IO sequence, the counts related to IO size distribution, and the statistics related to IO request intervals; At the end of the preset time window, calculate the sequential read ratio, sequential write ratio, random read ratio, and random write ratio based on the updated counts related to IO sequence; calculate the proportion of each type of IO based on the updated counts related to IO size distribution; and calculate the IO interval variance based on the updated statistical values ​​related to IO request interval. The feature vector is formed by combining at least one of the sequential read ratio, the proportion of each type of I / O, and the I / O interval variance.

[0008] Optionally, adjusting the quality of service control parameters of the block device layer based on the identified current AI workload stage includes: Based on the current AI workload stage, query the pre-stored stage-parameter mapping template to obtain the parameter set corresponding to the current AI workload stage; the parameter set includes burst capacity, smoothing suppression strength and time slice granularity. The current parameters of the block device layer token bucket are atomically updated to the corresponding values ​​in the parameter group.

[0009] Optionally, after adjusting the quality of service control parameters of the block device layer, the method further includes: Monitor the I / O request latency of the block device layer; When the detected IO request latency remains higher than a preset latency threshold for a preset duration, the service quality control parameters are reset to a preset conservative parameter set. The currently identified AI workload phase is marked as suspicious.

[0010] Optionally, at least one behavioral characteristic of the input / output I / O requests of the acquisition block device layer includes: Based on multiple preset IO size ranges, the received IO requests are divided into corresponding ranges; Maintain an independent statistical context for each of the aforementioned IO size ranges; Each sub-feature vector is calculated based on its respective statistical context, and all sub-feature vectors are used together as the feature vector.

[0011] Optionally, atomically updating the current parameters of the block device layer token bucket to the corresponding values ​​in the parameter group includes: When the new burst capacity in the parameter group is less than the available token amount in the current token bucket, the excess token amount exceeding the new burst capacity is determined. Within each time period, the available token amount is reduced at a preset decay rate until the available token amount does not exceed the new burst capacity; During the decay of the available token quota, the block device layer token bucket still uses the new burst capacity as a parameter.

[0012] Optionally, the consecutive preset number of time windows is at least three time windows.

[0013] This application discloses, in one aspect, an adaptive quality of service scheduling system for the block device layer, comprising: The acquisition module acquires at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector; The input module inputs the feature vector into the state machine and identifies the AI ​​workload stage corresponding to the current IO request based on the hysteresis mechanism. The state machine refers to a computational model that maintains a set of states corresponding to the semantic stages of the AI ​​task and defines the conditions for transitions between states. The hysteresis mechanism refers to a mechanism that requires the candidate stage to be consistently determined within multiple consecutive time windows before triggering the state transition of the state machine. The AI ​​workload stage refers to the semantic stage of the upper-layer artificial intelligence task inferred from the IO behavior of the block device layer. The scheduling module adjusts the quality of service control parameters of the block device layer based on the identified current AI workload stage in order to schedule subsequent IO requests.

[0014] As can be seen from the above technical solution, for sudden large-granularity sequential read / write traffic caused by upper-layer AI tasks during the data loading and checkpointing phases, the feature vector in this method quickly captures the characteristics of abnormally high sequential read or write ratios and prominent large IO ratios, and matches the corresponding candidate phases according to the judgment rules. At this time, the state machine will not switch immediately due to the instantaneous fluctuations of a single window. The hysteresis mechanism requires that the candidate phase be stably identified within multiple consecutive time windows, thereby filtering out boundary misjudgments caused by data shuffling or short-term batch operations. Once the state machine completes phase confirmation and updates to the data loading or checkpoint state, the system immediately increases the burst capacity of the token bucket, reduces the smoothing suppression strength, and increases the time slice granularity, so that the token bucket can generate tokens with a higher burst limit and faster speed, smoothly absorbing the staged flood of IO, and significantly shortening the GPU's data waiting time and model saving window. When the task turns into a computationally intensive period, the IO traffic drops significantly below the baseline. After the state machine stably determines that it is a computation phase within a consecutive window, it immediately reverts the token bucket parameters to a conservative configuration, actively giving up bandwidth for the IO of other hosts. In inference scenarios, the consistently occurring small-granularity random reads and low-interval variance characteristics are stably identified as inference phases. The system then switches to a parameter template with extremely strong smoothing suppression and extremely small time slices, ensuring that the latency of each read request is not affected by sudden traffic spikes, and tail latency is effectively controlled. Thus, the entire scheduling process, through a logical closed loop of feature acquisition, state machine matching, hysteresis confirmation, and atomic parameter adjustment, achieves accurate response and smooth transition to the IO requirements of each stage of the AI ​​task without making any modifications to the upper-layer AI framework. Ultimately, this improves overall GPU utilization and reduces checkpoint write time and inference service latency. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is one of the flowcharts of an adaptive quality of service scheduling method for the block device layer in the embodiments of this application; Figure 2 This is a flowchart illustrating how a block device layer adaptive quality of service scheduling method in this application inputs the feature vector into a state machine and identifies the AI ​​workload stage corresponding to the current IO request based on a hysteresis mechanism. Figure 3 This is a flowchart illustrating how a block device layer adaptive quality of service scheduling method in this application determines candidate AI workload stages by matching the feature vector with preset stage determination rules. Figure 4 This is a flowchart illustrating the process of collecting at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector in an adaptive quality of service scheduling method for the block device layer according to an embodiment of this application. Figure 5 This is a flowchart illustrating how a block device layer adaptive quality of service scheduling method adjusts the quality of service control parameters of the block device layer based on the identified current AI workload stage, according to an embodiment of this application. Figure 6 This is a flowchart illustrating the process after adjusting the quality of service control parameters of the block device layer in an embodiment of this application's adaptive quality of service scheduling method. Figure 7 This is a flowchart illustrating the process of collecting at least one behavioral characteristic of input / output I / O requests from the block device layer in an adaptive quality of service scheduling method for the block device layer according to an embodiment of this application. Figure 8 This is a flowchart illustrating how a block device layer adaptive quality of service scheduling method in this application atomically updates the current parameters of the block device layer token bucket to the corresponding values ​​in the parameter group. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not limiting, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without such specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0018] In this application embodiment, the specific implementation of the prior art is first described to provide a clearer understanding of the technical problem to be solved by this application. For example, in a multi-tenant shared storage system, multiple artificial intelligence tasks are deployed, where tenant A runs a large model training task and tenant B runs an online inference service. The system uses a traditional static token bucket strategy for service quality control, and the administrator configures a fixed maximum bandwidth and IO limit per second for each tenant. During the data loading phase of the training task, tenant A needs high-bandwidth sequential reads to quickly load the dataset, but due to the low bandwidth limit of the static configuration, the data loading speed is slow, causing the computing unit to be idle for a long time. When the training task enters the checkpoint phase, a sudden surge of large-granularity sequential write traffic occurs. At this time, the burst capacity of the static configuration is insufficient, resulting in excessively long checkpoint write times and prolonging the entire training cycle. At the same time, tenant B's inference service requires low-latency random reads, but when tenant A's checkpoint write traffic surges, it will preempt a large amount of storage resources, causing a significant increase in the latency of inference requests, with the latency of some requests even exceeding the threshold specified by the service level agreement. To mitigate these issues, administrators need to continuously and manually adjust the QoS parameters for each tenant. However, due to the unpredictable nature of AI task phase changes, manual adjustments cannot keep up with load changes in a timely manner, and parameter misconfiguration is prone to occur during the adjustment process, further impacting system performance. Furthermore, some existing technologies attempt to achieve adaptive scheduling at the distributed storage or virtualization layer, but the scheduling decision-making location of these solutions is far from the physical block devices, resulting in millisecond-level scheduling latency and an inability to perform timely traffic control for flash memory devices with microsecond-level response times. Other technologies classify workloads based on general I / O characteristics, but they cannot distinguish between sequential reads of training data loading and sequential writes of checkpoints, leading to a mismatch between scheduling strategies and actual task requirements.

[0019] Based on this, this application provides an adaptive quality of service scheduling method for the block device layer, such as... Figure 1 As shown, it includes: S100: Collect at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector; S200: Input the feature vector into the state machine, and identify the AI ​​workload stage corresponding to the current IO request according to the hysteresis mechanism; the state machine refers to a computational model that maintains a set of states corresponding to the semantic stages of the AI ​​task and defines the transition conditions between states; the hysteresis mechanism refers to a mechanism that requires the alternative stages to be consistently determined within multiple consecutive time windows before triggering the state transition of the state machine; the AI ​​workload stage refers to the semantic stage of the upper-layer artificial intelligence task inferred from the block device layer IO behavior; S300: Based on the identified current AI workload stage, adjust the quality of service control parameters of the block device layer to schedule subsequent IO requests.

[0020] This application provides an adaptive quality of service scheduling method for the block device layer. For sudden, large-granularity sequential read / write traffic triggered by upper-layer AI tasks during data loading and checkpointing phases, this method uses feature vectors to quickly capture characteristics such as abnormally high sequential read or write ratios and prominent large I / O ratios. Corresponding candidate phases are then matched using judgment rules. The state machine does not immediately switch due to instantaneous fluctuations in a single window; a hysteresis mechanism requires that the candidate phase be stably identified within multiple consecutive time windows, thus filtering out boundary misjudgments caused by data shuffling or brief batch operations. Once the state machine completes phase confirmation and updates to the data loading or checkpoint state, the system immediately increases the token bucket's burst capacity, reduces the smoothing suppression strength, and increases the time slice granularity. This allows the token bucket to generate tokens with a higher burst limit and faster speed, smoothly absorbing the staged flood of I / O and significantly shortening the GPU's data waiting time and model saving window. When the task transitions to a computationally intensive period, and I / O traffic drops significantly below the baseline, the state machine, after stably determining the computation phase within a consecutive window, immediately reverts the token bucket parameters to a conservative configuration, proactively relinquishing bandwidth for other hosts' I / O. In inference scenarios, the consistently occurring small-granularity random reads and low-interval variance characteristics are stably identified as inference phases. The system then switches to a parameter template with extremely strong smoothing suppression and extremely small time slices, ensuring that the latency of each read request is not affected by sudden traffic spikes, and tail latency is effectively controlled. Thus, the entire scheduling process, through a logical closed loop of feature acquisition, state machine matching, hysteresis confirmation, and atomic parameter adjustment, achieves accurate response and smooth transition to the IO requirements of each stage of the AI ​​task without making any modifications to the upper-layer AI framework. Ultimately, this improves overall GPU utilization and reduces checkpoint write time and inference service latency.

[0021] It should be noted that the core concept of this application is to construct an AI workload stage identification and adaptive scheduling method based on a state machine and hysteresis mechanism, deployed at the operating system kernel block device layer. This method does not rely on active notifications from the upper-layer AI framework, but directly collects the I / O sequence, size distribution, and request interval characteristics of the block device layer to form a feature vector. This feature vector is input into a predefined state machine and matched with the judgment rules of each AI semantic stage to obtain candidate stages. Crucially, the actual stage switching is not instantaneous, but introduces a hysteresis mechanism: only when the same candidate stage is consistently determined within multiple consecutive sampling time windows is the state transition of the state machine completed, and the confirmed stage is taken as the current AI workload stage. Finally, based on the confirmed stage, the system atomically adjusts the burst capacity, smoothing suppression strength, and time slice granularity of the token bucket, and immediately executes differentiated scheduling matching the semantic stage. This technical solution achieves deterministic, jitter-resistant real-time stage inference with extremely low computational complexity on the kernel hot path, bridging the gap between the general I / O characteristics of the lower-layer block device and the specific semantic requirements of the upper-layer AI task.

[0022] The embodiments of this application will be described in detail below.

[0023] In this embodiment, at least one behavioral feature of input / output I / O requests from the block device layer is collected to obtain a feature vector. The block device layer, located in the operating system kernel, is an intermediate layer connecting the file system and physical storage devices. All I / O requests from upper-layer applications ultimately pass through the block device layer to the physical storage devices. Therefore, collecting I / O behavioral features at the block device layer allows for the direct acquisition of the most accurate and real-time I / O data, avoiding latency and distortion introduced by upper-layer protocol stacks or virtualization layers. Feature collection can be performed at the I / O request entry point. When an I / O request arrives at the block device layer, the system extracts relevant information about the request and updates the corresponding statistical counters. These statistical counters can be maintained in the context structure of each registered device for quick access and updates. The context structure is a data structure in the operating system kernel used to store device-related state information. Each block device registered in the system has a corresponding context structure, which includes basic device information, driver interfaces, I / O queue pointers, and, in this application, newly added statistical counters and state machine information.

[0024] It's important to note that I / O behavior characteristics can include, but are not limited to, the order of I / O requests, I / O size distribution, I / O request intervals, and read / write ratios. The order of I / O requests can be determined by comparing the offset of the current I / O request with the offset of the previous I / O request. If the offset of the current I / O request equals the offset of the previous I / O request plus its size, then the I / O request is considered sequential; otherwise, it is considered random I / O. Here, the offset refers to the starting address of the I / O request on the storage device, in bytes. For example, if the offset of the previous I / O request was 1024 bytes and its size was 512 bytes, then the offset of the next sequential I / O request should be 1024 + 512 = 1536 bytes. If the offset of the next I / O request is 2048 bytes, then it is a random I / O request.

[0025] IO size distribution divides IO requests into multiple intervals based on size, and counts the number of IO requests and the total number of bytes in each interval. The purpose of dividing into intervals is to describe the size characteristics of IO requests in more detail, because different IO request sizes have different impacts on storage system performance. Generally speaking, large-granularity IO requests are more suitable for sequential access, making full use of the storage device's bandwidth; small-granularity IO requests are more suitable for random access, placing higher demands on the storage device's latency.

[0026] The IO request interval records the time difference between the arrival of two adjacent IO requests, calculating their average and variance. The average IO request interval reflects the arrival rate of IO requests, while the variance reflects the regularity of IO request arrivals. The smaller the variance, the more stable the arrival of IO requests, and the easier it is to predict and schedule them; the larger the variance, the more unstable the arrival of IO requests, and the greater the possibility of sudden traffic bursts.

[0027] The read / write ratio can be used to measure the proportion of read requests and write requests (in bytes) over a period of time. The read / write ratio is a crucial factor affecting storage system performance because read and write operations are processed differently on storage devices. Generally, read operations have lower latency than write operations because write operations require waiting for data to be written to the storage medium, while read operations only need to retrieve data from the storage medium. Furthermore, write operations incur additional overhead such as garbage collection and wear leveling, which has a greater impact on storage system performance.

[0028] For example, the system can maintain the following counters: sequential read count, random read count, sequential write count, random write count, request count and byte count for each IO size range, offset of the last IO request, and arrival time of the last IO request. Each time an IO request arrives, the system updates the corresponding counters based on the request type (read or write), offset, and size. For instance, if the current request is a read request and the offset equals the last IO offset plus the last IO size, the sequential read count is incremented by 1; otherwise, the random read count is incremented by 1. Simultaneously, based on the request size, it is categorized into the corresponding IO size range, and the request count and byte count for that range are updated. Furthermore, the arrival time of the current request is recorded, the interval between the current request's arrival time and the last request's arrival time is calculated, and the statistical value of the request interval is updated.

[0029] In this embodiment, the counter update operation requires the use of atomic operations to ensure concurrency safety. Because in a multiprocessor system, multiple CPU cores may process I / O requests simultaneously, thus updating the same counter concurrently. Without atomic operations, data races may occur, leading to inaccurate counter values. For example, two CPU cores might simultaneously read the counter sequentially, obtaining a value of 100, then both increment it by 1, and finally write back a value of 101, while the correct value should be 102. Atomic operations ensure that the counter update operation is indivisible; it either completes entirely or is not executed at all, without any intermediate states. In C, atomic updates can be implemented using variables of type `atomic_t` and corresponding atomic operation functions, such as `atomic_inc`, `atomic_add`, and `atomic_read`.

[0030] In this embodiment, after obtaining the feature vector, it is input into a state machine, and the AI ​​workload stage corresponding to the current IO request is identified according to the hysteresis mechanism. A state machine is a computational model that maintains a finite set of states and defines the transition conditions between states. In this application, each state of the state machine corresponds to a semantic stage of an AI workload, such as a data loading stage, checkpointing stage, inference stage, and computation stage. The transition conditions of the state machine are defined by preset stage determination rules. When the feature vector satisfies the determination rule for a certain stage, the system will consider that stage as a candidate stage.

[0031] It's important to note that the AI ​​workload stage refers to the semantic stage of the upper-layer AI task inferred from the block device layer's I / O behavior. Different AI tasks exhibit different I / O behavior characteristics at different semantic stages, and these characteristics possess certain regularity and stability. For example, the data loading stage typically generates a large number of sequential read requests, with large I / O sizes and relatively regular request intervals; the checkpointing stage typically generates a large number of sequential write requests, with very large I / O sizes and sudden surges in traffic within a short period; the inference stage typically generates a large number of random read requests, with smaller I / O sizes and relatively stable request intervals; and the computation stage has almost no I / O requests and very low I / O traffic. These characteristics are determined by the inherent logic of the AI ​​task. For instance, the data loading stage requires reading training data from storage devices, the checkpointing stage requires writing model parameters to storage devices, the inference stage requires reading model parameters and input data from storage devices, and the computation stage mainly performs matrix operations and other operations on computational units, requiring almost no access to storage devices.

[0032] A hysteresis mechanism refers to a mechanism that requires a candidate stage to be consistently determined within multiple consecutive time windows before triggering a state transition in the state machine. The purpose of introducing a hysteresis mechanism is to avoid frequent switching of the state machine between different stages due to instantaneous fluctuations in I / O characteristics, which would cause frequent adjustments to QoS parameters and affect system stability. For example, if the I / O characteristics in a certain time window just meet the determination conditions for the data loading stage, but not in the next time window, immediately switching the state would cause the QoS parameters to change repeatedly in a short period, actually degrading system performance. By introducing a hysteresis mechanism, a state transition is only triggered when the same candidate stage is determined to meet the conditions within multiple consecutive time windows. This filters out the impact of instantaneous fluctuations and improves the stability of stage identification.

[0033] For example, the system can set the time window size to 1 second, meaning that a feature vector is collected and a stage determination is performed every 1 second. The hysteresis mechanism requires that the same candidate stage be determined for three consecutive time windows before a state transition is triggered. For instance, in the first second, if the feature vector meets the determination criteria for the data loading stage, the system selects the data loading stage as a candidate stage; in the second second, if the feature vector still meets the determination criteria for the data loading stage, the candidate stage remains the data loading stage; in the third second, if the feature vector still meets the determination criteria for the data loading stage, the system triggers the state machine to transition from the current state to the data loading stage state. If, in any of these three time windows, the feature vector does not meet the determination criteria for the data loading stage, then the candidate stage will change, and the state transition will not be triggered.

[0034] In this embodiment, the size of the time window can be adjusted according to actual needs. A smaller time window results in higher real-time performance for stage identification, but also higher computational overhead and greater susceptibility to transient fluctuations; a larger time window results in higher stability for stage identification and lower computational overhead, but lower real-time performance. In practical applications, an appropriate time window size can be selected based on the system's hardware performance and the characteristics of the AI ​​task. For example, for high-performance storage devices and AI tasks with high real-time requirements, a smaller time window, such as 500 milliseconds, can be selected; for lower-performance storage devices and AI tasks with lower real-time requirements, a larger time window, such as 2 seconds, can be selected.

[0035] In this embodiment, after identifying the current AI workload stage, the service quality control parameters of the block device layer are adjusted according to this stage to schedule subsequent IO requests. The service quality control parameters may include the token bucket burst capacity, smoothing suppression strength, and time slice granularity. The token bucket is a commonly used flow control algorithm that controls the sending rate of IO requests by maintaining a token bucket. The token bucket generates tokens at a certain rate, and each IO request needs to acquire a certain number of tokens before it can be sent. If the number of tokens in the token bucket is insufficient, the IO request will be blocked or dropped. Burst capacity refers to the maximum number of tokens that the token bucket can hold, which determines the maximum burst traffic the system can withstand. Smoothing suppression strength refers to the strength of token generation suppression when the number of tokens in the token bucket exceeds the burst capacity, which determines the smoothness of the system's response to burst traffic. Time slice granularity refers to the time interval between token generation by the token bucket, which determines the precision of token generation.

[0036] It's important to note that different AI workload phases have varying quality of service requirements, necessitating the configuration of different token bucket parameters for each phase. For instance, the data loading phase requires high-bandwidth sequential reads, thus necessitating a larger burst capacity and weaker smoothing suppression to allow for larger bursts of traffic; the checkpointing phase requires bursty, large-granularity sequential writes, requiring even larger burst capacity and weaker smoothing suppression to quickly complete checkpoint writes; the inference phase requires low-latency random reads, thus requiring a smaller burst capacity and stronger smoothing suppression to avoid increased latency due to burst traffic; and the computation phase has almost no I / O requests, thus requiring a smaller burst capacity and stronger smoothing suppression to free up more resources for other tasks.

[0037] For example, the system can maintain a stage-parameter mapping table, which stores the token bucket parameters corresponding to each AI workload stage. When the current stage is identified, the system queries this mapping table to obtain the corresponding parameter set, and then updates the token bucket's current parameters to the values ​​in that parameter set. For instance, when the current stage is identified as the data loading stage, the system sets the burst capacity to 50% of the device's maximum bandwidth, the smoothing suppression strength to 30, and the time slice granularity to 10 milliseconds; when the current stage is identified as the checkpoint stage, the system sets the burst capacity to 80% of the device's maximum bandwidth, the smoothing suppression strength to 10, and the time slice granularity to 10 milliseconds; when the current stage is identified as the inference stage, the system sets the burst capacity to 10% of the device's maximum bandwidth, the smoothing suppression strength to 90, and the time slice granularity to 1 millisecond; when the current stage is identified as the computation stage, the system sets the burst capacity to 5% of the device's maximum bandwidth, the smoothing suppression strength to 95, and the time slice granularity to 1 millisecond.

[0038] In this embodiment, the maximum bandwidth of the device refers to the maximum theoretical or measured bandwidth that the physical storage device can provide, which can be obtained by reading the device's configuration information or performing performance tests. For example, the maximum bandwidth of a high-speed solid-state storage device may be 4GB per second. The unit of burst capacity is bytes, representing the maximum number of bytes that the token bucket can hold. For example, if the maximum bandwidth of the device is 4GB per second, then the burst capacity during the data loading phase is 4GB * 50% = 2GB. The smoothing suppression strength is a value between 0 and 100. The larger the value, the stronger the suppression strength and the more stable the token generation rate; the smaller the value, the weaker the suppression strength and the larger the allowed burst traffic. The unit of time slice granularity is milliseconds, representing how often the token bucket generates a token. For example, a time slice granularity of 10 milliseconds means that the token bucket generates a token every 10 milliseconds, and the number of tokens generated each time is the maximum bandwidth of the device * time slice granularity / 1000. For example, if the maximum bandwidth of the device is 4GB per second and the time slice granularity is 10 milliseconds, then the number of tokens generated each time is 4GB * 10ms / 1000ms = 40MB.

[0039] In alternative implementations, such as Figure 2 As shown, the step of inputting the feature vector into the state machine and identifying the AI ​​workload stage corresponding to the current IO request based on the hysteresis mechanism includes: S210: Match the feature vector with the preset stage determination rules to determine the candidate AI workload stage; S220: Determine whether the same candidate AI workload phase is continuously determined within a preset number of consecutive time windows; S230: When the judgment result is yes, the state machine is triggered to transition from the current state to a new state corresponding to the alternative AI workload stage, and the stage corresponding to the new state is taken as the current AI workload stage.

[0040] In this embodiment, the feature vector is input into the state machine, and the AI ​​workload stage corresponding to the current IO request is identified according to the hysteresis mechanism. Specifically, the following steps are included: First, the feature vector is matched with the preset stage determination rules to determine the candidate AI workload stage; then, it is determined whether the same candidate AI workload stage is continuously determined within a preset number of time windows; when the determination result is yes, the state machine is triggered to transition from the current state to a new state corresponding to the candidate AI workload stage, and the stage corresponding to the new state is taken as the current AI workload stage.

[0041] It's important to note that the preset stage determination rules are pre-defined based on the typical IO characteristics of different AI workload stages. Each stage's determination rule consists of multiple conditions; a stage is only determined when the feature vector satisfies all conditions. For example, the determination rules for the data loading stage might include sequential read ratios exceeding a certain threshold, medium and large granularity IO ratios exceeding a certain threshold, and IO interval variance being less than a certain threshold. These conditions are obtained through statistical analysis of the IO characteristics of a large number of AI tasks, enabling accurate differentiation of different AI workload stages.

[0042] For example, the system can predefine decision rules for four stages: data loading stage, checkpoint stage, inference stage, and computation stage. At the end of each time window, the system calculates the current feature vector and then matches it against the decision rules for each of the four stages in sequence. If the feature vector satisfies all the decision conditions for a certain stage, then that stage is identified as a candidate stage for the current time window. If the feature vector does not satisfy the decision conditions for any stage, the system can either keep the current stage unchanged or set the current stage as the default stage, such as the computation stage. In practical applications, the matching order of the stage decision rules can be adjusted according to the priority of each stage. For example, the priority of the checkpoint stage can be higher than that of the data loading stage because the burst traffic in the checkpoint stage is larger and has a greater impact on system performance. If the QoS parameters are not identified and adjusted in time, it may lead to more serious performance problems.

[0043] In this embodiment, it is determined whether the same candidate AI workload stage is continuously identified within a preset number of time windows. The preset number of time windows can be set according to actual needs, such as 3, 4, or 5. A higher number of windows results in higher stability for stage identification, but reduces real-time performance; conversely, a lower number of windows results in higher real-time performance, but reduces stability. In practical applications, a suitable preset number can be selected based on the system's performance requirements and the characteristics of the AI ​​task. For example, for production environments with high stability requirements, 4 or 5 time windows can be used; for testing environments with high real-time requirements, 2 or 3 time windows can be used.

[0044] For example, the preset number is 3 time windows. The system maintains a list of candidate stages for the three most recent time windows. Whenever a new time window ends and a new candidate stage is determined, the system adds that candidate stage to the end of the list and deletes the oldest candidate stage from the beginning of the list. Then, the system checks whether the three candidate stages in the list are the same. If all three candidate stages are the same, a state transition of the state machine is triggered; if the three candidate stages are not the same, the state machine remains in its current state. For example, if the candidate stages for the three most recent time windows are, in order, the data loading stage, and the data loading stage, the system will trigger a state transition to the data loading stage state; if the candidate stages for the three most recent time windows are, in order, the data loading stage, and the calculation stage, the system will not trigger a state transition, and the state machine will remain in its current state.

[0045] In this embodiment, when the determination result is yes, the state machine is triggered to transition from the current state to a new state corresponding to the candidate AI workload stage, and the stage corresponding to the new state is taken as the current AI workload stage. After the state transition is completed, the system will adjust the service quality control parameters according to the new stage and begin to schedule subsequent IO requests using the new parameters.

[0046] It should be noted that whether the state transitions of the state machine are unidirectional or bidirectional depends on the specific implementation. In this application, the state transitions of the state machine can be bidirectional, meaning it can transition from any state to any other state, as long as the corresponding transition conditions are met. For example, the system can transition from the data loading phase to the computation phase, from the computation phase to the checkpoint phase, and from the checkpoint phase to the inference phase, etc. This bidirectional transition design can adapt to the complex and varied phase changes of AI tasks. For example, during training, there may be multiple switches between the data loading phase and the computation phase, or multiple switches between the computation phase and the checkpoint phase.

[0047] In alternative implementations, such as Figure 3 As shown, the step of matching the feature vector with preset stage determination rules to determine the candidate AI workload stage includes: S240: Based on the sequential read ratio, IO size distribution ratio, and IO interval variance in the feature vector, determine the candidate AI workload stage as the data loading stage; S250: Based on the sequential write ratio, IO size distribution ratio and write traffic increase in the feature vector, determine the candidate AI workload stage as the checkpoint stage; S260: Based on the random read ratio, IO size distribution ratio and read request interval variance in the feature vector, determine the candidate AI workload stage as the inference stage; S270: Based on the comparison result between the total IO traffic in the feature vector and the preset baseline threshold, the candidate AI workload stage is determined to be the computation stage.

[0048] In this embodiment, candidate AI workload stages are determined by matching feature vectors with preset stage determination rules. Specifically, the following scenarios are considered: a data loading stage is determined based on the sequential read ratio, IO size distribution ratio, and IO interval variance in the feature vectors; a checkpoint stage is determined based on the sequential write ratio, IO size distribution ratio, and write traffic increase in the feature vectors; an inference stage is determined based on the random read ratio, IO size distribution ratio, and read request interval variance in the feature vectors; and a computation stage is determined based on the comparison between the total IO traffic in the feature vectors and a preset baseline threshold.

[0049] It should be noted that the thresholds in the judgment criteria for each stage can be adjusted according to the actual situation. For example, the threshold for sequential read ratio can be set to 80%, the threshold for medium and large granularity IO ratio can be set to 60%, and the threshold for IO interval variance can be set to 0.1, etc. These thresholds can be obtained through statistical analysis of the IO characteristics of a large number of AI tasks, or they can be dynamically adjusted according to the actual operation of the system. In practical applications, administrators can fine-tune these thresholds based on their experience and the system's performance to achieve the best stage recognition effect.

[0050] For example, the criteria for determining the data loading phase are: sequential read ratio greater than 80%, the sum of medium-granularity IO and large-granularity IO ratios greater than 60%, and IO interval variance less than 0.1. Here, medium-granularity IO refers to IO requests between 64KB and 512KB in size, and large-granularity IO refers to IO requests between 512KB and 4MB in size. IO interval variance refers to the variance between adjacent IO requests; the smaller the variance, the more regular the arrival of IO requests. These characteristics of the data loading phase are determined by the working method of the data loader. The data loader typically reads training data in batches, reading large data blocks each time, and the reading order is consecutive. Therefore, it generates a large number of sequential read requests, with large IO sizes and relatively regular request intervals.

[0051] The criteria for determining the checkpoint phase are: sequential write ratio greater than 80%, ultra-large granularity I / O ratio greater than 50%, and write traffic increasing more than three times within a short window. Ultra-large granularity I / O refers to I / O requests of 4MB or larger. The short window can be set to 1 second, comparing the write traffic in the current second with the write traffic in the previous second. If the current write traffic is more than three times that of the previous second, a sudden increase in write traffic is considered to have occurred. These characteristics of the checkpoint phase are determined by the model saving process. During the checkpoint phase, the training framework writes the entire model's parameters to the storage device. The model parameters are typically a large contiguous block of memory, thus generating a large number of sequential write requests, resulting in very large I / O sizes and a rapid increase in write traffic within a short period.

[0052] The criteria for determining the inference phase are: random read ratio greater than 60%, the sum of small-granularity I / O and medium-granularity I / O ratios greater than 70%, and read request interval variance less than 0.05. Small-granularity I / O refers to I / O requests with sizes between 4KB and 64KB. A smaller read request interval variance indicates more stable arrival of inference requests. These characteristics of the inference phase are determined by the working method of the inference service. The inference service typically handles a large number of concurrent requests, each request requiring the reading of a portion of the model's parameters and input data, thus generating a large number of random read requests with relatively small I / O sizes and relatively stable request intervals.

[0053] The rule for determining the computation phase is: the total I / O traffic is below a preset baseline threshold. Total I / O traffic can refer to the number of I / O requests or the total number of bytes per unit time. The preset baseline threshold can be set to 5% of the device's maximum bandwidth. If the current total I / O traffic is below this threshold, the system is considered to be in the computation phase. At this time, the computation unit is performing intensive computational operations with almost no I / O requests. The characteristic of the computation phase is very low I / O traffic because the computation unit is performing operations such as matrix operations and does not need to access the storage device.

[0054] In alternative implementations, such as Figure 4 As shown, the feature vector is obtained by collecting at least one behavioral feature of the input / output I / O requests of the acquisition block device layer, including: S310: When an IO request is completed, atomically update the counts related to IO sequence, the counts related to IO size distribution, and the statistics related to IO request intervals; S320: At the end of the preset time window, calculate the sequential read ratio, sequential write ratio, random read ratio, and random write ratio based on the updated counts related to IO sequence, calculate the proportion of each type of IO based on the updated counts related to IO size distribution, and calculate the IO interval variance based on the updated statistical values ​​related to IO request interval. S330: Combine at least one of the sequential read ratio, the proportion of each type of IO, and the IO interval variance into the feature vector.

[0055] In this embodiment, at least one behavioral feature of input / output I / O requests at the block device layer is collected to obtain a feature vector. Specifically, the steps include: when an I / O request is completed, atomically updating the counts related to I / O sequence, the counts related to I / O size distribution, and the statistical values ​​related to I / O request intervals; at the end of a preset time window, calculating the sequential read ratio, sequential write ratio, random read ratio, and random write ratio based on the updated counts related to I / O sequence, calculating the proportion of each type of I / O based on the updated counts related to I / O size distribution, and calculating the I / O interval variance based on the updated statistical values ​​related to I / O request intervals; and combining at least one of the sequential read ratio, the proportion of each type of I / O, and the I / O interval variance into a feature vector.

[0056] It's important to note that updating the counter when an I / O request completes, rather than when it's submitted, is crucial for ensuring accuracy. When an I / O request is submitted, it hasn't been actually processed and may be canceled or fail for various reasons. Updating the counter at submission would lead to inaccurate values. Updating when the I / O request completes ensures that only successfully processed I / O requests are counted in the counter, thus improving the accuracy of the feature vector.

[0057] For example, the system can use atomic variables in C to implement atomic updates of the counter. For instance, the sequential read counter can be defined as a variable of type `atomic_t`, and when the sequential read counter needs to be incremented, the `atomic_inc` function can be used to perform the atomic increment operation. This way, even if multiple CPU cores call the `atomic_inc` function simultaneously, no counting error will occur. In addition to the `atomic_inc` function, the `atomic_add` function can be used to increment a specified value, the `atomic_sub` function to decrement a specified value, and the `atomic_read` function to read the current value of the counter.

[0058] In this embodiment, the counts related to IO sequence include sequential read count, random read count, sequential write count, and random write count. Each time an IO request completes, the system determines whether the request is sequential or random IO based on the request type (read or write) and offset, and then updates the corresponding count. For example, if the current request is a read request and the offset equals the previous IO offset plus the previous IO size, the sequential read count is incremented by 1; otherwise, the random read count is incremented by 1. It is important to note that the previous IO offset and the previous IO size also need to be updated using atomic operations to ensure consistency in a multi-threaded concurrent environment.

[0059] The counts related to I / O size distribution include the request count and byte count for each I / O size range. I / O size ranges can be divided into five ranges: very small, small, medium, large, and very large. Very small refers to less than 4KB, small to 4KB, medium to 64KB, large to 512KB, and very large to 4MB or more. Each time an I / O request completes, the system categorizes it into the corresponding range based on its size and updates the request count and byte count for that range. For example, if the current request size is 128KB, it belongs to medium-granularity I / O, and the system will increment the request count and byte count for medium-granularity I / O by 1.

[0060] The statistics related to the IO request interval include the completion time of the last IO request and the exponentially weighted moving average of the IO interval. Each time an IO request completes, the system records the current time, calculates the interval between the completion time of the last IO request and the current time, and then updates the average IO interval using an exponentially weighted moving average algorithm. The exponentially weighted moving average algorithm can smooth historical data and reduce the impact of instantaneous fluctuations. The formula for calculating the exponentially weighted moving average is: EWMA(t) = α * X(t) + (1-α) * EWMA(t-1), where X(t) is the current observation, EWMA(t-1) is the previous exponentially weighted moving average, and α is the smoothing coefficient, ranging from 0 to 1. A larger α indicates a greater weight for the current observation and a smaller weight for historical data; a smaller α indicates a smaller weight for the current observation and a larger weight for historical data. In this application, α can be set to 0.1, which can better balance the influence of current and historical data.

[0061] In this embodiment, at the end of the preset time window, the system calculates the feature vector based on the updated counter. The size of the preset time window can be set to 1 second, meaning the feature vector is calculated once every 1 second. The formula for calculating the sequential read ratio is: Sequential read ratio = Sequential read count / (Sequential read count + Random read count). Similarly, the sequential write ratio, random read ratio, and random write ratio can be calculated. The formula for calculating the proportion of each type of IO is: Proportion of a certain type of IO = Byte count of that type of IO / Total IO byte count. The formula for calculating the IO interval variance is: IO interval variance = E[X^2] - (E[X])^2, where E[X] is the average value of the IO intervals, and E[X^2] is the average value of the squared IO intervals. To calculate the IO interval variance, the system also needs to maintain the exponentially weighted moving average of the squared IO intervals, which is calculated in the same way as the exponentially weighted moving average of the IO intervals.

[0062] It's important to note that a feature vector can contain all of the aforementioned features, or only a subset of them. For example, in some simple applications, the sequential read ratio and I / O size distribution might be sufficient for accurate stage identification; in this case, the feature vector could contain only these two features. In complex applications, more features may be needed to improve accuracy; in this case, the feature vector could include more features, such as the read / write ratio and the average size of I / O requests. In practical applications, appropriate feature combinations can be selected to construct the feature vector based on the characteristics of the AI ​​task and the system's performance requirements.

[0063] In alternative implementations, such as Figure 5 As shown, adjusting the quality of service control parameters of the block device layer based on the identified current AI workload stage includes: S340: Based on the current AI workload stage, query the pre-stored stage-parameter mapping template to obtain the parameter set corresponding to the current AI workload stage; the parameter set includes burst capacity, smoothing suppression strength, and time slice granularity. S350: Atomically update the current parameters of the block device layer token bucket to the corresponding values ​​in the parameter group.

[0064] In this embodiment, adjusting the quality of service control parameters of the block device layer according to the identified current AI workload stage specifically includes the following steps: querying a pre-stored stage-parameter mapping template according to the current AI workload stage to obtain a parameter group corresponding to the current AI workload stage; the parameter group includes burst capacity, smoothing suppression strength, and time slice granularity; atomically updating the current parameters of the block device layer token bucket to the corresponding values ​​in the parameter group.

[0065] It's important to note that the stage-parameter mapping template is pre-defined based on the requirements of different AI workload stages. Each stage corresponds to a set of parameters, which are obtained through performance testing and optimization on a large number of AI tasks. In practical applications, the parameters in the mapping template can be adjusted according to the system's hardware configuration and the characteristics of the AI ​​task to achieve optimal performance. For example, for high-performance storage devices, the burst capacity of each stage can be appropriately increased; for inference tasks with high latency requirements, the smoothing suppression strength of the inference stage can be appropriately increased.

[0066] For example, a stage-parameter mapping template can be shown below: Data loading phase: burst capacity is 50% of the device's maximum bandwidth, smoothing suppression strength is 30, and time slice granularity is 10 milliseconds; Checkpoint phase: Burst capacity is 80% of the device's maximum bandwidth, smoothing suppression strength is 10, and time slice granularity is 10 milliseconds; Inference phase: Burst capacity is 10% of the device's maximum bandwidth, smoothing suppression strength is 90, and time slice granularity is 1 millisecond; Calculation phase: Burst capacity is 5% of the device's maximum bandwidth, smoothing suppression strength is 95, and time slice granularity is 1 millisecond.

[0067] The device's maximum bandwidth refers to the maximum bandwidth that the physical storage device can provide, such as 4GB per second. Burst capacity represents the maximum number of tokens that the token bucket can hold, measured in bytes. For example, if the device's maximum bandwidth is 4GB per second, then the burst capacity during the data loading phase is 2GB. Smoothing suppression strength is a value between 0 and 100; a higher value indicates stronger suppression and a more stable token generation rate, while a lower value indicates weaker suppression and a larger allowable burst traffic. Time slice granularity refers to the time interval between token generation by the token bucket, measured in milliseconds. For example, a time slice granularity of 10 milliseconds means that the token bucket generates a token every 10 milliseconds.

[0068] In this embodiment, the current parameters of the block device layer token bucket are atomically updated to the corresponding values ​​in the parameter group. Atomic parameter updates ensure that data races and inconsistencies do not occur during parameter updates in a multi-threaded concurrent environment. If the parameter updates are not atomic, a thread might read both new and old parameters, leading to abnormal token bucket behavior. For example, if one thread is reading token bucket parameters to process an I / O request while another thread is updating these parameters, and the update is not atomic, the first thread might read the new burst capacity and the old smoothing suppression strength, resulting in incorrect token bucket calculations.

[0069] For example, the system can store the token bucket parameters in a structure and then use an atomic write operation to update the entire structure. Alternatively, the system can use memory barriers to ensure the order and visibility of parameter updates. A memory barrier is a CPU instruction that guarantees that all memory operations before the memory barrier are completed before any memory operations after the memory barrier are executed. This way, when one thread updates the token bucket parameters, other threads can immediately see the updated parameters. In C, the `smp_mb` function can be used to insert memory barriers.

[0070] It should be noted that the parameter update latency should be as low as possible to ensure that the QoS policy can keep up with the phase changes of the AI ​​workload. In this application, the parameter update latency can be controlled within 10 microseconds, which is sufficient for flash memory devices with microsecond-level response times. The parameter update latency mainly includes parameter query time, atomic write operation time, and memory barrier overhead. These overheads are very small, thus ensuring that the parameter update latency is within an acceptable range.

[0071] In alternative implementations, such as Figure 6 As shown, after adjusting the quality of service control parameters of the block device layer, the method further includes: S360: Monitor the I / O request latency of the block device layer; S370: When the detected IO request latency continues to exceed the preset latency threshold for a preset duration, the service quality control parameters are reset to the preset conservative parameter group. S380: Mark the currently identified AI workload phase as a suspicious state.

[0072] In this embodiment of the application, after adjusting the quality of service control parameters of the block device layer, the following steps are also included: monitoring the IO request latency of the block device layer; when the monitored IO request latency is higher than the preset latency threshold for a preset duration, resetting the quality of service control parameters to a preset conservative parameter group; and marking the currently identified AI workload stage as a suspicious state.

[0073] It's important to note that monitoring I / O request latency is crucial for detecting errors in phase identification. If an error occurs, leading to improper QoS parameter configuration, I / O request latency will significantly increase. For example, if the inference phase is misidentified as the data loading phase, the system will configure larger burst capacities and weaker smoothing suppression, resulting in increased burst traffic and higher inference request latency. By monitoring I / O request latency, such errors can be detected promptly, allowing for appropriate corrective measures.

[0074] For example, the system can record the submission and completion times of each IO request and calculate the latency of each IO request. Then, an exponentially weighted moving average algorithm is used to calculate the average IO latency. A preset latency threshold can be set according to the requirements of the service level agreement, such as 500 microseconds. The preset duration can be set to 100 milliseconds; that is, if the average IO latency is higher than the preset latency threshold for 100 consecutive milliseconds, then a stage identification error is considered to have occurred. Choosing 100 milliseconds as the preset duration is to avoid false triggering of the error correction mechanism due to instantaneous latency fluctuations. If the preset duration is too short, occasional latency spikes may incorrectly reset QoS parameters; if the preset duration is too long, the impact of stage identification errors will last for a longer period, significantly affecting system performance.

[0075] In this embodiment, when the IO request latency is detected to be consistently higher than a preset latency threshold for a preset duration, the system resets the service quality control parameters to a preset conservative parameter set. The conservative parameter set typically refers to parameter configurations that ensure system stability, such as parameters used in the inference or computation phases. This prevents incorrect QoS parameters from further degrading system performance.

[0076] For example, a preset set of conservative parameters could be: burst capacity of 10% of the device's maximum bandwidth, smoothing suppression strength of 90, and time slice granularity of 1 millisecond. This set of parameters can effectively suppress burst traffic and ensure stable latency for I / O requests. In practical applications, the conservative parameter set can be adjusted according to the actual situation of the system to achieve the best balance between stability and performance.

[0077] In this embodiment, the currently identified AI workload stage is marked as a suspicious state. After being marked as suspicious, the system increases the feature sampling density in the next time window to improve the accuracy of stage identification. For example, if the original time window was 1 second, it can now be shortened to 500 milliseconds, allowing for more frequent collection of feature vectors for stage determination. When the system does not detect any latency anomalies within multiple consecutive time windows, the suspicious state can be canceled, and the original sampling density can be restored. For example, if the average IO latency is lower than a preset latency threshold for 5 consecutive time windows, the suspicious state can be canceled, and the time window can be restored to 1 second.

[0078] It should be noted that delay monitoring and error correction mechanisms complement the stage identification mechanism, improving the system's robustness and reliability. Even if errors occur in stage identification, the system can detect and correct them promptly, preventing serious impacts on system performance. In practical applications, delay monitoring and error correction mechanisms are essential because no stage identification algorithm can guarantee 100% accuracy; there will always be special cases that lead to identification errors.

[0079] In alternative implementations, such as Figure 7 As shown, at least one behavioral characteristic of the input / output I / O requests of the acquisition block device layer includes: S410: Based on multiple preset IO size ranges, the received IO requests are divided into corresponding ranges; S420: Maintain an independent statistical context for the behavioral characteristics for each of the IO size intervals; S430: Calculate the corresponding sub-feature vectors according to each statistical context, and use all sub-feature vectors together as the feature vector.

[0080] In this embodiment of the application, collecting at least one behavioral feature of input / output IO requests from the block device layer specifically includes the following steps: dividing the received IO requests into corresponding intervals according to a plurality of preset IO size intervals; maintaining an independent statistical context for behavioral features for each IO size interval; calculating the corresponding sub-feature vectors according to each statistical context, and using each sub-feature vector as a feature vector.

[0081] It's important to note that this feature acquisition method is primarily used to handle the I / O features of multimodal AI tasks. Multimodal tasks typically process multiple types of data simultaneously, such as text, images, and videos, each with different I / O characteristics. For example, text data is usually small-granularity random reads, while video data is usually large-granularity sequential reads. If these different types of I / O requests are mixed together for statistical analysis, the resulting feature vector may not accurately reflect the characteristics of any single type of I / O, leading to errors in stage identification. By dividing requests into I / O size ranges and maintaining an independent statistical context for each range, different types of sub-streams can be identified separately, thereby improving the accuracy of stage identification.

[0082] For example, the preset IO size ranges can be divided into: less than 4KB, 4KB to 64KB, 64KB to 512KB, 512KB to 4MB, and greater than or equal to 4MB. The system maintains an independent statistical context for each range. Each statistical context contains counts related to IO sequence, counts related to IO size distribution, and statistical values ​​related to IO request intervals. Each time an IO request arrives, the system categorizes it into the corresponding range based on its size and then updates the statistical context for that range. For instance, if the current request size is 2MB, it belongs to the 512KB to 4MB range, and the system updates the statistical context for that range; if the current request size is 8MB, it belongs to the greater than or equal to 4MB range, and the system updates the statistical context for that range.

[0083] In this embodiment, sub-feature vectors are calculated based on each statistical context. The calculation method for each sub-feature vector is the same as that for the previously described feature vectors, i.e., features such as sequential read ratio, IO size distribution ratio, and IO interval variance are calculated based on the counters in the statistical context. Then, all sub-feature vectors are combined to form the final feature vector. For example, if the system has 5 IO size intervals, then 5 sub-feature vectors will be calculated, and the final feature vector is the combination of these 5 sub-feature vectors.

[0084] For example, in a multimodal training task, text data I / O requests are mainly concentrated in the range of less than 4KB and 4KB to 64KB, exhibiting small-granularity random reads; video data I / O requests are mainly concentrated in the range of greater than or equal to 4MB, exhibiting large-granularity sequential reads. The system calculates sub-feature vectors for these two ranges respectively. The sub-feature vectors for the ranges of less than 4KB and 4KB to 64KB satisfy the judgment rules for the inference stage, while the sub-feature vectors for the range of greater than or equal to 4MB satisfy the judgment rules for the data loading stage. The system can configure different QoS parameters for the text sub-stream and video sub-stream based on these two sub-feature vectors. For example, a larger burst capacity can be configured for the video sub-stream to ensure the loading speed of video data, while a smaller burst capacity can be configured for the text sub-stream to ensure the reading latency of text data.

[0085] It's important to note that the sub-stream separation mechanism can be flexibly configured according to actual needs. For example, sub-stream separation can be based on the type of I / O (read or write) and the source of the I / O (different processes or containers), rather than just the I / O size range. In practical applications, the appropriate sub-stream separation method can be selected based on the characteristics of the AI ​​task. For instance, for multi-tenant shared storage systems, sub-stream separation can be performed based on different tenants, maintaining independent statistical contexts and QoS parameters for each tenant; for AI tasks with mixed read and write operations, sub-stream separation can be performed based on the type of I / O, configuring different QoS parameters for read and write requests respectively.

[0086] In alternative implementations, such as Figure 8 As shown, atomically updating the current parameters of the block device layer token bucket to the corresponding values ​​in the parameter group includes: S440: When the new burst capacity in the parameter group is less than the available token amount in the current token bucket, determine the excess token amount that exceeds the new burst capacity; S450: In each time period, the available token amount is reduced at a preset decay rate until the available token amount does not exceed the new burst capacity; During the decay of the available token quota, the block device layer token bucket still uses the new burst capacity as a parameter.

[0087] In this embodiment, the current parameter of the block device layer token bucket is atomically updated to the corresponding value in the parameter group, specifically including the following steps: when the new burst capacity in the parameter group is less than the available token amount of the current token bucket, the excess token amount exceeding the new burst capacity is determined; in each time period, the available token amount is decayed at a preset decay rate until the available token amount does not exceed the new burst capacity; wherein, during the decay of the available token amount, the block device layer token bucket still uses the new burst capacity as the parameter.

[0088] It's important to note that when the token bucket's burst capacity is adjusted from a large value to a small value, if the available tokens in the current token bucket exceed the new burst capacity, directly truncating the available tokens to make up the new burst capacity would cause a large number of I / O requests to be suddenly blocked, resulting in I / O spikes and impacting system performance. To avoid this, this application introduces a smooth migration logic, gradually decaying and releasing excess tokens at a certain rate, rather than truncating them all at once. This ensures a smooth transition in the I / O request sending rate, preventing sudden drops and thus avoiding I / O spikes.

[0089] For example, suppose the current token bucket burst capacity is 2GB and the available token amount is 1.8GB. Now, the system adjusts the burst capacity to 1GB. At this point, the available token amount of 1.8GB is greater than the new burst capacity of 1GB, with an excess of 0.8GB. The system sets a decay rate, for example, decaying by 10% every time period. The time period can be set to 10 milliseconds, meaning it decays once every 10 milliseconds. In the first 10 milliseconds, the system will decay the available token amount by 10%, i.e., 1.8GB * 10% = 0.18GB, resulting in a decayed available token amount of 1.62GB. In the second 10 milliseconds, the system will decay the available token amount by another 10%, i.e., 1.62GB * 10% = 0.162GB, resulting in a decayed available token amount of 1.458GB. This continues until the available token amount is no greater than 1GB. Throughout the decay process, the token bucket still uses the new burst capacity of 1GB as a parameter; that is, the token generation rate is calculated based on the new burst capacity.

[0090] It's important to note that the decay rate can be set according to actual needs. A higher decay rate results in faster decay of available tokens and a shorter smooth migration time, but may cause some IO fluctuations. Conversely, a lower decay rate results in slower decay of available tokens and a longer smooth migration time, but with less IO fluctuation. In practical applications, an appropriate decay rate can be selected based on the system's performance requirements. For example, the decay rate can be set between 5% and 20% per time period. For systems sensitive to IO fluctuations, a lower decay rate, such as 5%, can be chosen; for systems with high requirements for smooth migration time, a higher decay rate, such as 20%, can be chosen.

[0091] In an optional implementation, the consecutive preset number of time windows is at least three time windows.

[0092] In this embodiment of the application, the number of consecutive preset time windows is at least three time windows.

[0093] It's important to note that the requirement of at least three time windows for the hysteresis mechanism is to strike a balance between real-time performance and stability. Requiring only one time window results in high real-time performance for stage recognition, but poor stability, making it susceptible to instantaneous fluctuations. Requiring two time windows improves stability somewhat, but it's still vulnerable to two consecutive instantaneous fluctuations. Requiring three or more time windows significantly improves stability because the probability of the same instantaneous fluctuation occurring in three consecutive time windows is extremely low. Simultaneously, the delay from three time windows is not excessive and is acceptable for most AI tasks.

[0094] For example, if the time window size is 1 second, then the total delay for three time windows is 3 seconds. This means that when a phase of an AI task changes, the system needs a maximum of 3 seconds to recognize the phase change and adjust the QoS parameters. For most AI tasks, a 3-second delay is acceptable because each phase of an AI task typically lasts tens of seconds or even minutes. For example, the data loading phase for training a large model might last several minutes, the checkpointing phase might last tens of seconds, and the inference phase might last several hours. Therefore, a 3-second recognition delay will not significantly impact system performance.

[0095] It should be noted that in some applications with high real-time requirements, the number of consecutive time windows can be appropriately reduced, for example, to two time windows. In some applications with extremely high stability requirements, the number of consecutive time windows can be appropriately increased, for example, to four or five time windows. In practical applications, the appropriate number of consecutive time windows can be selected based on the system's performance requirements and the characteristics of the AI ​​task. For example, for online inference services with high real-time requirements, two time windows can be set; for offline training tasks with high stability requirements, four time windows can be set.

[0096] It is understood that embodiments of this application further provide a block device layer adaptive quality of service scheduling system, including: The acquisition module acquires at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector; The input module inputs the feature vector into the state machine and identifies the AI ​​workload stage corresponding to the current IO request based on the hysteresis mechanism. The state machine refers to a computational model that maintains a set of states corresponding to the semantic stages of the AI ​​task and defines the conditions for transitions between states. The hysteresis mechanism refers to a mechanism that requires the candidate stage to be consistently determined within multiple consecutive time windows before triggering the state transition of the state machine. The AI ​​workload stage refers to the semantic stage of the upper-layer artificial intelligence task inferred from the IO behavior of the block device layer. The scheduling module adjusts the quality of service control parameters of the block device layer based on the identified current AI workload stage in order to schedule subsequent IO requests.

[0097] Those skilled in the art will understand that the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. Further details are omitted here.

Claims

1. A block device layer adaptive quality of service scheduling method, characterized in that, include: Collect at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector; The feature vector is input into the state machine, and the AI ​​workload stage corresponding to the current IO request is identified according to the hysteresis mechanism. The state machine refers to a computational model that maintains a set of states corresponding to the semantic stages of the AI ​​task and defines the conditions for transitions between states. The hysteresis mechanism refers to a mechanism that requires the alternative stages to be consistently determined within multiple consecutive time windows before triggering the state transition of the state machine. The AI ​​workload stage refers to the semantic stage of the upper-layer artificial intelligence task inferred from the IO behavior of the block device layer. Based on the identified current AI workload stage, adjust the quality of service control parameters of the block device layer to schedule subsequent IO requests; The step of inputting the feature vector into the state machine and identifying the AI ​​workload stage corresponding to the current IO request based on the hysteresis mechanism includes: The candidate AI workload stages are determined by matching the feature vectors with preset stage determination rules. Determine whether the same candidate AI workload phase is continuously identified within a preset number of consecutive time windows; When the judgment result is yes, the state machine is triggered to transition from the current state to a new state corresponding to the candidate AI workload stage, and the stage corresponding to the new state is taken as the current AI workload stage. The step of matching the feature vector with preset stage determination rules to determine candidate AI workload stages includes: Based on the sequential read ratio, IO size distribution ratio, and IO interval variance in the feature vector, the candidate AI workload stage is determined to be the data loading stage; Based on the sequential write ratio, IO size distribution ratio, and write traffic increase in the feature vector, the candidate AI workload stage is determined as the checkpoint stage; Based on the random read ratio, IO size distribution ratio, and read request interval variance in the feature vector, the candidate AI workload stage is determined to be the inference stage; Based on the comparison between the total IO traffic in the feature vector and the preset baseline threshold, the candidate AI workload stage is determined to be the computation stage; The feature vector is obtained by collecting at least one behavioral feature of the input / output I / O requests of the acquisition block device layer, including: When an IO request is completed, atomically update the counts related to IO sequence, the counts related to IO size distribution, and the statistics related to IO request intervals; At the end of the preset time window, calculate the sequential read ratio, sequential write ratio, random read ratio, and random write ratio based on the updated counts related to IO sequence; calculate the proportion of each type of IO based on the updated counts related to IO size distribution; and calculate the IO interval variance based on the updated statistical values ​​related to IO request interval. The feature vector is formed by combining at least one of the sequential read ratio, the proportion of each type of I / O, and the I / O interval variance.

2. The method according to claim 1, characterized in that, The step of adjusting the quality of service control parameters of the block device layer based on the identified current AI workload stage includes: Based on the current AI workload stage, query the pre-stored stage-parameter mapping template to obtain the parameter set corresponding to the current AI workload stage; the parameter set includes burst capacity, smoothing suppression strength and time slice granularity. The current parameters of the block device layer token bucket are atomically updated to the corresponding values ​​in the parameter group.

3. The method according to any one of claims 1 to 2, characterized in that, After adjusting the quality of service control parameters of the block device layer, the method further includes: Monitor the I / O request latency of the block device layer; When the detected IO request latency remains higher than a preset latency threshold for a preset duration, the service quality control parameters are reset to a preset conservative parameter set. The currently identified AI workload phase is marked as suspicious.

4. The method according to any one of claims 1 to 2, characterized in that, At least one behavioral characteristic of the input / output I / O requests of the acquisition block device layer includes: Based on multiple preset IO size ranges, the received IO requests are divided into corresponding ranges; Maintain an independent statistical context for each of the aforementioned IO size ranges; Each sub-feature vector is calculated based on its respective statistical context, and all sub-feature vectors are used together as the feature vector.

5. The method according to claim 2, characterized in that, The step of atomically updating the current parameters of the block device layer token bucket to the corresponding values ​​in the parameter group includes: When the new burst capacity in the parameter group is less than the available token amount in the current token bucket, the excess token amount exceeding the new burst capacity is determined. Within each time period, the available token amount is reduced at a preset decay rate until the available token amount does not exceed the new burst capacity; During the decay of the available token quota, the block device layer token bucket still uses the new burst capacity as a parameter.

6. The method according to claim 1, characterized in that, The number of consecutive preset time windows is at least three time windows.

7. A block device layer adaptive quality of service scheduling system, characterized in that, include: The acquisition module acquires at least one behavioral feature of input / output I / O requests from the block device layer to obtain a feature vector; The input module inputs the feature vector into the state machine and identifies the AI ​​workload stage corresponding to the current IO request based on the hysteresis mechanism. The state machine refers to a computational model that maintains a set of states corresponding to the semantic stages of the AI ​​task and defines the conditions for transitions between states. The hysteresis mechanism refers to a mechanism that requires the candidate stage to be consistently determined within multiple consecutive time windows before triggering the state transition of the state machine. The AI ​​workload stage refers to the semantic stage of the upper-layer artificial intelligence task inferred from the IO behavior of the block device layer. The scheduling module adjusts the quality of service control parameters of the block device layer based on the identified current AI workload stage to schedule subsequent IO requests; wherein... The input module is specifically used for: The candidate AI workload stages are determined by matching the feature vectors with preset stage determination rules. Determine whether the same candidate AI workload phase is continuously identified within a preset number of consecutive time windows; When the judgment result is yes, the state machine is triggered to transition from the current state to a new state corresponding to the candidate AI workload stage, and the stage corresponding to the new state is taken as the current AI workload stage. The step of matching the feature vector with preset stage determination rules to determine candidate AI workload stages includes: Based on the sequential read ratio, IO size distribution ratio, and IO interval variance in the feature vector, the candidate AI workload stage is determined to be the data loading stage; Based on the sequential write ratio, IO size distribution ratio, and write traffic increase in the feature vector, the candidate AI workload stage is determined as the checkpoint stage; Based on the random read ratio, IO size distribution ratio, and read request interval variance in the feature vector, the candidate AI workload stage is determined to be the inference stage; Based on the comparison between the total IO traffic in the feature vector and the preset baseline threshold, the candidate AI workload stage is determined to be the computation stage; The feature vector is obtained by collecting at least one behavioral feature of the input / output I / O requests of the acquisition block device layer, including: When an IO request is completed, atomically update the counts related to IO sequence, the counts related to IO size distribution, and the statistics related to IO request intervals; At the end of the preset time window, calculate the sequential read ratio, sequential write ratio, random read ratio, and random write ratio based on the updated counts related to IO sequence; calculate the proportion of each type of IO based on the updated counts related to IO size distribution; and calculate the IO interval variance based on the updated statistical values ​​related to IO request interval. The feature vector is formed by combining at least one of the sequential read ratio, the proportion of each type of I / O, and the I / O interval variance.

Images