Storage task scheduling method and computing device

Abstract

Embodiments of the present application relate to a storage task scheduling method and a computing device. The method comprises: obtaining storage tasks in a storage system, and dividing the storage tasks according to priorities; monitoring an IO pressure index of the storage system; when the IO pressure index is greater than a preset pressure threshold, reducing resource allocation to low-priority storage tasks in the storage system, so as to guarantee resource requirements of high-priority storage tasks in the storage system; and when the IO pressure index is not greater than the preset pressure threshold, driving execution of the low-priority storage tasks based on a system idle event. In this way, when the storage system is overloaded, resources are tilted to storage tasks with higher priorities, which can avoid problems that may be caused by directly suspending low-priority storage tasks, such as long-term stagnation of system background tasks. When the storage system is in an idle time, low-priority storage tasks can be processed, so that the long-term health and stability of the storage system can be maintained.

Description

Technical Field

[0001] This application relates to the field of server technology, and in particular to a storage task scheduling method and computing device. Background Technology

[0002] When storage systems face overload pressure, some related flow control solutions often adopt relatively simple and rigid strategies. For example, they prioritize user input / output (IO) services by directly suspending background tasks within the system (such as data migration and space reclamation).

[0003] However, this either-or control approach of related technologies may cause system background tasks to stagnate for a long time, affecting the health of the storage system and easily causing fluctuations in business performance. Summary of the Invention

[0004] This application provides a storage task scheduling method and computing device that can maintain the health and stability of the storage system.

[0005] According to a first aspect of the embodiments of this application, a storage task scheduling method is provided, the method comprising: The system acquires storage tasks from the storage system and categorizes them according to priority; it monitors the IO pressure metrics of the storage system; when the IO pressure metrics exceed a preset pressure threshold, it reduces resource allocation to low-priority storage tasks in the storage system to ensure the resource requirements of high-priority storage tasks; when the IO pressure metrics do not exceed the preset pressure threshold, it drives the execution of low-priority storage tasks based on system idle events; where low-priority storage tasks refer to storage tasks with a priority lower than a first preset priority, and high-priority storage tasks refer to storage tasks with a priority higher than the first preset priority.

[0006] This solution can allocate resources to higher-priority storage tasks when the storage system is overloaded, avoiding the problem of long-term stagnation of background tasks caused by directly pausing the processing of lower-priority storage tasks. When the storage system is idle, it can handle low-priority storage tasks, thereby maintaining the long-term health and stability of the storage system.

[0007] In one possible implementation, storage tasks are divided into external tasks with a first priority and internal tasks with a second priority; external tasks are user read / write I / O tasks, and internal tasks are system maintenance tasks, with the first priority being higher than the second priority.

[0008] This scheme, through its clear priority division, provides a clear strategic basis for subsequent dynamic resource scheduling based on system pressure, ensuring that external tasks such as user I / O are always given priority when resources are scarce.

[0009] In one possible implementation, the internal tasks are further divided into several different sub-priorities. Among the internal tasks, data repair, data migration, garbage collection, and data flushing have higher priority than bad sector scanning and data consistency verification.

[0010] This solution establishes a multi-layered internal task prioritization system, enabling the storage system to perform more granular scheduling when resources are scarce. When it is necessary to reduce internal task resources, the storage system can prioritize ensuring the minimum operational requirements of high-priority sub-tasks (such as data repair), while first limiting or suspending the execution of low-priority sub-tasks (such as bad sector scanning). This ensures a good user IO experience while maximizing the maintenance of the core health of the storage system.

[0011] In one possible implementation, resource allocation is based on a license mechanism, whereby a license represents a virtual credential that allows the execution of a single I / O operation. When the IO pressure metric exceeds a preset pressure threshold, reduce resource allocation to low-priority storage tasks in the storage system, including: Assign licenses to pending I / O operations; When the IO pressure index exceeds the preset pressure threshold, reduce the rate or number of licenses issued to low-priority storage tasks in the storage system.

[0012] This solution achieves precise flow control of IO operations through a licensing mechanism, which can quickly and smoothly limit background task resources when the storage system is overloaded, so as to ensure the performance of high-priority storage tasks such as front-end business.

[0013] In one possible implementation, the method further includes: If the number of licenses is less than the threshold, the allocation of licenses for I / O operations corresponding to internal tasks with a priority lower than the second preset priority will be suspended.

[0014] When license resources become scarce to a certain extent, this scheme completely stops supplying licenses to the lowest priority internal tasks, preventing their I / O operations from being executed. This allows the extremely limited system resources to be concentrated on ensuring user I / O tasks and critical internal maintenance tasks. This enables optimal resource allocation decisions under extreme pressure on the storage system.

[0015] In one possible implementation, the method further includes: For low-priority storage tasks, I / O operations that have not obtained a license are either deferred or discarded.

[0016] This solution ensures that storage system resources are concentrated on executing high-priority tasks to the maximum extent possible, while avoiding queue congestion that may be caused by excessive backlog of low-priority requests. By deferring or dropping I / O operations for non-critical tasks, the system can maintain the normal execution of core business operations even under overload conditions.

[0017] In one possible implementation, the method further includes: When the IO pressure metric drops from above the preset pressure threshold to below the preset pressure threshold, resource allocation for low-priority storage tasks is restored.

[0018] This solution enables the storage system to achieve dynamic balance during stress cycles, ensuring critical business operations are protected during overload and timely restoration of system maintenance functions when resources are plentiful, thus guaranteeing the long-term healthy operation of the storage system.

[0019] According to a second aspect of the embodiments of this application, a storage task scheduling apparatus is provided, the apparatus comprising: The task receiving and classification module is used to obtain storage tasks in the storage system and classify the storage tasks according to priority. The stress monitoring module is used to monitor the I / O stress indicators of the storage system; The task scheduling and execution module is used to reduce the resource allocation of low-priority storage tasks in the storage system when the IO pressure index is greater than the preset pressure threshold, so as to ensure the resource requirements of high-priority storage tasks in the storage system; when the IO pressure index is not greater than the preset pressure threshold, the module drives the execution of the low-priority storage tasks based on the system idle event. Among them, low-priority storage tasks refer to storage tasks with a priority lower than the first preset priority, and high-priority storage tasks refer to storage tasks with a priority higher than the first preset priority.

[0020] According to a third aspect of the embodiments of this application, a computing device is provided. The computing device includes a memory and a processor, the memory storing a computer program, and the processor executing the program to implement the method as described above.

[0021] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the methods described in the embodiments of this application.

[0022] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above in the embodiments of this application. Attached Figure Description

[0023] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1A A schematic diagram of the architecture of a storage system provided in an exemplary embodiment of this application; Figure 1B A schematic diagram of the license generation module and its controlled IO processing flow provided for an exemplary embodiment of this application; Figure 2 A flowchart illustrating a storage task scheduling method provided in an exemplary embodiment of this application; Figure 3 A flowchart of a license issuance method provided for an exemplary embodiment of this application; Figure 4 A flowchart of a storage task scheduling method provided in an exemplary embodiment of this application; Figure 5 A schematic block diagram of the functional modules of a storage task scheduling apparatus provided in an exemplary embodiment of this application; Figure 6 A structural block diagram of a computing device provided for an exemplary embodiment of this application. Detailed Implementation

[0024] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0025] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0027] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more". The embodiments provided in this application can be widely applied in enterprise-level distributed storage scenarios with high requirements for business continuity and system stability. When the storage cluster faces peak business pressure, it can intelligently identify overload states and, through a dynamic priority resource scheduling mechanism, prioritize the performance of critical user IO tasks such as database backup and cloud disk read / write, while smoothly limiting or postponing system maintenance tasks such as garbage collection and bad sector scanning, effectively avoiding business performance fluctuations. During off-peak business periods, the system automatically switches to idle drive mode, making full use of idle resources to accelerate backend maintenance, significantly improving the resource utilization efficiency and long-term health of the storage system, and achieving an optimal balance between business experience and system operation and maintenance efficiency.

[0028] In one specific embodiment provided in this application, such as Figure 1A As shown, Figure 1A This is a schematic diagram of the architecture of a storage system provided in an embodiment of this application. The storage system 10 may include: The task receiving and classification module 11 is used to receive user IO requests from the application server and maintenance tasks generated within the storage system 10.

[0029] The task queue management module 12 is used to manage the task queue, which can include storage tasks for external tasks and internal tasks. The external task queue stores user read / write I / O tasks, while the internal queue can contain multiple queues, each storing system maintenance tasks with different priorities, such as repair, migration, GC, and bad sector scanning.

[0030] The stress monitoring module 13 is used to continuously monitor the performance indicators of the storage system, such as average IO latency and long-tail latency ratio.

[0031] The license generation and management module 14 is used to dynamically generate and allocate licenses based on system stress levels.

[0032] The task scheduling and execution module 15 is used to retrieve licensed IO requests from various queues and hand them over to the underlying storage resources for processing.

[0033] Specifically, in the task reception and classification process of the embodiment, for example, when the application server issues a data write request, after the task reception and classification module 11 of the storage system 10 receives the data write request, the task queue management module 12 will identify the data write request as a user IO task with the highest priority according to the source (such as from an external client) and type (such as a user read / write operation), and put it into the corresponding external queue for processing. At the same time, the garbage collection process inside the system generates a data block reclamation task. This task is identified as an internal task and is put into the corresponding internal queue according to its priority (higher than bad sector scanning but lower than user IO tasks).

[0034] In this embodiment, during system stress monitoring and assessment, the stress monitoring module 13 continuously collects the system's I / O response data. At a certain moment, for example, the following situation is detected: The average latency of user I / O requests has risen to 150ms, exceeding the preset threshold of 100ms.

[0035] The proportion of IO requests with latency exceeding 2 seconds reached 1.5‰, exceeding the threshold of 1‰.

[0036] When the pressure monitoring module 13 determines that the storage system 10 is currently under overload pressure, it sends a pressure overload signal to the license generation and management module 14.

[0037] In the embodiment, during the dynamic license allocation process, the license generation and management module 14 can activate a dynamic control mechanism after receiving a pressure overload signal: Safeguarding external tasks: The license generation and management module 14 maintains or even prioritizes the rate at which licenses are issued to external queues, ensuring that user IO tasks can be processed continuously and quickly, thereby safeguarding the experience of front-end services.

[0038] Restricting internal tasks: The license generation and management module 14 begins to significantly or gradually reduce the rate or number of licenses issued to each internal task queue; the degree of resource reduction is proportional to the system pressure.

[0039] Internal task priority scheduling: Within internal tasks, license allocation follows a preset priority. For example, data migration tasks (higher priority internal tasks) can still obtain a small number of licenses to maintain minimal operation, while license allocation for bad sector scanning tasks (lowest priority internal tasks) is suspended, and their corresponding IO requests are postponed or discarded, thereby freeing up system resources (such as CPU cycles and IO bandwidth) for higher priority tasks.

[0040] In this embodiment, during task execution, the task scheduling and execution module 15 processes IO requests that have successfully obtained licenses. Therefore, under the current overload state, user IO tasks in the external queue are processed efficiently, while in the internal queue, a small number of high-priority internal tasks (such as migration) are executed, and low-priority tasks (such as bad sector scanning) are temporarily suspended.

[0041] In the implementation example, during the pressure reduction and recovery phase, as the peak business period passed, the pressure monitoring module 13 detected that the average latency had dropped to 80ms (below the 100ms threshold), and the long-tail latency ratio also returned to normal. The license generation and management module 14 then gradually restored the rate of issuing licenses to the internal task queue, first restoring normal supply to high-priority internal tasks, and then restoring supply to low-priority tasks, so that all system functions gradually returned to normal operation, forming a complete closed-loop control.

[0042] This embodiment of the application, through the aforementioned fine-grained scheduling mechanism based on real-time pressure awareness and dynamic license allocation, prioritizes the IO performance of critical user services during system overload, avoiding service jitter and interruptions. Internal tasks are handled differently rather than being completely suspended, ensuring user experience while maintaining the healthy operation of system background functions as much as possible. The overall scheduling process in this embodiment is adaptive and dynamic, requiring no manual intervention from administrators, effectively improving the system's automated operation and maintenance level and resource utilization efficiency.

[0043] Based on the above embodiments, the task queue management module 12 is the core component for realizing dynamic scheduling. It can adopt a hierarchical queue structure, specifically including an external queue 201 and an internal queue 202.

[0044] The external queue is used to receive and process user-initiated read / write I / O tasks. As shown in Table 1, the external queue contains multiple sub-queues (e.g., queue 1, queue 2... queue N), each corresponding to an independent client or application session. This design ensures that I / O requests from different users or applications are isolated at the entry point, laying the foundation for subsequent client-based resource allocation and quality assurance. All tasks in the external queue are designated as having the highest priority by the system.

[0045] Internal queues are used to receive and process maintenance and management tasks for the storage system itself. These internal queues are further divided into multiple priority sub-queues (e.g., queue a, queue b, ..., queue z) based on the task's impact on system data health and service continuity. For example, in a preferred embodiment, critical maintenance tasks such as data repair and garbage collection are located in high-priority internal queues (e.g., queue a), while non-urgent tasks such as bad sector scanning and consistency checks are located in low-priority internal queues (e.g., queue z).

[0046] In this embodiment, after receiving an IO request, the task receiving and classification module 11 can first determine the source and type of the IO request. If it is a read / write request from a user client, the IO request is routed to the corresponding sub-queue in the external queue; if it is a maintenance task initiated internally by the storage system 10, it is routed to the sub-queue with the corresponding priority in the internal queue according to the type of the IO request. When the license generation and management module 14 is executed, it can prioritize processing storage tasks in the external queue. When the pressure monitoring module 13 indicates that the storage system 10 is under high load, the license generation and management module 14 will dynamically limit the rate at which tasks are retrieved from the internal queue for processing, and prioritize the resource allocation of high-priority internal queues, suspending or slowing down task processing in low-priority internal queues (such as queue z).

[0047] Table 1:

[0048] Based on the above embodiments, in another embodiment provided in this application, the license generation and management module 14 may further include a license generation module, such as... Figure 1B As shown, this process clarifies the configuration, operation, and direct control over I / O processing of the license generation module, specifically including the following steps: (1) Set the rate.

[0049] This stage involves the initialization or dynamic adjustment of the control flow. The system administrator or upper-level scheduling policy sets a specific license generation rate for the license generation module. This rate determines the number of virtual credentials available for allocation per unit of time and is a direct control point for regulating the system's I / O throughput. The rate can be differentiated based on the priority of different task queues and the overall system load.

[0050] (2) License generation module.

[0051] This module is the core execution unit of the process, responsible for continuously and stably generating processing permissions (i.e., licenses) according to the set rate. Each license represents a permission to perform a single I / O operation.

[0052] (3) Uniform generation processing permission.

[0053] This step describes the specific operating mode of the license generation module. The module generates and processes licenses at a uniform and periodic rate, as set above. This uniform generation mode creates a stable license supply flow on a macroscopic level, providing a predictable resource supply baseline for the system and serving as a crucial mechanism to avoid traffic bursts and fluctuations. The generated licenses are stored in their corresponding license pools, awaiting retrieval by I / O requests.

[0054] (4) IO processing requests.

[0055] A single I / O processing request from within the application or system arrives at the scheduling point. This request attempts to obtain a processing license to gain execution rights.

[0056] (5) Decision and branching.

[0057] The system determines whether the I / O processing request successfully obtained a processing license from the license pool. Based on the determination result, the process proceeds to the following different branches: Permission to process I / O: If the request is successfully granted permission, it is authorized to use the underlying storage resources (such as disk I / O, CPU time slices) and can be executed immediately or placed in the ready queue to wait for execution.

[0058] I / O processing is prohibited, and requests are dropped: If a request fails to acquire a license (usually because the corresponding license pool is empty and a new license has not yet been generated), processing of the request is explicitly prohibited. To prevent queue congestion and resource waste, the I / O request is dropped directly. This mechanism is key to implementing hard flow control, ensuring that the actual processing volume of the system never exceeds the preset license supply rate, thereby protecting system resources during overload.

[0059] This embodiment allows for precise control of the maximum bandwidth of a specific task queue by setting the rate, achieving quantitative management of resource allocation. Generating licenses at a uniform rate avoids sudden issuance, helps smooth system traffic processing, and reduces performance fluctuations caused by uneven resource allocation.

[0060] Based on the above embodiments, in another embodiment provided in this application, such as Figure 2 The above, Figure 2 This is a flowchart illustrating a storage task scheduling method provided in an embodiment of this application. Figure 2 This paper details the process of resource allocation through a licensing mechanism, from perceiving overload levels to dynamically adjusting license generation speed and arbitrating licenses for each I / O request in a real-time, closed-loop manner. Specifically, this method may include the following steps: Step S201: Obtain the overload level.

[0061] At the start of the process, the implementation initiates a license acquisition process for a new IO processing request.

[0062] The implementation can obtain the current global overload level, which can reflect the overall stress state of the storage system, such as normal, light overload, or heavy overload, and determine the overall scarcity of licenses.

[0063] Step S202: Adjust the speed at which licenses are generated.

[0064] The license generation speed can be dynamically adjusted based on the obtained overload level.

[0065] If the storage system is in normal condition, the license will be generated at the baseline speed, indicating that resources are sufficient.

[0066] If the storage system is overloaded, the rate of license generation is reduced, especially for license supply to low-priority task queues, thereby limiting their I / O processing rate.

[0067] It should be noted that the high priority or low priority mentioned in the embodiments is relative. After dividing the storage task into multiple different priorities, the storage task with a priority higher than a certain priority can be determined as a high priority storage task, while the storage task with a priority no higher than a certain priority can be determined as a low priority storage task.

[0068] Step S203: IO license processing.

[0069] Licenses can be issued to high-priority storage tasks based on the rate at which licenses are generated.

[0070] Step S204: IO processing request.

[0071] A specific I / O request, for example, waiting to be processed when it arrives from an external or internal queue.

[0072] Step S205: Obtain data processing permission.

[0073] An IO request can attempt to obtain a license from a license pool that corresponds to its task type and priority.

[0074] Step S206: Determine whether a license has been obtained.

[0075] Determine whether the IO request successfully obtained the license.

[0076] If yes, the process proceeds to step S207. If no, the process proceeds to step S205.

[0077] Step S207: IO processing.

[0078] If the I / O request successfully obtains a license, it can be authorized to use system resources (such as CPU or I / O channels). The license generation and management module can retrieve the I / O request and perform specific read and write operations.

[0079] If a license is not granted due to storage system resource constraints (overload) and the I / O request's priority is insufficient to win the competition, the storage system will prohibit processing the I / O request and either discard or postpone it. This process primarily targets low-priority internal tasks to ensure resources are allocated to high-priority tasks.

[0080] In this embodiment, if the processing flow of the current IO request ends (whether it is successfully executed or discarded), the storage system immediately prepares to process the next IO request, and the flow returns to S204, forming a continuous processing loop.

[0081] Meanwhile, the storage system's stress monitoring module will continue to run in the background, for example, periodically checking the stress status of the storage system to ensure that the resource allocation strategy can be updated in real time according to the stress status of the storage system.

[0082] In this embodiment, by ensuring that licensed IO requests are executed and that the license generation rate is dynamically adjusted according to the storage system load, the control intensity is matched to the level of overload. This embodiment achieves control precision at the level of a single IO request, enabling fine-grained traffic processing and avoiding coarse limitations on the entire task queue, effectively preventing performance fluctuations in the storage system.

[0083] Based on the above embodiments, in another embodiment provided in this application, such as Figure 3 The above, Figure 3 A flowchart for license issuance provided in this application embodiment may specifically include the following steps: Step S301: Start / Wait for cycle trigger.

[0084] In this embodiment, the process begins or waits for the next scheduling cycle to arrive. The generation and allocation of licenses is a continuously running closed-loop process that can periodically evaluate and respond to the state of the storage system.

[0085] Step S302: Obtain the current system pressure status.

[0086] For example, the latest system stress metrics data can be obtained from the stress monitoring module, such as the average latency of IO requests or the proportion of long-tail latency.

[0087] Step S303: Determine whether the system pressure exceeds the threshold.

[0088] The example compares the acquired stress indicators with preset stress thresholds (such as average latency > 100ms) to determine whether the storage system is in normal control mode or overload control mode.

[0089] Step S304: License allocation strategy under normal pressure.

[0090] If the system pressure does not exceed the threshold (i.e., S303 determines "No"), the allocation strategy for the normal state is executed. In this state, the license generation and management module will issue licenses to all task queues (including external queues and all internal queues) at the maximum or baseline rate. This means that system resources are sufficient, all types of IO tasks can be processed in a timely manner, and internal tasks can also make full use of idle resources to complete quickly.

[0091] Step S305: License allocation strategy under excessive pressure.

[0092] If the system pressure exceeds the threshold (S303 determines "yes"), the system switches to overload control mode. In this state, the following operations can be performed: Safeguard external tasks: Maintain or even prioritize the rate of issuing licenses to external queues to ensure that user I / O tasks are not affected.

[0093] Reduce internal task resources: Dynamically reduce the overall rate or number of licenses issued to internal task queues. The reduction can be positively correlated with the level of pressure; that is, the greater the pressure, the greater the reduction.

[0094] Internal priority scheduling: Licenses are allocated according to preset priorities within internal tasks. High-priority internal tasks (such as data repair) can still obtain a small number of licenses, while license allocation for the lowest-priority internal tasks (such as bad sector scanning) is suspended.

[0095] Step S306: Issue licenses to each task queue.

[0096] Based on the strategy determined in S304 or S305, the embodiment can actually distribute the generated licenses to the corresponding external and internal queues. Only IO requests that have obtained a license are eligible to be retrieved and executed by the task scheduler.

[0097] After this scheduling cycle ends, the next cycle will begin. Once the license allocation is complete, the process can return to S301, waiting for the next cycle to start again. This allows the implementation to continuously monitor and respond in real time, forming an adaptive feedback loop.

[0098] Based on the above embodiments, this application also provides a storage task scheduling method, such as... Figure 4 As shown, the method may include the following steps: In step S410, the storage tasks in the storage system are obtained and the storage tasks are divided according to priority.

[0099] In this embodiment, the storage system categorizes received tasks into two main types: external tasks with high-priority storage and internal tasks with low-priority storage.

[0100] External tasks refer to read / write I / O operations (such as database queries, file uploads / downloads, etc.) directly initiated by users or applications. These external tasks directly affect the user experience of the front-end business and are therefore given the highest priority. In this embodiment, high-priority storage tasks can be these types of external tasks.

[0101] Internal tasks refer to background maintenance tasks initiated by the storage system to maintain its own health, data reliability, and performance. These tasks can be further subdivided into multiple sub-priorities. For example, tasks that significantly impact system availability, such as data repair, garbage collection, and data migration, have relatively high sub-priorities; while non-urgent tasks such as bad sector scanning and data consistency verification have relatively low sub-priorities. In this embodiment, low-priority storage tasks can be used to refer to this type of system maintenance task in general.

[0102] In step S420, the IO pressure metrics of the storage system are monitored.

[0103] In this embodiment, continuous monitoring of IO performance metrics that directly reflect user experience and service quality can serve as a basis for determining whether the system is overloaded, and may include average latency and / or long-tail latency ratio, etc.

[0104] Average latency can be the average processing time of all I / O requests. For example, an average latency exceeding 100ms may indicate that the system is starting to experience stress. Long-tail latency ratio refers to the proportion of I / O requests whose processing time exceeds a certain critical threshold (e.g., 2 seconds). This metric is used to capture abnormally slow requests that cause users to perceive "lag." For example, when this ratio exceeds 0.1%, it indicates that system performance has significantly deteriorated. The long-tail latency ratio can also refer to the proportion of I / O requests with latency exceeding a predetermined length.

[0105] In step S430, when the IO pressure index is greater than the preset pressure threshold, the resource allocation for low-priority storage tasks in the storage system is reduced to ensure the resource requirements of high-priority storage tasks in the storage system.

[0106] Among them, low-priority storage tasks refer to storage tasks with a priority lower than the first preset priority, and high-priority storage tasks refer to storage tasks with a priority higher than the first preset priority.

[0107] In this embodiment, when the IO pressure indicators (such as the ratio of average latency to long-tail latency) exceed a preset pressure threshold simultaneously as detected in step S420, the storage system triggers a dynamic resource control mechanism. By reducing resource allocation to low-priority storage tasks in the storage system, the resource requirements of high-priority storage tasks can be guaranteed. Specifically: Guarantee high-priority tasks: Ensure that system resources (such as I / O bandwidth, CPU time slices) allocated to user I / O tasks (external tasks) are not affected or enjoy the highest priority.

[0108] Reduce resources allocated to low-priority tasks: Decrease the resources allocated to internal system tasks (low-priority storage tasks). In a preferred embodiment, this can be achieved through a licensing mechanism. The system reduces the generation rate or number of IO licenses for internal tasks.

[0109] Differentiated handling of internal tasks: Within internal tasks, priority classification continues. Resource reduction primarily targets the lowest priority tasks (such as bad sector scanning), whose license allocation may be suspended, and the tasks may be directly discarded or postponed; while for relatively high priority internal tasks (such as data repair), a small amount of resources is still reserved to maintain their minimum operation.

[0110] In step S440, when the IO pressure index is not greater than the preset pressure threshold, the execution of low-priority storage tasks is driven by the system idle event.

[0111] Specifically, when the monitored IO pressure indicators (such as average latency and long-tail latency ratio) are not greater than the preset pressure threshold, it indicates that the storage system is currently in a normal or low-load state, and user IO requests are exerting relatively little pressure on system resources. At this time, the system will not mechanically wait for the next timer cycle, but will automatically trigger the following behavior: Idle event determination: The system continuously monitors metrics such as the depth of the IO request queue and the processor utilization. When a significant "resource idle window" is detected (for example, the external queue is continuously empty, or the CPU utilization is below a certain level for a specific period of time), it is determined that a system idle event has occurred.

[0112] Event-driven execution: Once an idle event is detected, the system immediately starts or accelerates the execution of low-priority storage tasks, such as garbage collection (GC), data consistency verification, and bad sector scanning. These tasks will receive temporary boosted resource allocation (such as more licenses), enabling them to efficiently and quickly utilize idle system resources to complete processing.

[0113] Dynamic exit mechanism: When the system detects an increase in user I / O requests and rising pressure indicators, it means the "idle window" is closed. The system will automatically and smoothly reduce the resource allocation of internal tasks, causing them to quickly give way, thereby ensuring that user I / O tasks can receive immediate resource responses at any time and avoiding interference with front-end services.

[0114] This step can resolve resource conflicts that may occur between maintenance tasks and user services in a timed-driven solution, transforming passive scheduling into proactive scheduling. This allows system maintenance tasks to be completed better during off-peak periods, greatly improving the execution efficiency of background tasks and the overall utilization of system resources. At the same time, it fundamentally eliminates business performance fluctuations caused by improper timing of internal task execution.

[0115] The storage task scheduling method provided in this application divides storage tasks according to priority. When the IO pressure index of the storage system exceeds a preset pressure threshold, resource allocation to low-priority storage tasks is reduced to ensure the resource needs of high-priority storage tasks. Conversely, when the IO pressure index is not greater than the preset pressure threshold, low-priority storage tasks are executed based on system idle events, allowing them to run during idle periods. This embodiment can allocate resources to higher-priority storage tasks when the storage system is overloaded, avoiding the potential for prolonged background task stagnation caused by directly pausing lower-priority tasks. Furthermore, it can handle low-priority storage tasks during idle periods, thus maintaining the long-term health and stability of the storage system.

[0116] In this embodiment, the storage tasks are divided into external tasks with a first priority and internal tasks with a second priority; the external tasks are user read / write I / O tasks, and the internal tasks are system maintenance tasks, with the first priority being higher than the second priority.

[0117] Specifically, external tasks can refer to user read / write I / O tasks initiated by users or upper-layer applications, such as database operations and file access. Their execution performance directly affects user experience and business continuity, and therefore they are given the highest priority.

[0118] Internal tasks can refer to system maintenance tasks initiated by the storage system to maintain its own operation, such as data repair, garbage collection, space reclamation, and bad sector scanning. These tasks are crucial to the long-term health of the system, but their operation should not affect immediate user services, and therefore are given a lower second priority.

[0119] This embodiment, through the clear priority division described above, provides a clear strategic basis for subsequent dynamic resource scheduling based on system pressure, ensuring that external tasks such as user I / O are always given priority when resources are scarce.

[0120] In this embodiment, internal tasks can be further divided into multiple different sub-priorities. For example, tasks that are crucial to preventing data loss and maintaining the basic functions of the system, such as data repair, data migration, garbage collection, and data flushing, are assigned higher sub-priorities; while routine inspection or non-urgent optimization tasks, such as bad sector scanning and data consistency verification, are assigned lower sub-priorities.

[0121] By establishing this multi-layered internal task prioritization system, the storage system can perform more granular scheduling when resources are scarce. When it is necessary to reduce internal task resources, the storage system can prioritize ensuring the minimum operating requirements of high-priority sub-tasks (such as data repair), while first limiting or suspending the execution of low-priority sub-tasks (such as bad sector scanning). This ensures the user's IO experience while maximizing the maintenance of the core health of the storage system.

[0122] Therefore, among the aforementioned internal tasks, data repair, data migration, garbage collection, and data flushing have a higher priority than bad sector scanning and data consistency verification.

[0123] Based on the above embodiments, in another embodiment provided in this application, the above resource allocation can be implemented based on a license mechanism, wherein the license represents a virtual credential that allows the execution of a single IO operation.

[0124] Therefore, when the IO pressure index exceeds the preset pressure threshold, in order to reduce resource allocation to low-priority storage tasks in the storage system, step S430 may further include the following steps: Step S431: Assign a license to the IO operation to be processed.

[0125] In this embodiment, the resource allocation is implemented based on a token mechanism. A token is a virtual credential, and each token represents the permission to perform a single I / O operation.

[0126] The implementation can maintain corresponding license pools for various task queues. When an I / O operation needs to be processed, a license can be successfully obtained from the corresponding license pool to gain execution permission.

[0127] Step S432: When the IO pressure index is greater than the preset pressure threshold, reduce the rate or number of licenses issued to the IO operations corresponding to low-priority storage tasks in the storage system.

[0128] When the IO pressure metric exceeds a preset threshold, a control mechanism is triggered, which reduces the rate or number of licenses issued to low-priority storage tasks (i.e., system maintenance tasks). For example, the license issuance rate for bad sector scanning tasks can be reduced to 30% of the normal level or even suspended, thereby effectively limiting its resource consumption.

[0129] At the same time, the storage system can maintain or prioritize the issuance of licenses for high-priority storage tasks (user read / write IO) to ensure that user business is not affected.

[0130] The implementation example achieves precise flow control of IO operations through a licensing mechanism, which can quickly and smoothly limit background task resources when the storage system is overloaded, so as to ensure the performance of high-priority storage tasks such as front-end services.

[0131] Step S433: If the number of licenses is less than the threshold, pause the allocation of licenses for IO operations corresponding to internal tasks with a priority lower than the second preset priority.

[0132] Specifically, the second preset priority is used to divide the internal tasks into higher-priority tasks that require protection (such as data repair and garbage collection) and lowest-priority tasks that can be temporarily suspended (such as bad sector scanning and consistency checks). When license resources become scarce to a certain extent, the storage system will completely stop supplying licenses to the lowest-priority internal tasks, preventing their I / O operations from being executed. This concentrates extremely limited system resources on ensuring user I / O tasks and critical internal maintenance tasks. This enables optimal resource allocation decisions under extreme pressure on the storage system.

[0133] Based on the above embodiments, in another embodiment provided in this application, the method may further include the following steps: Step S434: For IO operations in low-priority storage tasks that have not obtained a license, postpone or discard them.

[0134] In this embodiment, low-priority internal storage tasks, such as I / O operations like bad sector scanning and data consistency checking, may not obtain the necessary licenses for execution. For these unlicensed I / O operations, the storage system can take one of the following actions: Deferred processing: The IO request is kept in the queue and will be processed when the license becomes available in a later period.

[0135] Discard directly: Remove the IO request from the queue and do not process it again.

[0136] This processing mechanism ensures that storage system resources are concentrated on the execution of high-priority tasks to the greatest extent possible, while avoiding queue congestion that may be caused by excessive backlog of low-priority requests. By deferring or dropping I / O operations for non-critical tasks, the system can maintain the normal execution of core business operations under overload conditions.

[0137] Based on the above embodiments, in another embodiment provided in this application, the method may further include the following steps: Step S450: When the IO pressure index drops from above the preset pressure threshold to below the preset pressure threshold, restore resource allocation for low-priority storage tasks.

[0138] In this embodiment, when the pressure on the storage system eases and IO pressure indicators (such as average latency or the proportion of long-tail latency) drop from above a preset pressure threshold to below the preset pressure threshold, it indicates that the storage system has recovered from the overload state. At this time, the storage system will gradually restore resource allocation to low-priority storage tasks (such as system maintenance tasks).

[0139] Gradually restore license allocation: The storage system will gradually increase the rate and number of licenses issued to low-priority tasks, first restoring normal supply to higher sub-priority internal tasks (such as data repair and garbage collection), and then restoring supply to lower sub-priority tasks (such as bad sector scanning).

[0140] Normal processing mechanism restoration: As resource allocation is restored, previously suspended low-priority I / O operations will regain their licenses and be processed, ensuring that system maintenance functions gradually return to normal operation.

[0141] The recovery mechanism provided in the embodiments reflects the integrity and adaptability of the scheduling strategy of the embodiments of this application, enabling the storage system to achieve dynamic balance during pressure cycles. It can protect critical business during overload and restore system maintenance functions in a timely manner when resources are sufficient, thus ensuring the long-term healthy operation of the storage system.

[0142] Based on the above embodiments, in another embodiment provided in this application, when the IO pressure index is not greater than the preset pressure threshold, the processing of internal tasks can be changed from timed driving to system idle event driving, so as to automatically accelerate the processing of internal tasks when the system IO is idle.

[0143] To avoid the problems in traditional storage systems where internal tasks (such as garbage collection and data migration) are triggered at fixed time intervals (timed drive), which leads to internal tasks starting as scheduled during busy periods, competing for resources with user I / O tasks and impacting performance, and internal tasks potentially waiting during idle periods, thus failing to fully utilize available resources.

[0144] This embodiment employs an event-driven mechanism, intelligent acceleration processing, and dynamic resource allocation. Specifically: An event-driven mechanism means that the triggering of internal tasks no longer depends on fixed time intervals, but is based on system I / O idle events. When the storage system detects that the user's I / O load is low or that the system is idle, it can automatically trigger the execution of internal tasks.

[0145] Intelligent acceleration processing refers to the ability of internal tasks to obtain more system resources (such as increasing the license issuance rate) during idle periods of the storage system, thereby accelerating the completion of maintenance operations such as data repair and garbage collection.

[0146] Dynamic resource allocation refers to the automatic yielding of internal tasks to reduce resource consumption and ensure that user tasks are given priority when the storage system becomes busy again and user I / O requests increase.

[0147] The above-described system idle event-based driving mechanism provided in the embodiment enables internal tasks to run at the most appropriate time, which avoids competing for resources with users' critical business and makes full use of system idle time to improve processing efficiency, thereby achieving optimal utilization of system resources.

[0148] By dividing each functional module according to its corresponding function, this application provides a storage task scheduling device, which can be a server, a terminal, or a chip applied to a server. Figure 5 A schematic block diagram of the functional modules of a storage task scheduling apparatus provided for an exemplary embodiment of this application. For example... Figure 5 As shown, the storage task scheduling device includes: The task receiving and classification module 51 is used to obtain storage tasks in the storage system and classify the storage tasks according to priority. The pressure monitoring module 52 is used to monitor the IO pressure indicators of the storage system; The task scheduling and execution module 53 is used to reduce the resource allocation of low-priority storage tasks in the storage system when the IO pressure index is greater than the preset pressure threshold, so as to ensure the resource requirements of high-priority storage tasks in the storage system; when the IO pressure index is not greater than the preset pressure threshold, it drives the execution of the low-priority storage tasks based on the system idle event; wherein, low-priority storage tasks refer to storage tasks with a priority lower than the first preset priority, and high-priority storage tasks refer to storage tasks with a priority higher than the first preset priority.

[0149] This embodiment can allocate resources to higher-priority storage tasks when the storage system is overloaded, avoiding the problem of long-term stagnation of background tasks caused by directly pausing lower-priority storage tasks. When the IO pressure index is not greater than the preset pressure threshold, the execution of low-priority storage tasks is driven by the system idle event, thereby maintaining the long-term health and stability of the storage system.

[0150] In another embodiment provided in this application, storage tasks are divided into external tasks with a first priority and internal tasks with a second priority; external tasks are user read / write I / O tasks, and internal tasks are system maintenance tasks, with the first priority being higher than the second priority.

[0151] This embodiment, through the clear priority division described above, provides a clear strategic basis for subsequent dynamic resource scheduling based on system pressure, and can ensure that external tasks such as user I / O are always given priority when resources are scarce.

[0152] In another embodiment provided in this application, the internal tasks are further divided into multiple different sub-priorities. Among the internal tasks, data repair, data migration, garbage collection, and data flushing have higher priority than bad sector scanning and data consistency verification.

[0153] This embodiment establishes a multi-layered internal task priority system, enabling the storage system to perform more granular scheduling when resources are scarce. When it is necessary to reduce internal task resources, the storage system can prioritize ensuring the minimum operating requirements of high-priority sub-tasks (such as data repair), while first limiting or suspending the execution of low-priority sub-tasks (such as bad sector scanning). This ensures the user's IO experience while maximizing the maintenance of the core health of the storage system.

[0154] In another embodiment provided in this application, resource allocation is implemented based on a license mechanism, whereby a license represents a virtual credential that allows the execution of a single I / O operation; the aforementioned task scheduling and execution module 53 is specifically used for: Assign licenses to pending I / O operations; When the IO pressure index exceeds the preset pressure threshold, reduce the rate or number of licenses issued to low-priority storage tasks in the storage system.

[0155] This embodiment achieves precise flow control of IO operations through a license mechanism, which can quickly and smoothly limit background task resources when the storage system is overloaded, so as to ensure the performance of high-priority storage tasks such as front-end services.

[0156] In another embodiment provided in this application, the device further includes a first license allocation module, used for: If the number of licenses is less than the threshold, the allocation of licenses for I / O operations corresponding to internal tasks with a priority lower than the second preset priority will be suspended.

[0157] When license resources become scarce to a certain extent, this embodiment completely stops supplying licenses to the lowest priority internal tasks, preventing their I / O operations from being performed. This concentrates extremely limited system resources on ensuring user I / O tasks and critical internal maintenance tasks. This enables optimal resource allocation decisions under extreme pressure on the storage system.

[0158] In another embodiment provided in this application, the device further includes a task processing module for: For low-priority storage tasks, I / O operations that have not obtained a license are either deferred or discarded.

[0159] This implementation ensures that storage system resources are concentrated on the execution of high-priority tasks to the greatest extent possible, while avoiding queue congestion that may be caused by excessive backlog of low-priority requests. By deferring or dropping I / O operations for non-critical tasks, the system can maintain the normal execution of core business operations under overload conditions.

[0160] In another embodiment provided in this application, the device further includes a second license allocation module, specifically used for: When the IO pressure metric drops from above the preset pressure threshold to below the preset pressure threshold, resource allocation for low-priority storage tasks is restored.

[0161] This embodiment enables the storage system to achieve dynamic balance during stress cycles, ensuring critical business operations are protected during overload and timely restoration of system maintenance functions when resources are sufficient, thus guaranteeing the long-term healthy operation of the storage system.

[0162] This application also provides a computing device, including: at least one processor; a memory for storing at least one processor-executable instruction; wherein the at least one processor is configured to execute instructions to implement the methods disclosed in the embodiments of this application.

[0163] The aforementioned processor can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this application can be implemented by integrated logic circuits in the processor's hardware or by software instructions. The aforementioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in memory, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0164] Furthermore, various operations / processes according to embodiments of this application, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a computing device with a dedicated hardware architecture, such as... Figure 6 The computing device 600 shown is equipped with programs that constitute the software. When various programs are installed, the computing device is able to perform various functions, including functions such as those mentioned above. Figure 6 A structural block diagram of a computing device provided for an exemplary embodiment of this application.

[0165] Computing device 600 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0166] like Figure 6As shown, the computing device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the computing device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0167] Multiple components in the computing device 600 are connected to the I / O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 can be any type of device capable of inputting information to the computing device 600. The input unit 606 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the computing device. The output unit 607 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 608 may include, but is not limited to, a hard disk and an optical disk. The communication unit 609 allows the computing device 600 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, a modem, network card, infrared communication device, wireless communication transceiver, and / or chipset, such as Bluetooth™ device, WiFi device, WiMax device, cellular communication device, and / or the like.

[0168] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in the embodiments of this application can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on a computing device via ROM 602 and / or communication unit 609. In some embodiments, the computing unit 601 can be configured to perform the methods disclosed in the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0169] This application also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by the processor of a computing device, the computing device is able to perform the methods disclosed in the embodiments of this application.

[0170] The computer-readable storage medium in this application embodiment may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0171] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0172] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this application.

[0173] In embodiments of this application, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0174] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0175] The modules, components, or units described in the embodiments of this application can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0176] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0177] The above description is merely an embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0178] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A storage task scheduling method, characterized in that, The method includes: Obtain storage tasks from the storage system and divide the storage tasks according to priority; Monitor the I / O pressure metrics of the storage system; When the IO pressure index exceeds a preset pressure threshold, reduce the resource allocation for low-priority storage tasks in the storage system to ensure the resource requirements of high-priority storage tasks in the storage system. When the IO pressure index is not greater than the preset pressure threshold, the execution of the low-priority storage task is driven based on the system idle event; Wherein, the low-priority storage task refers to a storage task with a priority lower than the first preset priority, and the high-priority storage task refers to a storage task with a priority higher than the first preset priority.

2. The method according to claim 1, characterized in that, The storage tasks are divided into external tasks with a first priority and internal tasks with a second priority; the external tasks are user read / write I / O tasks, and the internal tasks are system maintenance tasks, with the first priority being higher than the second priority.

3. The method according to claim 2, characterized in that, The internal tasks are further divided into several different sub-priorities.

4. The method according to claim 3, characterized in that, Among the internal tasks, data repair, data migration, garbage collection, and data flushing have a higher priority than bad sector scanning and data consistency verification.

5. The method according to claim 1, characterized in that, The IO pressure metrics include the average latency of IO requests and / or the proportion of IO requests whose latency exceeds a predetermined time length.

6. The method according to claim 2, characterized in that, The resource allocation is implemented based on a license mechanism, whereby the license represents a virtual credential that allows the execution of a single I / O operation; When the IO pressure index exceeds a preset pressure threshold, reducing resource allocation to low-priority storage tasks in the storage system includes: Assign licenses to pending I / O operations; When the IO pressure index exceeds a preset pressure threshold, reduce the rate or number of licenses issued to IO operations corresponding to low-priority storage tasks in the storage system.

7. The method according to claim 6, characterized in that, The method further includes: If the number of licenses is less than the threshold, the allocation of licenses for IO operations corresponding to internal tasks with a priority lower than the second preset priority is suspended.

8. The method according to claim 6 or 7, characterized in that, The method further includes: For IO operations in the low-priority storage task that have not obtained a license, they should be deferred or discarded.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: When the IO pressure index drops from above the preset pressure threshold to below the preset pressure threshold, resource allocation for the low-priority storage task is restored.

10. A computing device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-9.

Images