Method, apparatus, and computer program product for managing a storage system

Abstract

Embodiments of the present disclosure relate to a method, an electronic device and a computer program product for managing a storage system. The method comprises assigning a plurality of tasks to a plurality of acceleration resources in the storage system for processing based on respective task types of the plurality of tasks; determining, at least for a first acceleration resource of the plurality of acceleration resources, a first polling interval based on an average task volume of a first set of tasks assigned to the first acceleration resource; and scheduling execution of the first set of tasks at the first acceleration resource at the first polling interval. Embodiments of the present disclosure can optimize scheduling of tasks to be executed on the plurality of acceleration resources, and in turn, optimize system performance.

Description

Technical Field

[0001] Embodiments of this disclosure generally relate to the field of data storage, and more specifically to methods, apparatus, and computer program products for managing storage systems. Background Technology

[0002] In storage systems, acceleration resources (such as coprocessors) are typically used to assist the central processing unit (CPU) in performing tasks at a faster pace. For example, Quick Assist Technology (QAT) cards are a common type of coprocessor used to accelerate computationally intensive tasks such as compression, encryption, and decryption. Storage systems usually include multiple acceleration resources, such as QAT cards. There are often a large number of tasks of different types that require the use of acceleration resources. How to better allocate tasks of various types to multiple acceleration resources is a problem worthy of attention. Summary of the Invention

[0003] Embodiments of this disclosure provide methods, apparatus, and computer program products for managing storage systems.

[0004] In a first aspect of this disclosure, a method for managing a storage system is provided. The method includes allocating multiple tasks to multiple acceleration resources in the storage system for processing based on the task types of each of the multiple tasks to be executed; determining a first polling interval based on the average task load of a first group of tasks allocated to the first acceleration resource, at least for a first acceleration resource; and scheduling the execution of the first group of tasks at the first acceleration resource at the first polling interval.

[0005] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform an action comprising: allocating a plurality of tasks to a plurality of acceleration resources in a storage system for processing based on the task types of each of the plurality of tasks to be executed; determining a first polling interval based on the average task load of a first group of tasks allocated to the first acceleration resource, at least for a first acceleration resource; and scheduling the execution of the first group of tasks at the first acceleration resource at the first polling interval.

[0006] In a third aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any step of the method described in the first aspect of this disclosure.

[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0008] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0009] Figure 1 A schematic diagram of an example system that can be implemented therein according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A schematic diagram illustrating the assignment of tasks to multiple acceleration resources is shown;

[0011] Figure 3 A flowchart illustrating an example method for managing a storage system according to some embodiments of this disclosure is shown;

[0012] Figure 4 A schematic diagram illustrating task allocation according to some embodiments of the present disclosure is shown;

[0013] Figure 5 Another schematic diagram illustrating task allocation according to some embodiments of the present disclosure is shown;

[0014] Figure 6 Another schematic diagram illustrating task allocation according to some embodiments of the present disclosure is shown; and

[0015] Figure 7 A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0016] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0017] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0018] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] Figure 1 A schematic diagram of a storage system 100 in which embodiments of the present disclosure may be implemented is shown. The storage system 100 is used to provide data storage related tasks, including tasks such as storage, data access, and data protection (e.g., deduplication, backup, encryption, decryption, etc.). It should be understood that... Figure 1 The system shown is merely an example. In real-world applications, there may be many other devices and / or components within the storage system, or the devices and / or components shown may be arranged in other ways.

[0020] like Figure 1 As shown, storage system 100 includes multiple (e.g., N) acceleration resources 110-1, 110-2, 110-3, ..., 110-N, where N is an integer greater than 1. In the following discussion, acceleration resources 110-1, 110-2, 110-3, ..., 110-N are sometimes collectively referred to as acceleration resource 110 or simply referred to as acceleration resource 110. Each acceleration resource 110 is configured to execute various tasks 120-1, 120-2, 120-3, ..., 120-N (collectively referred to as or simply referred to as task 120) of storage system 100. In this document, a task is also referred to as a job, request, or instance.

[0021] Acceleration resources 110 may reside on nodes (not shown) of the storage system 100, and nodes may be implemented by one or more computing devices, computing systems, servers, mainframes, edge devices, etc. In this document, nodes are also referred to as devices. In some embodiments, each node of the storage system 100 has one or more acceleration resources 110. Each node of the storage system 100 may be deployed with the same or different acceleration resources 110 to perform corresponding tasks.

[0022] In storage system 100, the tasks to be performed at acceleration resource 110 include various types of data storage-related tasks. As some examples, tasks may include compression tasks to compress user data; encryption tasks to encrypt user data; and encryption tasks (such as SHA tasks) to decrypt user data. Of course, these are just some specific examples. Other types of tasks may exist in storage system 100, such as data backup tasks, data recovery tasks, etc.

[0023] The use of acceleration resource 110 can accelerate the processing of various tasks, thereby improving the overall throughput of storage system 100. In some embodiments, acceleration resource 110 can be one or more QAT cards. It should be understood that although QAT cards are used as examples of acceleration resources in some embodiments of this disclosure, acceleration resource 110 may also include other hardware processing devices with acceleration capabilities.

[0024] Depending on the storage system employed, in some embodiments, storage system 100 may include two acceleration resources (i.e., N=2) 110. In some embodiments, storage system 100 may include a scalable multi-node (multi-device) system comprising at least two (i.e., N≥2) acceleration resources 110. In such a system, tasks can be scheduled to execute on the various acceleration resources 110. It should be understood that the number of nodes in storage system 100 can be arbitrary, and the number of acceleration resources 110 on each node can be arbitrary.

[0025] It should be understood that Figure 1 The storage system 100 shown is merely exemplary and not limiting. The storage system according to this disclosure may also have other forms or structures.

[0026] Figure 2 This illustrates a common scheme for allocating multiple tasks in storage system 200 to multiple acceleration resources. For example... Figure 2 As shown, the storage system 200 includes acceleration resources 210-1, 210-2, and 210-3 (collectively referred to as, or individually, acceleration resource 210). In conventional solutions, tasks are typically allocated to multiple acceleration resources by balancing the workload of tasks to be processed on different acceleration resources. For example... Figure 2As shown, acceleration resource 210-1 is allocated tasks 220-1, 220-2, 220-3, 220-5, 220-6, and 220-7 of SHA task type, and task 220-4 of compression task type. Acceleration resource 210-2 is allocated tasks 230-1 of encryption task type, tasks 230-2 of compression task type, and tasks 230-3, 230-4, and 230-5 of SHA task type. Acceleration resource 210-3 is allocated tasks 240-1 of encryption task type, tasks 240-2 and 240-3 of compression task type, and task 240-4 of SHA task type. The total number of tasks to be processed at each acceleration resource 210 is basically the same.

[0027] Traditionally, a polling pattern is typically used to schedule tasks awaiting execution on acceleration resources. For example, the polling interval is used to determine whether a task on the acceleration resource is currently complete. However, in conventional schemes, it is difficult to determine an appropriate polling interval when the tasks awaiting execution on the acceleration resource belong to different task types. Figure 2 Taking the acceleration resource 210-1 as an example, the execution time required by acceleration resource 210-1 for SHA type tasks is typically 17 microseconds (when the task size is 8KB), while the execution time required for compression type tasks is approximately 280 microseconds (when the task size is 100KB). In this case, it is difficult to determine an appropriate polling interval.

[0028] A common approach is to choose a polling interval close to the required execution time, such as 20 microseconds (close to 17 microseconds) in the example above. This shorter polling interval results in multiple unnecessary polls of incomplete compression tasks, increasing the context switching overhead of the compression tasks. Another common approach is to choose a polling interval close to the required execution time, such as 300 microseconds (close to 280 microseconds) in the example above. This longer polling interval increases the latency of the SHA task, as it needs to wait significantly longer to be polled. Therefore, it is difficult to reasonably allocate multiple acceleration resources for multiple tasks of different types and ensure that appropriate polling intervals are set for each acceleration resource.

[0029] Embodiments of this disclosure propose a scheme for managing a storage system to address one or more of the aforementioned problems and other potential problems. In this scheme, tasks are allocated to multiple acceleration resources in the storage system for processing based on the task type of the task to be executed. The scheme also includes determining the polling interval for scheduling the allocated set of tasks at each acceleration resource based on the average task load of a set of tasks allocated to each acceleration resource.

[0030] In this way, tasks can be rationally allocated by assigning them to different acceleration resources according to their task type. This ensures that tasks assigned to different acceleration resources have the same task type and therefore similar required execution times, making it easier to determine an appropriate polling interval.

[0031] The basic principles and several exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0032] The following will refer to Figure 3 A flowchart is provided to describe a method 300 for allocating multiple tasks to multiple acceleration resources according to some embodiments of the present disclosure. Method 300 may be implemented by a device or node of the storage system 100. It should be understood that method 300 may also be performed by other suitable devices or apparatuses. Method 300 may include additional actions not shown and / or actions shown may be omitted; the scope of the present disclosure is not limited in this respect. For ease of explanation, reference will be made to... Figure 1 Let's describe method 300.

[0033] like Figure 3 As shown, at point 310, based on the task types of the multiple tasks to be executed, the multiple tasks are assigned to multiple acceleration resources 110 in the storage system 100 for processing. For example, a task belonging to a first task type can be assigned to acceleration resource 110-1 among the multiple acceleration resources 110; a task belonging to a second task type can be assigned to acceleration resource 110-2. It should be understood that a task belonging to the first task type can be assigned to one acceleration resource 110-2 in the storage system 100, or it can be assigned to two or more acceleration resources 110 in the storage system 100. The terms "first" and "second" used herein are for ease of distinction only.

[0034] Figure 4 A schematic diagram illustrating task allocation according to some embodiments of the present disclosure is shown. For example... Figure 4 As shown, tasks 420-1, 420-2, 420-3, 420-4, 420-5, 420-6, and 420-7, belonging to the SHA task type, are assigned to acceleration resource 110-1. Tasks 440-1 and 440-2, belonging to the encryption task type, are assigned to acceleration resource 110-3. Tasks 430-1, 430-2, 430-3, and 430-4, belonging to the compression task type, are assigned to acceleration resource 110-2. Tasks 450-1, 450-2, 450-3, and 450-4, belonging to the compression task type, are assigned to acceleration resource 110-N. It should be understood that the storage system 100 may also include, in addition to... Figure 4 Other task types besides the few task types shown here. Figure 4 The example assignments are merely illustrative and not restrictive. Figure 4 The size of the boxes for each task type shown does not indicate the size of the corresponding task volume.

[0035] In this way, tasks of different types can be assigned to different acceleration resources 110. The execution time required for different types of tasks typically varies significantly. This allows the acceleration resource 110 to determine its polling interval based on the task type to which it is assigned.

[0036] Additionally or alternatively, in some embodiments, for a group of tasks of the same type among multiple tasks, the group of tasks can be allocated based on the execution time required by the group of tasks. For example, the execution time required for each task can be determined by the size of the workload of each task in the group. Taking a group of tasks belonging to the compression task type as an example, when the workload is small, the required execution time is usually short, while when the workload is large, the required execution time is usually long.

[0037] Figure 5 A schematic diagram illustrating task allocation based on the required execution time of tasks according to some embodiments of the present disclosure is shown. In some embodiments, the association between the required execution time and the workload of each task in a set of tasks belonging to a certain task type, such as a compression task type, is pre-stored in a lookup table. This lookup table can be derived by storing historical execution information of system 100, or by deriving data from multiple experiments. In some embodiments, the lookup table may also be provided by the producer of acceleration resource 110.

[0038] A threshold execution time T can be determined based on the workload and corresponding required execution time of each task in a set of tasks. Based on this threshold execution time T, the set of tasks can be allocated to two of the multiple acceleration resources 110 to ensure that the workload of these two acceleration resources is substantially balanced. It should be understood that in some embodiments, more than one threshold execution time can be determined to allocate the set of tasks to more than two acceleration resources.

[0039] Figure 5 A schematic diagram illustrating the allocation of a set of tasks according to the respective task sizes of each task, based on an embodiment of this disclosure, is shown. Figure 5In the example, taking compression tasks as an example, a group of compression tasks includes tasks 510-1, 510-2, and 510-3 with a size of 32KB, task 510-4 with a size of 48KB, task 520-1 with a size of 80KB, and task 520-1 with a size of 64KB. In the following discussion, tasks 510-1, 510-2, 510-3, and 510-4 are sometimes collectively referred to as task 510 or referred to individually. Tasks 520-1 and 520-2 are sometimes collectively referred to as task 520 or referred to individually.

[0040] According to the lookup table relating task size to execution time, a task with a size of 32KB requires approximately 109 microseconds to execute, a task with a size of 48KB requires approximately 149 microseconds to execute, a task with a size of 64KB requires approximately 187 microseconds to execute, and a task with a size of 80KB requires approximately 226 microseconds to execute.

[0041] Based on the task size and execution time required for each of the above tasks, the threshold execution time T can be determined to be 180 microseconds. According to the threshold execution time T, the above set of tasks can be... Figure 5 The allocation is as shown. Tasks with sizes of 32KB and 48KB require execution times less than the threshold execution time T, and therefore can be allocated to acceleration resource 110-2. Tasks with sizes of 64KB and 80KB require execution times greater than the threshold execution time T, and therefore can be allocated to acceleration resource 110-N.

[0042] It should be understood that, although Figure 5 The threshold execution time T shown is set to 180 microseconds, but in this example, the threshold execution time T can also be set to any length of time between 149 microseconds (i.e., the execution time required for a 48KB task) and 187 microseconds (i.e., the execution time required for a 64KB task).

[0043] After the above allocation, the total number of tasks allocated to acceleration resource 110-2 is 32KB + 32KB + 32KB + 48KB = 144KB. The total number of tasks allocated to acceleration resource 110-N is 80KB + 64KB = 144KB. Therefore, through this allocation method, the total number of tasks allocated to acceleration resources 110-2 and 110-N is balanced. This balanced allocation method can prevent a single acceleration resource 110 from becoming too busy and causing a failure. In addition, the execution time required for the tasks allocated to acceleration resources 110-2 or 110-N is also similar, making it easier to determine the polling interval in the following steps.

[0044] It should be understood that, for ease of explanation, in Figure 5 The example shown is a simple example of a group of tasks consisting of only 6 tasks. In practical applications, a group of tasks of the same task type can include more or fewer tasks, and the group can also include more or fewer types of task sizes.

[0045] Additionally or alternatively, for certain task types, such as SHA task types, the execution time required varies little across different task sizes. In this case, the polling interval can be determined based on the task type. For example, the execution time required for a SHA task type is approximately 17 microseconds. Acceleration resources allocated to tasks of the SHA task type, such as... Figure 4 In the example, the polling interval for acceleration resource 110-1 is set to a time length of approximately 17 microseconds, such as 20 microseconds.

[0046] Additionally or alternatively, in some embodiments, the execution time required for each task can be determined jointly based on both the task's execution level and workload. For example, the correlation between the required execution time, execution level, and workload for each task in a set of tasks is pre-stored in a lookup table. This lookup table can be derived from historical execution information stored in system 100, or from multiple experimental data. In some embodiments, the lookup table can also be provided by the producer of acceleration resource 110. Table 1 below shows example lookup tables according to some embodiments of this disclosure.

[0047] Table 1 Example Lookup Table

[0048]

[0049] Table 1 above uses compression tasks as an example to show the execution time required for tasks with different amounts of work at different execution levels (also known as compression levels). It should be understood that Table 1 is merely illustrative and not restrictive. Other task sizes are possible besides those shown in Table 1. Other execution levels are also possible besides those shown in Table 1. Similar lookup tables can be used for other task types, such as SHA task types. For example, SHA task types using the SHA1 algorithm and SHA task types using the SHA256 algorithm can correspond to different execution levels for SHA task types. The task size, execution level, and execution time for SHA task types can have a lookup table similar to Table 1.

[0050] A threshold execution time T can be determined based on the workload, execution level, and corresponding required execution time of each task in a set of tasks. Based on this threshold execution time T, the set of tasks can be allocated to two of the multiple acceleration resources 110 to ensure that the workload of these two acceleration resources is substantially balanced. It should be understood that in some embodiments, more than one threshold execution time can be determined to allocate the set of tasks to more than two acceleration resources.

[0051] Figure 6 A schematic diagram illustrating the allocation of a set of tasks according to an embodiment of the present disclosure based on the respective task size and execution level of each task. Figure 6 In the example, taking compression tasks as an example, a set of compression tasks includes task 610-2 with a low compression level (i.e., execution level 1 in Table 1) and a task size of 32KB, task 610-1 with a task size of 48KB, task 610-3 with a task size of 64KB, and task 620-3 with a task size of 80KB. A set of compression tasks also includes tasks 620-1 and 620-2 with a high compression level (i.e., execution level 4 in Table 1) and a task size of 32KB.

[0052] According to Table 1, the execution time required for each task in the above set of tasks can be determined. For example, the execution time required for task 610-2 is 109 microseconds (μs), and the execution time required for task 620-1 is 274 μs.

[0053] Based on the task size, execution level, and required execution time of each of the above tasks, the threshold execution time T can be determined to be 200 microseconds. Figure 5 Compared to the example shown, Figure 6 The high-compression tasks 620-1 and 620-2 shown require longer execution times compared to the low-compression task 610-2 with the same task size. Therefore, the same Figure 5 Compared to the example, Figure 6 The threshold time T determined in the example is relatively long.

[0054] Based on the threshold execution time T, the above set of tasks can be categorized as follows: Figure 6 The allocation is as shown. Tasks with low compression levels and sizes of 32KB, 48KB, and 64KB require execution time less than the threshold execution time T, and therefore can be allocated to acceleration resource 110-2. Tasks with low compression levels and a size of 80KB require execution time greater than the threshold execution time T, and therefore can be allocated to acceleration resource 110-N. Tasks with high compression levels and a size of 32KB require execution time greater than the threshold execution time T, and therefore can be allocated to acceleration resource 110-N.

[0055] After the above allocation, the total number of tasks allocated to acceleration resource 110-2 is 48KB + 32KB + 64KB = 144KB. The total number of tasks allocated to acceleration resource 110-N is 32KB + 32KB + 80KB = 144KB. Therefore, through this allocation method, the total number of tasks allocated to acceleration resources 110-2 and 110-N is balanced. This balanced allocation method can prevent one acceleration resource 110 from becoming too busy and causing a failure. In addition, the execution time required for the tasks allocated to acceleration resources 110-2 or 110-N is also similar, making it easier to determine the polling interval in the following steps.

[0056] It should be understood that, for ease of explanation, in Figure 6 The example shown is a simple example of a group of tasks consisting of only 6 tasks. In practical applications, a group of tasks of the same task type can include more or fewer tasks, and the group can also include more or fewer types of task sizes or more or fewer execution levels.

[0057] The above combination Figure 5 This illustrates an example of allocating a group of tasks belonging to the same task type based on task size. (Combined) Figure 6 An example is shown of allocating a group of tasks belonging to the same task type based on both task size and execution level. It should be understood that in some embodiments, a group of tasks belonging to the same task type may also be allocated based solely on execution level. It should also be understood that in some embodiments, in addition to the task size and execution level described above, other factors affecting task execution time may be considered, such as data patterns and data compressibility.

[0058] Additionally or alternatively, in some embodiments, if an imbalance in task processing is determined at multiple acceleration resources 110, at least a portion of the tasks can be reallocated. For example, if an imbalance in task processing is determined between acceleration resources 110-2 and 110-N, tasks at acceleration resources 110-2 and 110-N can be reallocated by re-determining the threshold execution time T. Figure 5 For example, if the current threshold execution time T is 180 microseconds, and during execution, it is found that there are too many tasks in acceleration resource 110-N, then the threshold execution time T can be gradually increased, for example, by 5 microseconds each time, and the balance between tasks in acceleration resource 110-2 and acceleration resource 110-N can be observed.

[0059] In this way, if an imbalance in task processing is identified, at least a portion of multiple tasks can be reallocated. This allows for the balancing of tasks across the various acceleration resources 110 within the storage system 100.

[0060] It should be understood that, although the above combination Figure 5 and Figure 6 Taking a compression task as an example, this illustrates the process of assigning tasks belonging to the same task type. A similar process can be performed for other task types.

[0061] Return to Figure 3 At point 320, for the first acceleration resource among multiple acceleration resources 110, the first polling interval is determined based on the average task load of the first group of tasks allocated to the first acceleration resource. For example, with Figure 5 For example, for acceleration resource 110-2, the first polling interval is determined based on the average number of tasks 510-1, 510-2, 510-3 and 510-4 allocated to acceleration resource 110-2.

[0062] exist Figure 5 In the example, the average task size of acceleration resource 110-2 is 36KB, and its corresponding required execution time is between 109 microseconds (i.e., the execution time required for 32KB) and 149 microseconds (i.e., the execution time required for 48KB). Therefore, the first polling interval can be determined to be a time length between 109 microseconds and 149 microseconds, for example, 120 microseconds.

[0063] In some embodiments, the first polling interval may also be determined based on the execution time required for each task in the first group of tasks allocated to the first acceleration resource. For example, with Figure 5 For example, the first polling interval can be determined by calculating the average execution time, or the weighted average execution time, of each task allocated to acceleration resource 110-2. Figure 5 For example, the average execution time required for tasks 510-1, 510-2, 510-3, and 510-4, which are allocated to acceleration resource 110-2, can be calculated as (109 + 109 + 109 + 149) microseconds / 4 = 119 microseconds. It should be understood that for... Figure 6 The polling interval can also be determined using a similar method.

[0064] It should be understood that although the process of determining the polling interval is described using acceleration resource 110-2 as an example, a similar process can be performed to determine the polling interval for other acceleration resources 110 in the storage system 100.

[0065] In this way, the polling interval can be determined to be close to the average execution time of the set of tasks allocated to acceleration resource 110. This avoids unnecessary polling operations caused by too short a polling interval, while avoiding unnecessary processing delays caused by too long a polling interval.

[0066] return Figure 3 At point 330, the execution of the first group of tasks at the first acceleration resource is scheduled using the first polling interval. For example, with... Figure 5 For example, task 510 can be scheduled with a polling interval of, for example, 119 microseconds.

[0067] In some embodiments, it can be determined at a first polling interval (e.g., 119 microseconds) whether acceleration resource 110-2 has completed task 510-1 in the first set of tasks 510. If it is determined that acceleration resource 110-2 has completed task 510-1, the execution result of the completed task 510-1 can be returned to each device, node, or program of storage system 100 that was assigned to the task. Additionally, task 510-1 can be removed from acceleration resource 110-2. Alternatively or additionally, acceleration resource 110-2 can also execute the next unexecuted task in task 510, such as task 510-2.

[0068] In some embodiments, at a first polling interval, it can be continuously determined whether each task in task 510 has been completed by acceleration resource 110-2, and completed tasks can be removed sequentially and the next task can be executed. It should be understood that for other acceleration resources in acceleration resource 110, a similar process can be performed to schedule the execution of a set of tasks allocated to acceleration resource 110 at that acceleration resource 110 at their respective determined polling intervals.

[0069] The above combination Figure 1 and Figures 3-6 Examples of task allocation according to some embodiments of the present disclosure are shown. The scheme of the present disclosure enables the allocation of different tasks to multiple acceleration resources based on task type, thereby enabling the determination of appropriate polling intervals for each of the multiple acceleration resources.

[0070] Furthermore, the disclosed solution can further allocate tasks to different acceleration resources based on the execution time required by each task belonging to the same task type. This allows the polling interval to be determined to be close to the execution time required by each task allocated to that acceleration resource. By doing so, not only can task balance be ensured across acceleration resources, but also an optimized polling interval can be determined for polling scheduling of tasks on each acceleration resource, thereby further improving the overall performance of the storage system.

[0071] Figure 7 A schematic block diagram of an example device 700 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1 The storage system 100 shown can be implemented by device 700. For example... Figure 7 As shown, device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 702 or loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. CPU 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0072] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0073] The various processes and handling described above, such as method 300, can be executed by processing unit 701. For example, in some embodiments, method 300 can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by CPU 701, one or more actions of method 300 described above can be performed.

[0074] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0075] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0076] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0077] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0078] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0079] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0080] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0082] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing a storage system, comprising: Based on the task type of each of the multiple tasks to be executed, the multiple tasks are allocated to multiple acceleration resources in the storage system for processing, including: For a group of tasks of the same type among the multiple tasks, the group of tasks is allocated based on both the task volume and execution level for that group of tasks; and At least for the first acceleration resource among the plurality of acceleration resources, The first polling interval is determined based on the average number of tasks allocated to the first group of tasks in the first acceleration resource; and The execution of the first group of tasks at the first acceleration resource is scheduled according to the first polling interval; and In response to determining an imbalance in task processing at the plurality of acceleration resources, the threshold execution time is changed at the plurality of acceleration resources.

2. The method of claim 1, wherein allocating the plurality of tasks to a plurality of acceleration resources in the storage system comprises: For a group of tasks of the same type among the multiple tasks, the group of tasks is allocated based on the execution time required for the group of tasks.

3. The method of claim 2, wherein the execution time is determined based on at least one of the execution level and the amount of tasks in the set of tasks.

4. The method of claim 3, wherein the association between the execution time, the execution level and the task quantity of each task in the set of tasks is pre-stored in a lookup table.

5. The method according to claim 1, further comprising: In response to determining the imbalance in task processing at the plurality of acceleration resources, at least a portion of the plurality of tasks are reallocated by changing the threshold execution time.

6. The method of claim 1, wherein determining the first polling interval comprises: The first polling interval is determined based on the execution time required for each task in the first group of tasks.

7. The method of claim 1, wherein scheduling the execution of the first group of tasks at the first acceleration resource at the first polling interval comprises: Based on the first polling interval, determine whether the first acceleration resource has completed the first task in the first group of tasks; as well as If it is determined that the first acceleration resource has completed the first task... Remove the completed first task from the first acceleration resource; as well as The first acceleration resource executes the next task in the first group of tasks that has not yet been executed.

8. The method according to claim 1, wherein the plurality of acceleration resources respectively include a Quick Assist Technology (QAT) card.

9. An electronic device, comprising: At least one processor; as well as At least one memory storing computer program instructions, the at least one memory and the computer program instructions being configured, together with the at least one processor, to cause the at least one processor to perform actions, the actions including: Based on the task type of each of the multiple tasks to be executed, the multiple tasks are allocated to multiple acceleration resources in the storage system for processing, including: For a group of tasks of the same type among the multiple tasks, the group of tasks is allocated based on both the task volume and execution level for that group of tasks; and At least for the first acceleration resource among the plurality of acceleration resources, The first polling interval is determined based on the average number of tasks allocated to the first group of tasks in the first acceleration resource; and The execution of the first group of tasks at the first acceleration resource is scheduled according to the first polling interval; and In response to determining an imbalance in task processing at the plurality of acceleration resources, the threshold execution time is changed at the plurality of acceleration resources.

10. The electronic device of claim 9, wherein allocating the plurality of tasks to a plurality of acceleration resources in the storage system comprises: For a group of tasks of the same type among the multiple tasks, the group of tasks is allocated based on the execution time required for the group of tasks.

11. The electronic device of claim 10, wherein the execution time is determined based on at least one of the execution level and the amount of tasks in the set of tasks.

12. The electronic device of claim 11, wherein the association between the execution time, the execution level, and the task quantity of each task in the set of tasks is pre-stored in a lookup table.

13. The electronic device of claim 9, wherein the action further comprises: In response to determining the imbalance in task processing at the plurality of acceleration resources, at least a portion of the plurality of tasks are reallocated by changing the threshold execution time.

14. The electronic device of claim 9, wherein determining the first polling interval comprises: The first polling interval is determined based on the execution time required for each task in the first group of tasks.

15. The electronic device of claim 9, wherein scheduling the execution of the first group of tasks at the first acceleration resource at the first polling interval comprises: Based on the first polling interval, determine whether the first acceleration resource has completed the first task in the first group of tasks; as well as If it is determined that the first acceleration resource has completed the first task... Remove the completed first task from the first acceleration resource; as well as The first acceleration resource executes the next task in the first group of tasks that has not yet been executed.

16. The electronic device of claim 9, wherein the plurality of acceleration resources each comprise a Quick Assist Technology (QAT) card.

17. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a device to perform the method according to any one of claims 1-8.

Images