Method, apparatus, and storage medium for allocating computing task amounts

Abstract

The present application discloses a method, apparatus, and storage medium for allocating computing task amounts, belonging to the field of storage technology. In the present application, resources that limit the computing speed of a storage device are determined, and computing tasks are divided into multiple subtasks according to the determined resources. A first subtask among the multiple subtasks is allocated to a computing unit, and a second subtask among the multiple subtasks is allocated to a second computing unit. On the basis of determining the resources that limit the computing speed of the storage device, allocating the computing amount according to the resources can, while making better use of the resources, cooperate with other resources to complete the computing of the computing task, so that the computing speed of the storage device when executing the computing task is better.

Description

Technical Field

[0001] This application relates to the technical field of data storage, and particularly to a method, device, and storage medium for allocating computing task amounts. Background Art

[0002] Currently, for a storage device including a controller and multiple hard disks, as the amount of data stored in the storage device increases, it is no longer possible to meet customer needs by only using the controller to complete data operations such as data reconstruction in the storage device. Based on this, other types of computing units with computing capabilities can be added between the controller of the storage device and the multiple hard disks to cooperate with the controller to complete data operations. In this way, for a data operation task, it is necessary to allocate the computing task amount between the controller and other computing units.

[0003] In the related art, a hard disk enclosure with computing capabilities can be added between the controller of the storage device and the multiple hard disks to cooperate with the controller for data operations. In this case, taking the data reconstruction task as an example, usually, the intermediate operations of half of the data in the reconstruction task are completed by this hard disk enclosure. After that, the controller completes the data reconstruction based on the intermediate operation results obtained by this hard disk enclosure and the other half of the data. However, this mechanical allocation method may not be able to ensure that the reconstruction speed reaches the optimum, resulting in a poor customer experience. Summary of the Invention

[0004] This application provides a method, device, and storage medium for allocating computing task amounts, which can improve the computing speed when the storage device executes computing tasks. The technical solutions are as follows:

[0005] In a first aspect, a method for allocating computing task amounts is provided. The method is applied to a storage device, and the storage device includes at least one first computing unit and a first memory, at least one second computing unit and a second memory. At least one of the first computing unit and the first memory is located in the controller or hard disk enclosure of the storage device, and at least one of the second computing unit and the second memory is located in the controller or the hard disk enclosure. The method includes: determining the resources that limit the computing speed of the storage device, where the resources include the memory bandwidth of the first memory, or the memory bandwidth of the second memory, or the link bandwidth between the controller and the hard disk enclosure, or the read / write capabilities of the hard disks included in the hard disk enclosure; dividing the computing task into multiple subtasks according to the determined resources that limit the computing speed of the storage device, and allocating a first subtask among the multiple subtasks to the first computing unit and a second subtask among the multiple subtasks to the second computing unit.

[0006] In the embodiments of the present application, the first arithmetic unit and the second arithmetic unit may be arithmetic units with different or the same forms. The so-called different forms may refer to different types of chips. For example, a CPU and AI chips such as a GPU and an NPU are arithmetic units with different forms. In addition, in the embodiments of the present application, the controller includes a CPU. In addition to this, it may also include other types of processors other than AI chips. The hard disk enclosure may or may not include an arithmetic unit. When the hard disk enclosure includes an arithmetic unit, the arithmetic unit included in the hard disk enclosure may be a CPU, or an AI chip such as a GPU or an NPU, or other types of processing chips. In addition, the first memory refers to the memory belonging to the first arithmetic unit, and the second memory refers to the memory belonging to the second arithmetic unit. The so-called memory belonging to a certain arithmetic unit means that the memory is used to store the instructions executed by the corresponding arithmetic unit and the data involved.

[0007] It should also be noted that the memory bandwidth refers to the rate at which the corresponding arithmetic unit can read data from or store data into the corresponding memory. The link bandwidth between the controller and the hard disk enclosure refers to the maximum amount of data allowed to pass through the communication channel between the controller and the connected hard disk enclosure per unit time. In addition, according to different arithmetic tasks, the read-write capabilities of the hard disks included in the hard disk enclosure can be characterized by different characteristic values. Among them, when the arithmetic task requires both data reading and writing to the hard disk at the same time, the read-write capability of the hard disk can be characterized by the maximum value among the maximum mixed read-write fluxes of each hard disk connected in the hard disk enclosure. The so-called maximum mixed read-write flux of each hard disk refers to the maximum amount of data allowed for the read-write mixture of each hard disk, that is, the maximum read-write bandwidth of each hard disk. When the arithmetic task only reads data from the hard disk without writing, the read-write capability of the hard disk can be characterized by the maximum value among the maximum read fluxes of each hard disk among multiple hard disks. The so-called maximum read flux of each hard disk refers to the maximum amount of data allowed to pass through when reading data from each hard disk, that is, the maximum read bandwidth of each hard disk. When the arithmetic task only writes data to the hard disk without reading, the read-write capability of the hard disk can be characterized by the maximum value among the maximum write fluxes of each hard disk among multiple hard disks. The so-called maximum write flux of each hard disk refers to the maximum amount of data allowed to pass through when writing data to each hard disk, that is, the maximum write bandwidth of each hard disk.

[0008] In the embodiments of the present application, the resources that limit the arithmetic speed of the storage device are determined, and then, based on the determined resources, the arithmetic task is divided into multiple subtasks for the distribution of the arithmetic amount. On the basis of determining the resources that limit the arithmetic speed of the storage device, allocating the arithmetic amount according to the resources can, while making better use of the resources, cooperate with other resources to complete the arithmetic of the arithmetic task, so that the arithmetic speed of the storage device when executing the arithmetic task is better.

[0009] Optionally, the implementation process of determining the resource that limits the computing speed of the storage device may be as follows: respectively determine the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the computing speed corresponding to the read / write capability of the hard disk; determine the resource with the minimum corresponding computing speed as the resource that limits the computing speed of the storage device.

[0010] That is to say, in the embodiments of the present application, the resource with the minimum corresponding computing speed is the problem that limits the further improvement of the computing speed of the storage device, that is, the resource bottleneck of the storage device.

[0011] Optionally, if the determined resource that limits the computing speed of the storage device is the memory bandwidth of the first memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the amount of computation of the first subtask is less than the amount of computation of the second subtask.

[0012] Wherein, the reference threshold range is a numerical interval from 0 to a specified value. When the difference value between the computing capabilities of the first computing unit and the second computing unit is within this reference threshold range, it can be considered that the computing capabilities of the first computing unit and the second computing unit are equivalent. In addition, the computing capability of the computing unit can be characterized by the maximum amount of data that can be processed per unit time, that is, characterized by the maximum amount of computation of the computing unit.

[0013] When the determined resource that limits the computing speed of the storage device is the memory bandwidth of the first memory, in the case where the computing capabilities of the first computing unit and the second computing unit are equivalent, the amount of data that the first computing unit can process will be limited by the memory bandwidth of the first memory, and thus less than the amount of data that the second computing unit can process. Based on this, the amount of computation of the first subtask allocated to the first computing unit can be less than the amount of computation of the second subtask allocated to the second computing unit.

[0014] Optionally, if the determined resource that limits the computing speed of the storage device is the memory bandwidth of the second memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the amount of computation of the first subtask is greater than the amount of computation of the second subtask.

[0015] Optionally, if the determined resource that limits the computing speed of the storage device is the link bandwidth between the controller and the hard disk enclosure, and the first computing unit is located in the controller and the second computing unit is located in the hard disk enclosure, then the amount of computation of the first subtask is less than the amount of computation of the second subtask.

[0016] When the resource restricting the storage device is the link bandwidth between the controller and the disk enclosure, as much computing workload as possible can be completed by the disk enclosure. In this way, the disk enclosure only needs to transmit the result data obtained after computing based on the data read from the hard disk back to the controller. Compared with the disk enclosure transmitting a large amount of raw data read from the hard disk to the controller for the arithmetic unit on the controller to perform arithmetic operations, the amount of data transmitted on the communication link between the disk enclosure and the controller can be reduced, thereby avoiding the limitation of the link bandwidth between the two on the computing speed of the storage device.

[0017] Optionally, if the resource that determines the computing speed of the storage device is the read / write ability of the hard disk, and the first arithmetic unit is located in the controller and the second arithmetic unit is located in the disk enclosure, then the computing workload of the first subtask is less than that of the second subtask.

[0018] If the resource restricting the computing speed of the storage device is the read / write ability of the hard disk, then for both the controller and the disk enclosure, no matter which of the two has a larger share of the computing workload, the computing speed will be restricted due to reading and / or writing data from the hard disk. However, considering that the controller is the main control unit of the entire storage device, the arithmetic units located on the controller often need to execute a relatively large variety of tasks. Therefore, as much computing workload as possible can be placed on the disk enclosure, which serves as an auxiliary unit, to complete.

[0019] Optionally, when the computing task is a data reconstruction task, the implementation process of dividing the computing task according to the determined resource restricting the computing speed of the storage device can be as follows: determining the resource consumption magnification factors of the data reconstruction task on the first arithmetic unit and the second arithmetic unit according to the reconstruction ratio of the data reconstruction task; obtaining the current degree of service pressure; and dividing the data reconstruction task into multiple subtasks according to the determined resource restricting the computing speed of the storage device, the computing capabilities of the first arithmetic unit and the second arithmetic unit, the resource consumption magnification factors of the data reconstruction task on the first arithmetic unit and the second arithmetic unit, and the degree of service pressure.

[0020] For an operation task to be allocated, during the operation process, the resource consumption will be amplified by a certain multiple when the operation task is performed in different operation units of the storage device. For example, for a data reconstruction task, the operation process of the EC operation will cause the memory bandwidth consumption of the operation unit to be amplified by a certain multiple. Based on this, in the embodiments of the present application, the resource consumption amplification multiples corresponding to different operation units when executing this type of operation task are obtained, and then the operation task is further allocated according to this resource consumption configuration, which can improve the accuracy of the operation volume allocation. In addition, in the embodiments of the present application, the current business pressure level and the operation capabilities of each operation unit are also considered. Although the operation capabilities of the operation units generally do not become a resource bottleneck when configuring the storage device, by considering the above two factors, the operation task volume can be allocated more reasonably to ensure that the operation speed of the storage device is better.

[0021] It should be noted that in the embodiments of the present application, the operation tasks are allocated in units of operation units. In some scenarios, the operation tasks can also be allocated in units of controllers and disk enclosures, that is, each of the controller and the disk enclosure is used as an operation unit for allocation. Or, in some scenarios, the operation units can also be divided according to other structural division methods to perform the corresponding operation task allocation. However, no matter which structural unit is used to allocate the operation tasks, the method provided in the embodiments of the present application can be referred to determine the resources that limit the operation speed of the storage device among the resources related to the operation process in each unit, and then the operation tasks are divided and allocated according to the determined resources. Among them, the parameter values used in the specific allocation process may change with the change of the structural unit, but this should also be included in the protection scope of the present application.

[0022] In a second aspect, a device for allocating an operation task volume is provided. The device for allocating an operation task volume has the function of implementing the method behavior of allocating an operation task volume in the first aspect above. The device for allocating an operation task volume includes at least one module, and the at least one module is used to implement the method for allocating an operation task volume provided in the first aspect.

[0023] In a third aspect, a storage device is provided. The storage device includes at least one first arithmetic unit and a first memory, at least one second arithmetic unit and a second memory. At least one of the first arithmetic unit and the first memory is located within a controller or a hard disk enclosure. At least one of the second arithmetic unit and the second memory is located within the controller or the hard disk enclosure. The arithmetic units included in the controller include a processor and the memory corresponding to the processor. The memory corresponding to the processor is used to store a program for supporting the processor to execute the method for allocating arithmetic task amounts provided in the first aspect above, and to store data involved in implementing the method for allocating arithmetic task amounts provided in the first aspect above. The processor is configured to execute the program stored in the memory corresponding to the processor.

[0024] In a fourth aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the method for allocating arithmetic task amounts described in the first aspect above.

[0025] In a fifth aspect, a computer program product containing instructions is provided. When it runs on a computer, it causes the computer to execute the method for allocating arithmetic task amounts described in the first aspect above.

[0026] The technical effects obtained in the second, third, fourth, and fifth aspects above are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be elaborated here.

[0027] The beneficial effects brought by the technical solution provided in this application at least include:

[0028] In the embodiments of this application, the resources that limit the arithmetic speed of the storage device are determined, and the arithmetic tasks are divided into multiple subtasks according to the determined resources. The first subtask among the multiple subtasks is allocated to the first arithmetic unit, and the second subtask among the multiple subtasks is allocated to the second arithmetic unit. On the basis of determining the resources that limit the arithmetic speed of the storage device, allocating the arithmetic amount according to this resource can, while making better use of this resource, cooperate with other resources to complete the arithmetic of the arithmetic task, so that the arithmetic speed of the storage device when executing the arithmetic task is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a system architecture diagram of a storage device provided by an embodiment of this application;

[0030] Figure 2 is a flowchart of a method for allocating arithmetic task amounts provided by an embodiment of this application;

[0031] Figure 3 is another system architecture diagram of a storage device provided by an embodiment of this application;

[0032] Figure 4 It is a schematic structural diagram of a device for allocating computing task amounts provided by an embodiment of the present application. Specific implementation manners

[0033] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0034] Before giving a detailed explanation of the embodiments of the present application, the application scenarios involved in the embodiments of the present application will be introduced first.

[0035] With the reduction in the volume of computing units, the improvement in computing power, and the decrease in price, the forms of hardware devices equipped with computing units have increased sharply. The structure of traditional storage devices includes a controller and multiple hard disks. However, with the explosion of data volume and the users' requirements for high capacity, high reliability, high throughput, and low latency of storage devices, it is no longer possible to meet the users' needs by implementing the computing of storage devices only through the controller. On this basis, it has become an irresistible trend to add computing units in different forms to the storage device to cooperate with the controller to complete the computing. For example, other forms of computing units can be added between the controller and the hard disks of the storage device to reduce the pressure on the controller by offloading some computing tasks on the controller to other computing units.

[0036] It should be noted that among the various computing tasks of the storage device, data reconstruction is a very important one. Data reconstruction is an effective way for the storage device to recover data. For large-capacity hard disks, the longer the data reconstruction time, the longer the impact time on the service, and, during the reconstruction process, the higher the probability of other hard disks failing, and the lower the data reliability. Therefore, accelerating data reconstruction is a common concern of major storage device manufacturers. On this basis, for storage devices with computing units in different forms, how to allocate the task amounts of data reconstruction tasks so that each computing unit can cooperate in computing to improve the reconstruction speed is one of the problems that need to be solved urgently. The method for allocating computing task amounts provided by the embodiments of the present application can be used in the above scenarios to allocate the task amounts of data reconstruction to multiple computing units (including the controller) of the storage device, so that multiple computing units can cooperate in computing to improve the reconstruction speed.

[0037] Of course, in addition to allocating the computing task amounts of the above data reconstruction tasks, the method for allocating computing task amounts provided by the embodiments of the present application can also be used to allocate the task amounts of other computing tasks in the storage device, such as some background computing tasks in the storage device, etc. The embodiments of the present application do not limit this.

[0038] Next, the system architecture involved in the method for allocating computing task amounts provided by the embodiments of the present application will be introduced.

[0039] Figure 1 is a system architecture diagram of a storage device 100 provided by an embodiment of the present application. As Figure 1 shown, the storage device 100 includes a controller 01, a hard disk enclosure 02, and hard disks 03.

[0040] As Figure 1 shown, the controller 01 includes a processor 011 and a memory 012.

[0041] The processor 011 refers to a central processing unit (CPU), which is used to process various computing tasks. Here, the computing tasks include data reconstruction tasks, deduplication tasks, data compression tasks, etc., and tasks such as executing read data requests or write data requests. In addition to the CPU, the controller 01 may also include other processors, such as a graphics processing unit (GPU), a neural-network processing unit (NPU), or other artificial intelligence (AI) chips. The above computing tasks are jointly shared and completed by these processors.

[0042] Taking the data reconstruction task as an example, its main operation is erasure coding (EC) operation. When the storage device 100 stores a piece of data, it can divide the data into multiple data blocks and generate a certain number of parity blocks according to the multiple data blocks according to the reconstruction ratio. For example, according to a reconstruction ratio of 23:2, the data can be divided into 23 data blocks, and 2 parity blocks can be generated based on the 23 data blocks. Then, the storage device can store the multiple data blocks and parity blocks in multiple different hard disks respectively. When a hard disk fails and some data blocks are damaged, the storage device can calculate the damaged data blocks based on the remaining undamaged data blocks, thereby realizing data recovery. This process is the EC operation. For example, for the above 23 data blocks and 2 parity blocks, assuming that any two of these 25 blocks are damaged, the damaged two blocks can be calculated through the remaining 23 blocks, thereby restoring the complete data. Among them, the process of calculating the damaged two blocks through the remaining 23 blocks is the EC operation.

[0043] Memory 012 is used to temporarily store the data carried in the write data request or the data read from the hard disk 03, as well as the intermediate data and result data during the operation process. For example, when the controller 01 receives multiple write data requests, it can temporarily store the data in the multiple write data requests in its own memory 012. When the capacity of the memory 012 reaches a certain threshold, the memory 012 is stored in the hard disk inserted in the hard disk frame 02. For another example, after reading data from the hard disk 03, the read data can be stored in the memory 012. The processor 011 can perform EC operations based on the data in the memory 012, and store the intermediate data or result data of the EC operations in the memory 012 as well. Among them, the memory 012 includes a volatile memory, a non-volatile memory or a combination thereof. The volatile memory is, for example, a random-access memory (English: random-access memory, RAM). The non-volatile memory is, for example, various machine-readable media such as a floppy disk, a hard disk, a solid state disk (SSD), and an optical disc. In addition, it should be noted that the memory 012 can also store program codes for implementing the solution of this application.

[0044] In this embodiment, the number of memories 012 is one or more. For example, some memories belong to the CPU and are used to store the data and instructions generated when the CPU executes various tasks. Some memories belong to the GPU and are used to store the data and instructions generated when the GPU executes various tasks. Generally speaking, each AI chip has a corresponding memory, and these memories are all used to store the data and instructions generated when the corresponding AI chip executes various tasks. However, this embodiment does not exclude the situation where the CPU and the AI chip share a memory, or two or more AI chips share a memory.

[0045] The hard disk frame 02 communicates with the controller 01 through an IP network or other networks. Multiple hard disks 03 can be inserted in the hard disk frame 02.

[0046] In the first possible case, the hard disk frame 02 does not have computing capabilities. For example, the hard disk frame 02 is only a control frame for inserting the hard disk 03.

[0047] In the second possible case, the hard disk frame 02 has computing capabilities, that is, as Figure 1As shown in the figure, the hard disk enclosure 02 includes a processor 021 and a memory 022. Among them, the processor 021 can be any chip with data processing capabilities. And the types of the processor 021 and the processor 011 can be the same or different. For example, the processor 011 can be a CPU, the processor 021 can be a GPU, or both the processor 011 and the processor 021 are CPUs. In the embodiment of the present application, the processor 021 of the hard disk enclosure 02 can receive a write data request or a read data request sent by the controller 01 to write data to or read data from the hard disk 03. In addition, the processor 021 of the hard disk enclosure 02 can also execute tasks assigned by the controller 01 to complete data operations.

[0048] The memory 021 is used to temporarily store the data carried in the write data request or the data read from the hard disk 03, as well as the intermediate data and result data during the operation process of the processor 021. For example, when the controller 01 receives multiple write data requests, it can first temporarily store the data in the multiple write data requests in its own memory 012. When the capacity of the memory 012 reaches a certain threshold, the data stored in its own memory 012 is sent to the hard disk enclosure 02. Similarly, the hard disk enclosure 02 can temporarily store the data sent by the controller 01 received in its own memory 022. When the capacity of the memory 022 reaches a certain threshold, the data stored in the memory 022 is sent to the hard disk 03 to store the data. For another example, the hard disk enclosure 02 reads data from the hard disk 03, stores a part of the read data in the memory 022, and sends the other part to the controller 01. The processor 011 of the controller 01 and the hard disk enclosure 02 can both perform EC operations based on the data in their respective memories, and store the intermediate data or result data of the EC operation in their respective memories as well. After the operation is completed, the hard disk enclosure 02 can send the result data stored in its own memory 022 to the controller 01. The controller 01 can combine the result data sent by the hard disk enclosure 02 received and the result data stored in its own memory 012 to obtain the restored data. Among them, the memory 022 also includes a volatile memory, a non-volatile memory, or a combination thereof. The volatile memory is, for example, a random-access memory (English: random-access memory, RAM). The non-volatile memory is, for example, various machine-readable media such as a floppy disk, a hard disk, a solid state disk (solid state disk, SSD), and an optical disc.

[0049] In the present embodiment of the application, the space of the hard disk 03 can be divided into multiple storage blocks (chunks), and the size of each storage block is the same. For example, the size of the storage block can be 256 KB, and each storage block has a segment of logical address. When the controller 01 receives a read data request, the read data request can address the logical address of the data to be read. According to the logical address of the data to be read, the storage block where the data to be read is located can be determined, and then the corresponding data is read from the storage block to the memory 012. Alternatively, when it is detected that a certain hard disk fails, the controller 01 can create a data reconstruction task, and according to the logical addresses of other undamaged data related to the damaged data in the failed hard disk, the corresponding data is read from other hard disks to the memory through the hard disk enclosure 02.

[0050] It should be noted that in the embodiment of the present application, the processors with computing capabilities included in the controller 01 and the hard disk enclosure 02 can both be referred to as computing units. Among them, some computing units can be referred to as the first computing units, and some computing units can be referred to as the second computing units. For detailed descriptions, please refer to the subsequent embodiments.

[0051] Next, a method for allocating the amount of computing tasks provided in the embodiment of the present application will be introduced.

[0052] Figure 2 is a flowchart of a method for allocating the amount of computing tasks provided in the embodiment of the present application. This method can be executed by the controller in the aforementioned storage device. Refer to Figure 2 , and this method includes the following steps:

[0053] Step 201: Determine the resources that limit the computing speed of the storage device. The resources include the memory bandwidth of the first memory, or the memory bandwidth of the second memory, or the link bandwidth between the controller and the hard disk enclosure, or the read / write capabilities of the hard disks included in the hard disk enclosure.

[0054] In the embodiment of the present application, the storage device includes at least one first computing unit and a first memory, at least one second computing unit and a second memory. At least one first computing unit and the first memory are located in the controller or the hard disk enclosure of the storage device, and at least one second computing unit and the second memory are located in the controller or the hard disk enclosure.

[0055] Among them, the first computing unit and the second computing unit refer to processing units with computing capabilities such as CPU, GPU, NPU, tensor processing unit (TPU), etc. In addition, the first memory belongs to the first computing unit and is used to store the data and instructions generated when the first computing unit executes various tasks. Among them, each first computing unit can correspond to a first memory, or multiple first computing units can share a first memory. The second memory belongs to the second computing unit and is used to store the data and instructions generated when the second computing unit executes various tasks. Similarly, each second computing unit can correspond to a second memory, or multiple second computing units can share a second memory.

[0056] It should also be noted that at least one first computing unit and the first memory, and at least one second computing unit and the second memory can all be located in the controller. At this time, the hard disk enclosure may not include a computing unit, that is, the hard disk enclosure does not have computing capabilities. Or, at least one first computing unit and the first memory are located in the controller, and at least one second computing unit and the second memory are located in the hard disk enclosure. At this time, both the controller and the hard disk enclosure have computing capabilities. Or, at least one first computing unit and the first memory are located in the hard disk enclosure, and at least one second computing unit and the second memory are located in the controller. In this case, both the controller and the hard disk enclosure also have computing capabilities. Or, at least one first computing unit and the first memory, and at least one second computing unit and the second memory are all located in the hard disk enclosure. Since the controller usually includes computing units such as a processor, in this case, both the hard disk enclosure and the controller have computing capabilities.

[0057] In the embodiments of the present application, when the above storage device executes an operation task, the operation speed of the storage device will be affected by various resources related to the operation process in the storage device. Exemplarily, the resources related to the operation process include the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the read / write capabilities of the hard disks included in the hard disk enclosure. Based on this, the controller can determine the resource that limits the operation speed of the storage device from the above multiple resources related to the operation process, that is, the resource bottleneck of the storage device.

[0058] Optionally, the computing power of the computing unit itself is also a resource that affects the computing speed of the storage device. However, in most cases, when configuring the architecture of the storage device, a computing unit with computing power that cannot match the memory or other resources will not be configured. That is to say, the computing power of the computing unit itself is unlikely to become a resource bottleneck that limits the computing speed of the storage device. Therefore, in the embodiments of the present application, when determining the resources that limit the computing speed of the storage device from multiple resources, the computing power of the computing unit may not be included in the multiple resources.

[0059] It should be noted that the memory bandwidth refers to the rate at which the corresponding computing unit can read data from or store data in the corresponding memory, that is, the maximum amount of data allowed to pass through the corresponding memory per unit time. Among them, the memory bandwidth is usually expressed in bytes per second, but for systems where the natural data size is not a multiple of 8 bits, the representation method may be different. The memory bandwidth determined for a specified memory or system usually refers to the maximum theoretical bandwidth, and the actual memory bandwidth will be less than this maximum theoretical bandwidth.

[0060] The link bandwidth between the controller and the disk enclosure refers to the maximum amount of data allowed to pass through the communication channel between the controller and the connected disk enclosure per unit time.

[0061] In addition, according to different computing tasks, the read / write capabilities of the hard disks included in the disk enclosure can be characterized by different characteristic values. Among them, when the computing task requires both data reading and writing to the hard disk at the same time, the read / write capabilities of the hard disk can be characterized by the maximum value among the maximum mixed read / write fluxes of each hard disk connected in the disk enclosure. The so-called maximum mixed read / write flux of each hard disk refers to the maximum amount of data allowed for the read / write mixing of each hard disk, that is, the maximum read / write bandwidth of each hard disk. When the computing task only reads data from the hard disk without writing, the read / write capabilities of the hard disk can be characterized by the maximum value among the maximum read fluxes of each hard disk. The so-called maximum read flux of each hard disk refers to the maximum amount of data allowed to pass through when reading data from each hard disk, that is, the maximum read bandwidth of each hard disk. When the computing task only writes data to the hard disk without reading, the read / write capabilities of the hard disk can be characterized by the maximum value among the maximum write fluxes of each hard disk. The so-called maximum write flux of each hard disk refers to the maximum amount of data allowed to pass through when writing data to each hard disk, that is, the maximum write bandwidth of each hard disk.

[0062] In an embodiment of the present application, the controller may respectively determine the computing speeds corresponding to each resource related to the computing process, and then determine the resource with the minimum computing speed as the resource that limits the computing speed of the storage device. Taking the resources related to the computing process mentioned above, including the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the read / write ability of the hard disk, as an example, the controller may respectively determine the computing speed corresponding to the memory bandwidth of the first memory, the computing speed corresponding to the memory bandwidth of the second memory, the computing speed corresponding to the link bandwidth, and the computing speed corresponding to the read / write ability of the hard disk. Then, select the minimum computing speed from them, and use the resource corresponding to the minimum computing speed as the resource that limits the computing speed of the storage device.

[0063] In some possible implementation manners, the controller may convert each resource so as to unify each resource into the amount of data passing through the corresponding component within the same unit time, that is, convert each resource into the corresponding computing speed.

[0064] Exemplarily, the controller may unify the units of the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the read / write ability of the hard disk, so as to obtain the computing speeds corresponding to each resource.

[0065] In some other possible implementation manners, the controller may combine the parameter values of the computing task to be allocated, the current business pressure level, and the resource consumption magnification factor during the execution process when the computing task is executed on different computing units, etc., to determine the resource that limits the computing speed of the storage device from multiple resources.

[0066] Among them, the parameter value of the computing task to be allocated can be used to represent the amount of the computing task to be allocated. The controller may obtain the parameter value of the computing task to be allocated in advance.

[0067] Exemplarily, taking the data reconstruction task as an example, when the controller detects a faulty hard disk among multiple hard disks, it may create a data reconstruction task. At this time, the data reconstruction task is the computing task to be allocated. Then, the controller may obtain the reconstruction ratio of the data reconstruction task as the parameter value of the data reconstruction task. Among them, the reconstruction ratio is used to indicate the number ratio between the data blocks and the parity blocks of a piece of data. For example, the reconstruction ratio may be a value such as 23:2. For a more detailed introduction to the reconstruction ratio, reference may be made to the relevant introduction to the reconstruction ratio in the foregoing embodiments. The embodiments of the present application will not elaborate herein.

[0068] It should be noted that the reconstruction ratio is determined according to the device structure of the storage device when storing data. Therefore, in the embodiments of the present application, after creating the data reconstruction task, the controller may directly obtain the pre-stored reconstruction ratio.

[0069] The current business pressure level can be the maximum business pressure level obtained most recently before the current moment, or can be the business pressure level predicted according to the business parameters of the current business by a machine learning model.

[0070] In addition, for the operation tasks to be allocated, during the operation process, the resource consumption will be amplified by a certain multiple when the operation tasks are carried out in different operation units of the storage device. For example, for the data reconstruction task, the operation process of the EC operation will cause the memory bandwidth consumption of the operation unit to be amplified by a certain multiple. Based on this, in the embodiments of the present application, the controller can also obtain the resource consumption amplification multiples corresponding to different operation units when executing this type of operation task.

[0071] Exemplarily, taking the data reconstruction task as an example, the controller can calculate the operation speed corresponding to each resource through the following calculation model (1).

[0072] V i = f i (e, α, p, R i ) (1)

[0073] Wherein, V i is the operation speed corresponding to the i-th resource, R i is the i-th resource, e is the reconstruction ratio of the data reconstruction task, α is the resource consumption amplification multiple corresponding to R i , and p is the current business pressure level.

[0074] It should be noted that in the above example, e can also adopt the number of data blocks in the reconstruction ratio. For example, if the reconstruction ratio is e1:e2, e can take e1.

[0075] Step 202: Divide the operation task into multiple subtasks according to the resources that determine the operation speed of the restricted storage device, allocate the first subtask among the multiple subtasks to the first operation unit, and allocate the second subtask among the multiple subtasks to the second operation unit.

[0076] After determining the resource that limits the computing speed of the storage device, the controller may divide the computing task into multiple subtasks according to the resource, and allocate the first subtask among the multiple subtasks to the first computing unit, and allocate the second subtask among the multiple subtasks to the second computing unit. As introduced in step 201 above, the computing speed corresponding to the determined resource that limits the computing speed of the storage device is the smallest among all resources, that is, this resource is the problem that limits the further improvement of the speed of the storage device. Based on this, when allocating computing tasks to each computing unit, while making the most of this resource, using other resources to complete the computing of the computing task at the same time can make the computing speed of the storage device better.

[0077] Taking the foregoing multiple resources as an example, in the first case, if the determined resource that limits the computing speed of the storage device is the memory bandwidth of the first memory, that is, the memory bandwidth of the first memory is the resource bottleneck, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the amount of computation of the first subtask is less than the amount of computation of the second subtask.

[0078] The reference threshold range refers to the numerical range between 0 and a specified value. When the difference value between the computing capabilities of the first computing unit and the second computing unit is within this reference threshold range, it can be considered that the computing capabilities of the first computing unit and the second computing unit are equivalent, and the impact of the difference between the two on the allocation of computing tasks is much smaller than the impact of the resource bottleneck. In addition, the computing capability of the computing unit can be characterized by the maximum amount of computation of the computing unit per unit time, or other characteristic values can also be used for characterization. The embodiments of the present application do not make any limitations in this regard.

[0079] When the memory bandwidth of the first memory is the resource bottleneck, it means that the speed at which the first computing unit reads or writes data from the first memory is relatively small. At this time, if the computing capabilities of the first computing unit and the second computing unit are equivalent, then affected by the speed of reading or writing data from the first memory, the amount of computation that the first computing unit can handle is also relatively small. Therefore, in this case, the amount of computation of the first subtask allocated to the first computing unit can be less than the amount of computation of the second subtask allocated to the second computing unit.

[0080] In the second case, if the determined resource that limits the computing speed of the storage device is the memory bandwidth of the second memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the amount of computation of the first subtask is greater than the amount of computation of the second subtask.

[0081] Among them, the relevant description of the reference threshold range can refer to the relevant description in the first case, and the embodiments of the present application will not repeat it here.

[0082] When the memory bandwidth of the second memory is the resource bottleneck, it indicates that the speed at which the second computing unit reads or writes data from the second memory is relatively low. At this time, if the computing power of the first computing unit is comparable to that of the second computing unit, then affected by the speed of reading or writing data from the second memory, the amount of computation that the second computing unit can handle is also relatively small. Therefore, in this case, the amount of computation of the second subtask assigned to the second computing unit can be less than the amount of computation of the first subtask assigned to the first computing unit.

[0083] In the third case, if the resource that determines the computing speed of the storage device is the link bandwidth between the controller and the hard disk enclosure, and the first computing unit is located in the controller and the second computing unit is located in the hard disk enclosure, then the amount of computation of the first subtask is less than the amount of computation of the second subtask.

[0084] When the resource of the storage device is the link bandwidth between the controller and the hard disk enclosure, data transmission between the controller and the hard disk enclosure through this link can be minimized as much as possible to avoid the limitation of this link bandwidth on the speed of the storage device. In this case, the controller can have as much computation as possible completed by the hard disk enclosure. In this way, the hard disk enclosure only needs to transmit the result data obtained after computing based on the data read from the hard disk back to the controller. Compared with the hard disk enclosure transmitting a large amount of raw data read from the hard disk to the controller for the computing unit on the controller to perform computations, the amount of data transmitted on the communication link between the hard disk enclosure and the controller can be reduced. Based on this, in the embodiments of the present application, the controller can allocate less computation to the first computing unit located in the controller and more computation to the second computing unit located in the hard disk enclosure, that is, the amount of computation of the first subtask is less than the amount of computation of the second subtask.

[0085] Of course, if the first computing unit is located in the hard disk enclosure and the second computing unit is located in the controller, then for the aforementioned reasons, in order to ensure that the amount of computation of the controller is less than that of the hard disk enclosure, the amount of computation of the first subtask allocated to the first computing unit can be greater than the amount of computation of the second subtask allocated to the second computing unit.

[0086] Optionally, for this case, in some possible embodiments, the amount of computation allocated to the computing unit on the controller can be 0, that is, when the link bandwidth between the controller and the hard disk enclosure is the bottleneck, all computing tasks can be directly completed by the hard disk enclosure.

[0087] In the fourth case, if the resource that limits the computing speed of the storage device is the read / write ability of the hard disk, and the first computing unit is located in the controller and the second computing unit is located in the hard disk enclosure, then the amount of computation of the first subtask is less than that of the second subtask.

[0088] If the resource that limits the computing speed of the storage device is the read / write ability of the hard disk, then for the controller and the hard disk enclosure, no matter which one of them is allocated more computation, the computing speed will be limited due to reading and / or writing data from the hard disk. However, considering that the controller is the main control unit of the entire storage device, the computing units located on the controller often need to execute a relatively large variety of tasks. Therefore, as many computations as possible can be completed by the hard disk enclosure, which serves as an auxiliary unit. Based on this, when the first computing unit is located in the controller and the second computing unit is located in the hard disk enclosure, the amount of computation of the first subtask allocated to the first computing unit can be less than that of the second subtask allocated to the second computing unit. Of course, if the second computing unit is located in the controller and the first computing unit is located in the hard disk enclosure, the amount of computation of the first subtask allocated to the first computing unit can be greater than that of the second subtask allocated to the second computing unit.

[0089] The above are several possible cases in the exemplary embodiments of the present application for computing amount allocation. When specifically performing the allocation, taking the data reconstruction task as an example, the controller can determine the resource consumption magnification factors of the data reconstruction task on the first computing unit and the second computing unit according to the reconstruction ratio of the data reconstruction task; obtain the current service pressure level; and divide the data reconstruction task into multiple subtasks according to the determined resource that limits the computing speed of the storage device, the computing capabilities of the first computing unit and the second computing unit, the resource consumption magnification factors of the data reconstruction task on the first computing unit and the second computing unit, and the service pressure level.

[0090] Among them, the implementation method of obtaining the current service pressure level can refer to the relevant introduction in the foregoing step 202 and will not be elaborated here.

[0091] In addition, regarding the resource consumption magnification factors of the data reconstruction task on each computing unit, the controller can determine the resource consumption magnification factors of the data reconstruction task on the corresponding computing unit in different ways for different computing units according to the reconstruction ratio of the data reconstruction task.

[0092] Exemplarily, if the computing unit is located in the controller, the controller can use the following calculation model (2) to determine the resource consumption magnification factor of the data reconstruction task when executed on the corresponding computing unit. If the computing unit is located in the hard disk enclosure, the following calculation model (3) can be used to determine the resource consumption magnification factor of the data reconstruction task on this computing unit.

[0093] α c = f c (e) (2)

[0094] α i = f i (e) (3)

[0095] Among them, α c is the resource consumption amplification factor when the data reconstruction task is executed on the arithmetic unit in the controller, and α i is the resource consumption amplification factor when the data reconstruction task is executed on the arithmetic unit in the hard disk enclosure connected to the controller. e is the reconstruction ratio of the data reconstruction task.

[0096] Optionally, e can also be the number of data blocks in the reconstruction ratio. For example, if the reconstruction ratio is e1:e2, e can take e1.

[0097] Based on obtaining the resource consumption amplification factor when the data reconstruction task is executed on each arithmetic unit and the current business pressure level, the controller can also obtain the computing power of each arithmetic unit. Among them, the computing power of the arithmetic unit can be characterized by the maximum amount of computation per unit time of the arithmetic unit, or can also be characterized by other characteristic values. This application embodiment does not make a limitation in this regard. After that, the controller can use a linear programming model to divide the data reconstruction task into multiple subtasks with different amounts of computation according to the resources that limit the computing speed of the storage device, the computing power of each arithmetic unit obtained, the resource consumption amplification factor of the data reconstruction task on each arithmetic unit, and the current business pressure level.

[0098] Exemplarily, the controller can determine the amount of computation of the data reconstruction task on different arithmetic units through the following linear programming model.

[0099] Objective function: max(r c + ∑r i );

[0100] Constraint condition: r c + ∑r i + f(p) ≤ R k ;

[0101] Among them, r c refers to the amount of computation of the data reconstruction task allocated to the arithmetic unit on the controller, r i refers to the amount of computation task of the data reconstruction task allocated to the i-th arithmetic unit on the hard disk enclosure, α c is the resource consumption amplification factor when the data reconstruction task is executed on the arithmetic unit in the controller, α iis the resource consumption magnification factor when the data reconstruction task is executed on the arithmetic unit in the hard disk enclosure connected to the controller. f(p) refers to the restricted resource bandwidth consumed under the current business pressure level, and ρ c and ρ i are the limit values set for the arithmetic unit on the controller and the arithmetic unit in the hard disk enclosure respectively. refers to the maximum amount of operations per unit time of the arithmetic unit in the controller. refers to the maximum amount of operations per unit time of the i-th arithmetic unit in the hard disk enclosure, and R k refers to the resource that limits the operation speed of the storage device.

[0102] It should be noted that in the above linear programming model, an example is given with one arithmetic unit included in the controller. Optionally, in some possible implementation manners, the controller may have multiple arithmetic units. In this case, the objective function in the above linear programming model may be to find the maximum value of the sum of the amounts of operations of multiple arithmetic units in the controller and multiple arithmetic units in the hard disk enclosure. Correspondingly, the constraint conditions can also be appropriately modified. For example, the amount of operation tasks of the arithmetic unit in the controller in the first condition of the above constraint conditions can be modified to the sum of the amounts of operation tasks of multiple arithmetic units in the controller, and the second condition can be modified to that the product of the amount of operation of each arithmetic unit in the controller and α c is less than the product of ρ c and the maximum amount of operations of the corresponding arithmetic unit.

[0103] Optionally, in some possible cases, the storage device may also include multiple controllers. Correspondingly, for any one of the controllers and the hard disk enclosures connected thereto, as well as between two controllers that can communicate with each other, the method provided in the embodiments of the present application can be referred to for allocating the amount of operation tasks of the operation tasks. Of course, it can also be extended to the allocation of the amounts of operation of multiple controllers and multiple hard disks.

[0104] The above takes the data reconstruction task as an example to illustrate the implementation process of allocating the amount of operation tasks. Optionally, for other types of operation tasks, the above method can also be used to allocate the amount of operation tasks. The difference is that the parameter values of the operation tasks for determining the resource consumption magnification factor of the operation tasks on each arithmetic unit are different, and according to different operation tasks, the above constraint conditions can also be partially adapted and modified.

[0105] Figure 3 is a system architecture diagram of a storage device shown in the embodiments of the present application. Taking this storage device as an example, it illustrates the change in the reconstruction speed of the storage device after allocating the amount of operation of the data reconstruction task by using the method for allocating the amount of operation tasks provided in the embodiments of the present application.

[0106] As shown Figure 3 in the figure, the storage device includes a control unit, which includes two controllers, namely controller A and controller B. Each controller includes an arithmetic unit, and each controller is connected to a control box, namely control box A and control box B. The control box A and control box B can be connected to multiple hard disks. The control unit can also be connected to an IP box, which also includes an arithmetic unit, and the IP box includes two boxes A and B for connecting hard disks. Suppose it is calculated by the above formula that the maximum operation speed on the IP box is about 600 - 700M / s. At this time, the resource that limits the operation speed of the storage device is that the memory bandwidth of the IP box reaches the upper limit. In this case, according to the memory bandwidth of the IP box, all EC calculations in the data reconstruction task can be completed by the controller. In this way, the reconstruction speed can be increased by 300M / s. Optionally, if a memory module is added to the IP box when the memory bandwidth of the above IP box reaches the upper limit, and the above allocation method of completing all EC calculations by the controller is adopted, the memory bandwidth of the memory of controller A may become the resource that limits the storage device. Therefore, in this case, according to the allocation method provided in the embodiment of the present application, part of the computing amount in the data reconstruction task is allocated to the IP box to complete, and part of the computing amount is allocated to the control box to complete according to the memory of controller A. In this way, the reconstruction speed can be increased by 300M / s again. It can be seen that in the embodiment of the present application, the resource that truly limits the operation speed of the storage device is determined. On this basis, the computing amount is allocated according to this resource to make the best use of this resource, and at the same time, other resource items are coordinated to complete the operation of the operation task. In this way, the operation speed of the storage device can be made better.

[0107] In the embodiment of the present application, the controller can determine the resource that limits the operation speed of the storage device, divide the operation task into multiple subtasks according to the determined resource, allocate the first subtask among the multiple subtasks to the arithmetic unit, and allocate the second subtask among the multiple subtasks to the second arithmetic unit. On the basis of determining the resource that limits the operation speed of the storage device, allocating the computing amount according to this resource can, while making better use of this resource, coordinate other resources to complete the operation of the operation task, so that the operation speed of the storage device is better when executing the operation task.

[0108] See Figure 4, an embodiment of the present application provides a device 400 for allocating computing tasks. This device can be applied to a storage device, which includes at least one first computing unit and a first memory, at least one second computing unit and a second memory. At least one first computing unit and the first memory are located within the controller or the hard disk enclosure of the storage device, and at least one second computing unit and the second memory are located within the controller or the hard disk enclosure. The device 400 includes: a determination module 401 and an allocation module 402;

[0109] The determination module 401 is configured to execute step 201 in the foregoing embodiment;

[0110] The allocation module 402 is configured to execute step 202 in the foregoing embodiment.

[0111] It should be noted that the functions of the determination module 401 and the allocation module 402 can be implemented by the CPU in the controller of the storage device reading and executing the instructions in the memory corresponding to the CPU. The structural schematic diagram of the storage device can refer to the foregoing Figure 1 shown storage device.

[0112] Optionally, the determination module 401 is mainly used for:

[0113] respectively determine the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the computing speed corresponding to the read / write ability of the hard disk;

[0114] Determine the resource with the minimum corresponding computing speed as the resource that limits the computing speed of the storage device.

[0115] Optionally, if the resource that limits the computing speed of the storage device determined is the memory bandwidth of the first memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the computing amount of the first subtask is less than that of the second subtask.

[0116] Optionally, if the resource that limits the computing speed of the storage device determined is the memory bandwidth of the second memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the computing amount of the first subtask is greater than that of the second subtask.

[0117] Optionally, if the resource that limits the computing speed of the storage device determined is the link bandwidth between the controller and the hard disk enclosure, and the first computing unit is located within the controller and the second computing unit is located within the hard disk enclosure, then the computing amount of the first subtask is less than that of the second subtask.

[0118] Optionally, if the resource that determines the computing speed of the storage device is the read / write ability of the hard disk, the first computing unit is located in the controller, and the second computing unit is located in the hard disk enclosure, then the computing amount of the first subtask is less than that of the second subtask.

[0119] Optionally, the allocation module 402 is mainly used for:

[0120] Determine the resource consumption magnification of the data reconstruction task on the first computing unit and the second computing unit according to the reconstruction ratio of the data reconstruction task;

[0121] Obtain the current business pressure level;

[0122] According to the determined resource that restricts the computing speed of the storage device, the computing ability of the first computing unit, the computing ability of the second computing unit, the resource consumption magnification of the data reconstruction task on the first computing unit and the second computing unit, and the business pressure level, divide the data reconstruction task into multiple subtasks.

[0123] In summary, in the embodiment of the present application, determine the resource that restricts the computing speed of the storage device, divide the computing task into multiple subtasks according to the determined resource, allocate the first subtask among the multiple subtasks to the first computing unit, and allocate the second subtask among the multiple subtasks to the second computing unit. On the basis of determining the resource that restricts the computing speed of the storage device, allocating the computing amount according to this resource can, while making better use of this resource, cooperate with other resources to complete the computing of the computing task, so that the computing speed of the storage device when executing the computing task is better.

[0124] It should be noted that: when the device for allocating the computing task amount provided in the above embodiment allocates the computing task amount, only the above division of each functional module is used as an example. In practical applications, the above functions can be allocated by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for allocating the computing task amount provided in the above embodiment and the method embodiment for allocating the computing task amount belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be elaborated here.

[0125] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc.

[0126] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc.

[0127] It should be understood that the "at least one" mentioned herein refers to one or more, and "a plurality" refers to two or more. In the description of the present application, unless otherwise specified, " / " means "or". For example, A / B can mean A or B. The "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit being different.

[0128] The above are the embodiments provided by the present application, which are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for allocating the amount of computing tasks, characterized in that The method is applied to a storage device, which includes at least one first operation unit and a first memory corresponding to the at least one first operation unit, at least one second operation unit and a second memory corresponding to the at least one second operation unit. The at least one first operation unit and the first memory are located within a controller or a hard disk enclosure of the storage device. The at least one second operation unit and the second memory are located within the controller or the hard disk enclosure. The method includes: Determine the resources that limit the operation speed of the storage device. The resources include the memory bandwidth of the first memory, or the memory bandwidth of the second memory, or the link bandwidth between the controller and the hard disk enclosure, or the read / write capabilities of the hard disks included in the hard disk enclosure; According to the determined resources that limit the operation speed of the storage device, divide the operation task into multiple subtasks, allocate the first subtask among the multiple subtasks to the first operation unit, and allocate the second subtask among the multiple subtasks to the second operation unit.

2. The method according to claim 1, wherein The determination of the resources that limit the operation speed of the storage device includes: Respectively determine the operation speeds corresponding to the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the read / write capabilities of the hard disks; Determine the resource with the minimum corresponding operation speed as the resource that limits the operation speed of the storage device.

3. The method according to claim 1 or 2, characterized in that If the determined resource that limits the operation speed of the storage device is the memory bandwidth of the first memory, and the difference value between the operation capabilities of the first operation unit and the second operation unit is within the reference threshold range, then the amount of computation of the first subtask is less than that of the second subtask.

4. The method according to claim 1 or 2, characterized in that If the determined resource that limits the operation speed of the storage device is the memory bandwidth of the second memory, and the difference value between the operation capabilities of the first operation unit and the second operation unit is within the reference threshold range, then the amount of computation of the first subtask is greater than that of the second subtask.

5. The method according to claim 1 or 2, characterized in that, If the determined resource that limits the operation speed of the storage device is the link bandwidth between the controller and the hard disk enclosure, and the first operation unit is located within the controller and the second operation unit is located within the hard disk enclosure, then the amount of computation of the first subtask is less than that of the second subtask.

6. The method according to claim 1 or 2, characterized in that, If the determined resource that limits the operation speed of the storage device is the read / write capabilities of the hard disks, and the first operation unit is located within the controller and the second operation unit is located within the hard disk enclosure, then the amount of computation of the first subtask is less than that of the second subtask.

7. The method according to claim 1 or 2, characterized in that, The operation task is a data reconstruction task. The division of the operation task into multiple subtasks according to the determined resources that limit the operation speed of the storage device includes: Determine the resource consumption magnification factors of the data reconstruction task on the first operation unit and the second operation unit according to the reconstruction ratio of the data reconstruction task; Obtain the current degree of service pressure; Divide the data reconstruction task into multiple subtasks according to the determined resources that limit the computing speed of the storage device, the computing capabilities of the first computing unit and the second computing unit, the resource consumption magnification of the data reconstruction task on the first computing unit and the second computing unit, and the degree of service pressure.

8. An apparatus for allocating the amount of operation tasks, characterized in that The device is applied to a storage device, which includes at least one first computing unit and a first memory corresponding to the at least one first computing unit, at least one second computing unit and a second memory corresponding to the at least one second computing unit. The at least one first computing unit and the first memory are located in the controller or the hard disk enclosure of the storage device, and the at least one second computing unit and the second memory are located in the controller or the hard disk enclosure. The device includes: A determination module, configured to determine the resources that limit the computing speed of the storage device, where the resources include the memory bandwidth of the first memory, or the memory bandwidth of the second memory, or the link bandwidth between the controller and the hard disk enclosure, or the read / write capabilities of the hard disks included in the hard disk enclosure; An allocation module, configured to divide the computing task into multiple subtasks according to the determined resources that limit the computing speed of the storage device, allocate the first subtask among the multiple subtasks to the first computing unit, and allocate the second subtask among the multiple subtasks to the second computing unit.

9. The device according to claim 8, characterized in that The determination module is mainly used for: Determine the computing speeds corresponding to the memory bandwidth of the first memory, the memory bandwidth of the second memory, the link bandwidth between the controller and the hard disk enclosure, and the read / write capabilities of the hard disks respectively; Determine the resource with the minimum corresponding computing speed as the resource that limits the computing speed of the storage device.

10. The device according to claim 8 or 9, characterized in that, If the determined resource that limits the computing speed of the storage device is the memory bandwidth of the first memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the amount of computation of the first subtask is less than that of the second subtask.

11. The device according to claim 8 or 9, characterized in that, If the determined resource that limits the computing speed of the storage device is the memory bandwidth of the second memory, and the difference value between the computing capabilities of the first computing unit and the second computing unit is within the reference threshold range, then the amount of computation of the first subtask is greater than that of the second subtask.

12. The device according to claim 8 or 9, characterized in that, If the determined resource that limits the computing speed of the storage device is the link bandwidth between the controller and the hard disk enclosure, and the first computing unit is located in the controller and the second computing unit is located in the hard disk enclosure, then the amount of computation of the first subtask is less than that of the second subtask.

13. The device according to claim 8 or 9, characterized in that, If the determined resource that limits the computing speed of the storage device is the read / write capabilities of the hard disks, and the first computing unit is located in the controller and the second computing unit is located in the hard disk enclosure, then the amount of computation of the first subtask is less than that of the second subtask.

14. The device according to claim 8 or 9, characterized in that, The operation task is a data reconstruction task, and the allocation module is mainly used for: Determining the resource consumption magnification factors of the data reconstruction task on the first operation unit and the second operation unit according to the reconstruction ratio of the data reconstruction task; Obtaining the current degree of service pressure; Dividing the data reconstruction task into multiple subtasks according to the determined resources that limit the operation speed of the storage device, the operation capabilities of the first operation unit and the second operation unit, the resource consumption magnification factors of the data reconstruction task on the first operation unit and the second operation unit, and the degree of service pressure.

15. A storage device, characterized in that, The storage device includes at least one first operation unit and a first memory corresponding to the at least one first operation unit, at least one second operation unit and a second memory corresponding to the at least one second operation unit. The at least one first operation unit and the first memory are located in a controller or a hard disk enclosure. The at least one second operation unit and the second memory are located in the controller or the hard disk enclosure. The operation units included in the controller include a processor and a memory corresponding to the processor. The memory corresponding to the processor is used to store a computer program for executing the method according to any one of claims 1-7, and to store data involved in implementing the method according to any one of claims 1-7. The processor is used to execute the computer program stored in the memory corresponding to the processor to implement the method according to any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium, and when the instructions run on a computer, the computer is caused to execute the method according to any one of claims 1-7.

Images