A method, device, storage medium and electronic device for executing a task

Abstract

This specification discloses a method, apparatus, storage medium, and electronic device for task execution. The task execution method includes: determining device information corresponding to each computing device for a computing task corresponding to each network layer in a target model; for each network layer, determining the computing time required for executing the computing task corresponding to the network layer through each computing device based on the number of calculations involved in executing the computing task corresponding to the network layer and the device information of each computing device; determining the target device corresponding to the network layer based on at least one of the computing time, the data transmission time between the computing device corresponding to the previous network layer and the other computing devices, the memory space required for the data of the network layer, and the remaining memory of each computing device; and executing the computing task through the target device corresponding to each network layer after receiving an execution request for the computing task corresponding to each network layer.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, device, storage medium, and electronic device for executing a task. Background Art

[0002] With the development of deep neural network technology, some extremely large deep neural network models have gradually emerged. Due to the extremely large number of parameters in these models, the computing systems used to train them have brought many challenges. Large-scale training not only requires a large amount of computing resources, but also places high demands on device memory and transmission resources. Therefore, parallel training of models has gradually become a mainstream development trend.

[0003] Currently, models are usually divided into multiple computing tasks and assigned to different graphics processing units (GPUs) to perform operations, while the central processing unit (CPU) is only responsible for data transmission and scheduling. This method requires a large amount of GPU computing resources, while the CPU computing resources are not effectively utilized. This not only causes a waste of resources during model training, but deploying more GPUs to perform computing tasks will further increase the cost of model training.

[0004] Therefore, how to improve the utilization of different types of equipment during model training and reduce the training cost of the model is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a method, device, storage medium, and electronic device for executing a task to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for executing a task, including:

[0008] Get the model data of the target model;

[0009] Parsing the model data to determine a computing task corresponding to each network layer in the target model, and determining device information corresponding to each computing device, wherein the computing device includes at least one central processing unit (CPU) and at least one graphics processing unit (GPU);

[0010] For each network layer, determine the computation time required for each computing device to perform the computation task corresponding to the network layer based on the number of computations involved in performing the computation task corresponding to the network layer and the device information corresponding to each computing device;

[0011] Determining, based on at least one of the computing duration, the data transmission duration between the computing device that performs the computing task corresponding to the previous network layer and the other computing devices, the memory space required for the data of the network layer, and the remaining memory of each computing device, the computing device that performs the computing task corresponding to the network layer is determined as the target device corresponding to the network layer, wherein the data transmission duration is determined based on the transmission information between the computing device that performs the computing task corresponding to the previous network layer and the other computing devices, and the amount of data output by the previous network layer;

[0012] Each network layer is deployed in a target device corresponding to each network layer, so that after receiving an execution request for a computing task corresponding to each network layer, the computing task is executed through the target device corresponding to each network layer.

[0013] Optionally, before determining the computing device corresponding to the network layer, the method further includes:

[0014] determining at least one execution unit in each computing device;

[0015] The computing capability corresponding to each execution unit in each computing device is determined based on the device information corresponding to each computing device.

[0016] Optionally, for each network layer, based on the number of calculations involved in executing the computing task corresponding to the network layer and the device information corresponding to each computing device, determining the computing time required for executing the computing task corresponding to the network layer by each computing device, specifically including:

[0017] For each network layer, the computing time required for each execution unit to execute the computing task corresponding to the network layer is determined based on the number of calculations involved in executing the computing task corresponding to the network layer and the computing capacity corresponding to each execution unit.

[0018] Optionally, determining the computing device that performs the computing task corresponding to the network layer as the target device corresponding to the network layer based on at least one of the computing duration, the data transmission duration between the computing device that performs the computing task corresponding to the previous network layer and other computing devices, the memory space required for the data of the network layer, and the remaining memory of each computing device, specifically includes:

[0019] Determine, based on the calculation duration corresponding to each execution unit, the data transmission duration between the execution unit that executes the calculation task corresponding to the previous network layer and other execution units, the memory space, and at least one of the remaining memory, the execution unit that executes the calculation task corresponding to the network layer as the target execution unit corresponding to the network layer;

[0020] Deploying each network layer in a target device corresponding to each network layer, so as to execute the computing task corresponding to each network layer through the target device corresponding to each network layer after receiving the execution request of the computing task corresponding to each network layer, specifically including:

[0021] Each network layer is deployed in a target device where an execution unit corresponding to each network layer is located, so that after receiving an execution request for a computing task corresponding to each network layer, the computing task is executed by the execution unit corresponding to each network layer.

[0022] Optionally, the data transmission duration is determined based on the transmission information between the execution unit that executes the computing task corresponding to the previous network layer and the other execution units, and the amount of data output by the previous network layer.

[0023] Optionally, determining an execution unit that executes the computing task corresponding to the network layer as a target execution unit corresponding to the network layer based on the computing time corresponding to each execution unit, the data transmission time between the execution unit that executes the computing task corresponding to the previous network layer and other execution units, the memory space, and at least one of the remaining memory, specifically includes:

[0024] If the network layer is not the starting network layer of the target model, then for each execution unit, determining the comprehensive duration corresponding to the execution unit according to the calculation duration and data transmission duration corresponding to the execution unit;

[0025] Determine, among the execution units, an execution unit whose remaining memory of the corresponding computing device is greater than the memory space and whose corresponding comprehensive duration is less than the computing duration required for the execution unit that executes the computing task corresponding to the previous network layer to execute the computing task corresponding to the network layer as a candidate execution unit;

[0026] The target execution unit corresponding to the network layer is determined according to the comprehensive duration corresponding to each candidate execution unit.

[0027] Optionally, determining an execution unit that executes the computing task corresponding to the network layer as a target execution unit corresponding to the network layer based on the computing time corresponding to each execution unit, the data transmission time between the execution unit that executes the computing task corresponding to the previous network layer and other execution units, the memory space, and at least one of the remaining memory, specifically includes:

[0028] If the network layer is the starting network layer of the target model, the calculation time corresponding to each execution unit is determined according to the number of calculations required when executing the corresponding calculation task of the network layer and the corresponding computing capacity of each execution unit;

[0029] The execution unit corresponding to the network layer is determined according to the calculation time corresponding to each execution unit, the memory space and the remaining memory.

[0030] Optionally, determining the execution unit corresponding to the network layer according to the calculation duration corresponding to each execution unit, the memory space, and the remaining memory specifically includes:

[0031] Determine an execution unit whose remaining memory is greater than the memory space as a candidate execution unit;

[0032] The target execution unit corresponding to the network layer is determined based on the computation time corresponding to each candidate execution unit.

[0033] Optionally, the method further includes:

[0034] For each network layer, if there are at least two transmission modes between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer of the network layer, the transmission status of the current transmission mode is monitored;

[0035] Determining whether the current transmission mode needs to be adjusted according to the transmission status;

[0036] If so, adjust the current transmission mode to another transmission mode.

[0037] Optionally, the transmission information includes: a transmission mode between each execution unit and a bandwidth corresponding to each transmission mode, and the transmission mode includes: at least one of a first transmission mode and a second transmission mode.

[0038] Optionally, the method further includes:

[0039] If the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer are both execution units in the GPU device, and the second transmission mode exists between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer, then the data between the network layer and the adjacent network layer is transmitted through the second transmission mode; otherwise, the data between the network layer and the adjacent network layer is transmitted through the first transmission mode.

[0040] Optionally, the first transmission mode includes: PCI-E serial computer expansion bus, and the second transmission mode includes: NVLINK.

[0041] This specification provides a task execution device, including:

[0042] Get module to get model data of target model;

[0043] a parsing module that parses the model data to determine a computing task corresponding to each network layer in the target model, and determines device information corresponding to each computing device, wherein the computing device includes at least one central processing unit (CPU) and at least one graphics processing unit (GPU);

[0044] A first determining module determines, for each network layer, a computing time required for each computing device to execute the computing task corresponding to the network layer based on the number of computations involved in executing the computing task corresponding to the network layer and device information corresponding to each computing device;

[0045] a second determining module, determining, based on at least one of the calculation duration, a data transmission duration between the computing device that performs the computing task corresponding to the previous network layer and other computing devices, a memory space required for data of the network layer, and remaining memory of each computing device, a computing device that performs the computing task corresponding to the previous network layer as a target device corresponding to the network layer, wherein the data transmission duration is determined based on transmission information between the computing device that performs the computing task corresponding to the previous network layer and the other computing devices, and an amount of data output by the previous network layer;

[0046] The execution module deploys each network layer in the target device corresponding to each network layer, so as to execute the computing task through the target device corresponding to each network layer after receiving the execution request of the computing task corresponding to each network layer.

[0047] This specification provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for executing the above-mentioned task is implemented.

[0048] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned task execution method when executing the program.

[0049] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0050] In the task execution method provided in this specification, the device information corresponding to each computing device of the computing task corresponding to each network layer in the target model is determined. For each network layer, the computing time required for executing the computing task corresponding to the network layer through each computing device is determined based on the number of calculations involved in executing the computing task corresponding to the network layer and the device information of each computing device. The target device corresponding to the network layer is determined based on the computing time, the data transmission time between the computing device corresponding to the previous network layer and other computing devices, the memory space required for the data of the network layer, and at least one of the remaining memory of each computing device. After receiving the execution request of the computing task corresponding to each network layer, the computing task is executed through the target device corresponding to each network layer.

[0051] It can be seen from the above method that this solution can allocate different computing devices to different network layers based on the device information of different types of computing devices (CPU and GPU) to perform corresponding computing tasks. Compared with the current method of only using GPU to perform major computing tasks, in this solution, both CPU and GPU can determine the network layer that matches it based on its device information, and then execute the computing tasks corresponding to the network layer, which greatly improves the utilization rate of different computing devices and further reduces the execution cost of computing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0053] Figure 1 A flowchart of a method for executing a task provided in this specification;

[0054] Figure 2 This is a diagram of the task execution process during model training provided in this manual;

[0055] Figure 3 A schematic diagram of a task execution system provided in this specification;

[0056] Figure 4 A schematic diagram of a device for performing a task provided in this specification;

[0057] Figure 5 This manual provides a corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0059] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0060] Figure 1 This is a flowchart of a method for executing a task provided in this specification, which includes the following steps:

[0061] S101: Acquire model data of a target model.

[0062] S102: Parse the model data to determine the computing tasks corresponding to each network layer in the target model, and determine the device information corresponding to each computing device, where the computing device includes at least one central processing unit (CPU) and at least one graphics processing unit (GPU).

[0063] During model training, the training processes of each network layer of the model usually have a sequential relationship. The input calculated by the next network layer depends on the output calculated by the previous network layer. Therefore, many GPUs spend most of the time waiting for the calculation of the upper layer input to be completed, resulting in idle computing resources.

[0064] Pipeline parallelism is a method that accelerates neural network training by combining model parallelism with data pipelining. The core idea is to partition the model layer by layer into several chunks, each assigned to a separate device. During the forward pass, each device passes intermediate activations to the next stage. During the backward pass, each device propagates the gradients of the input tensor back to the previous pipeline stage. This approach mitigates some of the resource waste associated with model parallelism.

[0065] However, during the current parallel model training process, most computing tasks are completed by the GPU, and the CPU only performs data transmission and control, resulting in low CPU resource utilization. The CPU is even idle most of the time, causing a waste of CPU resources.

[0066] Based on this, this specification provides a method for task execution, which determines the execution unit corresponding to the computing task of each network layer of the target model among the execution units of different types of computing devices, and then executes the computing task to improve the utilization rate of different types of computing resources.

[0067] In this specification, the execution subject of the method for executing the task execution may refer to a designated device such as a server. For the sake of convenience, this specification only takes the server as the execution subject as an example to illustrate a task execution method provided in this specification.

[0068] Among them, in the process of executing the computing task for the target model, the server needs to obtain the model data of the target model. The computing task can be a computing task in the model training process (including computing tasks in the forward transfer and backward transfer processes). After obtaining the training request of the model, the server can obtain the model data. Of course, the above computing task can also be a computing task of the model in the actual prediction process.

[0069] After obtaining the model data, the server can parse the model data. For example, the server can store the model structure in the form of a data flow graph. After parsing the structure, the server can determine the computing tasks corresponding to each network layer of the target model.

[0070] At the same time, the server can determine the device information corresponding to each computing device. Each computing device can be deployed in the same terminal device (such as a computer), and of course, can also be deployed in multiple nodes of a distributed computing system.

[0071] In this specification, the above-mentioned computing device may include at least one central processing unit (CPU) and at least one graphics processing unit (GPU). Of course, it may also include other computing devices such as a tensor processing unit (TPU) and an embedded neural network processor (NPU). This specification does not make specific limitations on this.

[0072] The device information of each computing device may include the number of CPUs, CPU frequency, CPU memory size, number of GPUs, number of GPU stream processors, GPU stream processor frequency, and GPU memory size, etc. This manual does not make specific restrictions on this.

[0073] In addition, the server may also determine transmission information between the computing devices, where the transmission information includes transmission modes between different computing devices and bandwidths corresponding to different transmission modes. In this specification, the transmission mode may include at least one of a first transmission mode and a second transmission mode, where the first transmission mode may include a high-speed serial computer expansion bus PCI-E (Peripheral Component Interconnect Express), and the second transmission mode may include NVLINK, where NVLINK is used for data transmission between GPUs.

[0074] Furthermore, the server may determine at least one execution unit included in each computing device and instantiate each execution unit. In this specification, an execution unit may be determined based on the thread warps of the computing device, or alternatively, based on the number of cores of the computing device, which is not specifically limited in this specification.

[0075] For each execution unit, the server may determine the computing capability corresponding to the execution unit based on the device information of the computing device to which the execution unit belongs. The computing capability may be expressed as the number of floating-point operations per second (flops) corresponding to the execution unit.

[0076] S 103: For each network layer, determine the computing time required for each computing device to execute the computing task corresponding to the network layer according to the number of calculations involved in executing the computing task corresponding to the network layer and the device information corresponding to each computing device.

[0077] S104: Determine the computing device that performs the computing task corresponding to the previous network layer as the target device corresponding to the network layer based on at least one of the computing duration, the data transmission duration between the computing device that performs the computing task corresponding to the previous network layer and other computing devices, the memory space required for the data of the network layer, and the remaining memory of each computing device, wherein the data transmission duration is determined based on the transmission information between the computing device that performs the computing task corresponding to the previous network layer and the other computing devices, and the amount of data output by the previous network layer.

[0078] In the process of determining the computing devices corresponding to each network layer, for each network layer, the server derives the number of calculations required to execute the computing tasks corresponding to the network layer (such as the number of floating-point operations) and the amount of data output by the network layer, and determines the memory space required for the data of the network layer (such as model parameters, tensors, etc.).

[0079] For the starting network layer of the target model, the server can determine the execution unit corresponding to the starting network layer based on the number of calculations and memory space corresponding to the starting network layer, the computing power corresponding to each execution unit, and the remaining memory of the computing device corresponding to each execution unit. The method for determining the execution unit can be expressed by the following formula:

[0080]

[0081] Among them, L a .flop is the number of calculations required to start the network layer a (number of floating-point operations), EC i is the computing power (floating point operations per second) corresponding to execution unit i, i is the remaining memory of the computing device corresponding to execution unit i, La .Mem is the memory space required by network layer a.

[0082] The server can determine the execution unit whose remaining memory is greater than the memory space required by network layer a according to the above formula, as a candidate execution unit, and determine the calculation time corresponding to each execution unit according to the number of calculations required for the starting network layer a and the computing power corresponding to each execution unit. Then, based on the calculation time corresponding to each candidate execution unit, determine the unique execution unit corresponding to the network layer (such as taking the candidate execution unit with the smallest calculation time as the execution unit corresponding to the starting network layer a) as the target execution unit corresponding to the network layer.

[0083] For network layers other than the starting network layer, the server can determine the execution unit corresponding to the network layer based on the number of calculations and the required memory space corresponding to the network layer, the computing power corresponding to each execution unit, the remaining memory of the computing device corresponding to each execution unit, the amount of data output by the network layer above the network layer, and the transmission information between the execution unit corresponding to the previous network layer (i.e., the execution unit that performs the computing task corresponding to the previous network layer) and each execution unit. The method for determining the execution unit can be expressed by the following formula:

[0084]

[0085] Among them, L a_1 .size is the amount of data output by the previous network layer, BW i,j,k It is the bandwidth between the execution unit j corresponding to the previous network layer and other execution units i. When the execution unit j and the execution unit i belong to different computing devices, k has two values, representing two transmission modes respectively. k = 0 means using PCI-e for data transmission, and k = 1 means using NVLINK for data transmission.

[0086] Specifically, the server can determine the calculation duration corresponding to each execution unit based on the above formula, according to the number of calculations corresponding to the network layer a (a>1) and the computing power corresponding to each execution unit, and determine the data transmission duration corresponding to each execution unit according to the transmission information between the computing device corresponding to the previous network layer and each computing device, as well as the amount of data output by the previous network layer. Then, based on the above calculation duration and data transmission duration, the comprehensive duration corresponding to each network layer is determined, that is,

[0087] Furthermore, in the process of determining the execution unit corresponding to network layer a (a>1), the server can determine, among each execution unit, an execution unit whose remaining memory of the corresponding computing device is greater than the memory space required by network layer a (a>1), and whose corresponding comprehensive duration is less than the computing time required to execute the corresponding computing task of the network layer through the execution unit of the previous network layer, as a candidate execution unit, and then determine the target execution unit corresponding to the network layer based on the comprehensive duration corresponding to each candidate execution unit (such as taking the candidate execution unit with the smallest comprehensive duration as the computing unit corresponding to network layer a).

[0088] It should be noted that in actual applications, it is not necessary to divide the execution units corresponding to each computing device. Instead, the computing devices corresponding to different network layers can be determined. Each computing device can be equivalent to an execution unit, so that the computing tasks corresponding to each network layer are executed through the computing devices corresponding to different network layers.

[0089] During this process, the server can determine, for each network layer, the computing time required for each computing device to execute the computing task corresponding to the network layer based on the number of calculations involved in executing the computing task corresponding to the network layer and the device information corresponding to each computing device, determine the data transmission time corresponding to each computing device based on the transmission information between the computing device corresponding to the previous network layer (i.e., the computing device that executes the computing task corresponding to the previous network layer) and each computing device, and the amount of data output by the previous network layer, and then determine the comprehensive time of each computing device based on the computing time and data transmission time corresponding to each computing device, and then determine the computing device corresponding to the network layer as the target device corresponding to the network layer based on the comprehensive time corresponding to each computing device, the remaining memory of each computing device and at least one of the above-mentioned memory spaces.

[0090] S105: deploying each network layer in a target device corresponding to each network layer, so as to execute the computing task corresponding to each network layer through the target device corresponding to each network layer after receiving an execution request of the computing task corresponding to each network layer.

[0091] Specifically, the server can output a tuple list based on the device information, transmission information, and inference information. The list may include information such as the neural model network layer, forward and backward operations, location in the computing device, whether direct memory access (DMA) is used, and whether NVLINK is used. The server can then execute the computing task based on the tuple list through the execution unit corresponding to each network layer.

[0092] In the process of executing computing tasks, the server can first deploy each network layer on the target device where the corresponding target execution unit is located. It should be noted that for each network layer, if there are at least two transmission modes (including NVLINK and PCI-e) between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer of the network layer, the transmission status of the current transmission mode is monitored, and based on the monitored transmission status, it is determined whether the current transmission mode needs to be adjusted. The above-mentioned transmission status may include the busyness or traffic size of the current transmission mode. If the current transmission mode is relatively busy, the current transmission mode can be adjusted to another transmission mode.

[0093] In actual applications, NVLINK (the second transmission mode) exists only between the execution units of the GPU, and does not exist between the execution units of the CPU or between the execution units of the CPU and GPU.

[0094] In addition, in this specification, for each network layer, if the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer are both execution units in the GPU device, and there is an NVLINK transmission method between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer, since the transmission efficiency of the NVLINK transmission method is much higher than PCI-e, the server can give priority to transmitting data between the network layer and the adjacent network layer through the NVLINK transmission method. If not, it is transmitted through PCI-e.

[0095] During the execution of computing tasks, when the NVLINK transmission status is detected to be relatively busy, NVLINK can be adjusted to PCI-e. In the case of two transmission modes at the same time, when the PCI-e transmission status is relatively busy, it can also be adjusted to NVLINK to ensure the execution efficiency of computing tasks. For ease of understanding, this manual provides a schematic diagram of the task execution process during model training, such as Figure 2 shown.

[0096] Figure 2 This is a diagram of the task execution process during model training provided in this manual.

[0097] Among them, in the process of training the target model, the server can first obtain the training request submitted by the user, and then parse the structure of the model to determine the computing tasks corresponding to each network layer and the computing amount (number of floating-point operations) corresponding to each network layer. At the same time, it obtains the device information of each computing device, determines the execution unit corresponding to each computing device, and determines whether there is NVLINK between the execution units. If so, its bandwidth is determined, and then the allocation strategy is determined based on the above device information, the estimated information of each network layer and the transmission information.

[0098] After determining the allocation strategy, the server can initialize each execution unit and monitor the transmission status after starting training. If it is determined that the transmission method needs to be modified, the transmission method will be adjusted after the conditions are met. The forward and backward operations in the model training process are executed through each target execution unit, and the optimizer outputs the model parameters. The training of the target model is stopped after the preset number of training times is reached.

[0099] In this specification, a corresponding task execution system can also be deployed in the server to complete the computing task through the task execution system. For ease of understanding, this specification provides a task execution system diagram, such as Figure 3 shown.

[0100] Figure 3 This is a schematic diagram of a task execution system provided in this manual.

[0101] Among them, the task execution system includes an execution unit manager, a monitoring module and a dynamic resource scheduler. The dynamic resource scheduler includes: a model analysis module, an allocation module, a judgment module and a configuration selection module. The execution unit manager includes the execution unit and transmission unit corresponding to each computing device. The monitoring module includes a function modification unit and a reallocation unit.

[0102] The execution unit manager is responsible for instantiating multiple execution units on each computing device. For example, a GPU can have one or more execution units. These execution units are responsible for completing the computing tasks corresponding to each network layer and providing input data for the next layer of the network; the transmission unit is responsible for transmitting the input data of the next layer. The number of transmission units and execution units is not consistent. If the next layer of the network and the execution unit exist on different computing devices, then a transmission unit will be instantiated. If the next layer of the network and the neural network layer that the execution unit is responsible for are in the same computing unit, then there is no need to generate a transmission unit. The transmission unit is generated based on whether data transmission is required in the allocation plan, and the transmission method of the transmission unit is also determined based on the allocation plan.

[0103] The monitoring module is responsible for monitoring the changes in node resources during each batch processing of the deep neural network to calculate whether it is necessary to modify the data transmission method between the deep neural network layers. When the allocation scheme changes (that is, the execution units corresponding to each network layer are changed), the function modification unit of the monitoring module can modify the transmission method of the corresponding transmission unit. Through such modifications, the overlap of data transmission and data calculation in the parallel process of the model can be dynamically adjusted to improve the efficiency of deep neural network training.

[0104] It can be seen from the above method that this solution can allocate different computing devices to different network layers based on the device information of different types of computing devices (CPU and GPU) to perform corresponding computing tasks. Compared with the current method of only using GPU to perform major computing tasks, in this solution, both CPU and GPU can determine the network layer that matches it based on its device information, and then execute the computing tasks corresponding to the network layer, which greatly improves the utilization rate of different computing devices and further reduces the execution cost of computing tasks.

[0105] In addition, this solution can also monitor the transmission status of the transmission mode between each execution unit, so as to adjust the transmission mode when the change conditions are met to ensure the data communication efficiency during model training.

[0106] The above are one or more methods for implementing task execution in this specification. Based on the same idea, this specification also provides corresponding task execution devices, such as Figure 4 shown.

[0107] Figure 4 A schematic diagram of a device for performing a task provided in this specification, including:

[0108] Acquisition module 401, used to acquire model data of the target model;

[0109] A parsing module 402 is configured to parse the model data, determine a computing task corresponding to each network layer in the target model, and determine device information corresponding to each computing device, wherein the computing device includes at least one central processing unit (CPU) and at least one graphics processing unit (GPU);

[0110] A first determining module 403 is configured to determine, for each network layer, a computing time required for each computing device to perform the computing task corresponding to the network layer based on the number of computations involved in performing the computing task corresponding to the network layer and device information corresponding to each computing device;

[0111] a second determining module 404 configured to determine, based on at least one of the computing duration, a data transmission duration between the computing device that performs the computing task corresponding to the previous network layer and other computing devices, a memory space required for data of the network layer, and remaining memory of each computing device, a computing device that performs the computing task corresponding to the previous network layer as a target device corresponding to the network layer, wherein the data transmission duration is determined based on transmission information between the computing device that performs the computing task corresponding to the previous network layer and the other computing devices, and an amount of data output by the previous network layer;

[0112] The execution module 405 is used to deploy each network layer in the target device corresponding to each network layer, so as to execute the computing task corresponding to each network layer through the target device corresponding to each network layer after receiving the execution request of the computing task corresponding to each network layer.

[0113] Optionally, before determining the computing device corresponding to the network layer, the first determination module 403 is also used to determine at least one execution unit in each computing device; and determine the computing capability corresponding to each execution unit in each computing device based on the device information corresponding to each computing device.

[0114] Optionally, the first determination module 403 is specifically used to determine, for each network layer, the computing time required for each execution unit to execute the computing task corresponding to the network layer based on the number of calculations involved in executing the computing task corresponding to the network layer and the computing capacity corresponding to each execution unit.

[0115] Optionally, the second determination module 404 is specifically used to determine the execution unit that executes the computing task corresponding to the network layer as the target execution unit corresponding to the network layer based on the computing time corresponding to each execution unit, the data transmission time between the execution unit that executes the computing task corresponding to the previous network layer and other execution units, the memory space, and at least one of the remaining memory; deploy each network layer in the target device corresponding to each network layer, so that after receiving the execution request of the computing task corresponding to each network layer, the computing task is executed through the target device corresponding to each network layer, specifically including: deploying each network layer in the target device where the execution unit corresponding to each network layer is located, so that after receiving the execution request of the computing task corresponding to each network layer, the computing task is executed through the execution unit corresponding to each network layer.

[0116] Optionally, the data transmission duration is determined based on the transmission information between the execution unit that executes the computing task corresponding to the previous network layer and the other execution units, and the amount of data output by the previous network layer.

[0117] Optionally, the second determination module 404 is specifically used to, if the network layer is not the starting network layer of the target model, determine, for each execution unit, the comprehensive duration corresponding to the execution unit based on the calculation duration and data transmission duration corresponding to the execution unit; determine, among each execution unit, an execution unit whose remaining memory of the corresponding computing device is greater than the memory space and whose corresponding comprehensive duration is less than the calculation time required for executing the calculation task corresponding to the previous network layer by the execution unit that executes the calculation task corresponding to the previous network layer, as a candidate execution unit; determine the target execution unit corresponding to the network layer based on the comprehensive duration corresponding to each candidate execution unit.

[0118] Optionally, the second determination module 404 is specifically used to, if the network layer is the starting network layer of the target model, determine the calculation time corresponding to each execution unit based on the number of calculations required to execute the corresponding calculation task of the network layer and the computing power corresponding to each execution unit; determine the execution unit corresponding to the network layer based on the calculation time corresponding to each execution unit, the memory space and the remaining memory.

[0119] Optionally, the second determination module 404 is specifically used to determine an execution unit whose remaining memory is greater than the memory space as a candidate execution unit; and determine a target execution unit corresponding to the network layer based on the calculation time corresponding to each candidate execution unit.

[0120] Optionally, the execution module 405 is also used to monitor the transmission status of the current transmission mode for each network layer if there are at least two transmission modes between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer of the network layer; determine whether the current transmission mode needs to be adjusted based on the transmission status; if so, adjust the current transmission mode to another transmission mode.

[0121] Optionally, the transmission information includes: a transmission mode between each execution unit and a bandwidth corresponding to each transmission mode, and the transmission mode includes: at least one of a first transmission mode and a second transmission mode.

[0122] Optionally, the execution module 405 is also used to, if the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer are both execution units in the GPU device, and the second transmission mode exists between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer, then transmit the data between the network layer and the adjacent network layer through the second transmission mode; otherwise, transmit the data between the network layer and the adjacent network layer through the first transmission mode.

[0123] Optionally, the first transmission mode includes: PCI-E serial computer expansion bus, and the second transmission mode includes: NVLINK.

[0124] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 Provides a method for executing a task.

[0125] This manual also provides Figure 5 The one shown corresponds to Figure 1 Schematic diagram of the electronic equipment. Figure 5 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0126] Improvements to a technology can be clearly distinguished as either hardware improvements (for example, improvements to circuit structures such as diodes, transistors, and switches) or software improvements (improvements to process flows). However, with technological advancements, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always program the improved process flow into the hardware circuit to obtain the corresponding hardware circuit structure. Therefore, it cannot be said that a process flow improvement cannot be implemented using a hardware module. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0127] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0128] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0129] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0130] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0136] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0138] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0140] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0141] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for executing a task, characterized in that: include: Get the model data of the target model; Parsing the model data to determine a computing task corresponding to each network layer in the target model, and determining device information corresponding to each computing device, wherein the computing device includes at least one central processing unit (CPU) and at least one graphics processing unit (GPU); For each network layer, determine the computation time required for executing the computation task corresponding to the network layer by the execution unit in each computing device based on the number of computations involved in executing the computation task corresponding to the network layer and the device information corresponding to each computing device; Determining, based on at least one of the computational duration corresponding to each execution unit, the data transmission duration between the execution unit that executes the computational task corresponding to the previous network layer and the other execution units, the memory space required for the data of the network layer, and the remaining memory of each computing device, the execution unit that executes the computational task corresponding to the network layer is determined as the target execution unit corresponding to the network layer; wherein the data transmission duration is determined based on the transmission information between the computing device that executes the computational task corresponding to the previous network layer and the other computing devices, and the amount of data output by the previous network layer; Each network layer is deployed in a target device where an execution unit corresponding to each network layer is located, so that after receiving an execution request for a computing task corresponding to each network layer, the computing task is executed by the execution unit corresponding to each network layer.

2. The method according to claim 1, wherein Before determining the computing device corresponding to the network layer, the method further includes: determining at least one execution unit in each computing device; The computing capability corresponding to each execution unit in each computing device is determined based on the device information corresponding to each computing device.

3. The method according to claim 2, wherein For each network layer, based on the number of calculations involved in executing the computing task corresponding to the network layer and the device information corresponding to each computing device, determine the computing time required for each computing device to execute the computing task corresponding to the network layer, specifically including: For each network layer, the computing time required for each execution unit to execute the computing task corresponding to the network layer is determined based on the number of calculations involved in executing the computing task corresponding to the network layer and the computing capacity corresponding to each execution unit.

4. The method according to claim 3, wherein The data transmission duration is determined based on the transmission information between the execution unit that executes the computing task corresponding to the previous network layer and the other execution units, as well as the amount of data output by the previous network layer.

5. The method according to claim 3, wherein Determining, based on the calculation duration corresponding to each execution unit, the data transmission duration between the execution unit that executes the calculation task corresponding to the previous network layer and other execution units, the memory space, and at least one of the remaining memory, the execution unit that executes the calculation task corresponding to the network layer is determined as the target execution unit corresponding to the network layer, specifically including: If the network layer is not the starting network layer of the target model, then for each execution unit, determining the comprehensive duration corresponding to the execution unit according to the calculation duration and data transmission duration corresponding to the execution unit; Determine, among the execution units, an execution unit whose remaining memory of the corresponding computing device is greater than the memory space and whose corresponding comprehensive duration is less than the computing duration required for the execution unit that executes the computing task corresponding to the previous network layer to execute the computing task corresponding to the network layer as a candidate execution unit; The target execution unit corresponding to the network layer is determined according to the comprehensive duration corresponding to each candidate execution unit.

6. The method according to claim 3, wherein Determining, based on the calculation duration corresponding to each execution unit, the data transmission duration between the execution unit that executes the calculation task corresponding to the previous network layer and other execution units, the memory space, and at least one of the remaining memory, the execution unit that executes the calculation task corresponding to the network layer is determined as the target execution unit corresponding to the network layer, specifically including: If the network layer is the starting network layer of the target model, the calculation time corresponding to each execution unit is determined according to the number of calculations required when executing the corresponding calculation task of the network layer and the corresponding computing capacity of each execution unit; The execution unit corresponding to the network layer is determined according to the calculation time corresponding to each execution unit, the memory space and the remaining memory.

7. The method according to claim 6, wherein Determining the execution unit corresponding to the network layer according to the computation time corresponding to each execution unit, the memory space, and the remaining memory specifically includes: Determine an execution unit whose remaining memory is greater than the memory space as a candidate execution unit; The target execution unit corresponding to the network layer is determined based on the computation time corresponding to each candidate execution unit.

8. The method according to claim 4, wherein The method further comprises: For each network layer, if there are at least two transmission modes between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer of the network layer, the transmission status of the current transmission mode is monitored; Determining whether the current transmission mode needs to be adjusted according to the transmission status; If so, adjust the current transmission mode to another transmission mode.

9. The method according to claim 4, wherein The transmission information includes: a transmission mode between each execution unit and a bandwidth corresponding to each transmission mode, and the transmission mode includes: at least one of a first transmission mode and a second transmission mode.

10. The method according to claim 9, wherein The method further comprises: If the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer are both execution units in the GPU device, and the second transmission mode exists between the target execution unit corresponding to the network layer and the target execution unit corresponding to the adjacent network layer, then the data between the network layer and the adjacent network layer is transmitted through the second transmission mode; otherwise, the data between the network layer and the adjacent network layer is transmitted through the first transmission mode.

11. The method according to claim 9, wherein The first transmission mode includes: PCI-E serial computer expansion bus, and the second transmission mode includes: NVLINK.

12. A device for executing a task, characterized in that: include: Get module to get model data of target model; a parsing module that parses the model data to determine a computing task corresponding to each network layer in the target model, and determines device information corresponding to each computing device, wherein the computing device includes at least one central processing unit (CPU) and at least one graphics processing unit (GPU); A first determining module determines, for each network layer, a computing time required for executing the computing task corresponding to the network layer by an execution unit in each computing device based on the number of computations involved in executing the computing task corresponding to the network layer and device information corresponding to each computing device; The second determination module determines, based on at least one of the calculation duration corresponding to each execution unit, the data transmission duration between the execution unit that executes the computing task corresponding to the previous network layer and the other execution units, the memory space required for the data of the network layer, and the remaining memory of each computing device, the execution unit that executes the computing task corresponding to the network layer as the target execution unit corresponding to the network layer; wherein the data transmission duration is determined based on the transmission information between the computing device that executes the computing task corresponding to the previous network layer and the other computing devices, and the amount of data output by the previous network layer; The execution module deploys each network layer in the target device where the execution unit corresponding to each network layer is located, so as to execute the computing task corresponding to each network layer through the execution unit corresponding to each network layer after receiving the execution request of the computing task corresponding to each network layer.

13. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 11 is implemented.

Images