A computing task planning method, device, storage medium and electronic device

Abstract

This specification discloses a computing task planning method, device, storage medium and electronic device, which obtains the data size of the computing data of the neural network layer to be calculated, determines the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the neural network layer calculation, divides the data size corresponding to the computing task that the computing core needs to undertake, and obtains the data size of the subtask data, so as to minimize the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data. The data size of the subtask data is the amount of data read by the computing core in one communication cycle. Because the difference between the storage space required for performing the computing task corresponding to the subtask data is minimized, the method in this specification can be used to fully utilize the storage capacity of the computing core in the calculation process of the neural network, maximize resource utilization, and thus speed up the neural network calculation.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a computing task planning method, device, storage medium and electronic device. Background Art

[0002] With the development of artificial intelligence technology, neural networks are being used more and more widely. The calculation of neural networks requires a lot of computing resources, and traditional single-core processors can no longer meet the computing needs of neural networks. To this end, multi-core processors specifically designed for large-scale neural network model calculations have been developed. Multi-core processors contain hundreds or even more cores, which can provide rich computing resources for the calculation of neural network models in hardware and have powerful parallel processing capabilities.

[0003] In order to give full play to the hardware performance of multi-core processors, a parallel computing architecture is usually used in the calculation of neural networks to divide the computing tasks of the neural network and plan the computing tasks of the neural network to be processed in parallel by multiple computing cores to speed up the computing efficiency of the neural network.

[0004] However, the current parallel computing architecture only divides the computing tasks equally among each computing core when dividing tasks. Since the data required for computing in the neural network layer is usually large, and the storage capacity of the computing core is limited, the data required for the computing tasks that each computing core needs to undertake after the division cannot be transmitted to the computing core at one time. The communication cost generated during the data transmission process will also limit the computing efficiency of the neural network. In the prior art, the computing tasks are not planned for multiple data transmissions of a single computing core, resulting in poor acceleration of the neural network.

[0005] Therefore, this specification provides a computing task planning method. Summary of the invention

[0006] This specification provides a computing task planning method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0007] This manual adopts the following technical solutions:

[0008] This specification provides a computing task planning method, including:

[0009] Obtain the data size of the calculation data of the neural network layer to be calculated;

[0010] Determine the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the computing of the neural network layer;

[0011] Dividing the data size corresponding to the computing task to be undertaken by the computing core to obtain the data size of the subtask data, so as to minimize the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data;

[0012] The data size of the subtask data is sent to the control core, so that the control core transmits data to the computing core according to the data size of the subtask data.

[0013] Optionally, according to the total number of computing cores participating in the calculation of the neural network layer, the data size corresponding to the computing task to be undertaken by each computing core is determined, specifically including:

[0014] Determine whether the data size of the input feature map data included in the calculation data is greater than the data size of the convolution kernel parameters included in the calculation data;

[0015] If yes, according to the total number of computing cores participating in the calculation of the neural network layer, the data size of the input feature map data is evenly divided along the batch number axis of the tensor corresponding to the input feature map data to obtain the data sizes of several sub-feature map data contained in the input feature map data, and the data size corresponding to the computing task that the computing core needs to undertake is determined according to the data size of the sub-feature map data and the data size of the convolution kernel parameter;

[0016] If not, according to the total number of computing cores participating in the calculation of the neural network layer, the data size of the convolution kernel parameters is evenly divided along the batch number axis of the tensor corresponding to the convolution kernel parameters to obtain the data sizes of several sub-convolution kernel parameters contained in the convolution kernel parameters, and the data size corresponding to the computing task that the computing core needs to undertake is determined according to the data size of the input feature map data and the data size of the sub-convolution kernel parameters.

[0017] Optionally, dividing the data size of the task data to obtain the data size of the subtask data so as to minimize the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data, specifically includes:

[0018] Taking the data size corresponding to the computing task to be undertaken by the computing core as the data size of the data to be divided, dividing the data size of the data to be divided in half, and obtaining the data sizes of several division result data included in the data to be divided;

[0019] Determine the storage space required for the calculation task corresponding to the division result data according to the calculation accuracy required by the neural network layer and the data size of the division result data;

[0020] Determining whether the storage space is greater than the storage capacity of the computing core;

[0021] If yes, use the division result data as the data to be divided, and continue the process of dividing in half;

[0022] If not, the half-division process is stopped, and the data sizes of several sub-task data included in the task data are determined according to the data size of the division result data.

[0023] Optionally, the data size corresponding to the computing task to be undertaken by the computing core is used as the data size of the data to be divided, and the data size of the data to be divided is divided in half to obtain the data sizes of several division result data contained in the data to be divided, specifically including:

[0024] Determine, among the tensor axes of the input feature map data contained in the calculation data and the tensor axes of the convolution kernel parameters contained in the calculation data, a divisible tensor axis as an axis to be divided;

[0025] The axis length of the axis to be divided is halved to determine the data size of a plurality of division result data included in the data to be divided.

[0026] Optionally, determining the storage space required for the computing task corresponding to the division result data according to the computing accuracy required by the neural network layer and the data size of the division result data specifically includes:

[0027] Inferring the data size of the output feature map data by performing the calculation task corresponding to the division result data according to the data size of the division result data;

[0028] Determine the total data size required to store the computing task corresponding to the division result data according to the data size of the division result data and the data size of the output feature map data;

[0029] Determining the storage bit width according to the calculation accuracy required by the neural network layer;

[0030] The storage space required for performing the calculation task corresponding to the divided result data is determined according to the product of the total data size and the storage bit width.

[0031] Optionally, determining the data sizes of several subtask data included in the task data according to the data size of the division result data specifically includes:

[0032] According to the data size of the division result data, inferring the data size of the output feature map data obtained by performing the calculation task corresponding to the division result data;

[0033] Determine the computing time required for the computing task corresponding to the partition result data according to the data size of the partition result data, the data size of the output feature map data, and the total number of computing cores participating in the computing of the neural network layer;

[0034] Determining whether the calculation time is less than the communication cycle of the calculation core;

[0035] If yes, the data size of the division result data is gradually increased along the specified tensor axis according to the specified axis length until the storage space required for the calculation task corresponding to the division result data after the increase is greater than the storage capacity, and the data size of the division result data obtained after the increase in the previous step of the last step is used as the data size of the subtask data;

[0036] If not, the data size of the division result data is used as the data size of the subtask data.

[0037] Optionally, the method further comprises:

[0038] Among the tensor axes of the input feature map data contained in the calculation data and the tensor axes of the convolution kernel parameters contained in the calculation data, at least one tensor axis whose dimension can be increased is determined as the designated tensor axis.

[0039] This specification provides a computing task planning device, the device comprising:

[0040] An acquisition module, for acquiring the data size of the calculation data of the neural network layer to be calculated;

[0041] An inter-core partitioning module determines the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the computing of the neural network layer;

[0042] The core division module divides the data size corresponding to the computing task to be undertaken by the computing core to obtain the data size of the subtask data, so as to minimize the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data;

[0043] The application module sends the data size of the subtask data to the control core, so that the control core transmits data to the computing core according to the data size of the subtask data.

[0044] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned computing task planning method is implemented.

[0045] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned computing task planning method when executing the program.

[0046] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0047] In the computing task planning method provided in this specification, the data size of the computing data of the neural network layer to be calculated is obtained, and the data size corresponding to the computing task that each computing core needs to undertake is determined according to the total number of computing cores participating in the neural network layer calculation, and the data size corresponding to the computing task that the computing core needs to undertake is divided to obtain the data size of the subtask data, so that the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data is minimized. The data size of the subtask data is sent to the control core so that the control core transmits data to the computing core according to the data size of the subtask data. The data size of the subtask data is the amount of data read by the computing core in one communication cycle. Because the difference between the storage space required for performing the computing task corresponding to the subtask data is minimized, the method in this specification can be applied to fully utilize the storage capacity of the computing core in the calculation process of the neural network, maximize resource utilization, and thus speed up the neural network calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0049] Figure 1 A flowchart of a computing task planning method in this specification;

[0050] Figure 2 It is an iterative partitioning process diagram provided in an embodiment of this specification;

[0051] Figure 3 It is an iterative addition process diagram provided in an embodiment of this specification;

[0052] Figure 4 A schematic diagram of data storage location division provided in an embodiment of this specification;

[0053] Figure 5 A schematic diagram of a computing task planning device provided in this specification;

[0054] Figure 6 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] All steps in the computing task planning method provided in this specification can be implemented by any electronic device with computing functions, such as a terminal, a server, etc. For ease of description, the computing task planning method provided in this specification is described below with the server as the execution subject.

[0057] The computing task planning method of this specification can be performed before the neural network model training or reasoning task starts, and the computing task planning results of each neural network layer of the neural network model are planned in advance. During the neural network training or reasoning process, data is transmitted according to the computing task planning results of each neural network layer. It can also be performed during the neural network training or reasoning process, that is, in the order of forward propagation, the computing task planning results of each neural network layer are planned in turn, and after the computing task planning results of a neural network layer are determined, the data of the neural network layer is transmitted according to the allocation result.

[0058] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0059] Figure 1 The following is a flowchart of a computing task planning method in this specification, which specifically includes the following steps:

[0060] S100: Obtain the data size of the calculation data of the neural network layer to be calculated.

[0061] The computational scale of the computational tasks of each neural network layer of the neural network model depends on the data size of the computational data of each neural network layer. The operations involved in the computational process of the neural network model are mainly matrix multiplication operations. Matrix multiplication operations are the process of multiplying the corresponding elements of two matrices and then summing them. In this process, each multiplication operation is independent and does not depend on the results of other multiplication operations. Therefore, matrix multiplication operations have the characteristics of parallelism.

[0062] The computing task planning method of this specification realizes the reasonable scheduling of computing tasks at the software level without changing the hardware configuration of the many-core processor. Therefore, this method can be applied to any type of many-core processor, such as graphics processing unit (GPU), tensor processing unit (TPU), neural network processor (NPU), etc.

[0063] The data planning method for computing data of different neural network layers is the same. Therefore, this manual takes the method of allocating computing tasks of a neural network layer as an example for explanation.

[0064] After the architecture of the neural network model is determined, the data size of the computing data of each neural network layer of the neural network model is known. This specification plans the computing tasks based on the data size of the computing data of each neural network layer.

[0065] The server first determines the neural network layer to be calculated, and obtains the data size of the calculation data of the neural network layer to be calculated for subsequent data planning.

[0066] S102: Determine the data size corresponding to the computing task that each computing core needs to undertake based on the total number of computing cores participating in the calculation of the neural network layer.

[0067] The computing data of the neural network layer includes input feature map data and convolution kernel parameters. When allocating the computing data to each computing core, the server can divide the data size of the input feature map data or convolution kernel parameters according to the number of computing cores participating in the calculation of the neural network layer, and obtain the data size of the computing data required for the computing task undertaken by a computing core, that is, the data size of the computing core's task data.

[0068] Specifically, the server may distribute the computation data to each computation core according to a preset division rule, in which one of the input feature map data or the convolution kernel parameter may be specified as the initial segmentation data. The division rule may also be a method for selecting the initial segmentation data, so that the server selects the initial segmentation data from the input feature map data or the convolution kernel parameter according to the division rule.

[0069] Then, the server divides the data size of the initial segmented data into equal parts corresponding to the number of computing cores, and obtains the initial segmented data required for the computing task of each computing core.

[0070] When the initial segmented data is the input feature map data, the server divides the batch axis of the tensor corresponding to the input feature map data equally according to the total number of computing cores to obtain the data size of each sub-feature map data contained in the input feature map data. The data size corresponding to the computing task that a computing core needs to undertake is the sum of the data size of the sub-feature map data and the data size of the convolution kernel parameter.

[0071] The data size of the input feature map data is recorded as N*H*W*C, and the total number of computing cores is recorded as . Where N represents the length of the batch number axis, H represents the length of the height axis, W represents the length of the width axis, and C represents the length of the channel axis. The data size of the sub-feature map data is .

[0072] When the initial segmented data is a convolution kernel parameter, the server divides the batch axis of the tensor corresponding to the convolution kernel parameter equally according to the number of computing kernels to obtain the data size of each sub-convolution kernel parameter contained in the convolution kernel parameter. Then the data size of the computational data required for the computational task undertaken by a computing kernel is the sum of the data size of the input feature map data and the data size of the sub-convolution kernel parameter.

[0073] The data size of the convolution kernel parameters is recorded as M*R*S*C, and the total number of calculation kernels is recorded as Among them, M represents the axis length of the batch number axis, R represents the axis length of the height axis, S represents the axis length of the width axis, and C represents the axis length of the channel axis. Then the data size of the sub-convolution kernel parameters is .

[0074] After the data size of the task data is determined, when performing the computing tasks of the neural network layer, the server can divide the computing data of the neural network layer according to the data size of the task data, so as to obtain the task data of each computing core, so that each computing core can perform parallel calculations on its corresponding task data.

[0075] The computing cores to which the computing data is allocated are the computing cores on the many-core processor that participate in the computing tasks of the neural network layer. The total number of computing cores can be the number of all computing cores included in the many-core processor, or a specified number within the number of computing cores included in the many-core processor. When the computing cores participating in the computing tasks of the neural network layer are the number of all computing cores included in the many-core processor, the utilization rate of the computing resources of the many-core processor is the highest, and the computing efficiency is also faster.

[0076] In this step, the data size of the calculation data is distributed to different calculation cores. Therefore, this step is the inter-core division process of the calculation data.

[0077] There is data dependency between the computational tasks of the data range corresponding to the height axis and width axis of the tensor corresponding to the input feature map data or convolution kernel parameters. Therefore, in this step, the computational data is divided along the batch number of the input feature map data or convolution kernel parameters, and the tensor axis dimension with data dependency is not split. In the computation process of the neural network, the computational tasks for each task data are relatively independent, and the task data can be computed in parallel through different computational kernels.

[0078] S104: Divide the data size corresponding to the computing task that the computing core needs to undertake to obtain the data size of the subtask data, so as to minimize the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data.

[0079] In the above process, the computing data is divided among the cores, and the data size of the computing data required for the computing tasks that each computing core needs to undertake is determined, that is, the data size of the task data of a computing core. Because the data size of the input feature map data of the neural network layer is usually large, and the internal storage capacity of the computing core is limited, the task data cannot be transferred to the computing core at one time, so the task data needs to be transferred to the computing core in batches, that is, the data size of the task data needs to be divided within the core.

[0080] The server can check the hardware configuration of the many-core processor to determine the storage capacity of the computing core. The storage capacity of the computing core varies in many-core processors of different models or types.

[0081] This specification adopts an iterative partitioning method, where the server divides the computing core into halves multiple times, dividing the data size of the task data into the data sizes of multiple sub-task data.

[0082] Figure 2 This is an iterative partitioning process diagram provided in an embodiment of this specification, such as Figure 2 As shown, the server takes the data size corresponding to the computing task that the computing core needs to undertake as the data size of the data to be divided, divides the data size of the data to be divided in half, and obtains the data sizes of several division result data contained in the data to be divided.

[0083] For convolution calculation, the calculation data includes input feature map data and convolution kernel parameters. The server determines the divisible tensor axis among each tensor axis of the input feature map data and each tensor axis of the convolution kernel parameters as the axis to be divided.

[0084] This manual does not restrict which tensor axis to be divided. According to actual needs, it can be selected from the batch axis, height axis, width axis, channel axis of the input feature map data, and the batch axis, height axis, width axis, channel axis of the convolution kernel parameters.

[0085] The server reduces the axis length of the axis to be divided by half, and determines the data size of several division result data contained in the data to be divided.

[0086] Secondly, the server determines the storage space required for the computing task corresponding to the divided result data based on the computing accuracy required by the neural network layer and the data size of the divided result data.

[0087] During the calculation process of the computing core, the calculation data corresponding to the calculation task performed by the computing core in one calculation cycle needs to be stored in the computing core, and the output data calculated based on the calculation data corresponding to the calculation task performed in this calculation cycle also needs to be cached in the computing core.

[0088] Therefore, the server can infer and calculate the data size of the output data obtained after calculating the calculation data of this data size based on the data size of the calculation data corresponding to the calculation task performed in a calculation cycle, and determine the total data size that needs to be stored for the calculation task corresponding to the divided result data based on the data size of the calculation data corresponding to the calculation task performed in a calculation cycle and the data size of the output data.

[0089] The server can determine the storage bit width according to the calculation accuracy required by the neural network layer, and determine the storage space required for the calculation task corresponding to the divided result data according to the product of the storage bit width and the total data size.

[0090] Then, the server determines whether the storage space required for the computing task corresponding to the divided result data is greater than the storage capacity of the computing core.

[0091] If so, the server uses the division result data as the data to be divided in the next division process and continues the half-division process.

[0092] If not, the server stops the half-division process and determines the data size of the subtask data according to the data size of the division result data. The server may use the data size of the division result data as the data size of the subtask data.

[0093] Because in this method, the server divides the data size of the task data in half in order from large to small, and after each division is completed, when it is determined that the data size of the divided result data obtained by the division corresponds to the storage space required for the calculation task of the calculation data is not greater than the storage capacity of the calculation core, the half division process is stopped. Therefore, after the division is completed, the storage space required for the calculation task of the data size of the subtask data corresponding to the calculation data is relatively close to the storage capacity of the calculation core. Data transmission based on the data size of the subtask data determined by this method can make full use of the storage resources of the calculation core.

[0094] S104: Send the data size of the subtask data to the control core, so that the control core transmits data to the computing core according to the data size of the subtask data.

[0095] Before the calculation task of the neural network layer to be calculated begins, the server sends the data size of the calculation sub-data determined above to the control core. The control core divides the calculation data into task data corresponding to the calculation core according to the data size of the sub-task data. In each communication cycle, the calculation data of the data scale corresponding to the data size of the sub-task data is transmitted to the calculation core so that the calculation core performs calculation.

[0096] Based on the above Figure 1 The provided computing task planning method obtains the data size of the computing data of the neural network layer to be calculated, determines the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the neural network layer calculation, divides the data size corresponding to the computing task that the computing core needs to undertake, and obtains the data size of the subtask data, so that the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data is minimized. The data size of the subtask data is sent to the control core so that the control core transmits data to the computing core according to the data size of the subtask data. The data size of the subtask data is the amount of data read by the computing core in one communication cycle. Because the difference between the storage space required for performing the computing task corresponding to the subtask data is minimized, the method in this specification can be applied to fully utilize the storage capacity of the computing core in the calculation process of the neural network, maximize resource utilization, and thus speed up the neural network calculation.

[0097] In the above step S100, when the division rule is a selection method for initially segmenting data, the division rule can be set as follows.

[0098] The server determines the total number of computing cores involved in the neural network layer calculation. It determines whether the data size of the input feature map data is greater than the data size of the convolution kernel parameter.

[0099] Because the axis length of the channel axis of the input feature map data is consistent with the axis length of the channel axis of the convolution kernel parameters, in one embodiment, in order to simplify the computational complexity of the judgment process, the server may compare the product of the batch quantity axis, height axis, and width axis lengths of the input feature map data with the product of the batch quantity axis, height axis, and width axis lengths of the convolution kernel parameters.

[0100] If the data size of the input feature map data is larger than the data size of the convolution kernel parameters, the input feature map data is used as the initial partition data to partition the input feature map data. Specifically, according to the total number of computing kernels participating in the neural network layer calculation, the data size of the input feature map data is evenly divided along the batch number axis of the tensor corresponding to the input feature map data to obtain the data sizes of several sub-feature map data contained in the input feature map data, and the data size of the task data corresponding to the computing kernel is determined according to the data size of the sub-feature map data and the data size of the convolution kernel parameters.

[0101] If the data size of the input feature map data is not larger than the data size of the convolution kernel parameters, the convolution kernel parameters are used as the initial partition data to divide the convolution kernel parameters. Specifically, according to the total number of computing kernels participating in the neural network layer calculation, the data size of the convolution kernel parameters is evenly divided along the batch number axis of the tensor corresponding to the convolution kernel parameters to obtain the data sizes of several sub-convolution kernel parameters contained in the convolution kernel parameters, and the data size of the task data corresponding to the computing kernel is determined according to the data size of the input feature map data and the data size of the sub-convolution kernel parameters.

[0102] In this embodiment, the data with larger data size in the input feature map data and the convolution kernel parameters is used as the initial partition data. For the initial partition data, the data size of the initialization data contained in the task data of the computing kernel is larger, which can enable the task data of a computing kernel to maintain a high data integrity, reduce the data dependency between the computing tasks of different computing kernels, and reduce the computing speed delay caused by the communication between different computing cores when performing the computing tasks of the neural network layer.

[0103] In the above step S102, the subtask data is determined in a half-division manner, that is, in each division process, the size of the task data is reduced to half of the original size. In fact, the server can also divide according to any division multiple, and is not limited to the division method described in step S102. For example, each time the data size of the task data is reduced to a quarter, an eighth, etc. of the original size.

[0104] When other division multiples are used to divide the data and the difference between the storage capacity and the storage space required for the computing task corresponding to the subtask data is minimized, the specific division process is similar to the half-and-half division process, and reference may be made to the corresponding description of S102.

[0105] In the above step S102, the server may determine at least one tensor axis from among the tensor axes of the input feature map data and the tensor axes of the convolution kernel parameters as the axis to be divided. Step S102 describes the case where the number of axes to be divided is one, and the following will describe an embodiment where the number of axes to be divided is multiple.

[0106] The server determines a preset division order for each axis to be divided, and this specification does not specifically limit the division order.

[0107] Because in the convolution calculation process, the calculation kernel corresponds to the calculation task in one calculation cycle, and the number of channels of the required input feature map data needs to be consistent with the number of channels of the convolution kernel parameters. Therefore, when determining the axis to be divided, you only need to select a channel axis from the channel axis of the input feature map data or the channel axis of the convolution kernel parameters to perform the division operation, that is, you only need to determine one axis to be divided from the channel axis of the input feature map data and the channel axis of the convolution kernel parameters.

[0108] In one embodiment, the divisible tensor axes in the input feature map data may be determined, the divisible tensor axes in the input feature map data may be sorted, and the divisible tensor axes in the convolution kernel parameters may be determined, and the divisible tensor axes in the convolution kernel parameters may be sorted. Then, the division priority of the input feature map data and the convolution kernel parameters may be determined, and the order of the divisible tensor axes in the input feature map data and the divisible tensor axes in the convolution kernel parameters may be determined according to the division priority.

[0109] According to the division order determined in this embodiment, it can be batch quantity axis of input feature map data → channel axis of input feature map data → height axis of input feature map data → width axis of input feature map data → batch quantity axis of convolution kernel parameters → height axis of convolution kernel parameters → width axis of convolution kernel parameters.

[0110] In another embodiment, considering the height axis and width axis of the tensor corresponding to the input feature map data, and the height axis and width axis of the convolution kernel parameters, the calculation tasks of the corresponding data range need to perform multiplication and accumulation operations, and have data dependence. If the data is divided according to the height axis and width axis of the tensor corresponding to the input feature map data, and the height axis and width axis of the convolution kernel parameters, the calculation tasks of a one-dimensional single feature map data, or the calculation tasks of a one-dimensional convolution kernel parameter will be assigned to different calculation cores, which will lead to data communication between different calculation cores and increase the calculation time.

[0111] Therefore, the server can increase the priority of the axis to be divided that has no data dependency when dividing. When the subtask data obtained by the division operation on the axis to be divided that has no data dependency corresponds to the storage space required by the computing task that is close to the storage capacity of the computing core, the division of the axis to be divided that has data dependency can be avoided.

[0112] Specifically, the server determines the tensor axes that do not have data dependency among the tensor axes of the input feature map data and the tensor axes of the convolution kernel parameters. That is, the batch number axis of the input feature map data, the channel axis of the input feature map data, and the batch number axis of the convolution kernel parameters. Then, the server increases the partitioning priority of the tensor axes that do not have data dependency to obtain the partitioning order.

[0113] In this embodiment, an example of the division order is: batch quantity axis of feature map data → channel axis of input feature map data → batch quantity axis of convolution kernel parameters → height axis of input feature map data → width axis of input feature map data → height axis of convolution kernel parameters → width axis of convolution kernel parameters.

[0114] Furthermore, because dividing the width axis and height axis of the convolution kernel parameters will affect the data continuity, and because the height axis and width axis of the convolution kernel parameters are usually small, it is possible not to divide the height axis and width axis of the convolution kernel parameters. That is, when determining the axis to be divided, exclude the height axis and width axis of the convolution kernel parameters.

[0115] In this embodiment, an exemplary division order is batch quantity axis of feature map data→channel axis of input feature map data→batch quantity axis of convolution kernel parameters→height axis of input feature map data→width axis of input feature map data.

[0116] In the above step S102, a half-division method is adopted, and in each division process, the data size of the division result data is reduced by half to determine the subtask data of the computing core.

[0117] In the last division process, although the storage space required for the computing task corresponding to the subtask data is smaller than the storage capacity of the computing core, there may still be a small gap between the storage space and the storage capacity of the computing core.

[0118] For example, the storage capacity of the computing core is 50M, and the storage space required for the computing task corresponding to the task data of the computing core is 160M. After one division process, the storage space required for the computing task corresponding to the subtask data becomes 80M, which is still greater than 50M. The second division process is continued, and the storage space required for the computing task corresponding to the subtask data becomes 40M. At this time, the storage space required for the computing task corresponding to the subtask data is less than 50M, but the computing core still has 10M of remaining space, which is not fully utilized.

[0119] In order to further reduce the difference between the storage capacity of the computing core and the storage space required for the computing tasks corresponding to the subtask data, when the server determines that the half-division process is completed, the data size of the division result data determined by the last division process is increased, and the increased data size of the division result data is used as the data size of the subtask data.

[0120] Because the data size of the computational sub-data is the amount of computational data transmitted to the computational core in one communication cycle. In the computational process of the neural network, the communication time of the computational core often limits the computational efficiency of the computational core.

[0121] That is, if the communication cycle of the computing core is greater than the computing time of the computing core for subtask data, then after the computing core completes the computing task of the subtask data transmitted into the computing core in the previous communication cycle, it is necessary to wait for the subtask data to be transmitted into the computing core in the next communication cycle. This waiting time causes the computing core to be idle, slowing down the computing speed of the neural network layer.

[0122] However, if the communication cycle of the computing core is equal to the calculation time of the subtask data, then after the computing core completes the calculation task of the subtask data transmitted into the computing core in the previous communication cycle, it just receives the subtask data transmitted into the computing core in the next communication cycle. The computing core can continue to calculate the subtask data transmitted into the computing core in the next communication cycle, and the computing core has no idle time.

[0123] When the computing core has idle time, increasing the data size of the subtask data can increase the computing time of the subtask data in one communication cycle accordingly, until the computing time is equal to the communication cycle, the computing core can run at full capacity, and the computing efficiency of the neural network layer is maximized. When the computing core has no idle time, continuing to increase the data size of the subtask data will not further increase the computing efficiency.

[0124] Therefore, in this embodiment, the server determines the data size of the division result data obtained in the last round of division process, and based on the data size of the division result data obtained in the last round of division process, iteratively increases the data size of the division result data to obtain the data size of the subtask data.

[0125] The server infers the data size of the output feature map data obtained by performing the calculation task corresponding to the division result data based on the data size of the division result data.

[0126] After the data size of the input feature map data and the data size of the convolution kernel parameters are determined, the data size of the corresponding output feature map data can be calculated.

[0127] For example, let the data size of the input feature map data be N*H*W*C, the data size of the convolution kernel parameters be M*R*S*C, the batch size of the input feature map data determines the batch size of the output feature map data, and the batch size of the convolution kernel parameters determines the number of channels of the output feature map data. Then the size of the output feature map data can be recorded as N*E*F*M, where E and F can be determined according to the following formula:

[0128]

[0129]

[0130] in, Represents the left expansion width of the input feature map data, indicates the right expansion width of the input feature map data, Represents the upper expansion width of the input feature map data, Represents the lower expansion width of the input feature map data, Represents the vertical step size of the convolution kernel parameters, Represents the horizontal step size of the convolution kernel parameters.

[0131] Secondly, the server determines the computing time required for the computing task corresponding to the division result data based on the data size of the division result data, the data size of the output feature map data obtained by performing the computing task corresponding to the division result data, and the total number of computing cores involved in the neural network layer calculation.

[0132] The specific formula for determining the calculation time is as follows:

[0133]

[0134] in, To calculate the total number of cores, The minimum data size to be read when calculating the kernel. To calculate the first shape parameter of the kernel, The second shape parameter of the computing core is used to represent the array arrangement scale of the computing units included in the computing core.

[0135] Then, the server may determine the communication cycle of the computing core according to the hardware configuration of the many-core processor, and judge whether the computing time required for the computing task corresponding to the divided result data is less than the communication cycle.

[0136] When the computing time required for the computing task corresponding to the partitioned result data is less than the communication cycle, the data size of the partitioned result data is gradually increased in the specified tensor axis according to the specified step size until the storage space required for the computing task corresponding to the partitioned result data after the increase is greater than the storage capacity of the computing core. The server uses the data size of the partitioned result data obtained after the last step of the previous step as the data size of the subtask data.

[0137] Figure 3 This is an iterative increase process diagram provided in the embodiments of this specification, such as Figure 3 As shown, the server may use the data size of the partition result data obtained in the last round of partitioning process as the data size of the data to be increased, and start a gradual size increase process.

[0138] First, the server determines the data size of the output feature graph of the computing task corresponding to the data to be added according to the data size of the data to be added. According to the data size of the data to be added, the data size of the output feature graph of the computing task corresponding to the data to be added, and the total number of computing cores participating in the neural network layer calculation, the computing time of the computing task corresponding to the data to be added is determined.

[0139] When the computing time is less than the communication cycle of the computing core, it means that the computing efficiency of the computing task needs to be improved by increasing the size. Then the server increases the data size of the data to be increased in the specified tensor axis to obtain the data size of the increased result data.

[0140] Then, it is determined whether the storage space required by the computing task corresponding to the data size of the increased result data is greater than the storage capacity of the computing core. If so, the data size of the increased result data is used as the data size of the data to be increased, and the size increase process is continued. If not, the size increase process is stopped, and the data size of the increased result data is used as the data size of the subtask data.

[0141] When the computing time is not less than the communication cycle of the computing core, it means that it is no longer necessary to increase the size to improve the computing efficiency of the computing task. The server uses the data size of the data to be increased as the data size of the subtask data.

[0142] After the last step of increasing is completed, the data size of the determined partitioned result data corresponding to the computing time required for the computing task begins to be greater than the storage capacity of the computing core, then the data size of the partitioned result data obtained after the last step of increasing is increased, corresponding to the computing time required for the computing task, is the closest to the storage capacity of the computing core. That is, the data size of the subtask data that can maximize the use of the storage resources of the computing core.

[0143] The above-mentioned specified tensor axis can be specifically determined by the server among the tensor axes of the input feature map data and the tensor axes of the convolution kernel parameters. The specified tensor axis can be one or more. This specification does not limit the specific content of the specified tensor axis.

[0144] When the computing time required for the computing task corresponding to the division result data is not less than the communication cycle, the data size of the division result data is used as the data size of the subtask data.

[0145] In this case, it means that the computing task performed according to the data size of the partitioned result data can already make the computing core run at full capacity. Although the storage resources of the computing core are fully utilized, increasing the data size cannot increase the computing rate of the neural network calculation. Therefore, the server uses the data size of the partitioned result data as the data size of the subtask data.

[0146] In the above embodiment, when there are multiple designated tensor axes, the server determines the preset order of increase of each designated tensor axis. Exemplarily, when each designated tensor axis includes a channel axis of input feature map data, a batch axis of input feature map data, and a batch axis of convolution kernel parameters, the order of increase of each designated tensor axis can be determined as needed.

[0147] For different specified tensor axes, the same specified step size can be set, or different specified step sizes can be set. This description does not limit the specific value of the specified step size, which can be set according to the array shape of the computing units contained in the computing core, or can be set to any value as needed. The minimum specified step size is 1, which means that one dimension is added to the subtask data at the specified tensor axis each time.

[0148] In this embodiment, the subtask data includes the data size of the input feature map data and the convolution kernel parameters that need to be transmitted in one communication cycle, and the data size of the subtask data can be recorded as: the data size of the input feature map data , the data size of the convolution kernel parameters , specify the step size as 32, and specify the tensor axis as the channel axis of the input feature map data. Because the channel axis length of the input feature map data is consistent with the channel axis length of the convolution kernel parameter, the data size of the subtask data after one step increase becomes: the data size of the input feature map data , the data size of the convolution kernel parameters .

[0149] The computing task planning method in this manual can also be applied to the convolution fusion operator, which is a fusion of multiple computing operations in the neural network layer. The intermediate results of the multiple computing operations in the convolution fusion operator are not output, which can reduce unnecessary computing steps and speed up the computing speed of the Shenjiang network layer.

[0150] A convolution fusion operator may include a batch normalization operation on the input feature map data and a convolution operation on the batch normalized input feature map data.

[0151] In this embodiment, the calculation data includes input feature map data, convolution kernel parameters and batch normalization parameters, and the batch normalization parameters include the mean and variance of the input feature map data and batch normalization scaling parameters.

[0152] When the computing core performs the computing task of the convolution fusion operator, the data that needs to be stored includes the input feature map data, convolution kernel parameters, batch normalization parameters, batch normalization feature map data and output feature map data corresponding to the computing task of each communication cycle. In the above step S102, the server can determine the storage space required for the computing task corresponding to the divided result data in the following manner.

[0153] The server determines the total data size required to store the computing task corresponding to the partition result data according to the data size of the partition result data, the data size of the output feature map data, and the data size of the batch normalization parameter.

[0154] The data size of the batch normalized feature map data is the same as the data size of the input feature map data. Therefore, the total data size is equal to the sum of the data size of the input feature map data corresponding to the division result data, the data size of the convolution kernel parameters corresponding to the division result data, the data size of the batch normalization parameters, the data size of the batch normalized feature map data of the computing task corresponding to the division result data, and the data size of the output feature map data of the computing task corresponding to the division result data.

[0155] Then, the server determines the storage space required for performing the computing task corresponding to the divided result data according to the product of the total data size and the storage bit width.

[0156] In one or more embodiments of the present specification, in the application scenario of the convolution fusion operator, the server may also determine the storage space required for the input feature map data, the storage space required for the convolution kernel parameters, the storage space required for the batch normalization parameters, the storage space required for the calculated batch normalized feature map data, and the storage space required for the calculated output feature map data when performing the calculation task corresponding to the subtask data based on the data size of the input feature map data contained in the subtask data, the data size of the convolution kernel parameters contained in the subtask data, the data size of the batch normalization parameters, the data size of the batch normalized feature map data of the calculation task corresponding to the subtask data, and the data size of the output feature map data of the calculation task corresponding to the subtask data.

[0157] In step S104, the server sends the data size of the subtask data and the sizes of various storage spaces to the control core, and the control block transmits data to the computing core according to the data size of the subtask data, and pre-divides the storage locations of the input feature map data, convolution kernel parameters, batch normalization parameters, batch normalization feature map data, and output feature map data in the storage space of the computing core according to the sizes of various storage spaces. Thus, when performing training or inference tasks of the neural network model, various data in the computing core are stored according to the pre-planned storage locations. Figure 4 A schematic diagram of data storage location division provided in an embodiment of this specification.

[0158] By applying this embodiment, the server can plan the computing tasks of the neural network layer and the storage locations of various data that need to be stored during the computing process of the neural network layer.

[0159] The above is the computing task planning method provided in this specification. Based on the same idea, this specification also provides a corresponding computing task planning device, such as Figure 5 shown.

[0160] Figure 5 This specification provides a schematic diagram of a computing task planning device, which specifically includes:

[0161] An acquisition module 200 is used to obtain the data size of the calculation data of the neural network layer to be calculated;

[0162] The core division module 202 is used to determine the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the neural network layer calculation;

[0163] The core division module 204 is used to divide the data size corresponding to the computing task to be undertaken by the computing core to obtain the data size of the subtask data, so as to minimize the difference between the storage capacity of the computing core and the storage space required for performing the computing task corresponding to the subtask data;

[0164] The application module 206 is used to send the data size of the subtask data to the control core, so that the control core transmits data to the computing core according to the data size of the subtask data.

[0165] Optionally, the inter-core division module 202 is specifically used to determine whether the data size of the input feature map data contained in the calculation data is greater than the data size of the convolution kernel parameters contained in the calculation data. If so, according to the total number of calculation kernels participating in the calculation of the neural network layer, the data size of the input feature map data is evenly divided along the batch number axis of the tensor corresponding to the input feature map data to obtain the data size of several sub-feature map data contained in the input feature map data, and the data size corresponding to the calculation task that the calculation kernel needs to undertake is determined according to the data size of the sub-feature map data and the data size of the convolution kernel parameters. If not, according to the total number of calculation kernels participating in the calculation of the neural network layer, the data size of the convolution kernel parameters is evenly divided along the batch number axis of the tensor corresponding to the convolution kernel parameters to obtain the data size of several sub-convolution kernel parameters contained in the convolution kernel parameters, and the data size corresponding to the calculation task that the calculation kernel needs to undertake is determined according to the data size of the input feature map data and the data size of the sub-convolution kernel parameters.

[0166] Optionally, the intra-core partitioning module 204 is specifically used to use the data size corresponding to the computing task that the computing core needs to undertake as the data size of the data to be divided, divide the data size of the data to be divided in half, and obtain the data size of several partition result data contained in the data to be divided, determine the storage space required for performing the computing task corresponding to the partition result data according to the computing accuracy required by the neural network layer and the data size of the partition result data, and judge whether the storage space is larger than the storage capacity of the computing core. If so, use the partition result data as the data to be divided and continue the half-partitioning process. If not, stop the half-partitioning process, and determine the data size of several sub-task data contained in the task data according to the data size of the partition result data.

[0167] Optionally, the intra-core division module 204 is specifically used to determine a divisible tensor axis among each tensor axis of the input feature map data contained in the calculation data and each tensor axis of the convolution kernel parameters contained in the calculation data, as the axis to be divided, reduce the axis length of the axis to be divided by half, and determine the data size of several division result data contained in the data to be divided.

[0168] Optionally, the in-core partitioning module 204 is specifically used to infer the computing task corresponding to the partitioning result data according to the data size of the partitioning result data, obtain the data size of the output feature map data, determine the total data size required to store for the computing task corresponding to the partitioning result data according to the data size of the partitioning result data and the data size of the output feature map data, determine the storage bit width according to the computing accuracy required by the neural network layer, and determine the storage space required for the computing task corresponding to the partitioning result data according to the product of the total data size and the storage bit width.

[0169] Optionally, the intra-core partitioning module 204 is specifically used to infer the data size of the output feature map data obtained by performing the computing task corresponding to the partition result data according to the data size of the partition result data, determine the computing time required for the computing task corresponding to the partition result data according to the data size of the partition result data, the data size of the output feature map data and the total number of computing cores participating in the calculation of the neural network layer, and determine whether the computing time is less than the communication cycle of the computing core. If so, gradually increase the data size of the partition result data along the specified tensor axis according to the specified axis length until the storage space required for the computing task corresponding to the partition result data after the increase is greater than the storage capacity, and use the data size of the partition result data obtained after the increase of the previous step of the last step as the data size of the subtask data. If not, use the data size of the partition result data as the data size of the subtask data.

[0170] Optionally, the intra-core division module 204 is specifically used to determine at least one tensor axis that can increase the dimension among the tensor axes of the input feature map data contained in the calculation data and the tensor axes of the convolution kernel parameters contained in the calculation data, as the designated tensor axis.

[0171] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 The computing task planning method provided.

[0172] This manual also provides Figure 6 The electronic equipment structure diagram shown in FIG. Figure 6 As shown in the figure, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0173] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0174] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0175] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0176] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0177] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0178] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0181] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0182] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0183] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0184] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0185] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0186] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0187] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0188] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A computing task planning method, characterized in that: include: Obtain the data size of the calculation data of the neural network layer to be calculated; Determine the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the computing of the neural network layer; The data size corresponding to the computing task to be undertaken by the computing core is used as the data size of the data to be divided, and the data size of the data to be divided is divided in half to obtain the data size of several division result data contained in the data to be divided; according to the computing accuracy required by the neural network layer and the data size of the division result data, the storage space required for the computing task corresponding to the division result data is determined; whether the storage space is larger than the storage capacity of the computing core is judged; if so, the division result data is used as the data to be divided and the half-division process is continued; if not, the half-division process is stopped, and according to the data size of the division result data, the data size of several sub-task data contained in the task data is determined, and the task data is the computing data required for the computing task undertaken by the computing core; The data size of the subtask data is sent to the control core, so that the control core transmits data to the computing core according to the data size of the subtask data.

2. The method according to claim 1, characterized in that According to the total number of computing cores participating in the calculation of the neural network layer, the data size corresponding to the computing task to be undertaken by each computing core is determined, specifically including: Determine whether the data size of the input feature map data included in the calculation data is greater than the data size of the convolution kernel parameters included in the calculation data; If yes, according to the total number of computing cores participating in the calculation of the neural network layer, the data size of the input feature map data is evenly divided along the batch number axis of the tensor corresponding to the input feature map data to obtain the data sizes of several sub-feature map data contained in the input feature map data, and the data size corresponding to the computing task that the computing core needs to undertake is determined according to the data size of the sub-feature map data and the data size of the convolution kernel parameter; If not, according to the total number of computing cores participating in the calculation of the neural network layer, the data size of the convolution kernel parameters is evenly divided along the batch number axis of the tensor corresponding to the convolution kernel parameters to obtain the data sizes of several sub-convolution kernel parameters contained in the convolution kernel parameters, and the data size corresponding to the computing task that the computing core needs to undertake is determined according to the data size of the input feature map data and the data size of the sub-convolution kernel parameters.

3. The method according to claim 1, characterized in that The data size corresponding to the computing task to be undertaken by the computing core is used as the data size of the data to be divided, and the data size of the data to be divided is divided in half to obtain the data sizes of several division result data contained in the data to be divided, specifically including: Determine, among the tensor axes of the input feature map data contained in the calculation data and the tensor axes of the convolution kernel parameters contained in the calculation data, a divisible tensor axis as an axis to be divided; The axis length of the axis to be divided is halved to determine the data size of a plurality of division result data included in the data to be divided.

4. The method according to claim 1, characterized in that According to the calculation accuracy required by the neural network layer and the data size of the division result data, the storage space required for the calculation task corresponding to the division result data is determined, specifically including: Inferring the data size of the output feature map data by performing the calculation task corresponding to the division result data according to the data size of the division result data; Determine the total data size required to store the computing task corresponding to the division result data according to the data size of the division result data and the data size of the output feature map data; Determining the storage bit width according to the calculation accuracy required by the neural network layer; The storage space required for performing the calculation task corresponding to the divided result data is determined according to the product of the total data size and the storage bit width.

5. The method according to claim 1, characterized in that According to the data size of the division result data, determining the data size of a plurality of subtask data included in the task data specifically includes: According to the data size of the division result data, inferring the data size of the output feature map data obtained by performing the calculation task corresponding to the division result data; Determine the computing time required for the computing task corresponding to the partitioning result data according to the data size of the partitioning result data, the data size of the output feature map data, and the total number of computing cores participating in the computing of the neural network layer; Determining whether the calculation time is less than the communication cycle of the calculation core; If yes, the data size of the division result data is gradually increased along the specified tensor axis according to the specified axis length until the storage space required for the calculation task corresponding to the division result data after the increase is greater than the storage capacity, and the data size of the division result data obtained after the increase in the previous step of the last step is used as the data size of the subtask data; If not, the data size of the division result data is used as the data size of the subtask data.

6. The method according to claim 5, characterized in that The method further comprises: Among the tensor axes of the input feature map data contained in the calculation data and the tensor axes of the convolution kernel parameters contained in the calculation data, at least one tensor axis whose dimension can be increased is determined as the designated tensor axis.

7. A computing task planning device, characterized in that: include: An acquisition module, for acquiring the data size of the calculation data of the neural network layer to be calculated; An inter-core partitioning module determines the data size corresponding to the computing task that each computing core needs to undertake according to the total number of computing cores participating in the computing of the neural network layer; The intra-core partitioning module uses the data size corresponding to the computing task to be undertaken by the computing core as the data size of the data to be partitioned, divides the data size of the data to be partitioned in half, and obtains the data size of several partition result data contained in the data to be partitioned; determines the storage space required for performing the computing task corresponding to the partition result data according to the computing accuracy required by the neural network layer and the data size of the partition result data; determines whether the storage space is larger than the storage capacity of the computing core; if so, uses the partition result data as the data to be partitioned and continues the half-partitioning process; if not, stops the half-partitioning process, and determines the data size of several sub-task data contained in the task data according to the data size of the partition result data, and the task data is the computing data required for the computing task undertaken by the computing core; The application module sends the data size of the subtask data to the control core, so that the control core transmits data to the computing core according to the data size of the subtask data.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 6 is implemented.

Images