Method and apparatus for data multiplexing of reverse weight gradient computation based on many-core processor

Abstract

The application discloses a kind of reverse weight gradient calculation data multiplexing method and device based on many-core processor.The method comprises the following steps of: firstly, according to the size of the depth learning model and input data set, the size of convolution layer is obtained, and the hardware information of many-core processor is obtained;Second, three candidate schemes are designed, the height and width of convolution kernel in each candidate scheme, the block size of input channel number and output channel number on each computing core and the design of calculation cycle sequence are determined;Then, the data access amount between on-chip local storage and global memory in each candidate scheme is calculated;Finally, the candidate scheme with the minimum data access amount is selected.Through the application, the problem of how to design acceleration strategy according to different convolution layer size in the development of high-performance acceleration library for reverse convolution kernel gradient calculation is solved, and the calculation performance is improved by reducing the data access amount between on-chip local storage and global memory.

Description

Technical Field

[0001] This invention relates to the field of dedicated processors for artificial intelligence, and in particular to a method and apparatus for reusing reverse weight gradient calculation data based on many-core processors. Background Technology

[0002] With the rapid development of artificial intelligence technology, the network scale of deep learning algorithms continues to increase, which puts forward increasingly higher requirements for the computing power of processors. Developing high-performance deep learning computing acceleration libraries on dedicated artificial intelligence processors has become a research hotspot.

[0003] Existing AI processors often employ a many-core architecture, where multiple computing cores process data in parallel to improve computational parallelism and performance. They also utilize a multi-level storage architecture, where multiple computing cores share global memory, and each core has its own on-chip local memory (Scratchpad Memory, SPM), allowing the computing unit to directly access data in registers for computation. Convolution operators are a computational hotspot in deep learning algorithms. To achieve better performance in deep learning applications, each computing core has a dedicated hardware acceleration unit—a systolic array—for accelerating convolution / matrix multiplication calculations. The systolic array typically has fixed weight data. Input feature map data, intermediate parts, and results flow sequentially in both horizontal and vertical directions. An accumulator buffer receives the intermediate parts and results, accumulates them with the original data in the accumulator buffer, and temporarily stores the result until all accumulation is complete, at which point the final accumulated result is output.

[0004] In the development of high-performance acceleration libraries for many-core processors for deep learning training, accelerating and optimizing the calculation of inverse convolution kernel gradients faces numerous challenges. First, designing convolution acceleration libraries on many-core processors coupled with systolic array (SPM) hardware acceleration units often results in memory access becoming a performance bottleneck for convolution operators. Second, the convolution process for calculating inverse convolution kernel gradients differs significantly from forward convolution. In inverse convolution, both inputs are feature maps or feature map gradients with relatively large height and width dimensions, while the result is a convolution kernel gradient with smaller height and width dimensions. However, in the multi-level memory architecture of many-core processors, the storage space of on-chip SPMs and registers is very limited, thus requiring convolution operations to be broken down into multiple loops. Finally, the sizes of convolutional layers in the network vary, and different strategies are employed in the development of acceleration libraries for different layer sizes. Finding a better-performing strategy based on the convolutional layer size is a challenge.

[0005] Based on the computational principle of convolution, the gradient calculation of the inverse convolution kernel involves the following data reuse: 1) reuse of the input feature map across different output channels; 2) reuse of the gradient of the feature map from the next layer across different input channels; 3) reuse of the input feature map across different heights and widths of the convolution kernel. When accelerating the gradient calculation of the inverse convolution kernel on many-core processors, the first two data reuse characteristics are typically utilized. However, when the height or width of the convolution kernel is greater than 1, the last reuse characteristic is often ignored and not fully utilized.

[0006] This invention aims to design a strategy scheme that leverages the data reuse characteristics of convolutional processing and the hardware features of many-core processor architecture to maximize the reuse of data in on-chip cache, thereby increasing data reuse and reducing data access volume between SPM and global memory, reducing memory access latency, and improving performance. This invention provides strategy design guidance and performance optimization for gradient calculation of inverse convolutional kernels with a kernel height or width greater than 1. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and apparatus for reusing reverse weight gradient calculation data based on many-core processors.

[0008] The objective of this invention is achieved through the following technical solution: a method for reusing inverse weight gradient calculation data based on many-core processors, comprising the following steps:

[0009] (1) Based on the size information of the deep learning model and the input dataset, obtain the size of the convolutional layer to be calculated; and obtain the hardware information of the many-core processor;

[0010] (2) Set the height R and width S of the convolution kernel in the block size bR and bS of a single computing core to obtain three candidate schemes: first candidate scheme: bR=bS=1; second candidate scheme: bR=1, bS=S; third candidate scheme: bR=R, bS=S; determine the block size bC and bM of the number of input channels C and the number of output channels M in a single computing core in each candidate scheme according to the hardware information of the many-core processor, determine the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core; and determine the loop order of the three candidate schemes.

[0011] (3) Based on the loop order and data reuse method calculated by the three candidate schemes, the number of times each element in the convolutional layer is accessed repeatedly is calculated, and combined with the convolutional layer size and bC, bR, bS, bM, the data access volume between on-chip local storage and global memory in each candidate scheme is calculated.

[0012] (4) Select the candidate solution with the least amount of data access.

[0013] Further, in step (1), the convolutional layer size includes: the size of the input feature map, the size of the weights, and the size of the feature map gradient from the next layer; the size of the input feature map includes the batch size, the number of input channels, and the height and width of the feature map; the hardware information of the many-core processor includes: the depth P of the accumulator buffer of each column of the systolic array, the number of columns SA_CO of the systolic array, and the number of rows RO and columns CO of the computing core array of the many-core processor.

[0014] Further, in step (2), determining the block size bC and bM of the number of input channels C and the number of output channels M in each candidate scheme on a single computing core specifically involves: determining the block size bC and bM of the number of input channels C and the number of output channels M in each candidate scheme on a single computing core based on the block size bS and bR in each scheme, the depth P of the accumulator buffer, and the number of columns SA_CO of the systolic array. bC is taken as no more than the largest power of 2 of P divided by the product of bS and bR, and bM is taken as SA_CO.

[0015] Specifically, determining the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core is as follows: the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core must satisfy the following: bC_core is the maximum value between the floor value of C divided by RO and bC; bM_core is the maximum value between the floor value of M divided by CO and bM.

[0016] Furthermore, the cyclic order of computation for the first candidate solution is determined by the following method:

[0017] (a.1) Each computational core traverses the width of the output gradient map F times, and each time calculates an intermediate width gradient block of size bC×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result.

[0018] (a.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result.

[0019] (a.3) The calculation of the input channel dimension is performed by dividing the block into loops with a block size of bC, resulting in a gradient block of size bC_core×bM, which is the third calculation result;

[0020] (a.4) The calculation of the output channel dimension is performed by dividing the block into loops with a block size of bM to obtain a gradient block of size bC_core×bM_core, which is the fourth calculation result;

[0021] (a.5) Traverse the width of the fourth calculation result S times to obtain a gradient block of size bC_core×S×bM_core, which is the fifth calculation result;

[0022] (a.6) Traverse the height of the fifth calculation result R times to obtain a gradient block of size bC_core×R×S×bM_core, which is the sixth calculation result;

[0023] (a.7) The sixth calculation result obtained from all the computation cores is processed in parallel to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

[0024] Furthermore, the cyclic order of the calculation of the second candidate scheme is determined by the following method:

[0025] (b.1) Each computational core traverses the width of the output gradient map F times. Each time, it calculates an intermediate width gradient block with size bC×S×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result.

[0026] (b.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×S×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result.

[0027] (b.3) The calculation of the input channel dimension is performed in a block loop with a block size of bC to obtain a third calculation result with a size of bC_core×S×bM;

[0028] (b.4) The calculation of the output channel dimension is performed in a block loop with a block size of bM, resulting in a fourth calculation result with a size of bC_core×S×bM_core;

[0029] (b.5) Iterate through the height of the fourth calculation result R times to obtain the fifth calculation result with size bC_core×R×S×bM_core;

[0030] (b.6) The fifth calculation result obtained from all the computation cores is processed in parallel to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

[0031] Furthermore, the cycle order for calculating the third candidate scheme is determined by the following method:

[0032] (c.1) Each computational core traverses the width of the output gradient map F times. Each time, it calculates an intermediate width gradient block with size bC×R×S×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result.

[0033] (c.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×R×S×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result.

[0034] (c.3) The calculation of the input channel dimension is performed in a block loop with a block size of bC to obtain a third calculation result with a size of bC_core×R×S×bM;

[0035] (c.4) The calculation of the output channel dimension is performed in a block loop with a block size of bM to obtain the fourth calculation result with a size of bC_core×R×S×bM_core;

[0036] (c.5) The fourth calculation result obtained from all the calculation cores is processed in parallel to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

[0037] Furthermore, step (3) specifically includes:

[0038] The data access quantity (memory_access_dw) associated with the gradient dw of the convolution kernel weights is calculated as follows:

[0039] memory_access_dw is the product of the number of bytes occupied in memory for C, R, S, M, and dw data types;

[0040] The first candidate solution satisfies the following conditions regarding the data access volume memory_access_x_1 for reading the input feature map from global storage and the data access volume memory_access_dy_1 for reading the gradient of the feature map from the next layer from global storage:

[0041] memory_access_x_1 is the product of R, S, N, H, W, C, M divided by the floor value of bM_1 and the number of bytes occupied by the x data type in memory;

[0042] memory_access_dy_1 is the product of R, S, N, E, F, M, C divided by bC_1 (rounded up) and the number of bytes occupied by the dy data type in memory;

[0043] The second candidate solution satisfies the following conditions: the data access volume memory_access_x_2 for reading the input feature map from global storage and the data access volume memory_access_dy_2 for reading the gradient of the feature map from the next layer from global storage.

[0044] memory_access_x_2 is the product of R, N, H, W, C, M divided by bM_2 (rounded up) and the number of bytes occupied by the x data type in memory;

[0045] memory_access_dy_2 is the product of R, N, E, F, M, C divided by bC_2 (rounded up) and the number of bytes occupied by the dy data type in memory;

[0046] The third candidate solution satisfies the following conditions: the data access volume memory_access_x_3 for reading the input feature map from global storage and the data access volume memory_access_dy_3 for reading the gradient of the feature map from the next layer from global storage.

[0047] memory_access_x_3 is the product of N, H, W, C, M divided by bM_3 (rounded up) and the number of bytes occupied by the x data type in memory;

[0048] memory_access_dy_3 is the product of N, E, F, M, C divided by the floor value of bC_3, and the number of bytes occupied by the dy data type in memory.

[0049] Further, step (4) specifically involves selecting the candidate scheme with the smallest total data access volume based on the total data access volume of each candidate scheme calculated in step (3); the total data access volume includes the data access volume of the input feature map x read from the global storage, the data access volume of the feature map gradient dy from the next layer read from the global storage, and the data access volume related to dw.

[0050] The present invention also includes a data reuse device for reverse weight gradient calculation based on a many-core processor, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-described method for data reuse of reverse weight gradient calculation based on a many-core processor.

[0051] The present invention also includes a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-described method for reusing reverse weight gradient calculation data based on a many-core processor.

[0052] The beneficial effects of this invention are: This invention utilizes the data reuse characteristics of inverse convolution kernel gradient calculation to optimize the calculation loop order, reduces the amount of data access between SPM and global memory, and improves performance by analyzing and selecting the candidate scheme with the least amount of data access. Attached Figure Description

[0053] Figure 1A flowchart of a data reuse method for inverse weight gradient calculation based on many-core processors;

[0054] Figure 2 This is a schematic diagram of a data reuse device for inverse weight gradient calculation based on a many-core processor.

[0055] Figure 3 A schematic diagram of the loop order calculated for the first candidate solution;

[0056] Figure 4 A schematic diagram of the loop order calculated for the second candidate solution;

[0057] Figure 5 A schematic diagram of the cyclical order of calculations for the third candidate scheme. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0059] Example 1: As Figure 1 As shown, this invention provides a method for reusing reverse weight gradient calculation data based on many-core processors, including the following sub-steps:

[0060] (1) Based on the size information of the deep learning model and the input dataset, obtain the size of the convolutional layer to be calculated; and obtain the hardware information of the many-core processor.

[0061] Specifically, based on the architecture definition of the deep learning model (such as ResNet50, SSD-ResNet34, etc.) and the size information of the input dataset (including batch size, number of channels, spatial resolution, etc.), the size of the input feature map of each convolutional layer is calculated, and the hardware information of the many-core processor is obtained for subsequent hardware-aware optimization.

[0062] The convolutional layer dimensions include the input feature map size NHWC, the weight size CRSM, and the gradient size NEFM from the next layer's feature map. The input feature map x dimensions include the batch size N, the number of input channels C, and the height H and width W of the feature map x. The input feature map dimensions are [N, H, W, C], where N represents the batch size, H represents the height of the input feature map x, W represents the width of the input feature map x, and C represents the number of input channels.

[0063] The hardware information of the many-core processor includes: the depth P of the accumulator buffer of each column of the systolic array, the number of columns SA_CO of the systolic array, and the number of rows RO and columns CO of the computing core array of the many-core processor.

[0064] (2) Set the height R and width S of the convolution kernel in the block size bR and bS of a single computing core to obtain three candidate schemes: first candidate scheme: bR=bS=1; second candidate scheme: bR=1, bS=S; third candidate scheme: bR=R, bS=S; determine the block size bC and bM of the number of input channels C and the number of output channels M in a single computing core in each candidate scheme according to the hardware information of the many-core processor, determine the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core; and determine the loop order of the three candidate schemes.

[0065] Specifically, candidate solutions are designed based on the principle of inverse weight gradient calculation and the hardware architecture characteristics of many-core processors. The formula for calculating the inverse weight gradient is:

[0066]

[0067] According to the principle of convolution, the gradient dy of the input feature map x of size [N, H, W, C] and the output y of size [N, E, F, M] is calculated by convolution to obtain the gradient dw of the weights w of size [C, R, S, M]. Here, E represents the height of the output feature map y, F represents the width of the output feature map y, R represents the height of the convolution kernel, S represents the width of the convolution kernel, and M represents the number of channels of the output feature map y. The dimensions of the input channels C and the output channels M are retained in the calculation result dw. The calculations between different input channels C are independent of each other, and the calculations between different output channels M are also independent of each other. The three dimensions N, E, and F are used as the accumulation dimensions for the convolution calculation. Each data point in the output calculation result dw is obtained by multiplying and accumulating N×E×F data points in the input feature map x and N×E×F data points in the gradient dy. A sliding window of size N×E×F slides R×S times over the N×H×W data points of the input feature map x, finally obtaining the output dw of size [C, R, S, M].

[0068] Based on the hardware architecture characteristics of many-core processors, data read from global memory via direct memory access (DMA) can be broadcast among multiple computing cores in the same column or row, meaning that reading one piece of data can be reused by multiple computing cores simultaneously.

[0069] The basic computational task splitting strategy is to partition C, M, and the accumulation dimension NEF onto different computing cores for computation. According to the capacity limit of the accumulator buffer, the computed result after chunking for each computing core in each operation is bC×bR×bS×bM, where bC, bR, bS, and bM respectively represent the chunk sizes of the input channel number C, the height R of the convolutional kernel, the width S of the convolutional kernel, and the output channel number M in each computation on each computing core. Since the computation between the input channel number C and the output channel number M is independent, priority is given to partitioning the dimensions of the input channel number C and the output channel number M onto different computing cores; when ceil(C / bC)×ceil(M / bM)<RO×CO, where ceil() represents the ceiling function, bC represents the chunk size when chunking the input channel C, and bM represents the chunk size when chunking the output channel M (ceil(C / bC) is the minimum number of chunks required after dividing C columns into chunks of bC columns each and rounding up, and ceil(M / bM) is the minimum number of chunks required after dividing M rows into chunks of bM rows each and rounding up), for the load balancing of the computing cores, the accumulation dimension NEF that needs to be accumulated is partitioned onto different computing cores, and a reduction operation is performed after the accumulation on a single core. Data reuse can be carried out between the dimensions of the height R and the width S of the convolutional kernel, and the sizes of the height R and the width S of common convolutional kernels are mostly odd numbers and are not easily divisible by the number of computing cores. Therefore, the dimensions of the height R and the width S of the convolutional kernel are not partitioned onto different computing cores, bR takes R or 1, and bS takes S or 1.

[0070] Since the weight data format is CRSM, in terms of the data storage continuity in the global memory, the S×M data for each R will be stored continuously, and then the S×M data for the next R will be placed. In the case where bS = 1 and bR = R, data cannot be continuously read from the global memory. Therefore, three candidate schemes are finally obtained: 1) bR = bS = 1; 2) bR = 1, bS = S; 3) bR = R, bS = S.

[0071] Furthermore, the specific method for determining the chunk sizes bC and bM of the input channel number C and the output channel number M in each candidate scheme on a single computing core is as follows: According to the chunk size bR, the chunk size bS, the depth P of the accumulator buffer in each column of the systolic array on the many-core processor, and the number of columns SA_CO of the systolic array, determine the chunk sizes bC and bM of the input channel number C and the output channel number M in each candidate scheme on a single computing core. Generally, bM takes SA_CO, and bC takes the largest power of 2 that does not exceed P / (bS×bR);

[0072] The determination of the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core specifically involves: according to the computing task partitioning strategy, determining the task partitioning of C and M on the computing core array based on parameters such as C, M, RO, and CO, thus obtaining the total number of input channels bC_core and the total number of output channels bM_core that need to be computed on each computing core. Since the computation between the number of input channels C and the number of output channels M is independent, and data read from global storage can be broadcast among multiple computing cores in the same column or row, meaning that reading one piece of data can be reused by multiple computing cores simultaneously. A common method is to allocate the number of input channels C to computing cores in row RO and the number of output channels M to computing cores in column CO. However, the bC_core allocated to each computing core cannot be less than bC, and the bM_core cannot be less than bM. Therefore, bC_core = max(ceil(C / RO), bC), bM_core = max(ceil(M / CO), bM), where max() is the maximum value function.

[0073] Furthermore, the computational cycle order of the three candidate schemes is determined for evaluation in subsequent steps. If the total number of input channels to be computed by each computation core, bC_core, is greater than the number of input channels computed by each computation core per cycle, bC_core needs to be computed time-divisionally on the same computation core, with a cycle count of ceil(bC_core / bC); similarly, the cycle count for output channels is ceil(bM_core / bM). Note that the values ​​of bC, bR, bS, and bM are different in different candidate schemes.

[0074] Determine the loop order for calculating the first candidate solution (bR=bS=1), such as... Figure 3 As shown, the specific steps include the following:

[0075] (a.1) Each computational core traverses the width of the output gradient map F times. Each time, it calculates an intermediate width gradient block with size bC×bM. After F iterations, it accumulates all intermediate width gradient blocks point-to-point along the width dimension to obtain the first calculation result.

[0076] (a.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×bM. After E iterations, accumulate all intermediate height gradient blocks point-to-point along the height dimension to obtain the second calculation result.

[0077] (a.3) Iterate through the input channel dimension ceil(bC_core / bC) times, processing bC input channels each time, and expanding the second calculation result along the input channel dimension to obtain a third calculation result with a size of bC_core×bM;

[0078] (a.4) Iterate through the output channel dimension ceil(bM_core / bM) times, processing bM input channels each time, and expanding the third calculation result along the output channel dimension to obtain the fourth calculation result with size bC_core×bM_core;

[0079] (a.5) Iterate through the width of the fourth calculation result S times, aligning the fourth calculation result with the width dimension each time, and obtain the fifth calculation result with size bC_core×S×bM_core through spatial convolution;

[0080] (a.6) Iterate through the height of the fifth calculation result R times, aligning the fifth calculation result with the height dimension each time, and obtain the sixth calculation result with size bC_core×R×S×bM_core through spatial convolution;

[0081] (a.7) The sixth calculation result obtained by all the computation cores is extended along the input channel dimension and the output channel dimension through parallel computation to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

[0082] Determine the loop order for calculating the second candidate solution (bR=1, bS=S), as follows: Figure 4 As shown, the specific steps include the following:

[0083] (b.1) Each computational core traverses the width of the output gradient map F times. The innermost loop is a systolic array multiplication and accumulation calculation. Each time, the data size of the input feature map read from the local storage is bC×S×N, and the data size of the output gradient map read is N×S×bM. Each calculation obtains an intermediate width gradient block with a size of bC×S×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result.

[0084] (b.2) Traverse the height of the output gradient map E times. Each time, read data from the global storage to the local storage through direct memory access. The data size of the input feature map read is bC×W×N, and the data size of the output gradient map read is N×F×bM. Each time, calculate the intermediate height gradient block with size bC×S×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result.

[0085] (b.3) Iterate through the input channel dimension ceil(bC_core / bC) times, processing bC input channels each time, and expanding the second calculation result along the input channel dimension to obtain the third calculation result with size bC_core×S×bM;

[0086] (b.4) Iterate through the output channel dimension ceil(bM_core / bM) times, processing bM input channels each time, and expanding the third calculation result along the output channel dimension to obtain the fourth calculation result with size bC_core×S×bM_core;

[0087] (b.5) Iterate through the height of the fourth calculation result R times, aligning the fourth calculation result with the height dimension each time, and obtain the fifth calculation result with size bC_core×R×S×bM_core through spatial convolution;

[0088] (b.6) The fifth calculation result obtained by all the computation cores is extended along the input channel dimension and the output channel dimension through parallel computation to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

[0089] Determine the loop order for calculating the third candidate solution (bR=R, bS=S), as follows: Figure 5 As shown, the specific steps include the following:

[0090] (c.1) Each computational core traverses the width of the output gradient map F times. The innermost loop is a systolic array multiplication and accumulation calculation. Each time, the data size of the input feature map read from the local storage is bC×R×S×N, and the data size of the output gradient map read is N×R×S×bM. Each time, an intermediate width gradient block with a size of bC×R×S×bM is obtained. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result.

[0091] (c.2) Traverse the height of the output gradient map E times. Each time, read data from the global storage to the local storage through direct memory access. The data size of the input feature map read is bC×N×R×W, and the data size of the output gradient map read is N×F×bM. Each time, calculate the intermediate height gradient block with size bC×R×S×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result.

[0092] (c.3) Iterate through the input channel dimension ceil(bC_core / bC) times, processing bC input channels each time, and expanding the second calculation result along the input channel dimension to obtain the third calculation result with size bC_core×R×S×bM;

[0093] (c.4) Iterate through the output channel dimension ceil(bM_core / bM) times, processing bM input channels each time, and expanding the third calculation result along the output channel dimension to obtain the fourth calculation result with size bC_core×R ×S×bM_core;

[0094] (c.5) The fourth calculation result obtained by all the computation cores is extended along the input channel dimension and the output channel dimension through parallel computation to obtain the gradient dw of the convolution kernel weight with size C×R×S×M.

[0095] (3) Based on the loop order and data reuse method calculated by the three candidate schemes, the number of times each element in the convolutional layer is accessed repeatedly is calculated, and combined with the convolutional layer size and bC, bR, bS, bM, the data access volume between on-chip local memory (Scratchpad Memory) and global memory in each candidate scheme is calculated.

[0096] The data read from the global memory includes the input feature map x and the feature map gradient dy from the next layer. The data written to the global memory includes the calculated weight gradient dw: dw = x conv dy.

[0097] Furthermore, data access related to data write (dw) will not be read repeatedly, but will only be written once, and will not change across the three candidate schemes. The formula for the data access volume related to data write (memory_access_dw) (in bytes, for writing data to global memory) is as follows:

[0098] memory_access_dw=C×R×S×M×sizeof(dtype(dw));

[0099] Here, sizeof() represents the number of bytes occupied in memory for the data format; dtype() represents the data type.

[0100] The input feature map x and the gradient dy from the feature map of the next layer need to be read repeatedly. The number of repeated reads varies depending on the computation loop design, resulting in different data access volumes. The specific calculation formulas for the data access volume of the input feature map x read from global memory (memory_access_x_i, in bytes, with the numeric suffix i indicating the candidate solution) and the data access volume of the gradient dy from the feature map of the next layer (memory_access_y_i, in bytes, with the numeric suffix i indicating the candidate solution) for each candidate solution are as follows:

[0101] Candidate Solution 1:

[0102] memory_access_x_1=R×S×ceil(M / (bM_1))×N×H×W×C×sizeof(dtype(x))

[0103] memory_access_dy_1=R×S×ceil(C / (bC_1))×N×E×F×M×sizeof(dtype(dy))

[0104] Candidate Solution 2:

[0105] memory_access_x_2=R×ceil(M / (bM_2))×N×H×W×C×sizeof(dtype(x))

[0106] memory_access_dy_2=R×ceil(C / (bC_2))×N×E×F×M×sizeof(dtype(dy))

[0107] Candidate Solution 3:

[0108] memory_access_x_3=ceil(M / (bM_3))×N×H×W×C×sizeof(dtype(x))

[0109] memory_access_dy_3=ceil(C / (bC_3))×N×E×F×M×sizeof(dtype(dy))

[0110] Where bC_1, bC_2, and bC_3 represent the number of input channels bC calculated per computation by each computing core in candidate schemes 1, 2, and 3, respectively, and bM_1, bM_2, and bM_3 represent the number of output channels bM calculated per computation by each computing core in candidate schemes 1, 2, and 3, respectively.

[0111] (4) Select the candidate solution with the least amount of data access.

[0112] Further, the candidate scheme with the smallest data access volume is selected based on the data access volume (memory_access_x_i, memory_access_dy_i, memory_access_dw) of each candidate scheme calculated in step (3); the total data access volume includes the data access volume of the input feature map x read from the global storage, the data access volume of the feature map gradient dy from the next layer read from the global storage, and the data access volume related to dw.

[0113] Specifically, a double-buffering acceleration strategy is adopted when developing operators, where computation time and memory access time are hidden from each other. Since the performance bottleneck is usually in memory access, the candidate solution with the minimum total data access volume is selected. The formula for the total data access volume memory_access_i is as follows:

[0114] It is worth noting that as the candidate scheme number increases, although the data of x and dy can be reused between different S or different R, the data access volume of x and dy can be reduced by multiples according to the R and S values. However, as bS and bR increase, due to the limited accumulator buffer capacity of the systolic array, the decrease in the bC value may lead to an increase in the data access volume of dy. Overall, the data access volume of x will decrease, but the data access volume of dy may not necessarily decrease. Therefore, it is necessary to calculate the total data access volume to determine this.

[0115] Example 2: This example relates to a data reuse device for reverse weight gradient calculation based on a many-core processor, including a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, it is used for the data reuse method for reverse weight gradient calculation based on a many-core processor in Example 1 above. The device example can be applied to any device with data processing capabilities, such as a computer or other equipment.

[0116] like Figure 2 As shown, at the hardware level, this model includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 1 The method is illustrated. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0117] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are the most commonly used. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0118] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0119] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0120] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

Claims

1. A method for reusing data in reverse weight gradient calculation based on many-core processors, characterized in that, Includes the following steps: (1) Based on the deep learning model and the size information of the input dataset, obtain the size of the convolutional layer to be calculated; And obtain the hardware information of the many-core processor; (2) Set the height R and width S of the convolution kernel in the block size bR and bS of a single computation kernel to obtain three candidate schemes: First candidate scheme: bR=bS=1; Second candidate scheme: bR=1, bS=S; Third candidate scheme: bR=R, bS=S. Based on the hardware information of the many-core processor, determine the block size bC and bM of the number of input channels C and the number of output channels M on a single computing core in each candidate scheme, and determine the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core; And determine the cyclic order of calculation for the three candidate schemes; (3) Based on the loop order and data reuse method calculated by the three candidate schemes, the number of times each element in the convolutional layer is accessed repeatedly is calculated, and combined with the convolutional layer size and bC, bR, bS, bM, the data access volume between on-chip local storage and global memory in each candidate scheme is calculated. The formula for the data access quantity memory_access_dw related to the gradient dw of the convolution kernel weights is as follows: memory_access_dw=C×R×S×M×sizeof(dtype(dw)); Here, sizeof() represents the number of bytes occupied in memory for the data format; dtype() represents the data type. The specific formulas for calculating the data access volume (memory_access_x_i) of the input feature map x read from global memory and the data access volume (memory_access_y_i) of the gradient of the feature map dy from the next layer read from global memory for each candidate scheme are as follows: Candidate Solution 1: memory_access_x_1=R×S×ceil(M / (bM_1))×N×H×W×C×sizeof(dtype(x)) memory_access_dy_1=R×S×ceil(C / (bC_1))×N×E×F×M×sizeof(dtype(dy)) Candidate Solution 2: memory_access_x_2=R×ceil(M / (bM_2))×N×H×W×C×sizeof(dtype(x)) memory_access_dy_2=R×ceil(C / (bC_2))×N×E×F×M×sizeof(dtype(dy)) Candidate Solution 3: memory_access_x_3=ceil(M / (bM_3))×N×H×W×C×sizeof(dtype(x)) memory_access_dy_3=ceil(C / (bC_3))×N×E×F×M×sizeof(dtype(dy)) Where bC_1, bC_2, and bC_3 represent the number of input channels bC calculated per computation in each computation core in candidate schemes 1, 2, and 3, respectively; bM_1, bM_2, and bM_3 represent the number of output channels bM calculated per computation in each computation core in candidate schemes 1, 2, and 3, respectively; N represents the batch size, H represents the height of the input feature map x, W represents the width of the input feature map x, E represents the height of the output feature map y, F represents the width of the output feature map y, M represents the number of channels in the output feature map y, and C represents the number of input channels; ceil() represents the floor function. (4) Select the candidate solution with the least amount of data access.

2. The method for reusing reverse weight gradient calculation data based on many-core processors according to claim 1, characterized in that, In step (1), the convolutional layer size includes: the size of the input feature map, the size of the weights, and the size of the feature map gradient from the next layer; the size of the input feature map includes the batch size, the number of input channels, and the height and width of the feature map; the hardware information of the many-core processor includes: the depth P of the accumulator buffer of each column of the systolic array, the number of columns SA_CO of the systolic array, and the number of rows RO and columns CO of the computing core array of the many-core processor.

3. The method for reusing reverse weight gradient calculation data based on many-core processors according to claim 2, characterized in that: In step (2), The determination of the block size bC and bM of the number of input channels C and the number of output channels M in each candidate scheme on a single computing core is specifically as follows: the block size bC and bM of the number of input channels C and the number of output channels M in each candidate scheme on a single computing core are determined according to the block size bS and bR in each scheme, the depth P of the accumulator buffer, and the number of columns SA_CO of the systolic array. bC is taken as the largest power of 2 of P divided by the product of bS and bR, and bM is taken as SA_CO. Specifically, determining the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core is as follows: the total number of input channels bC_core and the total number of output channels bM_core allocated to each computing core must satisfy the following: bC_core is the maximum value between the floor value of C divided by RO and bC; bM_core is the maximum value between the floor value of M divided by CO and bM.

4. The method for reusing reverse weight gradient calculation data based on many-core processors according to claim 1, characterized in that, The cycle order for calculating the first candidate solution is determined by the following method: (a.1) Each computational core traverses the width of the output gradient map F times, and each time calculates an intermediate width gradient block of size bC×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result. (a.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result. (a.3) The calculation of the input channel dimension is performed by dividing the block into loops with a block size of bC, resulting in a gradient block of size bC_core×bM, which is the third calculation result; (a.4) The calculation of the output channel dimension is performed by dividing the block into loops with a block size of bM to obtain a gradient block of size bC_core×bM_core, which is the fourth calculation result; (a.5) Traverse the width of the fourth calculation result S times to obtain a gradient block of size bC_core×S×bM_core, which is the fifth calculation result; (a.6) Traverse the height of the fifth calculation result R times to obtain a gradient block of size bC_core×R×S×bM_core, which is the sixth calculation result; (a.7) The sixth calculation result obtained from all the computation cores is processed in parallel to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

5. The method for reusing reverse weight gradient calculation data based on many-core processors according to claim 1, characterized in that, The cycle order for calculating the second candidate scheme is determined by the following method: (b.1) Each computational core traverses the width of the output gradient map F times. Each time, it calculates an intermediate width gradient block with size bC×S×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result. (b.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×S×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result. (b.3) The calculation of the input channel dimension is performed in a block loop with a block size of bC to obtain a third calculation result with a size of bC_core×S×bM; (b.4) The calculation of the output channel dimension is performed in a block loop with a block size of bM, resulting in a fourth calculation result with a size of bC_core×S×bM_core; (b.5) Iterate through the height of the fourth calculation result R times to obtain the fifth calculation result with size bC_core×R×S×bM_core; (b.6) The fifth calculation result obtained from all the computation cores is processed in parallel to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

6. The method for reusing reverse weight gradient calculation data based on many-core processors according to claim 1, characterized in that, The cycle order for calculating the third candidate solution is determined by the following method: (c.1) Each computational core traverses the width of the output gradient map F times. Each time, it calculates an intermediate width gradient block with size bC×R×S×bM. After F iterations, all intermediate width gradient blocks are accumulated point-to-point to obtain the first calculation result. (c.2) Traverse the height of the output gradient map E times. Each time, calculate the intermediate height gradient block with size bC×R×S×bM. After E iterations, accumulate all intermediate height gradient blocks point by point to obtain the second calculation result. (c.3) The calculation of the input channel dimension is performed in a block loop with a block size of bC to obtain a third calculation result with a size of bC_core×R×S×bM; (c.4) The calculation of the output channel dimension is performed in a block loop with a block size of bM to obtain a fourth calculation result with a size of bC_core×R ×S×bM_core; (c.5) The fourth calculation result obtained from all the calculation cores is processed in parallel to obtain the gradient dw of the convolution kernel weights with size C×R×S×M.

7. The method for reusing reverse weight gradient calculation data based on many-core processors according to claim 1, characterized in that, The specific steps (4) are as follows: based on the total data access volume of each candidate scheme calculated in step (3), select the candidate scheme with the smallest total data access volume; the total data access volume includes the data access volume of the input feature map x read from the global storage, the data access volume of the feature map gradient dy from the next layer read from the global storage, and the data access volume related to dw.

8. A data reuse device for reverse weight gradient calculation based on many-core processors, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the data reuse method for reverse weight gradient calculation based on a many-core processor as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a method for reusing reverse weight gradient calculation data based on a many-core processor, as described in any one of claims 1-7.

Images