Data processing method, device, equipment and storage medium

Abstract

Embodiments of the present disclosure provide a data processing method, apparatus, device, and computer-readable storage medium. The method provided by the embodiments of the present disclosure optimizes the performance of the back propagation calculation of the depth-separable two-dimensional convolution operator, by pre-loading part of the data of the output tensor from the off-chip memory into the group shared memory each time, and pre-loading the data of the weight matrix from the off-chip memory into the thread private register, so as to use the data of the output tensor loaded into the group shared memory and the data of the weight matrix to calculate the corresponding data in the input tensor, thereby obtaining the input tensor through back propagation based on the output tensor and the weight matrix, wherein the high memory access bandwidth of the group shared memory is used to cache data, which can improve the memory access efficiency of the data and improve the performance of the back propagation calculation of the depth-separable two-dimensional convolution operator.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and more specifically, to a data processing method, apparatus, device and storage medium. Background Art

[0002] The rapid development of artificial intelligence technology has led to the widespread application of artificial intelligence chips in various fields. As the basis for efficient computing of artificial intelligence chips, high-performance operators are crucial to improving computing efficiency and performance. In artificial intelligence applications, deep learning models usually contain a large number of convolution operations, and the depthwise separable convolution operator is one of the commonly used convolution operators.

[0003] The depthwise separable convolution operator is a convolution operation that decomposes the standard convolution operation into two steps: depthwise convolution and pointwise convolution. In this way, the number of parameters and the amount of calculation can be reduced, thereby improving computational efficiency. In practical applications, tuning the depthwise separable convolution operator can further improve its performance, including reducing memory usage, speeding up calculations, and improving accuracy.

[0004] Therefore, an effective data processing method is needed to further optimize the depthwise separable convolution operator to improve the computational efficiency and performance of the model. Summary of the invention

[0005] Embodiments of the present disclosure provide a data processing method, apparatus, device, and computer-readable storage medium.

[0006] An embodiment of the present disclosure provides a data processing method for obtaining an input tensor through back propagation based on an output tensor, the method comprising: loading a portion of the data in the output tensor from an off-chip memory into a group shared memory, and loading a weight tensor from the off-chip memory into a thread private register, wherein the dimension of the input channel of the weight tensor is one-dimensional; based on the portion of the data loaded from the group shared memory and the weight tensor loaded from the thread private register, determining the corresponding data in the input tensor.

[0007] An embodiment of the present disclosure provides a data processing device for obtaining an input tensor through back propagation based on an output tensor, the device comprising: a data loading module, configured to load a portion of the data in the output tensor from an off-chip memory into a group shared memory, and to load a weight tensor from the off-chip memory into a thread private register, wherein the dimension of the input channel of the weight tensor is one-dimensional; a data calculation module, configured to determine the corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register.

[0008] An embodiment of the present disclosure provides a data processing device, including: one or more processors; and one or more memories, wherein a computer executable program is stored in the one or more memories, and when the computer executable program is executed by the processor, the data processing method as described above is performed.

[0009] An embodiment of the present disclosure provides a computer-readable storage medium on which computer-executable instructions are stored. When the instructions are executed by a processor, the instructions are used to implement the data processing method as described above.

[0010] The embodiments of the present disclosure provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method according to the embodiments of the present disclosure.

[0011] The method provided by the embodiments of the present disclosure optimizes the performance of the back propagation calculation of the depthwise separable two-dimensional convolution operator, by pre-loading part of the data of the output tensor from the off-chip memory into the group shared memory each time, and pre-loading the data of the weight matrix from the off-chip memory into the thread private register, to use the data of the output tensor loaded into the group shared memory and the data of the weight matrix to calculate the corresponding data in the input tensor, so as to obtain the input tensor through back propagation based on the output tensor and the weight matrix, wherein the high memory access bandwidth of the group shared memory is used to cache data, so as to improve the memory access efficiency of the data and improve the performance of the back propagation calculation of the depthwise separable two-dimensional convolution operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0013] Figure 1 is a schematic diagram illustrating a conventional convolution process;

[0014] Figure 2 is a schematic diagram illustrating a grouped convolution process;

[0015] Figure 3 is a schematic diagram illustrating a depthwise separable convolution process;

[0016] Figure 4is a flow chart showing a data processing method according to an embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram illustrating a data processing method according to an embodiment of the present disclosure;

[0018] Figure 6 is a schematic diagram showing data processing based on a convolutional segmentation strategy according to an embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram showing a data processing device according to an embodiment of the present disclosure;

[0020] Figure 8 A schematic diagram showing a data processing device according to an embodiment of the present disclosure; and

[0021] Fig. 9 A schematic diagram illustrating the architecture of an exemplary computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0023] In this specification and the accompanying drawings, substantially the same or similar steps and elements are represented by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance or ranking.

[0024] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure pertains. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0026] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.

[0027] The data processing method disclosed in the present invention may be based on artificial intelligence (AI). Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.

[0028] The data processing method disclosed in the present invention can be based on depth separable convolution. Depth separable convolution is a convolution operation widely used in the fields of computer vision and deep learning. It is a special convolution operation that reduces the number of parameters and the amount of calculation by decomposing the standard convolution operation, thereby improving the efficiency and performance of the model. Depth separable convolution includes two steps: depth convolution and point-by-point convolution. In the depth convolution stage, each input channel is convolved with an independent convolution kernel, which helps to extract the features of the input data. In the point-by-point convolution stage, the output of the depth convolution is convolved with a 1x1 convolution kernel to integrate and interact the information between the channels. In this way, the number of parameters and the amount of calculation can be reduced, thereby improving the computational efficiency. In practical applications, tuning the depth separable convolution operator can further improve its performance, including reducing memory usage, speeding up calculation speed and improving accuracy.

[0029] In summary, the solutions provided by the embodiments of the present disclosure involve technologies such as artificial intelligence and depthwise separable convolution. The embodiments of the present disclosure will be further described below in conjunction with the accompanying drawings.

[0030] Figure 1 is a schematic diagram illustrating a conventional convolution process. Figure 2 is a schematic diagram illustrating grouped convolution processing. Figure 3 is a schematic diagram illustrating the depthwise separable convolution process.

[0031] As an integrated circuit chip specifically used to accelerate artificial intelligence applications, artificial intelligence chips are designed to efficiently execute deep learning algorithms and neural network models. The development and application of artificial intelligence chips have promoted the rapid development of artificial intelligence technology. They are widely used in image recognition, speech recognition, natural language processing, intelligent driving and other fields to meet the needs of different application scenarios. For the artificial intelligence chip software stack that runs on artificial intelligence chips to support the development and deployment of artificial intelligence applications, its design and implementation are crucial to the performance and efficiency of artificial intelligence chips. It can help developers develop efficient artificial intelligence applications more quickly and improve the utilization and performance of artificial intelligence chips. There are a large number of convolution operations in the artificial intelligence chip software stack. For convolution operations, the standard (conventional) convolution operation used in convolutional neural networks convolves the input data with the convolution kernel to generate a feature map. In conventional convolution processing, all input data will be convolved with the same convolution kernel to extract the features of the entire input data. Such as Figure 1 As shown, for input data (e.g., an input tensor of dimension c1*H*W), it can be convolved with c2 convolution kernels (where each convolution kernel has dimension c1*h1*w1) to generate output data (e.g., an output tensor of dimension c2*H*W). As a special convolution operation, grouped convolution processing divides the input data into multiple groups, and the data of each group is convolved with the corresponding convolution kernel respectively, and finally the convolution results of each group are merged together to generate the final output data. Figure 2 As shown, the input data (e.g., an input tensor with a dimension of c1*H*W) is divided into g groups, and the convolution kernels (where each convolution kernel has a dimension of c1*h1*w1) are also divided into corresponding g groups. By convolving the input data of each group with the corresponding convolution kernel and merging the convolution results of each group together, the output data (e.g., an output tensor with a dimension of c2*H*W) can be obtained. Through grouped convolution processing, the number of parameters can be effectively reduced, the computational efficiency can be improved, and the generalization ability of the model can be improved to a certain extent.

[0032] Based on the grouped convolution, the number of groups is set to the number of channels of the input tensor (e.g. Figure 3 In the depthwise separable convolution, the convolution kernel of the depthwise convolution is designed as a single-channel mode, that is, the convolution kernel is divided into (for example, C) single-channel convolution kernels of the number of channels of the input tensor according to its number of channels, and each channel of the input is convolved with a single-channel convolution kernel to obtain an output tensor with the same number of channels as the input tensor. Figure 3As shown, the input tensor can be divided into C groups according to its number of channels C, where the dimension of the input data of each group is 1*H*W. Similarly, the convolution kernel can be divided into C groups, namely C single-channel convolution kernels (the dimension of each single-channel convolution kernel is 1*h1*w1). Based on this, the data of each channel of the input tensor can be convolved with the corresponding single-channel convolution kernel to obtain the data of the corresponding channel of the output tensor, where the dimension of the output data of each channel is 1*H'*W'.

[0033] Depthwise separable convolution has become one of the commonly used convolution operations in convolutional neural networks by reducing the number of parameters, improving computational efficiency, and improving model performance. Therefore, in order to improve the performance of artificial intelligence chips, it is very important to optimize operators related to depthwise separable convolution.

[0034] In an embodiment of the present disclosure, the performance of the back propagation calculation of the depth-separable two-dimensional convolution operator can be optimized, that is, the process of obtaining the input tensor based on the output tensor is optimized. The back propagation calculation of the depth-separable convolution is a computationally intensive task, requiring efficient algorithms and data processing methods to accelerate the calculation process. For example, the back propagation calculation method for the depth-separable two-dimensional convolution operator can be designed to output the input tensor dx with the output tensor dy and the weight tensor w as input, wherein the dimension of the output tensor dy is N*C*P*Q, wherein N represents the batch size, C represents the number of channels, P and Q represent the height and width respectively, and the dimension of the weight tensor w is K*1*R*S, wherein K represents the dimension of the output channel, and for the depth-separable two-dimensional convolution operator, the dimension of the input channel of the weight tensor w is 1, R and S represent the height and width respectively, and the dimension of the input tensor dx is N*C*H*W, wherein N represents the batch size, C represents the number of channels, and H and W represent the height and width respectively.

[0035] Therefore, based on the output tensor dy and the weight tensor w, in the existing back propagation calculation method for the depthwise separable two-dimensional convolution operator, a point of the input tensor (i.e., dx[n][c][h][w]) is usually calculated separately in each thread of the CPU or GPU. The point is first initialized to a zero value, and then the data related to the point in the output tensor dy is determined by calculation, and the point of the input tensor is calculated based on the accumulation of the product of the data of dy and the corresponding data in the weight tensor.

[0036] In the above method, when calculating a point in the input tensor, since each thread performs the above calculation for a point, each thread needs to load and store a copy of the output tensor dy data separately, and the repeated loading of the output tensor dy data from the off-chip memory seriously affects the performance of the operator. In addition, since it is necessary to determine the data related to the point in the output tensor dy by calculation, for example, the indexes p and q of the height and width dimensions, the calculated indexes p and q may exceed the original value range, so it is necessary to perform conditional judgment on the coordinates of each point of the output tensor data, that is, to determine whether the coordinates are within the range of the output tensor. This conditional judgment operation is not friendly to GPU parallel computing, and it will increase the time consumption of thread synchronization while increasing the operation instructions.

[0037] Based on this, the present disclosure provides a data processing method, which optimizes the performance of the back propagation calculation of the depthwise separable two-dimensional convolution operator, by pre-loading part of the data of the output tensor from the off-chip memory into the group shared memory each time, and pre-loading the data of the weight matrix from the off-chip memory into the thread private register, to use the data of the output tensor loaded into the group shared memory and the data of the weight matrix to calculate the corresponding data in the input tensor, so as to obtain the input tensor through back propagation based on the output tensor and the weight matrix, wherein the high memory access bandwidth of the group shared memory is used to cache data, so as to improve the memory access efficiency of the data and improve the performance of the back propagation calculation of the depthwise separable two-dimensional convolution operator.

[0038] Figure 4 is a flow chart illustrating a data processing method 400 according to an embodiment of the present disclosure. Figure 5 is a schematic diagram illustrating a data processing method according to an embodiment of the present disclosure. Figure 6 It is a schematic diagram showing data processing based on a convolutional segmentation strategy according to an embodiment of the present disclosure.

[0039] In step S401, a portion of the data in the output tensor can be loaded from the off-chip memory into the group shared memory, and the weight tensor can be loaded from the off-chip memory into the thread private register, wherein the dimension of the input channel of the weight tensor is one-dimensional.

[0040] In an embodiment of the present disclosure, the data in the output tensor can be loaded from the off-chip memory into the group shared memory (Group Shared Memory, GSM) in batches, and the back propagation calculation is performed by reading the data of the output tensor from the group shared memory, so as to improve the memory access efficiency by using the group shared memory. As a shared memory mechanism, the threads in the group can work together, share data, and use resources such as shared memory for collaborative computing, thereby improving the collaborative computing efficiency between threads, reducing data redundancy, and improving computing performance.

[0041] like Figure 5 As shown, for the data of the output tensor stored in the off-chip memory (eg, high bandwidth memory HBM), part of the data dy therein can be loaded (501) into the group shared memory for calculation.

[0042] Optionally, the amount of data of the output tensor loaded from the off-chip memory each time can be determined based on the available hardware resources. For example, the amount of data of the output tensor loaded from the off-chip memory each time can be determined based on the bandwidth of the off-chip memory, hardware resource limitations (such as the size, speed, cache size, etc. of the off-chip memory), the memory capacity of the computing device, and the requirements of the computing task to ensure the efficiency and stability of data loading. The appropriate amount of data loading can be determined based on the specific application scenario to meet the requirements of the computing task and make full use of the hardware resources. The present disclosure does not limit the amount of data loaded each time.

[0043] Optionally, since the dimension of the input channel of the weight tensor is one-dimensional for the depthwise separable two-dimensional convolution operator, the amount of data of the weight tensor is small in this case. Therefore, before performing the convolution operation, the data of the weight tensor for the depthwise separable two-dimensional convolution operator can be pre-loaded from the off-chip memory into the thread-private register (Thread Local Register, TLR). As a register unique to each thread, the thread-private register can be used to store thread-private data and temporary variables. In multi-threaded processing, each thread has its own TLR, which can increase data access speed and reduce competition conditions, thereby improving computing performance and concurrency.

[0044] Therefore, by preloading the weight tensor data into the thread-private register, the number of accesses to the external memory can be reduced, the delay of data transmission can be reduced, thereby speeding up the execution of the algorithm and improving the efficiency and speed of calculation. In addition, since the thread-private registers usually have faster access speeds and lower delays, by reading data directly from the registers during the calculation process, the convolution operation can be completed faster, improving the overall computing performance.

[0045] In an embodiment of the present disclosure, in order to improve computing performance, the calculation based on the data of the output tensor and the data of the weight tensor can be split into multiple threads for parallel execution. Optionally, the data of the weight tensor can be loaded into the corresponding thread-private registers of different threads by slicing to perform the calculation of the corresponding weight tensor data on different threads, and the threads can share the data in the group shared memory. Therefore, on each thread, the corresponding input tensor calculation can be performed based on the weight tensor data read from the corresponding thread-private register and the output tensor data read from the group shared memory.

[0046] Optionally, for the back propagation calculation of the depth-separable two-dimensional convolution operator, when the stride of the convolution operation is 2, the stride of each movement of the convolution kernel is two elements, resulting in the convolution operation skipping one element in the horizontal and vertical directions, which will result in a reduction in the size of the output feature map, and "holes" (or "hole convolutions") will appear at certain positions of the input feature map, i.e., elements that are not considered by the convolution kernel. Therefore, in order to solve this problem, a convolution splitting strategy can be used to split the convolution operation. Optionally, the convolution calculation based on the data of the output tensor and the data of the weight tensor can be split into multiple threads for parallel execution, and the corresponding data of the input tensor can be obtained by merging the calculation results of each thread, thereby avoiding the loading and calculation hole problems that occur when the stride is 2, while improving the calculation efficiency and accuracy.

[0047] By preloading some output tensor data into group shared memory, the number of accesses to off-chip memory can be reduced, thereby improving data access efficiency. At the same time, loading weight matrix data into thread private registers can reduce the movement of data during the calculation process, further improving calculation efficiency. This method takes advantage of the high memory access bandwidth of group shared memory and effectively caches data, thereby improving the performance of backpropagation calculations.

[0048] Therefore, by preloading data into the group shared memory and the thread private register, the back propagation computing performance of the depth-separable two-dimensional convolution operator can be improved by using an efficient data access method. After loading part of the data of the output tensor from the off-chip memory into the group shared memory and loading the weight tensor from the off-chip memory into the thread private register, in step S402, the corresponding data in the input tensor can be determined based on the part of the data loaded from the group shared memory and the weight tensor loaded from the thread private register.

[0049] Optionally, when performing back-propagation calculations of a depth-separable two-dimensional convolution operator, in each thread, the input tensor data corresponding to each weight tensor data part can be calculated based on the weight tensor data in the corresponding thread private register and the output tensor data shared between the threads, and the corresponding data of the input tensor can be obtained by merging the calculation results of each thread.

[0050] According to an embodiment of the present disclosure, determining the corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register may include: dividing the weight tensor into multiple sub-weight tensors according to the odd-even row and column arrangement of the weight tensor; determining the corresponding data in the input tensor based on the portion of data and the multiple sub-weight tensors.

[0051] Optionally, the data of the weight tensor may be divided into corresponding thread-private registers of different threads according to the parity row and column arrangement of the weight tensor in a parity row and column splitting manner. Specifically, dividing the weight tensor into a plurality of sub-weight tensors according to the parity row and column arrangement of the weight tensor may include determining the parity of the row and column of each element in the weight tensor according to the position of the element in the row and column of the weight tensor, and grouping the elements in the weight tensor based on the row and column parity of each element.

[0052] Alternatively, if Figure 5 As shown, the elements in the weight tensor can be divided into four groups according to the parity of rows and columns, namely ee (rows and columns are even, even-even), eo (rows are even and columns are odd, even-odd), oe (rows are odd and columns are even, odd-even), and oo (rows and columns are odd, odd-odd). The elements of these four groups can each constitute a sub-weight tensor. Therefore, based on the parity of the rows and columns of each element, the corresponding four sub-weight tensors can be determined. For example Figure 5 For example, Figure 6As shown, for a weight tensor 620 of size 3x3, where both the row index and the column index are 0-2, it can be parity split (601) according to its parity row and column arrangement to obtain four corresponding sub-weight tensors 630, where the elements split into the same group are represented by the same group code. For example, for the weight tensor 620, the elements with column indexes 0 and 2 in the rows with row indexes 0 and 2 belong to the elements of the ee group, that is, the elements with even rows and columns, so they are all shown using the group code ee. These four elements constitute a sub-weight tensor w_ee corresponding to the ee group, and the other sub-weight tensors w_eo, w_oe, and w_oo can be determined similarly. Therefore, for the weight tensor w (520) on the off-chip memory, it can be loaded (502) from the off-chip memory into the thread private register, and it can be split to obtain multiple sub-weight tensors (540).

[0053] According to an embodiment of the present disclosure, determining the corresponding data in the input tensor based on the multiple sub-weight tensors and the portion of data may include: performing convolution calculations on each of the multiple sub-weight tensors with the portion of data; and rearranging the results of the convolution calculations in the group shared memory according to the odd-even row and column arrangement of the weight tensor to determine the corresponding data in the input tensor.

[0054] Optionally, the data of the output tensor (i.e., part of the data mentioned above) can be read from the group shared memory and convolved with each sub-weight tensor in the thread private register, and the result of the convolution calculation can be written to the group shared memory to rearrange the results of each convolution calculation according to the division method of the weight tensor, thereby determining the corresponding data in the input tensor. For example, Figure 5 As shown, the output tensor data dy in the GSM can be convolved (conv) calculated (503) with the sub-weight tensors w_ee, w_eo, w_oe, and w_oo in each TLR, respectively, to obtain the convolution calculation results 550, namely dx_ee, dx_eo, dx_oe, and dx_oo. These convolution calculation results can be written into the GSM to obtain the corresponding data dx in the final input tensor by merging (merge) (504).

[0055] According to an embodiment of the present disclosure, determining the corresponding data in the input tensor based on the multiple sub-weight tensors and the portion of data may include: for each sub-weight tensor in the multiple sub-weight tensors, performing a convolution calculation on the sub-weight tensor and the portion of data, and determining a temporary input tensor corresponding to the sub-weight tensor; filling the temporary input tensor corresponding to each sub-weight tensor with data based on the step size of the output tensor obtained from the input tensor based on the weight tensor; and rearranging the data-filled temporary input tensor in the group shared memory according to the even-odd row and column arrangement of the weight tensor to determine the corresponding data in the input tensor.

[0056] For the result of convolving each sub-weight tensor with the output tensor, since the dimensions of the sub-weight tensors may be different, the temporary input tensors generated corresponding to the sub-weight tensors may also have different dimensions. Therefore, optionally, the dimensions of each temporary input tensor can be unified by data padding, for example, it can be unified to the same dimension as the above-mentioned part of the data in the output tensor.

[0057] Optionally, the temporary input tensor can be padded based on the stride of the forward propagation of the depthwise separable 2D convolution operator. Figure 6 As shown in 602, when the forward propagation step length stride=2, each sub-weight tensor 630 can be convolved with the output tensor 610 to obtain temporary input tensors with different dimensions, and the temporary input tensors with different dimensions are data-filled, that is, corresponding rows or columns are inserted between any adjacent rows or columns and filled with zero values ​​to obtain data-filled temporary input tensors 640. These data-filled temporary input tensors all have the same dimension as the output tensor 610, for example, the dimension of the data-filled temporary input tensor is 4x4.

[0058] Based on this, optionally, each data-filled temporary input tensor can be rearranged according to the parity row and column arrangement of the weight tensor (603) to obtain the final input tensor data 650. For example, the data of each temporary input tensor can be fused according to the parity of the row and column of the sub-weight tensor (e.g., w_ee, w_eo, w_oe, w_oo) corresponding to each temporary input tensor, thereby generating the input tensor data.

[0059] According to an embodiment of the present disclosure, the data processing method of the present disclosure may further include: writing out corresponding data in the determined output tensor from the group shared memory to the off-chip memory.

[0060] As described above, through steps S401-S402, the corresponding partial data in the input tensor can be generated based on the partial data in the output tensor and the weight tensor data. Optionally, the corresponding data in the final input tensor obtained by the above processing can be loaded into an off-chip memory, for example, Figure 5 As shown, the data dx in the input tensor can be written into the HBM.

[0061] Based on the above data processing method disclosed in the present invention, the performance of the back propagation calculation of the deep separable two-dimensional convolution operator can be optimized. By pre-loading part of the data of the output tensor from the off-chip memory to the group shared memory each time, and pre-loading the data of the weight matrix from the off-chip memory to the thread private register, the data of the output tensor loaded into the group shared memory and the data of the weight matrix are used to calculate the corresponding data in the input tensor, thereby obtaining the input tensor through back propagation based on the output tensor and the weight matrix, wherein the high memory access bandwidth of the group shared memory is used to cache data, which can improve the memory access efficiency of the data and improve the performance of the back propagation calculation of the deep separable two-dimensional convolution operator. In addition, by adopting a convolution splitting strategy to perform the convolution calculation between the output tensor and the weight tensor, the cache pressure of the group shared memory can be reduced, thereby further improving the performance of the back propagation calculation of the deep separable two-dimensional convolution operator.

[0062] Figure 7 is a schematic diagram showing a data processing device 700 according to an embodiment of the present disclosure.

[0063] According to an embodiment of the present disclosure, the data processing device 700 may include a data loading module 701 and a data calculation module 702 .

[0064] The data loading module 701 may be configured to load a portion of the data in the output tensor from the off-chip memory into the group shared memory, and load the weight tensor from the off-chip memory into the thread private register, wherein the dimension of the input channel of the weight tensor is one-dimensional. Optionally, the data loading module 701 may perform the operation described above with reference to step S401.

[0065] Optionally, the data in the output tensor can be loaded from the off-chip memory into the group shared memory (GSM) in batches, and the back propagation calculation is performed by reading the data of the output tensor from the group shared memory to improve the memory access efficiency by utilizing the group shared memory.

[0066] Optionally, the amount of data of the output tensor loaded from the off-chip memory each time can be determined based on the available hardware resources. For example, the amount of data of the output tensor loaded from the off-chip memory each time can be determined based on the bandwidth of the off-chip memory, hardware resource limitations (such as the size, speed, cache size, etc. of the off-chip memory), the memory capacity of the computing device, and the requirements of the computing task to ensure the efficiency and stability of data loading. The appropriate amount of data loading can be determined based on the specific application scenario to meet the requirements of the computing task and make full use of the hardware resources. The present disclosure does not limit the amount of data loaded each time.

[0067] Optionally, since for the depthwise separable two-dimensional convolution operator, the dimension of the input channel of the weight tensor is one-dimensional, the amount of data of the weight tensor is small in this case. Therefore, before performing the convolution operation, the data of the weight tensor for the depthwise separable two-dimensional convolution operator can be pre-loaded from the off-chip memory into the thread-private register.

[0068] By preloading the weight tensor data into the thread-private register, the number of accesses to the external memory can be reduced, and the delay of data transmission can be reduced, thereby speeding up the execution of the algorithm and improving the efficiency and speed of calculation. In addition, since the thread-private registers usually have faster access speeds and lower delays, by reading data directly from the registers during the calculation process, the convolution operation can be completed faster, improving the overall computing performance.

[0069] By preloading some output tensor data into group shared memory, the number of accesses to off-chip memory can be reduced, thereby improving data access efficiency. At the same time, loading weight matrix data into thread private registers can reduce the movement of data during the calculation process, further improving calculation efficiency. This method takes advantage of the high memory access bandwidth of group shared memory and effectively caches data, thereby improving the performance of backpropagation calculations.

[0070] Therefore, by pre-loading data into group shared memory and thread private registers, efficient data access can be used to improve the back-propagation computation performance of the depth-wise separable 2D convolution operator.

[0071] The data calculation module 702 may be configured to determine the corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register. Optionally, the data calculation module 702 may perform the operations described above with reference to step S402.

[0072] Optionally, when performing back-propagation calculations of a depth-wise separable two-dimensional convolution operator, input tensor data corresponding to each weight tensor data part may be calculated based on the weight tensor data in the thread-private register and the output tensor data in the group shared memory, and the corresponding data of the input tensor may be obtained by merging the calculation results.

[0073] Optionally, the data of the weight tensor may be segmented according to the even-odd row and column arrangement of the weight tensor in an even-odd row and column segmentation manner. Specifically, segmenting the weight tensor into a plurality of sub-weight tensors according to the even-odd row and column arrangement of the weight tensor may include determining the parity of the row and column of each element in the weight tensor according to the position of the element in the row and column of the weight tensor, and grouping the elements in the weight tensor based on the parity of the row and column of each element.

[0074] For example, the data of the output tensor (i.e., part of the data mentioned above) can be read from the group shared memory and convolved with each sub-weight tensor in the thread private register, and the result of the convolution calculation can be written to the group shared memory to rearrange the results of each convolution calculation according to the division method of the weight tensor, thereby determining the corresponding data in the input tensor.

[0075] Optionally, the dimensions of each temporary input tensor may be unified by data padding, for example, they may be unified to the same dimension as the above-mentioned portion of data in the output tensor.

[0076] For example, the temporary input tensor may be data-filled based on the forward propagation step size of the depthwise separable two-dimensional convolution operator. For example, each sub-weight tensor may be convolved with the output tensor to obtain a temporary input tensor with different dimensions, and the temporary input tensor with different dimensions may be data-filled, that is, a corresponding row or column may be inserted between any adjacent rows or columns thereof and filled with zero values ​​to obtain a data-filled temporary input tensor, and these data-filled temporary input tensors all have the same dimension as the output tensor.

[0077] Based on this, each data-filled temporary input tensor can be rearranged according to the parity of the weight tensor to obtain the final input tensor data. For example, the data of each temporary input tensor can be fused according to the parity of the rows and columns of the sub-weight tensors corresponding to each temporary input tensor, thereby generating the input tensor data.

[0078] As described above, the data calculation module 702 can generate corresponding partial data in the input tensor based on partial data in the output tensor and weight tensor data. Therefore, the corresponding data in the final input tensor obtained by the above processing can be loaded into the off-chip memory, for example, the data dx in the input tensor can be written into the HBM.

[0079] According to yet another aspect of the present disclosure, a data processing device is provided. Figure 8 A schematic diagram of a data processing device 2000 according to an embodiment of the present disclosure is shown.

[0080] like Figure 8 As shown, the data processing device 2000 may include one or more processors 2010 and one or more memories 2020. The memory 2020 stores a computer readable code, and when the computer readable code is run by the one or more processors 2010, the data processing method described above may be executed.

[0081] The processor in the embodiments of the present disclosure may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc., and may be an X86 architecture or an ARM architecture.

[0082] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0083] For example, the method or device according to the embodiment of the present disclosure may also be implemented by Fig. 9 The architecture of the computing device 3000 shown in FIG. Fig. 9As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the data processing method provided in the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Fig. 9 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Fig. 9 One or more components of a computing device are shown.

[0084] According to another aspect of the present disclosure, a computer-readable storage medium is also provided. Computer-readable instructions are stored on the computer storage medium. When the computer-readable instructions are executed by a processor, the data processing method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium in the embodiment of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronously linked dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0085] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data processing method according to the embodiments of the present disclosure.

[0086] Embodiments of the present disclosure provide a data processing method, apparatus, device, and computer-readable storage medium.

[0087] The method provided by the embodiment of the present disclosure optimizes the performance of the back propagation calculation of the depth-separable two-dimensional convolution operator, by pre-loading part of the data of the output tensor from the off-chip memory into the group shared memory each time, and pre-loading the data of the weight matrix from the off-chip memory into the thread private register, so as to use the data of the output tensor loaded into the group shared memory and the data of the weight matrix to calculate the corresponding data in the input tensor, thereby obtaining the input tensor through back propagation based on the output tensor and the weight matrix, wherein, by utilizing the high memory access bandwidth of the group shared memory to cache data, the memory access efficiency of the data can be improved, and the performance of the back propagation calculation of the depth-separable two-dimensional convolution operator can be improved. In addition, by adopting a convolution splitting strategy to perform the convolution calculation between the output tensor and the weight tensor, the cache pressure of the group shared memory can be reduced, thereby further improving the performance of the back propagation calculation of the depth-separable two-dimensional convolution operator.

[0088] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0089] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0090] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A data processing method for obtaining an input tensor by back propagation based on an output tensor, the method comprising: Loading a portion of the data in the output tensor from the off-chip memory into the group shared memory, and loading the weight tensor from the off-chip memory into the thread private register, wherein the dimension of the input channel of the weight tensor is one-dimensional; Determining corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register; Wherein, determining the corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register includes: Splitting the weight tensor into a plurality of sub-weight tensors according to the even-odd row and column arrangement of the weight tensor; Based on the portion of data and the multiple sub-weight tensors, corresponding data in the input tensor is determined.

2. The method of claim 1, wherein: Based on the portion of data and the plurality of sub-weight tensors, determining corresponding data in the input tensor comprises: Perform convolution calculation on each of the plurality of sub-weight tensors and the portion of data respectively; The results of the convolution calculation are rearranged in the group shared memory according to the even-odd row and column arrangement of the weight tensor to determine corresponding data in the input tensor.

3. The method of claim 1, wherein: Based on the portion of data and the plurality of sub-weight tensors, determining corresponding data in the input tensor comprises: For each sub-weight tensor of the plurality of sub-weight tensors, convolution calculation is performed on the sub-weight tensor and the portion of data, and a temporary input tensor corresponding to the sub-weight tensor is determined; Filling data on a temporary input tensor corresponding to each sub-weight tensor based on a step size of obtaining the output tensor from the input tensor based on the weight tensor; According to the even-odd row and column arrangement of the weight tensor, the data-filled temporary input tensor is rearranged in the group shared memory to determine the corresponding data in the input tensor.

4. The method of claim 1, further comprising: Writing corresponding data in the determined output tensor from the group shared memory to the off-chip memory.

5. A data processing device for obtaining an input tensor by back propagation based on an output tensor, the device comprising: A data loading module is configured to load a portion of the data in the output tensor from the off-chip memory into the group shared memory, and load the weight tensor from the off-chip memory into the thread private register, wherein the dimension of the input channel of the weight tensor is one-dimensional; a data calculation module configured to determine corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register; Wherein, determining the corresponding data in the input tensor based on the portion of data loaded from the group shared memory and the weight tensor loaded from the thread private register includes: Splitting the weight tensor into a plurality of sub-weight tensors according to the even-odd row and column arrangement of the weight tensor; Based on the portion of data and the multiple sub-weight tensors, corresponding data in the input tensor is determined.

6. A data processing device comprising: one or more processors; as well as One or more memories, wherein a computer executable program is stored, and when the computer executable program is executed by the processor, the method according to any one of claims 1 to 4 is performed.

7. A computer program product, stored on a computer-readable storage medium, and comprising computer instructions, which, when executed by a processor, cause a computer device to perform the method of any one of claims 1 to 4.

8. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the instructions are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

Images