Data processing method, device, equipment and storage medium

Abstract

The embodiments of the present disclosure provide a data processing method, apparatus, device and computer-readable storage medium. The method provided by the embodiments of the present disclosure divides the input tensor into multiple data blocks based on the minimum alignment granularity of GPGPU, and converts the input tensor into a one-dimensional tensor based on the divided multiple data blocks, so as to decompose the computing task of the input tensor in the dimension of the one-dimensional tensor, thereby decomposing the computing task of the input tensor into multiple independent subtasks that can be executed in parallel on the hardware resources of the GPU. The method of the embodiments of the present disclosure can improve the computing parallelism of GPGPU by performing task decomposition in the dimension of the one-dimensional tensor, achieve higher hardware resource utilization, and thus improve the overall computing performance and efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and more specifically, to a data processing method, apparatus, device and storage medium. Background Art

[0002] In recent years, the computing power of graphics processing units (GPUs) has been significantly improved, making general-purpose computing on GPUs more and more common. General-purpose graphics processing units (GPGPUs) are a technology that uses GPUs for general-purpose computing. The computing core of GPGPU can execute multiple threads simultaneously, thereby achieving efficient parallel computing, which makes GPGPU advantageous when processing large-scale data and intensive computing tasks. GPGPU has a wide range of applications, including scientific computing, data analysis, machine learning, deep learning, cryptography, etc. By utilizing the parallel computing power of GPGPU, the execution speed of these tasks can be accelerated and computing efficiency can be improved.

[0003] When executing computing tasks on GPGPU, the parallel computing capability of GPGPU can be used to decompose the computing tasks into multiple independent subtasks, and these subtasks can be executed on GPGPU simultaneously to improve the overall computing performance and efficiency. Therefore, the allocation of computing tasks to hardware resources in GPGPU becomes a key issue, which involves how to reasonably allocate computing tasks to different hardware units to achieve efficient parallel computing.

[0004] Therefore, a more efficient data processing method is needed to improve the computational parallelism of GPGPU, thereby improving the overall computing performance and efficiency. Summary of the invention

[0005] In order to solve the above problems, the present invention folds the input tensor into a one-dimensional tensor based on the minimum alignment granularity of GPGPU, and performs task allocation to the hardware resources of GPGPU for the one-dimensional tensor, thereby improving the utilization of hardware resources, thereby improving the computational parallelism of GPGPU and the overall computational performance and efficiency.

[0006] Embodiments of the present disclosure provide a data processing method, apparatus, device, and computer-readable storage medium.

[0007] An embodiment of the present disclosure provides a data processing method, including: obtaining an input tensor; dividing the input tensor into multiple data blocks based on the minimum alignment granularity of the GPGPU, and converting the input tensor into a one-dimensional tensor, wherein each element in the one-dimensional tensor corresponds to a data block, and the minimum alignment granularity is used to indicate that the starting address of the storage address of each data block in the memory of the GPGPU of the multiple data blocks is aligned based on the minimum alignment granularity; according to the computing task for the input tensor, determining the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor; and executing the computing task for the input tensor based on the task allocation of the computing task on the hardware resources of the GPGPU.

[0008] An embodiment of the present disclosure provides a data processing device, including: a data acquisition module, configured to acquire an input tensor; a tensor folding module, configured to divide the input tensor into multiple data blocks based on the minimum alignment granularity of the GPGPU, and convert the input tensor into a one-dimensional tensor, wherein each element in the one-dimensional tensor corresponds to a data block, and the minimum alignment granularity is used to indicate that the starting address of the storage address of each data block in the memory of the GPGPU of the multiple data blocks is aligned based on the minimum alignment granularity; a task decomposition module, configured to determine the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor according to the computing task for the input tensor; and a task execution module, configured to execute the computing task for the input tensor based on the task allocation of the computing task on the hardware resources of the GPGPU.

[0009] An embodiment of the present disclosure provides a data processing device, including: one or more processors; and one or more memories, wherein a computer executable program is stored in the one or more memories, and when the computer executable program is executed by the processor, the data processing method as described above is performed.

[0010] An embodiment of the present disclosure provides a computer-readable storage medium on which computer-executable instructions are stored. When the instructions are executed by a processor, the instructions are used to implement the data processing method as described above.

[0011] The embodiments of the present disclosure provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method according to the embodiments of the present disclosure.

[0012] The method provided by the embodiment of the present disclosure divides the input tensor into multiple data blocks based on the minimum alignment granularity of GPGPU, and converts the input tensor into a one-dimensional tensor based on the divided multiple data blocks, so as to decompose the computing task of the input tensor in the dimension of the one-dimensional tensor, thereby decomposing the computing task of the input tensor into multiple independent subtasks that can be executed in parallel on the hardware resources of the GPU. The method of the embodiment of the present disclosure can improve the computing parallelism of GPGPU by performing task decomposition in the dimension of the one-dimensional tensor, achieve higher hardware resource utilization, and thus improve the overall computing performance and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0014] Figure 1 is a schematic diagram showing task decomposition in three dimensions according to an embodiment of the present disclosure;

[0015] Figure 2 is a flow chart showing a data processing method according to an embodiment of the present disclosure;

[0016] Figure 3 is a schematic diagram illustrating a data processing method according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic diagram showing a first example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure;

[0018] Figure 5A is a schematic diagram showing a second example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure;

[0019] Figure 5B is a schematic diagram showing task allocation to hardware resources of a GPGPU based on a second example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure;

[0020] Fig. 6A is a schematic diagram showing a third example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure;

[0021] Figure 6B is a schematic diagram showing task allocation to hardware resources of a GPGPU based on a third example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure;

[0022] Figure 7 is a schematic diagram showing a data processing device according to an embodiment of the present disclosure;

[0023] Figure 8 A schematic diagram showing a data processing device according to an embodiment of the present disclosure; and

[0024] Fig. 9 A schematic diagram illustrating the architecture of an exemplary computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0026] In this specification and the accompanying drawings, substantially the same or similar steps and elements are represented by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance or ranking.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure pertains. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0028] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.

[0029] The data processing method disclosed in the present invention can be based on GPGPU computing. GPGPU computing uses the parallel computing power of GPGPU to accelerate various computing-intensive tasks. The basic principle of GPGPU computing is to decompose the computing task into multiple parallel subtasks, and then assign these subtasks to the execution structure on the GPGPU for parallel execution. There are dense execution structures on the GPGPU chip, and these execution structures can form larger execution structures according to certain granularity and rules. The hierarchical structure of the execution structure on the GPGPU is described below. As an example, in GPGPU, the execution structure for processing parallel computing tasks may include, for example, threads, execution units (EU), computing units (CU), and stream processor cores (SPC). Among them, threads, as the most basic execution structure, can be responsible for executing an instruction or computing task. In GPGPU, a large number of threads are executed simultaneously to achieve a high degree of parallel computing. The execution unit EU can be composed of multiple threads, and each EU can execute the instructions of multiple threads to, for example, execute a set of related computing tasks, such as vector operations, scalar operations, etc. CU, as an execution structure at a higher level than EU, can be composed of multiple EUs. CU can execute multiple parallel computing tasks at the same time, thereby improving the overall computing throughput. As the basic computing unit in the GPGPU chip, SPC can be responsible for executing parallel computing tasks, which can usually be composed of multiple CUs. By adopting an appropriate execution structure organization method, higher parallel computing capabilities and flexibility can be provided.

[0030] In summary, the solutions provided by the embodiments of the present disclosure involve technologies such as GPGPU computing, and the embodiments of the present disclosure will be further described below in conjunction with the accompanying drawings.

[0031] As a highly parallel processor, GPGPU has a large number of computing cores and memory bandwidth, which is suitable for executing parallel computing tasks. Task decomposition on GPGPU is one of the key steps to achieve parallel computing.

[0032] In some existing methods for performing task decomposition on GPGPU, when assigning tasks to an execution structure, the execution structure is usually divided into multiple SPCs, EUs, and threads to process tensors that need to be processed in parallel. Figure 1 FIG. 1 is a schematic diagram showing task decomposition in three dimensions according to an embodiment of the present disclosure. Figure 1As shown in the figure, it is assumed that the tensor to be processed has three dimensions, namely N, H and W, where N represents the batch size, H represents the height, and W represents the width. In order to efficiently utilize the parallel computing capabilities of the GPU, these methods will map the execution structures SPC, EU and threads to these three dimensions for task decomposition.

[0033] For example, various execution structures can be mapped to various dimensions of the input tensor to perform task decomposition. For example, for a three-dimensional input tensor, different execution structures SPC, EU, and thread can be mapped to N, H, and W dimensions respectively to decompose the computing task into multiple subtasks (i.e., generate multiple subtask instructions) for execution by different execution structures, thereby achieving parallel computing.

[0034] However, there are two main problems in the above task decomposition method:

[0035] 1. Unbalanced task decomposition: Since the amount of tasks for input tensors in different dimensions may be different, the task decomposition may be unbalanced, that is, some execution structures may process more or less tasks than other execution structures, resulting in some execution structures being idle while other execution structures are overloaded, which in turn causes a waste of hardware resources and may also reduce the overall computing efficiency.

[0036] 2. Low memory access efficiency: Due to different data access patterns in different dimensions, for example, for the above three-dimensional tensor, if different execution structures are mapped to different dimensions, the memory address of the data accessed during the calculation process will frequently change significantly, resulting in low memory access efficiency and cache hits, reducing the calculation efficiency. In addition, under this task decomposition method, the memory bandwidth is difficult to be fully utilized, which also limits the speed of calculation.

[0037] In order to solve these problems, it is necessary to optimize the task decomposition method to improve the computing efficiency and the video memory access efficiency. In the embodiments of the present disclosure, the task decomposition can be performed according to the specific situation. For example, the appropriate decomposition strategy can be determined according to the characteristics of the hardware resources and the tasks to ensure balanced distribution of tasks and maximize the use of the video memory bandwidth. This requires comprehensive consideration of factors such as the size of the input tensor, the hardware architecture, and the parallelism of the tasks.

[0038] Based on this, the present disclosure provides a data processing method, in which the input tensor is folded into a one-dimensional tensor based on the minimum alignment granularity of GPGPU, and the task allocation of GPGPU hardware resources is performed for the one-dimensional tensor, thereby improving the hardware resource utilization, thereby improving the computational parallelism of GPGPU and the overall computational performance and efficiency.

[0039] The method provided by the embodiment of the present disclosure divides the input tensor into multiple data blocks based on the minimum alignment granularity of GPGPU, and converts the input tensor into a one-dimensional tensor based on the divided multiple data blocks, so as to decompose the computing task of the input tensor in the dimension of the one-dimensional tensor, thereby decomposing the computing task of the input tensor into multiple independent subtasks that can be executed in parallel on the hardware resources of the GPU. The method of the embodiment of the present disclosure can improve the computing parallelism of GPGPU by performing task decomposition in the dimension of the one-dimensional tensor, achieve higher hardware resource utilization, and thus improve the overall computing performance and efficiency.

[0040] Figure 2 is a flow chart illustrating a data processing method 200 according to an embodiment of the present disclosure. Figure 3 is a schematic diagram illustrating a data processing method according to an embodiment of the present disclosure.

[0041] Optionally, the data processing method of the present disclosure may be applicable to, for example, the development scenario of operators on GPGPU, where operators may be operators or functions used to perform specific computing tasks, including but not limited to mathematical operations, logical operations, bit operations, image processing, data compression, sorting, etc. The computing tasks of these operators usually require a large amount of parallel computing and data processing, and the parallel computing capabilities and high-bandwidth memory of GPGPU can significantly improve the execution efficiency and performance of operators. Therefore, parallel computing of operators on GPGPU can accelerate the computing process and improve the overall computing performance. For the development scenario of operators on GPGPU, it is necessary to reasonably allocate tasks to the execution units on GPGPU to improve the parallelism of operators. That is to say, in the embodiments of the present disclosure, the computing tasks of the operators can be decomposed into multiple independent sub-computing tasks, and these sub-computing tasks can be executed on GPGPU at the same time to make full use of the parallel computing capabilities of GPGPU and improve the overall computing performance and efficiency. Of course, it should be understood that the above-mentioned scenarios are only used as examples and not limitations in the present disclosure, and the data processing method of the present disclosure can also be applied to various other scenarios.

[0042] In step S201 , an input tensor may be obtained.

[0043] Optionally, a tensor can be obtained as an input to the data processing method of the present invention, that is, as an input to a computing task. An input tensor may refer to a multidimensional array data that needs to be calculated on a GPGPU, which may be various types of data, such as images, audio, text, or any other type of data. The dimension, shape, and data type of the input tensor may depend on the specific application scenario and task requirements. For example, for image processing tasks, the input tensor may typically have three or four dimensions, representing the height, width, number of channels, and batch size of the image, respectively. For audio processing tasks, the input tensor may typically have two or three dimensions, representing the number of sampling points, number of channels, and batch size of the audio, respectively. For text processing tasks, the input tensor may typically have two dimensions, representing the length of the text and the batch size, respectively.

[0044] Optionally, the data of the input tensor can be loaded and transferred to the GPGPU in a variety of ways. For example, common ways may include copying data from the host memory to the GPGPU memory, or directly allocating and initializing data in the GPGPU memory. After the calculation, the results can be copied from the GPGPU memory back to the host memory for further processing and analysis.

[0045] Specifically, as an example, the input tensor to be processed can be loaded from the host memory into the GPGPU memory, and the input tensor can be assigned to the execution unit for calculation. In this way, the execution unit can read the input data directly from the GPGPU memory and calculate the input tensor in parallel. For example, in the presence of multiple execution units, each execution unit can be responsible for processing different parts of the input tensor data to make full use of the parallel computing capabilities of the GPGPU and speed up the calculation. Therefore, after the calculation is completed in this way, the execution unit can write the calculation result into the output tensor, which is the output of the computing task. Therefore, by loading the input tensor into the GPGPU memory and assigning the input tensor to the execution unit for calculation, the overhead of data transmission and processing can be minimized and the computing efficiency can be improved, which is an important step in performing high-performance computing on the GPGPU.

[0046] In step S202, the input tensor can be divided into multiple data blocks based on the minimum alignment granularity of the GPGPU, and the input tensor can be converted into a one-dimensional tensor, wherein each element in the one-dimensional tensor corresponds to a data block, and the minimum alignment granularity is used to indicate that the starting address of the storage address of each data block in the memory of the GPGPU of the multiple data blocks is aligned based on the minimum alignment granularity.

[0047] Optionally, in an embodiment of the present disclosure, the dimension of the input tensor may be folded according to the minimum alignment granularity defined by the hardware layout of the GPGPU to convert the input tensor into a one-dimensional tensor, wherein the minimum alignment granularity defined by the hardware layout of the GPGPU may refer to the need for data to be stored in a certain byte alignment when accessed in the memory of the GPGPU to improve memory access efficiency. Specifically, the multi-dimensional input tensor may be reorganized into a one-dimensional array and memory aligned according to the hardware layout requirements, which defines how the dimension of the input tensor is mapped to the storage method in the memory of the GPGPU.

[0048] Therefore, the input tensor can be converted into a one-dimensional tensor according to the storage method of the data of the input tensor in the memory of the GPGPU. In an embodiment of the present disclosure, the input tensor can be divided into multiple data blocks according to the minimum alignment granularity defined by the hardware layout of the GPGPU, wherein each data block is aligned according to a certain byte alignment method. Among them, alignment refers to the alignment of the data stored in the memory according to certain rules. For example, when the above-mentioned minimum alignment granularity is 4 bytes, the data block will be stored with 4-byte boundary alignment, that is, the starting address of the data block must be a multiple of 4.

[0049] As an example, in an embodiment of the present disclosure, Figure 3 As shown, the input tensor can be converted into a one-dimensional tensor by performing data block partitioning 301 and tensor folding 302 on the input tensor.

[0050] For example, Figure 3 As shown, for a three-dimensional input tensor, it is assumed that the three dimensions of the input tensor are N, H and W, where N represents the batch size, H represents the height, and W represents the width. For the convenience of description, the data block partitioning and tensor folding in the H and W dimensions and the subsequent task allocation are introduced as examples here, but it should be understood that these processes can also be similarly applied to other dimensions (for example, including but not limited to N or other additional dimensions).

[0051] Optionally, different data blocks divided from the input tensor can be stored in different memory areas respectively, and these data blocks can be stored according to a certain byte alignment. Optionally, the minimum alignment granularity of the GPGPU can indicate the alignment of each data block, for example, it can include the number of consecutively aligned data blocks and the amount of data included in each data block. For example, each data block can include 4×32 bits of data (i.e., 16 bytes of data), that is, data corresponding to 4 rows of the H dimension and 32 columns of the W dimension in the input tensor, wherein the minimum alignment granularity defined by the hardware layout of the GPGPU can require alignment of every 4 data blocks (e.g., 64 bytes). Therefore, in this case, the actual data on the H and W dimensions (shown as shaded rectangles) can be divided into multiple data blocks and aligned in a manner that aligns every 4 data blocks (and each data block includes 4×32 bits of data). As Figure 3 As shown, the actual data of the input tensor in the H and W dimensions includes less than 7 data blocks in the H dimension, and the data amount in the W dimension is less than 6 data blocks. Therefore, according to the minimum alignment granularity of GPGPU, such as 4 4×32-bit data block alignments, the data block division of the input tensor in the H and W dimensions can include division into 8 data blocks in the H dimension (the portion outside the actual data can be filled with a predefined filling value, such as a zero value or other specific values) and division into 6 data blocks in the W dimension (the portion outside the actual data can be filled with a predefined filling value).

[0052] Therefore, as described above, based on the minimum alignment granularity of GPGPU, the input tensor can be divided into multiple data blocks.

[0053] According to an embodiment of the present disclosure, for a plurality of memory regions in the memory of the GPGPU for storing the input tensor, data in a data block corresponding to an element in the one-dimensional tensor may be stored in one of the plurality of memory regions. Optionally, all data in a data block divided from the input tensor may be stored in the same memory region, so that the GPGPU has more continuous access to the memory.

[0054] According to an embodiment of the present disclosure, for different elements in the one-dimensional tensor, different data blocks corresponding to the different elements are stored in one or more memory areas of the multiple memory areas based on the positions of the different data blocks in the input tensor and a predetermined storage layout, and the predetermined storage layout indicates how to arrange data blocks on different dimensions of the input tensor in the memory of the GPGPU.

[0055] Optionally, different data blocks divided from the input tensor can be stored in different memory areas, and the way these different data blocks are stored can be based on their positions in the input tensor and the predetermined storage layout. For example, the predetermined storage layout can indicate the storage order of the data blocks on each dimension of the input tensor in the memory of the GPGPU, for example, the data blocks in the input tensor are stored layer by layer in the order of the innermost layer as H dimension, the second outermost layer as W dimension, and the outermost layer as N dimension. Figure 3 As shown, taking the H and W dimensions as an example, the storage method of the data blocks is as shown by the multiple dotted arrows in the rectangular box, and P data blocks can be stored continuously in Q memory areas of the GPGPU, where Q memory areas represent the memory areas currently available for the input tensor, and Q is not greater than the total number of currently available memory areas, and the P data blocks can be P data blocks continuous along these dotted arrows, and these data blocks can be stored in the corresponding memory areas of the Q memory areas in the order shown by these dotted arrows. Optionally, several data blocks that are continuous based on the position in the input tensor and the predetermined storage layout can be written to the memory in a cyclic manner. For example, in the case of storing 12 data blocks continuously in 6 memory areas of the GPGPU, the first 6 data blocks can be first written into the 6 memory areas in sequence, and then the last 6 data blocks can be written into the 6 memory areas in sequence.

[0056] By using the above storage method, the data in the input tensor can be evenly stored in each memory area of ​​the GPGPU. Of course, it should be understood that the above data storage method is only used as an example and not a limitation in the present disclosure, and the method of the present disclosure can also adopt other data storage methods that can achieve the same effect.

[0057] According to an embodiment of the present disclosure, converting the input tensor into a one-dimensional tensor may include: arranging the multiple data blocks according to the predetermined storage layout based on the position of each data block in the input tensor to convert the input tensor into the one-dimensional tensor.

[0058] After completing the data block division 301 of the input tensor as described above, tensor folding 302 can be performed based on the determined multiple data blocks. Optionally, the data blocks can be spliced ​​according to the storage order of the data blocks indicated by the above-mentioned predetermined storage layout to generate a one-dimensional tensor corresponding to the input tensor. Optionally, the computing task of the input tensor in the present disclosure can be a computing task that is insensitive to the memory access order, that is, it does not depend on the relative position or order of the elements in the input tensor during the calculation process. In other words, the data processing method in the present disclosure can be applicable to data processing scenarios that do not require the access order of the elements of the input tensor, so the data blocks of the input tensor can be stored and accessed in the desired order.

[0059] like Figure 3 As shown, the input tensor is divided into H_tile (e.g., 8) and W_tile (e.g., 6) parts in the H and W dimensions, that is, the input tensor is divided into H_tile×W_tile data blocks, such as 8×6 data blocks b1-b48. The 48 data blocks can be spliced ​​into a one-dimensional tensor on the left based on a predetermined storage layout, where data blocks b1-b8 correspond to the first 8 elements of the one-dimensional tensor, and then data blocks b9-b16 correspond to the subsequent 8 elements of the one-dimensional tensor, and so on.

[0060] Based on the one-dimensional tensor obtained through the above steps, such as Figure 3 As shown, task allocation 303 of computing tasks for input tensors may be performed.

[0061] In step S203, task allocation of the computing task on the hardware resources of the GPGPU may be determined based on the one-dimensional tensor according to the computing task for the input tensor.

[0062] Optionally, for task allocation 303, the following aspects can be considered: ① Data parallelism: that is, consider whether the computing task for the input tensor can be decomposed into multiple independent sub-computing tasks for parallel computing. If there is no dependency between the sub-computing tasks and different data blocks can be processed in parallel, then the degree of parallelism can be improved through data parallelism; ② Task allocation: that is, the computing tasks are reasonably allocated to different execution units. The task allocation strategy can be determined based on factors such as task complexity, data dependency, and computing load. For example, computing-intensive tasks can be allocated to CUs, and data-intensive tasks can be allocated to EUs; ③ Data transmission: In distributed computing, data may need to be transferred from host memory to GPU memory, or data may need to be transferred between GPU memories. Therefore, it is necessary to reasonably manage data transmission and reduce the overhead of data transmission, thereby improving the execution efficiency of computing tasks.

[0063] In an embodiment of the present disclosure, for each element in the above-mentioned one-dimensional tensor, that is, each data block, a corresponding SPC in the hardware resources of the GPGPU can be used to perform a sub-computation task for the data block. Optionally, for a first number of SPCs that can be used to perform computing tasks for the input tensor, each SPC can be used to process sub-computation tasks for multiple data blocks at a time, and the multiple SPCs can be used to concurrently process sub-computation tasks for multiple groups of the first number of consecutive data blocks corresponding to the one-dimensional tensor according to the storage order of the data blocks indicated by the above-mentioned predetermined storage layout. For example, Figure 3As shown, assuming that there are 6 SPCs that can be used to perform computing tasks for the input tensor, the 6 SPCs can be used to process a group of 6 consecutive data blocks (e.g., data blocks b1-b6) corresponding to the one-dimensional tensor, and for the subsequent group of 6 consecutive data blocks (e.g., data blocks b7-b12), and so on, until the sub-computation tasks of all data blocks corresponding to the one-dimensional tensor are assigned to the SPC for processing. By performing task decomposition in one dimension as described above, a more uniform task allocation can be achieved, making the utilization of the hardware resources of the GPGPU more balanced.

[0064] Optionally, in the data processing method of the present disclosure, based on the actual data processing capability of the hardware resources of the GPGPU, multiple ways of converting the input tensor into a one-dimensional tensor can be adopted, and thus a task allocation method corresponding to these ways can be adopted. The following will introduce three ways of converting the input tensor into a one-dimensional tensor as examples.

[0065] Figure 4 is a schematic diagram showing a first example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure. Figure 4 In FIG. 5 , a first example of tensor folding for a three-dimensional input tensor is shown, that is, each thread of the GPGPU reads and processes one data in the input tensor at a time.

[0066] According to an embodiment of the present disclosure, according to the computing task for the input tensor, determining the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor may include: based on the position of the data block corresponding to each element in the one-dimensional tensor in the input tensor and the predetermined storage layout, decomposing the computing task for the input tensor into sub-computing tasks executed on each execution unit of a plurality of execution units of the GPGPU, wherein the sub-computing task executed on each execution unit can be used to process an element in the one-dimensional tensor; wherein each execution unit may include multiple threads, wherein each thread can be used to process a data in a data block corresponding to an element in the one-dimensional tensor.

[0067] Optionally, when folding several dimensions of the input tensor into one dimension, the dimension with the smallest memory address jump with the coordinates can be placed in the innermost layer to improve the memory access efficiency. Among them, the smaller jump of the memory address corresponds to a smaller difference between the addresses of adjacent data elements in the memory, that is, the storage location of the data in the memory is more continuous. On the contrary, the larger jump of the memory address corresponds to a larger difference between the addresses of adjacent data elements in the memory, that is, the storage location of the data in the memory is more discrete. By placing the dimension with the smallest memory address jump with the coordinates in the innermost layer of tensor folding and task allocation (for example, in Figure 3In the case shown, placing the H dimension of the input tensor in the innermost layer can make the data access in the memory more continuous. Therefore, when the execution unit accesses the data, better data continuity can be achieved, thereby reducing the latency of the memory access operation and improving the utilization of the memory access bandwidth.

[0068] Optionally, when each GPGPU thread reads and processes only one data in the input tensor at a time, the dimension in which the memory address jumps the least with the coordinates can be taken as the innermost layer, and the tensor folding can be performed with one data in this dimension as a unit. Figure 4 As shown, when the above-mentioned predetermined storage layout is adopted, the H dimension of the input tensor is taken as the innermost layer, and tensor folding is performed in units of a row on the H dimension.

[0069] like Figure 4 As shown, for an input tensor of dimension N×H×W, the dimension with the smallest memory address jump with coordinates can be the innermost layer. For example, when data is stored according to the above predetermined storage layout, the H dimension can be the innermost layer, and tasks are assigned in units of a row on the H dimension. For example, a row on the H dimension and several columns on the W dimension constitute a data block (corresponding to Figure 4 The smallest rectangular block in the W dimension), where the columns on the W dimension may correspond to the number of threads included in an execution unit (warp_size), and the data block may correspond to an execution unit (corresponding to Figure 4 The smallest rectangular block in . Figure 4 As shown, one execution unit may include warp_size threads, so for dimension W, a row on the H dimension may correspond to W_tile execution units, where W_tile corresponds to the value obtained by rounding up (W+warp_size-1) / warp_size, that is, the number of thread warps (execution units) corresponding to one unit of the H dimension. Therefore, when data block partitioning, tensor folding, and task allocation are performed in units of a row on the H dimension and a batch on the N dimension, the H dimension is divided into H_tile=H units, and the N dimension is divided into N_tile=N units.

[0070] Therefore, for the above reference Figure 4 In the described situation, the one-dimensional tensor corresponding to the input tensor may include N_tile*H_tile*W_tile data blocks, and these data blocks may be processed by corresponding N_tile*H_tile*W_tile execution units.

[0071] In addition, considering the burst transmission characteristics of GPGPU, that is, the ability of GPGPU to achieve efficient data transmission and computing operations by utilizing data parallelism and instruction-level parallelism, as well as hardware optimizations such as cache and memory bandwidth, in the embodiments of the present disclosure, it is also possible to consider the situation where each thread of GPGPU reads and processes two data at the same time, which corresponds to the second example below.

[0072] According to an embodiment of the present disclosure, according to the computing task for the input tensor, determining the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor may include: based on the position of the data block corresponding to each element in the one-dimensional tensor in the input tensor and the predetermined storage layout, decomposing the computing task for the input tensor into sub-computation tasks executed on each execution unit of the multiple execution units of the GPGPU, wherein the sub-computation task executed on each execution unit can be used to process an element in the one-dimensional tensor, and the one element can correspond to two or more data blocks of the input tensor that are continuous based on the predetermined storage layout; wherein each execution unit may include multiple threads, wherein each thread can be used to process two data, and the two data are respectively from two adjacent data blocks of the two or more data blocks corresponding to an element in the one-dimensional tensor.

[0073] Figure 5A is a schematic diagram illustrating a second example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure. Figure 5B is a schematic diagram showing task allocation to hardware resources of a GPGPU based on a second example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure.

[0074] Optionally, when each GPGPU thread can read and process two data in the input tensor at a time, the dimension in which the memory address jumps the least with the coordinates can be taken as the innermost layer, and the two data on this dimension can be used as units for tensor folding. Figure 5A As shown, when the above predetermined storage layout is adopted, the H dimension of the input tensor is taken as the innermost layer, and tensor folding is performed in units of two rows on the H dimension.

[0075] like Figure 5A and 5B As shown, for an input tensor of dimension N×H×W, the dimension with the smallest memory address jump with coordinates can be the innermost layer. For example, when data is stored according to the above predetermined storage layout, the H dimension can be the innermost layer, and tasks can be assigned in units of two rows on the H dimension. For example, a row on the H dimension and several columns on the W dimension constitute a data block (corresponding to Figure 5AThe smallest rectangular block in the H dimension), and the two rows in the H dimension and the columns in the W dimension constitute an element in the one-dimensional tensor (corresponding to Figure 5A Two consecutive minimum rectangular blocks in ), where several columns on the W dimension can correspond to the number of threads included in an execution unit (warp_size), and an element in the one-dimensional tensor can correspond to an execution unit (corresponding to Figure 5A Two consecutive minimum rectangular blocks in ). Figure 5A As shown, one execution unit may include warp_size threads, so for dimension W, two rows in the H dimension may correspond to W_tile execution units, where W_tile corresponds to the value obtained by rounding up (W+warp_size-1) / warp_size, that is, the number of thread warps (execution units) corresponding to one unit in the H dimension. Therefore, when tensor folding and task allocation are performed in units of two rows in the H dimension and a batch in the N dimension, the H dimension is divided into parts, of which It means rounding up, and the N dimension is divided into N_tile=N parts.

[0076] For example, Figure 5B As shown, assuming that there are 6 SPCs available for performing the computing task for the input tensor, the 6 SPCs can be used for processing a corresponding group of 6 elements in the one-dimensional tensor (i.e., 6×2 consecutive data blocks, for example, data blocks b1-b12), and for a subsequent group of 6 elements (i.e., 6×2 consecutive data blocks, for example, data blocks b13-b24, Figure 5B (not fully shown in the figure), and so on, until the sub-computing tasks of all data blocks corresponding to the one-dimensional tensor are assigned to the SPC for processing.

[0077] Therefore, for the above reference Figure 5A In the above case, the one-dimensional tensor corresponding to the input tensor may include N_tile*H_tile*W_tile data blocks, and these data blocks may be processed by corresponding N_tile*H_tile*W_tile execution units. By using the tensor folding and task allocation method described above with reference to the second example, the hardware resources of GPGPU can be more fully utilized to improve computing efficiency.

[0078] Furthermore, based on the above second example, the memory access efficiency of the method disclosed herein can be further improved. Fig. 6A is a schematic diagram illustrating a third example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure. Figure 6Bis a schematic diagram showing task allocation to hardware resources of a GPGPU based on a third example of converting an input tensor into a one-dimensional tensor according to an embodiment of the present disclosure.

[0079] Optionally, similar to the second example, when each thread of the GPGPU can read and process two data in the input tensor at a time, the dimension in which the memory address jumps the least with the coordinates can be taken as the innermost layer, and the tensor folding can be performed in units of four data on this dimension. Fig. 6A As shown, when the above predetermined storage layout is adopted, the H dimension of the input tensor is taken as the innermost layer, and tensor folding is performed in units of four rows on the H dimension.

[0080] like Fig. 6A and 6B As shown, for an input tensor of dimension N×H×W, the dimension with the smallest memory address jump with coordinates can be the innermost layer. For example, when data is stored according to the above predetermined storage layout, the H dimension can be the innermost layer, and tasks are assigned in units of four rows on the H dimension. For example, a row on the H dimension and several columns on the W dimension constitute a data block (corresponding to Fig. 6A The smallest rectangular block in the H dimension), and the four rows in the H dimension and the columns in the W dimension constitute an element in the one-dimensional tensor (corresponding to Fig. 6A ), where a number of columns on the W dimension may correspond to the number of threads included in an execution unit (warp_size), and an element in the one-dimensional tensor may correspond to an execution unit (corresponding to Fig. 6A Furthermore, an element in a one-dimensional tensor can be divided into two groups when processed by an execution unit (e.g. Fig. 6A As shown in the grouping of four consecutive minimum rectangular blocks filled with different colors in FIG) (for example, by two execution instructions) are processed in sequence. Fig. 6A As shown, one execution unit may include warp_size threads, so for dimension W, four rows in the H dimension may correspond to W_tile execution units, where W_tile corresponds to the value obtained by rounding up (W+warp_size-1) / warp-size, that is, the number of thread warps (execution units) corresponding to one unit in the H dimension. Therefore, when tensor folding and task allocation are performed in units of four rows in the H dimension and a batch in the N dimension, the H dimension is divided into parts, the N dimension is divided into N_tile=N parts.

[0081] For example, Figure 6BAs shown, assuming that there are 6 SPCs available for performing the computing task for the input tensor, the 6 SPCs can be used for processing a corresponding group of 6 elements in the one-dimensional tensor (i.e., 6×4 consecutive data blocks, for example, data blocks b1-b16), and for a subsequent group of 6 elements (i.e., 6×4 consecutive data blocks, for example, data blocks b17-b32, Figure 6B The processing of (not shown) is repeated in this way until all sub-computation tasks of all data blocks corresponding to the one-dimensional tensor are assigned to the SPC for processing.

[0082] Therefore, for the above reference Fig. 6A In the case described above, the one-dimensional tensor corresponding to the input tensor may include N_tile*H_tile*W_tile data blocks, and these data blocks may be processed by corresponding N_tile*H_tile*W_tile execution units. By using the tensor folding and task allocation method described above with reference to the third example, the coordinates of each data block can be reused for reading data, and then the data can be accessed in the memory more efficiently, thereby reducing the access delay of the data and maximizing the reuse of the data to improve the computing efficiency.

[0083] After completing the task allocation of the computing task for the input tensor, in step S204, the computing task for the input tensor may be executed based on the task allocation of the computing task on the hardware resources of the GPGPU.

[0084] According to an embodiment of the present disclosure, based on the task allocation of the computing task on the hardware resources of the GPGPU, executing the computing task for the input tensor may include: executing the computing task for the input tensor by executing corresponding sub-computing tasks on multiple execution units of the GPGPU.

[0085] Optionally, as described above, for computing tasks that are insensitive to memory access order, the data processing method disclosed in the present invention can decompose the computing tasks for the input tensor into multiple independent sub-computing tasks in one dimension, and reasonably allocate the sub-computing tasks to different execution units in one dimension. By folding the dimension of the input tensor into one dimension for memory access according to the minimum alignment granularity of the hardware, the parallelism can be improved through data parallelism, and the efficiency of memory access can be improved and the latency of memory access can be reduced by utilizing the hardware alignment requirements, so that the parallel computing capabilities of GPGPU can be better utilized and the execution speed of operators can be accelerated. In addition, by allocating tasks in one dimension, more uniform task distribution can be achieved, memory bandwidth utilization can be improved, and the utilization of GPGPU hardware resources can be more balanced.

[0086] Figure 7is a schematic diagram showing a data processing device 700 according to an embodiment of the present disclosure.

[0087] According to an embodiment of the present disclosure, the data processing device 700 may include a data acquisition module 701 , a tensor folding module 702 , a task decomposition module 703 and a task execution module 704 .

[0088] The data acquisition module 701 may be configured to acquire an input tensor. Optionally, the data acquisition module 701 may perform the operations described above with reference to step S201.

[0089] For example, a tensor can be obtained as an input to the data processing device of the present disclosure, that is, as an input to a computing task. An input tensor may refer to multidimensional array data that needs to be calculated on a GPGPU, which may be various types of data, such as images, audio, text, or any other type of data. The dimension, shape, and data type of the input tensor may depend on the specific application scenario and task requirements. For example, for image processing tasks, the input tensor may typically have three or four dimensions, representing the height, width, number of channels, and batch size of the image, respectively. For audio processing tasks, the input tensor may typically have two or three dimensions, representing the number of sampling points, number of channels, and batch size of the audio, respectively. For text processing tasks, the input tensor may typically have two dimensions, representing the length of the text and the batch size, respectively.

[0090] Optionally, the data of the input tensor can be loaded and transferred to the GPGPU in a variety of ways. For example, common ways may include copying data from the host memory to the GPGPU memory, or directly allocating and initializing data in the GPGPU memory. After the calculation, the results can be copied from the GPGPU memory back to the host memory for further processing and analysis.

[0091] The tensor folding module 702 may be configured to divide the input tensor into a plurality of data blocks based on the minimum alignment granularity of the GPGPU, and convert the input tensor into a one-dimensional tensor, wherein each element in the one-dimensional tensor corresponds to a data block, and the minimum alignment granularity is used to indicate that the starting address of the storage address of each data block in the memory of the GPGPU is aligned based on the minimum alignment granularity. Optionally, the tensor folding module 702 may perform the operations described above with reference to step S202.

[0092] For example, the dimension of the input tensor can be folded according to the minimum alignment granularity defined by the hardware layout of the GPGPU to convert the input tensor into a one-dimensional tensor, wherein the minimum alignment granularity defined by the hardware layout of the GPGPU may refer to the need for data to be stored in a certain byte alignment when accessed in the memory of the GPGPU to improve memory access efficiency. Specifically, the multi-dimensional input tensor can be reorganized into a one-dimensional array and memory aligned according to the hardware layout requirements, which defines how the dimensions of the input tensor are mapped to the storage method in the memory of the GPGPU.

[0093] Therefore, the input tensor can be converted into a one-dimensional tensor according to the storage method of the data of the input tensor in the memory of the GPGPU. In an embodiment of the present disclosure, the input tensor can be divided into multiple data blocks according to the minimum alignment granularity defined by the hardware layout of the GPGPU, wherein each data block is aligned according to a certain byte alignment method. Among them, alignment refers to the alignment of the data stored in the memory according to certain rules. For example, when the above-mentioned minimum alignment granularity is 4 bytes, the data block will be stored with 4-byte boundary alignment, that is, the starting address of the data block must be a multiple of 4.

[0094] Optionally, different data blocks divided from the input tensor can be stored in different memory areas, and these data blocks can be stored according to a certain byte alignment. Optionally, the minimum alignment granularity of the GPGPU can indicate the alignment of each data block, for example, the number of consecutively aligned data blocks and the amount of data included in each data block.

[0095] Optionally, all data in a data block divided from the input tensor can be stored in the same memory area to make GPGPU access to the memory more continuous.

[0096] Optionally, different data blocks divided from the input tensor can be stored in different memory areas, and the way these different data blocks are stored can be based on their positions in the input tensor and a predetermined storage layout. For example, the predetermined storage layout can indicate the storage order of data blocks on various dimensions of the input tensor in the memory of the GPGPU, for example, the data blocks in the input tensor are stored layer by layer in the order of the innermost layer being the H dimension, the second outermost layer being the W dimension, and the outermost layer being the N dimension.

[0097] The task decomposition module 703 may be configured to determine the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor according to the computing task for the input tensor. Optionally, the task decomposition module 703 may perform the operation described above with reference to step S203.

[0098] For example, in the task decomposition module 703, the following aspects can be considered for task allocation: ① Data parallelism: that is, consider whether the computing task for the input tensor can be decomposed into multiple independent sub-computing tasks for parallel computing. If there is no dependency between the sub-computing tasks and different data blocks can be processed in parallel, then the degree of parallelism can be improved through data parallelism; ② Task allocation: that is, the computing tasks are reasonably allocated to different execution units. The task allocation strategy can be determined based on factors such as task complexity, data dependency, and computing load. For example, computing-intensive tasks can be allocated to CUs, and data-intensive tasks can be allocated to EUs; ③ Data transmission: In distributed computing, data may need to be transferred from host memory to GPU memory, or data may need to be transferred between GPU memories. Therefore, it is necessary to reasonably manage data transmission and reduce the overhead of data transmission, thereby improving the execution efficiency of computing tasks.

[0099] Optionally, for each element in the one-dimensional tensor, i.e., each data block, a corresponding SPC in the hardware resources of the GPGPU may be used to perform a sub-computation task for the data block. Optionally, for a first number of SPCs that can be used to perform computational tasks for the input tensor, each SPC may be used to process sub-computation tasks for multiple data blocks at a time, and the multiple SPCs may be used to concurrently process sub-computation tasks for multiple groups of the first number of consecutive data blocks corresponding to the one-dimensional tensor according to the storage order of the data blocks indicated by the predetermined storage layout.

[0100] Optionally, in the data processing method of the present disclosure, based on the actual data processing capability of the hardware resources of the GPGPU, multiple ways of converting the input tensor into a one-dimensional tensor can be adopted, and thus task allocation methods corresponding to these ways can be adopted.

[0101] Optionally, when folding several dimensions of the input tensor into one dimension, the dimension with the smallest memory address jump with the coordinates can be placed in the innermost layer to improve the memory access efficiency. Among them, the smaller jump of the memory address corresponds to a smaller difference between the addresses of adjacent data elements in the memory, that is, the storage location of the data in the memory is more continuous. On the contrary, the larger jump of the memory address corresponds to a larger difference between the addresses of adjacent data elements in the memory, that is, the storage location of the data in the memory is more discrete. By placing the dimension with the smallest memory address jump with the coordinates in the innermost layer of tensor folding and task allocation (for example, in Figure 3 In the case shown, placing the H dimension of the input tensor in the innermost layer can make the data access in the memory more continuous. Therefore, when the execution unit accesses the data, better data continuity can be achieved, thereby reducing the latency of the memory access operation and improving the utilization of the memory access bandwidth.

[0102] For example, when each thread of the GPGPU reads and processes only one data in the input tensor at a time, the dimension in which the memory address jumps the least with the coordinates can be taken as the innermost layer, and the tensor folding can be performed with one data on this dimension as a unit.

[0103] In addition, considering the burst transmission characteristics of GPGPU, that is, in GPGPU, the ability to achieve efficient data transmission and computing operations by utilizing data parallelism and instruction-level parallelism, as well as hardware optimizations such as cache and memory bandwidth, in the embodiments of the present disclosure, the situation where each thread of GPGPU reads and processes two data at the same time can also be considered. For example, in the case where each thread of GPGPU can read and process two data in the input tensor at a time, the dimension with the smallest jump of the memory address with the coordinates can be used as the innermost layer, and the two data on this dimension can be used as units for tensor folding.

[0104] Furthermore, based on the above second example, the memory access efficiency of the method disclosed in the present invention can be further improved. For example, when each thread of the GPGPU can read and process two data in the input tensor at a time, the dimension in which the memory address jumps the least with the coordinates can be taken as the innermost layer, and the four data on this dimension can be used as units for tensor folding.

[0105] The task execution module 704 may be configured to execute the computing task for the input tensor based on the task allocation of the computing task on the hardware resources of the GPGPU. Optionally, the task execution module 704 may perform the operations described above with reference to step S204.

[0106] After completing the task allocation for the computing task of the input tensor, optionally, for the computing task that is insensitive to the memory access order, the computing task for the input tensor can be decomposed into multiple independent sub-computation tasks in one dimension, and the sub-computation tasks can be reasonably allocated to different execution units in one dimension. By folding the dimension of the input tensor into one dimension for memory access according to the minimum alignment granularity of the hardware, the parallelism can be improved through data parallelism, and the efficiency of memory access can be improved and the latency of memory access can be reduced by utilizing the hardware alignment requirements, so that the parallel computing capabilities of GPGPU can be better utilized and the execution speed of the operator can be accelerated. In addition, by performing task allocation in one dimension, a more uniform task allocation can be achieved, the memory bandwidth utilization can be improved, and the utilization of the hardware resources of GPGPU can be more balanced.

[0107] According to yet another aspect of the present disclosure, a data processing device is provided. Figure 8 A schematic diagram of a data processing device 2000 according to an embodiment of the present disclosure is shown.

[0108] like Figure 8As shown, the data processing device 2000 may include one or more processors 2010 and one or more memories 2020. The memory 2020 stores a computer readable code, and when the computer readable code is run by the one or more processors 2010, the data processing method described above may be executed.

[0109] The processor in the embodiments of the present disclosure may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc., and may be an X86 architecture or an ARM architecture.

[0110] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0111] For example, the method or device according to the embodiment of the present disclosure may also be implemented by Fig. 9 The architecture of the computing device 3000 shown in FIG. Fig. 9 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the data processing method provided in the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Fig. 9 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Fig. 9 One or more components of a computing device are shown.

[0112] According to another aspect of the present disclosure, a computer-readable storage medium is also provided. Computer-readable instructions are stored on the computer storage medium. When the computer-readable instructions are executed by a processor, the data processing method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0113] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data processing method according to the embodiments of the present disclosure.

[0114] Embodiments of the present disclosure provide a data processing method, apparatus, device, computer program product, and computer-readable storage medium.

[0115] The method provided by the embodiment of the present disclosure divides the input tensor into multiple data blocks based on the minimum alignment granularity of GPGPU, and converts the input tensor into a one-dimensional tensor based on the divided multiple data blocks, so as to decompose the computing task of the input tensor in the dimension of the one-dimensional tensor, thereby decomposing the computing task of the input tensor into multiple independent subtasks that can be executed in parallel on the hardware resources of the GPU. The method of the embodiment of the present disclosure can improve the computing parallelism of GPGPU by performing task decomposition in the dimension of the one-dimensional tensor, achieve higher hardware resource utilization, and thus improve the overall computing performance and efficiency.

[0116] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0118] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A data processing method, comprising: Get the input tensor; Based on a minimum alignment granularity of a general purpose graphics processing unit GPGPU, the input tensor is divided into a plurality of data blocks, and the input tensor is converted into a one-dimensional tensor by splicing the plurality of data blocks, wherein each element in the one-dimensional tensor corresponds to a data block, and the minimum alignment granularity is used to indicate that a starting address of a storage address of each data block in the memory of the GPGPU is aligned based on the minimum alignment granularity; According to the computing task for the input tensor, determining the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor; and Based on the task allocation of the computing task on the hardware resources of the GPGPU, executing the computing task for the input tensor; Among them, according to the computing task for the input tensor, determining the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor includes decomposing the computing task into sub-computing tasks executed on each execution unit of multiple execution units of the GPGPU, wherein the sub-computing task executed on each execution unit is used to process an element in the one-dimensional tensor.

2. The method of claim 1, wherein: For a plurality of memory regions in the memory of the GPGPU for storing the input tensor, data in a data block corresponding to an element in the one-dimensional tensor is stored in one of the plurality of memory regions, and For different elements in the one-dimensional tensor, different data blocks corresponding to the different elements are stored in one or more memory areas of the multiple memory areas based on the positions of the different data blocks in the input tensor and a predetermined storage layout, wherein the predetermined storage layout indicates how the data blocks on different dimensions of the input tensor are arranged in the memory of the GPGPU.

3. The method of claim 2, wherein: Converting the input tensor to a one-dimensional tensor includes: Based on a position of each data block of the multiple data blocks in the input tensor, the multiple data blocks are arranged according to the predetermined storage layout to convert the input tensor into the one-dimensional tensor.

4. The method of claim 3, wherein: Decomposing the computing task into sub-computing tasks executed on each of the plurality of execution units of the GPGPU comprises: Based on the position of the data block corresponding to each element in the one-dimensional tensor in the input tensor and the predetermined storage layout, decomposing the computing task into sub-computing tasks executed on each execution unit of the plurality of execution units of the GPGPU; Each execution unit includes multiple threads, and each thread is used to process a data in a data block corresponding to an element in the one-dimensional tensor.

5. The method of claim 3, wherein: Decomposing the computing task into sub-computing tasks executed on each of the plurality of execution units of the GPGPU comprises: Based on the position of the data block corresponding to each element in the one-dimensional tensor in the input tensor and the predetermined storage layout, decomposing the computing task into sub-computing tasks executed on each execution unit of the plurality of execution units of the GPGPU, wherein the one element corresponds to two or more data blocks of the input tensor that are continuous based on the predetermined storage layout; Each execution unit includes multiple threads, each thread is used to process two data, and the two data are respectively from two adjacent data blocks in two or more data blocks corresponding to an element in the one-dimensional tensor.

6. The method according to claim 4 or 5, wherein: Based on the task allocation of the computing task on the hardware resources of the GPGPU, executing the computing task for the input tensor includes: The computing task for the input tensor is performed by executing corresponding sub-computing tasks on multiple execution units of the GPGPU.

7. A data processing device, comprising: A data acquisition module, configured to acquire input tensors; A tensor folding module is configured to divide the input tensor into a plurality of data blocks based on a minimum alignment granularity of a general purpose graphics processing unit GPGPU, and convert the input tensor into a one-dimensional tensor by splicing the plurality of data blocks, wherein each element in the one-dimensional tensor corresponds to a data block, and the minimum alignment granularity is used to indicate that a starting address of a storage address of each data block in the memory of the GPGPU is aligned based on the minimum alignment granularity; a task decomposition module configured to determine, according to the computing task for the input tensor, task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor; and A task execution module, configured to execute the computing task for the input tensor based on the task allocation of the computing task on the hardware resources of the GPGPU; Among them, according to the computing task for the input tensor, determining the task allocation of the computing task on the hardware resources of the GPGPU based on the one-dimensional tensor includes decomposing the computing task into sub-computing tasks executed on each execution unit of multiple execution units of the GPGPU, wherein the sub-computing task executed on each execution unit is used to process an element in the one-dimensional tensor.

8. The device according to claim 7, wherein: For a plurality of memory regions in the memory of the GPGPU for storing the input tensor, data in a data block corresponding to an element in the one-dimensional tensor is stored in one of the plurality of memory regions, and For different elements in the one-dimensional tensor, different data blocks corresponding to the different elements are stored in one or more memory areas of the multiple memory areas based on the positions of the different data blocks in the input tensor and a predetermined storage layout, wherein the predetermined storage layout indicates how the data blocks on different dimensions of the input tensor are arranged in the memory of the GPGPU.

9. A data processing device comprising: one or more processors; as well as One or more memories storing a computer executable program, wherein when the computer executable program is executed by the processor, the method according to any one of claims 1 to 6 is performed.

10. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the instructions are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

Images