Transpose vector processing method and device, electronic equipment and storage medium

Abstract

The invention provides a transposition vector processing method and device, electronic equipment and a storage medium, and relates to the technical field of chips. The method comprises the following steps: mapping a transposition operation executed by each DMA kernel according to a transposition vector, wherein the transposition vector is used for transposing tensor data; determining the operation cost of the DMA kernel according to the operation amount corresponding to the transposition operation and the operation capability of the DMA kernel; searching according to the transposition operation mapped by the multiple DMA kernels and the operation cost of each DMA kernel, and determining a target transposition path; and controlling a DMA kernel to transpose tensor data according to the target transpose path. According to the method, the target transposition path can be determined according to the computing power of each DMA, and the transposition vector comprising a plurality of transposition steps is completed through the transposition operation mapped by each DMA on the target transposition path, so that the transposition efficiency of the tensor is improved. The target transposition path can complete the transposition of the tensor with the highest performance, the high-performance operation of hardware is realized, the transposition of the tensor is completed, and the bandwidth utilization rate of equipment is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of chip technology, and in particular to a transposed vector processing method, device, electronic device and storage medium. Background Art

[0002] With the development of chip technology, deep learning based on chip technology has become possible.

[0003] In modern deep learning accelerators (DLA), tensors need to be transposed during deep learning operations. Currently, there are two solutions: MAC arrays that handle two or three specific dimensions and direct memory access (DMA) with wide buses. However, the above two methods are inefficient in the process of tensor transposition. How to provide a complete tensor transposition solution and improve the efficiency of tensor transposition has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a transposed vector processing method, device, electronic device and storage medium to solve the problem of low tensor transposition efficiency in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a transposed vector processing method, which is applied to a general computing unit, including:

[0006] Mapping the transpose operation performed by each DMA core according to a transpose vector, which is used to transpose the tensor data;

[0007] Determining the computation cost of the DMA kernel according to the computation amount corresponding to the transposition operation and the computation capability of the DMA kernel;

[0008] Searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine a target transposition path;

[0009] The DMA core is controlled to perform transposition processing on the tensor data according to the target transposition path.

[0010] In a second aspect, an embodiment of the present application further provides a transposed vector processing device, which is applied to a general computing unit, including:

[0011] A mapping module for mapping the transposition operation performed by each DMA core according to a transposition vector, wherein the transposition vector is used to transpose the tensor data;

[0012] A calculation module, used for determining the operation cost of the DMA kernel according to the operation amount corresponding to the transposition operation and the operation capability of the DMA kernel;

[0013] A search module, used for searching according to the transposition operations mapped by multiple DMA cores and the operation cost of each DMA core to determine a target transposition path;

[0014] The transposition module is used to control the DMA kernel to perform transposition processing on the tensor data according to the target transposition path.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that it includes: a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory to implement the transposed vector processing method as shown in the embodiment of the present application.

[0018] In a fourth aspect, an embodiment of the present application further provides a storage medium comprising computer executable instructions, wherein the computer readable storage medium stores computer executable instructions, and when the computer executable instructions are executed by a processor, they are used to implement the transposed vector processing method as shown in the embodiment of the present application.

[0019] In a sixth aspect, an embodiment of the present application further provides a computer program product, characterized in that it includes a computer program, which, when executed by a processor, implements the transposed vector processing method as shown in the embodiment of the present application.

[0020] The transposition vector processing method provided by the present application maps the transposition operation performed by each DMA kernel according to the transposition vector, and the transposition vector is used to transpose the tensor data; the operation cost of the DMA kernel is determined according to the operation amount corresponding to the transposition operation and the operation power of the DMA kernel; the target transposition path is determined according to the transposition operations mapped by multiple DMA kernels and the operation cost of each DMA kernel; the DMA kernel is controlled to perform transposition processing on the tensor data according to the target transposition path. Compared with the current inability to efficiently achieve complete tensor transposition through DMA or MAC array, the present application can determine the target transposition path according to the computing power of each DMA, and complete the transposition vector containing multiple transposition steps through the transposition operations mapped by each DMA on the target transposition path, thereby improving the transposition efficiency of the tensor. The target transposition path can complete the transposition of the tensor with the highest performance, realize the hardware while operating at high performance, and complete the transposition of the tensor, thereby improving the bandwidth utilization of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a transposed vector processing method provided in an embodiment of the present application;

[0022] Figure 2This is a schematic diagram of DMA kernel node mapping provided in an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a group-based ping-pong operation provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of the structure of a transposed vector processing device provided in an embodiment of the present application;

[0025] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0027] With the development of chip technology, deep learning based on chip technology has become possible.

[0028] In modern deep learning accelerators (DLA), tensors need to be transposed during deep learning operations. There are two solutions: MAC arrays that handle two or three specific dimensions and direct memory access (DMA) with wide buses. However, the MAC array needs to meet the dimensional requirements at the lowest memory location, and a wide bus DMA requires that the size of the lowest dimension of a single transfer should match the bus width as much as possible.

[0029] However, the above two methods are inefficient in the process of tensor transposition. How to provide a complete tensor transposition solution and improve the efficiency of tensor transposition has become an urgent problem to be solved.

[0030] Data Terminal Equipment (DTE) is a device used for data transmission and deformation. In the process of data transmission, the data shape is changed by slicing, deslicing or reshaping, so as to obtain the data of the shape required by the computing engine. DTE data can be represented by five dimensions, and the highest dimension is used for loop control. The reshape operation of DTE corresponds to transpose, and the permute parameter can be used to express the dimensional mapping from the original shape to the target shape.

[0031] Due to the cost constraints of hardware implementation, DTE is incomplete in terms of function and performance. For example, a single DTE data is represented by five dimensions, and the highest dimension can only be used for loops, and each dimension has a maximum shape limit. Without considering the additional performance loss of the AXI bus, the bandwidth utilization of DTE can be expressed as: the minimum dimension size of the target shape / bus width, that is, we need to ensure the minimum dimension size of the DTE operation. If the hardware device cannot meet the above requirements, the efficiency of the deformation operation will be limited.

[0032] The following is a detailed description of the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The acquisition, transmission, storage, use, and processing of data in the technical solution of the present application are in compliance with the relevant provisions of national laws and regulations.

[0033] It should be noted that in the embodiments of the present application, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 This is a flow chart of a transposed vector processing method provided in an embodiment of the present application. This embodiment is applicable to the case where tensor transposition is performed through multiple DMAs in a chip system. This method can be executed by electronic devices such as a system on chip (SoC). Figure 1 As shown, the transposed vector processing method provided by the present application includes:

[0035] Step S101 : mapping the transposition operation performed by each DMA core according to a transposition vector, where the transposition vector is used to transpose tensor data.

[0036] Exemplarily, the transposed vector of a tensor can be expressed as torch.permute(input,dims)→Tensor. The transposed vector indicates that the input tensor input is used as the initial dimension and the dimension is transformed according to the deformation method represented by dims. The dimension memory arrangement of the result of the transpose transpose, the size of dims can be 64 bits. Assuming there are two dimensions, dims are 0 and 1 respectively, the transposed vector can be permute(input,{0,1}). At this time, the transposed vector indicates that no changes are made and the dimension is the same as before. If the transposed vector is permute(input,{1,0}), what is obtained is the transpose of the original matrix, that is, the positions of dimension 0 and dimension 1 are swapped.

[0037] Each DMA node maps a corresponding transposition operation, and the transposition operation is used to implement the transposition of two dimensions of the tensor. In the present application scenario, it is necessary to map the transposition operation performed by each DMA, and then determine the transposition operation performed by each DMA.

[0038] The DMA can be pre-configured to perform fixed-dimensional transposition operations. The transposition operations performed by each DMA can also be mapped in the transposition vector plan. By mapping the DMA transposition operations, the transposition vector can be converted into a combination of transposition operations of a limited number of DMAs, and each DMA performs transposition operations of two dimensions.

[0039] Optionally, before step S101, mapping the transposition operation performed by each DMA core according to the transposition vector, the method further includes: merging dimensions according to continuous dimensions in the transposition vector. Accordingly, mapping the transposition operation performed by each DMA core according to the transposition vector includes: mapping the transposition operation performed by each DMA core according to the merged transposition vector.

[0040] A compression algorithm can be used to compress and merge multiple continuous dimensions. The compression algorithm can be a Byte Pair Encoding (BPE) algorithm that simplifies the transposed vector in bytes. Continuous dimensions are dimensions that do not participate in the transposition. The compressed continuous dimensions do not participate in the dimension conversion, so the merge operation does not affect the implementation of the transposition operation.

[0041] The above implementation can merge continuous dimensions in the transposed vector, thereby simplifying the transposed vector, reducing the transposition complexity, and improving the tensor transposition efficiency and device performance.

[0042] Optionally, dimension merging is performed based on consecutive dimensions in the transposed vector, which can be implemented as follows:

[0043] If the transposed vector includes multiple continuous dimensions, the multiple continuous dimensions are merged into a merged dimension; and the merged transposed vector is determined according to the merged dimension and the remaining dimensions, where the remaining dimensions are the dimensions in the first transposed vector other than the multiple continuous dimensions.

[0044] Specifically, merging the multiple continuous dimensions into a merged dimension can be implemented as follows: determining the merged dimension according to the product corresponding to the multiple continuous dimensions. Correspondingly, determining the merged transposed vector according to the merged dimension and the remaining dimensions can be implemented as follows: using the lowest dimension identifier of the multiple continuous dimensions as the merged dimension identifier. If the original identifier values ​​of the remaining dimensions are greater than the merged dimension identifier, new identifiers of the remaining dimensions are determined according to the number of continuous dimensions of the multiple continuous dimensions and the original identifier values ​​of the remaining dimensions. The merged dimension and the remaining dimensions constitute the merged transposed vector.

[0045] For example, the transposed vector is (1,0,2,3,4,5), where only the 0th and 1st dimensions are transformed, and the 2nd, 3rd, 4th and 5th dimensions are not involved in the transposition. Therefore, the 2nd, 3rd, 4th and 5th dimensions can be merged into a new 2' dimension to achieve the effect of simplifying the transposed vector. The merged transposed vector is (1,0,2'), where the new 2' dimension is called the merged dimension, and the 0th and 1st dimensions are the remaining dimensions. Dimension merging can effectively reduce the number of dimensions of the transposed vector.

[0046] When merging dimensions, you can scan the continuous dim in permute. If there are multiple adjacent dimensions, select the smaller value A as the new dimension, and multiply the shapes corresponding to the multiple dimensions as the new shape. And subtract 1 from all values ​​greater than A in permute.

[0047] Only two consecutive dimensions or multiple consecutive dimensions can be merged to avoid too much dimensional data after merging. In the above example, after merging the transposed vector (1,0,2,3,4,5) every two consecutive dimensions, the merged transposed vector is (1,0,2',3').

[0048] The above implementation method can renumber the dimension representations while merging the dimensions when merging the dimensions, thereby simplifying the transposed vector and improving the transposition efficiency of the tensor without changing the mapped dimension operation.

[0049] Furthermore, after merging the dimensions according to the continuous dimensions in the transposed vector, it also includes:

[0050] The combined transposed vector is split according to the size of the dimension. Accordingly, the transposed operation performed by each DMA core is mapped according to the transposed vector, including: mapping the transposed operation performed by each DMA core according to the split transposed vector.

[0051] In order to ensure that the size of each dimension matches the amount of data that can be processed by DMA, after merging multiple dimensions, the merged dimensions can be split.

[0052] Optionally, splitting is performed based on the size of the dimension in the merged transposed vector, which can be implemented as follows:

[0053] If the size of the target dimension in the merged transposed vector is greater than the preset upper limit threshold, the target dimension is split to obtain a split transposed vector, where the target dimension is any dimension in the transposed vector; if the size of the target dimension in the merged transposed vector is less than or equal to the preset upper limit threshold, the splitting of the merged transposed vector is canceled.

[0054] The DMA involved in the transposed vector execution may have hardware limitations, such as a single dimension cannot handle a size greater than 65525 bits. In this case, the oversized dimension needs to be split.

[0055] The preset upper limit threshold may be 65525 bits, and the preset upper limit threshold may vary according to different DMAs, which is not limited here. The split dimension may be a merged dimension, or may be another dimension that is not merged. As long as the data volume of the dimension is greater than the preset upper limit threshold, the target dimension is split to obtain a split transposed vector.

[0056] When splitting, we are not limited to factorization. We can decompose into values ​​of similar size to ensure that the execution time of each sub-unit in the pipeline is similar.

[0057] The above implementation method can split a dimension when the amount of data in a certain dimension after merging is too large. The split dimensional data can match the DMA bandwidth, thereby improving performance and improving the efficiency of the tensor transpose machine.

[0058] Specifically, the target dimension is split to obtain the split transposed vector, which can be implemented as follows:

[0059] The target dimension is split into multiple dimensions. If the data size of the target dimension is less than a preset lower limit threshold, supplementary data is found from other dimensions and the supplementary data is added to the target dimension so that the size of the target dimension is greater than the preset lower limit threshold. The target dimension is any dimension in the transposed vector.

[0060] If the size of the lowest dimension is less than 128 bits, you need to prepare a suitable dimension for the subsequent DMA decomposition. According to the size of the lowest dimension, find a dimension before and after the dimension, and split the dimension into a high-performance size after merging with the target dimension.

[0061] The above implementation can supplement the data in the dimension when the data size of the split dimension is less than the preset lower limit threshold, thereby avoiding DMA bandwidth waste, improving DMA bandwidth utilization, and further improving performance and tensor transposition efficiency.

[0062] Step S102: determining the computation cost of the DMA kernel according to the computation amount corresponding to the transposition operation and the computation capability of the DMA kernel.

[0063] Step S103 : searching according to the transposition operations mapped by the multiple DMA cores and the computational cost of each DMA core to determine a target transposition path.

[0064] Optionally, searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine the target transposition path can be implemented in the following manner:

[0065] A directed graph is constructed, wherein the nodes in the directed graph are DMA kernels, and each node corresponds to a mapped transposition operation; the operation cost of the DMA kernel is used as the distance between the nodes; the starting point and the end point in the directed graph are determined according to the transposition vector; and the shortest path is searched in the directed graph according to the starting point and the end point, and the shortest path is used as the target transposition path.

[0066] Different DMA kernels have different bandwidths and belong to different storage levels. A DMA kernel can be located between the L3 cache and the L2 cache, between the L2 cache and the L1 cache, or between the L1 cache and the kernel. Based on the bandwidth and storage level of the DMA kernel, the computational cost required for the DMA to perform the transposition operation can be determined, that is, the time required to perform the transposition operation of the tensor data. The computational cost of each DMA kernel is calculated based on the DMA bandwidth and the storage level to which the DMA belongs.

[0067] In the directed graph, each DMA core is a node, and the connection relationship between nodes can be determined by the actual connection relationship of the DMA hardware. The length of the edge is determined by the computational cost of the node as the starting point of the edge, and then a directed graph is constructed. In the directed graph, the shortest path between the starting point and the end point is found. The shortest path is the path with the shortest edge length.

[0068] Figure 2 A schematic diagram of DMA kernel node mapping provided by an embodiment of the present invention is as follows: Figure 2As shown, exemplarily, the directed graph includes four nodes, each of which is mapped with a transposition operation.

[0069] DMA can support five-dimensional operations, but the highest dimension cannot be changed. For permute0-1-2-3-4, 4-3-2-1-0, 4-1-2-3-0, and 0-1-2-4-3, it can be seen that there is no arrow from node 0-1-2-3-4 to 4-3-2-1-0, indicating that these two states cannot be completed directly through a DMA, and additional intermediate nodes need to be added.

[0070] The search using wavefront detection can be done by setting each transposed state to a state and modeling the entire search strategy by finding the path of state transition. The path of state transition depends on the implementation of dataflow. If defined as a single DMA, the existence of the path is equivalent to being completed by a single DMA after dim merge and dim split and other possible operations.

[0071] Through the above implementation, the target transposition path with the minimum computational cost can be quickly and accurately searched through the directed graph, so that the optimal transposition path can be quickly determined and the transposition efficiency of the tensor can be improved.

[0072] Step S104: Control the DMA core to perform transposition processing on the tensor data according to the target transposition path.

[0073] Optionally, controlling the DMA kernel to perform transposition processing on the tensor data according to the target transposition path includes:

[0074] The tensor data is transposed in parallel according to the DMA cores included in the target transposition path.

[0075] Optionally, performing transposition of tensor data in parallel according to the DMA core included in the target transposition path may be implemented as follows:

[0076] The DMA cores included in the target transposition path are grouped; in each group, tensor data is transposed in parallel according to a ping-pong operation.

[0077] Figure 3 A schematic diagram of a group-based ping-pong operation provided by an embodiment of the present invention is shown in FIG. Figure 3 shown.

[0078] Specifically, grouping the DMA cores included in the target transposition path includes:

[0079] The DMA kernels included in the target transposition path are grouped according to their computing power so that the computing power of the DMA kernels in each group matches each other; the DMA kernels in the group are divided into ping operation kernels that perform ping operations and pong operation kernels that perform pong operations; and the transposition operation corresponding to each ping operation kernel and each pong operation kernel in the group is configured according to the transposition operation mapped by the DMA kernel.

[0080] DMA may include CDMA between the L2 cache and the L1 cache and SDMA between the L3 cache and the L2 cache.

[0081] Optionally, the ratio of the number of CDMA, SDMA, and instruction access modules can be 1:3:6. Considering the overhead of synchronization at each level, when only CDMA and SDMA are used, the SDMA is divided into two groups and designed with a ratio of 1:2:0. The two SDMAs can share the same local storage, which can achieve a relatively perfect balance between storage and handling.

[0082] It should be noted that the above ratio and grouping method are only an example of the final implementation, and various device grouping options can be selected according to needs. For example, in order to support deeper parallelism, you can choose 1:2:2 or use only CDMA for D2D+D2D or D2C+C2D grouping.

[0083] Through the above method, the computing power of each group DMA core can be balanced, and the computing power between each group can be balanced. Furthermore, when performing ping-pong operation, synchronous transposition operation can be achieved between different groups, thereby improving the parallel efficiency of transposition operation.

[0084] Specifically, tensor data is transposed in parallel according to the ping-pong operation, including: at a first moment, calling all ping-pong operation kernels in the group to execute the transposition operation corresponding to the ping-pong operation; at a second moment, calling all pong operation kernels in the group to execute the transposition operation corresponding to the pong operation; the first moment and the second moment are moments that occur cyclically according to a preset period.

[0085] After obtaining two groups through grouping, it is assumed that the grouping includes a high bandwidth memory (HBM) and a configuration space bus (CSB) connected to multiple different DMAs. The following operations are performed at the first moment: HBM writes data to the ping module of the DMA corresponding to CSB0, which is a CDTE-copy operation; the pong module of the DMA corresponding to CSB0 performs a transposition operation on the data written at the previous moment and writes it to the pong module of the DMA corresponding to CSB1. The ping module of the DMA corresponding to CSB1 performs a transposition operation on the data written at the previous moment and writes it to the ping module of the DMA corresponding to CSB3. The pong module of the DMA corresponding to CSB3 writes the transposition result obtained at the previous moment to HBM, which is a CDTE-copy operation. The above operation is a ping operation.

[0086] At the second moment, the following operations are performed: HBM writes data to the pong module of the DMA corresponding to CSB0, which is a CDTE-copy operation; the ping module of the DMA corresponding to CSB0 performs a transposition operation on the data written at the previous moment and writes it to the ping module of the DMA corresponding to CSB1. The pong module of the DMA corresponding to CSB1 performs a transposition operation on the data written at the previous moment and writes it to the pong module of the DMA corresponding to CSB3. The ping module of the DMA corresponding to CSB3 writes the transposition result obtained at the previous moment to the HBM, which is a CDTE-copy operation. The above operation is a pong operation.

[0087] The storage levels to which the DMAs corresponding to the above CSB0, CSB1 and CSB3 belong may be the same level or different levels.

[0088] In the above implementation, after the DMA is grouped, the tensor data is transposed in parallel in different groups in a ping-pong operation manner, thereby improving the transposition efficiency.

[0089] Figure 4 Schematic diagram of the structure of the transposed vector processing device provided in the embodiment of the present application. Figure 4 As shown, the transposed vector processing device is applied to a general computing unit and includes: a mapping module 21, a computing module 22, a searching module 23 and a transposing module 24.

[0090] A mapping module 21, used for mapping the transposition operation performed by each DMA kernel according to a transposition vector, wherein the transposition vector is used for transposing tensor data;

[0091] A calculation module 22, configured to determine a calculation cost of the DMA kernel according to the amount of calculation corresponding to the transposition operation and the calculation capability of the DMA kernel;

[0092] A search module 23, configured to search according to the transposition operations mapped by the multiple DMA cores and the operation cost of each DMA core to determine a target transposition path;

[0093] The transposition module 24 is used to control the DMA core to perform transposition processing on the tensor data according to the target transposition path.

[0094] In some embodiments, the search module 23 is used to:

[0095] Constructing a directed graph, wherein the nodes in the directed graph are DMA kernels, each node corresponds to a transposition operation of a mapping; the operation cost of the DMA kernel is used as the distance between the nodes;

[0096] Determine a starting point and an end point in the directed graph according to the transposed vector;

[0097] The shortest path is searched in the directed graph according to the starting point and the end point, and the shortest path is used as a target transposed path.

[0098] In some embodiments, a merging module is further included, for performing dimension merging according to consecutive dimensions in the transposed vector before mapping the transposed operation performed by each DMA core according to the transposed vector;

[0099] Correspondingly, the mapping module 21 is used to map the transposition operation performed by each DMA core according to the combined transposition vector.

[0100] In some embodiments, a splitting module is further included, which is used to split according to the size of the dimension in the merged transposed vector after the dimension is merged according to the continuous dimension in the transposed vector;

[0101] Correspondingly, the mapping module 21 is used to map the transposition operation performed by each DMA core according to the split transposition vectors.

[0102] In some embodiments, the merge module is used to:

[0103] If the transposed vector includes multiple continuous dimensions, merge the multiple continuous dimensions into one merged dimension;

[0104] The merged transposed vector is determined according to the merged dimension and the remaining dimensions, where the remaining dimensions are dimensions other than the multiple continuous dimensions in the first transposed vector.

[0105] In some embodiments, the merge module is used to:

[0106] Determine the merged dimension according to the product corresponding to multiple continuous dimensions;

[0107] Using the lowest dimension identifier of the multiple continuous dimensions as a merged dimension identifier;

[0108] If the original identification values ​​of the remaining dimensions are greater than the merged dimension identification, new identifications of the remaining dimensions are determined according to the number of continuous dimensions of the multiple continuous dimensions and the original identification values ​​of the remaining dimensions;

[0109] The merged dimension and the remaining dimensions form a merged transposed vector.

[0110] In some embodiments, the split module is used to:

[0111] If the size of the target dimension in the combined transposed vector is greater than a preset upper threshold, the target dimension is split to obtain a split transposed vector, where the target dimension is any dimension in the transposed vector;

[0112] If the size of the target dimension in the merged transposed vector is less than or equal to the preset upper limit threshold, the splitting of the merged transposed vector is canceled.

[0113] In some embodiments, the split module is used to:

[0114] Splitting the target dimension into multiple dimensions;

[0115] If the data size of the target dimension is smaller than the preset lower limit threshold, supplementary data is found from other dimensions and the supplementary data is added to the target dimension so that the size of the target dimension is larger than the preset lower limit threshold. The target dimension is any dimension in the transposed vector.

[0116] In some embodiments, the transposition module 24 is used to:

[0117] The tensor data is transposed in parallel according to the DMA cores included in the target transposition path.

[0118] In some embodiments, the transposition module 24 performs the transposition of the tensor data in parallel according to the DMA cores included in the target transposition path, including:

[0119] Grouping the DMA cores included in the target transposition path;

[0120] In each group, the tensor data is transposed in parallel according to the ping-pong operation.

[0121] In some embodiments, the transposition module 24 groups the DMA cores included in the target transposition path, including:

[0122] Grouping the DMA cores included in the target transposition path according to their computing power, so that the computing power of the DMA cores in each group matches each other;

[0123] The DMA cores in the group are divided into a ping operation core that performs a ping operation and a pong operation core that performs a pong operation;

[0124] The transposition operation corresponding to each ping operation kernel and each pong operation kernel in the group is configured according to the transposition operation mapped by the DMA kernel.

[0125] In some embodiments, the transposition module 24 performs parallel transposition of the tensor data according to a ping-pong operation, including:

[0126] At the first moment, all ping operation kernels in the group are called to perform the transposition operation corresponding to the ping operation;

[0127] At a second moment, all pong operation kernels in the group are called to perform a transposition operation corresponding to the pong operation; the first moment and the second moment are moments that occur cyclically according to a preset period.

[0128] The transposition vector processing device provided by the present application includes a mapping module 21, which is used to map the transposition operation performed by each DMA kernel according to the transposition vector, and the transposition vector is used to transpose the tensor data; a calculation module 22, which is used to determine the operation cost of the DMA kernel according to the operation amount corresponding to the transposition operation and the operation capacity of the DMA kernel; a search module 23, which is used to search according to the transposition operations mapped by multiple DMA kernels and the operation cost of each DMA kernel to determine the target transposition path; and a transposition module 24, which is used to control the DMA kernel to transpose the tensor data according to the target transposition path. Compared with the current inability to efficiently realize complete tensor transposition through DMA or MAC array, the present application can determine the target transposition path according to the computing power of each DMA, and complete the transposition vector containing multiple transposition steps through the transposition operations mapped by each DMA on the target transposition path, thereby improving the transposition efficiency of the tensor. The target transposition path can complete the transposition of the tensor with the highest performance, realize the hardware while operating at high performance, and complete the transposition of the tensor, thereby improving the bandwidth utilization of the device.

[0129] The transposed vector processing device provided in the embodiment of the present application can be used to execute the technical solution of the transposed vector processing method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0130] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the transposition module 24 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above transposition module 24. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0131] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device may include: a transceiver 31 , a processor 32 , and a memory 33 .

[0132] The processor 32 executes the computer execution instructions stored in the memory, so that the processor 32 executes the scheme in the above embodiment. The processor 32 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0133] The memory 33 is connected to the processor 32 via a system bus and completes communication between them. The memory 33 is used to store computer program instructions.

[0134] The transceiver 31 may be used to perform data exchange with a client.

[0135] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0136] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the transposed vector processing method in the above embodiment.

[0137] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the transposed vector processing method of the above embodiment.

[0138] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the transposed vector processing method in the above embodiment can be implemented.

[0139] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0140] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A transposed vector processing method, characterized in that: Applicable to general computing units, including: Mapping the transpose operation performed by each DMA core according to a transpose vector, which is used to transpose the tensor data; Determining the computation cost of the DMA kernel according to the computation amount corresponding to the transposition operation and the computation capability of the DMA kernel; Searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine a target transposition path; The DMA core is controlled to perform transposition processing on the tensor data according to the target transposition path.

2. The method according to claim 1, characterized in that The target transposition path is determined by searching based on the transposition operations mapped by multiple DMA cores and the operation cost of each DMA core, including: Constructing a directed graph, wherein the nodes in the directed graph are DMA kernels, each node corresponds to a transposition operation of a mapping; the operation cost of the DMA kernel is used as the distance between the nodes; Determine a starting point and an end point in the directed graph according to the transposed vector; The shortest path is searched in the directed graph according to the starting point and the end point, and the shortest path is used as a target transposed path.

3. The method according to claim 1, characterized in that Before mapping the transpose operation performed by each DMA core according to the transpose vector, it also includes: Merge dimensions based on consecutive dimensions in the transposed vector; Accordingly, the transposition operation performed by each DMA core is mapped according to the transposition vector, including: The transpose operation performed by each DMA core is mapped according to the combined transpose vector.

4. The method according to claim 3, characterized in that After dimension merging based on consecutive dimensions in the transposed vector, this also includes: Split according to the size of the dimension in the merged transposed vector; Accordingly, the transposition operation performed by each DMA core is mapped according to the transposition vector, including: The transpose operation performed by each DMA core is mapped according to the split transpose vector.

5. The method according to claim 3, characterized in that: Dimension concatenation is performed based on consecutive dimensions in the transposed vector, including: If the transposed vector includes multiple continuous dimensions, merge the multiple continuous dimensions into one merged dimension; The merged transposed vector is determined according to the merged dimension and the remaining dimensions, where the remaining dimensions are dimensions other than the multiple continuous dimensions in the first transposed vector.

6. The method according to claim 5, characterized in that Merging the multiple continuous dimensions into a merged dimension, including: Determine the merged dimension according to the product corresponding to multiple continuous dimensions; Accordingly, the merged transposed vector is determined according to the merged dimension and the remaining dimensions, including: Using the lowest dimension identifier of the multiple continuous dimensions as a merged dimension identifier; If the original identification values ​​of the remaining dimensions are greater than the merged dimension identification, new identifications of the remaining dimensions are determined according to the number of continuous dimensions of the multiple continuous dimensions and the original identification values ​​of the remaining dimensions; The merged dimension and the remaining dimensions form a merged transposed vector.

7. The method according to claim 4, characterized in that The split is performed according to the size of the dimension in the merged transposed vector, including: If the size of the target dimension in the combined transposed vector is greater than a preset upper threshold, the target dimension is split to obtain a split transposed vector, where the target dimension is any dimension in the transposed vector; If the size of the target dimension in the merged transposed vector is less than or equal to the preset upper limit threshold, the splitting of the merged transposed vector is canceled.

8. The method according to claim 7, characterized in that The target dimension is split to obtain a split transposed vector, including: Splitting the target dimension into multiple dimensions; If the data size of the target dimension is smaller than the preset lower limit threshold, supplementary data is found from other dimensions and the supplementary data is added to the target dimension so that the size of the target dimension is larger than the preset lower limit threshold. The target dimension is any dimension in the transposed vector.

9. The method according to claim 1, characterized in that: Controlling the DMA kernel to perform transposition processing on the tensor data according to the target transposition path includes: The tensor data is transposed in parallel according to the DMA cores included in the target transposition path.

10. The method according to claim 9, characterized in that According to the DMA cores included in the target transposition path, transposition of tensor data is performed in parallel, including: Grouping the DMA cores included in the target transposition path; In each group, the tensor data is transposed in parallel according to the ping-pong operation.

11. The method according to claim 10, characterized in that The DMA cores included in the target transposition path are grouped, including: Grouping the DMA cores included in the target transposition path according to their computing power, so that the computing power of the DMA cores in each group matches each other; The DMA cores in the group are divided into a ping operation core that performs a ping operation and a pong operation core that performs a pong operation; The transposition operation corresponding to each ping operation kernel and each pong operation kernel in the group is configured according to the transposition operation mapped by the DMA kernel.

12. The method according to claim 11, characterized in that Transpose tensor data in parallel using ping-pong operations, including: At the first moment, all ping operation kernels in the group are called to perform the transposition operation corresponding to the ping operation; At a second moment, all pong operation kernels in the group are called to perform a transpose operation corresponding to the pong operation; the first moment and the second moment are moments that appear cyclically according to a preset period.

13. A transposed vector processing device, characterized in that: Applicable to general computing units, including: A mapping module for mapping the transposition operation performed by each DMA core according to a transposition vector, wherein the transposition vector is used to transpose the tensor data; A calculation module, used for determining the operation cost of the DMA kernel according to the operation amount corresponding to the transposition operation and the operation capability of the DMA kernel; A search module, used for searching according to the transposition operations mapped by multiple DMA cores and the operation cost of each DMA core to determine a target transposition path; The transposition module is used to control the DMA kernel to perform transposition processing on the tensor data according to the target transposition path.

14. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 12 when executed by a processor.

16. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 12 when being executed by a processor.

Images