Transposed vector processing method, device, electronic device and storage medium

Abstract

The present application provides a transposition vector processing method, device, electronic device, and storage medium, relating to the field of chip technology. The method includes: mapping the transposition operation performed by each DMA core according to a transposition vector, the transposition vector being used to transpose tensor data; determining the computational cost of the DMA core according to the computational load corresponding to the transposition operation and the computational power of the DMA core; searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine a target transposition path; and controlling the DMA core to perform transposition processing on the tensor data according to the target transposition path. The target transposition path can be determined based on the computing power of each DMA, and a transposition vector containing multiple transposition steps is completed through the transposition operations mapped by each DMA on the target transposition path, thereby improving the transposition efficiency of the tensor. The target transposition path can complete the transposition of the tensor with the highest performance, enabling the hardware to complete the transposition of the tensor while operating at high performance, thereby improving the bandwidth utilization of the device.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of chip technology, and in particular to a transposed vector processing method, device, electronic device, and storage medium. Background Art

[0002] With the development of chip technology, deep learning based on chip technology has become possible.

[0003] In modern deep learning accelerators (DLAs), tensors need to be transposed during deep learning operations. Currently, two approaches exist: MAC arrays that handle two or three specific dimensions, and direct memory access (DMA) with wide buses. However, both approaches are inefficient in performing tensor transpositions. Providing a comprehensive tensor transposition solution to improve its efficiency has become a pressing issue. Summary of the Invention

[0004] The present application provides a transposed vector processing method, device, electronic device and storage medium to solve the problem of low tensor transposition efficiency in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a transposed vector processing method, which is applied to a general computing unit, including:

[0006] Map the transpose operation performed by each DMA core according to the transpose vector, which is used to transpose the tensor data;

[0007] Determining a computational cost of the DMA kernel according to an amount of computation corresponding to the transposition operation and a computational capability of the DMA kernel;

[0008] Determine the target transposition path by searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core;

[0009] The DMA core is controlled to perform transposition processing on the tensor data according to the target transposition path.

[0010] In a second aspect, an embodiment of the present application further provides a transposed vector processing device, which is applied to a general computing unit, comprising:

[0011] A mapping module for mapping the transposition operation performed by each DMA core according to a transposition vector, which is used to transpose the tensor data;

[0012] a calculation module, configured to determine a computation cost of the DMA kernel according to a computation amount corresponding to the transposition operation and a computation capability of the DMA kernel;

[0013] A search module is used to search based on the transposition operations mapped by multiple DMA cores and the operation cost of each DMA core to determine the target transposition path;

[0014] The transposition module is used to control the DMA core to perform transposition processing on the tensor data according to the target transposition path.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that it includes: a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory to implement the transposed vector processing method as shown in the embodiment of the present application.

[0018] In a fourth aspect, an embodiment of the present application further provides a storage medium comprising computer-executable instructions, wherein the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the transposed vector processing method as shown in the embodiment of the present application.

[0019] In a sixth aspect, an embodiment of the present application further provides a computer program product, characterized in that it includes a computer program, which, when executed by a processor, implements the transposed vector processing method shown in the embodiment of the present application.

[0020] The transposition vector processing method provided by the present application maps the transposition operation performed by each DMA core according to the transposition vector, and the transposition vector is used to transpose tensor data; the computational cost of the DMA core is determined according to the computational amount corresponding to the transposition operation and the computing power of the DMA core; a target transposition path is determined based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core; and the DMA core is controlled to perform transposition processing on the tensor data according to the target transposition path. Compared with the current inability to efficiently achieve complete tensor transposition through DMA or MAC arrays, the present application can determine the target transposition path according to the computing power of each DMA, and complete the transposition vector containing multiple transposition steps through the transposition operations mapped by each DMA on the target transposition path, thereby improving the transposition efficiency of the tensor. The target transposition path can complete the tensor transposition with the highest performance, enabling the hardware to complete the tensor transposition while operating at high performance, thereby improving the device bandwidth utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a transposed vector processing method provided in an embodiment of the present application;

[0022] Figure 2This is a schematic diagram of DMA kernel node mapping provided in an embodiment of the present application;

[0023] Figure 3 2 is a schematic diagram of a group-based ping-pong operation provided in an embodiment of the present application;

[0024] Figure 4 Schematic diagram of the structure of the transposed vector processing device provided in an embodiment of the present application;

[0025] Figure 5 It is a structural diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0027] With the development of chip technology, deep learning based on chip technology has become possible.

[0028] In modern deep learning accelerators (DLAs), tensors need to be transposed during deep learning operations. Currently, two approaches exist: MAC arrays that handle two or three specific dimensions, and wide-bus Direct Memory Access (DMA). However, MAC arrays require the lowest memory location to meet dimensional requirements, while wide-bus DMA requires the lowest dimension of a single transfer to match the bus width as closely as possible.

[0029] However, the above two methods are inefficient in the process of tensor transposition. How to provide a complete tensor transposition solution and improve the efficiency of tensor transposition has become an urgent problem to be solved.

[0030] Data Terminal Equipment (DTE) is a device used for data transmission and transformation. During data transmission, it performs operations such as slicing, deslicing, and reshaping to transform the data into the shape required by the computing engine. DTE data can be represented using five dimensions, with the highest dimension used for loop control. The DTE reshape operation corresponds to transpose, and the permute parameter can be used to express the dimensional mapping from the original shape to the target shape.

[0031] Due to hardware implementation cost constraints, DTE is incomplete in terms of functionality and performance. For example, a single DTE data entry is represented using five dimensions, with the highest dimension only being loopable, and each dimension having a maximum shape limit. Without considering the additional performance penalties of the AXI bus, DTE bandwidth utilization can be expressed as: the minimum dimension size of the target shape divided by the bus width. This means that DTE operations must maintain a minimum dimension size. If the hardware fails to meet these requirements, the efficiency of deformation operations will be limited.

[0032] The following specific embodiments describe in detail the technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0033] It should be noted that in the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary and are intended only to illustrate the feasibility of implementing the technical solutions of this application. This does not mean that the applicant has or will necessarily use such solutions. The embodiments of this application will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 This is a flow chart of the transposed vector processing method provided in an embodiment of the present application. This embodiment is applicable to the case where tensor transposition is performed through multiple DMAs in a chip system. This method can be executed by electronic devices such as system-on-chip (SoC). Figure 1 As shown, the transposed vector processing method provided by this application includes:

[0035] Step S101 : Mapping the transposition operation performed by each DMA core according to a transposition vector, where the transposition vector is used to transpose tensor data.

[0036] For example, the transposed vector of a tensor can be expressed as torch.permute(input,dims)→Tensor. The transposed vector indicates that the input tensor input is used as the initial dimension and the dimension is transformed according to the deformation method represented by dims. The dimension memory arrangement of the result of the transpose transpose, the size of dims can be 64 bits. Assuming there are two dimensions, dims are 0 and 1 respectively, the transposed vector can be permute(input,{0,1}). At this time, the transposed vector indicates that no changes are made and the dimension is the same as before. If the transposed vector is permute(input,{1,0}), what is obtained is the transpose of the original matrix, that is, the positions of dimension 0 and dimension 1 are swapped.

[0037] Each DMA node maps a corresponding transpose operation, which is used to transpose two dimensions of a tensor. In this application scenario, it is necessary to map the transpose operation performed by each DMA to determine the transpose operation performed by each DMA.

[0038] You can pre-configure a DMA to perform fixed-dimensional transposition operations. You can also map the transposition operations performed by each DMA to a transposition vector. By mapping the DMA transposition operations, you can transform the transposition vector into a combination of transposition operations performed by a limited number of DMAs, each performing two-dimensional transposition operations.

[0039] Optionally, before step S101, mapping the transposition operation performed by each DMA core according to the transposition vector, the method further includes: merging dimensions according to consecutive dimensions in the transposition vector. Accordingly, mapping the transposition operation performed by each DMA core according to the transposition vector includes: mapping the transposition operation performed by each DMA core according to the merged transposition vector.

[0040] Compression algorithms can be used to merge multiple consecutive dimensions. The compression algorithm can be Byte Pair Encoding (BPE), which simplifies the transposed vectors in byte units. Consecutive dimensions are those that do not participate in the transposition. Compressed consecutive dimensions do not participate in the dimension conversion, so the merge operation does not affect the transposition operation.

[0041] The above implementation can merge continuous dimensions in the transposed vector, thereby simplifying the transposed vector, reducing the transposition complexity, and improving the tensor transposition efficiency and device performance.

[0042] Optionally, dimension merging is performed based on consecutive dimensions in the transposed vector, which can be implemented as follows:

[0043] If the transposed vector includes multiple continuous dimensions, the multiple continuous dimensions are merged into a merged dimension; and the merged transposed vector is determined according to the merged dimension and the remaining dimensions, where the remaining dimensions are dimensions in the first transposed vector other than the multiple continuous dimensions.

[0044] Specifically, merging the multiple continuous dimensions into a merged dimension can be implemented by determining the merged dimension based on the product of the multiple continuous dimensions. Accordingly, determining the merged transposed vector based on the merged dimension and the remaining dimensions can be implemented by using the lowest dimension identifier of the multiple continuous dimensions as the merged dimension identifier. If the original identifier values ​​of the remaining dimensions are greater than the merged dimension identifier, new identifiers for the remaining dimensions are determined based on the number of continuous dimensions in the multiple continuous dimensions and the original identifier values ​​of the remaining dimensions. The merged dimension and the remaining dimensions constitute the merged transposed vector.

[0045] For example, if the transposed vector is (1,0,2,3,4,5), only the 0th and 1st dimensions are transformed; the 2nd, 3rd, 4th, and 5th dimensions are not involved in the transposition. Therefore, the 2nd, 3rd, 4th, and 5th dimensions can be merged into a new 2' dimension to simplify the transposed vector. The merged transposed vector is (1,0,2'), where the new 2' dimension is called the merged dimension, and the 0th and 1st dimensions are the remaining dimensions. Dimension merging effectively reduces the number of dimensions in the transposed vector.

[0046] When merging dimensions, you can scan the continuous dims in permute. If there are multiple adjacent dimensions, select the smaller value A as the new dimension, and multiply the shapes corresponding to the multiple dimensions as the new shape. In permute, all values ​​greater than A are subtracted by 1.

[0047] You can merge only every two consecutive dimensions or every multiple consecutive dimensions to avoid saturating the merged dimension data. In the above example, after merging every two consecutive dimensions of the transposed vector (1,0,2,3,4,5), the resulting merged transposed vector is (1,0,2',3').

[0048] The above implementation method can renumber the dimension representations while merging the dimensions, without changing the mapped dimension operation, thereby simplifying the transposed vector and improving the transposition efficiency of the tensor.

[0049] Furthermore, after merging the dimensions according to the continuous dimensions in the transposed vector, the following steps are further included:

[0050] The combined transposed vector is split according to the size of the dimension. Accordingly, the transposed operation performed by each DMA core is mapped according to the transposed vector, including: mapping the transposed operation performed by each DMA core according to the split transposed vector.

[0051] In order to ensure that the size of each dimension matches the amount of data that can be processed by DMA, after merging multiple dimensions, the merged dimensions can be split.

[0052] Optionally, splitting is performed based on the size of the dimension in the merged transposed vector, which can be implemented as follows:

[0053] If the size of the target dimension in the merged transposed vector is greater than the preset upper limit threshold, the target dimension is split to obtain a split transposed vector, where the target dimension is any dimension in the transposed vector; if the size of the target dimension in the merged transposed vector is less than or equal to the preset upper limit threshold, the splitting of the merged transposed vector is canceled.

[0054] The DMA involved in the vector transposition may have hardware limitations, such as being unable to process a single dimension larger than 65525 bits. In this case, the oversized dimension needs to be split.

[0055] The preset upper threshold value may be 65525 bits. The preset upper threshold value may vary depending on different DMAs and is not limited here. The dimension to be split may be a merged dimension or another unmerged dimension. As long as the amount of data in the dimension is greater than the preset upper threshold value, the target dimension is split to obtain a transposed vector after the split.

[0056] When splitting, we are not limited to factorization. We can decompose into values ​​of similar size to ensure that the execution time of each sub-unit in the pipeline is similar.

[0057] The above implementation method can split a dimension when the amount of data in a certain dimension after merging is too large. The split dimensional data can match the DMA bandwidth, thereby improving performance and improving the efficiency of the tensor transpose machine.

[0058] Specifically, the target dimension is split to obtain the transposed vector after splitting, which can be implemented as follows:

[0059] Split the target dimension into multiple dimensions. If the data size of the target dimension is less than a preset lower threshold, search for supplementary data from other dimensions and add the supplementary data to the target dimension so that the size of the target dimension exceeds the preset lower threshold. The target dimension is any dimension in the transposed vector.

[0060] If the lowest dimension is less than 128 bits, you need to prepare appropriate dimensions for subsequent DMA decomposition. Based on the size of the lowest dimension, find a dimension before and after the dimension and split it into a high-performance size after merging with the target dimension.

[0061] The above implementation can supplement the data in the dimension when the data size of the split dimension is less than the preset lower limit threshold, thereby avoiding DMA bandwidth waste, improving DMA bandwidth utilization, and thus improving performance and tensor transposition efficiency.

[0062] Step S102 : determining the computation cost of the DMA core according to the computation amount corresponding to the transposition operation and the computation capability of the DMA core.

[0063] Step S103 : searching based on the transposition operations mapped by the multiple DMA cores and the computational cost of each DMA core to determine a target transposition path.

[0064] Optionally, searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine a target transposition path can be implemented in the following manner:

[0065] A directed graph is constructed, wherein nodes in the directed graph are DMA kernels, and each node corresponds to a mapped transposition operation; the operation cost of the DMA kernel is used as the distance between the nodes; a starting point and an end point in the directed graph are determined according to the transposition vector; and a shortest path is searched in the directed graph according to the starting point and the end point, and the shortest path is used as a target transposition path.

[0066] Different DMA cores have different bandwidths and belong to different memory classes. A DMA core can be located between the L3 and L2 caches, between the L2 and L1 caches, or between the L1 cache and the core. The computational cost required to perform a DMA transpose operation—the time required to perform the transpose of tensor data—can be determined based on the DMA core bandwidth and memory class. The computational cost of each DMA core is calculated based on the DMA bandwidth and the memory class to which the DMA belongs.

[0067] In a directed graph, each DMA core acts as a node, and the connections between nodes are determined by the actual connections of the DMA hardware. The length of an edge is determined by the computational cost of the node at the edge's starting point, thus constructing a directed graph. In a directed graph, the shortest path between the starting point and the end point is found. The shortest path is the path with the shortest edge length.

[0068] Figure 2 A schematic diagram of DMA kernel node mapping provided by an embodiment of the present invention is shown as follows: Figure 2As shown, exemplarily, the directed graph includes four nodes, each of which is mapped with a transpose operation.

[0069] DMA can support five-dimensional operations, but the highest dimension cannot be changed. For permute0-1-2-3-4, 4-3-2-1-0, 4-1-2-3-0, and 0-1-2-4-3, you can see that there is no arrow from node 0-1-2-3-4 to 4-3-2-1-0, indicating that these two states cannot be directly completed through a DMA transaction and require additional intermediate nodes.

[0070] A search using wavefront detection can be modeled by setting each transposed state to a state and finding a state transition path. This state transition path depends on the dataflow implementation. If defined as a single DMA, the existence of the path is equivalent to being able to be completed by a single DMA after dim merge and dim split operations, as well as other possible operations.

[0071] The above implementation method can quickly and accurately search for the target transposition path with the minimum computational cost through the directed graph, thereby quickly determining the optimal transposition path and improving the transposition efficiency of the tensor.

[0072] Step S104 : controlling the DMA core to perform transposition processing on the tensor data according to the target transposition path.

[0073] Optionally, controlling the DMA core to perform transposition processing on the tensor data according to the target transposition path includes:

[0074] Transposition of tensor data is performed in parallel according to the DMA cores included in the target transposition path.

[0075] Optionally, performing the transposition of tensor data in parallel according to the DMA core included in the target transposition path may be implemented as follows:

[0076] The DMA cores included in the target transposition path are grouped; and in each group, tensor data are transposed in parallel according to a ping-pong operation.

[0077] Figure 3 A schematic diagram of a ping-pong operation based on grouping provided in an embodiment of the present invention is shown in FIG. Figure 3 shown.

[0078] Specifically, grouping the DMA cores included in the target transposition path includes:

[0079] The DMA cores included in the target transposition path are grouped according to their computing power so that the computing power of the DMA cores in each group matches each other; the DMA cores in the group are divided into ping operation cores that perform ping operations and pong operation cores that perform pong operations; and the transposition operations corresponding to each ping operation core and each pong operation core in the group are configured according to the transposition operations mapped by the DMA cores.

[0080] DMA may include CDMA between the L2 cache and the L1 cache and SDMA between the L3 cache and the L2 cache.

[0081] Optionally, the ratio of CDMA, SDMA, and instruction access modules can be 1:3:6. Considering the synchronization overhead at each level, when only CDMA and SDMA are used, the SDMA modules are divided into two groups, with a design ratio of 1:2:0. The two SDMA modules can share the same local memory, achieving a relatively perfect balance between storage and transfer.

[0082] It should be noted that the above ratios and grouping methods are only examples of the final implementation, and various device grouping options can be selected according to needs. For example, in order to support deeper parallelism, you can choose 1:2:2 or use only CDMA for D2D+D2D or D2C+C2D grouping.

[0083] This approach balances the computing power of each group's DMA cores, achieving a balance between the computing power of each group. Furthermore, during ping-pong operations, different groups can perform synchronous transposition operations, improving the parallel efficiency of transposition operations.

[0084] Specifically, tensor data is transposed in parallel according to the ping-pong operation, including: at a first moment, calling all ping-pong operation kernels in the group to execute the transposition operation corresponding to the ping-pong operation; at a second moment, calling all pong operation kernels in the group to execute the transposition operation corresponding to the pong operation; the first moment and the second moment are moments that occur cyclically according to a preset period.

[0085] After obtaining two groups through grouping, assuming that the group includes High Bandwidth Memory (HBM) and the configuration space bus (CSB) connected to multiple different DMAs, the following operations are performed at the first moment: the HBM writes data to the ping module of the DMA corresponding to CSB0, which is a CDTE-copy operation; the pong module of the DMA corresponding to CSB0 transposes the data written at the previous moment and writes it to the pong module of the DMA corresponding to CSB1. The ping module of the DMA corresponding to CSB1 transposes the data written at the previous moment and writes it to the ping module of the DMA corresponding to CSB3. The pong module of the DMA corresponding to CSB3 writes the transposed result obtained at the previous moment to the HBM, which is a CDTE-copy operation. The above operations constitute one ping operation.

[0086] At the second moment, the following operations are performed: the HBM writes data to the pong module of the DMA corresponding to CSB0 (a CDTE-copy operation). The ping module of the DMA corresponding to CSB0 transposes the data written previously and writes it to the ping module of the DMA corresponding to CSB1. The pong module of the DMA corresponding to CSB1 transposes the data written previously and writes it to the pong module of the DMA corresponding to CSB3. The ping module of the DMA corresponding to CSB3 writes the transposed result obtained previously to the HBM (a CDTE-copy operation). The above operations constitute one pong operation.

[0087] The storage levels to which the DMAs corresponding to the above CSB0, CSB1 and CSB3 belong may be the same level or different levels.

[0088] In the above implementation, after the DMAs are grouped, the tensor data in different groups are transposed in parallel in a ping-pong operation manner, thereby improving the transposition efficiency.

[0089] Figure 4 This is a schematic diagram of the structure of the transposed vector processing device provided in the embodiment of the present application. Figure 4 As shown, the transposed vector processing device is applied to a general computing unit and includes: a mapping module 21, a computing module 22, a searching module 23 and a transposing module 24.

[0090] a mapping module 21 for mapping the transposition operation performed by each DMA core according to a transposition vector, the transposition vector being used to transpose the tensor data;

[0091] a calculation module 22, configured to determine a computation cost of the DMA kernel according to the computation amount corresponding to the transposition operation and the computation capability of the DMA kernel;

[0092] A search module 23 is configured to search based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine a target transposition path;

[0093] The transposition module 24 is configured to control the DMA core to perform transposition processing on the tensor data according to the target transposition path.

[0094] In some embodiments, the search module 23 is used to:

[0095] Constructing a directed graph, wherein the nodes in the directed graph are DMA kernels, each node corresponds to a mapping transpose operation; the operation cost of the DMA kernel is used as the distance between the nodes;

[0096] Determine a starting point and an end point in the directed graph according to the transposed vector;

[0097] A shortest path is searched in the directed graph according to the starting point and the end point, and the shortest path is used as a target transposed path.

[0098] In some embodiments, a merging module is further included for performing dimension merging according to consecutive dimensions in the transposed vector before mapping the transposed operation performed by each DMA core according to the transposed vector;

[0099] Correspondingly, the mapping module 21 is used to map the transposition operation performed by each DMA core according to the combined transposition vector.

[0100] In some embodiments, a splitting module is further included, for splitting according to the size of the dimension in the merged transposed vector after merging the dimensions according to the continuous dimensions in the transposed vector;

[0101] Correspondingly, the mapping module 21 is used to map the transposition operation performed by each DMA core according to the split transposition vectors.

[0102] In some embodiments, the merge module is configured to:

[0103] If the transposed vector includes multiple continuous dimensions, merge the multiple continuous dimensions into one merged dimension;

[0104] A merged transposed vector is determined according to the merged dimension and the remaining dimensions, where the remaining dimensions are dimensions other than the multiple continuous dimensions in the first transposed vector.

[0105] In some embodiments, the merge module is configured to:

[0106] Determine the merged dimension based on the product corresponding to multiple continuous dimensions;

[0107] Using the lowest dimension identifier of the multiple consecutive dimensions as a merged dimension identifier;

[0108] If the original identification values ​​of the remaining dimensions are greater than the merged dimension identification, new identifications of the remaining dimensions are determined according to the number of consecutive dimensions of the multiple consecutive dimensions and the original identification values ​​of the remaining dimensions;

[0109] The merged dimension and the remaining dimensions form a merged transposed vector.

[0110] In some embodiments, the split module is used to:

[0111] If the size of the target dimension in the merged transposed vector is greater than a preset upper threshold, the target dimension is split to obtain a split transposed vector, where the target dimension is any dimension in the transposed vector;

[0112] If the size of the target dimension in the merged transposed vector is less than or equal to the preset upper threshold, the splitting of the merged transposed vector is canceled.

[0113] In some embodiments, the split module is used to:

[0114] Splitting the target dimension into multiple dimensions;

[0115] If the data size of the target dimension is smaller than the preset lower limit threshold, supplementary data is found from other dimensions and the supplementary data is added to the target dimension so that the size of the target dimension is larger than the preset lower limit threshold. The target dimension is any dimension in the transposed vector.

[0116] In some embodiments, the transposition module 24 is configured to:

[0117] Transposition of tensor data is performed in parallel according to the DMA cores included in the target transposition path.

[0118] In some embodiments, the transposition module 24 performs the transposition of the tensor data in parallel according to the DMA cores included in the target transposition path, including:

[0119] Grouping the DMA cores included in the target transposition path;

[0120] In each group, the tensor data is transposed in parallel according to the ping-pong operation.

[0121] In some embodiments, the transposition module 24 groups the DMA cores included in the target transposition path, including:

[0122] Grouping the DMA cores included in the target transposition path according to their computing power, so that the computing power of the DMA cores in each group matches each other;

[0123] The DMA cores in the group are divided into a ping operation core for performing a ping operation and a pong operation core for performing a pong operation;

[0124] The transposition operation corresponding to each ping operation kernel and each pong operation kernel in the group is configured according to the transposition operation mapped by the DMA kernel.

[0125] In some embodiments, the transposition module 24 performs parallel transposition of the tensor data according to a ping-pong operation, including:

[0126] At the first moment, all ping operation kernels in the group are called to perform the transpose operation corresponding to the ping operation;

[0127] At a second moment, all pong operation kernels in the group are called to perform a transpose operation corresponding to the pong operation; the first moment and the second moment are moments that appear cyclically according to a preset period.

[0128] The transposition vector processing device provided by this application includes a mapping module 21 for mapping the transposition operations performed by each DMA core according to a transposition vector, which is used to transpose tensor data; a calculation module 22 for determining the computational cost of the DMA core based on the computational load corresponding to the transposition operation and the computing power of the DMA core; a search module 23 for searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core to determine a target transposition path; and a transposition module 24 for controlling the DMA core to transpose tensor data according to the target transposition path. Compared to the current inability to efficiently implement complete tensor transposition using DMA or MAC arrays, this application can determine a target transposition path based on the computing power of each DMA core. The transposition operations mapped by each DMA on the target transposition path complete a transposition vector containing multiple transposition steps, thereby improving tensor transposition efficiency. The target transposition path can complete tensor transposition with the highest performance, enabling hardware to complete tensor transposition while maintaining high performance, thereby improving device bandwidth utilization.

[0129] The transposed vector processing device provided in the embodiment of the present application can be used to execute the technical solution of the transposed vector processing method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0130] It should be noted that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity or physically separated. Furthermore, these modules can be implemented entirely in software called by a processing element, or entirely in hardware. Alternatively, some modules can be implemented in software called by a processing element, while others can be implemented in hardware. For example, transposition module 24 can be a separate processing element, or it can be integrated into a chip of the above device. Furthermore, it can be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above transposition module 24. The implementation of other modules is similar. Furthermore, these modules can be fully or partially integrated together, or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, the steps of the above method or the above modules can be completed by hardware integrated logic circuits in the processor element or by software instructions.

[0131] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 5 As shown, the electronic device may include: a transceiver 31 , a processor 32 , and a memory 33 .

[0132] The processor 32 executes the computer-executable instructions stored in the memory, so that the processor 32 implements the solutions in the above embodiments. The processor 32 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0133] The memory 33 is connected to the processor 32 via a system bus and communicates with the processor 32. The memory 33 is used to store computer program instructions.

[0134] The transceiver 31 can be used to exchange data with the client.

[0135] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. The system bus can be divided into an address bus, a data bus, a control bus, and so on. For ease of illustration, the figure shows only one thick line, but this does not imply that there is only one bus or only one type of bus. Transceivers are used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and non-volatile memory.

[0136] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the transposed vector processing method in the above embodiment.

[0137] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the transposed vector processing method of the above embodiment.

[0138] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the transposed vector processing method in the above embodiment.

[0139] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0140] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A transposed vector processing method, characterized in that: Applicable to general computing units, including: Map the transpose operation performed by each DMA core according to the transpose vector, which is used to transpose the tensor data; Determining a computational cost of the DMA kernel according to an amount of computation corresponding to the transposition operation and a computational capability of the DMA kernel; Determine the target transposition path by searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core; The DMA core is controlled to perform transposition processing on the tensor data according to the target transposition path.

2. The method according to claim 1, characterized in that The target transposition path is determined by searching based on the transposition operations mapped by multiple DMA cores and the computational cost of each DMA core, including: Constructing a directed graph, wherein the nodes in the directed graph are DMA kernels, each node corresponds to a mapping transpose operation; the operation cost of the DMA kernel is used as the distance between the nodes; Determine a starting point and an end point in the directed graph according to the transposed vector; A shortest path is searched in the directed graph according to the starting point and the end point, and the shortest path is used as a target transposed path.

3. The method according to claim 1, characterized in that Before mapping the transpose operation performed by each DMA core according to the transpose vector, it also includes: Merge dimensions based on consecutive dimensions in the transposed vector; Accordingly, the transposition operation performed by each DMA core is mapped according to the transposition vector, including: The transpose operation performed by each DMA core is mapped according to the combined transpose vector.

4. The method according to claim 3, characterized in that After dimension concatenation based on consecutive dimensions in the transposed vector, this also includes: Split according to the size of the dimension in the merged transposed vector; Accordingly, the transposition operation performed by each DMA core is mapped according to the transposition vector, including: The transpose operation performed by each DMA core is mapped according to the split transpose vector.

5. The method according to claim 3, characterized in that Dimension merging is performed based on consecutive dimensions in the transposed vector, including: If the transposed vector includes multiple continuous dimensions, merge the multiple continuous dimensions into one merged dimension; A merged transposed vector is determined according to the merged dimension and the remaining dimensions, where the remaining dimensions are dimensions other than the multiple continuous dimensions in the first transposed vector.

6. The method according to claim 5, characterized in that Merging the multiple continuous dimensions into a merged dimension, including: Determine the merged dimension based on the product corresponding to multiple continuous dimensions; Accordingly, the merged transposed vector is determined according to the merged dimension and the remaining dimensions, including: Using the lowest dimension identifier of the multiple consecutive dimensions as a merged dimension identifier; If the original identification values ​​of the remaining dimensions are greater than the merged dimension identification, new identifications of the remaining dimensions are determined according to the number of consecutive dimensions of the multiple consecutive dimensions and the original identification values ​​of the remaining dimensions; The merged dimension and the remaining dimensions form a merged transposed vector.

7. The method according to claim 4, characterized in that Split according to the size of the dimension in the merged transposed vector, including: If the size of the target dimension in the merged transposed vector is greater than a preset upper threshold, the target dimension is split to obtain a split transposed vector, where the target dimension is any dimension in the transposed vector; If the size of the target dimension in the merged transposed vector is less than or equal to the preset upper threshold, the splitting of the merged transposed vector is canceled.

8. The method according to claim 7, characterized in that The target dimension is split to obtain a split transposed vector, including: Splitting the target dimension into multiple dimensions; If the data size of the target dimension is smaller than the preset lower limit threshold, supplementary data is found from other dimensions and the supplementary data is added to the target dimension so that the size of the target dimension is larger than the preset lower limit threshold. The target dimension is any dimension in the transposed vector.

9. The method according to claim 1, characterized in that Controlling the DMA core to perform transposition processing on the tensor data according to the target transposition path includes: Transposition of tensor data is performed in parallel according to the DMA cores included in the target transposition path.

10. The method according to claim 9, characterized in that Executing transposition of tensor data in parallel according to the DMA cores included in the target transposition path includes: Grouping the DMA cores included in the target transposition path; In each group, the tensor data is transposed in parallel according to the ping-pong operation.

11. The method according to claim 10, characterized in that The DMA cores included in the target transposition path are grouped, including: Grouping the DMA cores included in the target transposition path according to their computing power, so that the computing power of the DMA cores in each group matches each other; The DMA cores in the group are divided into a ping operation core for performing a ping operation and a pong operation core for performing a pong operation; The transposition operation corresponding to each ping operation kernel and each pong operation kernel in the group is configured according to the transposition operation mapped by the DMA kernel.

12. The method according to claim 11, characterized in that Performs parallel transposition of tensor data using ping-pong operations, including: At the first moment, all ping operation kernels in the group are called to perform the transpose operation corresponding to the ping operation; At a second moment, all pong operation kernels in the group are called to perform a transpose operation corresponding to the pong operation; the first moment and the second moment are moments that appear cyclically according to a preset period.

13. A transposed vector processing device, characterized in that: Applicable to general computing units, including: A mapping module for mapping the transposition operation performed by each DMA core according to a transposition vector, which is used to transpose the tensor data; a calculation module, configured to determine a computation cost of the DMA kernel according to a computation amount corresponding to the transposition operation and a computation capability of the DMA kernel; A search module is used to search based on the transposition operations mapped by multiple DMA cores and the operation cost of each DMA core to determine the target transposition path; The transposition module is used to control the DMA core to perform transposition processing on the tensor data according to the target transposition path.

14. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 12 when executed by a processor.

16. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 12 when being executed by a processor.

Images