Data processing method and apparatus, and computing device

Abstract

A data processing method, comprising: acquiring a source matrix stored in a source memory; performing a transpose operation on the source matrix to obtain a transposed matrix; splitting the transposed matrix to obtain a plurality of sub-matrices; sequentially performing a zero-padding operation on the plurality of sub-matrices, and caching in a storage unit a zero-padded sub-matrix obtained upon completion of each zero-padding operation; and generating a target matrix on the basis of the storage unit, and writing the target matrix to a target memory. In the method of embodiments of the present application, the source matrix can be read from the source memory in a single operation, the target matrix is generated by performing operations such as transposing the source matrix, concatenating the transposed matrix with a zero matrix, and then transposing the concatenated matrix, and the target matrix is written to the target memory in a single operation. Therefore, the copy between the source memory and the target memory uses a larger data packet, eliminating the need to split a source matrix into a plurality of small data packets by rows for transmission, thereby reducing waste of data bandwidth.

Description

A data processing method, apparatus and computing device

[0001] This application claims priority to Chinese Patent Application No. 202510339922.3, filed on March 20, 2025, entitled “A Data Processing Method, Apparatus and Computing Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of chip technology, and in particular to a data processing method, apparatus and computing device. Background Technology

[0003] Matrix multiplication, as a core computational operation in neural network models, accounts for a significant portion of computational time. Therefore, accelerating matrix multiplication has become a key issue in building computing infrastructure. Current mainstream solutions employ dedicated processors equipped with tensor units, using hardware-level parallel circuits to complete matrix operations in one go, thereby greatly improving computing power. However, such tensor units typically only support matrix calculations of limited shapes. The input matrix of a general shape must be segmented and padded to a suitable size before the hardware unit can be invoked to perform the calculation. In this process, how to efficiently perform matrix segmentation and padding transformations becomes a crucial step in optimizing overall computational efficiency. Summary of the Invention

[0004] To address the aforementioned issues, embodiments of this application provide a data processing method that allows copying between source and target memory to use larger data packets, eliminating the need to divide the source matrix into multiple small data packets for transmission, thereby reducing data bandwidth waste.

[0005] In addition, this application also provides a data processing apparatus and a computing device corresponding to the data processing method.

[0006] Therefore, the following technical solutions are adopted in the embodiments of this application:

[0007] In a first aspect, embodiments of this application provide a data processing method. The computing device includes a storage module and a vector module. The storage module includes source memory and target memory. The source memory is used to store a source matrix, and the target memory is used to store a target matrix. The vector module includes a computing unit and a storage unit. The method is executed by the computing unit and includes: obtaining the source matrix stored in the source memory; performing a row-column axis transformation operation on the source matrix to obtain a transformation matrix, wherein the row-column axis transformation operation is an operation of performing row-column transformation on each element in the matrix; splitting the transformation matrix to obtain multiple sub-matrices; sequentially performing zero-padding operation on the multiple sub-matrices, and caching the zero-padding sub-matrices after each zero-padding operation in the storage unit; determining that the storage unit generates a target matrix, and writing the target matrix into the target memory.

[0008] In this implementation, the method can read the source matrix from the source memory in one go. After performing row and column axis transformations on the source matrix, concatenating it with a zero matrix, and then performing row and column axis transformations on the concatenated matrix again, the target matrix is ​​generated and written to the target memory in one go. Therefore, the copy between the source and target memory uses larger data packets, eliminating the need to divide the source matrix into multiple small data packets for transmission, thus reducing wasted data bandwidth.

[0009] In one implementation, before performing row and column axis transformation on the source matrix to obtain the transformation matrix, the method further includes: rearranging the source matrix to obtain the rearranged matrix.

[0010] In this embodiment, the method utilizes the reshape operation to flexibly adjust the shape of the matrix, thereby adjusting the shape of the source matrix so that the shape of the source matrix can approach the shape of the target matrix after two row and column axis transformation operations.

[0011] In one implementation, a target matrix is ​​obtained by sequentially padding multiple submatrices with zero elements and caching the padded submatrices after each padding operation in a storage unit. This includes: padding a first submatric with zero elements to obtain a first padded submatric; and storing the first padded submatric in the corresponding position in the storage unit according to the position of the elements in the first padded submatric.

[0012] In this embodiment, the method caches each element in the corresponding position in the storage unit according to the position of each element in the processed zero-padding submatrix, thereby achieving the goal of caching some elements in the target matrix in the storage unit.

[0013] In one implementation, the first submatrix is ​​padded with zero elements to obtain a first zero-padded submatrix, comprising: configuring a zero matrix according to the shape of the first submatrix and the shape of the target matrix, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first submatrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first submatrix; inserting each zero element in each row of the zero matrix after each element in each row of the first submatrix to obtain a first complete submatrix; and performing a row-column axis transformation operation on the first complete submatrix to obtain the first zero-padded submatrix.

[0014] In this implementation, the method inserts zero elements into the submatrix before performing the second row-column axis transformation operation. During this insertion process, the method can perform the insertion within storage cells, thereby enabling copying between source and target memory using larger data packets.

[0015] In one implementation, the first submatrix is ​​padded with zero elements to obtain a first zero-padded submatrix, comprising: configuring a zero matrix according to the shape of the first submatrix and the shape of the target matrix, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first submatrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first submatrix; performing a row-column axis transformation operation on the first submatrix and the zero matrix to obtain a first transformed submatrix and a transformed zero matrix; and inserting each zero element in each row of the transformed zero matrix after each element in each row of the first transformed submatrix to obtain the first zero-padded submatrix.

[0016] In this embodiment, the method inserts zero elements into the submatrix after performing the second row-column axis transformation operation. Since the row-column axis transformation operation allows computation on discontinuous input data, this method processes the submatrix and zero matrix together in the storage unit, thereby enabling copying between source and target memory using larger data packets and reducing the number of storage unit copying operations.

[0017] In one embodiment, before obtaining the source matrix, the method further includes: detecting whether the shape of the source matrix is ​​larger than a set shape; when the shape of the source matrix is ​​larger than the set shape, dividing the source matrix in the row direction and / or column direction to obtain multiple sub-source matrices.

[0018] In this implementation, if the size of the source matrix does not meet the requirements, the method needs to divide it in the column or row direction, decompose the source matrix into several sub-source matrices whose sizes meet the requirements of the input matrix, thereby processing the source matrix of arbitrary shape into the size of the input matrix that meets the requirements, thereby improving the applicable scenarios of the computing chip.

[0019] In one implementation, the rearranged matrix is ​​split into multiple sub-matrices, including: detecting whether the shape of the source matrix is ​​larger than a set shape; when the shape of the source matrix is ​​larger than the set shape, splitting the rearranged matrix into a first type of sub-matrices and a second type of sub-matrices, each type of sub-matrices including at least one sub-matrice, the elements in each first type of sub-matrices and the elements in each second type of sub-matrices being distributed alternately in the rearranged matrix, the first type of sub-matrices being used to generate a first sub-target matrix smaller than or equal to the set shape, the second type of sub-matrices being used to generate a second sub-target matrix smaller than or equal to the set shape, and the target matrix including the first sub-target matrix and the second sub-target matrix.

[0020] In this embodiment, if the size of the source matrix does not meet the requirements, the method divides the source matrix during the filling process, thereby processing the source matrix of arbitrary shape into the size of the input matrix that meets the requirements, thereby improving the applicable scenarios of the computing chip.

[0021] Secondly, this application provides a data processing apparatus, comprising: a first processing module for acquiring a source matrix stored in source memory; a second processing module for performing row and column axis transformation operations on the source matrix to obtain a transformation matrix, wherein the row and column axis transformation operation is an operation of performing row and column transformation on each element in the matrix; a third processing module for splitting the transformation matrix to obtain multiple sub-matrices; a fourth processing module for sequentially performing zero-padding operations on the multiple sub-matrices and caching the zero-padding sub-matrices after each zero-padding operation in a storage unit; and a fifth processing module for determining the target matrix generated by the storage unit and writing the target matrix into the target memory.

[0022] In one implementation, the second processing module, before performing row and column axis transformation on the source matrix to obtain the transformed matrix, is also used to rearrange the source matrix to obtain the rearranged matrix.

[0023] In one embodiment, the fourth processing module is used to perform zero-padding operation on the first submatrix to obtain a first zero-padding submatrix; and to store the first zero-padding submatrix in the corresponding position in the storage unit according to the position of the elements in the first zero-padding submatrix.

[0024] In one implementation, the fourth processing module is configured to configure a zero matrix according to the shape of the first sub-matrix and the shape of the target matrix, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first sub-matrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first sub-matrix; insert the zero elements of each row of the zero matrix after each row of the first sub-matrix to obtain the first completed sub-matrix; and perform a row and column axis transformation operation on the first completed sub-matrix to obtain the first zero-padded sub-matrix.

[0025] In one implementation, the fourth processing module is configured to configure a zero matrix according to the shape of the first sub-matrix and the shape of the target matrix, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first sub-matrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first sub-matrix; perform row and column axis transformation operations on the first sub-matrix and the zero matrix to obtain a first transformed sub-matrix and a transformed zero matrix; and insert each zero element in each row of the transformed zero matrix after each element in each row of the first transformed sub-matrix to obtain a first zero-filled sub-matrix.

[0026] In one embodiment, the first processing module is further configured to detect whether the shape of the source matrix is ​​larger than a set shape before obtaining the source matrix; when the shape of the source matrix is ​​larger than the set shape, the source matrix is ​​divided in the row direction and / or column direction to obtain multiple sub-source matrices.

[0027] In one embodiment, a third processing module is used to detect whether the shape of the source matrix is ​​larger than a set shape; when the shape of the source matrix is ​​larger than the set shape, the rearranged matrix is ​​split into a first type of sub-matrix and a second type of sub-matrix, each type of sub-matrix including at least one sub-matrix, and the elements in each first type of sub-matrix and the elements in each second type of sub-matrix are distributed alternately in the rearranged matrix. The first type of sub-matrix is ​​used to generate a first sub-target matrix that is smaller than or equal to the set shape, and the second type of sub-matrix is ​​used to generate a second sub-target matrix that is smaller than or equal to the set shape. The target matrix includes the first sub-target matrix and the second sub-target matrix.

[0028] Thirdly, embodiments of this application provide a computing device, including: at least one memory; and at least one processor, the processor being configured to execute instructions stored in the memory to cause the computing device to perform the embodiments as described in the first aspect.

[0029] Fourthly, embodiments of this application provide a computer-readable storage medium including computer program instructions, which, when executed by a computing device, cause the computing device to perform various possible implementations of the first aspect.

[0030] Fifthly, embodiments of this application provide a computer program product containing instructions that, when executed by a computing device, cause the computing device to implement various possible implementations of the first aspect.

[0031] In a sixth aspect, embodiments of this application provide a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster performs the various possible implementations of the first aspect.

[0032] In a seventh aspect, embodiments of this application provide a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the various possible implementations of the first aspect.

[0033] Eighthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to implement various possible implementations of the first aspect. Attached Figure Description

[0034] The accompanying drawings used in the description of the embodiments or prior art are briefly introduced below.

[0035] Figure 1 is a schematic diagram of the row and column axis conversion operation provided in the embodiment of this application;

[0036] Figure 2 is a schematic diagram of the data processing method for filling in related technologies;

[0037] Figure 3 is a schematic diagram of the structure of a computing chip provided in an embodiment of this application;

[0038] Figure 4 is a schematic diagram of the execution flow of a data processing method provided in an embodiment of this application;

[0039] Figure 5 is a schematic diagram of the source matrix filling process provided in the embodiment of this application;

[0040] Figure 6(a) is one of the schematic diagrams of the process of filling a submatrix with zero elements provided in the embodiments of this application;

[0041] Figure 6(b) is a second schematic diagram of the process of filling a submatrix with zero elements provided in the embodiments of this application;

[0042] Figure 7 is a schematic diagram of the direct segmentation process of the source matrix provided in the embodiment of this application;

[0043] Figure 8 is a schematic diagram of the segmentation process during the source matrix filling process provided in the embodiments of this application;

[0044] Figure 9 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0045] Figure 10 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0046] Figure 11 is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of this application;

[0047] Figure 12 is a schematic diagram of another computing device cluster architecture provided in the embodiments of this application. Detailed Implementation

[0048] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0049] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0050] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0051] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0052] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0053] Before introducing the technical solution protected by this application, several technical terms involved in the technical solution protected by this application will be explained in advance, namely:

[0054] The row and column transformation operation, similar to the transpose operation, transforms all elements of a matrix to obtain a new matrix. As shown in Figure 1, matrix 1, after undergoing the row and column transformation operation, results in matrix 2. When performing the row and column transformation operation, the matrix to be executed and the resulting matrix do not need to be stored contiguously in memory.

[0055] Next, the technical solution provided in this application will be introduced.

[0056] Generally, when performing matrix padding optimization, padding methods in related technologies (such as Torch and NumPy) typically first allocate an initial zero matrix in memory, then fill the corresponding positions of the initial zero matrix with the data from the source matrix, and finally copy the padded matrix to the corresponding positions in the target matrix. As shown in Figure 2, to padded an 8×2 source matrix into an 8×4 target matrix, an 8×4 zero matrix needs to be configured, and then the source matrix is ​​filled into the zero matrix to form the target matrix.

[0057] However, the elements in the source matrix are stored contiguously in memory (hereinafter referred to as "source memory"), while the locations in the target matrix where data needs to be filled are not contiguous in memory (hereinafter referred to as "target memory"). As shown in Figure 2, the elements in the source matrix are stored contiguously in the source memory. In the target memory, since each row of the target matrix contains both elements from the source matrix and zero elements, there is storage space for zero elements between consecutive rows in the target matrix. Before filling, the target memory is cleared to provide zero elements. During the filling process, elements "1" and "2" from the first row of the source matrix can be read from the source memory and treated as a data packet. This data packet is then sent and written to the target memory. Similarly, elements "3" and "4" from the second row of the source matrix are read from the source memory and treated as a data packet, which is then sent to the target memory, and so on.

[0058] Therefore, it is evident that filling the source matrix requires multiple data packet transfers between the source and target memory. If the source matrix has few columns and many rows, the data packets transferred between the source and target memory are small but numerous, leading to wasted bandwidth between the source and target memory and potentially causing bandwidth bottlenecks, thus significantly reducing the computation speed of matrix multiplication.

[0059] In view of this, embodiments of this application provide a data processing method that can read a source matrix from source memory in one go, generate a target matrix by performing row and column axis transformations on the source matrix, concatenating it with a zero matrix, and then performing row and column axis transformations on the concatenated matrix, and then write it into the target memory in one go. Therefore, the copy between source memory and target memory uses larger data packets, eliminating the need to divide the source matrix into multiple small data packets for transmission, thereby reducing the waste of data bandwidth.

[0060] This method splits the matrix after the first row and column axis transformation into multiple sub-matrices. Then, it sequentially concatenates these sub-matrices with the zero matrix, performs a second row and column axis transformation on the concatenated sub-matrices, and stores the results in the target memory to construct the target matrix. This method improves filling efficiency by transforming segmented filling into continuous filling. Furthermore, by taking into account the computing power and specifications of the processing chip, this method splits the matrix after the first row and column axis transformation into multiple sub-matrices, making the matrix filling calculation more compatible with the hardware and further improving filling efficiency.

[0061] The data processing method provided in this application is used to accelerate matrix multiplication calculations. It is commonly used in the implementation of matrix multiplication-related operators in the operator libraries and acceleration libraries of computing chips such as neural processing units (NPUs) and graphics processing units (GPUs), such as general matrix multiplication (GEMM) and matrix multiplication (Mat-Mul). This method can also be used for other operators that may emerge in the future that take matrices as input and have requirements for the smallest computational unit.

[0062] Figure 3 is a schematic diagram of a computing chip provided in an embodiment of this application. As shown in Figure 3, the computing chip 300 may include a storage module 310, a vector module 320, a matrix module 330, and a bus 340. The storage module 310, the vector module 320, and the matrix module 330 can establish a communication connection through the bus 340.

[0063] Storage module 310 may include volatile memory, such as random-access memory (RAM). Storage module 310 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. Storage module 310 may be a combination of the various types of memory described above.

[0064] Storage module 310 can act as a cache to store data such as matrices to be processed and processed matrices. Storage module 310 is also used to store the application program running in vector module 320, allowing vector module 320 to execute the aforementioned data processing methods. Storage module 310 is further used to store the application program running in matrix module 330, allowing matrix module 330 to perform matrix multiplication calculations. Storage module 310 may also have other functions, which are not limited herein.

[0065] In this embodiment, a vector module 320 is added to the computing chip 300. This vector module 320 may include a storage unit 321 and a computing unit 322. The vector module 320 can read the matrix to be processed from the storage module 310 and then cache the read matrix in the storage unit 321. The computing unit 322 can execute the data processing method described above based on the matrix to be processed.

[0066] Figure 4 is a schematic diagram of the execution flow of a data processing method provided in an embodiment of this application. As shown in Figure 4, the method can be executed by the aforementioned computing unit 322, and the specific implementation process is as follows:

[0067] Step S401: Obtain the source matrix.

[0068] The source matrix refers to the matrix to be processed. If the source matrix is ​​stored in storage module 310, computation unit 322 can first read the source matrix from storage module 310 all at once and cache it in storage unit 321. Subsequently, computation unit 322 reads the source matrix from storage unit 321 all at once, waiting for subsequent processing. If the source matrix is ​​stored in storage unit 321, computation unit 322 can directly read the source matrix from storage unit 321 all at once, waiting for subsequent processing.

[0069] Step S402: Perform a reshape operation on the source matrix to obtain a rearranged matrix.

[0070] The reshape operation refers to rearranging a matrix into a new shape while maintaining the total number of its elements. This operation is widely used in many programming languages ​​and mathematical tools, such as NumPy (Python), MATLAB, and TensorFlow. In this embodiment, the computation unit 322 can expand all elements of the source matrix into a one-dimensional array according to a set order, such as row-major or column-major order. Subsequently, the computation unit 322 can rearrange the one-dimensional array into a rearranged matrix of the target shape. The computation unit 322 can flexibly adjust the shape of the matrix using the reshape operation, adjusting the shape of the source matrix so that its shape can approach the shape of the target matrix after two row and column axis transformation operations.

[0071] For example, as shown in Figure 5, the source matrix has a shape of 8×2, that is, the source matrix has 8 rows and 2 columns, and a total of 8×2=16 elements. Through the reshape operation, the calculation unit 322 rearranges the source matrix into a new rearranged matrix with a shape of 4×4, that is, the rearranged matrix has 4 rows and 4 columns, and a total of 4×4=16 elements.

[0072] Step S403: Perform row and column axis transformation operations on the rearranged matrix to obtain the transformation matrix.

[0073] When computation unit 322 performs row and column axis transformation, it swaps the row and column coordinates of all elements in the rearranged matrix, changing the coordinate positions of the elements within the matrix to obtain a new matrix. When the row and column coordinates of all elements in the rearranged matrix are swapped, the number of columns and rows of the rearranged matrix also changes, thus altering the shape of the rearranged matrix. Assuming the rearranged matrix has a shape of M×N, after computation unit 322 performs the row and column axis transformation, the resulting transformed matrix has a shape of N×M.

[0074] For example, as shown in Figure 5, the rearranged matrix has a shape of 4 (rows) × 4 (columns). After the rearranged matrix is ​​transformed by the row and column axes, the transformed matrix has a shape of 4 (columns) × 4 (rows).

[0075] In this embodiment, the computing unit 322 performs row and column axis transformation operations on the continuously stored source matrix, so that the transformed matrix can be concatenated with the zero matrix in the subsequent operation, thereby reducing the copying operation.

[0076] Step S404: Split the transformation matrix to obtain multiple sub-matrices.

[0077] When the computing unit 322 performs the splitting, it can divide the transformation matrix into multiple sub-matrices with shapes that meet the requirements of the computing chip, based on factors such as the computing power and rules of the computing chip corresponding to the computing unit 322. This makes the matrix filling calculation more compatible with the hardware, thereby improving the filling efficiency. For example, in a GPU, a core can only fill a matrix of shape X×Y at a time, but the computing unit 322 can split the transformation matrix into multiple sub-matrices with shapes smaller than X×Y.

[0078] In this application, the calculation unit 322 can split the source matrix into multiple submatrices with the same number of rows as the source matrix, based on the number of columns. This splitting ensures that all elements in a column of a submatrix are the same as all elements in a row of the source matrix, thus avoiding disrupting the distribution of elements in each row of the source matrix.

[0079] Step S405: Fill the submatrix with zero elements to obtain a zero-filled submatrix.

[0080] Step S406: The zero-padding submatrix that has completed the zero-padding operation is cached in storage unit 322.

[0081] There are several ways to implement the zero-padding operation. The following uses Figures 6(a) and 6(b) to introduce two zero-padding operations.

[0082] As shown in Figure 6(a), when the calculation unit 322 performs the zero-padding operation, it can configure a zero matrix with the same number of columns as the submatrix and the number of rows equal to the number of rows in the target matrix minus the number of rows in the submatrix, based on the shapes of the submatrix and the target matrix. Then, the calculation unit 322 can insert the zero matrix into the submatrix, so that the zero elements in the zero matrix are appended to the end of the submatrix, forming a completed submatrix. Finally, the calculation unit 322 can perform another row and column axis transformation operation on the completed submatrix, swapping the row and column coordinates of all elements in the completed submatrix, changing the coordinate positions of the elements in the matrix, thus obtaining the zero-padding submatrix. Compared to the source matrix, the zero-padding submatrix undergoes two row and column axis transformation operations, causing the non-zero elements in the zero-padding submatrix to return to their original positions, and all zero elements to be moved to the columns following the zero-padding submatrix.

[0083] As shown in Figure 6(b), when performing the zero-padding operation, the calculation unit 322 can configure a zero matrix with the same number of columns as the submatrix and the number of rows equal to the number of rows in the target matrix minus the number of rows in the submatrix, based on the shapes of the submatrix and the target matrix. Then, the calculation unit 322 can perform another row-column axis transformation operation on the submatrix, swapping the row and column coordinates of all elements in the submatrix, thus changing the coordinate positions of the elements in the matrix, resulting in a transformed submatrix. Simultaneously, the calculation unit 322 performs a row-column axis transformation operation on the zero matrix, transforming its shape to be the same as the transformed submatrix, resulting in a transformed zero matrix. Finally, the calculation unit 322 can insert the transformed zero matrix into the transformed submatrix, so that the zero elements in the transformed zero matrix are appended to the end of the transformed submatrix, resulting in a zero-padding submatrix. Compared to the source matrix, the zero-padding submatrix undergoes two row-column axis transformation operations, causing the non-zero elements in the zero-padding submatrix to return to their original positions, and all zero elements to be moved to the columns following the zero-padding submatrix.

[0084] In the zero-padding operation shown in Figure 6(a), the computation unit 322 inserts zero elements into the submatrix before performing the second row-column axis transformation operation. During this insertion process, the computation unit 322 needs to perform the insertion in the storage unit 321. In the zero-padding operation shown in Figure 6(b), the computation unit 322 inserts zero elements into the submatrix after performing the second row-column axis transformation operation. Since the row-column axis transformation operation allows for computation on discontinuous input data, the computation unit 322 processes the submatrix and zero matrix together in the storage unit 321, reducing the copying operations in the storage unit 321.

[0085] After processing a submatrix, the computation unit 322 caches each element in the processing zero-padded submatrix at the corresponding position in the storage unit 321, thereby caching a portion of the target matrix in the storage unit 321. Each time the computation unit 322 processes a submatrix, it caches each element in the processing zero-padded submatrix at the corresponding position in the storage unit 321, thus writing the elements into the storage unit 321 in a continuous filling manner to generate the target matrix. During this process, the computation unit 322 merges the currently processed zero-padded submatrix with the zero-padded submatrix already stored in the storage unit 321, so steps S405 and S407 are essentially performed simultaneously.

[0086] Step S407: Check if all submatrices have been processed. If not, proceed to step S405. If all submatrices have been processed, proceed to step S408.

[0087] Step S408: Store the target matrix cached in storage unit 322 into storage module 310.

[0088] After completing the zero-padding operation on the current submatrix, the calculation unit 322 checks whether all submatrices split from the transformation matrix have completed the zero-padding operation. If not, the calculation unit 322 can repeat steps S405-S407 until all submatrices split from the transformation matrix have completed the zero-padding operation. If all submatrices have completed the zero-padding operation, the calculation unit 322 caches the complete target matrix in the storage unit 321, and then writes the target matrix into the target memory in the storage module 310 all at once.

[0089] For example, as shown in Figure 6(a), the zero-padded submatrix corresponding to the first submatrix includes four rows of elements, namely (1,2,0,0), (5,6,0,0), (9,10,0,0), and (13,14,0,0). The zero-padded submatrix corresponding to the second submatrix includes four rows of elements, namely (3,4,0,0), (7,8,0,0), (11,12,0,0), and (15,16,0,0). When the calculation unit 322 performs the merging, it will sequentially insert (1,2,0,0), (3,4,0,0), (5,6,0,0), (7,8,0,0), (9,10,0,0), (11,12,0,0), (13,14,0,0), and (15,16,0,0) to form a target matrix with a total of 8 rows of elements including (1,2,0,0), (3,4,0,0), (5,6,0,0), (7,8,0,0), (9,10,0,0), (11,12,0,0), (13,14,0,0), and (15,16,0,0).

[0090] In this embodiment, the computing unit 322 reads the source matrix from the source memory of the storage module 310 in one go. After performing operations such as row and column axis transformation on the source matrix, concatenating it with the zero matrix, and then performing row and column axis transformation on the concatenated matrix, the target matrix is ​​generated in the storage unit 321 and written into the target memory of the storage module 310 in one go. Therefore, the copy between the source memory and the target memory of the storage module 310 uses a larger data packet, eliminating the need to divide the source matrix into multiple small data packets for transmission by row, thereby reducing the waste of data bandwidth.

[0091] The computation unit 322 splits the matrix after the first row and column axis transformation operation into multiple sub-matrices, then sequentially concatenates these sub-matrices with the zero matrix, performs a second row and column axis transformation on the concatenated sub-matrices, and stores the results in the storage unit 321 to construct the target matrix. By transforming segmented filling into continuous filling, the computation unit 322 improves filling efficiency. Furthermore, by splitting the matrix after the first row and column axis transformation operation into multiple sub-matrices based on the processing chip's computing power and specifications, the computation unit 322 makes the matrix filling calculation more compatible with the hardware, thereby further improving filling efficiency.

[0092] Matrix module 330 performs accelerated matrix multiplication calculations, which typically has requirements regarding the size of the input matrix. If the size of the source matrix exceeds the size range of the input matrix, matrix module 330 will be unable to perform matrix multiplication calculations on the source matrix. Therefore, calculation unit 322 needs to first check whether the size of the source matrix meets the requirements. If the size of the source matrix meets the requirements of the input matrix, calculation unit 322 can directly perform a filling operation. If the size of the source matrix does not meet the requirements, calculation unit 322 needs to split it in the column or row direction, decomposing the source matrix into several sub-source matrices with sizes that meet the requirements of the input matrix.

[0093] Taking column-wise segmentation as an example, as shown in Figure 7, assume the source matrix has a shape of N×6 and the input matrix has a size of N×4. When the calculation unit 322 detects that the number of columns in the source matrix does not meet the requirements, it can segment the source matrix into two sub-source matrices from the column direction, resulting in matrix A with a shape of N×4 and matrix B with a shape of N×2.

[0094] Since the number of columns in matrix B does not meet the requirements, the calculation unit 322 can fill matrix B using the technical solutions described in Figures 4-6 and the corresponding content to obtain matrix B' with a shape of N×4, so that the size of matrix B meets the requirements.

[0095] The computation unit 322 can perform the partitioning of the source matrix during the filling process, instead of directly partitioning it. As shown in Figure 8, after performing a reshape operation and the first row and column axis transformation operation on the source matrix, the computation unit 322 obtains the transformation matrix. When partitioning the transformation matrix, the computation unit 322 can partition the elements of the first to fourth rows of the transformation matrix into a submatrix in a top-down order, then partition the elements of the fifth to sixth rows into a submatrix, and so on. Each time the computation unit 322 partitions a submatrix containing four rows of elements, it will partition a submatrix containing two rows of elements.

[0096] The calculation unit 322 can classify multiple submatrices containing four rows of elements into one class, called submatrices A, and multiple submatrices containing two rows of elements into another class, called submatrices B. The calculation unit 322 can directly perform a second row and column axis transformation operation on the submatrices of class A, and then merge the transformed submatrices of class A to obtain a matrix A of shape N×4. The calculation unit 322 can execute steps S405-S407 in Figure 4 on the submatrices of class B to obtain a matrix B' of shape N×4.

[0097] In this embodiment, the computing unit 322 can process a source matrix of arbitrary shape into a size that conforms to the input matrix of the matrix module 330, thereby improving the applicable scenarios of the computing chip.

[0098] Based on the above description, this application provides a data processing apparatus 900. As shown in FIG9, the apparatus 900 includes:

[0099] The first processing module 910 is used to obtain the source matrix stored in the source memory; the second processing module 920 is used to perform row and column axis transformation operations on the source matrix to obtain a transformation matrix, wherein the row and column axis transformation operation is to perform row and column transformation operations on each element in the matrix; the third processing module 930 is used to split the transformation matrix to obtain multiple sub-matrices; the fourth processing module 940 is used to perform zero-padding operations on the multiple sub-matrices in sequence, and cache the zero-padding sub-matrices after each zero-padding operation in the storage unit; the fifth processing module 950 is used to determine the target matrix generated by the storage unit and write the target matrix into the target memory.

[0100] In one embodiment, before performing row and column axis transformation on the source matrix to obtain the transformed matrix, the second processing module 920 also performs a rearrangement operation on the source matrix to obtain the rearranged matrix.

[0101] In one embodiment, the fourth processing module 940 is used to perform zero-padding operation on the first submatrix to obtain a first zero-padding submatrix; and to store the first zero-padding submatrix in the corresponding position in the storage unit according to the position of the elements in the first zero-padding submatrix.

[0102] In one embodiment, the fourth processing module 940 is configured to configure a zero matrix according to the shape of the first sub-matrix and the shape of the target matrix, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first sub-matrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first sub-matrix; insert the zero elements of each row of the zero matrix after each row of the first sub-matrix to obtain the first completed sub-matrix; and perform a row and column axis transformation operation on the first completed sub-matrix to obtain the first zero-padded sub-matrix.

[0103] In one embodiment, the fourth processing module 940 is configured to configure a zero matrix according to the shape of the first sub-matrix and the shape of the target matrix, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first sub-matrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first sub-matrix; perform row and column axis transformation operations on the first sub-matrix and the zero matrix to obtain a first transformed sub-matrix and a transformed zero matrix; and insert each zero element in each row of the transformed zero matrix after each element in each row of the first transformed sub-matrix to obtain a first zero-padded sub-matrix.

[0104] In one embodiment, before acquiring the source matrix, the first processing module 910 is further configured to detect whether the shape of the source matrix is ​​larger than a set shape; when the shape of the source matrix is ​​larger than the set shape, the source matrix is ​​divided in the row direction and / or column direction to obtain multiple sub-source matrices.

[0105] In one embodiment, the third processing module 930 is used to detect whether the shape of the source matrix is ​​larger than a set shape; when the shape of the source matrix is ​​larger than the set shape, the rearranged matrix is ​​split into a first type of sub-matrix and a second type of sub-matrix. Each type of sub-matrix includes at least one sub-matrix. The elements in each first type of sub-matrix and the elements in each second type of sub-matrix are distributed alternately in the rearranged matrix. The first type of sub-matrix is ​​used to generate a first sub-target matrix that is smaller than or equal to the set shape, and the second type of sub-matrix is ​​used to generate a second sub-target matrix that is smaller than or equal to the set shape. The target matrix includes the first sub-target matrix and the second sub-target matrix.

[0106] The first processing module 910, the second processing module 920, the third processing module 930, the fourth processing module 940, and the fifth processing module 950 can all be implemented in software or in hardware. For example, the implementation of the first processing module 910 will be described below. Similarly, the implementation of the second processing module 920, the third processing module 930, the fourth processing module 940, and the fifth processing module 950 can refer to the implementation of the first processing module 910.

[0107] As an example of a software functional unit, the first processing module 910 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, the first processing module 910 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0108] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0109] As an example of a hardware functional unit, the first processing module 910 may include at least one computing device, such as a server. Alternatively, the first processing module 910 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0110] The multiple computing devices included in the first processing module 910 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the first processing module 910 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the first processing module 910 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0111] It should be noted that, in other embodiments, the first processing module 910 can be used to execute any step of the method shown in FIG. 4, the second processing module 920 can be used to execute any step of the method shown in FIG. 4, the third processing module 930 can be used to execute any step of the method shown in FIG. 4, the fourth processing module 940 can be used to execute any step of the method shown in FIG. 4, and the fifth processing module 950 can be used to execute any step of the method shown in FIG. 4.

[0112] The steps implemented by the first processing module 910, the second processing module 920, the third processing module 930, the fourth processing module 940, and the fifth processing module 950 can be specified as needed. The first processing module 910, the second processing module 920, the third processing module 930, the fourth processing module 940, and the fifth processing module 950 respectively implement different steps in the method shown in Figure 4 to realize all the functions of the device 1000.

[0113] Figure 10 is a schematic diagram of a computing device provided in an embodiment of this application. As shown in Figure 10, the computing device 1000 includes a bus 1010, a processor 1020, a memory 1030, and a communication interface 1040. The processor 1020, the memory 1030, and the communication interface 1040 communicate with each other via the bus 1010. The computing device 1000 can be a server, computer, laptop, server rack, etc. It should be understood that this application does not limit the number of processors and memories in the computing device 1000.

[0114] Bus 1010 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 10, but this does not imply that there is only one bus or one type of bus. Bus 1010 can include pathways for transmitting information between various components of computing device 1000 (e.g., processor 1020, memory 1030, communication interface 1040).

[0115] The processor 1020 can be any one or more of the following processors: CPU, GPU, microprocessor (MP), or digital signal processor (DSP).

[0116] The memory 1030 may include volatile memory, such as random access memory (RAM). The memory 1030 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0117] The memory 1030 stores executable program code, and the processor 1020 executes this executable program code to implement the functions of the aforementioned multiple modules, such as the first processing module 910, the second processing module 920, the third processing module 930, the fourth processing module 940, and the fifth processing module 950, thereby realizing the method shown in FIG4. That is, the memory 1030 stores instructions for executing the method shown in FIG4.

[0118] Alternatively, the memory 1030 may store executable code, which the processor 1020 executes to implement the functions of the aforementioned modules, thereby achieving the method shown in FIG4. That is, the memory 1030 stores instructions for executing the method shown in FIG4.

[0119] The communication interface 1040 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1000 and other devices or communication networks.

[0120] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0121] As shown in Figure 11, the computing device cluster includes at least one computing device 1000. The memory 1030 in one or more computing devices 1000 in the computing device cluster may store the same instructions for performing the method shown in Figure 4.

[0122] In some possible implementations, the memory 1030 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the method shown in FIG. 4. In other words, a combination of one or more computing devices 1000 can jointly execute the instructions for executing the method shown in FIG. 4.

[0123] It should be noted that the memories 1030 in different computing devices 1000 within the computing device cluster can store different instructions, which are used to execute parts of the functions of the first processing module 910, the second processing module 920, the third processing module 930, the fourth processing module 940, and the fifth processing module 950, respectively. That is, the instructions stored in the memories 1030 of different computing devices 1000 can implement the functions of one or more of the aforementioned first processing module 910, second processing module 920, third processing module 930, fourth processing module 940, and fifth processing module 950.

[0124] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 12 illustrates one possible implementation. As shown in Figure 12, two computing devices, computing device 1000A and computing device 1000B, are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1030 in computing device 1000A stores instructions for executing some of the functions of the first processing module 910 and the second processing module 920. Simultaneously, the memory 1030 in computing device 1000B stores instructions for executing other parts of the functions of the third processing module 930, the fourth processing module 940, and the fifth processing module 950.

[0125] The connection method between the computing device clusters shown in Figure 12 can be considered in light of the fact that the method provided in this application as shown in Figure 4 requires a large amount of data storage. Therefore, it is considered that the functions implemented by another part of the modules in the first processing module 910, the second processing module 920, the third processing module 930, the fourth processing module 940 and the fifth processing module 950 are handed over to the computing device 1000B for execution.

[0126] It should be understood that the functions of computing device 1000A shown in Figure 12 can also be performed by multiple computing devices 1000. Similarly, the functions of computing device 1000B can also be performed by multiple computing devices 1000.

[0127] This application embodiment also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to in FIG12 and the connection method of the computing device cluster in FIG12. The difference is that the memory 1030 in one or more computing devices 1000 in this computing device cluster can store the same instructions for executing the method shown in FIG4.

[0128] In some possible implementations, the memory 1030 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the method shown in FIG. 4. In other words, a combination of one or more computing devices 1000 can jointly execute the instructions for executing the method shown in FIG. 4.

[0129] It should be noted that the memory 1030 in different computing devices 1000 within the computing device cluster can store different instructions for executing some functions of the computing device 1000. That is, the instructions stored in the memory 1030 of different computing devices 1000 can implement the functions of one or more of the aforementioned first processing module 910, second processing module 920, third processing module 930, fourth processing module 940, and fifth processing module 950.

[0130] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the method shown in FIG4.

[0131] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the method shown in FIG. 4.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method, characterized by, The computing device includes a storage module and a vector module. The storage module includes source memory and target memory. The source memory is used to store a source matrix, and the target memory is used to store a target matrix. The vector module includes a computing unit and a storage unit. The method is executed by the computing unit and includes: Obtain the source matrix stored in the source memory; Perform row and column transformation operations on the source matrix to obtain a transformed matrix. The row and column transformation operations are operations that transform each element in the matrix by row and column transformation. The transformation matrix is ​​split into multiple sub-matrices; The zero-padding operation is performed sequentially on the plurality of sub-matrices, and the zero-padding sub-matrices that have completed the zero-padding operation are cached in the storage unit each time; The target matrix is ​​generated by the storage unit and then written into the target memory.

2. The method of claim 1, wherein, Before performing row and column axis transformation operations on the source matrix to obtain the transformed matrix, the method further includes: The source matrix is ​​reshaped to obtain a rearranged matrix.

3. The method according to claim 1 or 2, characterized in that, The step of sequentially padding the plurality of submatrices with zero elements and caching the padded submatrices after each padding operation in the storage unit includes: The first submatrix is ​​padded with zero elements to obtain the first zero-padded submatrix; Based on the position of the element in the first zero-padding submatrix, the first zero-padding submatrix is ​​stored in the corresponding position in the storage unit.

4. The method of claim 3, wherein, The step of padding the first submatrix with zero elements to obtain the first zero-padded submatrix includes: Based on the shape of the first sub-matrix and the shape of the target matrix, a zero matrix is ​​configured, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first sub-matrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first sub-matrix; The zero elements in each row of the zero matrix are inserted after each row of the first submatrix to obtain the first completed submatrix; Perform the row and column axis transformation operation on the first padded submatrix to obtain the first zero-padded submatrix.

5. The method of claim 3, wherein, The step of padding the first submatrix with zero elements to obtain the first zero-padded submatrix includes: Based on the shape of the first sub-matrix and the shape of the target matrix, a zero matrix is ​​configured, wherein the number of columns of the zero matrix is ​​the same as the number of columns of the first sub-matrix, and the number of rows of the zero matrix is ​​the number of rows of the target matrix minus the number of rows of the first sub-matrix; Perform the row and column axis transformation operation on the first submatrix and the zero matrix to obtain the first transformed submatrix and the transformed zero matrix; The zero elements in each row of the transformation zero matrix are inserted after each row of the first transformation submatrix to obtain the first zero-padded submatrix.

6. The method according to any one of claims 1 to 5, characterized in that, Before obtaining the source matrix, the method further includes: Detect whether the shape of the source matrix is ​​larger than a set shape; When the shape of the source matrix is ​​larger than the set shape, the source matrix is ​​divided in the row direction and / or column direction to obtain multiple sub-source matrices.

7. The method according to any one of claims 1 to 6, characterized in that, The step of splitting the rearranged matrix to obtain multiple sub-matrices includes: Detect whether the shape of the source matrix is ​​larger than a set shape; When the shape of the source matrix is ​​larger than the set shape, the rearranged matrix is ​​split into a first type of submatrix and a second type of submatrix. Each type of submatrix includes at least one submatrix. The elements in each first type of submatrix and the elements in each second type of submatrix are distributed alternately in the rearranged matrix. The first type of submatrix is ​​used to generate a first sub-target matrix that is smaller than or equal to the set shape, and the second type of submatrix is ​​used to generate a second sub-target matrix that is smaller than or equal to the set shape. The target matrix includes the first sub-target matrix and the second sub-target matrix.

8. A data processing apparatus, characterized by, include: The first processing module is used to obtain the source matrix stored in the source memory; The second processing module is used to perform row and column axis transformation operations on the source matrix to obtain a transformation matrix. The row and column axis transformation operation is to perform row and column transformation operations on each element in the matrix. The third processing module is used to split the transformation matrix into multiple sub-matrices; The fourth processing module is used to sequentially perform zero-padding operations on the plurality of sub-matrices and cache the zero-padding sub-matrices that have completed the zero-padding operations in the storage unit. The fifth processing module is used to determine that the storage unit generates the target matrix and write the target matrix into the target memory.

9. A computing device, comprising: include: At least one memory; At least one processor, the processor being configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1-7.

Images