Matrix transposition apparatus and method, ai processor, and computer device

Abstract

The present application belongs to the technical field of artificial intelligence. Disclosed in the embodiments of the present application are a matrix transposition apparatus and method, an AI processor, and a computer device. The matrix transposition apparatus comprises a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer and a controller. The controller is used for: controlling the matrix reader to read from an upper-level memory matrix elements of a matrix to be transposed, the matrix reader reading, for each time, at least one row of matrix elements of the matrix to be transposed; controlling the shifter to execute data shift on the matrix elements read by the matrix reader, and controlling the shifter to write the shifted matrix elements into the matrix buffer; controlling the transposer to read sub-matrixes from the matrix buffer, and controlling the transposer to execute transposition processing on the read sub-matrixes to obtain transposed sub-matrixes, each sub-matrix comprising at least one column of matrix elements of the matrix to be transposed; and controlling the matrix writer to write into a lower-level memory the transposed sub-matrixes output by the transposer.

Description

Matrix transposition device, method, AI processor and computer equipment

[0001] This application claims priority to Chinese patent application number 202311171547.3, filed on September 12, 2023, entitled “Matrix transposition device, method, AI processor and computer equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a matrix transposition device, method, AI processor, and computer equipment. Background Art

[0003] With the development of artificial intelligence technology, more and more AI technologies are being used in life and production.

[0004] Matrix operations are widely used as the foundation of artificial intelligence technology, and matrix transposition is a common matrix operation. For example, in image processing, matrix transposition can be used to flip an image; in signal processing, matrix transposition can be used to perform fast Fourier transforms.

[0005] In order to realize matrix transposition, a special matrix transposition device needs to be set up in the AI ​​processor. Therefore, how to reduce the design complexity of the matrix transposition device and reduce the on-chip circuit area becomes the key to improving the performance of the AI ​​processor.

[0006] Summary of the Invention

[0007] The present invention provides a matrix transposition device, method, AI processor, and computer equipment that can reduce the processing delay of matrix transposition operations. The technical solution is as follows:

[0008] In one aspect, an embodiment of the present application provides a matrix transposition device, comprising:

[0009] Matrix readers, shifters, matrix buffers, transposers, matrix writers, and controllers;

[0010] The controller is configured to control the matrix reader to read matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time;

[0011] The controller is configured to control the shifter to perform data shifting on the matrix elements read by the matrix reader, and control the shifter to write the shifted matrix elements into the matrix buffer;

[0012] The controller is configured to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed;

[0013] The controller is used to control the matrix writer to write the transposed submatrix output by the transposer into a lower-level memory.

[0014] On the other hand, an embodiment of the present application provides a matrix transposition method, which is used in the matrix transposition device according to the above aspect, and the method includes:

[0015] The controller controls the matrix reader to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time;

[0016] The controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, and controls the shifter to write the shifted matrix elements into the matrix buffer;

[0017] The controller controls the transposer to read a submatrix from the matrix buffer, and controls the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed;

[0018] The controller controls the matrix writer to write the transposed submatrix output by the transposer into a lower-level memory.

[0019] On the other hand, an embodiment of the present application provides an AI processor, which includes the matrix transposition device as described in the above aspects.

[0020] On the other hand, an embodiment of the present application provides a computer device, which includes the AI ​​processor and memory as described in the above aspects, and the AI ​​processor and the memory are connected via a bus.

[0021] In an embodiment of the present application, a matrix transposition device performs data shift processing on matrix elements read by a matrix reader, performs block reading on the matrix stored in a matrix buffer after shifting, obtains a sub-matrix containing at least one column of matrix elements in the matrix to be transposed, and performs transposition on the sub-matrix, and then obtains the transposed result of the matrix to be transposed based on the transposed sub-matrix corresponding to each sub-matrix, thereby realizing matrix transposition along the matrix column direction, which can reduce the processing delay of the matrix transposition compared to performing matrix transposition along the matrix diagonal direction; in addition, when performing matrix transposition along the matrix column direction, since the number of matrix elements extracted from the matrix buffer each time is consistent, the same matrix only needs to use one transposition logic, which helps to reduce the design complexity of the matrix transposition device and reduce the on-chip area occupied by the matrix transposition device. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a schematic diagram illustrating an exemplary embodiment of the present invention showing a process of performing matrix transposition along a matrix diagonal direction;

[0023] FIG2 shows a schematic structural diagram of a matrix transposition device provided by an exemplary embodiment of the present application;

[0024] FIG3 is a schematic diagram illustrating an implementation of a matrix filling and filling removal process according to an exemplary embodiment of the present application;

[0025] FIG4 is a schematic diagram illustrating an implementation of a matrix element right shift process according to an exemplary embodiment of the present application;

[0026] FIG5 is a schematic structural diagram of a matrix buffer according to an exemplary embodiment of the present application;

[0027] FIG6 is a schematic diagram showing the principle of a matrix transposition process provided by an exemplary embodiment of the present application;

[0028] FIG7 is a schematic diagram illustrating an implementation of a matrix transposition process according to an exemplary embodiment of the present application;

[0029] FIG8 is a schematic diagram illustrating an implementation of a matrix transposition process according to another exemplary embodiment of the present application;

[0030] FIG9 is a schematic diagram illustrating an implementation of a matrix transposition process according to another exemplary embodiment of the present application;

[0031] FIG10 shows a flowchart of a matrix transposition method provided by an exemplary embodiment of the present application;

[0032] FIG11 shows a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0033] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0034] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0035] Since storage and processing are integrated in neural networks, while storage and processing are separated in classical computer structures (implemented by memory and arithmetic units, respectively), when using classical computers to run neural networks, it will inevitably be restricted by the separate storage and processing structure, affecting processing efficiency. In order to improve the processing efficiency of neural networks, neural network processors (Neural-Network Processing Units, NPU) (also known as AI processors) came into being. Among them, AI processors use circuits to simulate neurons and synaptic structures. Each neuron is abstracted as an excitation function, the input of which is determined by the output of the neurons connected to it and the synapses connecting the neurons. In order to express specific knowledge, it is necessary to adjust the values ​​of synapses in artificial neural networks, the topological structure of the network, etc. This process is called "learning". After learning, artificial neural networks can solve specific problems through the acquired knowledge.

[0036] AI processors need to perform a large number of matrix operations during operation, including matrix multiplication, matrix inversion, matrix transposition, etc. In order to enable AI processors to perform matrix transposition, related technologies will specifically set up a matrix transposition device in the AI ​​processor.

[0037] In one possible matrix transposition method, the matrix transposition device performs matrix transposition along the diagonal direction of the matrix. As shown in Figure 1, when performing matrix transposition, the matrix transposition device extracts a group of vectors (consisting of at least one matrix element) perpendicular to the diagonal direction of the matrix to be transposed each time. Schematically, in (a) shown in Figure 1, the extracted vector contains the matrix element (0,0), in (b) shown in Figure 1, the extracted vector contains the matrix elements (0,1) and (1,0), in (c) shown in Figure 1, the extracted vector contains the matrix elements (0,2), (1,1) and (2,0), in (d) shown in Figure 1, the extracted vector contains the matrix elements (0,3), (1,2), (2,1) and (3,0), and so on.

[0038] Furthermore, the matrix transposition device performs a mirror inversion on the read vector and writes the inverted vector into the memory in the same manner as when it was read. Schematically, in (a) shown in FIG1 , the matrix transposition device writes the matrix element (0,0) into the memory, in (b) shown in FIG1 , the matrix transposition device writes the matrix elements (1,0) and (0,1) into the memory, in (c) shown in FIG1 , the matrix transposition device writes the matrix elements (2,0), (1,1) and (0,2) into the memory, in (d) shown in FIG1 , the matrix transposition device writes the matrix elements (3,0), (2,1) (1,2) and (0,3) into the memory, and so on.

[0039] However, when performing matrix transposition using the above approach, the latency of matrix processing is greater than the latency of matrix writing, resulting in poor performance. For example, for the 4×4 matrix shown in Figure 1, writing to the memory requires four clock cycles (writing row by row, a total of four rows), while reading the vector along the diagonal to perform the flip requires seven clock cycles to complete the matrix processing. Therefore, in the scenario of performing continuous multi-matrix transposition operations, the startup latency of the above scheme is greater than the matrix access latency; and in the scenario of performing matrix transposition operations one by one, the above scheme will cause matrix access interruption, resulting in reduced data bandwidth.

[0040] In addition, since the number of matrix elements taken out along the diagonal each time is different, matrix transposition of the same size requires the configuration of multiple vector inversion logics, which increases the design complexity of the inversion circuit and leads to an increase in the on-chip area occupied by the matrix transposition device.

[0041] Unlike the above-mentioned scheme in which matrix transposition is performed along the matrix diagonal direction, an embodiment of the present application provides a matrix transposition device that performs transposition along the matrix column direction. By performing shifting and block processing on the matrix elements, the matrix transposition device performs transposition on the submatrix containing at least one column of matrix elements in the matrix to be transposed each time, and then obtains the transposition result of the matrix to be transposed based on the transposed submatrix corresponding to each submatrix. Compared with performing matrix transposition along the matrix diagonal direction, the matrix transposition device provided by the embodiment of the present application can achieve a delay consistent with matrix access, that is, reduce the processing delay of the matrix transposition; and, since the number of matrix elements extracted from the matrix buffer along the matrix column direction each time is consistent, the matrix of the same length and width size only needs to use one transposition logic in the entire transposition process, which helps to reduce the design complexity of the matrix transposition device and reduce the on-chip area occupied by the matrix transposition device.

[0042] The matrix transposition device provided in the embodiments of the present application can be applied to the following scenarios:

[0043] 1. The training and inference process of neural networks in artificial intelligence chips, including the calculation of backpropagation gradients when training neural networks, matrix operations at specific layers in the neural network inference process, etc.

[0044] 2. Image processing: In image processing applications, matrix transposition calculations are performed. For example, when performing image flipping or rotation operations through transposition, a digital image represented as a matrix can be transformed into a rotated or mirrored image through the transposition operation.

[0045] 3. Signal processing: Use matrix transposition to implement the Fast Fourier Transform (FFT) algorithm.

[0046] 4. Correlation analysis in graph networks. Social network or other network analysis can use matrix transposition calculations to determine the source of relationships between nodes in the network, or to determine the relationship patterns between nodes in the network.

[0047] Please refer to FIG2 , which shows a schematic diagram of a matrix transposition device according to an exemplary embodiment of the present application. The matrix transposition device includes a matrix reader 210 , a shifter 220 , a matrix buffer 230 , a transposer 240 , a matrix writer 250 , and a controller 260 .

[0048] The input of matrix reader 210 is connected to the upper-level memory, the output of matrix reader 210 is connected to the input of shifter 220, the output of shifter 220 is connected to the write end of matrix buffer 230, the read end of matrix buffer 230 is connected to the input of transposer 240, the output of transposer 240 is connected to the input of matrix writer 250, and the output of matrix writer 250 is connected to the lower-level memory. Controller 260, serving as the control end for the matrix transposition process, is connected to matrix reader 210, shifter 220, matrix buffer 230, transposer 240, and matrix writer 250, respectively.

[0049] The controller 260 is used to control the matrix reader 210 to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time.

[0050] The meanings of the matrix elements in the matrix to be transposed may vary in different application scenarios. For example, the matrix elements in the matrix to be transposed may be the color values ​​of pixels in an image under a certain color channel, or they may be the eigenvalues ​​of a graph node in a graph network. The embodiments of this application do not limit the specific meanings of the matrix elements in the matrix to be transposed.

[0051] In some embodiments, the matrix reader 210 reads one row of data (i.e., one burst) from the upper-level memory at a time based on the read interface bit width of the upper-level memory. Because the total bit width of each row of matrix elements in the matrix to be transposed may be less than or equal to the read interface bit width, each row of data read by the matrix reader 210 includes at least one row of matrix elements in the matrix to be transposed. For example, when the total bit width of each row of matrix elements in the matrix to be transposed is equal to the read interface bit width, one row of matrix elements in the matrix to be transposed is read each time; when the total bit width of each row of matrix elements in the matrix to be transposed is 1 / 2 of the read interface bit width, two rows of matrix elements in the matrix to be transposed are read each time.

[0052] In order to avoid bank (memory block) read conflicts when subsequently reading matrix elements (i.e., to avoid multiple threads attempting to access different locations in the same bank at the same time), before writing the read matrix elements into the matrix buffer 230, the controller 260 is used to control the shifter 220 to perform data shifting on the matrix elements read by the matrix reader 210, and to control the shifter 220 to write the shifted matrix elements into the matrix buffer 230.

[0053] Optionally, the shift operation may be a right-shifted circular shift operation. For example, when the matrix elements read are 00, 01, 02, 03, 04, 05, after performing a 1-bit right-shifted circular shift operation, the resulting matrix element sequence is 05, 00, 01, 02, 03, 04.

[0054] After the shifted matrix elements are written to the matrix buffer, the matrix elements that were originally in the same column in the matrix to be transposed are moved to different columns. That is, the matrix elements that were originally in the same column in the matrix to be transposed are located in different banks. Therefore, when one or more columns of the matrix elements in the matrix to be transposed are subsequently read, no bank read conflict occurs.

[0055] In some embodiments, the data shift lengths of the matrix elements read in each row are different, and the difference between the data shift lengths of the matrix elements in two adjacent rows is the same. For example, the data shift lengths of the matrix elements in four rows are 0, 1, 2, and 3 respectively.

[0056] In some embodiments, the matrix buffer 230 may be implemented as a static random-access memory (SRAM) or a register. The embodiment of the present application does not limit the specific type of the matrix buffer.

[0057] When the matrix to be transposed is completely written into the matrix buffer 230 , the controller 260 controls the transposer 240 to perform matrix transposition.

[0058] The controller 260 is used to control the transposer 240 to read a submatrix from the matrix buffer 230 and control the transposer 240 to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed.

[0059] In some embodiments, the matrix to be transposed can be divided into several sub-matrices of uniform size in the column direction (matrix filling may be involved, which will be introduced in a separate embodiment below). When the controller 260 controls the transposer 240 to read a sub-matrix from the matrix buffer 230, one sub-matrix among the several sub-matrices is read each time.

[0060] Since the size of the sub-matrices read each time is consistent, the transpose unit 240 can use consistent transposition logic to perform transposition processing on each sub-matrix to obtain a transposed sub-matrix.

[0061] In an illustrative example, when the size of the matrix to be transposed is 8×8, the matrix to be transposed can be divided into eight 8×1 sub-matrices along the column direction. The controller 260 controls the transposer 240 to read one sub-matrix at a time, and a total of eight sub-matrices are read. The transposer 240 performs a transposition process on each sub-matrix to obtain a transposed sub-matrix of size 1×8.

[0062] Furthermore, the controller 260 is configured to control the matrix writer 250 to write the transposed sub-matrix output by the transposer 240 into the lower-level memory.

[0063] In some embodiments, after each transposition process is completed by the transposer 240, the matrix writer 250 writes the matrix elements of the transposed submatrix obtained by the transposition to the lower-level memory. After the matrix writer 250 writes all the transposed submatrices to the lower-level memory, the lower-level memory stores the transposed matrix corresponding to the matrix to be transposed.

[0064] Taking the example of a matrix to be transposed having a size of 8×8 and a memory requiring 8 clock cycles to read and write the matrix, in the related art, when performing matrix transposition along the diagonal direction, the matrix transposition requires 15 clock cycles; while when performing matrix transposition along the column direction using the matrix transposition device provided in an embodiment of the present application, the matrix transposition only requires 8 clock cycles. The matrix transposition is more efficient, and the clock cycles required for the matrix transposition are consistent with the read and write delay of the memory, which helps to improve performance and increase transmission bandwidth.

[0065] In addition, in the related art, when performing matrix transposition along the diagonal direction, since the number of matrix elements read each time is 1 to 8, 8 types of inversion logic need to be set; however, with the solution provided in the embodiment of the present application, since the number of matrix elements in the sub-matrix read each time is 8, only one transposition logic needs to be set for the transposer, and there is no need to process different numbers of matrix elements, which reduces the design complexity of the matrix transposition device and helps to reduce the on-chip area occupied by the matrix transposition device.

[0066] To summarize, in an embodiment of the present application, the matrix transposition device performs data shift processing on the matrix elements read by the matrix reader, performs block reading on the matrix stored in the matrix buffer after the shift, obtains a sub-matrix containing at least one column of matrix elements in the matrix to be transposed, and performs transposition on the sub-matrix, and then obtains the transposed result of the matrix to be transposed based on the transposed sub-matrix corresponding to each sub-matrix, thereby realizing matrix transposition along the matrix column direction, which can reduce the processing delay of the matrix transposition compared to performing matrix transposition along the matrix diagonal direction; in addition, when performing matrix transposition along the matrix column direction, since the number of matrix elements extracted from the matrix buffer each time is the same, the same matrix only needs to use one transposition logic, which helps to reduce the design complexity of the matrix transposition device and reduce the on-chip area occupied by the matrix transposition device.

[0067] In a method of dividing a matrix to be transposed into several sub-matrices, for a matrix to be transposed of size N×M, the matrix to be transposed is divided into k sub-matrices, and the total bit width of matrix elements in each sub-matrix is ​​greater than or equal to the read interface bit width. That is, the total bit width of matrix elements in the sub-matrix processed by a single transposition is greater than or equal to the read interface bit width, thereby ensuring that there is no transmission bandwidth loss during the matrix transposition process.

[0068] Regarding the method for determining the number of submatrix columns, in one possible implementation, the controller determines the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed.

[0069] The matrix parameters of the matrix to be transposed include the number of rows and columns of the matrix to be transposed and the number of bits of each matrix element.

[0070] Since the matrix elements in the matrix are usually stored in the memory in the order of rows first and columns later, when the bit width of the row matrix elements of the matrix to be transposed is greater than or equal to the read interface bit width, it can be guaranteed that the total bit width of the matrix elements in each matrix processing is greater than or equal to the read interface bit width.

[0071] In a possible implementation, when the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, the column-row ratio of the matrix to be transposed is rounded up to an integer and determined as the number of submatrix columns.

[0072] The row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element. For example, for an N×M matrix to be transposed, the row matrix element bit width = M (number of columns of the matrix to be transposed) × Element_size (number of bits of the matrix element).

[0073] For example, when the size of the matrix to be transposed is 8×8 and the number of bits of the matrix elements is 128 bits, the bit width of the row matrix elements of the matrix to be transposed is 1024 bits.

[0074] When the row matrix element width is equal to the read interface width, the controller determines k=ceiling[M / N], where ceilling is a rounding-up operation, that is, dividing the matrix to be transposed into N sub-matrices, each sub-matrix having a size of N×k.

[0075] Correspondingly, the total bit width of the matrix elements in each sub-matrix obtained by division is N×k×Element_size. Since k≥M / N, and M×Element_size=read_port_width (read interface bit width), N×k×Element_size≥read_port_width.

[0076] In an illustrative example, when the row matrix element bit width of the matrix to be transposed is 2048 bits and the read interface bit width is 2048 bits, if the size of the matrix to be transposed is 8×8, the number of submatrix columns is determined to be 1, that is, the size of each submatrix is ​​8×1; if the size of the matrix to be transposed is 4×8, the number of submatrix columns is determined to be 2, that is, the size of each submatrix is ​​4×2; if the size of the matrix to be transposed is 6×8, the number of submatrix columns is determined to be 2, that is, the size of each submatrix is ​​6×2.

[0077] In another possible implementation, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the result of rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed is determined as the number of submatrix columns, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element.

[0078] The column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element. For example, for an N×M matrix to be transposed, the column matrix element bit width = N (number of columns of the matrix to be transposed) × Element_size (number of bits of the matrix element).

[0079] For example, when the size of the matrix to be transposed is 8×8 and the number of bits of the matrix elements is 128 bits, the bit width of the column matrix elements of the matrix to be transposed is 1024 bits.

[0080] When the row matrix element width is equal to the read port width, the controller determines k=ceilling[read_port_width / (N*Element_size)], that is, the matrix to be transposed is divided into N sub-matrices, and the size of each sub-matrix is ​​N×k.

[0081] Correspondingly, the total bit width of the matrix elements in each sub-matrix obtained by division is N×k×Element_size. Since k≥read_port_width / N*Element_size, N×k×Element_size≥read_port_width.

[0082] In an illustrative example, when the row matrix element bit width of the matrix to be transposed is 1024 bits and the read interface bit width is 2048 bits, if the size of the matrix to be transposed is 8×8 (the matrix element is 128 bits), the number of sub-matrix columns is determined to be 2, that is, the size of each sub-matrix is ​​8×2.

[0083] To ensure that the sizes of the sub-matrices are consistent, when N cannot be divided by M, padding processing needs to be performed on the transposed matrix. Accordingly, when the transposed sub-matrix is ​​subsequently written to the lower-level memory, padding removal needs to be performed.

[0084] Schematically, as shown in Figure 3, for an N×M matrix to be transposed, since M cannot divide N evenly, the matrix reader pads the read matrix elements with M'-M matrix elements, where M' = k*N. Since the resulting size after transposition is M'×N, the last M'-M rows of matrix elements must be removed to restore the M×N matrix.

[0085] In one possible implementation, when the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-row ratio is a non-integer greater than 1, the controller controls the matrix reader to perform padding after the read matrix elements so that the column-row ratio of the matrix to be transposed after padding is an integer.

[0086] Optionally, the matrix reader fills the read matrix elements with 0 or other preset data.

[0087] In one possible implementation, the controller determines the number of padding columns based on the number of rows and columns of the matrix to be transposed, and controls the matrix reader to perform padding after the read matrix elements based on the number of padding columns.

[0088] The number of padding columns is k × NM. For example, when the size of the matrix to be transposed is 6 × 8, the matrix reader needs to pad 4 columns of matrix elements each time it reads a matrix element.

[0089] It should be noted that the matrix reader fills in the matrix elements as it reads them, without taking up any additional time.

[0090] Accordingly, the controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer, and writes the matrix elements in the at least one transposed submatrix after padding removal into the lower-level memory. It should be noted that the at least one transposed submatrix after padding removal is located at the last position. In other words, the data position in the at least one transposed submatrix is ​​located at the end of one or more rows of the transposed result of the matrix to be transposed after padding. It should be noted that, in some examples, performing padding removal on the transposed submatrix output by the transposer can be implemented by removing part or all of the data in the transposed submatrix.

[0091] Because the padded columns become rows after the submatrix is ​​transposed, the matrix writer needs to perform a de-padding operation on at least one transposed submatrix to remove the rows that became the padded columns after the transposition. It should be noted that the de-padding operation here does not refer to a specific removal operation, but rather means that the matrix writer does not write some or all rows in the transposed submatrix to the lower-level memory.

[0092] In one possible implementation, the controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer based on the number of padding columns, wherein the sum of the number of rows of matrix elements to be removed from the at least one transposed submatrix is ​​the same as the number of padding columns.

[0093] The number of rows of matrix elements removed is the same as the number of padded columns, that is, k×NM. For example, when the size of the matrix to be transposed is 6×8, in order to ensure that the column-row ratio of the padded matrix to be transposed is an integer, 4 columns of matrix elements need to be padded, and the size of the padded matrix to be transposed is 6×12. As mentioned above, the size of each submatrix is ​​6×2, and the size of the corresponding transposed submatrix is ​​2×6. The last 4 rows of matrix elements need to be removed to remove the rows formed by the transposed 4 columns of matrix elements. The size of each transposed submatrix is ​​2×6, that is, it includes two rows and six columns. It is necessary to perform padding removal on the two transposed submatrices at the last position to remove all data in the transposed submatrix to achieve the removal of the last 4 rows of matrix elements.

[0094] To avoid bank read conflicts when reading a submatrix from a matrix buffer, in one possible implementation, the controller determines a shift step corresponding to each matrix element read by the matrix reader based on the number of submatrix columns of the submatrix, and thereby controls the shifter to perform a data right shift on the matrix elements read by the matrix reader based on the shift step.

[0095] In one possible implementation, for the matrix element read by the matrix reader for the nth time, the controller determines that the shift step length of the matrix element is (n-1)×k, that is, the shift step length of the matrix element read for the first time is 0, the shift step length of the matrix element read for the second time is k, the shift step length of the matrix element read for the third time is 2k, and so on.

[0096] After the shifting process, matrix elements originally located in the same column of the matrix to be transposed are moved to different columns, thereby avoiding bank read conflicts when reading sub-matrices from the matrix buffer.

[0097] Schematically, as shown in Figure 4, when the number of rows of the matrix to be transposed is 5 and the number of submatrix columns of the submatrix is ​​k, the matrix elements in the first row are shifted right by 0 steps and written into the matrix buffer, the matrix elements in the second row are shifted right by k steps and written into the matrix buffer, the matrix elements in the third row are shifted right by 2k steps and written into the matrix buffer, the matrix elements in the fourth row are shifted right by 3k steps and written into the matrix buffer, and the matrix elements in the fifth row are shifted right by 4k steps and written into the matrix buffer.

[0098] In one possible design, the matrix buffer is composed of several memory banks.

[0099] In an illustrative example, as shown in Figure 5, the matrix buffer is provided with 2n memory banks, each of which is connected to a write bus and a write data interface, a read bus and a read data interface, and an address bus and an address interface. Therefore, the shifter can write data to different memory banks in parallel via the write data interface, and the transposer can read data from different memory banks in parallel via the read data interface.

[0100] Regarding the method of writing the shifted matrix elements into the matrix buffer, in one possible implementation, when performing matrix element writing, the controller is used to control the shifter to write the shifted matrix elements into multiple memory banks in the matrix buffer; and when performing matrix element reading, the controller is used to control the transposer to read the matrix elements of the sub-matrix in parallel from the multiple memory banks of the matrix buffer based on the number of sub-matrix columns.

[0101] In some embodiments, the controller controls the shifter to write different shifted matrix elements into different memory banks in the matrix buffer. For example, when a row of 8 matrix elements is read, the shifter writes the 8 matrix elements into 8 memory banks respectively.

[0102] Correspondingly, when the subsequent transposer reads a column of matrix elements from the matrix buffer, the transposer reads 8 matrix elements from 8 memory banks respectively.

[0103] In one possible implementation, when reading matrix elements in the same submatrix from different memory banks, the controller determines a read address for each memory bank based on the number of columns of the submatrix, wherein the read address of the memory bank is incremented, and the increment of the address increment is the bit width of k matrix elements.

[0104] Schematically, when reading the first sub-matrix, the transposer reads matrix elements from the first memory bank based on the starting address 0, reads matrix elements from the second memory bank based on the starting address k×Element_size, reads matrix elements from the third memory bank based on the starting address 2k×Element_size, and so on. When reading the i-th sub-matrix, the transposer reads matrix elements from the i-th memory bank based on the starting address 0, reads matrix elements from the i+1-th memory bank based on the starting address k×Element_size, reads matrix elements from the i+2-th memory bank based on the starting address 2k×Element_size, and so on. It should be noted that after reading the last memory bank storing matrix elements, matrix elements will be read starting from the first memory bank.

[0105] Since data shifting is performed when writing matrix elements into the matrix buffer, in order to ensure the correctness of the order of the matrix elements in the sub-matrix, in one possible embodiment, the controller is further used to control the transposer to perform data left shift processing on the matrix elements of the sub-matrix based on the number of columns of the sub-matrix.

[0106] Optionally, the shift step of the matrix element left shift is (n-1)×k, that is, the matrix element read for the first time is left shifted by 0, the matrix element read for the second time is left shifted by k, the matrix element read for the third time is left shifted by 2k, and so on.

[0107] Schematically, as shown in FIG6 , the matrix to be transposed is divided into 5×k sub-matrices. After the matrix to be transposed after the right shift is completely written into the matrix buffer, in the first step, the transposer reads the matrix elements in the first sub-matrix from the matrix buffer, and shifts the matrix elements left by 0 steps, and then performs transposition on the first sub-matrix to obtain a k×5 matrix obtained by transposing the matrix elements from the 1st to the kth columns in the matrix to be transposed, and writes the matrix elements of the first k×5 matrix into the lower-level memory.

[0108] In the second step, the transposer reads the matrix elements in the second submatrix from the matrix buffer, shifts the matrix elements left by k steps, and then performs a transposition on the second submatrix to obtain a k×5 matrix obtained by transposing the matrix elements in the k+1th to 2kth columns in the matrix to be transposed, and writes the matrix elements of the second k×5 matrix into the lower-level memory.

[0109] The processes of the third and fourth steps are similar to the second step (the left shift step sizes are 2k and 3k respectively), and will not be repeated in this embodiment.

[0110] In the fifth step, the transposer reads the matrix elements in the fifth submatrix from the matrix buffer and left-shifts them by 4k steps. This transposes the fifth submatrix, resulting in a k×5 matrix consisting of the transposed matrix elements in columns 4k+1 through 5k of the matrix to be transposed. Because the k×5 matrix contains padding data, the last several columns of the fifth k×5 matrix must be removed based on the number of padded columns read before writing to the lower-level memory. The remaining matrix elements are then written to the lower-level memory.

[0111] Because the number of matrix elements read from the matrix buffer is consistent each time, a transposer with unified transposition logic can be used for the same matrix to be transposed. In one possible design, the transposer can be composed of multiple multiplexers. When performing the transposition process on the matrix elements, each multiplexer only needs to perform a gated output based on the configured data output terminal.

[0112] In one possible implementation, the controller is configured to configure the data output end of the multiplexer in the transposer based on the matrix size of the matrix to be transposed, and the transposer is configured to perform a gating output on the matrix elements of the submatrix through the configured multiplexer to obtain a transposed submatrix.

[0113] The controller configures different multiplexer gating logics for matrices to be transposed of different matrix sizes.

[0114] For example, for a matrix to be transposed with a Class A matrix size, the controller configures the multiplexer selection logic for it so that data input terminal 1 selects data output terminal 1, data input terminal 2 selects data output terminal 2, and data input terminal 3 selects data output terminal 3; for a matrix to be transposed with a Class B matrix size, the controller configures the multiplexer selection logic for it so that data input terminal 1 selects data output terminal 1, data input terminal 2 selects data output terminal 3, data input terminal 3 selects data output terminal 5, data input terminal 4 selects data output terminal 7, data input terminal 5 selects data output terminal 2, data input terminal 6 selects data output terminal 4, data input terminal 7 selects data output terminal 6, and data input terminal 8 selects data output terminal 8.

[0115] Exemplarily, since the number of matrix elements in each transposition sub-matrix is ​​related to the read interface bit width of the upper-level memory, the number of multiplexers in the transposer matches the read interface bit width of the upper-level memory.

[0116] Furthermore, in some embodiments, the number of multiplexers is positively correlated with the read interface width. For example, when the number of bits of a matrix element is 32 bits and the read interface width is 1024 bits, the transposer is configured with 32 multiplexers. If the read interface width is 2048 bits, the transposer is configured with 64 multiplexers.

[0117] In combination with the above embodiments, several typical matrix transposition processes are described below using illustrative examples.

[0118] 1. The number of rows and columns of the matrix to be transposed is equal, and the bit width of the row matrix elements is the same as the read interface bit width of the upper-level memory.

[0119] Taking an 8×8 matrix as an example, as shown in FIG7 , the matrix reader reads one row of matrix elements in the matrix to be transposed each time, and right-shifts the read matrix elements with a unit step size of k=1, that is, the matrix elements in the first row are right-shifted by 0, the matrix elements in the second row are right-shifted by 1, the matrix elements in the third row are right-shifted by 2, and so on, thereby writing the matrix to be transposed after the data shift into the matrix buffer.

[0120] The transposer reads the matrix elements in the first submatrix from the matrix buffer, performs a left shift of 0 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory.

[0121] Similarly, the transposer reads the matrix elements in the second sub-matrix from the matrix buffer, performs a left shift of 1 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed sub-matrix obtained by transposition into the lower-level memory.

[0122] Similarly, the transposer completes the transposition of the remaining 6 sub-matrices and finally writes the transposed 8×8 matrix into the lower-level cache.

[0123] 2. The number of rows and columns of the matrix to be transposed is equal, but the bit width of the row matrix element is smaller than the read interface bit width of the upper-level memory.

[0124] Taking an 8×8 matrix with a row matrix element width of 1024 bits and a read interface width of 2048 bits as an example, as shown in Figure 8, the matrix reader reads 2 rows of matrix elements in the matrix to be transposed each time, with a unit step size of k=2, and performs a right shift on the read matrix elements, that is, the first row of matrix elements is right shifted by 0, the second row of matrix elements is right shifted by 2, the third row of matrix elements is right shifted by 4, and so on, thereby writing the matrix to be transposed after the data shift is written into the matrix buffer.

[0125] The transposer reads the matrix elements in the first submatrix from the matrix buffer, performs a left shift of 0 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory.

[0126] Similarly, the transposer reads the matrix elements in the second sub-matrix from the matrix buffer, performs a left shift of 2 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed sub-matrix obtained by transposition into the lower-level memory.

[0127] Similarly, the transposer completes the transposition of the remaining two sub-matrices and finally writes the transposed 8×8 matrix into the lower-level cache.

[0128] 3. The number of rows and columns of the matrix to be transposed is not equal (M is an integer multiple of N), and the bit width of the row matrix element is equal to the read interface bit width of the upper-level memory.

[0129] Taking a 4×8 matrix as an example, as shown in FIG9 , the matrix reader reads one row of matrix elements in the matrix to be transposed each time, and right-shifts the read matrix elements with a unit step size of k=2, that is, the matrix elements in the first row are right-shifted by 0, the matrix elements in the second row are right-shifted by 2, the matrix elements in the third row are right-shifted by 4, and so on, thereby writing the matrix to be transposed after the data shift into the matrix buffer.

[0130] The transposer reads the matrix elements in the first submatrix from the matrix buffer, performs a left shift of 0 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory.

[0131] Similarly, the transposer reads the matrix elements in the second sub-matrix from the matrix buffer, performs a left shift of 2 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed sub-matrix obtained by transposition into the lower-level memory.

[0132] Similarly, the transposer completes the transposition of the remaining two sub-matrices and finally writes the transposed 8×4 matrix into the lower-level cache.

[0133] It should be noted that when M cannot divide N, additional padding operations are required before performing data shifts, and padding removal is required before writing the last transposed submatrix. For example, taking a 4×7 matrix as an example, an additional column of matrix elements is required before performing data shifts, and the last row of the transposed submatrix needs to be removed before writing the last transposed submatrix.

[0134] In one application scenario of the present application, a matrix transposition device is used to process the training and inference process of a neural network. The matrix transposition device is called a parameter matrix transposition device of a neural network. The device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller.

[0135] The controller is used to control the matrix reader to read the parameter elements of the neural network parameter matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of parameter elements in the neural network parameter matrix each time;

[0136] The controller is used to control the shifter to perform data shift on the parameter elements read by the matrix reader, and to control the shifter to write the shifted parameter elements into the matrix buffer;

[0137] Optionally, the controller is further configured to determine the number of submatrix columns of the submatrix based on a read interface bit width of the upper-level memory and a matrix parameter of the neural network parameter matrix;

[0138] Determine the shift step size corresponding to the parameter elements read by the matrix reader each time based on the number of submatrix columns;

[0139] Based on the shift step size, the control shifter performs data right shift on the parameter elements read by the matrix reader.

[0140] Optionally, the matrix buffer is composed of several memory banks;

[0141] The controller is used to control the shifter to write the shifted parameter elements into a plurality of memory banks in the matrix buffer;

[0142] The controller is further configured to control the transposer to read parameter elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix.

[0143] The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of parameter elements in the neural network parameter matrix;

[0144] Optionally, the transposer is composed of a plurality of multiplexers;

[0145] The controller is further configured to configure a data output terminal of a multiplexer in the transposer based on a matrix size of a neural network parameter matrix;

[0146] The transposer is used for performing gated output on matrix elements of the submatrix through the configured multiplexer to obtain a transposed submatrix.

[0147] The controller is used for controlling the matrix writer to write the transposed submatrix output by the transposer into the lower-level memory.

[0148] The lower-level memory is used to store the transposed result of the neural network parameter matrix, and / or provide the transposed result of the neural network parameter matrix to a processor (or processing device) that performs the training and inference process of the neural network.

[0149] In another application scenario of the present application, a matrix transposition device is used to perform image processing, such as image flipping or rotation through matrix transposition. The matrix transposition device is called an image transposition device; the device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller;

[0150] The controller is used to control the matrix reader to read pixel elements of the image pixel matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of pixel elements in the image pixel matrix each time;

[0151] The controller is used to control the shifter to perform data shift on the pixel elements read by the matrix reader, and to control the shifter to write the shifted pixel elements into the matrix buffer;

[0152] Optionally, the controller is further configured to determine the number of submatrix columns of the submatrix based on a read interface bit width of the upper-level memory and matrix parameters of the image pixel matrix;

[0153] Determine the shift step size corresponding to each pixel element read by the matrix reader based on the number of submatrix columns;

[0154] Based on the shift step size, the shifter is controlled to perform right shift of the data on the pixel elements read by the matrix reader.

[0155] Optionally, the matrix buffer is composed of several memory banks;

[0156] The controller is used to control the shifter to write the shifted pixel elements into multiple memory banks in the matrix buffer;

[0157] The controller is further configured to control the transposer to read pixel elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix.

[0158] The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of pixel elements in the image pixel matrix;

[0159] Optionally, the transposer is composed of a plurality of multiplexers;

[0160] The controller is further configured to configure the data output terminals of the multiplexer in the transposer based on the matrix size of the image pixel matrix;

[0161] The transposer is used for performing gate output on pixel elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

[0162] The controller is used for controlling the matrix writer to write the transposed submatrix output by the transposer into the lower-level memory.

[0163] The lower-level memory is used to store the transposed result of the image pixel matrix and / or provide the transposed result of the image pixel matrix to a processor (or processing device) that performs the training and inference process of the neural network.

[0164] In another application scenario of the present application, a matrix transposition device is used to perform signal processing, such as implementing a Fourier transform (FFT) of a signal or a sub-process of the Fourier transform through matrix transposition; the matrix transposition device is called a signal transposition device; the device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller;

[0165] The controller is used to control the matrix reader to read the signal elements of the signal matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of signal elements in the signal matrix each time;

[0166] The controller is used to control the shifter to perform data shift on the signal elements read by the matrix reader, and to control the shifter to write the shifted signal elements into the matrix buffer;

[0167] Optionally, the controller is further configured to determine the number of sub-matrix columns of the sub-matrix based on a read interface bit width of the upper-level memory and matrix parameters of the signal matrix;

[0168] Determine the shift step size corresponding to each signal element read by the matrix reader based on the number of submatrix columns;

[0169] Based on the shift step size, the shifter is controlled to perform data right shift on the signal elements read by the matrix reader.

[0170] Optionally, the matrix buffer is composed of several memory banks;

[0171] The controller is used to control the shifter to write the shifted signal elements into a plurality of memory banks in the matrix buffer;

[0172] The controller is further configured to control the transposer to read signal elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix.

[0173] The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of signal elements in the signal matrix;

[0174] Optionally, the transposer is composed of a plurality of multiplexers;

[0175] The controller is further configured to configure the data output terminals of the multiplexer in the transposer based on the matrix size of the signal matrix;

[0176] The transposer is used to perform gating output on the signal elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

[0177] The controller is used for controlling the matrix writer to write the transposed submatrix output by the transposer into the lower-level memory.

[0178] The lower-level memory is used to store the transposed result of the signal matrix and / or provide the transposed result of the signal matrix to a processor (or processing device) that performs the training and inference process of the neural network.

[0179] In another application scenario of the present application, a matrix transposition device is used to perform correlation analysis of a graph network, such as calculating the relationship source between nodes in the network or determining the relationship pattern between nodes in the network through matrix transposition; the matrix transposition device is called an information transposition device; the device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller;

[0180] The controller is used to control the matrix reader to read information elements of the graph network information matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of information elements in the graph network information matrix each time;

[0181] The controller is used to control the shifter to perform data shift on the information elements read by the matrix reader, and to control the shifter to write the shifted information elements into the matrix buffer;

[0182] Optionally, the controller is further configured to determine the number of submatrix columns of the submatrix based on a read interface bit width of the upper-level memory and matrix parameters of the graph network information matrix;

[0183] Determining the shift step size corresponding to each information element read by the matrix reader based on the number of submatrix columns;

[0184] Based on the shift step size, the shifter is controlled to perform data right shift on the information elements read by the matrix reader.

[0185] Optionally, the matrix buffer is composed of several memory banks;

[0186] The controller is used for controlling the shifter to write the shifted information elements into a plurality of memory banks in the matrix buffer;

[0187] The controller is further configured to control the transposer to read information elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix.

[0188] The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of information elements in the graph network information matrix;

[0189] Optionally, the transposer is composed of a plurality of multiplexers;

[0190] The controller is further configured to configure a data output terminal of a multiplexer in the transposer based on a matrix size of the graph network information matrix;

[0191] The transposer is used to perform gating output on the information elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

[0192] The controller is used for controlling the matrix writer to write the transposed submatrix output by the transposer into the lower-level memory.

[0193] The lower-level memory is used to store the transposed result of the graph network information matrix, and / or provide the transposed result of the graph network information matrix to a processor (or processing device) that performs the training and inference process of the neural network.

[0194] Please refer to FIG10 , which shows a flowchart of a matrix transposition method provided by an exemplary embodiment of the present application. The method is applied to a matrix transposition device, which includes a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller. The method includes:

[0195] In step 1001 , a controller controls a matrix reader to read matrix elements of a matrix to be transposed from a superior memory, wherein the matrix reader reads at least one row of matrix elements of the matrix to be transposed each time.

[0196] In step 1002 , the controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, and controls the shifter to write the shifted matrix elements into the matrix buffer.

[0197] In step 1003 , the controller controls the transposer to read a submatrix from the matrix buffer and to perform transposition on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed.

[0198] Step 1004 : The controller controls the matrix writer to write the transposed sub-matrix output by the transposer into the lower-level memory.

[0199] In some embodiments, the controller controls the shifter to perform data shifting on the matrix elements read by the matrix reader, including:

[0200] Determine the number of submatrix columns of the submatrix based on the read interface bit width of the upper memory and the matrix parameters of the matrix to be transposed;

[0201] Determine the shift step size corresponding to each matrix element read by the matrix reader based on the number of submatrix columns;

[0202] Based on the shift step size, the shifter is controlled to perform data right shift on the matrix elements read by the matrix reader.

[0203] In some embodiments, determining the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed includes:

[0204] When the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, the column-row ratio of the matrix to be transposed is rounded up to determine the number of submatrix columns, and the row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element;

[0205] Alternatively, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the result of rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed is determined as the number of columns of the submatrix, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element.

[0206] In some embodiments, the method further comprises:

[0207] When the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-row ratio is a non-integer greater than 1, the controller controls the matrix reader to perform padding after the read matrix element;

[0208] The controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer, and writes matrix elements in the at least one transposed submatrix after padding removal into the lower-level memory.

[0209] In some embodiments, the controller controls the matrix reader to perform padding after the read matrix element, including:

[0210] Determine the number of padding columns based on the number of rows and columns of the matrix to be transposed;

[0211] Controls the matrix reader to perform padding after the read matrix elements based on the number of padding columns;

[0212] The controller controls the matrix writer to perform padding removal on at least one transposed sub-matrix output by the transposer, including:

[0213] The controller controls the matrix writer to perform padding removal on the at least one transposed submatrix output by the transposer based on the number of padding columns, wherein the number of rows of the sum of matrix elements to be removed from the at least one transposed submatrix is ​​the same as the number of padding columns.

[0214] In some embodiments, the matrix buffer is composed of several memory banks;

[0215] The controller controls the transposer to read the sub-matrix from the matrix buffer, including:

[0216] Controlling the shifter to write the shifted matrix elements into multiple memory banks in the matrix buffer;

[0217] Based on the number of submatrix columns, the transposer is controlled to read the matrix elements of the submatrix in parallel from multiple memory banks of the matrix buffer.

[0218] In some embodiments, controlling the transpose device to read matrix elements of a sub-matrix in parallel from a plurality of memory banks of a matrix buffer includes:

[0219] Determine the read address of each memory bank based on the number of sub-matrix columns, and the read address of the memory bank is incremented;

[0220] The transposer is controlled to read the matrix elements of the sub-matrix in parallel from the plurality of memory banks of the matrix buffer based on the read address.

[0221] In some embodiments, after the controller controls the transposer to read the sub-matrix from the matrix buffer, the method further includes:

[0222] Controls the transposer to perform data left shift on the matrix elements of the submatrix based on the number of columns of the submatrix.

[0223] In some embodiments, the transposer is composed of a plurality of multiplexers;

[0224] The method further includes:

[0225] The controller configures the data output terminal of the multiplexer in the transposer based on the matrix size of the matrix to be transposed;

[0226] The transposer performs gated output on matrix elements of the submatrix through the configured multiplexer to obtain a transposed submatrix.

[0227] In some embodiments, the number of multiplexers in the transposer matches the read interface width of the upper-level memory.

[0228] The detailed process of the matrix transposition device performing matrix transposition can be referred to the above device embodiment, and will not be described in detail in this embodiment.

[0229] In some embodiments, the matrix transposition device in the embodiments of the present application can be integrated into an AI processor.

[0230] Please refer to Figure 11, which shows a block diagram of a computer device 1100 according to an exemplary embodiment of the present application. Computer device 1100 may be a portable mobile terminal, such as a smartphone, a tablet computer, a Moving Picture Experts Group Audio Layer III (MP3) player, or a Moving Picture Experts Group Audio Layer IV (MP4) player. Computer device 1100 may also be referred to as a user device, a portable terminal, a workstation, a server, or other similar names.

[0231] Typically, the computer device 1100 includes a processor 1101 and a memory 1102 .

[0232] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1101 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0233] In some embodiments, the processor 1101 may be integrated with the matrix transposition device provided in the above embodiments. When the processor 1101 has a matrix transposition requirement, the matrix transposition device may be used to perform the transposition operation.

[0234] The memory 1102 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.

[0235] In some embodiments, the computer device 1100 may optionally further include a peripheral device interface 1103 and at least one peripheral device.

[0236] Those skilled in the art will appreciate that the structure shown in FIG11 does not limit the computer device 1100 and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0237] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0238] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A matrix transposition device, characterized in that: The device comprises: Matrix readers, shifters, matrix buffers, transposers, matrix writers, and controllers; The controller is used to control the matrix reader to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time; The controller is used to control the shifter to perform data shift on the matrix elements read by the matrix reader, and control the shifter to write the shifted matrix elements into the matrix buffer; The controller is used to control the transposer to read a submatrix from the matrix buffer, and control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed; The controller is used to control the matrix writer to write the transposed sub-matrix output by the transposer into a lower-level memory.

2. The device according to claim 1, characterized in that The controller is used to: Determine the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed; Based on the number of columns of the submatrix, determining the shift step length corresponding to the matrix elements read by the matrix reader each time; Based on the shift step length, the shifter is controlled to perform data right shift on the matrix elements read by the matrix reader.

3. The device according to claim 2, characterized in that The controller is used to: In a case where the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, a result of rounding up the column-row ratio of the matrix to be transposed is determined as the number of columns of the submatrix, and the row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element; Or, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the result of rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed is determined as the number of columns of the submatrix, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element.

4. The device according to claim 3, characterized in that The controller is further used for: When the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-to-row ratio is a non-integer greater than 1, controlling the matrix reader to perform padding after the read matrix element; The matrix writer is controlled to perform padding removal on at least one transposed submatrix output by the transposer, and matrix elements in at least one transposed submatrix after padding removal are written into the lower-level memory.

5. The device according to claim 4, characterized in that The controller is used to: Determine the number of filled columns based on the number of rows and columns of the matrix to be transposed; Controlling the matrix reader to perform padding after the read matrix elements based on the number of padding columns; The controller is further used for: The matrix writer is controlled to perform padding removal on the at least one transposed submatrix output by the transposer based on the padding column number, wherein the sum of the number of rows of matrix elements to be removed from the at least one transposed submatrix is ​​the same as the padding column number.

6. The device according to any one of claims 2 to 5, characterized in that: The matrix buffer is composed of a plurality of memory banks; The controller is used to control the shifter to write the shifted matrix elements into a plurality of memory banks in the matrix buffer; The controller is further used to control the transposer to read the matrix elements of the submatrix in parallel from the multiple memory banks of the matrix buffer based on the number of columns of the submatrix.

7. The device according to claim 6, characterized in that The controller is used to: Determining a read address of each memory bank based on the number of columns of the sub-matrix, the read address of the memory bank being incremented; The transposer is controlled based on the read address to read matrix elements of the sub-matrix in parallel from a plurality of memory banks of the matrix buffer.

8. The device according to any one of claims 2 to 7, characterized in that: The controller is further used for: Based on the number of columns of the sub-matrix, the transposer is controlled to perform data left shift on the matrix elements of the sub-matrix.

9. The device according to any one of claims 1 to 8, characterized in that: The transposer is composed of a plurality of multiplexers; The controller is further used to configure the data output end of the multiplexer in the transposer based on the matrix size of the matrix to be transposed; The transposer is used to perform gating output on the matrix elements of the sub-matrix through the configured multiplexer to obtain the transposed sub-matrix.

10. The device according to claim 9, characterized in that The number of multiplexers in the transposer matches the read interface bit width of the upper-level memory.

11. A matrix transposition method, characterized in that: The method is applied to a matrix transposition device, the matrix transposition device comprising a matrix reader, a shifter, a matrix buffer, a transposition device, a matrix writer and a controller, and the method comprises: The controller controls the matrix reader to read matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time; The controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, and controls the shifter to write the shifted matrix elements into the matrix buffer; The controller controls the transposer to read a submatrix from the matrix buffer, and controls the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed; The controller controls the matrix writer to write the transposed submatrix output by the transposer into a lower-level memory.

12. The method according to claim 11, characterized in that The controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, including: The controller determines the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed; The controller determines the shift step length corresponding to the matrix elements read by the matrix reader each time based on the number of columns of the submatrix; The controller controls the shifter to perform data right shift on the matrix elements read by the matrix reader based on the shift step size.

13. The method according to claim 12, characterized in that The controller determines the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed, including: In a case where the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, the controller determines a result of rounding up the column-row ratio of the matrix to be transposed as the number of columns of the submatrix, and the row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element; Or, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the controller determines the number of columns of the submatrix by rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element.

14. The method according to claim 13, characterized in that The method further comprises: In the case where the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-to-row ratio is a non-integer greater than 1, the controller controls the matrix reader to perform padding after the read matrix element; The controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer, and writes matrix elements in at least one transposed submatrix after padding removal into the lower-level memory.

15. The method according to claim 14, characterized in that The controller controls the matrix reader to perform padding after the read matrix elements, including: The controller determines the number of filling columns based on the number of rows and the number of columns of the matrix to be transposed; The controller controls the matrix reader to perform padding after the read matrix element based on the number of padding columns; The controller controls the matrix writer to perform padding removal on at least one transposed sub-matrix output by the transposer, including: The controller controls the matrix writer to perform padding removal on the at least one transposed submatrix output by the transposer based on the number of padding columns, wherein the number of rows of the sum of matrix elements to be removed from the at least one transposed submatrix is ​​the same as the number of padding columns.

16. The method according to any one of claims 12 to 15, characterized in that: The method further comprises: Based on the number of columns of the sub-matrix, the controller controls the transposer to perform data left shift on matrix elements of the sub-matrix.

17. The method according to any one of claims 11 to 16, characterized in that: The transposer is composed of a plurality of multiplexers; The method further comprises: The controller configures the data output terminal of the multiplexer in the transposer based on the matrix size of the matrix to be transposed; The transposer performs gating output on the matrix elements of the sub-matrix through the configured multiplexer to obtain the transposed sub-matrix.

18. An AI processor, characterized in that: The processor comprises the matrix transposition device according to any one of claims 1 to 10.

19. A computer device, characterized in that: The computer device comprises the AI ​​processor as claimed in claim 18 and a memory, wherein the AI ​​processor is connected to the memory via a bus.

Images