A matrix transposition method, system, device and computer readable storage medium
By using separate write and broadcast write modes in the AI hardware accelerator to store matrix data into VSRAM and combining it with HSRAM for matrix operations, the problems of increased chip area and low computational efficiency caused by matrix transpose modules in the prior art are solved, and efficient matrix transpose operation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SIENGINE TECH CO LTD
- Filing Date
- 2026-04-17
- Publication Date
- 2026-07-07
AI Technical Summary
The existing technology has problems with increasing chip area and low computational efficiency due to the design of a dedicated matrix transpose module.
The matrix transpose operation is achieved by storing the matrix data to be transposed into vertical static random access memory (VSRAM) based on separate write mode and broadcast write mode, and then performing matrix operations by combining multiply-accumulate array (MAC) and horizontal static random access memory (HSRAM).
Performing matrix transpose at runtime reduces additional control logic and storage requirements, saves chip area, and improves computational efficiency.
Smart Images
Figure CN122044522B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence hardware accelerator technology, specifically to a matrix transpose method, system, device, and computer-readable storage medium. Background Technology
[0002] Currently, to perform machine learning, especially deep learning tasks, efficiently and with low power consumption, specialized chips or modules (such as neural processing units, NPUs) are often designed to accelerate the application of artificial intelligence (AI) and machine learning (ML). In modern AI hardware accelerators, matrix operations account for the vast majority of computation, with matrix transpose being a common operation.
[0003] In related technologies, a dedicated matrix transpose module is typically designed to perform data transpose. This involves first writing the data to be transposed into this module, then processing it according to certain rules, and finally reading the transposed data and storing it in a storage module. The storage module then writes the data into VSRAM (Vertical SRAM) for subsequent matrix multiplication and other processing. This approach requires additional digital logic control circuitry and independent storage resources (such as SRAM and registers), increasing chip area. Furthermore, the need to write data to the transpose module first, process it, and then read it increases processing latency and reduces computational efficiency. Summary of the Invention
[0004] This application provides a matrix transposition method, system, device, and computer-readable storage medium, which can solve the technical problems of large chip area and low computational efficiency caused by designing a dedicated matrix transposition module to achieve data transposition in the prior art.
[0005] In a first aspect, embodiments of this application provide a matrix transposition method, the matrix transposition method comprising:
[0006] When the matrix data to be transposed is a non-intermediate operation result, the matrix data to be transposed is cyclically stored into each vertical static random access memory (VSRAM) based on the individual write mode to achieve matrix transposition.
[0007] When the data of the matrix to be transposed is an intermediate operation result, the data of the matrix to be transposed is broadcast and stored into each VSRAM based on the broadcast write mode to achieve matrix transposition;
[0008] The multiply-accumulate-add array (MAC) performs matrix operations based on data stored in VSRAM and horizontal static random access memory (HSRAM), with HSRAM used to store data that does not require transposition.
[0009] In conjunction with the first aspect, in one implementation, the step of cyclically storing the transpose matrix data into each VSRAM based on a separate write mode includes:
[0010] If the data to be transposed is stored in external memory in row-major order, the data to be transposed is read row by row using direct memory access and based on 2D data transfer function, and the data of the nth row is written into the nth VSRAM in sequence, where n is a non-negative integer;
[0011] When the m-th row of data is read, the data is written back to the 0th VSRAM and the process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs.
[0012] In conjunction with the first aspect, in one implementation, the step of cyclically storing the transpose matrix data into each VSRAM based on a separate write mode includes:
[0013] If the data to be transposed is stored in external memory in column-major order, the data to be transposed is read column by column using direct memory access and based on a one-dimensional data transfer mode, and the data of the nth column is written sequentially into the nth VSRAM, where n is a non-negative integer;
[0014] When the data in column m is read, the data is written back to the 0th VSRAM and this process is repeated until the data of the transpose matrix is written. Here, m is an integer multiple of the number of VSRAMs.
[0015] In conjunction with the first aspect, in one implementation, the step of broadcasting the transpose matrix data into each VSRAM based on the broadcast write mode includes:
[0016] Broadcast the k-th row of the matrix data to be transposed to all VSRAMs, where k is a non-negative integer representing the row index of the matrix;
[0017] By controlling the byte enable signal and data shifting, each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0018] In conjunction with the first aspect, in one implementation, the step of controlling the byte enable signal and data shifting to ensure that each element of the k-th row of data is stored at the k-th byte position of each VSRAM includes:
[0019] Set the byte enable signal according to the row index k to be written, so that only the k-th byte of data is valid;
[0020] The valid byte index, valid_byte_id, is determined by finding the position of the valid bit in the byte enable signal.
[0021] The valid_byte_id is compared with the VSRAM number vsram_id to determine the data shift direction and the number of bits to shift.
[0022] Based on the data shift direction and the number of data shift bits, the elements of the k-th row of data are shifted so that each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0023] In conjunction with the first aspect, in one implementation, comparing the valid_byte_id with the VSRAM number vsram_id to determine the data shift direction includes:
[0024] When vsram_id is greater than valid_byte_id, the data shift direction is to the right;
[0025] When vsram_id is less than valid_byte_id, the data shift direction is to the left;
[0026] When vsram_id equals valid_byte_id, the data shift direction is no shift.
[0027] Secondly, embodiments of this application provide a matrix transpose system, the matrix transpose system comprising:
[0028] The SRAM bus matrix module is used to cyclically store the matrix data to be transposed into each vertical static random access memory (VSRAM) based on a separate write mode when the matrix data to be transposed is a non-intermediate operation result, in order to achieve matrix transposition; and to broadcast the matrix data to be transposed into each VSRAM based on a broadcast write mode when the matrix data to be transposed is an intermediate operation result, in order to achieve matrix transposition.
[0029] The multiply-accumulate array (MAC) is used to perform matrix operations based on data stored in VSRAM and lateral static random access memory (HSRAM), with HSRAM used to store data that does not require transposition.
[0030] In conjunction with the second aspect, in one implementation, the SRAM bus matrix module is specifically used for:
[0031] If the data to be transposed is stored in external memory in row-major order, the data to be transposed is read row by row using direct memory access and based on 2D data transfer function, and the data of the nth row is written into the nth VSRAM in sequence, where n is a non-negative integer;
[0032] When the m-th row of data is read, the data is written back to the 0th VSRAM and the process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs.
[0033] In conjunction with the second aspect, in one embodiment, the SRAM bus matrix module is further configured to:
[0034] If the data to be transposed is stored in external memory in column-major order, the data to be transposed is read column by column using direct memory access and based on a one-dimensional data transfer mode, and the data of the nth column is written sequentially into the nth VSRAM, where n is a non-negative integer;
[0035] When the data in column m is read, the data is written back to the 0th VSRAM and this process is repeated until the data of the transpose matrix is written. Here, m is an integer multiple of the number of VSRAMs.
[0036] In conjunction with the second aspect, in one embodiment, the SRAM bus matrix module is further configured to:
[0037] Broadcast the k-th row of the matrix data to be transposed to all VSRAMs, where k is a non-negative integer representing the row index of the matrix;
[0038] By controlling the byte enable signal and data shifting, each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0039] In conjunction with the second aspect, in one embodiment, the SRAM bus matrix module is further configured to:
[0040] Set the byte enable signal according to the row index k to be written, so that only the k-th byte of data is valid;
[0041] The valid byte index, valid_byte_id, is determined by finding the position of the valid bit in the byte enable signal.
[0042] The valid_byte_id is compared with the VSRAM number vsram_id to determine the data shift direction and the number of bits to shift.
[0043] Based on the data shift direction and the number of data shift bits, the elements of the k-th row of data are shifted so that each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0044] In conjunction with the second aspect, in one embodiment, the SRAM bus matrix module is further configured to:
[0045] When vsram_id is greater than valid_byte_id, the data shift direction is to the right;
[0046] When vsram_id is less than valid_byte_id, the data shift direction is to the left;
[0047] When vsram_id equals valid_byte_id, the data shift direction is no shift.
[0048] Thirdly, embodiments of this application provide a matrix transposition device, which includes a processor, a memory, and a matrix transposition program stored in the memory and executable by the processor, wherein when the matrix transposition program is executed by the processor, it implements the steps of the matrix transposition method as described above.
[0049] Fourthly, embodiments of this application provide a computer-readable storage medium storing a matrix transpose program, wherein when the matrix transpose program is executed by a processor, it implements the steps of the matrix transpose method as described above.
[0050] The beneficial effects of the technical solutions provided in this application include:
[0051] When the data to be transposed is a non-intermediate operation result, the data is cyclically stored into each VSRAM using a separate write mode to achieve matrix transposition. When the data to be transposed is an intermediate operation result, it is broadcast into each VSRAM using a broadcast write mode to achieve matrix transposition. The MAC array performs matrix operations based on the data stored in the HSRAM (used to store data that does not need to be transposed) and the data stored in the VSRAM. Therefore, this application achieves matrix transposition at runtime (on-the-fly), integrating the matrix transposition operation with other operations, eliminating the need for additional control logic and storage, and avoiding additional latency. This effectively reduces chip area and improves the computational efficiency of AI hardware accelerators. Attached Figure Description
[0052] Figure 1 This is a data flow architecture diagram involving the embodiments of this application;
[0053] Figure 2 This is a flowchart illustrating an embodiment of the matrix transpose method of this application;
[0054] Figure 3 This is a schematic diagram illustrating the intermediate computation result data conversion involved in the embodiments of this application;
[0055] Figure 4 This is a schematic diagram of the hardware structure of the matrix transpose device involved in the embodiments of this application. Detailed Implementation
[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0057] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.
[0058] HSRAM (Horizontal SRAM): See also Figure 1 As shown, it refers to the SRAM located on the left side of the MAC array, whose data appears to enter the MAC Array (Multiply Accumulate Array, also known as the MAC array) horizontally.
[0059] VSRAM (Vertical SRAM): See also Figure 1 As shown, this refers to the SRAM located on top of the MAC array, whose data appears to enter the MAC array vertically.
[0060] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0061] In a first aspect, embodiments of this application provide a matrix transpose method.
[0062] In one embodiment, reference is made to Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the matrix transpose method of this application. Figure 2 As shown, matrix transpose methods include:
[0063] Step S10: When the matrix data to be transposed is a non-intermediate operation result, the matrix data to be transposed is cyclically stored into each vertical static random access memory (VSRAM) based on the individual write mode to achieve matrix transposition; wherein, the multiply-accumulate operation array (MAC) performs matrix operations based on the data stored in the VSRAM and the horizontal static random access memory (HSRAM), and the HSRAM is used to store data that does not need to be transposed.
[0064] In this exemplary embodiment, it is assumed that two matrices A and B need to be multiplied, where both A and B are 16×16 in size and have an INT8 data format. Matrix A needs to be stored in HSRAM, which stores data that does not need to be transposed; matrix B needs to be stored in VSRAM, which stores transposed data. Based on this, data is provided to a MAC array containing 16×16×16 = 4096 multipliers through 16 horizontal SRAMs (i.e., HSRAMs numbered hsram_id from 0 to 15) and 16 vertical SRAMs (i.e., VSRAMs numbered vsram_id from 0 to 15), so that the MAC array can perform matrix multiplication of A and B.
[0065] Since matrix multiplication involves multiplying the rows of matrix A by the columns of matrix B, matrix B usually needs to be transposed before performing the multiplication operation. Therefore, for the sake of simplicity in the accompanying diagram, Figure 1 The related logic of the 16 horizontal SRAMs is not described, and its specific implementation logic is common knowledge in the field. Furthermore, to enable parallel processing of MAC array computation and DMA (Direct Memory Access) data transfer, this embodiment will use a ping-pong buffer (a technique for efficient data processing through alternating double buffers), meaning there are two sets of HSRAM and VSRAM. However, for the sake of simplicity in the accompanying drawings, Figure 1 Only one set was drawn in the painting.
[0066] It should be noted that the data to be transposed refers to the data that needs to be transposed, that is, the data corresponding to matrix B, which needs to be written to VSRAM; non-intermediate operation results refer to the complete operation results stored in external memory; for the data to be transposed matrix of type non-intermediate operation results, the SRAM bus matrix (i.e., SRAM_BUS_MATRIX) in this embodiment completes the matrix transposition when reading it from external memory and storing it into VSRAM, that is, it is cyclically stored into each VSRAM through a separate write mode, so that VSRAM0 stores the 0th column of matrix B, VSRAM1 stores the 1st column of matrix B, and so on; when matrix multiplication is performed subsequently, VSRAM0 to VSRAM15 naturally provide the row data of the transposed matrix B, thus completing the matrix transposition operation. In summary, the data stored in VSRAM can be used for subsequent matrix multiplication operations after writing, that is, it is integrated with other operations; based on this, see Figure 1 As shown, the MAC array will perform operations such as matrix multiplication based on the data stored in VSRAM and HSRAM.
[0067] Furthermore, in one embodiment, the step of cyclically storing the transpose matrix data into each VSRAM based on a separate write mode includes:
[0068] If the data to be transposed is stored in external memory in row-major order, the data to be transposed is read row by row using direct memory access and based on 2D data transfer function, and the data of the nth row is written into the nth VSRAM in sequence, where n is a non-negative integer;
[0069] When the m-th row of data is read, the data is written back to the 0th VSRAM and the process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs.
[0070] As an example, in this embodiment, when the array data to be transposed is stored in memory in row-major order, the DMA will use 2D data transfer to read the data (128 bits per row) of the array data to be transposed row by row and transfer it to the SRAM bus matrix. It should be noted that 2D data... The specific workflow and principle of the transfer are common knowledge in this field, and will not be elaborated here for the sake of brevity. Then, the SRAM bus matrix sequentially writes the 0th row of data transferred by DMA into VSRAM0, the 1st row of data into VSRAM1, and so on. That is, VSRAM0 stores the 0th row of data (B[0][0] to B[0]
[15] ), VSRAM1 stores the 1st row of data (B[1][0] to B[1]
[15] )...VSRAM15 stores the 15th row of data (B
[15] [0] to B
[15]
[15] ). When the 16th row is processed, the data is returned to VSRAM0 to continue writing, and so on, until the data to be transposed matrix is written. In this way, the matrix transposition operation is naturally completed during the process of data transfer from memory to VSRAM. It should be noted that the value of m is an integer multiple of the number of VSRAMs. If the number of VSRAMs is 16, then when m=16, the data needs to be written back to VSRAM0. When m=32, the data also needs to be written back to VSRAM0, and so on.
[0071] Furthermore, in one embodiment, the step of cyclically storing the transpose matrix data into each VSRAM based on a separate write mode includes:
[0072] If the data to be transposed is stored in external memory in column-major order, the data to be transposed is read column by column using direct memory access and based on a one-dimensional data transfer mode, and the data of the nth column is written sequentially into the nth VSRAM, where n is a non-negative integer;
[0073] When the data in column m is read, the data is written back to the 0th VSRAM and this process is repeated until the data of the transpose matrix is written. Here, m is an integer multiple of the number of VSRAMs.
[0074] As an example, in this embodiment, when the transpose array data is stored in memory in column-major order, the DMA will use a 1D data transfer mode to read the transpose matrix data (128 bits per row) row by row and transfer it to the SRAM bus matrix. It should be noted that the specific workflow and principle of the 1D data transfer mode are common knowledge in the art and will not be elaborated here for the sake of brevity. Then, the SRAM bus matrix sequentially writes the data from column 0 of the DMA transfer into VSRAM0, the data from column 1 into VSRAM1, and so on. That is, VSRAM0 stores the data in column 0 (B[0][0] to B[0]
[15] ), VSRAM1 stores the data in column 1 (B[1][0] to B[1]
[15] )...VSRAM15 stores the data in column 15 (B
[15] [0] to B
[15]
[15] ); when processing the 16th column, the data is returned to VSRAM0 to continue writing, and so on, until the data of the matrix to be transposed is written. In this way, the matrix transposition operation is naturally completed during the process of data being transferred from memory to VSRAM.
[0075] Step S20: When the data to be transposed is an intermediate operation result, the data to be transposed is broadcast and stored in each VSRAM based on the broadcast write mode to achieve matrix transposition; wherein, the MAC array performs matrix operations based on the data stored in the VSRAM and HSRAM.
[0076] As an example, it is understandable that in certain computational scenarios, such as the attention computation in the Transformer architecture, it is necessary to transpose the intermediate result V matrix: Where Q represents the query, K represents the key, V represents the value, and d k This represents the dimensions of Q and K, or the dimensions of the model. The result of softmax() is stored in HSRAM, while V is the result of the previous matrix calculation. Therefore, when calculating Attention, it is obvious that the V matrix needs to be transposed. Based on this, see... Figure 1 As shown, the intermediate calculation results in this embodiment refer to incomplete calculation results output by the MAC array or other functional modules that are not stored in external memory. It should be understood that the V matrix itself can be calculated using the same MAC array. Therefore, when outputting the V matrix, the result can be written to another set of SRAM in the VSRAM ping-pong buffer, thereby effectively saving intermediate storage and handling time.
[0077] In this embodiment, for matrix data to be transposed that is of the type of intermediate operation result, the SRAM bus matrix completes the matrix transposition when storing it into VSRAM via broadcast write mode. For details, see [link to documentation]. Figure 3 As shown, assuming the horizontal axis represents the rows of the calculated V matrix, each row contains 16 elements, and each element is 8 bits (INT8), it needs to be transposed and stored in VSRAM; based on this, see... Figure 3 As shown, the SRAM bus matrix broadcasts and stores the 16 elements of V[0] into 16 VSRAMs, all stored in the 0th byte of each VSRAM; the 16 elements of V[1] are broadcast and stored into 16 VSRAMs, all stored in the 1st byte of each VSRAM; and so on, with the 16 elements of V
[15] broadcast and stored into 16 VSRAMs, all stored in the 15th byte of each VSRAM. It is worth noting that the data stored in the VSRAM in this embodiment can be used for subsequent matrix multiplication operations after writing, that is, it is integrated with other operations; based on this, the matrix transpose operation is completed simultaneously during the data writing process to the VSRAM, without the need for an additional transpose module.
[0078] In summary, this embodiment achieves matrix transposition at runtime (on-the-fly), eliminating a separate / dedicated matrix transposition module, significantly saving chip area. Furthermore, by integrating matrix transposition with other computational operations, it enables real-time transposition during data transmission, eliminating additional latency and improving the computational efficiency of the NPU. This is particularly suitable for AI models such as Transformer that require frequent matrix transposition. In addition, this embodiment effectively reduces design complexity and implementation costs by reusing existing hardware structures.
[0079] Furthermore, in one embodiment, the step of broadcasting the transpose matrix data into each VSRAM based on the broadcast write mode includes:
[0080] Broadcast the k-th row of the matrix data to be transposed to all VSRAMs, where k is a non-negative integer representing the row index of the matrix;
[0081] By controlling the byte enable signal and data shifting, each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0082] In this exemplary embodiment, if the data to be transposed is an intermediate operation result, the SRAM bus matrix adopts a broadcast write mode to broadcast the data of the kth row of matrix V output by the MAC Array or other functional modules to all VSRAMs; then, by controlling the byte enable signal and data shift, each element of the kth row in matrix V is stored in the kth byte position of each VSRAM, where k is a non-negative integer representing the row index of matrix V. It can be understood that byte enable refers to the byte enable when writing data. The byte enable of VSRAM is used as the input of the SRAM bus matrix and the byte enable values of the 16 VSRAMs are all the same, so that the SRAM bus matrix can implement data write control based on this. For example, for V[0], the byte enable value of the 16 VSRAMs is set to 16'h0001, indicating that only the 0th byte data is valid; for V[1], the byte enable value of the 16 VSRAMs is set to 16'h0002, indicating that only the 1st byte data is valid; and so on.
[0083] For details, see Figure 3 As shown, the 16 elements of V[0] need to be broadcast and stored in 16 VSRAMs, and are all stored in the 0th byte of each VSRAM. Therefore, the value of byte enable controlling the 16 VSRAMs is 16'h0001, which means that only the 0th byte data is valid, and the data of the other bytes is invalid and will not be written to the VSRAM. The 16 elements of V[1] need to be broadcast and stored in 16 VSRAMs, and are all stored in the 1st byte of each VSRAM. Therefore, the value of byte enable controlling the 16 VSRAMs is 16'h0002, which means that only the 1st byte data is valid, and the data of the other bytes is invalid and will not be written to the VSRAM. Similarly, the 16 elements of V
[15] need to be broadcast and stored in 16 VSRAMs, and are all stored in the 15th byte of each VSRAM. Therefore, the value of byte enable controlling the 16 VSRAMs is 16'h8000, which means that only the 15th byte data is valid, and the data of the other bytes is invalid and will not be written to the VSRAM.
[0084] Further, in one embodiment, the step of controlling the byte enable signal and data shifting to ensure that each element of the k-th row of data is stored at the k-th byte position of each VSRAM includes:
[0085] Set the byte enable signal according to the row index k to be written, so that only the k-th byte of data is valid;
[0086] The valid byte index, valid_byte_id, is determined by finding the position of the valid bit in the byte enable signal.
[0087] The valid_byte_id is compared with the VSRAM number vsram_id to determine the data shift direction and the number of bits to shift. Specifically, when vsram_id is greater than valid_byte_id, the data shift direction is to the right; when vsram_id is less than valid_byte_id, the data shift direction is to the left; and when vsram_id is equal to valid_byte_id, the data shift direction is no shift.
[0088] Based on the data shift direction and the number of data shift bits, the elements of the k-th row of data are shifted so that each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0089] As an example, in this embodiment, the data bit width and storage bit width of each VSRAM are both 128 bits / 16 bytes. However, when broadcasting storage, only one byte can be written to the corresponding position of the 16-byte storage bit width at a time, so it is necessary to control data shifting and byte enabling.
[0090] Specifically, firstly, set the byte enable signal according to the row index k to be written, so that only the kth byte of data is valid: for V[0], set the byte enable value of 16 VSRAMs to 16'h0001, indicating that only the 0th byte of data is valid; for V[1], set the byte enable value of 16 VSRAMs to 16'h0002, indicating that only the 1st byte of data is valid, and so on. Then, by finding the position of the valid bits in byte_enable[15:0], valid_byte_id[3:0] is determined, i.e., valid_byte_id[3:0] = find_first_1(byte_enable); where valid_byte_id[3:0] represents the 4-bit wide signal of the valid byte index position, i.e., [3:0] represents a 4-bit wide binary signal, which can represent the decimal value range of 0-15. valid_byte_id[3:0] is used to identify which byte position in the 128-bit wide VSRAM is the valid data position of the current operation; taking V[1] as an example, at this time byte enable=16'h0002, valid_byte_id[3:0] can be generated by the find_first_1 module or function, which is similar to a lookup table, i.e., the position of 1 in byte_enable is encoded to obtain valid_byte_id[3:0]. For example, after encoding 16'h0001, valid_byte_id[3:0] = Encoding 16'h0002 with 4'h0 results in valid_byte_id[3:0] = 4'h1. Similarly, encoding 16'h8000 results in valid_byte_id[3:0] = 4'hf.
[0091] Next, based on the input byte enable and VSRAM number (0~15), write data is generated for each VSRAM. That is, valid_byte_id[3:0] is compared with the VSRAM number vsram_id[3:0] (which represents the 4-bit wide signal of the VSRAM number) to determine the data shift method. Specifically, taking V[1] as an example, the data of V[1] needs to be broadcast and stored in 16 VSRAMs, and all are written in the first byte of each VSRAM. V[1] has 16 elements, each element is 8 bits long (i.e. 1 byte), and the total length is 128 bits, which can be recorded as V[1][127:0]. If vsram_id[3:0]>valid_byte_id[3:0], such as VSRAM[2], then vsram_id[3:0] (=2)>valid_byte_id[3:0] (=1), indicating that the data shift direction is to the right. Then, according to the formula vsram_wdata (i.e., transposed data) = The shift control is performed using wdata (i.e., the data before transpose) >> ((vsram_id[3:0]–valid_byte_id[3:0])<<3). For example, if the data to be written to VSRAM[2] is in V[1][23:16], but this byte of data needs to be written to the first byte position of VSRAM[2], then wdata (i.e., V[1][127:0]) needs to be shifted 8 bits to the right. The <<3 inside the parentheses in the above formula is shifted 3 bits to the left, which is equivalent to multiplying by 8, and vsram_id[3:0] - valid_byte_id[3:0] = 1. Thus, the whole is shifted 8 bits to the right. This will move the [23:16] data in V[1][127:0] to the [15:8] position, which corresponds to the first byte position.
[0092] If vsram_id[3:0] < valid_byte_id[3:0], such as VSRAM[0], then vsram_id[3:0] (=0) < valid_byte_id[3:0] (=1), indicating that the data shift direction is left shift at this time. Then, the shift control is performed according to the formula vsram_wdata = wdata << ((valid_byte_id – vsram_id) << 3): For example, the data to be written to VSRAM[0] is in V[1][7:0], but this byte of data needs to be written at the 1st byte position of VSRAM[0]. Therefore, wdata (that is, V[1][127:0]) needs to be shifted 8 bits to the left. The <<3 inside the parentheses of the above formula is to shift 3 bits to the left, which is equivalent to multiplying by 8, and valid_byte_id[3:0] – vsram_id[3:0] = 1. In this way, the overall shift is 8 bits to the left. Thus, the lower 8 bits of V[1][127:0] can be moved to the position of [15:8], which exactly corresponds to the 1st byte position. If vsram_id[3:0] = valid_byte_id[3:0], such as VSRAM[1], at this time, vsram_id[3:0] (=1) and valid_byte_id[3:0] (=1) are equal, indicating that no shift is required at this time, that is, vsram_wdata = wdata. The data to be written to VSRAM[1] exactly falls on V[1][15:8]. Therefore, no data shift is required, and the data is naturally in the correct position. In summary, through the above method, during the process of writing data to VSRAM, the matrix transpose operation is completed simultaneously, eliminating a separate / special matrix transpose module to save chip area, and integrating with other operations to effectively eliminate additional delay and improve computing efficiency.
[0093] In a second aspect, an embodiment of the present application further provides a matrix transpose system.
[0094] In one embodiment, the matrix transpose system includes:
[0095] An SRAM bus matrix module, which is used to cyclically store the matrix data to be transposed into each vertical static random access memory VSRAM based on the individual write mode to implement matrix transpose when the matrix data to be transposed is a non-intermediate operation result; and broadcast and store the matrix data to be transposed into each VSRAM based on the broadcast write mode to implement matrix transpose when the matrix data to be transposed is an intermediate operation result.
[0096] A multiply-accumulate operation array MAC, which is used to perform matrix operations based on the data stored in VSRAM and the horizontal static random access memory HSRAM, and HSRAM is used to store data that does not need to be transposed.
[0097] Furthermore, in one embodiment, the SRAM bus matrix module is specifically used for:
[0098] If the data to be transposed is stored in external memory in row-major order, the data to be transposed is read row by row using direct memory access and based on 2D data transfer function, and the data of the nth row is written into the nth VSRAM in sequence, where n is a non-negative integer;
[0099] When the m-th row of data is read, the data is written back to the 0th VSRAM and the process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs.
[0100] Furthermore, in one embodiment, the SRAM bus matrix module is specifically used for:
[0101] If the data to be transposed is stored in external memory in column-major order, the data to be transposed is read column by column using direct memory access and based on a one-dimensional data transfer mode, and the data of the nth column is written sequentially into the nth VSRAM, where n is a non-negative integer;
[0102] When the data in column m is read, the data is written back to the 0th VSRAM and this process is repeated until the data of the transpose matrix is written. Here, m is an integer multiple of the number of VSRAMs.
[0103] Furthermore, in one embodiment, the SRAM bus matrix module is specifically used for:
[0104] Broadcast the k-th row of the matrix data to be transposed to all VSRAMs, where k is a non-negative integer representing the row index of the matrix;
[0105] By controlling the byte enable signal and data shifting, each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0106] Furthermore, in one embodiment, the SRAM bus matrix module is specifically used for:
[0107] Set the byte enable signal according to the row index k to be written, so that only the k-th byte of data is valid;
[0108] The valid byte index, valid_byte_id, is determined by finding the position of the valid bit in the byte enable signal.
[0109] The valid_byte_id is compared with the VSRAM number vsram_id to determine the data shift direction and the number of bits to shift.
[0110] Based on the data shift direction and the number of data shift bits, the elements of the k-th row of data are shifted so that each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
[0111] Furthermore, in one embodiment, the SRAM bus matrix module is specifically used for:
[0112] When vsram_id is greater than valid_byte_id, the data shift direction is to the right;
[0113] When vsram_id is less than valid_byte_id, the data shift direction is to the left;
[0114] When vsram_id equals valid_byte_id, the data shift direction is no shift.
[0115] The functions of each module in the matrix transpose system correspond to the steps in the matrix transpose method embodiment described above, and their functions and implementation processes will not be described in detail here.
[0116] Thirdly, embodiments of this application provide a matrix transposition device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.
[0117] Reference Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the matrix transposition device involved in the embodiments of this application. In the embodiments of this application, the matrix transposition device may include a processor, a memory, a communication interface, and a communication bus.
[0118] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.
[0119] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the matrix transposition device, as well as interfaces used for interconnecting the matrix transposition device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.
[0120] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0121] The processor can be a general-purpose processor, which can call a matrix transpose program stored in memory and execute the matrix transpose method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the matrix transpose program is called can be referred to in the various embodiments of the matrix transpose method of this application, and will not be repeated here.
[0122] Those skilled in the art will understand that Figure 4 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0123] Fourthly, embodiments of this application also provide a computer-readable storage medium.
[0124] The present application has a matrix transpose program stored on a readable storage medium, wherein when the matrix transpose program is executed by a processor, it implements the steps of the matrix transpose method as described above.
[0125] The method implemented when the matrix transpose procedure is executed can be referred to in various embodiments of the matrix transpose method of this application, and will not be repeated here.
[0126] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0127] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0128] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0129] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0130] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0132] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A matrix transpose method, characterized in that, The matrix transpose method includes: When the matrix data to be transposed is a non-intermediate operation result, the matrix data to be transposed is cyclically stored into each vertical static random access memory (VSRAM) based on the individual write mode to achieve matrix transposition. When the data of the matrix to be transposed is an intermediate operation result, the data of the matrix to be transposed is broadcast and stored into each VSRAM based on the broadcast write mode to achieve matrix transposition; Among them, the multiply-accumulate-add array (MAC) performs matrix operations based on the data stored in VSRAM and horizontal static random access memory HSRAM. HSRAM is used to store data that does not need to be transposed. The method of cyclically storing the transpose matrix data into each vertical static random access memory (VSRAM) based on a separate write mode includes: using direct memory access and based on 2D data transfer function or 1D data transfer mode to read the transpose matrix data row by row or column by column, and sequentially writing the nth row or nth column data into the nth VSRAM, where n is a non-negative integer; when the mth row or mth column data is read, the data is returned to the 0th VSRAM to continue writing, and this process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs; The method of broadcasting the data of the matrix to be transposed into each VSRAM based on the broadcast write mode includes: broadcasting the k-th row of the data of the matrix to be transposed into all VSRAMs, where k is a non-negative integer representing the row index of the matrix; and using the control byte enable signal and data shifting to store each element of the k-th row of the data in the k-th byte position of each VSRAM.
2. The matrix transpose method as described in claim 1, characterized in that, The step of cyclically storing the transpose matrix data into each VSRAM based on a separate write mode includes: If the data to be transposed is stored in external memory in row-major order, the data to be transposed is read row by row using direct memory access and based on 2D data transfer function, and the data of the nth row is written into the nth VSRAM in sequence, where n is a non-negative integer; When the m-th row of data is read, the data is written back to the 0th VSRAM and the process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs.
3. The matrix transpose method as described in claim 1, characterized in that, The step of cyclically storing the transpose matrix data into each VSRAM based on a separate write mode includes: If the data to be transposed is stored in external memory in column-major order, the data to be transposed is read column by column using direct memory access and based on a one-dimensional data transfer mode, and the data of the nth column is written sequentially into the nth VSRAM, where n is a non-negative integer; When the data in column m is read, the data is written back to the 0th VSRAM and this process is repeated until the data of the transpose matrix is written. Here, m is an integer multiple of the number of VSRAMs.
4. The matrix transpose method as described in claim 1, characterized in that, The method of controlling the byte enable signal and data shifting to ensure that each element of the k-th row of data is stored in the k-th byte position of each VSRAM includes: Set the byte enable signal according to the row index k to be written, so that only the k-th byte of data is valid; The valid byte index, valid_byte_id, is determined by finding the position of the valid bit in the byte enable signal. The valid_byte_id is compared with the VSRAM number vsram_id to determine the data shift direction and the number of bits to shift. Based on the data shift direction and the number of data shift bits, the elements of the k-th row of data are shifted so that each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
5. The matrix transpose method as described in claim 4, characterized in that, The step of comparing valid_byte_id with the VSRAM number vsram_id to determine the data shift direction includes: When vsram_id is greater than valid_byte_id, the data shift direction is to the right; When vsram_id is less than valid_byte_id, the data shift direction is to the left; When vsram_id equals valid_byte_id, the data shift direction is no shift.
6. A matrix transpose system, characterized in that, The matrix transpose system includes: The SRAM bus matrix module is used to cyclically store the matrix data to be transposed into each vertical static random access memory (VSRAM) based on a separate write mode when the matrix data to be transposed is a non-intermediate operation result, in order to achieve matrix transposition; and to broadcast the matrix data to be transposed into each VSRAM based on a broadcast write mode when the matrix data to be transposed is an intermediate operation result, in order to achieve matrix transposition. The multiply-accumulate-add array (MAC) is used to perform matrix operations based on data stored in VSRAM and horizontal static random access memory HSRAM, with HSRAM used to store data that does not need to be transposed. The SRAM bus matrix module is specifically used to: read the data of the matrix to be transposed by row or column using direct memory access and based on 2D data transmission function or 1D data transmission mode, and write the data of the nth row or the nth column into the nth VSRAM in sequence, where n is a non-negative integer; when the data of the mth row or the mth column is read, the data is returned to the 0th VSRAM to continue writing, and so on, until the data of the matrix to be transposed is written, where m is an integer multiple of the number of VSRAMs; The k-th row of the matrix data to be transposed is broadcast and written to all VSRAMs, where k is a non-negative integer representing the row index of the matrix; by controlling the byte enable signal and data shifting, each element of the k-th row of data is stored in the k-th byte position of each VSRAM.
7. The matrix transpose system as described in claim 6, characterized in that, The SRAM bus matrix module is specifically used for: If the data to be transposed is stored in external memory in row-major order, the data to be transposed is read row by row using direct memory access and based on 2D data transfer function, and the data of the nth row is written into the nth VSRAM in sequence, where n is a non-negative integer; When the m-th row of data is read, the data is written back to the 0th VSRAM and the process is repeated until the transpose matrix data is completely written, where m is an integer multiple of the number of VSRAMs.
8. A matrix transpose device, characterized in that, The matrix transpose device includes a processor, a memory, and a matrix transpose program stored in the memory and executable by the processor, wherein when the matrix transpose program is executed by the processor, it implements the steps of the matrix transpose method as described in any one of claims 1 to 5.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a matrix transpose program, wherein when the matrix transpose program is executed by a processor, it implements the steps of the matrix transpose method as described in any one of claims 1 to 5.