Data read-write method, product, data accelerator and medium

By setting the mapping relationship between the pre-designed counter and the target shard position and row and column dimensions in the data accelerator, the problem of inappropriate data storage order and non-matrix multiplication unit writing order is solved, and efficient data processing is achieved. Reduced delay in speech recognition time.

CN119045776AActive Publication Date: 2024-11-29LANGCHAO ELECTRONIC INFORMATION IND CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411534508.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-31
Publication Date
2024-11-29
Estimated Expiration
2044-10-31

AI Technical Summary

Technical Problem

In natural language processing, the data storage order output by the matrix multiplication unit cannot adapt to the writing order of the non-matrix multiplication unit, resulting in high delays written to the cache unit, affecting the speech recognition efficiency.

Method used

By setting the mapping relationship between the predesigned counter and the target shard position and row-column dimension in the data accelerator, the output address of the matrix multiplying the acceleration unit is converted to adapt to the write operation address of the non-matrix multiplying the acceleration unit, and the matrix is ​​performed in the read operation mode Dimension fills to ensure that data is stored in the cache in the appropriate order.

Benefits of technology

It effectively reduces the delay in speech recognition time, improves data processing efficiency, and avoids high-delay operations of writing the matrix multiplication acceleration unit data back to the off-chip memory before reading it.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045776A_ABST
    Figure CN119045776A_ABST
Patent Text Reader

Abstract

The invention discloses a data read-write method, a product, a data accelerator and a medium, and relates to the technical field of data processing. The method comprises the following steps of: storing data of an incomplete row sequence directly from a matrix multiplication acceleration unit in a cache according to a complete row sequence, and correspondingly performing conversion processing on a result data output sequence of the matrix multiplication acceleration unit through the sizes of fragments and row and column dimension positions so as to adapt to a write-in sequence corresponding to a non-matrix multiplication unit; and the high-delay operation of writing the data of the matrix multiplication acceleration unit back to the off-chip memory and then reading the data is avoided. The data storage sequence output by the non-matrix multiplication unit is filled, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, and the function of generating the data dimension required by the matrix multiplication acceleration unit for the actual calculation result is realized; and the high-delay operation of writing the data of the non-matrix multiplication acceleration unit back to the off-chip memory and then reading the data is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data reading and writing method, a product, a data accelerator and a medium. Background Art

[0002] Natural language processing is an important branch of artificial intelligence, which enables computers to understand and process human language. Neural network models have become the core technology in the field of natural language processing and have been extended to other fields such as computer vision.

[0003] In the process of natural language processing, such as speech recognition, the linear transformation of the neural network model will rely on a large number of matrix multiplication calculations to save data cache space. For the layers or functions of the model involving non-linear calculations, non-matrix multiplication calculations are used, which takes a long time to calculate. At the same time, if there are matrix multiplication units and non-matrix multiplication units adjacent to each other between different layers, the data storage order output by the matrix multiplication unit cannot adapt to the corresponding writing order of the non-matrix multiplication unit, and the data storage order output by the non-matrix multiplication unit cannot adapt to the corresponding writing order of the matrix multiplication unit, which will cause high delays in writing to the corresponding cache unit in each unit, making speech recognition take a long time.

[0004] Therefore, how to avoid the high delay in writing to the memory to improve the speech recognition efficiency is an urgent problem that needs to be solved by those skilled in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a data reading and writing method, product, data accelerator and medium to solve the problem that the data storage order output by the matrix multiplication unit cannot adapt to the writing order corresponding to the non-matrix multiplication unit, and the data storage order output by the non-matrix multiplication unit cannot adapt to the writing order corresponding to the matrix multiplication unit, both of which will cause high delay in writing to the corresponding cache unit in each unit, resulting in a longer speech recognition time.

[0006] In order to solve the above technical problems, the present invention provides a method for reading and writing data, comprising: Acquire a read and write mode of the first data in a non-matrix multiplication acceleration unit in the data accelerator; If the first data is in a write operation mode in the non-matrix multiplication acceleration unit, a sending address of the first data is converted according to a target slice position and a row and column dimension position of a systolic array under an output matrix of the matrix multiplication acceleration unit to determine a write operation address of the converted second data in the non-matrix multiplication acceleration unit, so as to write the converted second data into the non-matrix multiplication acceleration unit; If the first data is in a read operation mode in a non-matrix multiplication acceleration unit, the write matrix dimension of the first data in the matrix multiplication acceleration unit fills the current output matrix of the first data in the non-matrix multiplication acceleration unit, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to be read out to the matrix multiplication acceleration unit.

[0007] On the one hand, the sending address of the first data is converted according to the target slice position and row and column dimension position of the systolic array under the output matrix of the matrix multiplication acceleration unit of the first data to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, including: Pre-set the corresponding mapping relationship between the preset counter and the target shard position and the row and column dimension position; According to the mapping relationship between the target sharding setting and the row and column dimension positions and the preset counter, and the address count of the preset counter, the sending address of the first data is converted to determine the write operation address of the second data in the non-matrix multiplication acceleration unit.

[0008] On the other hand, the preset counter includes a first counter, a second counter, a third counter and a fourth counter; The first counter is used to record the address count of the submatrix in the horizontal direction under each slice of the output matrix; the address count of the first counter is limited by the number of consecutive preset write data in the horizontal direction, and is the column dimension of the submatrix - 1; The second counter is used to record the first reference address count in the horizontal direction of the non-first row of the submatrix under each slice of the output matrix; the first reference address count is limited by the number of consecutive preset write data in the vertical direction and is the row dimension of the submatrix-1; The third counter is used to record the second reference address count in the horizontal direction of the submatrix of the non-first slice of the output matrix based on the row; The fourth counter is used to record the third reference address count in the horizontal direction of the submatrix under each slice of the output matrix based on the non-first row; wherein the value of the third reference address count is the address value of the last data of the last slice of the previous row of the current row of the output matrix.

[0009] On the other hand, the target slice position is obtained by dividing the first data into corresponding positions of all systolic arrays under the output matrix of the matrix multiplication acceleration unit to obtain the position information corresponding to the slice size; wherein the row and column dimensions of the output matrix are integer multiples of the slice size.

[0010] On the other hand, the mapping relationship between the preset counter and the target shard position and the row and column dimension position is pre-set, including: If the target slice position is the first slice of the first row of the output matrix, a first mapping relationship is established between the column dimension positions under the first row of the submatrix where the first data of the first slice is located and the address count of the first counter; wherein the address counts of the remaining counters except the first counter are 0; a second mapping relationship is established between the column dimension positions corresponding to the rows under the non-first row of the submatrix where the first data of the first slice is located, the address count of the first counter, and the address count of the second counter; wherein the address counts of the remaining counters except the first counter and the second counter are 0; If the target slice position is the remaining slices except the first slice under the first row of the output matrix, a third mapping relationship is established between the column dimension positions under the first row of the submatrix where the first data of the remaining slices are located, the address count of the first counter, and the address count of the third counter; wherein the address counts of the remaining counters except the first counter and the third counter are 0; a fourth mapping relationship is established between the column dimension positions corresponding to each row under the non-first row of the submatrix where the first data of the remaining slices are located, and the address counts corresponding to the first counter, the second counter, and the third counter respectively; wherein the address count of the fourth counter is 0; If the target slice position is the first slice in the non-first row of the output matrix, then on the basis of establishing the first mapping relationship and the second mapping relationship, the address count of the fourth counter is added respectively to form the fifth mapping relationship and the sixth mapping relationship respectively; if the target slice position is the remaining slices except the first slice in the non-first row of the output matrix, then on the basis of establishing the third mapping relationship and the fourth mapping relationship, the address count of the fourth counter is added respectively to form the seventh mapping relationship and the eighth mapping relationship respectively.

[0011] On the other hand, if the target slice position is the first slice of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the first mapping relationship between the row and column dimension positions and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: When the sending address of the current first sub-data in the first data is the first address data under the first row of the sub-matrix, the address counts of all preset counters, the number of continuous preset write data in the horizontal direction, and the number of continuous preset write data in the vertical direction are all set to initial values; the column dimension position of the sub-matrix corresponding to the current first sub-data is determined, and the write operation address of the current data is determined according to the column dimension position of the sub-matrix corresponding to the current first sub-data and the initial values ​​of the address counts corresponding to the first counter, the second counter, the third counter, and the fourth counter respectively; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If not, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next first subdata of the current first subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next first subdata, and the write operation address of the next first subdata is determined according to the current address count of the first counter, the sum of the initial values ​​of the address counters corresponding to the second counter, the third counter and the fourth counter; and the next first subdata is used as the new current first subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current first sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0012] On the other hand, if the target slice position is the first slice of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the second mapping relationship between the row and column dimension positions and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1; if the updated number of continuous preset write data in the vertical direction is not equal to the row dimension of the sub-matrix -1, determine that the number of continuous preset write data in the vertical direction is updated by adding 1; Determine whether the address count of the second counter is the row dimension of the submatrix - 1; if the address count of the second counter is not the row dimension of the submatrix - 1, obtain the remaining unwritten column dimensions corresponding to the remaining slices except the first slice in the first row; and use the sum of the number of remaining unwritten data and the row dimension as the address count of the second counter; When the sending address of the current second sub-data in the first data is the first address data under the non-first row of the sub-matrix, the column dimension position of the sub-matrix corresponding to the current second sub-data is determined, and the write operation address of the current second sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current second sub-data and the address count of the second counter, the first counter, the third counter and the fourth counter respectively corresponding to the initial value of the address count; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next second subdata of the current second subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next second subdata, and the write operation address of the next second subdata is determined according to the current address count of the first counter, the address count of the second counter, and the sum of the initial values ​​of the address counts corresponding to the third counter and the fourth counter; and the next second subdata is used as the new current second subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current second sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0013] On the other hand, if the target slice position is the remaining slices of the first row of the output matrix, the sending address of the first data is converted and processed according to the third mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1, set the updated number of continuous preset written data in the vertical direction to 0; Determine whether the address count of the second counter is the row dimension of the sub-matrix - 1; if the address count of the second counter is the row dimension of the sub-matrix - 1, reset the address count of the second counter to an initial value; Determine whether the remaining unwritten column dimension after the update is 0; if the remaining unwritten column dimension after the update is not 0, perform difference processing on the remaining unwritten column dimension after the update and the column dimension of the submatrix to obtain a new remaining unwritten column dimension; Setting the address count of the third counter to a value corresponding to the column dimension of the submatrix; When the sending address of the current third sub-data in the first data is the first address data under the first row of the sub-matrix of the remaining slices, determine the column dimension position of the sub-matrix corresponding to the current third sub-data, and determine the write operation address of the current second sub-data according to the column dimension position of the sub-matrix corresponding to the current third sub-data and the address count of the third counter, and the initial values ​​of the address counts corresponding to the first counter, the second counter, and the fourth counter; and update the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next third subdata of the current third subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next third subdata, and the write operation address of the next third subdata is determined according to the current address count of the first counter, the address count of the third counter, and the sum of the initial values ​​of the address counts corresponding to the second counter and the fourth counter; and the next third subdata is used as the new current third subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current third sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0014] On the other hand, if the target slice position is the remaining slices of the first row of the output matrix, the sending address of the first data is converted and processed according to the fourth mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1; if the updated number of continuous preset write data in the vertical direction is not equal to the row dimension of the sub-matrix -1, determine that the number of continuous preset write data in the vertical direction is updated by adding 1; Determine whether the address count of the second counter is the row dimension of the submatrix - 1; if the address count of the second counter is not the row dimension of the submatrix - 1, obtain the remaining unwritten column dimensions corresponding to the remaining slices except the first slice in the first row; and use the sum of the number of remaining unwritten data and the row dimension as the address count of the second counter; When the sending address of the current fourth sub-data in the first data is the first address data under the non-first row of the sub-matrix, the column dimension position of the sub-matrix corresponding to the current fourth sub-data is determined, and the write operation address of the current fourth sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current fourth sub-data and the address count of the second counter, the address count of the third counter, and the initial values ​​of the address counts corresponding to the first counter and the fourth counter respectively; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next fourth subdata of the current fourth subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next fourth subdata, and the write operation address of the next fourth subdata is determined according to the sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter; and the next second subdata is used as the new current fourth subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current fourth sub-data in the first data reaches the number of continuous preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0015] On the other hand, if the target slice position is the first slice of the non-first row of the output matrix, the sending address of the first data is converted and processed according to the fifth mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1, set the updated number of continuous preset written data in the vertical direction to 0; Determine whether the address count of the second counter is the row dimension of the sub-matrix - 1; if the address count of the second counter is the row dimension of the sub-matrix - 1, reset the address count of the second counter to an initial value; Determine whether the remaining unwritten column dimension after the update is 0; if the remaining unwritten column dimension after the update is 0, set the remaining unwritten column dimension after the update to the remaining unwritten column dimension before the update; Resetting the address count of the third counter to an initial value; The sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter is added by 1 to obtain the current address count of the fourth counter; and the process returns to the step of converting the sending address of the first data according to the first mapping relationship between the target sharding setting and the row and column dimension positions and the preset counter and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit to determine the write operation address of the current fifth sub-data in the first data.

[0016] On the other hand, the write matrix dimension of the first data in the matrix multiplication acceleration unit fills the current output matrix of the first data in the non-matrix multiplication acceleration unit so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, including: Obtaining the first column dimension and the first row dimension of the current output matrix of the non-matrix multiplication acceleration unit of the first data; Obtain a second column dimension and a second row dimension of a write matrix dimension of the first data in the matrix multiplication acceleration unit; Determine the third column dimension and the third row dimension to be filled according to the first column dimension and the second column dimension, the first row dimension and the second row dimension; The current output matrix of the first data in the non-matrix multiplication acceleration unit is filled according to the third column dimension and the third row dimension, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit.

[0017] On the other hand, the current output matrix of the first data in the non-matrix multiplication acceleration unit is filled according to the third column dimension and the third row dimension, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, including: Writing the first data into the cache of the matrix multiplication acceleration unit according to each row of data of the current output matrix; Performing filling processing on the third column dimension in the first column dimension of the last column of each row of data to obtain filled fourth data; When the last row of data is written, a first matrix is ​​generated according to the third row dimension and the second column dimension; and each data of the first matrix is ​​filled to obtain the filled fifth data; The padded fourth data, fifth data and first data are used as padded third data.

[0018] In order to solve the above technical problem, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the data reading and writing method when executed by a processor.

[0019] In order to solve the above technical problems, the present invention provides a data accelerator, comprising: Memory for storing computer programs; A processor is used to implement the steps of the data reading and writing method as described above when executing the computer program.

[0020] In order to solve the above technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data reading and writing method as described above are implemented.

[0021] The present invention provides a data reading and writing method, which obtains the reading and writing mode of the first data in a non-matrix multiplication acceleration unit in a data accelerator; wherein the non-matrix multiplication acceleration unit is a data processing unit for realizing a non-matrix multiplication calculation method; if the first data is in a write operation mode in the non-matrix multiplication acceleration unit, the sending address of the first data is converted and processed according to the target slice position and row and column dimension position of the pulsation array under the output matrix of the matrix multiplication acceleration unit of the first data to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, so as to write it to the non-matrix multiplication acceleration unit; wherein the matrix multiplication acceleration unit is a data processing unit for realizing a matrix multiplication calculation method; if the first data is in a read operation mode in the non-matrix multiplication acceleration unit, the write matrix dimension of the first data in the matrix multiplication acceleration unit is filled with the current output matrix of the first data in the non-matrix multiplication acceleration unit, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to be read out to the matrix multiplication acceleration unit.

[0022] The beneficial effect of the present invention is that when the matrix multiplication unit and the non-matrix multiplication unit are adjacent, when the data storage order output by the matrix multiplication unit is written to the non-matrix unit, the result data output order of the matrix multiplication acceleration unit needs to be converted and processed by the size of the slice and the row and column dimension position correspondingly to adapt to the corresponding writing order of the non-matrix multiplication unit; the data in the incomplete row order directly from the matrix multiplication acceleration unit is saved in the cache in the order of the complete row, avoiding the high-latency operation of writing the data of the matrix multiplication acceleration unit back to the off-chip memory and then reading it. The data storage order output by the non-matrix multiplication unit is padded, so that the matrix dimension of the third data after padded is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to adapt to the corresponding writing order of the matrix multiplication unit, and the read cache controller supporting the automatic padding mode realizes the function of generating the data dimension required by the matrix multiplication acceleration unit for the actual calculation result, avoiding the high-latency operation of writing the data of the non-matrix multiplication acceleration unit back to the off-chip memory and then reading it, and reducing the time of speech recognition.

[0023] Secondly, the function limitation of each counter is aimed at the specific counting rules of different counters, which improves the conversion accuracy and orderliness of the entire output matrix and avoids the conversion error caused by human calculation. The mapping relationship established by different counters realizes the conversion according to the functions of different counters in the subsequent conversion process, and improves the efficiency and accuracy of data conversion through the combination of the counts of different counters. The reading process of the first row of data under the first slice is completed by the counting function of the first counter, which improves the accuracy of the double insurance work of data conversion. The reading process of the non-first row of data under the first slice is completed by the counting function of the first counter and the first reference address counting function of the second counter, which is distinguished from the reading and counting process of the first row of data and the non-first row of data under the slice, and improves the versatility of data conversion. The reading process of the first row of data under the remaining slices is completed by the counting function of the first counter and the second reference address counting function of the third counter, which is distinguished from the reading and counting process under the first slice, and improves the flexibility of data conversion. The reading process of the sub-matrix of the non-first row under the output matrix is ​​completed by adding the address counting of the fourth counter, which is completed with the relationship between the first counter, the second counter and the third counter, and distinguishes the hierarchical definition of the sub-matrices of different rows, and improves the accuracy of data conversion. The automatic filling method in the read operation mode realizes the function of generating data dimensions that meet the requirements of the matrix multiplication acceleration unit for the actual calculation results, avoiding the high-latency operation of writing the data of the non-matrix multiplication acceleration unit back to the off-chip memory and then reading it.

[0024] In addition, the present invention also provides a computer program product, a data accelerator and a medium, which have the same beneficial effects as the above-mentioned data reading and writing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is a schematic diagram of the calculation of a single Block corresponding to the Transformer neural network model structure; Figure 2 This is a schematic diagram of the proportion of non-matrix multiplication operations and the proportion of latency; Figure 3 This is a schematic diagram of the structure of the data accelerator for the Transformer neural network model structure; Figure 4 A schematic diagram of the order of reading and outputting the result data output by the matrix multiplication acceleration unit; Figure 5A schematic diagram of the reading and outputting order of the result data output by another matrix multiplication acceleration unit; Figure 6 A flowchart of a method for reading and writing data provided by an embodiment of the present invention; Figure 7 A schematic diagram of reading and writing of a non-matrix multiplication acceleration unit of a data accelerator provided by an embodiment of the present invention; Figure 8 A schematic diagram of matrix multiplication of a systolic array provided in an embodiment of the present invention; Fig. 9 A schematic diagram of a conversion of a shard write operation mode provided by an embodiment of the present invention; Fig.10 A parameter diagram of a service sharding write mode provided in an embodiment of the present invention; Fig.11 A flowchart of a shard writing mode provided by an embodiment of the present invention; Fig.12 A schematic diagram of parameters of a service filling read mode provided in an embodiment of the present invention; Fig.13 A schematic diagram of data in a read operation mode provided by an embodiment of the present invention; Fig.14 A structural diagram of a data reading and writing device provided by an embodiment of the present invention; Fig.15 A structural diagram of an accelerator provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The core of the present invention is to provide a data reading and writing method, product, data accelerator and medium to solve the problem that the data storage order output by the matrix multiplication unit cannot adapt to the writing order corresponding to the non-matrix multiplication unit, and the data storage order output by the non-matrix multiplication unit cannot adapt to the writing order corresponding to the matrix multiplication unit, both of which will cause high delay in writing to the corresponding cache unit in each unit, resulting in a longer speech recognition time.

[0029] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] Data accelerators are accelerators used to accelerate the training and reasoning processes of learning models, such as the Tensor Processing Unit (TPU) accelerator, which is a dedicated integrated circuit designed for machine learning and artificial intelligence applications. It focuses on optimizing the operations of deep learning frameworks, corresponding to the acceleration of tensor operations, thereby improving the performance of machine learning related tasks.

[0031] The data accelerator includes a non-matrix multiplication acceleration unit and a matrix multiplication acceleration unit. The matrix multiplication acceleration unit contains a systolic array of units responsible for calculating matrix multiplication-related operations, and the non-matrix multiplication acceleration unit is an acceleration unit for non-matrix multiplication operations of units that process special functions. Generally speaking, the matrix multiplication acceleration unit focuses on the acceleration of linear algebra operations, and each data element can be reused by multiple multiplication units, thereby reducing the need for data movement and storage. In a neural network, the output of a layer can be regarded as a linear transformation of the input data, which is mainly achieved through matrix multiplication. Corresponding to non-matrix multiplication operations, it can be achieved by nonlinearly transformed data. As a hardware accelerator, the data accelerator corresponds to the hardware acceleration unit of the matrix multiplication acceleration unit and the non-matrix multiplication acceleration unit to perform matrix operations, non-matrix operations, and parallel computing tasks of the learning model.

[0032] Figure 1 The following is a calculation diagram of a single Block corresponding to the Transformer neural network model structure, as shown in the figure: Figure 1 As shown in the figure, a typical Transformer network model consists of a certain number of blocks, each of which contains a number of calculation steps. The calculation steps contained in each block are exactly the same, except for the first block, where the output of the previous block is the input of the next block. In addition to relying on a large number of traditional matrix multiplication calculations, some complex nonlinear calculations are also referenced, such as Figure 1 Layer normalization, Softmax activation function, etc. Although matrix multiplication accounts for a large proportion from the perspective of the amount of operations per second (OPS), the calculation time (delay time, etc.) of the layer where the non-matrix multiplication operation is located accounts for a relatively high proportion. Figure 2 This is a schematic diagram of the ratio of non-matrix multiplication operations to latency ratio. Figure 2 As shown, the delay of its non-matrix multiplication operation occupies more delay time.

[0033] Data accelerator for Transformer neural network model structure, Figure 3 The schematic diagram of the structure of the data accelerator of the Transformer neural network model is as follows: Figure 3As shown in the figure, the matrix multiplication acceleration unit and the non-matrix multiplication acceleration unit have their own dedicated caches to provide a much higher memory access bandwidth than the off-chip memory. After completing the data calculation in the matrix multiplication acceleration unit, the result data needs to be written back to the off-chip memory, and then additional data movement instructions are used to adjust the storage order of the data in the off-chip memory to the order of rows, so as to write it to the non-matrix multiplication acceleration unit for subsequent non-matrix multiplication operations. The above two calculation operations require that the complete row data be loaded into the cache corresponding to the non-matrix multiplication acceleration unit in order to operate normally. Figure 4 This is a schematic diagram of the order of reading and outputting the result data output by the matrix multiplication acceleration unit, such as Figure 4 As shown, the current large matrix is ​​obtained by performing matrix multiplication calculation on two matrices, and is divided into sub-matrices of different regions. The sub-matrices of different regions are represented by box diagrams of different colors. Each sub-matrix corresponds to the matrix multiplication calculation of each row and column of the two matrices. The output order of each sub-matrix is ​​to output the second row after the output of the first row of the sub-matrix is ​​completed. After the output order of the first sub-matrix is ​​completed, the first row of the second sub-matrix in the same row as the first sub-matrix is ​​read according to the order of the rows, and so on, until the same row as the first sub-matrix is ​​read, and then the sub-matrix in the second row is read.

[0034] In the writing order of the non-matrix multiplication acceleration unit, the sub-matrices are not distinguished, but the first row of the large matrix to which all sub-matrices belong is read, and then the second row is read until all rows of data are read. Figure 5 A schematic diagram of the reading and output order of the result data output by another matrix multiplication acceleration unit is shown in FIG. Figure 5 As shown, it is divided into 9 sub-matrices (A, B, C, D, E, F, G, H, I). Taking sub-matrices A, B, and C as an example, the output order of the matrix multiplication acceleration unit is to first output the data from the first row (a11-a13) to the last row (a31-a33) in sub-matrix A, and then output the data from the first row (b11-b13) to the last row in sub-matrix B. The writing order in the non-matrix acceleration unit is to output all sub-matrices according to each row, and then read the second row (a21-a23, b21-b23, c21-c23) after reading the first row (a11-a13, b11-b13, c11-c13). The order of the output results of all matrix multiplication acceleration units cannot meet the writing requirements of the non-matrix multiplication acceleration unit.

[0035] After the non-matrix multiplication acceleration unit has completed the calculation, since the systolic array in the matrix multiplication acceleration unit requires the input data dimension to be at least an integer multiple of the matrix, it is not necessarily satisfied that the matrix dimension is an integer multiple of the systolic array, resulting in a delay in the writing order of the matrix multiplication acceleration unit. Therefore, the data reading and writing method provided by the present invention can solve the above problem.

[0036] Figure 6 A flowchart of a method for reading and writing data provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the method includes: S11: Acquire a read / write mode of a non-matrix multiplication acceleration unit in a data accelerator for first data; S12: Determine whether the read / write mode of the first data is a write operation mode; if so, proceed to step S13; if not, proceed to step S14; S13: converting the sending address of the first data according to the target slice position and row and column dimension position of the systolic array under the output matrix of the matrix multiplication acceleration unit to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, so as to write the converted second data into the non-matrix multiplication acceleration unit; S14: Fill the current output matrix of the first data in the non-matrix multiplication acceleration unit according to the write matrix dimension of the first data in the matrix multiplication acceleration unit, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to be read out to the matrix multiplication acceleration unit.

[0037] Among them, the non-matrix multiplication acceleration unit is used to implement the data processing unit of the non-matrix multiplication calculation method; the matrix multiplication acceleration unit is used to implement the data processing unit of the matrix multiplication calculation method.

[0038] Figure 7 A read and write schematic diagram of a non-matrix multiplication acceleration unit of a data accelerator provided by an embodiment of the present invention, such as Figure 7 As shown, data from Ethernet or other links are received to configure the status register instructions, and control signals are provided for the write operation mode and the read operation mode. Based on the data accelerator, the read and write mode corresponding to the first data in the non-matrix multiplication acceleration unit is obtained. It should be noted that this embodiment is only aimed at the subsequent data address conversion problem under the current need for effective read and write modes, that is, the address conversion in this embodiment is performed corresponding to the non-matrix multiplication operation and matrix multiplication operation corresponding to the non-matrix multiplication acceleration unit and the matrix multiplication acceleration unit respectively. Combined with Figure 1 From the above, we can see that there is an activation function (belonging to the non-matrix multiplication acceleration unit) between the two matrix multiplication acceleration units in the fully connected layer. Layer normalization involves many complex nonlinear calculations, so it also belongs to the non-matrix multiplication acceleration unit. When the non-matrix multiplication acceleration unit and the matrix multiplication acceleration unit are adjacent and there are many nonlinear calculations, the corresponding address conversion is performed in the data reading and writing mode.

[0039] The determination of the read-write mode can be determined by the flag bit of the status register, or by the data flow, system call or other methods. When the first data is a write operation in the non-matrix multiplication acceleration unit, that is, the output result data of the matrix multiplication acceleration unit is stored in the cache area of ​​the non-matrix multiplication acceleration unit to perform a write operation in the non-matrix multiplication acceleration unit. The corresponding address conversion is intended to adapt to the write order of the non-matrix multiplication acceleration unit to reduce the delay operation time. The specific address conversion process can be based on the sending address of each data in the matrix multiplication acceleration unit. The gradual transfer can be achieved if the matrix dimension is small. If the matrix dimension is large, the gradual transfer will increase the additional transfer time, which will form another dimension of delay operation. Here, the address conversion can be based on a certain calculation method, or it can be realized by multiplying the sending address of the current first data with a certain weight parameter. It can also be combined with the target slice position of each systolic array corresponding to the output matrix of the matrix multiplication acceleration unit and the row and column dimension position of the matrix to convert to the write operation address of the non-matrix multiplication acceleration unit. The target slice position is based on the slice processing of the sub-matrix composed of the systolic array under the output matrix to obtain the sub-matrix under each slice, and one slice corresponds to one sub-matrix.

[0040] Figure 4 For example, each systolic array forms a 2*3 sub-matrix, that is, a systolic array corresponds to a box in the sub-matrix, and the target slice position corresponding to the systolic array corresponds to the slice position of the output matrix. The row and column dimension position corresponds to the row and column of the box where the systolic array is located under the sub-matrix. For example, according to the writing order of the non-matrix multiplication acceleration unit, Figure 4 The first entire row of the output matrix is ​​written first, and then the second row of data of the output matrix is ​​written. Therefore, it is necessary to store them in this order in the cache of the non-matrix multiplication acceleration unit to facilitate subsequent writing of the non-matrix multiplication acceleration unit.

[0041] The storage process is to first slice the output matrix so that each slice corresponds to a sub-matrix. The slice size is determined by combining the rows and columns of the two input matrices before the matrix multiplication calculation of the output matrix.

[0042] For the first row of the first submatrix (first slice) in the submatrix of the first row of the output matrix, all are written in the order of row writing, so there will be no address change. According to the matrix multiplication acceleration unit, after the first row of the first submatrix is ​​read, the second row of the first submatrix will be read, but in the non-matrix multiplication acceleration unit, the first row of the second submatrix of the first row will be read. Therefore, here it is necessary to add the column dimensions of the remaining submatrices in the first row except the first submatrix as the interval address based on the sending address of the data of the last column dimension of the first row of the first submatrix, so as to set it as the write address of the data of the first column dimension of the second row of the first submatrix. The remaining column dimensions corresponding to the second row of the first submatrix are processed by sequentially adding 1 to the write address of the data of the first column dimension of the second row of the first submatrix.

[0043] exist Figure 4 In the example, if the data of the last column dimension of the second row of the first submatrix is ​​read, the matrix multiplication acceleration unit will continue to read the data of the first column dimension of the first row of the second submatrix. At this time, it is necessary to determine it within the original interval address and deduce it one by one. Since it involves a process of gradually adding 1 in sequence, the mark conversion can be performed by counting with a counter. The number of corresponding counters is not limited and can be set according to actual conditions. It should be noted that multiple counters can be set with different functions. For example, a counter is only for the counting function of each row of each submatrix, a counter is for the address mark from the last column dimension of a row of each submatrix to the first column dimension of the next row, a counter is for the address mark conversion between each submatrix and the submatrix, and a counter is for the address mark of the last submatrix of a row of the output matrix and the first submatrix of the next row, etc., which are not limited here.

[0044] Step S13 converts the sending address of the first data corresponding to the output matrix of the matrix multiplication acceleration unit to the write operation address of the non-matrix multiplication acceleration unit to reduce delayed operations caused by address mismatch, or even the inability to read and write normally.

[0045] In step S14, the first data of the output matrix corresponding to the non-matrix multiplication acceleration unit, the output matrix where the first data is located is not constrained by the dimension of the pulsating array, resulting in the output matrix where the first data is located having a matrix dimension different from the matrix dimension of the matrix multiplication acceleration unit. The rows and columns of the matrix dimension of the overall output matrix of the non-matrix multiplication acceleration unit are smaller than the rows and columns of the matrix dimension of the matrix multiplication acceleration unit. Therefore, in order to maintain the consistency of the matrix dimension, it is automatically filled and processed, and combined with an additional control register circuit, the output of the non-matrix multiplication acceleration unit and the input of the matrix multiplication acceleration unit are adapted to each other. In the filling process here, in order not to affect the change of the first data, the corresponding filling is filled with 0 at this time. It should be noted that the first data as the entire data packet corresponds to an output matrix, such as Figure 4 A large matrix.

[0046] An embodiment of the present invention provides a data reading and writing method, which obtains the reading and writing mode of the first data in a non-matrix multiplication acceleration unit in a data accelerator; wherein the non-matrix multiplication acceleration unit is a data processing unit used to implement a non-matrix multiplication calculation method; if the first data is in a write operation mode in the non-matrix multiplication acceleration unit, the sending address of the first data is converted and processed according to the target slice position and row and column dimension position of the pulsation array under the output matrix of the matrix multiplication acceleration unit of the first data to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, so as to write it to the non-matrix multiplication acceleration unit; wherein the matrix multiplication acceleration unit is a data processing unit used to implement a matrix multiplication calculation method; if the first data is in a read operation mode in the non-matrix multiplication acceleration unit, the write matrix dimension of the first data in the matrix multiplication acceleration unit is filled with the current output matrix of the first data in the non-matrix multiplication acceleration unit, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to be read out to the matrix multiplication acceleration unit. When the matrix multiplication unit and the non-matrix multiplication unit are adjacent, when the data storage order output by the matrix multiplication unit is written to the non-matrix unit, the result data output order of the matrix multiplication acceleration unit needs to be converted and processed by the size of the slice and the row and column dimension position to adapt to the corresponding writing order of the non-matrix multiplication unit; the data in the incomplete row order directly from the matrix multiplication acceleration unit is saved in the cache in the order of the complete row, avoiding the high-latency operation of writing the data of the matrix multiplication acceleration unit back to the off-chip memory and then reading it. The data storage order output by the non-matrix multiplication unit is padded so that the matrix dimension of the third data after padding is the same as the matrix dimension of the matrix multiplication acceleration unit to adapt to the corresponding writing order of the matrix multiplication unit. The read cache controller supports the automatic padding mode, and realizes the function of generating the data dimension required by the matrix multiplication acceleration unit for the actual calculation result, avoiding the high-latency operation of writing the data of the non-matrix multiplication acceleration unit back to the off-chip memory and then reading it, and reducing the time of speech recognition.

[0047] In some embodiments, the sending address of the first data is converted according to the target slice position and row and column dimension position of the systolic array under the output matrix of the matrix multiplication acceleration unit to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, including: Pre-set the corresponding mapping relationship between the preset counter and the target shard position and the row and column dimension position; According to the mapping relationship between the target sharding setting and the row and column dimension positions and the preset counter, and the address count of the preset counter, the sending address of the first data is converted to determine the write operation address of the second data in the non-matrix multiplication acceleration unit.

[0048] Specifically, the preset counters, the corresponding preset number here can be involved according to the actual situation. No matter how many counters there are, it is necessary to pre-set the mapping relationship between the counters and the target slice positions and the row and column dimensional positions. That is to say, for the arrangement positions of multiple slices under an output matrix and the corresponding row and column positions, it is necessary to consider the count value of the function of the counter, when it changes, and when it is set to 0. It is worth noting that no matter how many counters are set, the corresponding write operation address is the sum of the current specific count values ​​of all counters.

[0049] In combination with the functions of different counters mentioned in the above embodiments, a mapping is performed between the target slicing positions and the row and column dimension positions to determine the specific values ​​of the counters at different target slicing positions and row and column dimension positions. One counter is only for the counting function of each row of each sub-matrix, one counter is for the address mark from the last column dimension of a row of each sub-matrix to the first column dimension of the next row, one counter is for the address mark conversion between each sub-matrix and sub-matrix, one counter is for the address mark of the last sub-matrix of a row of the output matrix and the first sub-matrix of the next row, etc., so that the write operation address of each sub-data in the first data is known.

[0050] The present embodiment provides a method for converting the subsequent first data sending address to obtain the converted second data write operation address by setting a mapping relationship between a preset counter and a target slice position and a row and column dimension position of an output matrix. The corresponding counting function is realized at specific positions of different row and column dimensions by preset counters, thereby improving the accuracy of data conversion.

[0051] In some embodiments, the preset counter includes a first counter, a second counter, a third counter, and a fourth counter; The first counter is used to record the address count of the submatrix in the horizontal direction under each slice of the output matrix; the address count of the first counter is limited by the number of consecutive preset write data in the horizontal direction, and is the column dimension of the submatrix - 1; The second counter is used to record the first reference address count in the horizontal direction of the non-first row of the submatrix under each slice of the output matrix; the first reference address count is limited by the number of consecutive preset write data in the vertical direction and is the row dimension of the submatrix-1; The third counter is used to record the second reference address count in the horizontal direction of the submatrix under the non-first slice of the output matrix based on the row; The fourth counter is used to record the third reference address count in the horizontal direction of the submatrix under each slice of the output matrix based on the non-first row; wherein the value of the third reference address count is the address value of the last data of the last slice of the previous row of the current row of the output matrix.

[0052] It should be noted that the horizontal and vertical directions are based on the column and row dimensions of the matrix, the horizontal order is based on the column of the output matrix, and the vertical order is based on the row of the output matrix. The number of counters provided in this embodiment is four, namely the first counter, the second counter, the third counter and the fourth counter. The first counter is based on the gradually increasing address count of each sub-matrix based on the line standard, and the address count starts from 0. For example, the slice size (sub-matrix) is 2*3, and there are three data in the horizontal direction, corresponding to the column dimension of 3, and its address count is limited to the number of consecutively written data -1, that is, the column dimension of the sub-matrix -1=2, that is, the sending address is 0-2, and the corresponding write operation address is 0-2. In other words, if the normal gradual self-increment conversion is performed, but when the corresponding column dimension is equal to the sub-matrix -1, it means that the data of the current row has been converted, and it is necessary to convert to the data of the next row.

[0053] The second counter is based on the first reference address count in the horizontal direction of the non-first row of each sub-matrix, that is, as long as it involves a non-first row, the second counter needs to be set, and the first reference address count of the counter is limited to the row dimension of the sub-matrix -1, that is, the number of continuous preset write data in the vertical direction -1. When the first reference address count of the second counter is not equal to the row dimension of the sub-matrix -1, it means that the conversion has not been completed under the sub-matrix. If it is equal, it means that the data conversion of the next sub-matrix can be carried out.

[0054] The third counter is for the arrangement of multiple sub-matrices under the output matrix, and the second reference address count in the horizontal direction of the sub-matrix under the non-first slice based on the row. The fourth counter is used to record the arrangement of multiple sub-matrices under the output matrix, and the third reference address count of the corresponding sub-matrix under each slice based on the non-first row. Figure 4From the perspective of the second row, it is the reference address data corresponding to the second row of the color matrix. The third reference address count is the address value of the last data of the last slice of the previous row of the current row, which remains unchanged during the data conversion process of all sub-matrices of the current row. On this basis, the first counter, the second counter and the third counter work together to obtain the write operation address of the subsequent data.

[0055] The functional limitation of each counter provided in this embodiment is aimed at the specific counting rules of different counters, so as to improve the conversion accuracy and orderliness of the entire output matrix and avoid the conversion errors caused by human calculation.

[0056] In some embodiments, the target slice position is obtained by dividing the first data corresponding to all systolic arrays under the output matrix of the matrix multiplication acceleration unit to obtain position information corresponding to the slice size; wherein the row and column dimensions of the output matrix are integer multiples of the slice size.

[0057] Specifically, the target slice position is to segment all systolic arrays under the output matrix to obtain the position information corresponding to the slice size. Figure 8 A schematic diagram of matrix multiplication of a systolic array provided in an embodiment of the present invention is shown in FIG. Figure 8 As shown in the figure, a 4*4 systolic array can calculate the multiplication of two 4*4 matrices. When the workload is a computationally intensive task such as matrix multiplication, it can provide a powerful computing acceleration effect. The disadvantage is that it has certain requirements on the shape of the input data. The dimension of the input data must be an integer multiple of the size of the systolic array. When calculating a matrix multiplication that exceeds the size of the systolic array, the final result matrix of the accumulated output is not output completely in the row direction, but in the order of Figure 4 Output in the direction of the arrow.

[0058] by Figure 4 For example, the slice size is 2*3 sub-matrices distinguished by different colors, and the output matrix is ​​4*9.

[0059] The relationship between the slice size and the output matrix, as well as the process of determining the target slice position, provided in this embodiment make the conversion process of the sub-matrices corresponding to each slice orderly and strictly perform the conversion according to the output order of the matrix multiplication acceleration unit, thereby improving the accuracy and reliability of address conversion.

[0060] In some embodiments, the mapping relationship between the preset counter and the target shard position and the row and column dimension position is pre-set, including: If the target slice position is the first slice of the first row of the output matrix, a first mapping relationship is established between the column dimension positions under the first row of the submatrix where the first data of the first slice is located and the address count of the first counter; wherein the address counts of the remaining counters except the first counter are 0; a second mapping relationship is established between the column dimension positions corresponding to the rows under the non-first row of the submatrix where the first data of the first slice is located, the address count of the first counter, and the address count of the second counter; wherein the address counts of the remaining counters except the first counter and the second counter are 0; If the target slice position is the remaining slices except the first slice under the first row of the output matrix, a third mapping relationship is established between the column dimension positions under the first row of the submatrix where the first data of the remaining slices are located, the address count of the first counter, and the address count of the third counter; wherein the address counts of the remaining counters except the first counter and the third counter are 0; a fourth mapping relationship is established between the column dimension positions corresponding to each row under the non-first row of the submatrix where the first data of the remaining slices are located, and the address counts corresponding to the first counter, the second counter, and the third counter respectively; wherein the address count of the fourth counter is 0; If the target slice position is the first slice in the non-first row of the output matrix, then on the basis of establishing the first mapping relationship and the second mapping relationship, the address count of the fourth counter is added respectively to form the fifth mapping relationship and the sixth mapping relationship respectively; if the target slice position is the remaining slices except the first slice in the non-first row of the output matrix, then on the basis of establishing the third mapping relationship and the fourth mapping relationship, the address count of the fourth counter is added respectively to form the seventh mapping relationship and the eighth mapping relationship respectively.

[0061] Specifically, Fig. 9 A schematic diagram of a conversion of a shard write operation mode provided by an embodiment of the present invention, such as Fig. 9 As shown, if the target slice position is the first slice of the first row of the output matrix, the corresponding first counter and second counter are counted at this time, and the other counters are 0, but there is a distinction between the first row and the non-first row of the first slice. When it is the first row of the first slice, it is necessary to establish a first mapping relationship between the position of each column dimension and the address count of the first counter, and the address count of the second counter is also 0. When it is not the first row of the first slice, the value of the second counter changes, and a second mapping relationship is established between the first reference address count and the address count of the first counter and the position of each column dimension.

[0062] If the target slice position is the remaining slices except the first slice in the first row of the output matrix, a third mapping relationship is established between the corresponding column dimension position and the address counts of the first and third counters in the first row of the remaining slices. In the non-first row, a fourth mapping relationship is established between the corresponding column dimension position and the address counts of the first, second, and third counters.

[0063] If the target slice position is in a slice not in the first row of the output matrix, an additional address count of a fourth counter needs to be added, which is added based on the first mapping relationship to the fourth mapping relationship to convert and obtain the write operation address of the second data.

[0064] The mapping relationship established by different counters provided in this embodiment realizes conversion according to the functions of different counters in the subsequent conversion process, and improves the efficiency and accuracy of data conversion by combining the counts of different counters.

[0065] In some embodiments, if the target slice position is the first slice of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the first mapping relationship between the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: When the sending address of the current first sub-data in the first data is the first address data under the first row of the sub-matrix, the address counts of all preset counters, the number of continuous preset write data in the horizontal direction, and the number of continuous preset write data in the vertical direction are all set to initial values; the column dimension position of the sub-matrix corresponding to the current first sub-data is determined, and the write operation address of the current data is determined according to the column dimension position of the sub-matrix corresponding to the current first sub-data and the initial values ​​of the address counts corresponding to the first counter, the second counter, the third counter, and the fourth counter respectively; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If not, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next first subdata of the current first subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next first subdata, and the write operation address of the next first subdata is determined according to the current address count of the first counter, the sum of the initial values ​​of the address counters corresponding to the second counter, the third counter and the fourth counter; and the next first subdata is used as the new current first subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current first sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0066] Fig.10 A parameter diagram of the service slicing write mode provided in an embodiment of the present invention, such as Fig.10 As shown, there is a 2*3 matrix. When writing the sub-data of the first box, the corresponding remaining unwritten column dimension (horizontal_tile_num) is 2, and the number of consecutive preset written data in the horizontal direction (horizontal_num) is also called the written column dimension; the number of consecutive preset written data in the vertical direction (vertical_num) is also called the written row dimension.

[0067] Specifically, Fig. 9 As shown, for the first slice, that is, the data 1-3 of the first sub-matrix of the output matrix (the sub-matrix composed of data 1-6), the conversion is performed through the first mapping relationship and the address count of the counter. When the first sub-data arrives, sfu_waddr_cnt0, sfu_waddr_cnt1, sfu_waddr_cnt2, and sfu_waddr_block_base represent the first counter, the second counter, the third counter, and the fourth counter respectively, and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are represented by horizontal_tile_num_left and vertical_num_cnt respectively (because the current process is the continuous writing of data, this is the distinction under the continuous process), and the initial values ​​of the above 6 types of data are 0, so the cache address of the first sub-data written into the non-matrix multiplication acceleration unit is 0. It is necessary to update the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction.

[0068] Then determine whether the address count of the first counter is equal to the column dimension of the sub-matrix - 1. If not, the initial value of the address count of the first counter needs to be increased by 1. If it is equal to 1, it means that the data of the current first row has been read and it is necessary to switch to the next row of the sub-matrix for reading and processing.

[0069] Combination Fig. 9, when the second sub-data arrives, the first counter increments by 1, and the data of the other three counters are 0, so the write address of the second sub-data is 1, and the first counter continues to increment by 1. When the third sub-data arrives, the address count of the first counter is 2, and the counts of the other three counters are 0, and the write address of the third sub-data is 2. At this time, the data of the first row under the first slice has been read, and the data of the second row under the first slice needs to be transferred to read. At this time, the task of the first counter in the first row is completed, and it needs to be reset to the initial value to facilitate the reading and counting of the data of the next row.

[0070] The reading process of the first row of data under the first slice provided by this embodiment is completed by the counting function of the first counter, thereby improving the accuracy of the double insurance work of data conversion.

[0071] In some embodiments, if the target slice position is the first slice of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the second mapping relationship between the row and column dimension positions and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1; if the updated number of continuous preset write data in the vertical direction is not equal to the row dimension of the sub-matrix -1, determine that the number of continuous preset write data in the vertical direction is updated by adding 1; Determine whether the address count of the second counter is the row dimension of the submatrix - 1; if the address count of the second counter is not the row dimension of the submatrix - 1, obtain the remaining unwritten column dimensions corresponding to the remaining slices except the first slice in the first row; and use the sum of the number of remaining unwritten data and the row dimension as the address count of the second counter; When the sending address of the current second sub-data in the first data is the first address data under the non-first row of the sub-matrix, the column dimension position of the sub-matrix corresponding to the current second sub-data is determined, and the write operation address of the current second sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current second sub-data and the address count of the second counter, the first counter, the third counter and the fourth counter respectively corresponding to the initial value of the address count; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix - 1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next second subdata of the current second subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next second subdata, and the write operation address of the next second subdata is determined according to the current address count of the first counter, the address count of the second counter, and the sum of the initial values ​​of the address counts corresponding to the third counter and the fourth counter; and the next second subdata is used as the new current second subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current second sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0072] Specifically, when performing the first sub-data conversion of the non-first row of the first slice, it is necessary to pre-determine whether the number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1, that is, to determine whether the current row is the last row. If not, the corresponding number of continuous preset write data in the vertical direction is updated.

[0073] Next, determine whether the address count of the second counter is the row dimension of the submatrix - 1. If not, you need to obtain the remaining unwritten column dimensions (horizontal_tile_num) corresponding to the remaining slices. Here, you need to combine the remaining unwritten column dimensions (horizontal_tile_num) with the row dimensions (combined Fig. 9 From the above, the address count of the second counter is obtained by the addition process in 3).

[0074] Combination Fig. 9For the first slice, take the data 4-6 of the first sub-matrix of the output matrix as an example, and convert it through the second mapping relationship and the address count of the counter. The column dimension position of the sub-matrix corresponding to the second sub-data and the address count of the second counter currently obtained, the sum of the initial values ​​of the address counts corresponding to the first counter, the third counter and the fourth counter determine the write operation address of the current second sub-data. When the fourth sub-data arrives, the address count of the second counter is 0, and the other three counters are 0, so the cache address written to the fourth sub-data is 9. Continue to read the remaining sub-data of the current row. For each increment of one data, the address count of the corresponding first counter is increased by 1 until the address count of the first counter is equal to the column dimension of the sub-matrix -1, then the sixth sub-data has been read, so it is necessary to read the data of the next sub-matrix, and the address count of the first counter is reset to the initial value. If not equal, continue to add 1 to the count of the first counter, and complete the reading of the non-first row of data under the sub-matrix.

[0075] The reading process of non-first row data under the first segment provided in this embodiment is completed through the counting function of the first counter and the first reference address counting function of the second counter, which is distinguished from the reading counting process of the first row data and the non-first row data under the segment, thereby improving the versatility of data conversion.

[0076] In some other embodiments, if the target slice position is the remaining slices of the first row of the output matrix, the sending address of the first data is converted and processed according to the third mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1, set the updated number of continuous preset written data in the vertical direction to 0; Determine whether the address count of the second counter is the row dimension of the sub-matrix - 1; if the address count of the second counter is the row dimension of the sub-matrix - 1, reset the address count of the second counter to an initial value; Determine whether the remaining unwritten column dimension after the update is 0; if the remaining unwritten column dimension after the update is not 0, perform difference processing on the remaining unwritten column dimension after the update and the column dimension of the submatrix to obtain a new remaining unwritten column dimension; Setting the address count of the third counter to a value corresponding to the column dimension of the submatrix; When the sending address of the current third sub-data in the first data is the first address data under the first row of the sub-matrix of the remaining slices, the column dimension position of the sub-matrix corresponding to the current third sub-data is determined, and the write operation address of the current third sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current third sub-data and the address count of the third counter, and the initial values ​​of the address counts corresponding to the first counter, the second counter, and the fourth counter; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next third subdata of the current third subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next third subdata, and the write operation address of the next third subdata is determined according to the current address count of the first counter, the address count of the third counter, and the sum of the initial values ​​of the address counts corresponding to the second counter and the fourth counter; and the next third subdata is used as the new current third subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current third sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0077] Specifically, the update of the number of consecutive preset write data in the vertical direction mentioned in the above embodiment will not be described in detail, and is the same as in the above embodiment. Fig. 9 For the remaining sub-matrices under the first row of each sub-matrix of the output matrix, take the 7-9 sub-data of the 7-12 sub-data as an example, determine whether the address count of the second counter is the row dimension of the sub-matrix -1. At this time, since the address count of the second counter is set to the initial value in the first row of the remaining slices, continue to determine whether the remaining unwritten column dimension after the update is 0. If it is not 0, it is updated to the new remaining unwritten column dimension, that is, the column dimension of the remaining unwritten slices under the current remaining slices ( Fig. 9 The corresponding dimension in is 3). It should be noted that the determination of the address count of the second counter and the determination of the remaining unwritten column dimensions are not in any particular order, and either determination can be performed first.

[0078] Set the address count of the third counter to the value corresponding to the column dimension of the submatrix, combined with Fig. 9It is 3. At this time, the corresponding write operation address of the third sub-data is the sum of the address counts of the four counters. When the 7th sub-data arrives, the address count of the third counter is 3, and the other 3 address counters are all 0. At this time, the write cache address is 3, and the reading of the remaining sub-data of the current row continues. Each time a data is incremented, the address count of the corresponding first counter is increased by 1, until the address count of the first counter is equal to the column dimension of the sub-matrix -1, then the corresponding 9th sub-data has been read, so it is necessary to read the data of the next sub-matrix, and the address count of the first counter is reset to the initial value. If it is not equal, the count of the first counter continues to be increased by 1, and the data of the first row of the remaining slices under the sub-matrix is ​​read.

[0079] At this time, in the sum of the corresponding counters, the current addresses of the second counter and the fourth counter are initial values, and the current address count of the first counter and the address count (second reference address count) of the third counter determine the write operation address.

[0080] The reading process of the first row of data under the remaining slices provided in this embodiment completes the conversion through the counting function of the first counter and the second reference address counting function of the third counter, which is different from the reading counting process under the first slice, thereby improving the flexibility of data conversion.

[0081] In some embodiments, if the target slice position is the remaining slices of the first row of the output matrix, the sending address of the first data is converted and processed according to the fourth mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1; if the updated number of continuous preset write data in the vertical direction is not equal to the row dimension of the sub-matrix -1, determine that the number of continuous preset write data in the vertical direction is updated by adding 1; Determine whether the address count of the second counter is the row dimension of the submatrix - 1; if the address count of the second counter is not the row dimension of the submatrix - 1, obtain the remaining unwritten column dimensions corresponding to the remaining slices except the first slice in the first row; and use the sum of the number of remaining unwritten data and the row dimension as the address count of the second counter; When the sending address of the current fourth sub-data in the first data is the first address data under the non-first row of the sub-matrix, the column dimension position of the sub-matrix corresponding to the current fourth sub-data is determined, and the write operation address of the current fourth sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current fourth sub-data and the address count of the second counter, the address count of the third counter, and the initial values ​​of the address counts corresponding to the first counter and the fourth counter respectively; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next fourth subdata of the current fourth subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next fourth subdata, and the write operation address of the next fourth subdata is determined according to the sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter; and the next second subdata is used as the new current fourth subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current fourth sub-data in the first data reaches the number of continuous preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

[0082] Specifically, the update of the number of consecutive preset write data in the vertical direction mentioned in the above embodiment will not be described in detail, and is the same as in the above embodiment. Fig. 9 For the remaining sub-matrices under the first row of each sub-matrix of the output matrix, take the 10th to 12th sub-data of the 7-12 sub-data as an example, determine whether the address count of the second counter is the row dimension of the sub-matrix -1. At this time, since the address count of the second counter is set to the initial value in the first row of the remaining slices, continue to determine whether the remaining unwritten column dimension after the update is 0. If it is not 0, it is updated to the new remaining unwritten column dimension, that is, the column dimension of the remaining unwritten slices under the current remaining slices ( Fig. 9 The corresponding dimension is the number of remaining unwritten data 6 + row dimension 3 = 9), and the address count corresponding to the second counter is 9.

[0083] At this time, the write operation address of the corresponding fourth sub-data is the sum of the address counts of the four counters. When the 10th sub-data arrives, the address count of the second counter is 9, the address count of the third counter is 3, and the remaining two address counters are 0. At this time, the write cache address is 12, and the reading of the remaining sub-data of the current row continues. Each time a data is incremented, the address count of the corresponding first counter is increased by 1, until the address count of the first counter is equal to the column dimension of the sub-matrix -1, then the 12th sub-data has been read, so it is necessary to read the data of the next sub-matrix, and the address count of the first counter is reset to the initial value. If it is not equal, the count of the first counter continues to be increased by 1, and the data of the first row of the remaining slices under the sub-matrix is ​​read.

[0084] At this time, in the sum of the corresponding counters, the current address of the fourth counter is the initial value, and the current address count of the first counter, the address count of the second counter (first reference address count) and the address count of the third counter (second reference address count) determine the write operation address.

[0085] Similarly, corresponding to Fig. 9 The reading principle for the 13th to 18th sub-data is the same as that for the other shards in the above embodiment and the present embodiment, and will not be elaborated here.

[0086] The reading process of non-first row data under the remaining slices provided in this embodiment completes the conversion through the counting function of the first counter, the first reference address counting function of the second counter and the second reference address counting function of the third counter, which is different from the reading counting process under the first slice and improves the flexibility of data conversion.

[0087] In some other embodiments, if the target slice position is the first slice of the non-first row of the output matrix, the sending address of the first data is converted according to the fifth mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1, set the updated number of continuous preset written data in the vertical direction to 0; Determine whether the address count of the second counter is the row dimension of the sub-matrix - 1; if the address count of the second counter is the row dimension of the sub-matrix - 1, reset the address count of the second counter to an initial value; Determine whether the remaining unwritten column dimension after the update is 0; if the remaining unwritten column dimension after the update is 0, set the remaining unwritten column dimension after the update to the remaining unwritten column dimension before the update; Resetting the address count of the third counter to an initial value; The sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter is added by 1 to obtain the current address count of the fourth counter; and the process returns to the step of converting the sending address of the first data according to the first mapping relationship between the target sharding setting and the row and column dimension positions and the preset counter and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, and determines the write operation address of the current fifth sub-data in the first data.

[0088] Specifically, the update of the number of consecutive preset write data in the vertical direction mentioned in the above embodiment will not be described in detail, and is the same as in the above embodiment. Fig. 9 For the sub-matrices in the non-first row of each sub-matrix of the output matrix, take the 19-21 sub-data of the 19-24 sub-data as an example, determine whether the address count of the second counter is the row dimension of the sub-matrix -1. At this time, since it is in the first row of the first slice of the non-first row, the address count of the second counter is set to the initial value, and continue to determine whether the remaining unwritten column dimension after the update is 0. If it is 0, the remaining unwritten column dimension after the update is set to the remaining unwritten column dimension before the update. At this time, the address count of the third counter is reset to the initial value. The current count of the fourth counter is the sum of the current address counts of all counters, that is, the write operation address of the last data in the remaining slices of the first row + 1.

[0089] It should be noted that the difference between the first slice of the current row and the first slice of the first row lies in the address count of the fourth counter, but the subsequent reading process of the first slice of the current row is the same as the data reading process of the first slice of the first row. Similarly, the reading process of the remaining slices of the current row is the same as the data reading process of the remaining slices of the first row, except that the address count of the fourth counter is added to the original address, which will not be elaborated here.

[0090] The reading process of the sub-matrix of the non-first row under the output matrix provided in this embodiment adds the address count of the fourth counter, and completes the relationship between the first counter, the second counter and the third counter, distinguishes the hierarchical definition of the sub-matrices in different rows, and improves the accuracy of data conversion.

[0091] Fig.11 A flowchart of a shard writing mode provided by an embodiment of the present invention, such as Fig.11 As shown, the specific steps are as follows: It should be noted that this embodiment is based on the situation of continuous reading and writing of multiple data. Therefore, for the general association logic corresponding to the arrival of the Nth data: S20: Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; if so, trigger step S21; if not, trigger step S22; S21: Reset the address count of the first counter to an initial value; and trigger steps S23, S24, S28 and S29; S22: increment the address count of the first counter by 1; and trigger step S23; S23: Output the current address count of the first counter; S24: Determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if so, proceed to step S26; if not, proceed to step S25; S25: the updated number of continuous preset written data in the vertical direction is incremented by 1; and the process proceeds to step S37; S26: the updated number of continuous preset write data in the vertical direction is 0; and the process proceeds to steps S27, S28, S33, S37, S38 and S40; S27: Outputting the updated number of continuous preset write data in the vertical direction; S28: setting the address count of the second counter to an initial value; and proceeding to step S32; S29: Determine whether the address count of the second counter is equal to the row dimension of the submatrix - 1; if so, proceed to step S30; if not, proceed to step S31; S30: setting the address count of the second counter to an initial value; and proceeding to step S32; S31: taking the sum of the remaining unwritten column dimensions and row dimensions corresponding to the remaining slices as the address count of the second counter; and proceeding to step S32; S32: Output the address count of the second counter; S33: Determine whether the remaining unwritten column dimension after the update is 0; if not, proceed to step S34; if yes, proceed to step S35; S34: performing difference processing on the updated remaining unwritten column dimension and the column dimension of the submatrix to obtain a new remaining unwritten column dimension; and proceeding to step S36; S35: setting the remaining unwritten column dimensions after the update to the remaining unwritten column dimensions before the update, and proceeding to steps S36 and S40; S36: Output the current remaining unwritten column dimensions; S37: Set the address count of the third counter to 0; and proceed to step S39; S38: setting the address count of the third counter to the value corresponding to the column dimension of the sub-matrix; and proceeding to step S39; S39: Output the address count of the third counter; S40: setting the address count of the fourth counter to the sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter plus 1; and entering step S40; S41: Output the address count of the fourth counter.

[0092] Specifically, combined Fig. 9 Let's take a look at it. 1) When the first data arrives, the four address counters sfu_waddr_cnt0, sfu_waddr_cnt1, sfu_waddr_cnt2, and sfu_waddr_block_base, as well as vertical_num_cnt and horizontal_tile_num_left are all initialized to 0, so the first data is written to the special function unit (Special Function Unit) cache address 0; then the result of S20 is no, and S22 is executed, sfu_waddr_cnt0++, which becomes 1. The special function unit (Special Function Unit, SFU) here is a non-matrix multiplication acceleration unit.

[0093] 2) When the second data arrives, sfu_waddr_cnt0 is 1, and the other three address counters are 0, so the second data is written to SFU cache address 1; then the result of S20 is no, and S22 is executed, sfu_waddr_cnt0++, which becomes 2.

[0094] 3) When the third data arrives, sfu_waddr_cnt0 is 2, and the other three address counters are 0, so the third data is written to SFU cache address 2; trigger Fig.10 In S20, S24 and S29, the result of S20 is yes, S21 is executed, so that sfu_waddr_cnt0 becomes 0 again; the result of S24 is no, S25 is executed, so that vertical_num_cnt becomes 1; the result of S29 is no, S31 is executed, so that sfu_waddr_cnt1 becomes 6+3=9.

[0095] 4) When the fourth data arrives, sfu_waddr_cnt1 is 9, and the other three address counters are 0, so the fourth data is written to the SFU cache address 9; and the result of S20 is no, S22 is executed, sfu_waddr_cnt0++ becomes 1.

[0096] 5) When the fifth data arrives, sfu_waddr_cnt0 is 1, sfu_waddr_cnt1 is 9, and the other two address counters are 0, so the fifth data is written to the SFU cache address 10; and the result of S20 is no, S22 is executed, sfu_waddr_cnt0++, which becomes 2.

[0097] 6) When the sixth data arrives, sfu_waddr_cnt0 is 2, sfu_waddr_cnt1 is 9, and the other two address counters are 0, so the sixth data is written to the SFU cache address 11, triggering again Fig.10 In S20 and S24, if the result of S20 is yes, S21 is executed, so that sfu_waddr_cnt0 becomes 0 again; if the result of S24 is yes, S26 is executed, so that vertical_num_cnt becomes 0. At this time, since both conditions for triggering S28 are met (i.e., S21 and S26), S28 is directly executed, so that sfu_waddr_cnt1 becomes 0, and S29 is not executed; S26 will also trigger S33, and if the result is no, S34 is executed, so that horizontal_tile_num_left becomes 6-3=3; S26 will also trigger S221, so that sfu_waddr_cnt2 becomes 3.

[0098] 7) When the seventh data arrives, sfu_waddr_cnt2 is 3, and the other three address counters are all 0; so the seventh data is written to SFU cache address 3; and the result of S20 is no, execute S22, sfu_waddr_cnt0++, and become 1.

[0099] 8) When the eighth data arrives, sfu_waddr_cnt2 is 3, sfu_waddr_cnt0 is 1, and the other two address counters are 0, so the eighth data is written to SFU cache address 4, and the result of S20 is no, S22 is executed, and sfu_waddr_cnt0 becomes 2.

[0100] 9) When the 9th data arrives, sfu_waddr_cnt2 is 3, sfu_waddr_cnt0 is 2, and the other two address counters are 0, so the 9th data is written to the SFU cache address 5, triggering S20, the result is yes, executing S21, sfu_waddr_cnt0 becomes 0; triggering S24, the result is no, executing S25, vertical_num_cnt becomes 1; triggering S29, the result is no, executing S31, sfu_waddr_cnt1 becomes 9.

[0101] 10) When the 10th data arrives, sfu_waddr_cnt1 is 9, sfu_waddr_cnt2 is 3, and the other two address counters are 0; so the 10th data is written to the SFU cache address 12; S20 is triggered, the result is no, sfu_waddr_cnt0++ is executed, and sfu_waddr_cnt0 becomes 1.

[0102] 11) When the 11th data arrives, sfu_waddr_cnt0 is 1, sfu_waddr_cnt1 is 9, and fu_waddr_cnt2 is 3, so the 11th data is written to the SFU cache address 13; and the result of S20 is no, S22 is executed, sfu_waddr_cnt0++, which becomes 2.

[0103] 12) When the 12th data arrives, sfu_waddr_cnt0 is 2, sfu_waddr_cnt1 is 9, and sfu_waddr_cnt2 is 3, so the 12th input is written to SFU cache address 14; and the result of S20 is yes, S21 is executed, sfu_waddr_cnt0 becomes 0; S24 is executed, the result is yes, S26 is executed, vertical_num_cnt becomes 0; at this time, S21 and S26 are both true, so S28 is executed, sfu_waddr_cnt1 becomes 0; S26 will also trigger S33, the result is no, S34 is executed, horizontal_tile_num_left becomes 3-3=0; S26 will also trigger S38, sfu_waddr_cnt2 becomes 3+3=6.

[0104] 13) When the 13th data arrives, sfu_waddr_cnt2 is 6 and the other three counters are 0; so the 13th input data is written to SFU cache address 6; and the result of S20 is no, S22 is executed, sfu_waddr_cnt0++, and becomes 1.

[0105] 14) When the 14th data arrives, sfu_waddr_cnt2 is 6, sfu_waddr_cnt0 is 1, and the other two address counters are 0, so the 14th input data is written to the SFU cache address 7; and the result of S20 is no, S22 is executed, and sfu_waddr_cnt0 becomes 2.

[0106] 15) When the 15th data arrives, sfu_waddr_cnt2 is 6, sfu_waddr_cnt0 is 2, and the other two address counters are 0, so the 15th input data is written to SFU cache address 8; triggering S20, the result is yes, executing S21, sfu_waddr_cnt0 becomes 0; triggering S24, the result is no, executing S25, vertical_num_cnt becomes 1; triggering S29, the result is no, executing S31, sfu_waddr_cnt1 becomes 9.

[0107] 16) When the 16th data arrives, sfu_waddr_cnt1 is 9, sfu_waddr_cnt2 is 6, and the other two address counters are 0; so the 16th data is written to the SFU cache address 15; trigger S20, the result is no, execute sfu_waddr_cnt0++, and sfu_waddr_cnt0 becomes 1.

[0108] 17) When the 17th data arrives, sfu_waddr_cnt0 is 1, sfu_waddr_cnt1 is 9, and fu_waddr_cnt2 is 6, so the 11th data is written to the SFU cache address 16; and the result of S20 is no, S22 is executed, sfu_waddr_cnt0++, which becomes 2.

[0109] 18) When the 18th data arrives, sfu_waddr_cnt0 is 2, sfu_waddr_cnt1 is 9, and sfu_waddr_cnt2 is 6, so the 18th input is written to the SFU cache address 17; and the result of S20 is yes, S21 is executed, sfu_waddr_cnt0 becomes 0; S24 is executed, the result is yes, S26 is executed, vertical_num_cnt becomes 0; at this time, S21 and S26 are both true, so S28 is executed, making sfu_waddr_c nt1 becomes 0; S26 will also trigger S33, the result is yes, S35 is executed, making horizontal_tile_num_left become 6; if S26 and S35 are both true, S37 will be triggered (S38 will no longer be triggered), making sfu_waddr_cnt2 become 0; if S26 and S35 are both true, S40 will be triggered, making sfu_waddr_block_base become 17+1=18 (sfu_waddr_cnt_all is a sequential counter, and each incoming data will be +1).

[0110] 19) When the 19th data arrives, sfu_waddr_cnt0 is 0, sfu_waddr_cnt1 is 0, sfu_waddr_cnt2 is 0, and sfu_waddr_block_base is 18, so the 19th input is written to SFU cache address 18.

[0111] Currently, only the 1st to 19th data are read as an example. The subsequent data are read in the same way and will not be described in detail here.

[0112] It is understandable that the above-mentioned segmented write mode is not always used in the write operation mode. Therefore, when performing less nonlinear transformation on data, the normal write operation mode can be used instead of the segmented write mode, so it is necessary to determine when to perform the write operation mode.

[0113] In some embodiments, the process of determining the write operation mode includes: Determine whether the corresponding write operation mode under the current write operation is the shard write mode; If yes, proceed to step S11; If not, the write operation mode is determined to be the normal write mode. When the read operation of the first data of the matrix multiplication acceleration unit is valid and the write operation mode is valid, the sending address of the first data is directly assigned to the write operation address of the non-matrix multiplication acceleration unit.

[0114] Specifically, when the write operation started this time is in normal mode, whenever matrix_out_valid is valid, sfu_write_valid is valid, and sfu_write_addr is incremented (+1 each time), and matrix_out_data is directly assigned to sfu_write_data to complete the operation of writing data into the SFU cache.

[0115] The present embodiment provides that when performing a write operation mode, it is necessary to determine whether it is a normal write operation mode or a sharded write operation mode. Through this determination, the non-linear transformation calculation corresponding to the non-matrix multiplication acceleration unit that does not require a large number of address conversion operations is reduced, thereby accelerating the data conversion efficiency and improving the data writing efficiency to facilitate subsequent data processing.

[0116] In some embodiments, the write matrix dimension of the first data in the matrix multiplication acceleration unit fills the current output matrix of the first data in the non-matrix multiplication acceleration unit so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, including: Obtain the first column dimension and the first row dimension of the current output matrix of the non-matrix multiplication acceleration unit of the first data; Obtain a second column dimension and a second row dimension of a write matrix dimension of the first data in the matrix multiplication acceleration unit; Determine the third column dimension and the third row dimension to be filled according to the first column dimension and the second column dimension, the first row dimension and the second row dimension; The current output matrix of the first data in the non-matrix multiplication acceleration unit is filled according to the third column dimension and the third row dimension, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit.

[0117] Fig.12 A schematic diagram of parameters of a service filling read mode provided in an embodiment of the present invention, such as Fig.12 As shown in the figure, the area filled with zeros is filled based on the row and column dimensions of the matrix multiplication acceleration unit. padding_enable is whether to enable the padding read mode; actual_cols: the number of columns of the matrix actually to be calculated; actual_rows: the number of rows of the matrix actually to be calculated; complete_cols: the number of all columns. When this value is greater than actual_cols, it means that zeros need to be padded in the horizontal direction; zero_rows: how many rows of data with all zeros need to be padded; combined Fig.12 When the data dimension of the row or column of actual_data cannot satisfy the data input dimension of the systolic array, padding operation will be automatically performed according to the control status register.

[0118] The third column dimension and the third row dimension to be filled need to be calculated, and zero filling processing will be performed according to the row and column dimensions to be filled to ensure that the matrix dimension of the third data is the same as the matrix dimension of the matrix multiplication acceleration unit.

[0119] In some embodiments, a filling process is performed on a current output matrix of the first data in the non-matrix multiplication acceleration unit according to a third column dimension and a third row dimension, so that a matrix dimension of the filled third data is the same as a matrix dimension of the matrix multiplication acceleration unit, including: Writing the first data into the cache of the matrix multiplication acceleration unit according to each row of data of the current output matrix; Performing filling processing on the third column dimension in the first column dimension of the last column of each row of data to obtain filled fourth data; When the last row of data is written, a first matrix is ​​generated according to the third row dimension and the second column dimension; and each data of the first matrix is ​​filled to obtain the filled fifth data; The padded fourth data, fifth data and first data are used as padded third data.

[0120] Specifically, the non-matrix multiplication acceleration unit of each row is read first. After reading the data of the first row, padding processing is performed through the third column dimension to obtain the padded fourth data, and then the data of the second row is read, and padding processing is performed according to the third column dimension. After the data of the non-matrix multiplication acceleration unit of all rows is read, it is read according to the second column dimension. The first matrix dimension read is composed of the third row dimension * (second column dimension + third column dimension), which is used as the padded fifth data; the fourth data, the fifth data and the first data are used as the final third data.

[0121] Fig.13 A schematic diagram of data in a read operation mode provided by an embodiment of the present invention, such as Fig.13 As shown in the figure, the actual data read by the SFU read cache controller from the SFU cache consists of the purple area in the figure. Assuming that there are 7 data in a row and there are 3 rows in total, and the systolic array requires the column dimension to be 9 and the row dimension to be 5 (generally, these two dimensions are required to be consistent, and this is only used as an example), then we set actual_cols to 7, actual_rows to 3, complete_cols to 7+2=9, and zero_rows to 2; 1) The SFU read cache controller reads the first line of the purple area and sends it to the SFU computing control unit. The SFU computing control unit schedules the SFU computing core unit to perform calculations and sends the generated calculation results to the result data aggregation module.

[0122] 2) The SFU calculation control unit uses the internal sfu_row_rnum_cnt to count: sfu_row_cnum_cnt: Every time the SFU calculation control unit sends a result data to the result data aggregation module, it is +1. When sfu_row_cnum_cnt==actual_cols, it returns to 0 and reports to the SFU read cache controller.

[0123] 3) After the SFU read cache controller detects the report when sfu_row_cnum_cnt==actual_cols, it immediately sends complete_cols-actual_cols result data with a value of 0 to the result data aggregation module, then increases the internal sfu_row_rnum_cnt counter by 1, and then continues to read the purple data of the second row.

[0124] 4) Repeat the above process until 3 rows of purple data are read, generating 3 paddings of size complete_cols-actual_cols; at this time sfu_row_rnum_cnt==actual_rows, the SFU read cache controller will directly generate zero_rows*complete_cols result data with a value of 0 and send it to the result data aggregation module.

[0125] The automatic filling method in the read operation mode provided in this embodiment realizes the function of generating data dimensions that meet the requirements of the matrix multiplication acceleration unit for the actual calculation results, avoiding the high-latency operation of writing data of non-matrix multiplication acceleration units back to the off-chip memory and then reading it.

[0126] The above describes in detail various embodiments corresponding to the data reading and writing method. On this basis, the present invention also discloses a data reading and writing device corresponding to the above method. Fig.14 The structure diagram of a data reading and writing device provided by an embodiment of the present invention. Fig.14 As shown, the data reading and writing device includes: An acquisition module 11 is used to acquire a read and write mode of the first data in a non-matrix multiplication acceleration unit in a data accelerator; The first processing module 12 is used for, if the first data is in a write operation mode in the non-matrix multiplication acceleration unit, converting the sending address of the first data according to the target slice position and row and column dimension position of the systolic array under the output matrix of the matrix multiplication acceleration unit of the first data to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, so as to write the converted second data into the non-matrix multiplication acceleration unit; The second processing module 13 is used to fill the current output matrix of the first data in the non-matrix multiplication acceleration unit with the write matrix dimension of the first data in the matrix multiplication acceleration unit if the first data is in the read operation mode in the non-matrix multiplication acceleration unit, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to be read out to the matrix multiplication acceleration unit.

[0127] Since the embodiments of the device part correspond to the above embodiments, please refer to the description of the embodiments of the method part for the embodiments of the device part, and will not be repeated here.

[0128] For an introduction to a data reading and writing device provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above data reading and writing method.

[0129] Fig.15 A structural diagram of an accelerator provided by an embodiment of the present invention, such as Fig.15 As shown, the device comprises: A memory 21, used for storing computer programs; The processor 22 is used to implement the steps of the data reading and writing method when executing the computer program.

[0130] Among them, the processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 22 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 22 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 22 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0131] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used to store the following computer program 211, wherein, after the computer program is loaded and executed by the processor 22, the relevant steps of the data reading and writing method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include but is not limited to the data involved in the data reading and writing method, etc.

[0132] In some embodiments, the accelerator may further include a display screen 23 , an input / output interface 24 , a communication interface 25 , a power supply 26 , and a communication bus 27 .

[0133] Those skilled in the art can understand that Fig.15The structure shown in the figure does not constitute a limitation of the accelerator, and may include more or fewer components than shown in the figure.

[0134] The processor 22 implements the data reading and writing method provided by any of the above embodiments by calling the instructions stored in the memory 21 .

[0135] For an introduction to an accelerator provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. It has the same beneficial effects as the above data reading and writing method.

[0136] Furthermore, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor 22, the steps of the above-mentioned data reading and writing method are implemented.

[0137] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0138] For an introduction to a computer-readable storage medium provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. It has the same beneficial effects as the above data reading and writing method.

[0139] Furthermore, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the data reading and writing method when executed by a processor.

[0140] For an introduction to a computer program product provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. It has the same beneficial effects as the above data reading and writing method.

[0141] The above is a detailed introduction to a data reading and writing method, product, data accelerator and medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

[0142] It should also be noted that, in this specification, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for reading and writing data, characterized in that: include: Acquire a read and write mode of the first data in a non-matrix multiplication acceleration unit in the data accelerator; If the first data is in a write operation mode in the non-matrix multiplication acceleration unit, a sending address of the first data is converted according to a target slice position and a row and column dimension position of a systolic array under an output matrix of the matrix multiplication acceleration unit to determine a write operation address of the converted second data in the non-matrix multiplication acceleration unit, so as to write the converted second data into the non-matrix multiplication acceleration unit; If the first data is in a read operation mode in a non-matrix multiplication acceleration unit, the write matrix dimension of the first data in the matrix multiplication acceleration unit fills the current output matrix of the first data in the non-matrix multiplication acceleration unit, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, so as to be read out to the matrix multiplication acceleration unit.

2. The method for reading and writing data according to claim 1, characterized in that: The sending address of the first data is converted according to the target slice position and row and column dimension position of the systolic array under the output matrix of the matrix multiplication acceleration unit to determine the write operation address of the converted second data in the non-matrix multiplication acceleration unit, including: Preset the corresponding mapping relationship between the preset counter and the target shard position and the row and column dimension position; According to the mapping relationship between the target sharding setting and the row and column dimension positions and the preset counter, and the address count of the preset counter, the sending address of the first data is converted to determine the write operation address of the second data in the non-matrix multiplication acceleration unit.

3. The method for reading and writing data according to claim 2, characterized in that: The preset counter includes a first counter, a second counter, a third counter and a fourth counter; The first counter is used to record the address count of the submatrix in the horizontal direction under each slice of the output matrix; the address count of the first counter is limited by the number of consecutive preset write data in the horizontal direction, and is the column dimension of the submatrix - 1; The second counter is used to record the first reference address count in the horizontal direction of the non-first row of the submatrix under each slice of the output matrix; the first reference address count is limited by the number of consecutive preset write data in the vertical direction and is the row dimension of the submatrix-1; The third counter is used to record the second reference address count in the horizontal direction of the submatrix under the non-first slice of the output matrix based on the row; The fourth counter is used to record the third reference address count in the horizontal direction of the submatrix under each slice of the output matrix based on the non-first row; wherein the value of the third reference address count is the address value of the last data of the last slice of the previous row of the current row of the output matrix.

4. The method for reading and writing data according to claim 2 or 3, characterized in that: The target slice position is obtained by dividing the first data into corresponding positions of all systolic arrays under the output matrix of the matrix multiplication acceleration unit to obtain the position information corresponding to the slice size; wherein the row and column dimensions of the output matrix are integer multiples of the slice size.

5. The method for reading and writing data according to claim 4, characterized in that: The mapping relationship between the preset counter and the target shard position and the row and column dimension position is pre-set, including: If the target slice position is the first slice of the first row of the output matrix, a first mapping relationship is established between the column dimension positions under the first row of the submatrix where the first data of the first slice is located and the address count of the first counter; wherein the address counts of the remaining counters except the first counter are 0; a second mapping relationship is established between the column dimension positions corresponding to the rows under the non-first row of the submatrix where the first data of the first slice is located, the address count of the first counter, and the address count of the second counter; wherein the address counts of the remaining counters except the first counter and the second counter are 0; If the target slice position is the remaining slices except the first slice under the first row of the output matrix, a third mapping relationship is established between the column dimension positions under the first row of the submatrix where the first data of the remaining slices are located, the address count of the first counter, and the address count of the third counter; wherein the address counts of the remaining counters except the first counter and the third counter are 0; a fourth mapping relationship is established between the column dimension positions corresponding to each row under the non-first row of the submatrix where the first data of the remaining slices are located, and the address counts corresponding to the first counter, the second counter, and the third counter respectively; wherein the address count of the fourth counter is 0; If the target slice position is the first slice in the non-first row of the output matrix, then on the basis of establishing the first mapping relationship and the second mapping relationship, the address count of the fourth counter is added respectively to form the fifth mapping relationship and the sixth mapping relationship respectively; if the target slice position is the remaining slices except the first slice in the non-first row of the output matrix, then on the basis of establishing the third mapping relationship and the fourth mapping relationship, the address count of the fourth counter is added respectively to form the seventh mapping relationship and the eighth mapping relationship respectively.

6. The method for reading and writing data according to claim 5, characterized in that: If the target slice position is the first slice of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the first mapping relationship between the row and column dimension positions and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: When the sending address of the current first sub-data in the first data is the first address data under the first row of the sub-matrix, the address counts of all preset counters, the number of continuous preset write data in the horizontal direction, and the number of continuous preset write data in the vertical direction are all set to initial values; the column dimension position of the sub-matrix corresponding to the current first sub-data is determined, and the write operation address of the current data is determined according to the column dimension position of the sub-matrix corresponding to the current first sub-data and the initial values ​​of the address counts corresponding to the first counter, the second counter, the third counter, and the fourth counter respectively; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If not, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next first subdata of the current first subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next first subdata, and the write operation address of the next first subdata is determined according to the current address count of the first counter, the sum of the initial values ​​of the address counters corresponding to the second counter, the third counter and the fourth counter; and the next first subdata is used as the new current first subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current first sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

7. The method for reading and writing data according to claim 6, characterized in that: If the target slice position is the first slice of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the second mapping relationship between the row and column dimension positions and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1; if the updated number of continuous preset write data in the vertical direction is not equal to the row dimension of the sub-matrix -1, determine that the number of continuous preset write data in the vertical direction is updated by adding 1; Determine whether the address count of the second counter is the row dimension of the submatrix - 1; if the address count of the second counter is not the row dimension of the submatrix - 1, obtain the remaining unwritten column dimensions corresponding to the remaining slices except the first slice in the first row; and use the sum of the number of remaining unwritten data and the row dimension as the address count of the second counter; When the sending address of the current second sub-data in the first data is the first address data under the non-first row of the sub-matrix, the column dimension position of the sub-matrix corresponding to the current second sub-data is determined, and the write operation address of the current second sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current second sub-data and the address count of the second counter, the first counter, the third counter and the fourth counter respectively corresponding to the initial value of the address count; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next second subdata of the current second subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next second subdata, and the write operation address of the next second subdata is determined according to the current address count of the first counter, the address count of the second counter, and the sum of the initial values ​​of the address counts corresponding to the third counter and the fourth counter; and the next second subdata is used as the new current second subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current second sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

8. The method for reading and writing data according to claim 7, characterized in that: If the target slice position is the remaining slices of the first row of the output matrix, the sending address of the first data is converted and processed according to the target slice setting and the third mapping relationship between the row and column dimension positions and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1, set the updated number of continuous preset written data in the vertical direction to 0; Determine whether the address count of the second counter is the row dimension of the sub-matrix - 1; if the address count of the second counter is the row dimension of the sub-matrix - 1, reset the address count of the second counter to an initial value; Determine whether the remaining unwritten column dimension after the update is 0; if the remaining unwritten column dimension after the update is not 0, perform difference processing on the remaining unwritten column dimension after the update and the column dimension of the submatrix to obtain a new remaining unwritten column dimension; Setting the address count of the third counter to a value corresponding to the column dimension of the submatrix; When the sending address of the current third sub-data in the first data is the first address data under the first row of the sub-matrix of the remaining slices, determine the column dimension position of the sub-matrix corresponding to the current third sub-data, and determine the write operation address of the current second sub-data according to the column dimension position of the sub-matrix corresponding to the current third sub-data and the address count of the third counter, and the initial values ​​of the address counts corresponding to the first counter, the second counter, and the fourth counter; and update the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next third subdata of the current third subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next third subdata, and the write operation address of the next third subdata is determined according to the current address count of the first counter, the address count of the third counter, and the sum of the initial values ​​of the address counts corresponding to the second counter and the fourth counter; and the next third subdata is used as the new current third subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current third sub-data in the first data reaches the number of consecutive preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

9. The method for reading and writing data according to claim 8, characterized in that: If the target slice position is the remaining slices of the first row of the output matrix, the sending address of the first data is converted and processed according to the fourth mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset write data in the vertical direction is equal to the row dimension of the sub-matrix -1; if the updated number of continuous preset write data in the vertical direction is not equal to the row dimension of the sub-matrix -1, determine that the number of continuous preset write data in the vertical direction is updated by adding 1; Determine whether the address count of the second counter is the row dimension of the submatrix - 1; if the address count of the second counter is not the row dimension of the submatrix - 1, obtain the remaining unwritten column dimensions corresponding to the remaining slices except the first slice in the first row; and use the sum of the number of remaining unwritten data and the row dimension as the address count of the second counter; When the sending address of the current fourth sub-data in the first data is the first address data under the non-first row of the sub-matrix, the column dimension position of the sub-matrix corresponding to the current fourth sub-data is determined, and the write operation address of the current fourth sub-data is determined according to the column dimension position of the sub-matrix corresponding to the current fourth sub-data and the address count of the second counter, the address count of the third counter, and the initial values ​​of the address counts corresponding to the first counter and the fourth counter respectively; and the number of continuous preset write data in the horizontal direction and the number of continuous preset write data in the vertical direction are updated; Determine whether the address count of the first counter is equal to the column dimension of the submatrix - 1; If the address count of the first counter is not equal to the column dimension of the submatrix -1, the initial value of the address count of the first counter is incremented by 1; the column dimension position of the submatrix corresponding to the next fourth subdata of the current fourth subdata in the first data is determined, and the initial value of the address count of the first counter is incremented by 1 according to the column dimension position of the submatrix corresponding to the next fourth subdata, and the write operation address of the next fourth subdata is determined according to the sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter; and the next second subdata is used as the new current fourth subdata, and the process returns to the step of determining whether the address count of the first counter is equal to the column dimension of the submatrix -1; If so, when it is determined that the column dimension position of the first row of the submatrix corresponding to the new current fourth sub-data in the first data reaches the number of continuous preset written data in the horizontal direction, the address count of the first counter is reset to the initial value.

10. The method for reading and writing data according to claim 9, characterized in that: If the target slice position is the first slice of the non-first row of the output matrix, the sending address of the first data is converted and processed according to the fifth mapping relationship between the target slice setting and the row and column dimension position and the preset counter, and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit, including: Pre-determine whether the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1; if the updated number of continuous preset written data in the vertical direction is equal to the row dimension of the sub-matrix - 1, set the updated number of continuous preset written data in the vertical direction to 0; Determine whether the address count of the second counter is the row dimension of the sub-matrix - 1; if the address count of the second counter is the row dimension of the sub-matrix - 1, reset the address count of the second counter to an initial value; Determine whether the remaining unwritten column dimension after the update is 0; if the remaining unwritten column dimension after the update is 0, set the remaining unwritten column dimension after the update to the remaining unwritten column dimension before the update; Resetting the address count of the third counter to an initial value; The sum of the current address count of the first counter, the address count of the second counter, the address count of the third counter and the initial value of the address count of the fourth counter is added by 1 to obtain the current address count of the fourth counter; and the process returns to the step of converting the sending address of the first data according to the first mapping relationship between the target sharding setting and the row and column dimension positions and the preset counter and the address count of the preset counter to determine the write operation address of the second data in the non-matrix multiplication acceleration unit to determine the write operation address of the current fifth sub-data in the first data.

11. The method for reading and writing data according to claim 1, characterized in that: The write matrix dimension of the first data in the matrix multiplication acceleration unit fills the current output matrix of the first data in the non-matrix multiplication acceleration unit so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, including: Obtaining the first column dimension and the first row dimension of the current output matrix of the non-matrix multiplication acceleration unit of the first data; Obtain a second column dimension and a second row dimension of a write matrix dimension of the first data in the matrix multiplication acceleration unit; Determine the third column dimension and the third row dimension to be filled according to the first column dimension and the second column dimension, the first row dimension and the second row dimension; The current output matrix of the first data in the non-matrix multiplication acceleration unit is filled according to the third column dimension and the third row dimension, so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit.

12. The method for reading and writing data according to claim 11, characterized in that: Filling the current output matrix of the first data in the non-matrix multiplication acceleration unit according to the third column dimension and the third row dimension so that the matrix dimension of the filled third data is the same as the matrix dimension of the matrix multiplication acceleration unit, including: Writing the first data into the cache of the matrix multiplication acceleration unit according to each row of data of the current output matrix; Performing filling processing on the third column dimension in the first column dimension of the last column of each row of data to obtain filled fourth data; When the last row of data is written, a first matrix is ​​generated according to the third row dimension and the second column dimension; and each data of the first matrix is ​​filled to obtain the filled fifth data; The padded fourth data, fifth data and first data are used as padded third data.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method for reading and writing data described in any one of claims 1 to 12 are implemented.

14. A data accelerator, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data reading and writing method as claimed in any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for reading and writing data according to any one of claims 1 to 12 are implemented.

Citation Information

Patent Citations

  • Accelerator capable of configuring matrix multiplication

    CN111124360A

  • Operation accelerator, processing method, and related device

    CN112840356A

  • Neural network training method and device

    CN117910537A

  • Method for realizing local caching of data in systolic array calculation

    CN118012340A

  • Accelerator, data processing method, device, medium, program product and system

    CN118536073A