Accelerated units and systems on a chip

Abstract

The application provides an acceleration unit and a system on chip, the acceleration unit comprising: a memory for storing matrix data; multiply-add modules, each multiply-add module comprising at least two multiply-add units and a two-stage accumulator, for completing multiply-add operations of matrices and vectors; a controller for loading the matrix data from the memory to the multiply-add units in the multiply-add modules in the column order of the matrix; and a data output unit for outputting the multiply-add operation results of matrices and vectors.

Description

Technical Field

[0001] This application relates to the field of digital signal processing technology, and in particular to an acceleration unit and a system-on-a-chip. Background Technology

[0002] In recent years, large-scale language models based on Transformers have primarily relied on matrix and vector multiplication operations to generate word output during the decoding phase. This has resulted in a large number of such operations being required for dialogue tasks. In tasks processing massive amounts of data, matrix and vector multiplication involves enormous input data that cannot be reused in computation, making it a data-intensive application. Therefore, optimizing the utilization of storage bandwidth is crucial. Furthermore, designing dedicated acceleration units can effectively improve computational efficiency.

[0003] In related technologies, FPGAs (Field-Programmable Gate Arrays) are often used to accelerate matrix and vector multiplication due to their programmability. However, in FPGA systems, maximizing memory bandwidth and DSP (Digital Signal Processor) resource utilization efficiency while reducing power consumption is a key design challenge. Furthermore, batch processing of multiple vectors is a common requirement, and DSP frequency multiplication technology can improve computational performance to meet this need.

[0004] Furthermore, supporting multiplication and accumulation calculations for different data types, such as simultaneously supporting int8 (8-bit integer) and int4 (4-bit integer), further increases the complexity of the design. Therefore, when customizing hardware solutions, it is necessary to comprehensively consider multiple factors such as storage bandwidth, DSP resources, power consumption, and computing performance to optimize matrix and vector multiplication operations in the decoding stage of large language models.

[0005] However, the FPGA matrix computing acceleration system in related technologies is quite complex in designing data flow control, involving multi-level data transmission and storage buffering mechanisms, resulting in unsatisfactory data transmission energy efficiency.

[0006] Although related technologies can achieve double-frequency operation of on-chip DSP units while maximizing FPGA memory access bandwidth utilization, their support for mixed-precision calculations, such as int8 and int4 data types, is still insufficient. Summary of the Invention

[0007] This application provides an acceleration unit and a system-on-a-chip that solves the problems in the prior art, such as complex data flow control, multiple layers of data transmission and storage buffers in the computing system, low data transmission energy efficiency, and inability to well support calculations of mixed precision data types such as int8 and int4.

[0008] To achieve the above objectives, this application adopts the following technical solution:

[0009] In a first aspect, an acceleration unit is provided, characterized in that the acceleration unit comprises:

[0010] Memory, used to store matrix data;

[0011] The multiply-accumulate module, each of which includes at least two multiply-accumulate units and a two-stage accumulator, is used to perform multiply-accumulate operations on matrices and vectors;

[0012] A controller is used to load the matrix data from the memory into the multiply-accumulate units of the multiply-accumulate module in the column order of the matrix.

[0013] The data output unit is used to output the results of matrix and vector multiplication and addition operations.

[0014] Optionally, the elements in the same column of data in the matrix are sequentially input into different multiply-accumulate units in the same column of the multiply-accumulate module.

[0015] Optionally, the inputs of the multipliers in different multiply-accumulate modules may store different vector data.

[0016] Optionally, different rows of data in the same column of the matrix are input into the corresponding multiply-accumulate units after being delayed by different periods; the different delay periods are determined according to the delay of the multiply-accumulate operation of the matrix and the vector.

[0017] Optionally, the multiple acceleration units can be cascaded into a group, and the result of the multiplication-addition operation of one acceleration unit in the group can be used as the input of another acceleration unit.

[0018] Optionally, the final multiply-accumulate result of the set of acceleration units is input into the accumulator.

[0019] Optionally, the multiply-accumulate unit includes at least two digital signal processing units.

[0020] Each of the at least two digital signal processing units includes: a first register, a third register, and a fourth register for storing vector data;

[0021] The first register is connected to the third register and the fourth register;

[0022] The matrix data register and the second register are used to store the matrix data read from the matrix's memory;

[0023] The matrix data registers of the at least two digital signal processing units are interconnected.

[0024] Optionally, in one of the at least two digital signal processing units, the third register and the fourth register store different vector data;

[0025] The multiplication and addition operations of the vector data in the third register and the matrix data in the second register are calculated within one clock cycle;

[0026] In the next clock cycle, perform a multiplication and addition operation between the vector data in the fourth register and the matrix data in the second register.

[0027] Optionally, the matrix data in the at least two digital signal processing units are the same, wherein the matrix data of the first digital signal processing unit is delayed before entering the second digital signal processing unit.

[0028] Optionally, the matrix data in the at least two digital signal processing units are different, wherein the matrix data of the first digital signal processing unit is the low-order data in a column of data, and the matrix data of the second digital signal processing unit is the high-order data in the column of data.

[0029] The first register of the first digital signal processing unit and the first register of the second digital signal processing unit store different portions of the same vector data.

[0030] In a second aspect, a system-on-a-chip is provided, the system-on-a-chip including an acceleration unit as described in any of the first aspects, for performing matrix and vector multiplication and addition operations.

[0031] The acceleration unit provided in this application embodiment can complete the calculation of 4 sets of vectors by loading matrix data once, which improves the calculation efficiency of the system batch processing, makes full use of the calculation bit width of the multiply-accumulate unit, and reduces the hardware connection overhead of the input port of the multiply-accumulate unit; by forming a group of two columns of multiply-accumulate units, it can simultaneously process the multiplication and addition of one int8 matrix data and two vectors, or reuse the data path to process the multiplication and addition of two int4 matrix data and one vector, reducing the wiring overhead of the FPGA system and improving the system calculation efficiency; and through the two-level accumulation structure, the system can support matrix-vector multiplication calculations of different scales and batch sizes. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the structure of an acceleration unit provided in an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of a connection structure for loading matrix data into a multiply-accumulate unit, provided in an embodiment of this application.

[0034] Figure 3A schematic diagram of an accumulation structure provided in an embodiment of this application;

[0035] Figure 4 This application provides a schematic diagram of the internal connection structure of a DSP module.

[0036] Figure 5 This is a schematic diagram of the internal connection structure of a multiply-accumulate unit provided in an embodiment of this application. Detailed Implementation

[0037] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known principles of matrix multiplication and electronic devices are omitted so as not to obscure the description of this application with unnecessary detail.

[0038] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “the,” “the,” and “the” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise.

[0039] In recent years, large-scale language models based on Transformers have primarily relied on matrix-vector multiplication operations to generate word output during the decoding phase. This results in a large number of such operations being required for dialogue tasks. In tasks processing massive amounts of data, matrix-vector multiplication involves enormous input data that cannot be reused in computation, making it a data-intensive application. Therefore, optimizing the utilization of storage bandwidth is crucial. Furthermore, designing dedicated acceleration units can effectively improve computational efficiency.

[0040] In related technologies, FPGAs (Field-Programmable Gate Arrays) are often used to accelerate matrix-vector multiplication due to their programmability. However, in FPGA systems, maximizing memory bandwidth and DSP (Digital Signal Processor) resource utilization efficiency while reducing power consumption is a key design challenge. Furthermore, batch processing of multiple vectors is a common requirement, and DSP frequency multiplication technology can improve computational performance to meet this need.

[0041] Furthermore, supporting multiplication and accumulation calculations for different data types, such as simultaneously supporting int8 (8-bit integer) and int4 (4-bit integer), further increases the complexity of the design. Therefore, when customizing hardware solutions, it is necessary to comprehensively consider multiple factors such as storage bandwidth, DSP resources, power consumption, and computing performance to optimize matrix-vector multiplication operations in the decoding stage of large language models.

[0042] However, the FPGA matrix computing acceleration system in related technologies is quite complex in designing data flow control, involving multi-level data transmission and storage buffering mechanisms, resulting in unsatisfactory data transmission energy efficiency.

[0043] Although related technologies can achieve double-frequency operation of on-chip DSP units while maximizing FPGA memory access bandwidth utilization, their support for mixed-precision calculations, such as int8 and int4 data types, is still insufficient.

[0044] Therefore, this application proposes an acceleration unit that can complete the calculation of 4 sets of vectors by loading matrix data once, improving the efficiency of batch processing calculations in the system, making full use of the calculation bit width of the multiply-accumulate unit, and reducing the hardware wiring overhead of the input port of the multiply-accumulate unit; by forming a group of two columns of multiply-accumulate units, it can simultaneously process the multiplication and addition of one int8 matrix data and two vectors, or reuse the data path to process the multiplication and addition calculation of two int4 matrix data and one vector, reducing the wiring overhead of the FPGA system and improving the system's calculation efficiency; and through a two-level accumulation structure, the system can support matrix-vector multiplication calculations of different scales and batch sizes.

[0045] The acceleration unit provided in the embodiments of this application will be described below first. Figure 1 This is a schematic diagram of an acceleration unit provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0046] Reference Figure 1 The acceleration unit includes:

[0047] Memory 101 is used to store matrix data, and each column of data in the matrix is ​​loaded from the memory into the input data queue in sequence;

[0048] Multiply-accumulate module 102, such as Figure 1 As shown, the multiply-accumulate module 102 includes a multiply-accumulate unit and a two-stage accumulator for performing matrix and vector multiplication and accumulation operations; Figure 2In the embodiment shown, the acceleration unit may further include a controller (not shown). The controller controls the matrix column data in the input data queue to be input into the corresponding multiply-accumulate unit. Different multiply-accumulate units in a multiply-accumulate module input different elements in the same column of data in the matrix in sequence. It should be noted that a column of data in the matrix in the input data queue is input into the first to the Nth multiply-accumulate modules respectively, where N is a positive integer greater than or equal to 2.

[0049] It should also be noted that one input port of the multiplier in the multiply-add module already stores vector data. Therefore, after receiving matrix data, multiply-add operations can be performed directly to complete matrix-vector multiplication and addition calculations.

[0050] Furthermore, the vector data stored at one input port of the multiplier in each multiply-accumulate module is different, that is, N multiply-accumulate modules store N different vectors.

[0051] It should be further explained that a multiply-accumulate column contains multiple multiply-accumulate units. Therefore, the output of the multiply-accumulate unit is connected to a two-stage accumulator to further accumulate the calculation results, thereby completing the multiply-accumulate calculation of the vector.

[0052] Furthermore, all multiply-accumulate units perform multiply-accumulate processing on the data flowing through them simultaneously and in parallel, thereby achieving high parallel processing speed.

[0053] Preferably, in a system-on-a-chip with abundant hardware resources, if there are enough hardware multiply-accumulate units on the chip, the number of multiply-accumulate calculation columns can be increased, and batch calculation of matrix-vector multiplication can be achieved by sharing the input matrix. In other words, multiply-accumulate operations of the same matrix with different vectors can be performed simultaneously.

[0054] Data output unit 103 is used to output the results of matrix and vector multiplication and addition calculations.

[0055] The above provides an overall overview of the entire acceleration unit. The following section details each module within the acceleration unit, starting with a schematic diagram of the connection structure for loading matrix data in the multiply-accumulate unit. (See attached diagram.) Figure 2 , Figure 2 This diagram illustrates a connection structure for loading matrix data into a multiply-accumulate unit, as provided in an embodiment of this application. This connection structure is, for example, a systolic connection structure. Figure 2As shown, the elements of a column of data in the matrix loaded in the input data queue are loaded into different multiply-accumulate units sequentially according to their row order. In reality, to complete the multiplication and addition operation between a column of data in the matrix and a vector, element-wise multiplication and subsequent accumulation are required. Therefore, the number of times the input data corresponding to each multiply-accumulate unit in the same column needs to be registered increases sequentially according to the calculation order. For example, the element in the last row of a column of data, in row order, has the fewest times its input data is registered in the multiply-accumulate unit and is calculated first. In this embodiment, the number of registration cycles corresponds to the multiply-accumulate operation delay of the multiply-accumulate unit. For example, if the multiply-accumulate operation requires three clock cycles to complete, then the number of registration cycles for one registration is three clock cycles. That is, the number of times the input data corresponding to each multiply-accumulate unit in the same column of the matrix data needs to be registered increases sequentially, with each registration requiring multiple clock cycles. The number of clock cycles required for each registration is equal to the multiply-accumulate operation delay of the multiply-accumulate unit.

[0056] Furthermore, in practical matrix and vector multiplication and addition applications, the vectors are generally quite large. In order to improve the computational efficiency of the system, an ultra-long addition tree can be designed to improve the computational efficiency of accumulation.

[0057] The multiply-accumulate module is the core computing unit in a DSP, and a DSP must include at least one multiply-accumulate module. In practical applications, many DSPs are connected to perform matrix and vector multiplication and addition operations. The following section provides a detailed introduction to the accumulation structure of a DSP; see [link to relevant documentation]. Figure 3 . Figure 3 This is a schematic diagram of an accumulation structure provided in an embodiment of this application. In practical applications, multiplication and accumulation calculations require support for multiplying two int8 data. Therefore, the lower-order int8 data within a single DSP unit can use a maximum of 18 bits to complete the accumulation of 16 data (input data greater than or equal to -127 and less than or equal to 127). Therefore, for illustrative purposes, the 16 DSP units in this embodiment are divided into one group; however, this application does not impose a specific limitation on the number of DSP units contained in a group. Figure 3 As shown, within a group of DSPs (DSP0, DSP1, ..., DSPn), the multiply-accumulate sum of DSP0 is passed to the PCIN of DSP1 via DSP0's PCOUT, and so on. The multiply-accumulate sum of DSP(n-1) is passed to the PCIN of DSPn via DSP(n-1)'s PCOUT. Finally, the multiply-accumulate sum of DSPn is output to the first accumulator. In other words, the sum is passed to the next DSP unit for accumulation through the cascaded accumulation path (P(COUT) and P(CIN)) of the DSP unit itself. The accumulated sum of the last DSP unit in the group is output to the first-stage accumulator. Figure 3The connection architecture shown ensures full utilization of the wiring resources provided within the DSP unit, saving configurable wiring resources in the FPGA.

[0058] Furthermore, to improve computational efficiency, in addition to the cascaded accumulation of DSPs, this application employs a two-level addition tree structure for larger-scale partial sum accumulation. After the partial sum accumulation result of each DSP group is output, it is sent to the first-level accumulator for accumulation, which also performs accumulation calculations using a systolic array method. After the first-level accumulation is completed, based on the length of the vector, the partial sums of different columns can be sent to the accumulator for further accumulation. Therefore, Figure 3 The illustrated embodiment includes a second-stage accumulator. The sum of the first-stage accumulator can be fed into the second-stage accumulator for further accumulation, yielding the final accumulated result. In the second-stage accumulator, the accumulation calculation still uses a pulsating array approach, accumulating the partial sums of different columns. To further enhance the system's flexibility, both accumulators are configurable; whether data needs to be accumulated can be determined by configuring different instructions based on the vector length.

[0059] The above describes the connection structure of multiple DSP units. The following section details the internal connection structure of a DSP module containing two DSP units. (See attached image.) Figure 4 . Figure 4 This is a schematic diagram of the internal connection structure of a DSP module provided in an embodiment of this application. For illustrative purposes, this application provides a connection diagram of two DSP units, but it does not impose a specific limitation on the number of cascaded DSPs.

[0060] like Figure 4 As shown, the multiply-accumulate unit is the hard-core computing unit of the DSP in the FPGA. Figure 4The illustrated embodiment includes a first DSP unit and a second DSP unit. In matrix-vector multiplication, the calculation controller pre-places four different vectors in registers A1 and A2 (examples of the third and fourth registers) at port A of the multiply-accumulate unit via the vector registers (V) of the first and second DSP units (example of the first register) before the calculation begins. Since the multiplication unit in the DSP unit can perform 25-bit * 18-bit multiplication, two sets of int8 vectors (this embodiment uses two sets of int8 vectors as an example for illustration, but does not impose a specific limit on the number of stored vector sets) multiplication operations can be completed per clock cycle. At this time, each register (A1 and A2) at port A can store two sets of data; therefore, the two DSP units can store four different vectors. In one DSP unit, one set of data is stored in the high 8 bits of register A, and another set is stored in the low 8 bits of register A. The input data of the matrix is ​​read from memory into register M (example of the matrix data register), such as... Figure 4 As shown, the M register of the first DSP unit is connected to the M register of the second DSP unit, and then the data is sequentially sent to the B registers of the two DSP units (example of the second register). During calculation, the DSP unit reads matrix data from the M register, loads it into the register at port B, and simultaneously calculates A1*B. In the next clock cycle, the data at port B remains unchanged, while the data at port A switches from A1 (example of the third register) to A2 (example of the fourth register), calculating A2*B. After the calculation is complete, new matrix input data is loaded from the M register again into port B, and port A is switched to register A1 to begin a new round of multiplication and addition calculations. Since the DSP units can operate at twice the logic frequency, the vector at port A allows port B to use only one input data to meet the data requirements for frequency multiplication of the DSP units. This structure reduces the hardware routing requirements of the computing units.

[0061] Figure 4 The cascading method of the first and second DSP units shown can be adopted as follows: Figure 3 The cascading methods provided in the illustrated embodiments will not be described in detail here.

[0062] The internal connection structure of the multiply-accumulate unit is described in detail below. (See also...) Figure 5 . Figure 5 This is a schematic diagram of the internal connection structure of a multiply-accumulate unit provided in an embodiment of this application. To simultaneously support the calculation of matrix data of int4 and int8, and to reduce the demand for hardware routing resources, this embodiment of the application employs a grouping and cascading method to improve the connection efficiency of the DSP unit. For example... Figure 5As shown, two DSP units in different columns can form a DSP calculation group. When the input matrix is ​​of data type int8, the two DSP units are cascaded. The input int8 data first enters the left DSP unit (first DSP unit). At this time, the M register on the left sends 8 bits of data (the high 4 bits and the low 4 bits) to the multiply-accumulate unit for calculation. After a delay of several clock cycles, the 8 bits of data in the M register enter the right DSP unit (second DSP unit), and the matrix-vector multiplication calculation of different vectors is completed in the same way. If the input data is of data type int4, the left and right DSP units calculate the low 4 bits and high 4 bits of the 8-bit data respectively. Therefore, when the left DSP unit performs calculation, the low 4 bits of data are sent to the DSP unit through the selector; while when the right DSP unit performs calculation, the high 4 bits of data are sent to the DSP unit through the selector, together completing the matrix-vector multiplication of two int4 data. Therefore, when performing multiply-accumulate operations on int4 data type, the vector data stored in the two DSP units are different parts of the same vector.

[0063] The above embodiments use int8 and int4 data types as examples for illustration. This application does not impose specific restrictions on data types.

[0064] In summary, the acceleration unit provided in this application embodiment can complete the calculation of 4 sets of vectors by loading matrix data once, which improves the efficiency of batch processing calculation in the system, makes full use of the calculation bit width of the multiply-accumulate unit, and reduces the hardware wiring overhead of the input port of the multiply-accumulate unit; by forming a group of two columns of multiply-accumulate units, it can simultaneously process the multiplication and addition of one int8 matrix data and two vectors, or reuse the data path to process the multiplication and addition of two int4 matrix data and one vector, reducing the wiring overhead of the FPGA system and improving the system's calculation efficiency; and through the two-level accumulation structure, the system can support matrix-vector multiplication calculations of different scales and batch sizes.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0066] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0067] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0068] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0069] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0070] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0071] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0072] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0073] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An acceleration unit, characterized in that, The acceleration unit includes: Memory, used to store matrix data; The multiply-accumulate module includes at least two multiply-accumulate units and a two-stage accumulator, which are used to perform multiply-accumulate operations on matrices and vectors. The vector data stored at the input of the multiplier in different multiply-accumulate modules are different. A controller is used to load the matrix data from the memory into the multiply-accumulate units of the multiply-accumulate module in the column order of the matrix. The data output unit is used to output the results of matrix and vector multiplication and addition operations; Each of the at least two multiply-accumulate units includes at least two digital signal processing units. Each of the at least two digital signal processing units includes: A first register, a third register, and a fourth register are used to store vector data, wherein the first register is connected to the third register and the fourth register; and A matrix data register and a second register are provided. The matrix data register stores matrix data read from the matrix's memory, and the second register reads the matrix data from the matrix data register for multiplication and addition calculations. The matrix data registers of the at least two digital signal processing units are interconnected; When the matrix data in the at least two digital signal processing units are the same, the matrix data of the first digital signal processing unit is delayed before entering the second digital signal processing unit; When the matrix data in the at least two digital signal processing units are different, the matrix data of the first digital signal processing unit is the low-order data in a column of data, and the matrix data of the second digital signal processing unit is the high-order data in the column of data. The first register of the first digital signal processing unit and the first register of the second digital signal processing unit store different portions of the same vector data.

2. The acceleration unit according to claim 1, characterized in that, The elements in the same column of data in the matrix are sequentially input into different multiply-accumulate units in the same column of a multiply-accumulate module.

3. The acceleration unit according to claim 2, characterized in that, The data in the same column of the matrix is ​​input into the corresponding multiply-accumulate unit after different rows are delayed by different periods. The different delay periods are determined based on the delay of the multiplication and addition operations of the matrix and vector.

4. The acceleration unit according to claim 1, characterized in that, Multiple acceleration units are cascaded into a group, and the result of the multiplication and addition operation of one acceleration unit in the group is used as the input of another acceleration unit; The final multiply-accumulate result of a set of acceleration units is input into the accumulator.

5. The acceleration unit according to claim 1, characterized in that, In one of the at least two digital signal processing units, the third register and the fourth register store different vector data; The multiplication and addition operation between the vector data in the third register and the matrix data in the second register is calculated within one clock cycle; In the next clock cycle, perform a multiplication and addition operation between the vector data in the fourth register and the matrix data in the second register.

6. A system-on-a-chip, characterized in that, The system-on-chip includes an acceleration unit as described in any one of claims 1-5, for performing matrix and vector multiplication and addition operations.

Images