Low-complexity matrix multiplication calculation method and calculation architecture

Abstract

The invention discloses a low-complexity matrix multiplication calculation method and a calculation framework, and the method comprises the steps: taking a matrix with a relatively fixed value in two matrixes participating in multiplication as a weight matrix, taking a matrix with a relatively variable value as a signal matrix, taking each element of the matrix in matrix multiplication as a discrete signal sample, and taking the discrete signal sample as a signal matrix; each row of the first matrix and each column of the second matrix are respectively regarded as a time sequence of a discrete signal, up-sampling with the same multiple is carried out on the time sequence, a weight matrix after up-sampling is compressed and coded, then the weight matrix after compressed and coded and a signal matrix after up-sampling are subjected to multiplication operation, and a signal matrix after up-sampling is obtained; the obtained product is approximately equal to the product of the two original matrixes. According to the method, data can be compressed to a great extent, ultra-low space occupation is realized, meanwhile, high-efficiency data reading speed is ensured, and most of operation precision is still kept under the condition of limited calculation cost.

Description

Technical Field

[0001] The present invention relates to the field of numerical calculation technology of large models in the field of artificial intelligence and large-scale antennas in the field of communications, and in particular to a low-complexity matrix multiplication calculation method and calculation architecture. Background Art

[0002] In recent years, with the rise of a series of GPT large models and technologies such as large-scale antennas, large-scale matrix multiplication has become a widely used mathematical tool, and the data storage and computing overhead involved have also increased dramatically. Therefore, a series of matrix compression and simplification algorithms have been proposed, which can not only reduce the storage overhead of the matrix through quantization, but also reduce the computational overhead of matrix multiplication by using additive operations instead of multiplication. In terms of computing architecture, due to the exponential growth of matrix size, large-scale data movement has brought about, and the computing architecture has increased the requirements for storage bandwidth. Therefore, a storage-computing integrated architecture has emerged. During the chip design process, the storage unit and the computing unit are integrated to increase the data reading bandwidth, reduce the data movement distance, and read directly from the storage unit for calculation, which can improve the system energy efficiency. However, the existing matrix compression and simplification algorithms and their architectures have the problem of low efficiency.

[0003] The existing matrix compression quantization scheme decomposes the matrix into two parts: the product of a binary quantization matrix and a high-precision vector. The elements of the binary quantization matrix have only two values, +1 and -1, while the elements of the high-precision vector are multi-bit quantized. The corresponding matrix multiplication is also decomposed into two steps, namely, binary quantization matrix multiplication and high-precision vector multiplication. The binary quantization matrix multiplication only needs to be multiplied with +1 and -1, so it can be replaced by an additive operation. Although vector multiplication reduces the number of multiplication operations compared to matrix multiplication, it still requires high-precision multiplication operations, and the increase in operation steps requires more data movement, so the algorithm complexity needs to be further reduced. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present invention proposes a low-complexity matrix multiplication calculation method, which reduces the storage overhead of the matrix by compressing and encoding a matrix in the matrix multiplication. When the compression coding adopts binary or ternary quantization, additive operations can also be used to replace multiplication, thereby reducing the computational overhead of matrix multiplication and still retaining most of the calculation accuracy.

[0006] Another object of the present invention is to propose a computational architecture for low-complexity matrix multiplication.

[0007] To achieve the above object, the present invention provides a low-complexity matrix multiplication calculation method, which comprises:

[0008] The matrix with relatively fixed values ​​among the two matrices involved in the multiplication is used as the weight matrix, and the matrix with relatively variable values ​​is used as the signal matrix;

[0009] Each element of the matrix in the matrix multiplication is regarded as a discrete signal sample to perform the first upsampling or the second upsampling; wherein,

[0010] The first upsampling includes upsampling the weight matrix row by row and upsampling the signal matrix column by column; or upsampling the signal matrix row by row and upsampling the weight matrix column by column;

[0011] The second upsampling includes removing outliers from the weight matrix to split it into an outlier weight matrix and a normal value weight matrix; wherein the normal value weight matrix is ​​upsampled row by row and the signal matrix is ​​upsampled column by column; or the signal matrix is ​​upsampled row by row and the normal value weight matrix is ​​upsampled column by column;

[0012] After the first upsampling, the upsampled weight matrix is ​​compressed and encoded, and then the compressed and encoded weight matrix and the upsampled signal matrix are multiplied;

[0013] Or, after performing the second upsampling, the product of the outlier weight matrix and the signal matrix is ​​calculated to obtain a first product result, the upsampled normal value weight matrix is ​​compressed and encoded, and then multiplied with the upsampled signal matrix to obtain a second product result; the first product result is added to the second product result.

[0014] The low-complexity matrix multiplication calculation method of the embodiment of the present invention may also have the following additional technical features:

[0015] In one embodiment of the present invention, the weight matrix is ​​scaled before upsampling. Assuming that the weight matrix The scaled weight matrix The calculation formula is:

[0016]

[0017] The scaling factor s is set as:

[0018]

[0019] Among them, σ is the standard deviation of the weight matrix elements, and μ is the mean value of the weight matrix elements.

[0020] In one embodiment of the present invention, the up-sampling includes a method of using a low-pass filter, a method of performing discrete Fourier transform interpolation in the frequency domain, and a method of an analog circuit; wherein,

[0021] The low-pass filtering method: Assuming that the upsampling rate is L, insert L-1 zero-value samples between two adjacent sampling points of the original sequence, and filter out the frequency components outside the bandwidth of the original signal through low-pass filtering to obtain the up-sampled sequence;

[0022] The method for performing discrete Fourier transform interpolation in the frequency domain: assuming that the length of the original sequence is M and the upsampling rate is L, the original sequence is transformed into the frequency domain through M-point Fourier transform, (L-1)×M zeros are inserted in the high frequency band, and then transformed back to the time domain through L×M-point inverse Fourier transform to obtain a sequence of L×M points after upsampling;

[0023] The method of the analog circuit is as follows: firstly, the original sequence is converted into a continuous-time analog signal, and then the continuous-time analog signal is sampled at a higher sampling rate to obtain an up-sampled sequence.

[0024] In one embodiment of the present invention, the compression coding includes: using sigma-delta coding; wherein,

[0025] If the signal matrix is ​​multiplied by the weight matrix, sigma-delta encoding is performed on each column of the weight matrix, and each encoded sequence is combined column by column to obtain the compressed encoded weight matrix;

[0026] If the weight matrix is ​​multiplied by the signal matrix, sigma-delta encoding is performed on each row of the weight matrix, and each encoded sequence is combined row by row in turn to obtain the compressed encoded weight matrix.

[0027] In one embodiment of the present invention, the sigma-delta encoding includes first-order binary quantization, first-order ternary quantization, and second-order binary quantization; wherein,

[0028] The first-order binary quantization: Assume that the input sequence is The encoded sequence is Define register variable s, the initial value of s is 0, and each input u k , k=1,2,…M, then perform the following steps: integral s=s+u k ; Incremental encoding Calculate the difference s = sq k ;

[0029] The first-order ternary quantization: Assume that the input sequence is The encoded sequence is Define register variable s, the initial value of s is 0, and each input u k , k=1,2,…M, then perform the following steps: integral s=s+u k ; Incremental encoding Calculate the difference s = sq k;

[0030] The second-order binary quantization: Assume that the input sequence is The encoded sequence is Define two register variables as s n and 1 ,s 0 and 1 The initial value is 0. k , k=1,2,…M, then perform the following steps: Integral s 0 =s 0 +u k ,s 1 =s 1 +u k ; Incremental encoding Calculate the difference s 0 =s 0 -q k ;s 1 =s 1 +s 0 -q k .

[0031] In one embodiment of the present invention, the incremental coding in the sigma-delta coding is binary quantization or ternary quantization, wherein:

[0032] The binary quantization defines the incremental coding as two levels, using 1-bit quantization, with only two values ​​of +1 and -1. When the incremental is positive, +1 is output, and when the incremental is negative, -1 is output.

[0033] The ternary quantization defines the incremental coding as three levels, with three values ​​of +1, 0, and -1. When the increment exceeds 1 / 3, +1 is output, when the increment is less than -1 / 3, -1 is output, and when the increment is between -1 / 3 and 1 / 3, 0 is output.

[0034] In one embodiment of the present invention, it is assumed that matrix multiplication is represented by Y=X·W, where X=[x k,m ] K×M is a K×M-dimensional signal matrix, W = [w m,n ] M×N represents an M×N dimensional weight matrix, Y=[y k,n ] K×N is a K×N dimensional signal matrix;

[0035] The maximum absolute value of the normal value is set to T, that is, the lower boundary and the upper boundary are -T and T respectively, and the normal value weight matrix is ​​recorded as The outlier weight matrix is ​​denoted as but:

[0036]

[0037] In one embodiment of the present invention, the normal value weight matrix is ​​scaled element by element:

[0038]

[0039] Get the scaled weight matrix

[0040] Scaling factor s:

[0041]

[0042] but:

[0043]

[0044] In one embodiment of the present invention, based on Y=X·W, it is decomposed into multiplication of each row of X and each column of W, and the scaled weight matrix Each column of is upsampled, the upsampling rate is L, and the output matrix after upsampling It is L×M rows and N columns;

[0045] by Take any column of as an example, denoted as right Perform Fourier transform at M points, then:

[0046]

[0047] Insert (L-1)*M zeros in the high frequency band of v(n):

[0048]

[0049] Then perform an inverse Fourier transform on v(n) to obtain the output upsampling sequence

[0050]

[0051] Each upsampling sequence is Combine columns to get the upsampling weight matrix

[0052] In one embodiment of the present invention, the up-sampled weight matrix Each column of Take any column of as an example, denoted as Using first-order binary quantization coding, the initial value of register variable s is set to 0, and the sequence after sigma-delta coding is

[0053] s=s+u k

[0054]

[0055] s=sq k

[0056] Where k = 1, 2…LM;

[0057] Each sequence in turn Combine by columns to get the processed weight matrix

[0058] In one embodiment of the present invention, each row of X is upsampled, and the upsampling rate is also L. After upsampling, the output matrix It is K rows and L columns;

[0059] Using low-pass filtering method:

[0060] Take any row of X as an example, recorded as Assume that the upsampling rate L = 2, insert a zero-value sample between two adjacent sampling points.

[0061] Assume that the impulse response of the half-band filter is h(n), n=0,1,2…Q-1, Q is the order of the filter, and the filtering formula is:

[0062]

[0063] Sequentially transform the filtered sequence [u k ] 1×LM Combine rows to get the upsampled signal matrix

[0064] In one embodiment of the present invention, the matrix product is calculated in is the weight matrix of compressed quantization, whose values ​​are limited to -1 and +1, and the multiplexer is used according to Select an addend to accumulate the value.

[0065] In one embodiment of the present invention, the final result of adding the first product result and the second product result is corrected: Among them, the outlier signal matrix is a sparse matrix.

[0066] To achieve the above-mentioned purpose, a second embodiment of the present invention provides a computing architecture of low-complexity matrix multiplication, wherein the computing architecture is composed of a first plurality of functional units or a second plurality of functional units, wherein:

[0067] The first plurality of functional units include an upsampling unit, a weight storage unit, a calculation unit, and a result storage unit;

[0068] Or, the second plurality of functional units include an upsampling unit, a weight storage unit, a calculation unit, a result storage unit and an outlier processing unit;

[0069] In the computing architecture, multiple functional units are integrated using one die, or multiple chips or multiple dies.

[0070] A low-complexity matrix multiplication calculation method and computing architecture of an embodiment of the present invention can compress data to a great extent, achieve ultra-low space occupancy, while ensuring efficient data reading speed, and still retain most of the calculation accuracy under limited computing costs.

[0071] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0073] Figure 1 A flow chart of a calculation method of low-complexity matrix multiplication according to the present invention;

[0074] Figure 2 A flowchart of a low-complexity matrix multiplication method including outlier correction according to the present invention;

[0075] Figure 3 is a structural diagram of a matrix multiplication operation unit according to the present invention;

[0076] Figure 4 A structural diagram of a computing architecture of a low-complexity matrix multiplication according to the present invention;

[0077] Figure 5 A structural diagram of using parallel addition trees to calculate matrix multiplication according to the present invention;

[0078] Figure 6 It is a structural diagram of using a capacitor integration circuit to calculate matrix multiplication according to the present invention. DETAILED DESCRIPTION

[0079] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0080] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0081] The following describes a low-complexity matrix multiplication calculation method and a calculation architecture proposed according to an embodiment of the present invention with reference to the accompanying drawings.

[0082] Figure 1 and Figure 2 A low-complexity matrix multiplication calculation method according to an embodiment of the present invention comprises:

[0083] The matrix with relatively fixed values ​​among the two matrices involved in the multiplication is used as the weight matrix, and the matrix with relatively variable values ​​is used as the signal matrix;

[0084] Each element of the matrix in the matrix multiplication is regarded as a discrete signal sample to perform the first upsampling or the second upsampling; wherein,

[0085] The first upsampling includes upsampling the weight matrix row by row and upsampling the signal matrix column by column; or upsampling the signal matrix row by row and upsampling the weight matrix column by column;

[0086] The second upsampling includes removing outliers from the weight matrix to split it into an outlier weight matrix and a normal value weight matrix; wherein the normal value weight matrix is ​​upsampled row by row and the signal matrix is ​​upsampled column by column; or the signal matrix is ​​upsampled row by row and the normal value weight matrix is ​​upsampled column by column;

[0087] After the first upsampling, the upsampled weight matrix is ​​compressed and encoded, and then the compressed and encoded weight matrix and the upsampled signal matrix are multiplied;

[0088] Or, after performing the second upsampling, the product of the outlier weight matrix and the signal matrix is ​​calculated to obtain a first product result, the upsampled normal value weight matrix is ​​compressed and encoded, and then multiplied with the upsampled signal matrix to obtain a second product result; the first product result is added to the second product result.

[0089] In one embodiment of the present invention, the weight matrix is ​​scaled before upsampling, in order to adjust the signal to a suitable amplitude, suitable for the quantization interval of subsequent compression coding. Assume that the weight matrix W = [w m,n ] M×N , the scaled weight matrix The calculation formula is:

[0090]

[0091] The scaling factor s can be set as:

[0092]

[0093] σ is the standard deviation of the weight matrix elements, and μ is the mean value of the weight matrix elements.

[0094] In one embodiment of the present invention, up-sampling can be performed using a low-pass filtering method. Assuming that the up-sampling rate is L, L-1 zero-value samples are inserted between two adjacent sampling points of the original sequence, and the frequency components outside the bandwidth of the original signal are filtered out by low-pass filtering to obtain the up-sampled sequence. In particular, when L=2, the low-pass filtering can use a half-band filter, whose impulse response is not zero except for the zero point, while the other even points are all zero, requiring only half the amount of calculation, which is particularly suitable for real-time processing.

[0095] In one embodiment of the present invention, upsampling can be performed by using a discrete Fourier transform interpolation method in the frequency domain. Assuming that the original sequence length is M and the upsampling rate is L, the original sequence is transformed to the frequency domain through M-point Fourier transform, (L-1)×M zeros are inserted in the high frequency band, and then transformed back to the time domain through L×M-point inverse Fourier transform to obtain a sequence of L×M points after upsampling.

[0096] In one embodiment of the present invention, upsampling can be implemented using an analog circuit, where the original sequence is first converted into a continuous-time analog signal, and then the continuous-time analog signal is sampled at a higher sampling rate to obtain an upsampled sequence.

[0097] In one embodiment of the present invention, the weight matrix is ​​compressed and sigma-delta coding can be used. If the signal matrix is ​​multiplied by the weight matrix, sigma-delta coding is performed on each column of the weight matrix, and each encoded sequence is combined by column in turn to obtain a compressed and coded weight matrix; if the weight matrix is ​​multiplied by the signal matrix, sigma-delta coding is performed on each row of the weight matrix, and each encoded sequence is combined by row in turn to obtain a compressed and coded weight matrix.

[0098] In one embodiment of the present invention, sigma-delta coding may use first-order binary quantization, specifically including assuming that the input sequence is The encoded sequence is Define register variable s, the initial value of s is 0, and each input u k , k=1,2,…M, then perform the following steps:

[0099] Integration: s = s + uk ,

[0100] Incremental encoding:

[0101] Calculate the difference: s = sq k .

[0102] In one embodiment of the present invention, sigma-delta coding may use first-order ternary quantization, specifically including assuming that the input sequence is The encoded sequence is Define register variable s, the initial value of s is 0, and each input u k , k=1,2,…M, then perform the following steps:

[0103] Integration: s = s + u k ,

[0104] Incremental encoding:

[0105] Calculate the difference: s = sq k .

[0106] In one embodiment of the present invention, sigma-delta coding may use second-order binary quantization, specifically including assuming that the input sequence is The encoded sequence is Define two register variables as s 0 and 1 ,s 0 and 1 The initial value is 0. k , k=1,2,…M, then perform the following steps:

[0107] Points: s 0 =s 0 +u k ,s 1 =s 1 +u k

[0108] Incremental encoding:

[0109] Calculate the difference:

[0110] s 0 =s 0 -q k

[0111] s 1 =s 1 +s 0 -q k

[0112] In one embodiment of the present invention, the incremental coding in the sigma-delta coding can be binary quantization. Binary quantization defines the incremental coding as two levels, using 1-bit quantization, with only two values ​​of +1 and -1. When the increment is positive, "+1" is output, and when the increment is negative, "-1" is output.

[0113] In one embodiment of the present invention, the incremental coding in the sigma-delta coding can be ternary quantization. Ternary quantization defines the incremental coding as three levels, with three values ​​of +1, 0, and -1. When the increment exceeds 1 / 3, "+1" is output, when the increment is less than -1 / 3, "-1" is output, and when the increment is between -1 / 3 and 1 / 3, "0" is output.

[0114] In the calculation method of low-complexity matrix multiplication of an embodiment of the present invention, the row vectors of the signal matrix are upsampled to make the signal smoother and suppress the mutation components therein, so that the signal is concentrated in the low-frequency part and the high-frequency part of the signal is zero, the column vectors of the weight matrix are upsampled and then sigma-delta encoding is performed, the quantization error is concentrated in the high-frequency part, and the new signal matrix and the weight matrix are multiplied to eliminate the quantization error of the high-frequency part, so the result is approximately equal to the product of the original matrices.

[0115] It is understandable that the present embodiment can improve the calculation accuracy by adding a separate treatment of the outliers of the weight matrix, specifically including removing the outliers of the weight matrix and correcting the results for outliers. The specific calculation process is to remove the outliers from the weight matrix, split it into an outlier weight matrix and a normal value weight matrix, calculate the product of the outlier and the signal matrix separately, upsample and compress the normal value weight matrix, and then multiply it with the upsampled signal matrix. The obtained matrix product is added to the product of the outlier and the signal matrix, so that the result is closer to the product of the original two matrices. An outlier is an abnormal value in the data distribution that is significantly different from other data. The lower boundary and the upper boundary can be set. When the data is less than the lower boundary or greater than the upper boundary, it is judged as an outlier.

[0116] It can be known that the outlier correction of the matrix product result is to record the outliers and their locations, calculate the product of these outliers and the weight matrix separately, and include them in the final matrix product result. Matrix product The advantage of splitting the calculation into two parts is that It can be calculated by low-complexity matrix multiplication based on upsampling and compression coding. Although the outliers are large in magnitude, their number is very small, and the matrix composed of is a sparse matrix, Only a few multiplications are actually required.

[0117] Specifically, the calculation process of the low-complexity matrix multiplication calculation method of the embodiment of the present invention is described in detail through the accompanying drawings.

[0118] Assume that matrix multiplication can be expressed as Y = X·W, where X = [x k,m ] K×M is a K×M-dimensional signal matrix, W = [w m,n ] M×N represents an M×N dimensional weight matrix, Y=[y k,n ] K×N is a K×N dimensional signal matrix.

[0119] 1) Processing of weight matrix:

[0120] Remove outliers:

[0121] The maximum absolute value of the normal value is set to T, that is, the lower boundary and the upper boundary are -T and T respectively, and the normal value weight matrix is ​​recorded as The outlier weight matrix is ​​denoted as but:

[0122]

[0123] Scale the normal value weight matrix element-wise:

[0124]

[0125] Get the scaled weight matrix

[0126] Scaling factor s:

[0127]

[0128] Where σ is the standard deviation of the weight matrix elements and μ is the mean of the weight matrix elements.

[0129]

[0130] Upsampling:

[0131] Since Y = X·W, it can be decomposed into the multiplication of each row of X and each column of W, so for the scaled weight matrix Each column of is upsampled, the upsampling rate is L, and the output matrix after upsampling It is L×M rows and N columns.

[0132] by Take any column of as an example, denoted as right Perform Fourier transform at M points, then

[0133]

[0134] Insert (L-1)*M zeros in the high frequency band of v(n),

[0135]

[0136] Then perform an inverse Fourier transform on v(n) to obtain the output upsampling sequence

[0137]

[0138] Each upsampling sequence is Combine columns to get the upsampling weight matrix

[0139] Sigma-delta encoding: weight matrix for upsampling Each column of Take any column of as an example, denoted as Using first-order binary quantization coding, the initial value of register variable s is set to 0, and the sequence after sigma-delta coding is

[0140] s=s+u k

[0141]

[0142] s=sq k

[0143] Where k = 1, 2…LM.

[0144] Each sequence in turn Combine by columns to get the processed weight matrix

[0145] 2) Processing of signal matrix

[0146] Upsample the signal matrix:

[0147] Since Y = X·W, it can be decomposed into the multiplication of each row of X and each column of W, so each row of X is upsampled, and the upsampling rate is also L. After upsampling, the output matrix It is K rows and L columns.

[0148] Upsampling can be performed using a low-pass filtering method. When the upsampling rate is 2 or a power of 2, a half-band filter is more suitable. Its impulse response is not zero except for the zero point, and all other even-numbered points are zero. It only requires half the amount of calculation and is particularly suitable for real-time processing.

[0149] Take any row of X as an example, recorded as Assume that the upsampling rate L = 2, insert a zero-value sample between two adjacent sampling points.

[0150] Assume that the impulse response of the half-band filter is h(n), n=0,1,2…Q-1, Q is the order of the filter, and the filtering formula is:

[0151]

[0152] Sequentially transform the filtered sequence [u k ] 1×LM Combine rows to get the upsampled signal matrix

[0153] Compute matrix products in is the weight matrix of compressed quantization, and its values ​​are limited to -1 and +1 (ternary quantization values ​​are also limited to -1, +1 and 0). The multiplexer can be used according to The value of selects an addend to accumulate, thereby replacing the multiplication operation with the addition operation. The calculation process is as follows Figure 3 shown.

[0154] Correct the final output The outlier signal matrix is a sparse matrix, Only a few multiplications are actually required.

[0155] According to the low-complexity matrix multiplication calculation method of an embodiment of the present invention, the storage overhead of the matrix is ​​reduced by compressing and encoding a matrix in the matrix multiplication. When the compression encoding adopts binary or ternary quantization, additive operations can also be used instead of multiplication to reduce the computational overhead of the matrix multiplication while still retaining most of the calculation accuracy.

[0156] In order to implement the above embodiment, Figure 4 As shown, this embodiment also provides a computing architecture of low-complexity matrix multiplication, the computing architecture is composed of a first plurality of functional units or a second plurality of functional units, wherein:

[0157] The first plurality of functional units include an upsampling unit, a weight storage unit, a calculation unit, and a result storage unit;

[0158] Or, the second plurality of functional units include an upsampling unit, a weight storage unit, a calculation unit, a result storage unit and an outlier processing unit;

[0159] In the computing architecture, multiple functional units are integrated using one die, or multiple chips or multiple dies.

[0160] It is understandable that the computing architecture of the low-complexity matrix multiplication can use digital circuits or analog circuits. The entire chip can use only one die (Die, also known as bare chip, crystal grain) to complete upsampling, storage and computing functions at the same time, or it can include multiple dies, which are divided into upsampling die, storage die and computing die according to function, and multiple dies are stacked together, wherein upsampling can also use independent chips. The weight matrix after upsampling and compression coding is stored in the chip, and the signal matrix is ​​upsampled when calculating matrix multiplication, and the weight matrix is ​​sequentially read out from the chip, and the matrix multiplication is performed, and the product result is output. By shortening the storage and computing distance of data, and reading in sequence in the process of matrix multiplication, the data reading bandwidth can be improved.

[0161] Specifically, the entire chip of the computing architecture of the embodiment of the present invention is integrated with multiple tube cores, including an upsampling tube core of the input signal and a structure formed by stacking three layers of tube cores, wherein:

[0162] The weight storage unit of the first layer of tube core is used to store the weight matrix after upsampling and compression coding;

[0163] The arithmetic unit of the second-layer tube core is used to read out the weight matrix and the upsampled signal matrix from the chip in sequence to perform matrix multiplication when calculating matrix multiplication;

[0164] The result storage unit of the third layer of tube core is used to store the calculation results;

[0165] Two adjacent tube cores are connected and supported by a transfer structure.

[0166] The entire chip of the matrix multiplication computing architecture of the embodiment of the present invention can be integrated with multiple tube cores, including an upsampling tube core for the input signal and a structure composed of three layers of tube cores stacked together, specifically including a storage unit of the first layer of tube cores for storing weight matrices, an operation unit of the second layer of tube cores responsible for calculating matrix products, and a storage unit of the third layer of tube cores for storing calculation results, and two adjacent tube cores are connected and supported by a switching structure.

[0167] In one embodiment of the present invention, a FLASH die can be used to store the compressed and encoded weight matrix, and the operation unit die and the FLASH die use a vertical interconnection channel to achieve high-speed data reading. Wherein FLASH uses a NAND chip, and its read and write operations are in pages. The read and write operations must start from the starting position of the block (512 bytes), thus limiting the random access speed, but the sequential reading speed is fast, and the storage density is very high. In this architecture, only the weight matrix needs to be read sequentially, which can give full play to the advantages of NAND FLASH.

[0168] In one embodiment of the present invention, the computing unit of the second-layer tube core can use a parallel addition tree to complete the matrix multiplication operation. If the weight adopts binary quantization, the signal or the complement of the signal is selected according to the weight value. If the weight adopts ternary quantization, zero or the signal or the complement of the signal is selected according to the weight value. The selected result is input into the adder. Multiple adders calculate simultaneously. Using additive operations instead of multiplication can reduce the complexity of the chip.

[0169] Specifically, a specific implementation of an operation die using a parallel addition tree circuit is as follows: Figure 5 As shown, each upsampled signal enters a selector and selects the output according to the corresponding weight. If the weight is 1, the output is +x k,1 If the weight is -1, then output -x k,1 If the weight is 0, output 0, sum the outputs of the selectors, and multiple adders calculate in parallel to output

[0170] In one embodiment of the present invention, the computing unit of the second layer of tube core can use an analog circuit including capacitor integration to complete matrix multiplication operations, read the weight matrix in sequence, and control the integration circuit to perform positive integration, negative integration or no integration according to the weight matrix. The size of the elements of the signal matrix determines the amount of integration. The use of analog circuits can reduce the size of the circuit, reduce power consumption, improve performance, and facilitate manufacturing and use.

[0171] Specifically, a specific implementation of the operation tube core using the capacitance integration circuit is as follows: Figure 6 As shown, the first step is to disconnect the S3 switch, according to the weight w 1,n Select the position of switches S1 and S2. If the weight is 1, switches S1 and S2 are both down, that is, S2 is grounded, S1 is connected to the signal, and the charge of capacitor C1 is positively integrated +x k,1 If the weight is -1, switches S1 and S2 are both on, and the charge of capacitor C1 is negatively integrated -x k,1 , if the weight is 0, switch S1 is on the top and S2 is on the bottom, that is, S1 and S2 are both grounded, and the charge of capacitor C1 is reset to zero; the second step is that S1 is in the middle and S2 is on the bottom, and the S3 switch is closed, that is, capacitor C1 is connected to C0, and the charge of capacitor C0 is equal to the sum of the charges of multiple C1s, and the output

[0172] The computing architecture of low-complexity matrix multiplication according to the embodiment of the present invention is more suitable for adopting a structure in which storage unit dies and computing unit dies are stacked, which simplifies chip design and improves data reading bandwidth.

[0173] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0174] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. A low-complexity matrix multiplication calculation method, characterized in that: The method comprises: The matrix with relatively fixed values ​​among the two matrices involved in the multiplication is used as the weight matrix, and the matrix with relatively variable values ​​is used as the signal matrix; Each element of the matrix in the matrix multiplication is regarded as a discrete signal sample to perform the first upsampling or the second upsampling; wherein, The first upsampling includes upsampling the weight matrix row by row and upsampling the signal matrix column by column; or upsampling the signal matrix row by row and upsampling the weight matrix column by column; The second upsampling includes removing outliers from the weight matrix to split it into an outlier weight matrix and a normal value weight matrix; wherein the normal value weight matrix is ​​upsampled row by row and the signal matrix is ​​upsampled column by column; or the signal matrix is ​​upsampled row by row and the normal value weight matrix is ​​upsampled column by column; After the first upsampling, the upsampled weight matrix is ​​compressed and encoded, and then the compressed and encoded weight matrix and the upsampled signal matrix are multiplied; Or, after performing the second upsampling, the product of the outlier weight matrix and the signal matrix is ​​calculated to obtain a first product result, the upsampled normal value weight matrix is ​​compressed and encoded, and then multiplied with the upsampled signal matrix to obtain a second product result; the first product result is added to the second product result.

2. The method according to claim 1, characterized in that The weight matrix is ​​scaled before upsampling, assuming that the weight matrix W = [w m,n ] M×N , the scaled weight matrix The calculation formula is: The scaling factor s is set as: Among them, σ is the standard deviation of the weight matrix elements, and μ is the mean value of the weight matrix elements.

3. The method according to claim 1, characterized in that The up-sampling includes a low-pass filtering method, a discrete Fourier transform interpolation method in the frequency domain, and an analog circuit method; wherein, The low-pass filtering method: Assuming that the upsampling rate is L, insert L-1 zero-value samples between two adjacent sampling points of the original sequence, and filter out the frequency components outside the bandwidth of the original signal through low-pass filtering to obtain the up-sampled sequence; The method for performing discrete Fourier transform interpolation in the frequency domain: assuming that the length of the original sequence is M and the upsampling rate is L, the original sequence is transformed into the frequency domain through M-point Fourier transform, (L-1)×M zeros are inserted in the high frequency band, and then transformed back to the time domain through L×M-point inverse Fourier transform to obtain a sequence of L×M points after upsampling; The method of the analog circuit is as follows: firstly, the original sequence is converted into a continuous-time analog signal, and then the continuous-time analog signal is sampled at a higher sampling rate to obtain an up-sampled sequence.

4. The method according to claim 1, characterized in that: The compression coding includes: using sigma-delta coding; wherein, If the signal matrix is ​​multiplied by the weight matrix, sigma-delta encoding is performed on each column of the weight matrix, and each encoded sequence is combined column by column to obtain the compressed encoded weight matrix; If the weight matrix is ​​multiplied by the signal matrix, sigma-delta encoding is performed on each row of the weight matrix, and each encoded sequence is combined row by row in turn to obtain the compressed encoded weight matrix.

5. The method according to claim 4, characterized in that The sigma-delta coding includes first-order binary quantization, first-order ternary quantization, and second-order binary quantization; wherein, The first-order binary quantization: Assume that the input sequence is The encoded sequence is Define register variable s, the initial value of s is 0, and each input u k , k=1,2,…M, then perform the following steps: integral s=s+u k ; Incremental encoding Calculate the difference s = sq k ; The first-order ternary quantization: Assume that the input sequence is The encoded sequence is Define register variable s, the initial value of s is 0, and each input u k , k=1,2,…M, then perform the following steps: integral s=s+u k ; Incremental encoding Calculate the difference s = sq k ; The second-order binary quantization: Assume that the input sequence is The encoded sequence is Define two register variables s0 and s1. The initial values ​​of s0 and s1 are both 0. k , k=1,2,…M, then perform the following steps: integral s0=s0+u k , s1=s1+u k ; Incremental encoding Calculate the difference s0 = s0 - q k ; s1=s1+s0-q k .

6. The method according to claim 5, characterized in that The incremental coding in the sigma-delta coding is binary quantization or ternary quantization, wherein: The binary quantization defines the incremental coding as two levels, using 1-bit quantization, with only two values ​​of +1 and -1. When the incremental is positive, +1 is output, and when the incremental is negative, -1 is output. The ternary quantization defines the incremental coding as three levels, with three values ​​of +1, 0, and -1. When the increment exceeds 1 / 3, +1 is output, when the increment is less than -1 / 3, -1 is output, and when the increment is between -1 / 3 and 1 / 3, 0 is output.

7. The method according to claim 1, characterized in that Assume that matrix multiplication is represented by Y = X·W, where X = [x k,m ] K×M is a K×M-dimensional signal matrix, W = [w m,n ] M×N represents an M×N dimensional weight matrix, Y=[y k,n ] K×N is a K×N dimensional signal matrix; The maximum absolute value of the normal value is set to T, that is, the lower boundary and the upper boundary are -T and T respectively, and the normal value weight matrix is ​​recorded as The outlier weight matrix is ​​denoted as but:

8. The method according to claim 7, characterized in that Scale the normal value weight matrix element by element: Get the scaled weight matrix Scaling factor s: but:

9. The method according to claim 8, characterized in that Based on Y = X·W, it is decomposed into the multiplication of each row of X and each column of W, and the scaled weight matrix Each column of is upsampled, the upsampling rate is L, and the output matrix after upsampling It is L×M rows and N columns; by Take any column of as an example, denoted as right Perform Fourier transform at M points, then: Insert (L-1)*M zeros in the high frequency band of v(n): Then perform an inverse Fourier transform on v(n) to obtain the output upsampling sequence Each upsampling sequence is Combine columns to get the upsampling weight matrix 10. The method according to claim 9, characterized in that The up-sampled weight matrix Each column of Take any column of as an example, denoted as Using first-order binary quantization coding, the initial value of register variable s is set to 0, and the sequence after sigma-delta coding is s=s+u k s=s-q k Where k = 1, 2…LM; Each sequence in turn Combine by columns to get the processed weight matrix 11. The method according to claim 10, characterized in that Upsample each row of X, the upsampling rate is also L, and the output matrix after upsampling is It is K rows and L columns; Using low-pass filtering method: Take any row of X as an example, recorded as Assume that the upsampling rate L = 2, insert a zero-value sample between two adjacent sampling points. Assume that the impulse response of the half-band filter is h(n), n=0,1,2…Q-1, Q is the order of the filter, and the filtering formula is: Sequentially transform the filtered sequence [u k ] 1×LM Combine rows to get the upsampled signal matrix 12. The method according to claim 11, characterized in that Compute matrix products in is the weight matrix of compressed quantization, whose values ​​are limited to -1 and +1, and the multiplexer is used according to Select an addend to accumulate the value.

13. The method according to claim 12, characterized in that Correct the final result of adding the first product result to the second product result: Among them, the outlier signal matrix is a sparse matrix.

14. A computational architecture for low-complexity matrix multiplication, characterized in that: The computing architecture is composed of a first plurality of functional units or a second plurality of functional units, wherein: The first plurality of functional units include an upsampling unit, a weight storage unit, a calculation unit, and a result storage unit; Or, the second plurality of functional units include an upsampling unit, a weight storage unit, a calculation unit, a result storage unit and an outlier processing unit; In the computing architecture, multiple functional units are integrated using one die, or multiple chips or multiple dies.

15. The computing architecture of low-complexity matrix multiplication according to claim 14, characterized in that: The entire chip of the computing architecture is integrated with multiple tube cores, including an upsampling tube core of the input signal and a structure formed by stacking three layers of tube cores, wherein: The weight storage unit of the first layer of tube core is used to store the weight matrix after upsampling and compression coding; The arithmetic unit of the second-layer tube core is used to read out the weight matrix and the upsampled signal matrix from the chip in sequence for matrix multiplication when calculating matrix multiplication; The result storage unit of the third layer of tube core is used to store the calculation results; Two adjacent tube cores are connected and supported by a transfer structure.

16. The computing architecture of low-complexity matrix multiplication according to claim 15, characterized in that: When a FLASH die is used to store the compressed and encoded weight matrix, the die of the computing unit and the FLASH die use a vertical interconnection channel to achieve high-speed data reading.

17. The computing architecture of low-complexity matrix multiplication according to claim 15, characterized in that: When the computing unit of the second-layer tube core uses a parallel addition tree to complete the matrix multiplication operation; if the weight adopts binary quantization, the signal or the complement of the signal is selected according to the weight value; if the weight adopts ternary quantization, zero or the signal or the complement of the signal is selected according to the weight value, and the selected result is input into the adder, and multiple adders calculate simultaneously.

18. The computing architecture of low-complexity matrix multiplication according to claim 17, characterized in that: When the operation unit uses a parallel addition tree to complete the matrix multiplication operation, it includes: inputting each upsampled signal into a selector, selecting the output according to the corresponding weight; if the weight is 1, output +x k,1 If the weight is -1, then output -x k,1 , if the weight is 0, then output 0; sum the outputs of the selectors, multiple adders calculate in parallel, and output 19. The computing architecture of low-complexity matrix multiplication according to claim 15, characterized in that: When the computing unit of the second-layer tube core uses an analog circuit including capacitor integration to complete matrix multiplication operations; the weight matrix is ​​read in a preset order, and the integration circuit is controlled to be positively integrated, negatively integrated or not integrated according to the weight matrix, and the size of the elements of the signal matrix determines the amount of integration.

20. The computing architecture of low-complexity matrix multiplication according to claim 19, characterized in that: When the operation unit uses an analog circuit including capacitor integration to complete the matrix multiplication operation, Including: Step 1: Disconnect the preset S3 switch, according to the weight w 1,n Select the preset positions of switches S1 and S2. If the weight is 1, switches S1 and S2 are both down, that is, S2 is grounded, S1 is connected to the signal, and the charge of capacitor C1 is positively integrated +x k,1 If the weight is -1, switches S1 and S2 are both on, and the charge of capacitor C1 is negatively integrated -x k,1 , if the weight is 0, switch S1 is on the top and S2 is on the bottom, that is, S1 and S2 are both grounded, and the charge of capacitor C1 is reset to zero; Step 2: S1 is in the middle, S2 is on the bottom, and the S3 switch is closed, that is, capacitor C1 is connected to C0, and the charge of capacitor C0 is equal to the sum of the charges of multiple C1s, and the output

Images