A compression and transmission method for a neural network large-scale weight matrix

Abstract

The application discloses a compression and transmission method for a large-scale weight matrix of a neural network, and belongs to the technical field of neural network processors. In order to reduce data transmission cost, the application comprises the following steps: arranging the weight of each layer of a DNN model into a weight matrix; using a singular value decomposition method to process the weight matrix to obtain a left coefficient matrix, a base matrix and a right coefficient matrix of the weight matrix of each layer; using a bubble algorithm to process the base matrix, and using a quantization and approximation compression method to process the left coefficient matrix and the right coefficient matrix; loading the processed left coefficient matrix and the right coefficient matrix of each layer into an off-chip dynamic random access memory, and loading the processed base matrix of the weight matrix into an on-chip cache; using a data recovery array design method to process the loaded data to obtain a reconstructed weight matrix of each layer, and inputting the reconstructed weight matrix into a DNN accelerator for a calculation process of the DNN accelerator. The application greatly reduces memory access cost.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of neural network processors, and particularly relates to a compression and transmission method for a large-scale weight matrix of a neural network. BACKGROUND

[0002] Deep neural network (DNN) is an important tool for solving many real-world applications, and can be used for image processing, speech recognition and many other applications, including edge computing devices, mobile embedded systems and various Internet of Things (IoT) devices, and is widely used in many resource-constrained scenarios.

[0003] In order to make DNN run more efficiently, academic and industrial circles will design DNN inference accelerators. As a special computing chip, the accelerator has higher energy consumption and computing efficiency than general-purpose computing chips such as CPU and GPU. The computing power of the DNN accelerator is often improved with the increase of the DNN scale, and the large-scale DNN will cause a large amount of computation and data movement, causing a large amount of energy consumption of the accelerator, which is not conducive to deploying the model in energy-constrained scenarios. In order to apply DNN to such computing resource and energy-constrained scenarios, it is crucial to reduce the energy consumption of the DNN accelerator.

[0004] Approximate computing uses the inherent error tolerance of some applications to trade off computing accuracy for energy efficiency, improves the energy efficiency of the system without affecting the computing quality, and achieves good results. Past research on approximate computing has mostly focused on reducing energy consumption in the computing process by designing low-power arithmetic units or developing approximate training and execution frameworks. However, research has shown that because the large-scale weight matrix of the DNN needs to be stored in the off-chip dynamic random access memory (DRAM) with high energy consumption, the storage and access operations of the weight values may still cause the problem of high energy consumption of the DNN. SUMMARY

[0005] The problem to be solved by the application is to reduce the data transmission cost as much as possible while reducing the data scale and ensuring small data precision loss, and a compression and transmission method for a large-scale weight matrix of a neural network is proposed.

[0006] To achieve the above purpose, the application realizes the following technical scheme:

[0007] A compression and transmission method for a large-scale weight matrix of a neural network, comprising the following steps:

[0008] S1. The DNN accelerator loads the DNN model from the software layer using the CPU, parses the weight values of the DNN model, and arranges the weight values of each layer of the DNN model into a weight matrix W1…Wn. i , Wn n , where W i is the weight matrix of the i-th layer.

[0009] S2. For each weight matrix of each layer obtained in step S1, a singular value decomposition method is used for processing to obtain the left coefficient matrix, the basis matrix and the right coefficient matrix of the weight matrix of each layer.

[0010] S3. The basis matrix of the weight matrix of the 2nd layer to the nth layer in the basis matrix of the weight matrix of each layer obtained in step S2 is processed using the bubble algorithm respectively, and then each element in the basis matrix of the weight matrix of the 2nd layer to the nth layer is arranged from large to small, and the elements within the set percentage value range are retained, to obtain the processed basis matrix of the weight matrix.

[0011] S4. For the left coefficient matrix and the right coefficient matrix of each layer obtained in step S2, a quantization and approximation compression method is used for processing to obtain the processed left coefficient matrix and the right coefficient matrix of each layer.

[0012] S5. The processed left coefficient matrix and the right coefficient matrix of each layer obtained in step S3 are loaded into the off-chip dynamic random access memory, and the processed basis matrix of the weight matrix obtained in step S2 is loaded into the on-chip cache.

[0013] S6. For the data loaded in step S5, a data recovery array design method is used to obtain the reconstructed weight matrix of each layer, which is input into the DNN accelerator for the calculation process of the DNN accelerator.

[0014] Further, in step S1, if the operation of each layer of the DNN model is based on multiple weight matrices, the multiple weight matrices are arranged into one weight matrix.

[0015] Further, the singular value decomposition method in step S2 includes the following steps:

[0016] S2.1. For any weight matrix W i , solve W i W i T and W i T W i , W i T denotes the transpose of W i .

[0017] S2.2. For the W i obtained in step S2.1,i W i T To find the eigenvector u, the expression for the characteristic equation is set as follows:

[0018] |λE-W i W i T |=0

[0019] Where λ is the eigenvalue to be solved, E represents the unit diagonal matrix, and λE is a diagonal matrix in which all diagonals are λ.

[0020] Then for each eigenvalue λ k Solve for the expression:

[0021] (λ k EW i W i T u = 0

[0022] Where u is the eigenvector of the left coefficient matrix, and each eigenvector is a column vector. Concatenating u horizontally yields the left coefficient matrix U. i ;

[0023] S2.3. The W obtained in step S2.1 i T W i Following the method in step S2.2, solve for the eigenvectors v of the right coefficient matrix. Then, since each eigenvector is a column vector, concatenate v horizontally to obtain the right coefficient matrix V. i ;

[0024] S2.4. Setting W i W i T and W i T W i eigenvalues ​​λ k Similarly, the square roots of the positive eigenvalues ​​are calculated, and the square roots are used to construct the basis matrix ∑. i .

[0025] Furthermore, the specific implementation method of step S3 includes the following steps:

[0026] S3.1. Retain all parameters of ∑1 corresponding to the first layer weight matrix W1 of the DNN model;

[0027] S3.2. For the weight matrix of the second layer to the weight matrix of the nth layer W2……W n The corresponding basis matrix ∑2……∑ n First, process them individually using the bubble sort algorithm, then process each element e within each basis matrix. i Compare them one by one, if ei+1 >e i , then the two elements are exchanged in order, and finally the base matrix with the elements in each base matrix arranged in descending order is obtained, and the number of data is counted during the arrangement, and the total number of elements in each base matrix s is obtained;

[0028] According to the arrangement, the value of the element arranged at 0.1*s is recorded as e th , and then the value of e th is used to determine whether the element is retained or set to 0. i+1 >e th , the element is retained, otherwise the element is set to 0, and the base matrix of the processed weight matrix is obtained.

[0029] Further, the specific implementation method of step S4 includes the following steps:

[0030] S4.1. Each element in the left coefficient matrix and the right coefficient matrix of each layer obtained in step S2 is quantized, and the expression is calculated as:

[0031] lu ij =log2(u ij )

[0032] lv ij =log2(v ij )

[0033] wherein u ij is the element in the i-th row and the j-th column of the left coefficient matrix of each layer; v ij is the element in the i-th row and the j-th column of the right coefficient matrix of each layer.

[0034] Each element in the coefficient matrix is converted into an 8-bit base-2 power index l through quantization;

[0035] S4.2. Each element obtained in step S4.1 is converted into an 8-bit base-2 power index, and an approximate compression processing is performed, the highest two significant digits are retained, the first significant digit l ch is represented by a 3-bit binary number, and the difference between the second significant digit and the first significant digit is represented by a 2-bit binary number l cl , so that the 8-bit base-2 power index is converted into a 5-bit binary number lc, and the expression is:

[0036] l c =[l ch , l cl ];

[0037] S4.3. Process each element in the left coefficient matrix and right coefficient matrix of each layer obtained in step S2 using the methods of steps S4.1-S4.2 to obtain the processed left coefficient matrix and right coefficient matrix of each layer.

[0038] Furthermore, the data recovery array design method in step S6 is as follows:

[0039] S6.1. First calculate U i ∑ i Since the elements inside U represent the number of shifts, then according to the elements u in U... ij The instruction is to add each element e in ∑ i According to u ij The first 3 l ch Shift left, and then rearrange the result according to the last two bits. cl Perform a left shift and then add the shifted results. The expression is:

[0040] h ij =e i < <l ch +(e i < <l ch )< <l cl

[0041] After all calculations on the matrix are completed, the intermediate variable H is obtained;

[0042] S6.2. Recalculate HV T , using V i The elements in H i The elements in V are shifted and summed. Since the elements inside V represent the number of shifts, the values ​​of V and V are used to calculate the number of shifts. ij The instruction is to add each element h in ∑ ij According to v ij The first 3 l ch Shift left, and then rearrange the result according to the last two bits. cl Perform a left shift and then add the shifted results. The expression is:

[0043] w ij =h ij < <l ch +(h ij < <l ch )< <l cl ;

[0044] After all calculations are performed on the matrix, the reconstructed weight matrix W is obtained. i .

[0045] Further, the hardware module of the data recovery unit in step S6 includes two shifters and an adder, two inputs of the first shifter are respectively element data in the Cache loaded and elements in U loaded from the DRAM, two inputs of the second shifter are respectively the output of the first shifter and elements in V loaded from the DRAM; two inputs of the adder are respectively the outputs of the two shifters, to constitute the data recovery unit; and 32 data recovery units are tiled to constitute a data recovery array.

[0046] The beneficial effects of the present application are as follows:

[0047] The compression and transmission method for the large-scale weight matrix of the neural network provided by the present application replaces high-cost multiplication with low-cost shifting and addition, and uses shorter compressed data representation to reduce storage cost.

[0048] The compression and transmission method for the large-scale weight matrix of the neural network provided by the present application has a compression effect on the overall weight matrix that is comparable to the most advanced compression work, and can compress the base matrix to be smaller, which is suitable for storing in a cache with lower access cost. The weight scale is compressed, the memory access cost is greatly reduced, and the energy efficiency level of the neural network accelerator is improved.

[0049] The compression and transmission method for the large-scale weight matrix of the neural network provided by the present application is designed for a DNN accelerator in a 28nm process library, and the energy consumption and delay are evaluated by using Cadence Genes 15.0 software. The weight compression scale and accuracy of two DNN models, VGG11 and VGG19, are tested on a software platform based on the open-source PyTorch framework. The method proposed in the present application can compress the weights of the DNN to 3.96%-6.38% of the original, reduce the energy consumption of memory access by 86.47%-93.62%, and the precision loss is less than 2%, and the additional time delay is only 1.7%. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The flowchart of the compression and transmission method for the large-scale weight matrix of the neural network provided by the present application;

[0051] Figure 2 The flowchart of the compression and transmission method for the large-scale weight matrix of the neural network provided by the present application;

[0052] Figure 3 The schematic diagram of the compressed data representation of the present application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application, i.e., the specific embodiments described are only a part of the embodiments of the present application, but not all the specific embodiments. The components of the specific embodiments of the present application generally described and shown in the drawings can be arranged and designed in various different configurations, and the present application can also have other embodiments.

[0054] Therefore, the detailed description of the specific embodiments of the present application provided below in the drawings is not intended to limit the scope of the claimed present application, but only represents selected specific embodiments of the present application. Based on the specific embodiments of the present application, all other specific embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0055] In order to further understand the invention content, characteristics and effects of the present application, the following specific embodiments are exemplified, and the drawings are combined Figure 1 - the drawings Figure 3 The detailed description is as follows:

[0056] Example 1:

[0057] A compression and transmission method for a neural network large-scale weight matrix, comprising the following steps:

[0058] S1. The DNN accelerator loads the DNN model from the software layer using the CPU, parses the weight of the DNN model, and arranges the weight of each layer of the DNN model into a weight matrix W1…W i , W n , wherein W i is the weight matrix of the i-th layer;

[0059] Further, if the operation of each layer of the DNN model is based on multiple weight matrices in step S1, the multiple weight matrices are arranged into one weight matrix;

[0060] Further, if the weight matrix of one convolution operation is multiple convolution kernels, the convolution kernels are spliced to form one weight matrix;

[0061] S2. For each layer of the weight matrix obtained in step S1, a singular value decomposition method is used for processing to obtain the left coefficient matrix, the basis matrix and the right coefficient matrix of each layer of the weight matrix;

[0062] Further, the specific implementation method of the singular value decomposition method in step S2 comprises the following steps:

[0063] S2.1. For any one weight matrix Wi Solve W i W i T and W i T W i , W i T denotes the transpose of W i ;

[0064] S2.2. Solve W i W i T for eigenvectors u, and set the expression of the eigen-equation as:

[0065] |λE-W i W i T |=0

[0066] where λ is the eigenvalue to be solved, E represents the unit diagonal matrix, and λE is a diagonal matrix with λ on the diagonal;

[0067] Then solve for each eigenvalue λ k , and the expression is:

[0068] (λ k E-W i W i T )u=0

[0069] where u is the eigenvector of the left coefficient matrix, each eigenvector is a column vector, and u is transversely spliced to obtain the left coefficient matrix U i ;

[0070] S2.3. Solve W i T W i for the eigenvectors of the right coefficient matrix v according to the method of step S2.2, and then each eigenvector is a column vector, and v is transversely spliced to obtain the right coefficient matrix V i ;

[0071] S2.4. Set W i W i T and W i T W i the eigenvalues λ k are the same, and the positive values in the eigenvalues are solved for square roots, and the square roots form the basis matrix ∑ i .

[0072] Further, three weight matrices U i , ∑i , V i , the weight matrix W can be reconstructed by multiplying the three matrices i , the expression is:

[0073] W i = U i ∑ i V i T .

[0074] S3. The second layer to the nth layer of the weight matrix of the base matrix of each layer obtained in step S2 are processed respectively using the bubble algorithm, and then each element in the second layer to the nth layer of the weight matrix of the base matrix is arranged from large to small, and the elements within a certain percentage value range are retained, to obtain the processed weight matrix of the base matrix;

[0075] Further, the specific implementation method of step S3 includes the following steps:

[0076] S3.1. Retain all parameters of ∑1 corresponding to the first layer weight matrix W1 of the DNN model;

[0077] S3.2. The second layer weight matrix to the nth layer weight matrix W2……W n corresponding base matrix ∑2……∑ n First, use the bubble algorithm one by one and separately, and then compare each element e i one by one, if e i+1 >e i , the two elements are exchanged in order, and finally the base matrix of each element arranged from large to small is obtained, and the number of data is counted at the same time, to obtain the total number s of elements in each base matrix;

[0078] Then record the value e th of the element ranked 0.1*s according to the arrangement, and further process each ∑ according to e th If an element e i+1 >e th , the element is retained, otherwise the element is set to 0, to obtain the processed weight matrix of the base matrix;

[0079] S4. The left coefficient matrix and the right coefficient matrix of each layer obtained in step S2 are processed using the quantization and approximation compression method to obtain the processed left coefficient matrix and the right coefficient matrix of each layer.

[0080] Further, the specific implementation method of step S4 includes the following steps:

[0081] S4.1. quantize each element in the left coefficient matrix and the right coefficient matrix of each layer obtained in step S2, and calculate the expression:

[0082] lu ij = log2(u ij )

[0083] lv ij = log2(v ij )

[0084] wherein u ij is the element in the i-th row and the j-th column of the left coefficient matrix of each layer; v ij is the element in the i-th row and the j-th column of the right coefficient matrix of each layer;

[0085] Each element in the coefficient matrix is converted to the power index l with base 2 of 8-bit through quantization;

[0086] S4.2. each element obtained in step S4.1 is converted to the power index with base 2 of 8-bit, and is subjected to approximate compression processing, the highest two significant digits are reserved, the first significant digit l ch is represented by 3-bit binary number, the difference between the second significant digit and the first significant digit is represented by 2-bit binary number l cl , thereby converting the power index with base 2 of 8-bit to binary number lc of 5-bit, and the expression is:

[0087] l c = [l ch , l cl ];

[0088] S4.3. each element in the left coefficient matrix and the right coefficient matrix of each layer obtained in step S2 is processed by using the method of step S4.1-step S4.2, and the left coefficient matrix and the right coefficient matrix of each layer after processing are obtained;

[0089] S5. the left coefficient matrix and the right coefficient matrix of each layer after processing obtained in step S3 are loaded into the off-chip dynamic random access memory, and the base matrix of the weight matrix after processing obtained in step S2 is loaded into the on-chip cache;

[0090] Furthermore, all parameters obtained in the preceding steps need to be stored and allocated according to the characteristics of the data during the accelerator's operation. The DNN accelerator involves two types of memory: off-chip Dynamic Random Access Memory (DRAM) and on-chip cache. DRAM is external to the accelerator, with high access power consumption but large capacity; cache has low access power consumption but small capacity. The coefficient matrices U and V, after passing through S3 and S4, yield a compressed data representation. Due to the large size of its matrix elements, this is stored in off-chip DRAM; because the basis matrix is ​​a diagonal matrix, essentially a vector, its data size is small and can be directly stored in the cache, thus ensuring that storage overhead is minimized.

[0091] S6. For the data loaded in step S5, perform a data recovery array design method to obtain the weight matrix of each reconstructed layer, which is then input into the DNN accelerator for the DNN accelerator's computation process.

[0092] Furthermore, the data recovery array design method in step S6 is as follows:

[0093] S6.1. First calculate U i ∑ i Since the elements inside U represent the number of shifts, then according to the elements u in U... ij The instruction is to add each element e in ∑ i According to u ij The first 3 l ch Shift left, and then rearrange the result according to the last two bits. cl Perform a left shift and then add the shifted results. The expression is:

[0094] h ij =e i < <l ch +(e i < <l ch )< <l cl

[0095] After all calculations on the matrix are completed, the intermediate variable H is obtained;

[0096] S6.2. Recalculate HV T , using V i The elements in H i The elements in V are shifted and summed. Since the elements inside V represent the number of shifts, the values ​​of V and V are used to calculate the number of shifts. ij The instruction is to add each element h in ∑ ij According to v ij The first 3 l ch Shift left, and then rearrange the result according to the last two bits. clThe left shift is performed, and the shifted results are added, and the expression is:

[0097] w ij = h ij <<1 ch + (h ij <<1 ch ) <<1 cl ;

[0098] After all the matrix calculations are completed, the reconstructed weight matrix W i is obtained.

[0099] Further, the hardware module of the data recovery unit in step S6 includes two shifters and an adder, two inputs of the first shifter are respectively the element data in ∑ loaded by Cache and the element in U loaded from DRAM, two inputs of the second shifter are respectively the output of the first shifter and the element in V loaded from DRAM; two inputs of the adder are respectively the outputs of the two shifters, constituting the data recovery unit; and 32 data recovery units are tiled to constitute a data recovery array.

[0100] Further, because the time and energy cost of shift and addition calculation are very low, the time and energy consumed by the data recovery array to obtain W are much lower than the time and energy consumed to load the complete W from the external DRAM.

[0101] The present embodiment designs a DNN accelerator in a 28nm process library, and uses Cadence Genus 15.0 software to evaluate the energy consumption and delay. In addition, the weight compression scale and accuracy of two DNN models, VGG11 and VGG19, are tested on a software platform based on the open-source PyTorch framework. The method proposed in the present patent can compress the weights of the DNN to 3.96%-6.38% of the original, reduce the energy consumption of memory by 86.47%-93.62%, and the accuracy loss is less than 2%, and the additional delay is only 1.7%.

[0102] It has to be noted that the terms "first", "second", and the like in connection with an entity or action refer to this entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without further constraints, exclude the presence of additional elements of the process, method, article, or apparatus.

[0103] While the application has been described with reference to specific implementations thereof, it should be understood that various modifications and substitutions can be made by those skilled in the art without departing from the scope of the present application. In particular, any one of the features of the present application disclosed above can be utilized independently of any other and the scope of the application should not be limited by the specific embodiments disclosed herein, but should be given the widest coverage possible in its true scope.

Claims

1. A method for compressing and transmitting large-scale weight matrices for neural networks, characterized in that, Includes the following steps: S1. The DNN accelerator uses the CPU to load the DNN model from the software level, parses the weights of the DNN model, and organizes the weights of each layer of the DNN model into a weight matrix W1…W i W n Among them, W i Let be the weight matrix of the i-th layer; S2. For the weight matrix of each layer obtained in step S1, use the singular value decomposition method to process it, and obtain the left coefficient matrix, basis matrix and right coefficient matrix of the weight matrix of each layer. S3. For the basis matrices of the weight matrices of the second to nth layers obtained in step S2, the bubble sort algorithm is used to process them respectively. Then, each element in the basis matrices of the weight matrices of the second to nth layers is arranged from largest to smallest, and elements within a set percentage range are retained to obtain the basis matrices of the processed weight matrices. S4. The left and right coefficient matrices of each layer obtained in step S2 are processed by the method of quantization and approximation compression to obtain the processed left and right coefficient matrices of each layer. S5. Load the left and right coefficient matrices of each layer after processing in step S3 into off-chip dynamic random access memory, and load the base matrix of the weight matrix after processing in step S2 into on-chip cache. S6. For the data loaded in step S5, perform a data recovery array design method to obtain the weight matrix of each reconstructed layer, which is then input into the DNN accelerator for the DNN accelerator's computation process.

2. The method for compression and transmission of large-scale weight matrices for neural networks according to claim 1, characterized in that, If the operation of each layer of the DNN model is based on multiple weight matrices in step S1, then the multiple weight matrices are combined into one weight matrix.

3. A method for compressing and transmitting large-scale weight matrices for neural networks according to claim 1 or 2, characterized in that, The specific implementation method of the singular value decomposition method in step S2 includes the following steps: S2.

1. For any weight matrix W i Solve for W i W i T and W i T W i W i T W i transpose; S2.

2. The W obtained in step S2.1 i W i T To find the eigenvector u, the expression for the characteristic equation is set as follows: |λE-W i W i T |=0 Where λ is the eigenvalue to be solved, E represents the unit diagonal matrix, and λE is a diagonal matrix in which all diagonals are λ. Then for each eigenvalue λ k Solve for the expression: (λ k E-W i W i T )u=0 Where u is the eigenvector of the left coefficient matrix, and each eigenvector is a column vector. Concatenating u horizontally yields the left coefficient matrix U. i ; S2.

3. The W obtained in step S2.1 i T W i Following the method in step S2.2, solve for the eigenvectors v of the right coefficient matrix. Then, since each eigenvector is a column vector, concatenate v horizontally to obtain the right coefficient matrix V. i ; S2.

4. Setting W i W i T and W i T W i eigenvalues ​​λ k Similarly, the square roots of the positive eigenvalues ​​are calculated, and the square roots are used to construct the basis matrix ∑. i .

4. The method for compression and transmission of large-scale weight matrices for neural networks according to claim 3, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. Retain all parameters of ∑1 corresponding to the first layer weight matrix W1 of the DNN model; S3.

2. For the weight matrix of the second layer to the weight matrix of the nth layer W2……W n The corresponding basis matrix ∑2……∑ n First, process them individually using the bubble sort algorithm, then process each element e within each basis matrix. i Compare them one by one, if e i+1 >e i If the two elements are swapped, the base matrix is ​​obtained by arranging the elements in each base matrix from largest to smallest. The number of data is counted while arranging the elements to obtain the total number of elements s in each base matrix. Then, based on the arrangement, record the value e of the element located at 0.1*s. th Then according to e th Further processing for each ∑: If an element e i+1 >e th If the element is true, then keep it; otherwise, set the element to 0 to obtain the basis matrix of the processed weight matrix.

5. The method for compression and transmission of large-scale weight matrices for neural networks according to claim 4, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. For each element in the left and right coefficient matrices of each layer obtained in step S2, perform logarithmic quantization, and calculate the expression as follows: read ij <log2(u ij ) lv ij =log2(v ij ) Among them, u ij v is the element in the i-th row and j-th column of the left coefficient matrix of each level; ij For each level, the element in the i-th row and j-th column of the right coefficient matrix; The coefficient matrix is ​​converted into an 8-bit power of 2 by quantization; S4.

2. Convert each element obtained in step S4.1 into an 8-bit power of 2, perform approximate compression, retain the two most significant digits, and represent the first significant digit l using a 3-bit binary number. ch Using 2-bit binary numbers l cl This represents the difference between the second and first significant digits, thus converting an 8-bit power of 2 to a 5-bit binary number lc. The expression is: l c ＝[l ch ，l cl ]； S4.

3. Process each element in the left coefficient matrix and right coefficient matrix of each layer obtained in step S2 using the methods of steps S4.1-S4.2 to obtain the processed left coefficient matrix and right coefficient matrix of each layer.

6. The method for compression and transmission of large-scale weight matrices for neural networks according to claim 5, characterized in that, The data recovery array design method in step S6 is as follows: S6.

1. First calculate U i ∑ i Since the elements inside U represent the number of shifts, then according to the elements u in U... ij The instruction is to add each element e in ∑ i According to u ij The first 3 l ch Shift left, and then rearrange the result according to the last two bits. cl Perform a left shift, then add the shifted results together. The expression is: h ij =e i <<l ch +(e i <<l ch )<<l cl After all calculations on the matrix are completed, the intermediate variable H is obtained; S6.

2. Recalculate HV T , using V i The elements in H i The elements in V are shifted and summed. Since the elements inside V represent the number of shifts, the values ​​of V and V are used to calculate the number of shifts. ij The instruction is to add each element h in ∑ ij According to v ij The first 3 l ch Shift left, and then rearrange the result according to the last two bits. cl Perform a left shift, then add the shifted results together. The expression is: w ij =h ij <<l ch +(h ij <<l ch )<<l cl ; After all calculations are performed on the matrix, the reconstructed weight matrix W is obtained. i .

7. The method for compression and transmission of large-scale weight matrices for neural networks according to claim 6, characterized in that, The hardware module for designing the data recovery unit in step S6 includes two shifters and one adder. The two inputs of the first shifter are the element data in ∑ loaded from the cache and the element in U loaded from the DRAM, respectively. The two inputs of the second shifter are the output of the first shifter and the element in V loaded from the DRAM, respectively. The two inputs of the adder are the outputs of the two shifters, respectively, which constitute the data recovery unit. Then, the 32 data recovery units are laid flat to form a data recovery array.

Images