Modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration system

Abstract

The present invention discloses a modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration method and system. The method comprises: a user end receives input weight data of a neural network model, quantizes it using the addition quadratic power method, and then rearranges and transposes it; the user end receives input source data of the neural network model, rearranges it using the img2col method, transposes it, and encodes it into a plaintext polynomial; the user end encrypts the encoded plaintext polynomial into ciphertext corresponding to the homomorphic encryption scheme, and sends it together with the processed weight data to a hardware accelerator; the hardware accelerator initiates a neural network linear layer operation under homomorphic encryption conditions on the ciphertext and weight data, and feeds back the operation result to the user end. The present invention reduces the calculation cycle and resource overhead of the modular multiplication operator, thereby improving calculation efficiency.

Description

Technical Field

[0001] The present invention relates to the research field of hardware acceleration of homomorphic encryption and neural networks. Specifically, the present invention is a hardware system based on the BFV scheme, which accelerates the linear layer operations of neural networks using a specific method under homomorphic encryption conditions. Background Art

[0002] The concept of machine learning as a service (MLaaS) has been proposed in recent years and has gradually become a hot field. Its practical connotation is the numerous machine learning-based cloud services provided by cloud providers to their customers. However, with the expansion of these cloud services, the security of user privacy data has gradually become a major concern. If traditional methods are used to encrypt and upload data, the cloud service provider still needs to obtain the key and decrypt the data before performing operations. However, this undoubtedly poses a significant security risk to user privacy. Homomorphic encryption (HE), on the other hand, allows operations to be performed directly on ciphertext without decryption; therefore, homomorphic encryption technology has become the most promising solution for protecting user data security in the cloud computing era.

[0003] Neural networks (NNs) are the most widely used models in machine learning. Among neural networks, the most common algorithmic model is the convolutional neural network (CNN), primarily used in image processing. This model consists of several convolutional and fully connected layers. In fact, the underlying logic of convolutional and fully connected operations in CNNs is multiplication-accumulation, which aligns well with homomorphic encryption's support for homomorphic multiplication and addition. Therefore, among many machine learning algorithms, neural network models are particularly well-suited for implementation using homomorphic encryption.

[0004] Currently, the most intensively researched and mainstream homomorphic encryption schemes for numerical computation include BFV, BGV, and CKKS. These schemes are primarily based on the learning with error (LWE) and ring-learning with error (RLWE) problems. To ensure security, the ciphertext domain of these schemes is a high-dimensional ring of large integer polynomials. The original data is encrypted and mapped onto the ring, existing as a polynomial on the ring. While this encrypted mapping ensures security, it also introduces certain issues, such as large ciphertext size and low computational efficiency. This also creates difficulties in building neural network models under homomorphic encryption conditions.

[0005] In 2016, researchers including Dowlin proposed a homomorphic encryption neural network scheme called Cryptonets, combining convolutional neural networks with homomorphic encryption for the first time. This scheme is based on the YASHE homomorphic encryption scheme. The team also proposed that the weight information in the neural network does not actually involve user privacy and can therefore exist in plaintext. In fact, plaintext is smaller in size, is faster when compared to ciphertext, and does not generate additional noise. Inspired by this team's work, researchers in related fields have also conducted extensive research based on various homomorphic encryption schemes and different neural network models, attempting to improve the computational efficiency and practicality of neural networks under homomorphic encryption conditions. However, given the large data volumes and computational complexity of mainstream homomorphic encryption schemes, further improving the computational efficiency of homomorphic encrypted neural networks remains a challenge. Summary of the Invention

[0006] The purpose of the present invention is to solve the above-mentioned problem of how to improve the computational efficiency of homomorphic encryption neural networks, and to provide a modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration method and system, which can simplify ciphertext scalar operations, reduce computational memory access overhead, and better meet practical application needs. The specific implementation is as follows:

[0007] Technical solution: A modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration method, including the following steps:

[0008] Step S1: The user terminal receives the input weight data of the neural network model, quantizes it using the addition quadratic method, and then rearranges and transposes it;

[0009] Step S2: The user terminal receives the source data of the input neural network model, rearranges it using the img2col method, then transposes it, and encodes it into a plaintext polynomial.

[0010] Step S3: The user end encrypts the encoded plaintext polynomial into ciphertext corresponding to the homomorphic encryption scheme, and sends it together with the processed weight data to the hardware accelerator; the hardware accelerator starts the neural network linear layer operation under the homomorphic encryption condition on the ciphertext and weight data, and feeds back the operation results to the user end.

[0011] According to one aspect of the present application, in step S1, the process of quantization using the addition quadratic power method is further as follows:

[0012] Step S11: Get the original weight data x and quantize it to get an n-bit fixed-point number

[0013]

[0014] Q set ={x|x=x1+x2,x1∈Q1,x2∈Q2}∩[-1,1);

[0015]

[0016] Among them, the number sign(x) is used to determine the sign of the input value x. When x ≥ 0, the function returns 1, otherwise it returns 0; the function findnearest() returns the set Q set The value closest to x; set Q set The elements in are obtained by adding the elements in two other specific sets Q1 and Q2 to each other;

[0017] Step S12: Take a1≤a2=log2n, and transform the above formula into:

[0018]

[0019] Where, 0≤k1,k2≤n-1,φ1,φ2∈{0,1}.

[0020] According to one aspect of the present application, step S3 further includes a data segmentation process:

[0021] Step S31: Send the source data and weights into corresponding buffers respectively;

[0022] Step S32: start the multiplication and accumulation operations of each multiplication and accumulation calculation unit;

[0023] Step S33: The multiplication and accumulation results within the same group are summed with the results of the last designated block, and the sum is output to the output buffer to wait for the next operation.

[0024] According to one aspect of the present application, step 3 further includes modular multiplication calculation:

[0025] Step S3a, input source data a, quantization weight w and modulus q;

[0026] Step S3b, judging whether the parameter φ1 is 0, if not 0, the value of the source data a after being shifted left by k1 bits is sent to the adder, otherwise 0 is sent to the adder; judging whether the parameter φ2 is 0, if not 0, the value of the source data a after being shifted left by k2 bits is sent to the adder, otherwise 0 is sent to the adder;

[0027] Step S3c: perform an addition operation to calculate the sum of the data sent to the adder in the previous step;

[0028] Step S3d, sending the result of the previous step and the modulus q to the modulus operator to perform a modulus operation;

[0029] Step S3e: Determine the final output result according to the sign bit of the weight. If sign(w)=1, output parameter y; otherwise, output y=qy.

[0030] According to one aspect of the present application, step S3 further includes a ciphertext segmentation step:

[0031] The parameter u of the multiplication and accumulation array is used to split the ciphertext into several (d 2 u)×u small blocks, and at the same time divide the weights into several u×(d 2 u) Small blocks: During calculation, each time a split ciphertext is taken, all the splits on a column corresponding to the calculation weight can be traversed.

[0032] According to another aspect of the present application, a modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration system is provided, comprising a homomorphic encryption ciphertext scalar multiplication-accumulation operation array, wherein the homomorphic encryption ciphertext scalar multiplication-accumulation operation array comprises a predetermined number of operation units, each operation unit comprises a source data buffer, an output buffer, and a multiplication-accumulation cluster, each multiplication-accumulation cluster comprises a controller, a weight buffer, a predetermined number of sequentially connected multiplication-accumulation operators, and a modular adder,

[0033] The number of source data buffers, weight buffers and multiplication-accumulation operators is equal. Each source data buffer outputs to the multiplication-accumulation operator of the same level of each operation unit, and each weight buffer outputs to all multiplication-accumulation operators of the operation unit where it is located.

[0034] According to another aspect of the present application, the multiply-accumulate cluster is designed based on an optimized additive quadratic quantization algorithm, and the optimized additive quadratic quantization algorithm includes the following steps:

[0035] Step S11: Get the original weight data x and quantize it to get an n-bit fixed-point number

[0036]

[0037] Q set ={x|x=x1+x2,x1∈Q1,x2∈Q2}∩[-1,1);

[0038]

[0039] Among them, the number sign(x) is used to determine the sign of the input value x. When x ≥ 0, the function returns 1, otherwise it returns 0; the function findnearest() returns the set Q set The value closest to x; set Q set The elements in are obtained by adding the elements in two other specific sets Q1 and Q2 to each other;

[0040] Step S12: Take a1≤a2=log2n, and transform the above formula into:

[0041]

[0042] Where, 0≤k1,k2≤n-1,φ1,φ2∈{0,1}.

[0043] According to another aspect of the present application, the calculation process of the multiply-accumulate cluster is:

[0044] Step S3a, input source data a, quantization weight w and modulus q;

[0045] Step S3b, judging whether the parameter φ1 is 0, if not 0, the value of the source data a after being shifted left by k1 bits is sent to the adder, otherwise 0 is sent to the adder; judging whether the parameter φ2 is 0, if not 0, the value of the source data a after being shifted left by k2 bits is sent to the adder, otherwise 0 is sent to the adder;

[0046] Step S3c: perform an addition operation to calculate the sum of the data sent to the adder in the previous step;

[0047] Step S3d, sending the result of the previous step and the modulus q to the modulus operator to perform a modulus operation;

[0048] Step S3e: Determine the final output result according to the sign bit of the weight. If sign(w)=1, output parameter y; otherwise, output y=qy.

[0049] The advantages of the present invention are as follows:

[0050] First, the present invention implements a highly parallel and configurable homomorphic encryption neural network linear layer hardware acceleration system, which can perform efficient operations on the convolutional and fully connected layers in the neural network under the background of homomorphic encryption.

[0051] Second, the present invention adopts an optimized additive quadratic power weight quantization scheme, and on this basis proposes a streamlined modular multiplication operator, which reduces the calculation cycle and resource overhead of the modular multiplication operator and improves the calculation efficiency.

[0052] Third, the present invention adopts the operation and data acquisition logic of ciphertext splitting, which eliminates the need to treat the ciphertext as a whole. On this basis, ciphertext data with a large amount of data only needs to be moved once during operation, reducing the time overhead of data transmission.

[0053] Fourth, the present invention flexibly uses data preprocessing methods such as img2col, single instruction multiple data encoding and matrix transposition to encode and encrypt the source data, and can configure the weight and data bit width of the source data according to the accuracy requirements, thereby reducing the overall data volume of the ciphertext, reducing storage overhead and memory access difficulty.

[0054] In summary, the present invention can effectively improve computing efficiency, simplify computing procedures and memory access overhead, and has good practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a schematic diagram of the operation flow of the simplified modular multiplication operator built in this system.

[0056] Figure 2 This is a schematic diagram of the homomorphic encryption ciphertext scalar multiplication and accumulation operation array built by this system.

[0057] Figure 3 This is a schematic diagram of the homomorphic encryption ciphertext scalar multiplication and accumulation operator built in this system.

[0058] Figure 4 This is the ciphertext splitting and data retrieval logic diagram used by this system.

[0059] Figure 5 Flowchart of the linear layer operations of the homomorphically encrypted neural network of this system. DETAILED DESCRIPTION

[0060] The specific embodiments of the present invention are now described in conjunction with the accompanying drawings. The present invention constructs a highly parallel homomorphic encryption ciphertext scalar multiplication and accumulation operation array, which can perform scalar multiplication and accumulation on encrypted ciphertext and quantized weight scalars, thereby obtaining the operation results of neural network linear layers such as convolution and fully connected layers under homomorphic encryption conditions.

[0061] A modular multiplication-simplified hardware acceleration method for the linear layer of a homomorphic encrypted neural network mainly includes: an optimized weight quantization method for quantizing the weights in the neural network; a simplified modular multiplication scheme, built based on the optimized weight quantization scheme, which can significantly reduce the complexity of modular multiplication operations; an efficient data scheduling method, which can reduce the memory access time overhead and memory access difficulty; a highly parallel homomorphic encryption ciphertext scalar operation hardware acceleration system built based on the above optimization method, which can perform specific operations on the encrypted ciphertext and quantized weight scalars, thereby obtaining the operation results of the convolution, full connection and other neural network linear layers under homomorphic encryption conditions.

[0062] This application uses an efficient data scheduling method. In the input preprocessing stage, the img2col and transposition methods are used to rearrange the input to reduce the difficulty of memory access. In the specific operation stage, a ciphertext splitting operation and data acquisition logic is adopted, which can split the complete ciphertext. When moving data, the ciphertext with larger data volume is moved individually according to each split, and the ciphertext with smaller data volume is moved in a weighted traversal to improve the moving efficiency.

[0063] The hardware acceleration system is mainly composed of system peripherals, homomorphic encryption ciphertext scalar multiplication and accumulation operation array and storage module.

[0064] Based on an optimized additive quadratic quantization algorithm, a simplified modular multiplier is constructed to perform simplified modular multiplications on neural network inputs and quantized weights. This simplification allows expensive large-number multiplications to be converted into cheap shift-and-sum operations.

[0065] The modular multiplication simplified homomorphic encryption neural network linear layer hardware acceleration method and hardware acceleration system are further designed in that the single instruction multiple data encoding method described in step 2 of the calculation process specifically adopts the remainder theorem in the polynomial dimension, which can encode multiple source data into a plaintext polynomial. The specific encoding method is briefly described below. The plaintext space of this encryption scheme is in the polynomial ring Z m [x] / (x n +1), by selecting a suitable plaintext domain modulus m, (x n +1) is decomposed into n polynomial bases F1(x), F2(x), ... F n-1 (x), and referring to the CRT in the integer field, it is extended to the polynomial field. With the help of this set of bases, n parallel information can be encoded in the plaintext polynomial without affecting the secret state calculation.

[0066] During actual encoding, the number of batches of single-time single-instruction multiple-data encoding can be configured according to the actual data volume of each layer of the neural network to improve parallel efficiency. The specific implementation method is to assume that the dimension of the ciphertext polynomial is n, and in each linear layer of the current neural network, its source data are rearranged and transposed according to img2col. Assume that among all preprocessed source data matrices, the matrix with the most elements in one row has m elements per row. Then the maximum number of parallel batches that the current network can support is floor(n / m). The following example illustrates the benefits of such processing. For convolution operations, assume that the size of the output image is I w ×I h ×I c (width × height × input channel), assuming the convolution kernel size is d × d × I c ×O c Where d is the side length of the convolution kernel, O c is the number of output channels of the convolution. According to the characteristics of the BFV homomorphic encryption scheme, when performing scalar multiplication on the ciphertext, each coefficient of the ciphertext polynomial must be multiplied by the same scalar, otherwise it will violate the operation principle. Therefore, the traditional encoding method often flattens the source data into one dimension and encrypts each source data into a ciphertext separately. Moreover, after using single instruction multiple data encoding, if the required number of batches is less than the ciphertext dimension, it will cause a waste of ciphertext space. In this design, we first rearrange and transpose the input by img2col, and then convert the source data into d 2 I c ×O w O n The size of the convolution operation is converted from S·W (source data·weight) to W after transposition. T ·S T , for each row of the transposed source data, the same number will be multiplied by the scalar when the operation is performed, so that the row can be encoded and encrypted into the same ciphertext, while ensuring the ciphertext operation principle. In particular, for a batch of data, we only need d 2 I c ciphertext, while the traditional method requires I w ×I h ×I c Obviously, the encoding and encryption method of this design is more flexible and economical overall.

[0067] The bit width of the weights and ciphertext modulus can be flexibly configured according to the actual accuracy requirements of the network, and the amount of computational data can be targeted. The specific implementation method is that after the network to be deployed is trained, under the premise of meeting the accuracy requirements, the smallest possible bit width is selected to represent the source data and weights during network quantization, thereby reducing the amount of computation. In fact, in the hardware designed by the present invention, the relevant parameters of the weights and source data are configurable through software. For example, if for a trained network, the bit width of its weights and source data are reduced to half of the original after quantization without affecting the inference accuracy, then on this basis, the reduced bit width can be configured in the hardware accordingly, so that the complexity of the calculation and the storage overhead will also be reduced by half.

[0068] like Figure 1 As shown in the figure, a highly parallel homomorphic encryption ciphertext scalar multiplication and accumulation operation array is constructed. The specific structure mainly includes u groups of source data buffers, u groups of output buffers and u multiplication and accumulation clusters. Each multiplication and accumulation cluster includes a controller, a weight buffer, u multiplication and accumulation operators and a modular adder. The array can parallelize part of the source data and weights after the ciphertext is split in both row and column dimensions. For the same multiplication and accumulation group, the same set of weights is shared to complete the parallelism of the first dimension; for the i-th multiplication and accumulation unit of each multiplication and accumulation group, the data of the i-th source data buffer is shared to complete the parallelism of the second dimension. The specific operation process for a split is as follows:

[0069] Step 1: Send the source data and weights into the corresponding buffers respectively.

[0070] Step 2: Enable the multiplication and accumulation operations of each multiplication and accumulation calculation unit.

[0071] Step 3: The multiplication and accumulation results within the same group are summed up and the results of the last specified block are added bit by bit, and then output to the output buffer to wait for the next operation.

[0072] In the operation array we constructed, the main operation unit is the ciphertext scalar multiplication and accumulation unit. We constructed a multiplication and accumulation unit based on modular operation, and its specific structure is as follows: Figure 2 As shown in the figure, the multiplication-accumulation unit consists of a simplified modular multiplier and a modular adder. The inputs to the modular multiplier include the source data, quantized weights, and the modulus. The inputs to the modular adder include the output of the modular multiplier, the accumulated result from the previous stage, and the modulus. Each cycle, a set of source data and weights is fed into the multiplication-accumulation unit. Once the pipeline is established, the output terminal can output the accumulated result of each cycle in a continuous manner.

[0073] The multiplication-accumulation unit includes a simplified modular multiplier based on an optimized additive quadratic quantization method. The specific description of the quantization method is as follows:

[0074] Let the original data be x, and quantize it to get an n-bit fixed-point number The specific quantitative formula is:

[0075]

[0076] Among them, the function sign(x) is used to determine the sign of the input value x. When x ≥ 0, the function returns 1, otherwise it returns 0. The function findnearest() returns the set Q set The value closest to x. Set Q set The elements in are obtained by adding the elements in two other specific sets Q1 and Q2 to each other, that is:

[0077] Q set ={x|x=x1+x2,x1∈Q1,x2∈Q2}∩[-1,1)(2)

[0078] The sets Q1 and Q2 are represented as follows:

[0079]

[0080]

[0081] Here, we choose a1≤a2=log2n. Then formula (1) can be transformed into:

[0082]

[0083] Among them, 0≤k1,k2≤n-1,φ1,φ2∈{0,1}. Therefore, The operation can be transformed into a simple shift operation. Unlike the original addition quadratic quantization method that selects a1+a2+1=n, in our improved method, we select a1≤a2=log2n, which can reduce our shift operations and reduce resource consumption when implementing hardware. The brief reasoning is as follows: Note that in the sets Q1 and Q2, the elements are If we choose a1+a2+1=n, then a1 and a2 may be greater than log2 n. The probability is less than 1 / 2 n-1 . In this way, in formula 1, the integer coefficient 2 n-1 It should be changed to 2 accordingly k, Obviously, k≥n - 1. In this case, the quantized bit width will be greater than n, and a larger shift range is required, which will bring additional overhead. Selecting a1≤a2 = log2 n can avoid this problem, which enables the quantized data to still be represented by n bits and has a more compact shift range. More importantly, a weight with a smaller bit width can effectively reduce the bit width of the plaintext domain modulus and further streamline our hardware.

[0084] Based on this, the structure of the streamlined multiplier is as Figure 3 shown. The inputs of this multiplier include the source data a, which is an integer with a large bit width; the weight w, which can be represented in the form of sign(w)·(φ1<<k1 + φ2<<k2) after quantization. In actual operations, the parameters involved in the operation are also sign(w), φ1, φ2, k1, k2, etc.; the modulus q, which is the modulus for the modulo operation after multiplication.

[0085] Combined with Figure 2 , the working process of this streamlined multiplier is as follows:

[0086] Step 1, input the source data a, the quantized weight w, and the modulus q.

[0087] Step 2, determine whether the parameter φ1 is 0. If it is not 0, send the value of a shifted left by k1 bits to the adder; otherwise, send 0 to the adder.

[0088] Step 3, determine whether the parameter φ2 is 0. If it is not 0, send the value of a shifted left by k2 bits to the adder; otherwise, send 0 to the adder. This step is carried out simultaneously with Step 2.

[0089] Step 4, perform an addition operation to calculate the sum of the data sent to the adder in the previous steps.

[0090] Step 5, send the result of the previous step and the modulus q to the modulo arithmetic unit to perform a modulo operation.

[0091] Step 6, determine the final output result according to the sign bit of the weight. If sign(w) = 1, output y; otherwise, output y = q - y. Converting the multiplication of the multiplier into at most two shifts greatly improves the calculation efficiency.

[0092] Thus, through the optimized addition quadratic power quantization method, the large number multiplication part before the multiplier can be optimized into shift operations and addition operations, greatly simplifying the operation complexity.

[0093] Based on this multiply-accumulate operation array, the present invention designs a fetching logic that can split the ciphertext. The specific fetching steps are as Figure 5As shown. Taking the convolution operation as an example, assuming the size of the convolution kernel is d×d, after the Img2col and transpose operations, the size of the weight matrix is ​​(number of output channels × (d 2 * number of input channels); the size of the source data matrix is ​​((d 2 * number of input channels) × (convolution output length * convolution output width)), and then encrypt it in the row direction, we can get (d 2 *output channel number) ciphertext polynomials, and the dimension of the ciphertext polynomial is n. According to the parameter u of the multiplication and accumulation array, we can split the ciphertext into several (d 2 u)×u small blocks, and at the same time divide the weights into several u×(d 2 u) small blocks. The numbers in the figure represent the order of operations. As can be seen from the figure, each time a block of ciphertext is taken, all the blocks corresponding to the weight in a column can be traversed and calculated, eliminating the need for repeated transfer of the source data. For weights quantized using additive quadratic quantization, the total data volume is far smaller than the data volume of the ciphertext source data, so the transfer overhead caused by traversing it is very small. Another advantage is that, thanks to the unique encoding method designed in this article, the ciphertext can be split and calculated by referring to the matrix block method. In traditional algorithms, a ciphertext polynomial must be calculated as a whole, which makes its transfer logic relatively complex. With this encoding method, we can transfer blocks without having to complete the calculation of one ciphertext before starting the next, thereby reducing our transfer complexity and improving data transfer efficiency.

[0094] Based on the modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration method and hardware acceleration system we built, the overall workflow is as follows: Figure 5 As shown in the figure, the process of performing the neural network linear layer acceleration operation includes:

[0095] Step 1: Quantification

[0096] The weights in the user-provided neural network model are quantized and transposed to fit the architecture, using an optimized additive quadratic quantization method.

[0097] Step 2: Coding

[0098] On the user side, the source data that needs to be input into the neural network will be rearranged using the img2col method, and the source data will be transposed and encoded into plaintext polynomials by row, specifically using a single instruction multiple data encoding method.

[0099] Step 3: Encryption

[0100] The user side encrypts the encoded plaintext polynomial into the ciphertext corresponding to the homomorphic encryption scheme, and sends it together with the weight to the hardware accelerator for processing and then performing other operations.

[0101] Step 4: Calculation

[0102] After the accelerator obtains the quantized weights and encrypted source data, it starts the neural network linear layer operation under homomorphic encryption conditions.

[0103] The calculation results are fed back to the user end for subsequent operations.

[0104] In step 1, the optimized additive quadratic quantization method is specifically as follows:

[0105] In summary, the modular multiplication simplified homomorphic encryption neural network linear layer hardware acceleration method and hardware acceleration system proposed in the present invention can effectively improve computing efficiency, simplify the computing process and memory access overhead, and have good practical application value.

Claims

1. A modular multiplication-simplified homomorphic encryption neural network linear layer hardware acceleration system, characterized by: The method comprises a homomorphically encrypted ciphertext scalar multiplication-accumulation operation array, wherein the homomorphically encrypted ciphertext scalar multiplication-accumulation operation array comprises a predetermined number of operation units, each operation unit comprises a source data buffer, an output buffer, and a multiplication-accumulation cluster, each multiplication-accumulation cluster comprises a controller, a weight buffer, a predetermined number of sequentially connected multiplication-accumulation operators, and a modular adder. The number of source data buffers, weight buffers and multiplication-accumulation operators is equal, each source data buffer is output to the multiplication-accumulation operator of the same level of each operation unit, and each weight buffer is output to all multiplication-accumulation operators of the operation unit where it is located; The multiplication-accumulation unit consists of a simplified modular multiplier and a modular adder. The input of the modular multiplier includes the source data, quantized weights and modulus, and the input of the modular adder includes the output of the modular multiplier, the accumulated result of the previous stage and the modulus. Each cycle, a set of source data and weights are fed into the multiplication-accumulation unit. After the pipeline is established, the output end can output the accumulated result of each beat in a pipeline. The multiply-accumulate cluster is designed based on an optimized additive quadratic quantization algorithm and includes the following steps: Step S11: Get the original weight data x and quantize it to get an n-bit fixed-point number Q set ={x|x=x1+x2,x1∈Q1,x2∈Q2}∩[-1,1); Among them, the function sign(x) is used to determine the sign of the input value x. When x ≥ 0, the function sign(x) returns 1, otherwise it returns 0; the function findnearest() returns the set Q set The value closest to x; set Q set The elements in are obtained by adding the elements in two other specific sets Q1 and Q2 to each other; Step S12: Take a1≤a2=log2n, and transform the above formula into: Where, 0≤k1,k2≤n-1,φ1,φ2∈{0,1}.

2. The system according to claim 1, wherein The calculation process of the multiplication-accumulation cluster is: Step S3a, input source data a, quantization weight w and modulus q; Step S3b, judging whether the parameter φ1 is 0, if not 0, the value of the source data a after being shifted left by k1 bits is sent to the adder, otherwise 0 is sent to the adder; judging whether the parameter φ2 is 0, if not 0, the value of the source data a after being shifted left by k2 bits is sent to the adder, otherwise 0 is sent to the adder; Step S3c, performing an addition operation to calculate the sum of the data sent to the adder in the previous step; Step S3d, sending the result of the previous step and the modulus q to the modulus operator to perform a modulus operation; Step S3e: Determine the final output result according to the sign bit of the weight. If sign(w)=1, output the parameter y; otherwise, output y=qy.

Images