A homomorphic convolution acceleration method based on approximate fast fourier transform

Abstract

The application provides a homomorphic convolution acceleration method based on approximate fast Fourier transform, and belongs to the technical field of algorithm optimization of privacy calculation.The method mainly comprises client encryption, server-side calculation and client decryption, uses the fault tolerance feature of homomorphic convolution to perform optimized calculation on the homomorphic convolution, replaces the number theory transform NTT with fast Fourier transform FFT, and introduces an approximate method to further reduce the bit width, so as to reduce the hardware cost of each operation; in the process of determining the approximate FFT bit width, a multi-objective design space exploration method is adopted, different bit widths are used in multiple stages of fast Fourier transform, a space evaluation method based on a lookup table is designed, and multi-objective space exploration is performed on the space evaluation method, so that a design balance between calculation accuracy and power consumption is realized. The application is suitable for any convolution layer, reduces the overall overhead of homomorphic convolution, improves the calculation efficiency, and has a wide application prospect.

Description

Technical Field

[0001] This invention belongs to the field of algorithm optimization for privacy computing, specifically involving a method for accelerating homomorphic convolution using approximate fast Fourier transform. Background Technology

[0002] Privacy protection has become a major concern when deploying deep neural networks (DNNs) in the cloud. Homomorphic encryption (HE) has recently been proposed and has attracted widespread attention. By encrypting data into ciphertext polynomials, HE allows computation on encrypted data without revealing any information about the data itself. To apply HE to private DNN inference, there are two main approaches: fully homomorphic encryption (FHE) schemes and hybrid HE / two-party computation (2PC) schemes. The main difference between these two lies in the implementation of the nonlinear activation function. The hybrid scheme uses a two-party computation protocol, which helps avoid activation function approximation and costly bootstrapping operations in FHE schemes. Therefore, this invention focuses on optimizing the hybrid scheme.

[0003] Inference in privacy-preserving neural networks remains significantly slower than plaintext inference, primarily due to higher computational costs. Homomorphic convolution (HConv) becomes the main computational bottleneck compared to the computation of nonlinear layers, and its computational flow is as follows: Figure 1As shown, the client has the input activation vector X of the convolutional layer, encodes it into a polynomial, encrypts it, and then sends it to the server for preprocessing and N-point NTT transformation calculation. The server has the weight vector W, encodes it into a polynomial, preprocesses it, and performs NTT transformation calculation. It then performs a dot product with the encrypted NTT transformation result and sends the result back to the client for decryption and post-processing. In this way, the client can obtain the calculated data without leaking the input data, and the client does not know the neural network information on the server. Specifically, the polynomial calculation of weights W and X is accelerated by number theoretic transform (NTT). The high computational cost of homomorphic convolution is due to the repeated calculation of numerous number theoretic transforms and their inverse number theoretic transforms (INTTs), especially in the calculation of the weight polynomial. Although the weight polynomial in the NTT domain can be pre-computed and stored, this leads to significant memory overhead. For example, storing all the weights of a 4-bit quantized ResNet-50 in the NTT domain requires 23GB of memory, which results in memory consumption that is more than 1,000 times higher than usual.

[0004] In recent years, various homomorphic encryption accelerators have been proposed to speed up costly number-theoretic transformations. Some studies focus on optimizing the data flow and avoiding pauses between stages to improve parallelism [1][2]. Other studies achieve acceleration by decomposing large-scale NTTs into multiple smaller NTTs [3]. Although these accelerators have made promising progress in speedup, they generally face high area and power consumption costs. For example, the accelerator proposed in [3] has an area exceeding 150 mm². 2 The power consumption is approximately 100W. Research has found that homomorphic privacy inference possesses three levels of fault tolerance, stemming from homomorphic decryption, neural network quantization, and the fault tolerance of the network itself. However, number-theoretical transformations are based on modular arithmetic, and the modulo operation renders them intolerant of faults, even amplifying the proportion of error to the original data. The Fast Fourier Transform (FFT) not only accelerates polynomial multiplication but also has the potential for approximation. Therefore, researching how to replace number-theoretical transformations with the Fast Fourier Transform and further reduce computational overhead through approximation to improve the computational efficiency of homomorphic convolution is of great significance.

[0005] Fast Fourier Transform (FFT) and number-theoretic transforms can often be used to accelerate polynomial multiplication. FFT and NTT share the same dataflow structure and employ the Cooley-Turkey (CT) butterfly algorithm, such as... Figure 2 As shown, the complexity of polynomial multiplication is reduced from O(N) to O(N). 2The computation time is reduced to O(NlogN). The difference between FFT and NTT lies in the data type and the arithmetic units used. FFT processes data with complex floating-point coefficients, requiring complex adders and multipliers, while NTT processes polynomials on a ring, requiring modular adders and multipliers. After encoding and encryption, the computation is performed on the polynomial ring. Therefore, traditional schemes use NTT to calculate the exact multiplication results of the weights and activation polynomials. In contrast, directly using FFT instead of NTT due to the data type issue will introduce computational errors unless the data width is large enough. Mainstream homomorphic convolution accelerators are based on NTT and deconvolution theorems, such as... Figure 1 As shown. However, NTT itself is more sensitive to errors, and the rotation factor varies with different moduli, while FFT does not have these problems. Therefore, based on the fault tolerance in homomorphic privacy inference applications, it is of great significance to study a method that uses Fast Fourier Transform instead of Number Theory Transform and introduces approximate homomorphic convolution to accelerate the process.

[0006] [1] MS Riazi, K. Laine, B. Pelton, and W. Dai, "HEAX: An architecture for computing on encrypted data," in Proceedings of the twenty-fifth inter-national conference on architectural support for programming languages ​​and operating systems, 2020, pp.1295–1309.

[0007] [2]X.Ren,Z.Chen,Z.Gu,Y.Lu,R.Zhong,W.-J.Lu,J.Zhang,Y.Zhang,H.Wu,X.Zheng et al., "CHAM: Customized homomorphic encryption accelerator for fastmatrix-vectorproduct," in 2023 60thACM / IEEE DesignAutomation Conference(DAC).IEEE,2023,pp.1–6.

[0008] [3] N.Samardzic, A.Feldmann, A.Krastev, S.Devadas, R.Dreslinski, C.Peikert, andD.Sanchez, "F1: Afast and programmable accelerator for fully homomorphicencryption," inMICRO-54:54thAnnual IEEE / ACM International Symposium onMicroarchitecture,2021,pp.238–252. Summary of the Invention

[0009] To address the problems existing in the prior art, this invention proposes a homomorphic convolution acceleration method based on approximate fast Fourier transform. By utilizing the fault-tolerant characteristics of homomorphic convolution, the number-theoretic transformation of privacy inference is replaced with fast Fourier transform, and an approximation method is introduced to further reduce the bit width, thereby reducing the hardware cost per operation.

[0010] The technical solution of the present invention is as follows:

[0011] A method for accelerating homomorphic convolution based on approximate Fast Fourier Transform (FFT) is characterized by utilizing the fault-tolerant nature of homomorphic convolution to replace the number-theoretic transformation for privacy inference with FFT, and introducing an approximation method to further reduce the bit width, thereby reducing the hardware cost per operation. During the introduction of the approximation, a parameter space is constructed, and different bit widths are used in multiple stages of the FFT and explored to find the optimal balance between power consumption and accuracy. The specific steps are as follows:

[0012] Step 1: Client-side encryption

[0013] The client encodes the input activation vector X of the convolutional neural network into a polynomial of length N and encrypts it according to the hybrid HE / 2PC encryption protocol; the resulting ciphertext polynomial X... EN Send the data to the server for homomorphic convolution.

[0014] Step 2: Server-side calculation

[0015] The server-side calculations include the following steps:

[0016] S1) Input the ciphertext polynomial X EN Preprocessing of the weighted polynomial W; the preprocessing includes two steps: folding and rotating, resulting in the complex form of the input polynomial X. rot Weighted polynomial W rot ;

[0017] S2) Input polynomial X rot Weighted polynomial W rot The Fast Fourier Transform (FFT) is used to obtain the input polynomial and weight polynomial in the FFT domain.

[0018] S3) Perform a dot product between the input polynomial and the weight polynomial in the FFT domain to obtain the processed result Y. EN ;

[0019] Step 3: Client Decryption

[0020] The client receives the processing result Y from the server. EN Then, Y is obtained by decryption calculation according to the encryption protocol. fft Then, the corresponding IFFT calculation is performed to obtain Y. rot The decrypted polynomial is then post-processed to obtain the polynomial Y. appr The coefficients are rounded and modulo q are used to obtain the output polynomial Y of the homomorphic convolution.

[0021] Furthermore, in the aforementioned homomorphic convolution acceleration method based on approximate fast Fourier transform, the preprocessing in step S1) of the second step server-side computation includes two operations: folding and rotation, specifically:

[0022] The S1-1) folding operation will result in the elements belonging to Z[X] / (X) being folded. N +1) mod q polynomial X EN And W, respectively transformed into Complex polynomial X in the form of fold and W fold ;

[0023] S1-2) Rotation operation, which rotates the folded N / 2-order complex polynomial X fold and W fold X is obtained by multiplying it by a specific rotation factor. rot and W rot The rotation factor is

[0024] Furthermore, in the aforementioned homomorphic convolution acceleration method based on approximate fast Fourier transform, step S2) of the second step server-side computation involves inputting the polynomial X. rot Weighted polynomial W rotThe Fast Fourier Transform (FFT) is implemented using a butterfly network. For an N / 2-point FFT, the input consists of N / 2 complex numbers, and the computation is performed at log₂N / 2 levels. Each level contains N / 4 butterfly units, and within each butterfly unit, complex multiplication and addition are performed pairwise. In determining the bit width of the FFT computation, a multi-objective design space exploration is used to approximate the intermediate data and rotation factors at each level of the butterfly network, converting the original floating-point numbers to fixed-point numbers and reducing the bit width. Specifically, the following steps are included:

[0025] S2-1) Constructing the parameter space: The N / 2-point FFT butterfly network has a total of log2N / 2 stages. Complex multiplication is only necessary in the last log2N / 2-2 stages. The power consumption of complex multiplication is related to the intermediate data bit width dw. i and the bit width of the rotation factor tw i Related to, among them Selected The dimension parameter set Ω is represented as:

[0026] Each parameter in each dimension represents the bit width of the intermediate data or twitch factor input to the complex multiplier, ranging from b... L to b H upper limit b H This indicates the highest supported bit width, while the lower limit is b. L This indicates the minimum bit width allowed for the FFT calculation error tolerance;

[0027] S2-2) Design Space Evaluation: For samples taken in the constructed parameter space, construct the corresponding approximate FFT, and perform error estimation and hardware overhead calculation; for error evaluation, use multiple simulations; for hardware overhead evaluation, obtain the estimated power consumption by looking up the sampled approximate FFT parameters through a lookup table.

[0028] S2-3) Multi-objective design space exploration: Searching for Pareto optimal design using multi-objective Bayesian optimization methods;

[0029] Through steps S2-1) to S2-3), a set of Pareto fronts for the power consumption and error of the approximate FFT are obtained. Based on the maximum error tolerance preset in practical applications, a set of approximate FFT parameters with the lowest power consumption is selected. Approximate fast Fourier transforms are then performed on the input polynomial and weight polynomial according to these parameters to obtain the input polynomial and weight polynomial in the FFT domain.

[0030] Furthermore, in the aforementioned homomorphic convolution acceleration method based on approximate fast Fourier transform, the second step of server-side computation, specifically step S2-2), involves design space evaluation:

[0031] For error assessment, a multi-simulation approach is adopted. First, a set of corresponding activation tensors and weight tensors are randomly obtained from the real network input. In Matlab, the input values ​​are encoded according to an encryption protocol to obtain a plaintext polynomial. The plaintext activation polynomial is then encrypted in the SEAL library to obtain a ciphertext polynomial. Next, homomorphic convolution calculation based on the current approximate FFT is performed in Matlab, and the result is sent back to the SEAL library for decryption. The decrypted plaintext polynomial is then post-processed in Matlab, including coefficient decimation, activation function, and weighting, to obtain the final calculated value. This value is then compared with the accurate calculated value to obtain the calculation error.

[0032] For hardware cost assessment, the estimated power consumption is obtained by using the sampled approximate FFT parameters through a lookup table; a lookup table-based LUT method is adopted to obtain hardware cost estimates for different data width configurations from the LUT by aggregating the cost of the pre-synthesized butterfly cells.

[0033] Furthermore, in the aforementioned homomorphic convolution acceleration method based on approximate fast Fourier transform, the post-processing of the decrypted polynomial in the third step of client decryption includes two steps: rotation and unfolding, specifically:

[0034] 1) Rotation operation, transforming the N / 2 level complex polynomial Y rot Dot product with a specific rotation factor, corresponding rotation factors to be multiplied Get Y fold ;

[0035] 2) Expand the operation to include... Complex polynomial Y in form flod Revert to R[X] / (X) N +1) form Y appr .

[0036] The technical effects of this invention are as follows:

[0037] This invention presents an accelerated homomorphic convolution method based on approximate Fast Fourier Transform (FFT). It optimizes the computation of homomorphic convolution by replacing the Number Theory Transform (NTT) with the Fast Fourier Transform (FFT) and introducing an approximation method to further reduce the bit width, thereby reducing hardware costs per operation. In determining the approximate FFT bit width, a multi-objective design space exploration method is employed. Different bit widths are used in multiple stages of the FFT, and a lookup table-based space evaluation method is designed and subjected to multi-objective space exploration to achieve a design balance between computational accuracy and power consumption. This invention is applicable to arbitrary convolutional layers, reducing the overall overhead of homomorphic convolution and improving computational efficiency. In practical applications, different approximation levels are determined based on acceptable levels of accuracy to achieve optimal hardware efficiency. Compared to common NTT-based homomorphic convolution schemes, this invention's approximate FFT-based homomorphic convolution can achieve more than 10 times power reduction while maintaining acceptable error growth. Attached Figure Description

[0038] Figure 1 The flowchart shows the homomorphic convolution implementation based on number theory transformation in the hybrid HE / 2PC encryption scheme.

[0039] Figure 2 A schematic diagram illustrating the implementation of the butterfly network algorithm for the 8-point number theory transform in the Fast Fourier Transform algorithm.

[0040] Figure 3 This is a flowchart of the homomorphic convolution based on the Fast Fourier Transform proposed in this invention.

[0041] Figure 4 This is a flowchart illustrating the design space evaluation process in the multi-objective design space exploration proposed in this invention. Detailed Implementation

[0042] The present invention will be further clearly and completely described below with reference to the accompanying drawings and specific embodiments.

[0043] This invention proposes a method for accelerating homomorphic convolution based on approximate fast Fourier transform. The overall process is as follows: Figure 3 As shown, the process mainly consists of three steps: the client encrypts the data and sends it to the server; the server performs homomorphic computation on the encrypted data; and the client receives the computed data, decrypts it, and obtains the final computation result. The following is a detailed explanation with reference to the attached diagram.

[0044] Step 1: Client-side encryption

[0045] The client encodes the input activation vector X of the convolutional neural network into a polynomial of length N and encrypts it according to the hybrid HE / 2PC encryption protocol; the resulting ciphertext polynomial X... EN Send the data to the server for homomorphic convolution.

[0046] Step 2: Server-side calculation

[0047] The server-side calculations include the following steps:

[0048] S1) Input the ciphertext polynomial X EN Preprocessing of the weighted polynomial W; X EN Both W and Z[X] / (X) belong to the polynomial ring Z[X] / (X) N +1)modq, where the series of the polynomial is N and the coefficients are integers modulo q; the preprocessing consists of two steps: folding and rotation.

[0049] The S1-1) folding operation will result in the elements belonging to Z[X] / (X) being folded. N +1) mod q polynomial X EN And W, respectively transformed into Complex polynomial X in the form of fold and W fold Specifically, this is achieved by taking the N coefficients {X} of the input polynomial. EN [j]}, j=0,…,N-1, mapped to N / 2 complex coefficients {X} of an N / 2 order complex polynomial. fold [k]}, Where X fold [k] = X EN [2j]+X EN [2j+1]i, where i is the imaginary unit, X EN [2j] is the real part, X EN [2j+1] is the imaginary part; for example, X EN =x 0 +2x 1 +3x 2 +4x 3 +5x 4 +6x 5 +7x 6 +8x 7 The folded complex polynomial is X fold =(1+2i)x 0 +(3+4i)x 1 +(5+6i)x 2 +(7+8i)x 3 Similarly, W can be folded and transformed into W. fold .

[0050] S1-2) Rotation operation, which rotates the folded N / 2-order complex polynomial X fold X is obtained by multiplying by a specific rotation factor. rot The folded N / 2 level complex polynomial W fold W is obtained by dot product with a specific rotation factor.rot The corresponding rotation factor for multiplication is Right now For example:

[0051] S2) Input polynomial X rot Weighted polynomial W rot The Fast Fourier Transform (FFT) is implemented using a butterfly network, where each butterfly unit contains one complex multiplication and two complex addition / subtraction operations. Figure 2 The diagram illustrates the computation flow of an 8-point FFT butterfly network. For the more general N / 2-point FFT computation, the input consists of N / 2 complex numbers, and the computation is performed in log2N / 2 stages. Each stage contains N / 4 butterfly units. Within each butterfly unit, complex multiplication and addition are performed pairwise between data points, as shown below. Figure 2 As shown, the intermediate data m[i] and m[j] are related to the rotation factor. Perform butterfly calculations to obtain M[i] and M[j]:

[0052]

[0053] In the FFT calculation process, this invention proposes to approximate the intermediate data and rotation factors, converting the original floating-point quantization into fixed-point quantization and reducing the bit width. Reducing the approximate bit width significantly reduces the hardware overhead of the multiplier, but introduces some error into the calculation. As mentioned earlier, neural network privacy inference itself has a certain degree of fault tolerance; therefore, determining the degree of approximation to balance hardware cost and computational accuracy is a key issue. This invention proposes using multi-objective design space exploration (DSE) to determine the fixed-point bit width of the intermediate data and rotation factors at each stage in the FFT calculation process, mainly including the following steps:

[0054] S2-1) Constructing the parameter space: As mentioned earlier, the N / 2-point FFT butterfly network has a total of log2N / 2 stages. Importantly, complex multiplication is only necessary in the final log2N / 2-2 stage. In the first stage, since all twitch factors are 1, no multiplication is required; in the second stage, since the twitch factors are 1 and -j, simple operations such as data swapping and sign flipping are involved. The power consumption of complex multiplication is related to the intermediate data bit width dw. i and the bit width of the rotation factor tw i Related to, among them Therefore, the selected The dimension parameter set Ω is represented as: Each parameter in each dimension represents the bit width of the intermediate data or twitch factor input to the complex multiplier, ranging from b... L to bH Upper limit b H This indicates the highest supported bit width, while the lower limit is b. L This represents the minimum bit width allowed for the FFT computation error tolerance. For example, when N = 4096, taking a traditional NTT-based homomorphic convolution with a bit width of 39 bits as an example, the upper limit b of the bit width of the replaced FFT intermediate data and rotation factor is... H It is generally set to 25 bits, while the lower limit b L This represents the minimum bit width allowed for FFT computation error tolerance. Coarse-grained experimental results show that when the bit width is reduced to 15 bits, the introduced computational error leads to very low network accuracy. Therefore, b L It can be set to 15 bits.

[0055] S2-2) Design Space Evaluation: For samples taken in the constructed parameter space, construct the corresponding approximate FFT, perform error estimation and hardware overhead calculation, as shown in the flowchart below. Figure 4 As shown. For error assessment, this invention employs multiple simulations. First, a set of corresponding activation and weight tensors is randomly obtained from the real network input. In Matlab, this set of input values ​​is encoded according to an encryption protocol to obtain a plaintext polynomial. The plaintext activation polynomial is then input into the SEAL library for encryption to obtain a ciphertext polynomial. Next, homomorphic convolution calculation based on the current approximate FFT is implemented in Matlab, and the result is sent back to the SEAL library for decryption. The decrypted plaintext polynomial undergoes further post-processing in Matlab, including coefficient decimation (the reverse of encoding), activation function addition, and weighting, to obtain the final calculated value. This value is then compared with the accurate calculated value to determine the computational error. Simultaneously, for hardware overhead assessment, the estimated power consumption is obtained by using a lookup table to calculate the approximate FFT parameters obtained from the sampled parameters. Estimating power consumption for each configuration through register-transfer-level (RTL) simulation can be very time-consuming. To quickly estimate hardware costs, a lookup table (LUT)-based method is used. This method aggregates the costs of pre-synthesized butterfly cells to obtain hardware cost estimates for different data width configurations from the LUT. For example, this invention constructs a two-dimensional lookup table containing the power consumption of two multipliers with input bit widths ranging from 15 to 25 bits. Then, based on the bit width parameters in the samples, the power consumption of the approximate FFT is calculated by looking up the table. Although the LUT-based method does not provide designers with sufficiently accurate hardware cost estimates, it enables rapid evaluation, supporting subsequent multi-objective exploration.

[0056] S2-3) Multi-objective design space exploration: This invention has two design objectives: hardware cost and computational accuracy. Based on the above evaluation method, a multi-objective Bayesian optimization method is used to search for the Pareto optimal design.

[0057] Using the design space exploration method described above, a set of Pareto fronts regarding the power consumption and error of the approximate FFT can be obtained. Based on the maximum tolerable error in practical applications, a set of approximate FFT parameters with the lowest power consumption can be selected. Then, approximate fast Fourier transforms are performed on the input polynomial and the weight polynomial according to these parameters to obtain two polynomials in the FFT domain.

[0058] S3) Perform a dot product of the input polynomial and the weight polynomial obtained in the FFT field in the previous step to obtain Y. EN .

[0059] Step 3: Client-side decryption. For example... Figure 3 As shown, the client receives the processing result Y from the server. EN Then, Y is obtained by decryption calculation according to the encryption protocol. fft Then, the corresponding IFFT calculation is performed to obtain Y. rot Corresponding to the preprocessing above, post-processing is also needed to obtain the final result of the decrypted polynomial, which includes two steps: rotation and expansion.

[0060] 1) The rotation operation is to rotate the N / 2 level complex polynomial Y rot Perform a dot product with a specific rotation factor. The rotation factors to be multiplied are... Get Y fold .

[0061] 2) Expanding the operation will belong to Complex polynomial Y in form flod Revert to R[X] / (X) N +1) form. Specifically, the implementation involves assigning N / 2 complex coefficients {Y} to the N / 2-order complex polynomial. fold [k]}, Among them, Y fold [k] = Y appr [2j]+Y appr [2j+1]i, where i is the imaginary unit, Y appr [2j] is the real part, Y appr [2j+1] is the imaginary part, which is mapped to an Nth-order polynomial Y. appr N coefficients {Y appr [j]},j=0,…,N-1.

[0062] The post-processed result is R[X] / (X) N A polynomial of the form +1) Y appr The coefficients need to be rounded and modulo q to transform them into Z[X] / (X). N +1) mod q form, to obtain the output polynomial Y of the homomorphic convolution, such as Figure 3 As shown.

[0063] The above method can optimize the computation of homomorphic convolution. Compared with common NTT-based homomorphic convolution schemes, the homomorphic convolution based on approximate FFT in this invention can achieve a power consumption reduction of more than 10 times while maintaining an acceptable error growth.

[0064] Finally, it should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. A method for accelerating homomorphic convolution based on approximate fast Fourier transform, characterized in that, Leveraging the fault-tolerant nature of homomorphic convolution, the number-theoretic transformation for privacy inference is replaced with a Fast Fourier Transform (FFT). An approximation method is introduced to further reduce the bit width, thereby decreasing hardware cost per operation. During the approximation process, a parameter space is constructed, and different bit widths are used in multiple stages of the FFT for exploration to find the optimal balance between power consumption and accuracy. The specific steps are as follows: Step 1: Client-side encryption The client, following a hybrid HE / 2PC encryption protocol, transmits the input activation vector of the convolutional neural network. Encoded as a length of The polynomial is then encrypted; the resulting ciphertext polynomial is... Send the data to the server for homomorphic convolution. Step 2: Server-side calculation The server-side calculations include the following steps: S1) Input ciphertext polynomial and weighted polynomial The preprocessing includes two steps: folding and rotating. After preprocessing, the input polynomial in complex form is obtained. Weighted polynomial ; S2) Input polynomial Weighted polynomial The Fast Fourier Transform (FFT) is used to obtain the input polynomial and weight polynomial in the FFT domain. S3) Perform a dot product between the input polynomial and the weight polynomial in the FFT domain to obtain the processed result. ; Step 3: Client Decryption The client receives the processing result from the server. Then, decryption calculations are performed according to the encryption protocol to obtain... Then, the corresponding IFFT calculation is performed to obtain The decrypted polynomial is then post-processed to obtain the polynomial. Rounding and modulo the coefficients q The operation ultimately yields the output polynomial of the homomorphic convolution. Y ; Step S2 in the second server-side calculation: Input polynomial Weighted polynomial The Fast Fourier Transform (FFT) is implemented using a butterfly network. The FFT is calculated for N / 2 points, with the input being... N / 2 complex numbers, proceed Level calculation, each level of calculation contains 1 N / 4 butterfly unit, within each butterfly unit, complex multiplication and addition are performed pairwise between data; in determining the bit width of the FFT calculation, a multi-objective design space exploration is used to approximate the intermediate data and rotation factor of each level of the butterfly network, converting the original floating-point quantization into fixed-point quantization and reducing the bit width, specifically including the following steps: S2-1) Constructing the parameter space: The N / 2-point FFT butterfly network has Complex multiplication only occurs at the end of the series. In the intermediate stage, it is necessary to consider the power consumption and intermediate data bit width of complex multiplication. and the bit width of the rotation factor Related to, among them ; Selected Dimensional parameter set Represented as: Each parameter in each dimension represents the bit width of the intermediate data or twitch factor input to the complex multiplier, with values ​​ranging from... arrive upper limit Indicates the highest supported bit width, and the lower limit. This indicates the minimum bit width allowed for the FFT calculation error tolerance; S2-2) Design Space Evaluation: For samples taken in the constructed parameter space, construct the corresponding approximate FFT, perform error estimation and hardware overhead calculation; for error evaluation, use multiple simulations; for hardware overhead evaluation, obtain the estimated power consumption by looking up the sampled approximate FFT parameters through a lookup table. S2-3) Multi-objective design space exploration: Searching for Pareto optimal designs using multi-objective Bayesian optimization methods; Through steps S2-1) to S2-3), a set of Pareto fronts for the power consumption and error of the approximate FFT are obtained. Based on the maximum error tolerance preset in practical applications, a set of approximate FFT parameters with the lowest power consumption is selected. Approximate fast Fourier transforms are then performed on the input polynomial and weight polynomial according to these parameters to obtain the input polynomial and weight polynomial in the FFT domain.

2. The homomorphic convolution acceleration method based on approximate fast Fourier transform as described in claim 1, characterized in that, The preprocessing in step S1) of the second server-side calculation includes two operations: folding and rotating, specifically: S1-1) Folding operation, which belongs to polynomial and , respectively transformed into Complex polynomials of the form and ; S1-2) Rotation operation, to rotate the folded... N / 2nd level complex polynomial and Dot product with a specific rotation factor to obtain and The rotation factor is .

3. The homomorphic convolution acceleration method based on approximate fast Fourier transform as described in claim 1, characterized in that, Step S2-2) Design Space Assessment specifically includes: For error evaluation, multiple simulations are used. First, a set of corresponding activation tensors and weight tensors are randomly obtained from the real network input. Then, in Matlab, the input values ​​are encoded according to the encryption protocol to obtain the plaintext polynomial. The plaintext polynomial of the activation value is input into the SEAL library for encryption to obtain the ciphertext polynomial; then, the homomorphic convolution calculation based on the current approximate FFT is implemented in Matlab, and the result is sent back to the SEAL library for decryption. The decrypted plaintext polynomial is then post-processed in Matlab, including coefficient extraction, activation function, and weighting, to obtain the final calculated value. This value is then compared with the accurate calculated value to obtain the calculation error. For hardware cost assessment, the estimated power consumption is obtained by using the sampled approximate FFT parameters through a lookup table; a lookup table-based LUT method is adopted to obtain hardware cost estimates for different data width configurations from the LUT by aggregating the cost of the pre-synthesized butterfly cells.

4. The homomorphic convolution acceleration method based on approximate fast Fourier transform as described in claim 1, characterized in that, The third step of client-side decryption involves post-processing of the decrypted polynomial, including two steps: rotation and expansion. Specifically: Rotation operation, N / 2nd level complex polynomial Dot product with a specific rotation factor, corresponding to the rotation factor being multiplied. son ,get ; Expand the operation, which will belong to Complex polynomials of the form Return Form .

Images