An image processing hardware accelerator and image processing method based on homomorphic encryption

Abstract

The present invention discloses an image processing hardware accelerator and an image processing method based on homomorphic encryption. The processor includes: an FV module, an image conversion module, a data storage module and an HDMI display module; the FV module is an internal module for processing image data encryption and decryption; the data storage module is used for storing and outputting data; the image conversion module splits, expands and caches the received original image data and outputs it to the FV module, and receives the data processed by the FV module and processes and outputs it to the display module; the HDMI module is a display module for image data. The present invention designs a hardware accelerator using an FPGA development board, which greatly shortens the data processing time, thereby improving the efficiency of image grayscale processing in homomorphic encryption.

Description

Technical Field

[0001] The present invention belongs to the technical field of cryptography, and specifically relates to an image processing hardware accelerator based on homomorphic encryption and an image processing method thereof. Background Art

[0002] With the advancement of science and technology and the development of network information, privacy protection in mobile Internet has become increasingly important, and network security has emerged. Traditional encryption schemes can reduce privacy leakage to a great extent, but their biggest limitation is that they must be decrypted when processing and analyzing sensitive data. However, there are a large number of fields that require data analysis and processing, including medicine, law, manufacturers, finance, etc. If the data is analyzed directly without using encryption keys, the goal can be achieved while ensuring the privacy of the data, and these problems can be easily solved.

[0003] Therefore, Rivest et al. proposed the concept of homomorphic encryption in the 1970s. This concept was realized in 2009. Craig Gentry designed the first fully homomorphic encryption mechanism in his doctoral thesis at Stanford, which perfectly solved the above problems. However, this technology means that the algorithm is very slow and may have very high storage requirements to meet the requirements of full homomorphism. For example, IBM released an improved version of the HELIb C++ library for homomorphic encryption in 2018. This version is 25-75 times faster than the previous version and 2 million times faster than the original version released three years ago. However, the mathematical operation speed of the original version is about 100 trillion times slower than performing the same operation on the corresponding plaintext. This means that the new and improved version is still 1 million times slower than the average speed of plaintext operations. The effect of a million-fold slowdown is quite terrible. It takes an average of 11.5 days to perform a one-second calculation using plaintext using the 2018 version of HElib. It can be seen that the existing homomorphic encryption algorithm is relatively complex and the data processing efficiency is too low to meet people's needs for data processing. Summary of the invention

[0004] In order to solve the deficiencies existing in the above-mentioned prior art, the present invention proposes an image processing hardware accelerator and an image processing method based on homomorphic encryption, in order to reduce data processing time, shorten the circuit operation cycle, and thus improve the efficiency of image grayscale processing in homomorphic encryption.

[0005] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical scheme:

[0006] The characteristics of the image processing hardware accelerator based on homomorphic encryption of the present invention include: FV module, image conversion module, data storage module and HDMI module;

[0007] The FV module is composed of a key generation module, an encryption module, a calculation module, a decryption module, and a control module;

[0008] The key generation module is composed of a private key generation module, a public key generation module and a public key preprocessing module;

[0009] The private key generation module is provided with a private key storage device and stores private key data;

[0010] The public key generation module is provided with a second public key storage, and the second public key storage stores a second public key and noise data;

[0011] The public key preprocessing module is provided with a public key weight memory and a parameter memory;

[0012] The control module is provided with a data memory and an instruction code memory, and the instruction code memory stores a Y-bit data;

[0013] The private key memory outputs the private key data to the public key generation module and the control module respectively;

[0014] After receiving the private key data, the public key generation module reads the second public key and noise data in the second public key storage, and obtains the first public key after calculating with the private key data, and then transmits the first public key and the second public key to the public key preprocessing module;

[0015] The public key preprocessing module reads the randomly generated N-bit public key weight u from the public key weight memory, and calculates the public key weight u with the first public key and the second public key to obtain a first weighted public key and a second weighted public key, and outputs them to the encryption module;

[0016] The image conversion module performs truncation and expansion on the image data of the data storage module or external input to obtain plain text data and outputs the plain text data to the encryption module;

[0017] The encryption module receives the first weighted public key, the second weighted public key and the plaintext data, and reads the noise data from the second public key memory and the parameter data from the parameter memory, so as to obtain the first encrypted data after computing together and output them to the computing module and the data memory for storage respectively;

[0018] The encryption module obtains second encrypted data after performing calculations based on the second weighted public key and the noise data, and outputs the second encrypted data to the calculation module;

[0019] The control module uses the first two bits of the Y-bit data in the instruction code memory as a selection instruction for addition or multiplication, and outputs the remaining bits of data as coefficient data to the operation module, and outputs the private key data to the decryption module;

[0020] The operation module operates the first encrypted data and the second encrypted data respectively. If the selected instruction is a multiplication instruction, the first encrypted data and the second encrypted data are multiplied by the coefficient data respectively to obtain corresponding results and send them to the data memory for temporary storage; if the selected instruction is an addition instruction, all the first encrypted data are multiplied and then added to obtain the first encrypted grayscale data, all the second encrypted data are multiplied and then added to obtain the second encrypted grayscale data, and then the first encrypted grayscale data is output to the data memory for storage, and the first encrypted grayscale data and the second encrypted grayscale data are output to the decryption module;

[0021] The decryption module receives the first encrypted grayscale data, the second encrypted grayscale data and the private key data and generates decrypted data after performing calculations and outputs the decrypted data to the image conversion module;

[0022] The image conversion module shifts the decrypted data, generates grayscale plaintext data, and then splices it into an X-bit binary data and outputs it to the data storage module for storage;

[0023] The image conversion module reads the first encrypted grayscale data and the first encrypted data in the data storage and performs bit processing respectively, and then outputs the encrypted grayscale data and the encrypted data accordingly and transmits them to the data storage module for storage;

[0024] The data storage module sends the stored data to the HDMI module, so that the data is output to the display screen for display.

[0025] The image processing method based on homomorphic encryption of the present invention is characterized in that data processing is performed according to the following steps:

[0026] Step 1: Read the ciphertext modulus q and the plaintext modulus t, and generate a bitwise AND parameter q_value and two judgment parameters half_t and half_q;

[0027] Step 2: Key Generation:

[0028] Step 2.0: Read the private key sk, the second public key pk2 and the noise e; define and initialize the first public key pk1=0, the variable m1=0 and the variable n1=0, define d as the number of pixels processed at a time, and define the variables i1 and j1;

[0029] Step 2.1: Initialize i1=d;

[0030] Step 2.2: Initialize j1=d;

[0031] Step 2.3: Read the i1th value of the second public key pk2;

[0032] Step 2.4: Read the j1th value of the private key sk;

[0033] Step 2.5: Determine whether the remainder of m1+n1 divided by d is 0; if it is 0, assign pk2+e to pk2; otherwise, assign -pk2+e to pk2;

[0034] Step 2.6: After assigning j1-1 to j1 and n1+1 to n1, determine whether n1 is less than d. If so, return to step 2.4 and execute sequentially. Otherwise, execute step 2.7.

[0035] Step 2.7: Assign i1-1 to i1, assign m1+1 to m1, and determine whether m1 is less than d. If so, return to step 2.2 and execute sequentially; otherwise, output the first public key pk1;

[0036] Step 2.8: After generating an N-bit binary number u as the private key sk, the first public key pk1 and the second public key pk2 are respectively operated according to the process of steps 2.1 to 2.7 to obtain the first weighted public key pk 1-u and the second weighted public key pk 2-u ;

[0037] Step 3: Plaintext generation:

[0038] Step 3.1: Get the number of pixels R of the input image, define the number of pixels to be processed at a time as d, the pixel period of data calculation as poly_num, and round up the result of dividing R by d and assign it to poly_num;

[0039] Step 3.2: Expand the data bit width of each pixel in the input image containing R pixels into an M-bit binary number and use it as the plaintext data message;

[0040] Step 4: Ciphertext generation:

[0041] Step 4.1: define and initialize the first address addr1=d, define the first noise e1 and the second noise e2 and assign the noise e to the first noise e1 and the second noise e2 respectively; define the first encrypted data ct1 and initialize ct1=0; define the first encrypted data ct2 and initialize ct2=0;

[0042] Step 4.2: Use formula (1) to calculate the first noise e1 and the first weighted public key pk 1-u Operate with the plaintext data message to obtain the first encrypted data process value ct1_mod_din:

[0043] ct1_mod_din=pk 1_u[addr1]+e1[addr1]+message[addr1]×(q / t); (1)

[0044] Step 4.3: Determine whether the highest bit of ct1_mod_din is equal to 0. If so, set the first encrypted registered data mid1 = ct1_mod_din; otherwise, set the first encrypted registered data mid1 to be ct1_mod_din minus 1 and then inverted;

[0045] Step 4.4: Performing a bitwise AND operation on the bitwise AND parameter q_value and mid1-q to obtain the first encrypted data ct1;

[0046] Step 4.5: Use formula (2) to calculate the second noise e2 and the second weighted public key pk 2-u Perform calculation to obtain the second encrypted data process value ct2_mod_din;

[0047] ct2_mod_din=pk 2-u [addr1]+e2[addr1]; (2)

[0048] Step 4.6: Determine whether the highest bit of the second encrypted data process value ct2_mod_din is equal to 0. If so, set the second encrypted registered data mid2 = ct2_mod_din; otherwise, set the second encrypted registered data mid2 to be ct2_mod_din minus "1" and then inverted;

[0049] Step 4.7: The second encrypted data ct2 is obtained by performing a bitwise AND operation on the bitwise AND parameter q_value and mid2-q;

[0050] Step 5: Grayscale processing of encrypted data:

[0051] Step 5.1: Initialize the second address addr2, the encryption coefficient ct_scale, read the W-bit parameter scale from the instruction code memory, and initialize the counter cnt_poly to 0;

[0052] Step 5.2: Calculate the first encryption coefficient ct1_scale=ct_scale[addr2]+ct1[addr2]×scale[addr2];

[0053] Step 5.3: assign addr2+1 to addr2, and determine whether addr2 is less than d. If so, return to step 5.2 to execute sequentially; otherwise, it indicates that the encryption is completed, and the first grayscale encrypted data ct_mid1=ct1_scale is output;

[0054] Step 5.4: Calculate the second encryption coefficient:

[0055] ct2_scale=ct_scale[cnt_poly]+ct2[cnt_poly]×scale[cnt_poly];

[0056] Step 5.5: assign the counter cnt_poly+1 to cnt_poly, and determine whether cnt_poly is less than poly_num; if so, return to step 5.4 to execute sequentially; otherwise, it means that the calculation is completed, and the second grayscale encrypted data ct_mid2=ct2_scale is output;

[0057] Step 6: Decrypt the encrypted grayscale data:

[0058] Step 6.0: define and initialize the decrypted data dec_m=0, variable m2=0 and variable n2=0, initialize the variables i2 and j2; define and initialize the grayscale encrypted data process value q_mod to 0; define and initialize the fifth decrypted data process value t_mod_din to 0; define and initialize the decryption completion signal dem_flag to 0;

[0059] Step 6.1: Initialize i2=d;

[0060] Step 6.2: Initialize j2 = d;

[0061] Step 6.2.1: Read the i2th value of the second grayscale encrypted data ct_mid2;

[0062] Step 6.2.2: Read the j2th value of the private key sk;

[0063] Step 6.3: Determine whether the remainder of m2+n2 divided by d is 0. If so, assign the second grayscale encrypted data ct_mid2 to the grayscale encrypted data process value q_mod; otherwise, set the highest position of the second grayscale encrypted data ct_mid2 to "1" and assign it to the grayscale encrypted data process value q_mod;

[0064] Step 6.4: After assigning j2-1 to j2 and n2+1 to n2, determine whether n2 is less than d. If so, return to step 6.2.2 and execute sequentially. Otherwise, execute step 6.5.

[0065] Step 6.5: After assigning i2-1 to i2 and m2+1 to m2, determine whether m2 is less than d. If so, return to step 6.2 and execute sequentially; otherwise, execute step 6.6;

[0066] Step 6.6: Multiply the grayscale encrypted data process value q_mod by the plaintext modulus t to obtain the first decrypted data process value dec_mid1, and take the modulus of the value of dec_mid1 to obtain the second decrypted data process value dec_mid2;

[0067] Step 6.7: Divide dec_mid2 by d to obtain the third decrypted data process value dec_mid3;

[0068] Step 6.8: Perform a bitwise “OR” operation on the second decrypted data process value dec_mid2 and the bitwise parameter q_value to obtain a fourth decrypted data process value dec_mid4;

[0069] Step 6.9: Determine whether the highest bit of dec_mid1 is 0. If so, execute step 6.9.1; otherwise, execute step 6.9.2;

[0070] Step 6.9.1: Determine whether the fourth decrypted data process value dec_mid4 is greater than the judgment parameter half_q. If so, assign dec_mid3+1 to the fifth decrypted data process value t_mod_din; otherwise, set t_mod_din=dec_mid3;

[0071] Step 6.9.2: Determine whether the fourth decrypted data process value dec_mid4 is greater than the judgment parameter half_q. If so, assign the result of negating dec_mid3+1 and adding "1" to the fifth decrypted data process value t_mod_din; otherwise, assign dec_mid3+1 to t_mod_din;

[0072] Step 6.10: Determine whether the fifth decrypted data process value t_mod_din is greater than 0. If so, set the first decrypted data register mid1 = t_mod_din; otherwise, subtract "1" from t_mod_din and then assign the inverted result to the first decrypted data register mid1;

[0073] Step 6.11: Perform a bitwise AND operation on the fifth decrypted data process value t_mod_din and the bitwise AND parameter q_value to obtain the second decrypted data storage mid2, and determine whether mid1 is greater than the judgment parameter half_t; if so, execute step 6.11.1; otherwise, execute step 6.11.2;

[0074] Step 6.11.1: Determine whether mid2 is greater than half_t. If so, assign mid2-t to the decrypted data dec_m. Otherwise, set dec_m=mid2.

[0075] Step 6.11.2: Let the decrypted data dec_m = mid1;

[0076] Step 6.12: Determine whether the decrypted data dec_m is 0, if so, set dem_flag to 1; otherwise, set dem_flag to 0;

[0077] Step 7: Image Generation:

[0078] Step 7.1: Define and initialize an image processing completion signal tx_flag to 0;

[0079] Step 7.2: intercept the Pth to P+7th binary numbers of the decrypted data dec_m and copy them multiple times as multiple channels of grayscale plaintext data respectively, and then splice the multiple channels into X bits of data as the cached k×d to (k+1)×dth data;

[0080] Step 7.3: When dem_flag is 1, assign k+1 to k, and determine whether k is less than the pixel period poly_num of the calculated data. If so, return to step 7.2 for sequential execution; otherwise, set the image processing completion signal tx_flag to 1 and prepare for display;

[0081] Step 8: Image Display:

[0082] Step 8.1: Define and initialize a row counter h_cnt to 0 and a column counter l_cnt to 0;

[0083] Step 8.2: In each clock cycle, the column counter l_cnt is incremented. When the column counter l_cnt reaches the maximum value L, the row counter h_cnt is incremented by 1.

[0084] Step 8.3: Determine whether h_cnt is greater than H. If so, set the row counter h_cnt to 0 and the column counter l_cnt to 0. Otherwise, when the row counter range is between 0 and H / 2 and the column counter range is between 0 and L / 2, the original image data processed by the plaintext data message is output to the display screen; when the row counter range is between H / 2 and H and the column counter range is between 0 and L / 2, the original image ciphertext data processed by the first encrypted data ct1 is output to the display screen; when the row counter range is between 0 and H / 2 and the column counter range is between L / 2 and L, the grayscale image ciphertext data processed by the first grayscale encrypted data ct_mid1 is output to the display screen; when the row counter range is between H / 2 and H and the column counter range is between L / 2 and L, the grayscale image plaintext data processed by the decrypted data dec_m is output to the display screen.

[0085] Compared with the prior art, the beneficial technical effects of the present invention are embodied in:

[0086] 1. In the calculation process, the present invention uses the feature of random numbers sampled from the memory as the key, limits the bit width of the key to two bits, and directly operates on the value of another set of full-bit-width polynomials by judging the value of the key, which greatly reduces the complexity of the calculation, saves a lot of computing resources, and improves the image calculation speed by more than 60%;

[0087] 2. The present invention moves the calculation process of the weighted key pk_u to the public key preprocessing module, thereby precalculating the calculation of the encryption module. Compared with the initial pk_u calculation, which needs to be repeated every time a batch of data is encrypted, this calculation is now moved to the key generation module. During the entire picture encryption process, pk_u only needs to be calculated once, which saves more than 90% of the calculation time and greatly reduces power consumption by reducing repeated calculations.

[0088] 3. In the data flow process of the present invention, a multi-stage pipeline is inserted into the complex calculation process, and a ping-pong operation is designed, which can increase the data throughput rate while saving the data storage area to the greatest extent;

[0089] 4. The encryption, homomorphic processing, and decryption processes of the present invention adopt pipeline operations, and there is no need to wait for the entire data flow to be processed before reading the next batch of data, thereby ensuring that the image data can be processed effectively, quickly, and uninterruptedly. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 The overall structure diagram adopted by the present invention;

[0091] Figure 2 This is a data flow chart of the image conversion module of the present invention;

[0092] Figure 3 A schematic diagram of the data decryption process of the present invention;

[0093] Figure 4 This is a second schematic diagram of the data decryption process of the present invention;

[0094] Figure 5 The encrypted data flow chart of the present invention. DETAILED DESCRIPTION

[0095] In this embodiment, Figure 1 As shown, an image processing hardware accelerator based on homomorphic encryption includes: an FV module, an image conversion module, a data storage module and an HDMI display module;

[0096] The FV module consists of a key generation module, an encryption module, a calculation module, a decryption module, and a control module;

[0097] The key generation module consists of a private key generation module, a public key generation module and a public key preprocessing module;

[0098] The private key generation module is provided with a private key storage device and stores private key data;

[0099] A second public key memory is provided in the public key generation module, and the second public key and noise data are stored in the second public key memory;

[0100] The public key preprocessing module is provided with a public key weight memory and a parameter memory;

[0101] The control module is provided with a data memory and an instruction code memory, and the instruction code memory stores a Y-bit data;

[0102] The data storage module is composed of an original image storage module, an original image ciphertext storage module, a grayscale image ciphertext storage module, and a grayscale image plaintext storage module;

[0103] The private key storage device outputs the private key data to the public key generation module and the control module respectively;

[0104] After receiving the private key data, the public key generation module obtains the first public key by reading the second public key and the noise data in the second public key storage and calculating them together with the private key data, and then transmits the first public key and the second public key to the public key preprocessing module;

[0105] The public key preprocessing module reads the randomly generated N=2-bit public key weight u from the public key weight memory, and calculates the public key weight u with the first public key and the second public key to obtain the first weight public key and the second weight public key, and outputs them to the encryption module;

[0106] like Figure 2 As shown, the image conversion module truncates the image data from the data storage module or the external input into three data, namely, R, G, and B, and expands them into 65-bit plaintext message data respectively, and then obtains the plaintext data and outputs it to the encryption module;

[0107] The encryption module receives the first weighted public key, the second weighted public key and the plaintext data, and reads the noise data from the second public key memory and the parameter data from the parameter memory, thereby obtaining the first encrypted data after computing together and outputting them to the computing module and the data memory for storage respectively;

[0108] The encryption module obtains second encrypted data after performing calculations based on the second weighted public key and the noise data, and outputs the second encrypted data to the calculation module;

[0109] The control module uses the first two bits of the Y=34-bit data in the instruction code memory as the selection instruction of addition or multiplication, and outputs the remaining bits of data as coefficient data to the operation module, and outputs the private key data to the decryption module;

[0110] The operation module operates the first encrypted data and the second encrypted data respectively. If the selected instruction is a multiplication instruction, the first encrypted data and the second encrypted data are multiplied by the coefficient data respectively to obtain corresponding results and send them to the data storage for temporary storage; if the selected instruction is an addition instruction, all the first encrypted data are multiplied and then added to obtain the first encrypted grayscale data, all the second encrypted data are multiplied and then added to obtain the second encrypted grayscale data, and then the first encrypted grayscale data is output to the data storage for storage, and the first encrypted grayscale data and the second encrypted grayscale data are output to the decryption module;

[0111] The decryption module receives the first encrypted grayscale data, the second encrypted grayscale data and the private key data and generates decrypted data after performing calculations and outputs it to the image conversion module;

[0112] The image conversion module shifts the decrypted data, generates grayscale plaintext data, and then splices it into an X=24-bit binary data and outputs it to the data storage module for storage;

[0113] The image conversion module reads the first encrypted grayscale data and the first encrypted data in the data storage and performs bit processing respectively, then outputs the encrypted grayscale data and the encrypted data accordingly and transmits them to the data storage module for storage;

[0114] The data storage module sends the stored data to the HDMI module, so that the data is output to the display screen for display.

[0115] In this embodiment, an image processing method based on homomorphic encryption image processing hardware accelerator is to perform data processing according to the following steps:

[0116] Step 1: Read the ciphertext modulus q and the plaintext modulus t from the parameter memory, and generate a bitwise AND parameter q_value and two judgment parameters half_t and half_q;

[0117] Step 2: Key Generation:

[0118] Step 2.0: Read the private key sk from the private key memory, the second public key pk2 and the noise e from the second public key memory; define and initialize the first public key pk1=0, the variable m1=0 and the variable n1=0, define d as the number of pixels processed at one time, and define the variables i1 and j1;

[0119] Step 2.1: Initialize i1=d;

[0120] Step 2.2: Initialize j1=d;

[0121] Step 2.3: Read the i1th value of the second public key pk2;

[0122] Step 2.4: Read the j1th value of the private key sk;

[0123] Step 2.5: Determine whether the remainder of m1+n1 divided by d is 0; if it is 0, assign pk2+e to pk2; otherwise, assign -pk2+e to pk2;

[0124] Step 2.6: After assigning j1-1 to j1 and n1+1 to n1, determine whether n1 is less than d. If so, return to step 2.4 and execute sequentially. Otherwise, execute step 2.7.

[0125] Step 2.7: Assign i1-1 to i1, assign m1+1 to m1, and determine whether m1 is less than d. If so, return to step 2.2 and execute sequentially; otherwise, output the first public key pk1;

[0126] Step 2.8: After reading the randomly generated N-bit binary number u from the public key weight memory and using it as the private key sk, the first public key pk1 and the second public key pk2 are respectively calculated according to the process of steps 2.1 to 2.7 to obtain the first weighted public key pk 1-u and the second weighted public key pk 2-u ;

[0127] Step 3: Plaintext generation:

[0128] Step 3.1: Get the number of pixels of the input image R = 307200, define the number of pixels to be processed at one time as d = 256, the pixel period of data calculation as poly_num, and round up the result of dividing R by d and assign it to poly_num;

[0129] Step 3.2: Expand the data bit width of each pixel in the input image containing R pixels into a binary number of M=65 bits and use it as the plaintext data message;

[0130] Step 4: Figure 5 As shown, the ciphertext is generated:

[0131] Step 4.1: define and initialize the first address addr1=d, define the first noise e1 and the second noise e2 and assign the noise e to the first noise e1 and the second noise e2 respectively; define the first encrypted data ct1 and initialize ct1=0; define the first encrypted data ct2 and initialize ct2=0;

[0132] Step 4.2: Use formula (1) to calculate the first noise e1 and the first weighted public key pk 1-u Operate with the plaintext data message to obtain the first encrypted data process value ct1_mod_din:

[0133] ct1_mod_din=pk 1_u [addr1]+e1[addr1]+message[addr1]×(q / t); (1)

[0134] Step 4.3: Determine whether the highest bit of ct1_mod_din is equal to 0. If so, set the first encrypted registered data mid1 = ct1_mod_din; otherwise, set the first encrypted registered data mid1 to be ct1_mod_din minus 1 and then inverted;

[0135] Step 4.4: Perform a bitwise AND operation on the bitwise AND parameter q_value and mid1-q to obtain the first encrypted data ct1;

[0136] Step 4.5: Use formula (2) to calculate the second noise e2 and the second weighted public key pk 2-u Perform calculation to obtain the second encrypted data process value ct2_mod_din;

[0137] ct2_mod_din=pk 2-u [addr1]+e2[addr1]; (2)

[0138] Step 4.6: Determine whether the highest bit of the second encrypted data process value ct2_mod_din is equal to 0. If so, set the second encrypted registered data mid2 = ct2_mod_din; otherwise, set the second encrypted registered data mid2 to be ct2_mod_din minus "1" and then inverted;

[0139] Step 4.7: The second encrypted data ct2 is obtained by performing a bitwise AND operation on the bitwise AND parameter q_value and mid2-q;

[0140] Step 5: Grayscale processing of encrypted data:

[0141] Step 5.1: Initialize the second address addr2 and the encryption coefficient ct_scale, read W=32-bit parameter scale from the instruction code memory, and initialize the counter cnt_poly to 0;

[0142] Step 5.2: Calculate the first encryption coefficient ct1_scale=ct_scale[addr2]+ct1[addr2]×scale[addr2];

[0143] Step 5.3: assign addr2+1 to addr2, and determine whether addr2 is less than d. If so, return to step 5.2 to execute sequentially; otherwise, it indicates that the encryption is completed, and the first grayscale encrypted data ct_mid1=ct1_scale is output;

[0144] Step 5.4: Calculate the second encryption coefficient:

[0145] ct2_scale=ct_scale[cnt_poly]+ct2[cnt_poly]×scale[cnt_poly];

[0146] Step 5.5: assign the counter cnt_poly+1 to cnt_poly, and determine whether cnt_poly is less than poly_num; if so, return to step 5.4 to execute sequentially; otherwise, it means that the calculation is completed, and the second grayscale encrypted data ct_mid2=ct2_scale is output;

[0147] Step 6: Figure 3 and Figure 4 As shown, decrypt the encrypted grayscale data:

[0148] Step 6.0: define and initialize the decrypted data dec_m=0, variable m2=0 and variable n2=0, initialize the variables i2 and j2; define and initialize the grayscale encrypted data process value q_mod to 0; define and initialize the fifth decrypted data process value t_mod_din to 0; define and initialize the decryption completion signal dem_flag to 0;

[0149] Step 6.1: Initialize i2=d;

[0150] Step 6.2: Initialize j2 = d;

[0151] Step 6.2.1: Read the i2th value of the second grayscale encrypted data ct_mid2;

[0152] Step 6.2.2: Read the j2th value of the private key sk;

[0153] Step 6.3: Determine whether the remainder of m2+n2 divided by d is 0. If so, assign the second grayscale encrypted data ct_mid2 to the grayscale encrypted data process value q_mod; otherwise, set the highest position of the second grayscale encrypted data ct_mid2 to "1" and assign it to the grayscale encrypted data process value q_mod;

[0154] Step 6.4: After assigning j2-1 to j2 and n2+1 to n2, determine whether n2 is less than d. If so, return to step 6.2.2 and execute sequentially. Otherwise, execute step 6.5.

[0155] Step 6.5: After assigning i2-1 to i2 and m2+1 to m2, determine whether m2 is less than d. If so, return to step 6.2 and execute sequentially; otherwise, execute step 6.6;

[0156] Step 6.6: Multiply the grayscale encrypted data process value q_mod by the plaintext modulus t to obtain the first decrypted data process value dec_mid1, and take the modulus of the value of dec_mid1 to obtain the second decrypted data process value dec_mid2;

[0157] Step 6.7: Divide dec_mid2 by d to obtain the third decrypted data process value dec_mid3;

[0158] Step 6.8: Perform a bitwise “OR” operation on the second decrypted data process value dec_mid2 and the parameter q_value to obtain a fourth decrypted data process value dec_mid4;

[0159] Step 6.9: Determine whether the highest bit of dec_mid1 is 0. If so, execute step 6.9.1; otherwise, execute step 6.9.2;

[0160] Step 6.9.1: Determine whether the fourth decrypted data process value dec_mid4 is greater than the judgment parameter half_q. If so, assign dec_mid3+1 to the fifth decrypted data process value t_mod_din; otherwise, set t_mod_din=dec_mid3;

[0161] Step 6.9.2: Determine whether the fourth decrypted data process value dec_mid4 is greater than the judgment parameter half_q. If so, assign the result of negating dec_mid3+1 and adding "1" to the fifth decrypted data process value t_mod_din; otherwise, assign dec_mid3+1 to t_mod_din;

[0162] Step 6.10: Determine whether the fifth decrypted data process value t_mod_din is greater than 0. If so, set the first decrypted data register mid1 = t_mod_din; otherwise, subtract "1" from t_mod_din and then assign the inverted result to the first decrypted data register mid1;

[0163] Step 6.11: Perform a bitwise AND operation on the fifth decrypted data process value t_mod_din and the bitwise AND parameter q_value to obtain the second decrypted data register mid2, and determine whether mid1 is greater than the judgment parameter half_t; if so, execute step 6.11.1; otherwise, execute step 6.11.2;

[0164] Step 6.11.1: Determine whether mid2 is greater than half_t. If so, assign mid2-t to the decrypted data dec_m. Otherwise, set dec_m=mid2.

[0165] Step 6.11.2: Let the decrypted data dec_m = mid1;

[0166] Step 6.12: Determine whether the decrypted data dec_m is 0. If so, set dem_flag to 1; otherwise, set dem_flag to 0;

[0167] Step 7: Image Generation:

[0168] Step 7.1: Define and initialize an image processing completion signal tx_flag to 0;

[0169] Step 7.2: intercept the P=8 to 15th binary digits of the decrypted data dec_m and copy them multiple times as multiple channels of grayscale plaintext data respectively, and splice the multiple channels into X=24 bits of data as the cached k×d to (k+1)×d data;

[0170] Step 7.3: When dem_flag is 1, assign k+1 to k, and determine whether k is less than the pixel period poly_num of the calculated data. If so, return to step 7.2 for sequential execution; otherwise, set the image processing completion signal tx_flag to 1 and prepare for display;

[0171] Step 8: Image Display:

[0172] Step 8.1: Define and initialize a row counter h_cnt to 0 and a column counter l_cnt to 0;

[0173] Step 8.2: In each clock cycle, the column counter l_cnt is incremented. When the column counter l_cnt reaches the maximum value L=1280, the row counter h_cnt is incremented by 1.

[0174] Step 8.3: Determine whether h_cnt is greater than H=1024. If so, set the row counter h_cnt to 0 and the column counter l_cnt to 0. Otherwise, when the row counter range is between 0 and H / 2 and the column counter range is between 0 and L / 2, the original image data processed by the plaintext data message is output to the display screen; when the row counter range is between H / 2 and H and the column counter range is between 0 and L / 2, the original image ciphertext data processed by the first encrypted data ct1 is output to the display screen; when the row counter range is between 0 and H / 2 and the column counter range is between L / 2 and L, the grayscale image ciphertext data processed by the first grayscale encrypted data ct_mid1 is output to the display screen; when the row counter range is between H / 2 and H and the column counter range is between L / 2 and L, the grayscale image plaintext data processed by the decrypted data dec_m is output to the display screen.

Claims

1. An image processing hardware accelerator based on homomorphic encryption, the features of which include: FV module, image conversion module, data storage module and HDMI module; The FV module is composed of a key generation module, an encryption module, a calculation module, a decryption module, and a control module; The key generation module is composed of a private key generation module, a public key generation module and a public key preprocessing module; The private key generation module is provided with a private key storage device and stores private key data; The public key generation module is provided with a second public key storage, and the second public key storage stores a second public key and noise data; The public key preprocessing module is provided with a public key weight memory and a parameter memory; The control module is provided with a data memory and an instruction code memory, and the instruction code memory stores a Y-bit data; The private key memory outputs the private key data to the public key generation module and the control module respectively; After receiving the private key data, the public key generation module reads the second public key and noise data in the second public key storage, and obtains the first public key after calculating with the private key data, and then transmits the first public key and the second public key to the public key preprocessing module; The public key preprocessing module reads the randomly generated N-bit public key weight u from the public key weight memory, and calculates the public key weight u with the first public key and the second public key to obtain a first weighted public key and a second weighted public key, and outputs them to the encryption module; The image conversion module performs truncation and expansion on the image data of the data storage module or external input to obtain plain text data and outputs the plain text data to the encryption module; The encryption module receives the first weighted public key, the second weighted public key and the plaintext data, and reads the noise data from the second public key memory and the parameter data from the parameter memory, so as to obtain the first encrypted data after computing together and output them to the computing module and the data memory for storage respectively; The encryption module obtains second encrypted data after performing calculations based on the second weighted public key and the noise data, and outputs the second encrypted data to the calculation module; The control module uses the first two bits of the Y-bit data in the instruction code memory as a selection instruction for addition or multiplication, and outputs the remaining bits of data as coefficient data to the operation module, and outputs the private key data to the decryption module; The operation module operates the first encrypted data and the second encrypted data respectively. If the selected instruction is a multiplication instruction, the first encrypted data and the second encrypted data are multiplied by the coefficient data respectively to obtain corresponding results and send them to the data memory for temporary storage; if the selected instruction is an addition instruction, all the first encrypted data are multiplied and then added to obtain the first encrypted grayscale data, all the second encrypted data are multiplied and then added to obtain the second encrypted grayscale data, and then the first encrypted grayscale data is output to the data memory for storage, and the first encrypted grayscale data and the second encrypted grayscale data are output to the decryption module; The decryption module receives the first encrypted grayscale data, the second encrypted grayscale data and the private key data and generates decrypted data after performing calculations and outputs the decrypted data to the image conversion module; The image conversion module shifts the decrypted data, generates grayscale plaintext data, and then splices it into an X-bit binary data and outputs it to the data storage module for storage; The image conversion module reads the first encrypted grayscale data and the first encrypted data in the data storage and performs bit processing respectively, and then outputs the encrypted grayscale data and the encrypted data accordingly and transmits them to the data storage module for storage; The data storage module sends the stored data to the HDMI module, so that the data is output to the display screen for display.

2. An image processing method based on homomorphic encryption, characterized in that: The data processing is carried out in the following steps: Step 1: Read the ciphertext modulus q and the plaintext modulus t, and generate a bitwise AND parameter q_value and two judgment parameters half_t and half_q; Step 2: Key Generation: Step 2.0: Read the private key sk, the second public key pk2 and the noise e; define and initialize the first public key pk1=0, the variable m1=0 and the variable n1=0, define d as the number of pixels processed at a time, and define the variables i1 and j1; Step 2.1: Initialize i1=d; Step 2.2: Initialize j1=d; Step 2.3: Read the i1th value of the second public key pk2; Step 2.4: Read the j1th value of the private key sk; Step 2.5: Determine whether the remainder of m1+n1 divided by d is 0; if it is 0, assign pk2+e to pk2; otherwise, assign -pk2+e to pk2; Step 2.6: After assigning j1-1 to j1 and n1+1 to n1, determine whether n1 is less than d. If so, return to step 2.4 and execute sequentially. Otherwise, execute step 2.

7. Step 2.7: Assign i1-1 to i1, assign m1+1 to m1, and determine whether m1 is less than d. If so, return to step 2.2 and execute sequentially; otherwise, output the first public key pk1; Step 2.8: After generating an N-bit binary number u as the private key sk, the first public key pk1 and the second public key pk2 are respectively operated according to the process of steps 2.1 to 2.7 to obtain the first weighted public key pk 1-u and the second weighted public key pk 2-u ; Step 3: Plaintext generation: Step 3.1: Get the number of pixels R of the input image, define the number of pixels to be processed at a time as d, the pixel period of data calculation as poly_num, and round up the result of dividing R by d and assign it to poly_num; Step 3.2: Expand the data bit width of each pixel in the input image containing R pixels into an M-bit binary number and use it as the plaintext data message; Step 4: Ciphertext generation: Step 4.1: define and initialize the first address addr1=d, define the first noise e1 and the second noise e2 and assign the noise e to the first noise e1 and the second noise e2 respectively; define the first encrypted data ct1 and initialize ct1=0; define the first encrypted data ct2 and initialize ct2=0; Step 4.2: Use formula (1) to calculate the first noise e1 and the first weighted public key pk 1-u Operate with the plaintext data message to obtain the first encrypted data process value ct1_mod_din: ct1_mod_din=pk 1_u [addr1]+e1[addr1]+message[addr1]×(q / t); (1) Step 4.3: Determine whether the highest bit of ct1_mod_din is equal to 0. If so, set the first encrypted registered data mid1 = ct1_mod_din; otherwise, set the first encrypted registered data mid1 to be ct1_mod_din minus 1 and then inverted; Step 4.4: Perform a bitwise AND operation on the bitwise AND parameter q_value and mid1-q to obtain the first encrypted data ct1; Step 4.5: Use formula (2) to calculate the second noise e2 and the second weighted public key pk 2-u Perform calculation to obtain the second encrypted data process value ct2_mod_din; ct2_mod_din=pk 2-u [addr1]+e2[addr1]; (2) Step 4.6: Determine whether the highest bit of the second encrypted data process value ct2_mod_din is equal to 0. If so, set the second encrypted registered data mid2 = ct2_mod_din; otherwise, set the second encrypted registered data mid2 to be ct2_mod_din minus "1" and then inverted; Step 4.7: The second encrypted data ct2 is obtained by performing a bitwise AND operation on the bitwise AND parameter q_value and mid2-q; Step 5: Grayscale processing of encrypted data: Step 5.1: Initialize the second address addr2, the encryption coefficient ct_scale, read the W-bit parameter scale from the instruction code memory, and initialize the counter cnt_poly to 0; Step 5.2: Calculate the first encryption coefficient ct1_scale=ct_scale[addr2]+ct1[addr2]×scale[addr2]; Step 5.3: assign addr2+1 to addr2, and determine whether addr2 is less than d. If so, return to step 5.2 to execute sequentially; otherwise, it indicates that the encryption is completed, and the first grayscale encrypted data ct_mid1=ct1_scale is output; Step 5.4: Calculate the second encryption coefficient: ct2_scale=ct_scale[cnt_poly]+ct2[cnt_poly]×scale[cnt_poly]; Step 5.5: assign the counter cnt_poly+1 to cnt_poly, and determine whether cnt_poly is less than poly_num; if so, return to step 5.4 to execute sequentially; otherwise, it means that the calculation is completed, and the second grayscale encrypted data ct_mid2=ct2_scale is output; Step 6: Decrypt the encrypted grayscale data: Step 6.0: define and initialize the decrypted data dec_m=0, variable m2=0 and variable n2=0, initialize the variables i2 and j2; define and initialize the grayscale encrypted data process value q_mod to 0; define and initialize the fifth decrypted data process value t_mod_din to 0; define and initialize the decryption completion signal dem_flag to 0; Step 6.1: Initialize i2=d; Step 6.2: Initialize j2 = d; Step 6.2.1: Read the i2th value of the second grayscale encrypted data ct_mid2; Step 6.2.2: Read the j2th value of the private key sk; Step 6.3: Determine whether the remainder of m2+n2 divided by d is 0. If so, assign the second grayscale encrypted data ct_mid2 to the grayscale encrypted data process value q_mod; otherwise, set the highest position of the second grayscale encrypted data ct_mid2 to "1" and assign it to the grayscale encrypted data process value q_mod; Step 6.4: After assigning j2-1 to j2 and n2+1 to n2, determine whether n2 is less than d. If so, return to step 6.2.2 and execute sequentially. Otherwise, execute step 6.

5. Step 6.5: After assigning i2-1 to i2 and m2+1 to m2, determine whether m2 is less than d. If so, return to step 6.2 and execute sequentially; Otherwise, proceed to step 6.6; Step 6.6: Multiply the grayscale encrypted data process value q_mod by the plaintext modulus t to obtain the first decrypted data process value dec_mid1, and take the modulus of the value of dec_mid1 to obtain the second decrypted data process value dec_mid2; Step 6.7: Divide dec_mid2 by d to obtain the third decrypted data process value dec_mid3; Step 6.8: Perform a bitwise OR operation on the second decrypted data process value dec_mid2 and the bitwise parameter q_value to obtain a fourth decrypted data process value dec_mid4; Step 6.9: Determine whether the highest bit of dec_mid1 is 0. If so, execute step 6.9.1; Otherwise, proceed to step 6.9.2; Step 6.9.1: Determine whether the fourth decrypted data process value dec_mid4 is greater than the judgment parameter half_q. If so, assign dec_mid3+1 to the fifth decrypted data process value t_mod_din; otherwise, set t_mod_din=dec_mid3; Step 6.9.2: Determine whether the fourth decrypted data process value dec_mid4 is greater than the judgment parameter half_q. If so, assign the result of negating dec_mid3+1 and adding "1" to the fifth decrypted data process value t_mod_din; otherwise, assign dec_mid3+1 to t_mod_din; Step 6.10: Determine whether the fifth decrypted data process value t_mod_din is greater than 0. If so, set the first decrypted data register mid1 = t_mod_din; otherwise, subtract "1" from t_mod_din and then assign the inverted result to the first decrypted data register mid1; Step 6.11: Perform a bitwise AND operation on the fifth decrypted data process value t_mod_din and the bitwise AND parameter q_value to obtain the second decrypted data register mid2, and determine whether mid1 is greater than the judgment parameter half_t; if so, execute step 6.11.1; Otherwise, proceed to step 6.11.2; Step 6.11.1: Determine whether mid2 is greater than half_t. If so, assign mid2-t to the decrypted data dec_m. Otherwise, set dec_m=mid2. Step 6.11.2: Let the decrypted data dec_m = mid1; Step 6.12: Determine whether the decrypted data dec_m is 0, if so, set dem_flag to 1; otherwise, set dem_flag to 0; Step 7: Image Generation: Step 7.1: Define and initialize an image processing completion signal tx_flag to 0; Step 7.2: intercept the Pth to P+7th binary numbers of the decrypted data dec_m and copy them multiple times as multiple channels of grayscale plaintext data respectively, and then splice the multiple channels into X bits of data as the cached k×d to (k+1)×dth data; Step 7.3: When dem_flag is 1, assign k+1 to k, and determine whether k is less than the pixel period poly_num of the calculated data. If so, return to step 7.2 for sequential execution; otherwise, set the image processing completion signal tx_flag to 1 and prepare for display; Step 8: Image Display: Step 8.1: Define and initialize a row counter h_cnt to 0 and a column counter l_cnt to 0; Step 8.2: In each clock cycle, the column counter l_cnt is incremented. When the column counter l_cnt reaches the maximum value L, the row counter h_cnt is incremented by 1. Step 8.3: Determine whether h_cnt is greater than H. If so, set the row counter h_cnt to 0 and the column counter l_cnt to 0. Otherwise, when the row counter range is between 0 and H / 2 and the column counter range is between 0 and L / 2, the original image data processed by the plaintext data message is output to the display screen; when the row counter range is between H / 2 and H and the column counter range is between 0 and L / 2, the original image ciphertext data processed by the first encrypted data ct1 is output to the display screen; when the row counter range is between 0 and H / 2 and the column counter range is between L / 2 and L, the grayscale image ciphertext data processed by the first grayscale encrypted data ct_mid1 is output to the display screen; when the row counter range is between H / 2 and H and the column counter range is between L / 2 and L, the grayscale image plaintext data processed by the decrypted data dec_m is output to the display screen.

Images