A verifiable neural network security reasoning method, system and device based on homomorphic encryption and blockchain and a storage medium

Abstract

The application discloses a kind of based on homomorphic encryption and block chain's auditable neural network security reasoning method, system, device and storage medium, based on homomorphic encryption and block chain's verifiable neural network security reasoning method, comprising: step 1: initialization;Step 2: data processing and ciphertext packing;Step 3: model security reasoning;Step 4: upload model reasoning result;Step 5: service demand side obtains service result;Step 6: result verification.The application combines blockchain and a variety of homomorphic encryption technology, proposes a new ciphertext packing algorithm for reasoning efficiency optimization, and uses conversion method to avoid activation function polynomial approximation problem, improves the accuracy of model reasoning.At the same time, it provides a verification function for users, guarantees the authenticity and reliability of reasoning result, improves the accuracy of model reasoning.

Description

Technical Field

[0001] This invention relates to an auditable neural network secure inference method, system, device, and storage medium based on homomorphic encryption and blockchain, belonging to the fields of privacy-preserving machine learning and data security technology. Background Technology

[0002] In recent years, with the rapid development of big data and computing power, the inference performance of neural network models has been significantly improved. This progress has made complex artificial intelligence tasks more feasible and efficient. To meet the needs of resource-constrained enterprises and individuals, Machine Learning as a Service (MLaaS) has emerged. This service model allows enterprise and individual users to conveniently use powerful machine learning models without having to build and maintain complex machine learning infrastructure. However, when using MLaaS services, users send raw data to service providers, which can easily lead to privacy leaks. Furthermore, due to a lack of transparency and verifiability, users cannot determine whether the MLaaS provider has actually performed inference operations or whether the inference process meets expectations.

[0003] Ensuring user privacy and security while maintaining transparency and trustworthiness in the inference process during MLaaS service usage has become a crucial issue in research and practice, leading to numerous solutions. Among these, blockchain technology offers a potential solution for ensuring transparency and trustworthiness in the inference process. Blockchain's distributed ledger technology ensures the transparency and immutability of data and operations. By recording model parameters, inference processes, and results on the blockchain, users can verify the authenticity of each step. However, traditional encryption methods only guarantee the security of data storage and circulation; during computation, data remains in plaintext, failing to guarantee privacy during the computation process.

[0004] In terms of data privacy and security, homomorphic encryption technology is widely recognized due to its high security and the ability to perform operations directly on the ciphertext. However, message-based homomorphic encryption schemes can only perform addition and multiplication operations, and often use polynomial approximations for nonlinear functions such as activation functions, thus affecting the accuracy of model inference. While bit-based homomorphic encryption schemes can perform arbitrary forms of operations, they are difficult to apply to real-world scenarios due to their huge computational overhead.

[0005] Therefore, there is an urgent need to propose a verifiable neural network secure inference scheme that can improve inference efficiency while protecting user privacy and security, and ensure the authenticity and reliability of the inference process and results. Summary of the Invention

[0006] This invention provides a verifiable secure inference method and system for neural networks based on homomorphic encryption and blockchain. Targeting the inference characteristics of each network layer in the neural network model, we employ multiple homomorphic encryption schemes and design corresponding conversion methods and ciphertext packaging algorithms to improve the model's efficiency and accuracy. Furthermore, this invention provides a verifiable function, allowing users or enterprises to verify the inference results and ensure their authenticity.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A verifiable neural network secure reasoning method based on homomorphic encryption and blockchain includes the following steps:

[0009] Step 1: Initialization: The service requester (e.g., user, enterprise, etc.) and the model service provider generate a public and private key for the CKKS encryption scheme locally; the public and private keys of the service requester are denoted as PK. c SK c The model service provider is denoted as PK. s SK s ;

[0010] Step 2: Data Processing and Ciphertext Packaging: The service requester preprocesses the collected image data, such as resizing, noise reduction, and normalization, converting the images into matrix data. Leveraging the SIMD feature of CKKS, which allows multiple (two or more) data sets to be packaged into a single ciphertext for computation, we designed a ciphertext packaging algorithm. By arranging the data before encryption, we improve computational efficiency, thereby accelerating model inference. Then, the private key SK is used... c The rearranged data is encrypted into ciphertext and sent to the blockchain for storage.

[0011] Step 3: Model-safe inference: including convolutional layer inference, RLWE ciphertext to LWE ciphertext conversion, activation function layer inference, LWE ciphertext to RLWE ciphertext conversion, and fully connected layer inference;

[0012] Step 4: Upload Model Inference Results: The model service provider uploads the encrypted inference results to the blockchain;

[0013] Step 5: Service requester obtains service result: The service requester obtains the encrypted reasoning result from the blockchain and decrypts it using their own private key to obtain the reasoning result;

[0014] Step 6: Result Verification: When the service requester (user or enterprise) has doubts about the inference results of the model service provider, the service requester can send a verification request to an authoritative institution. The authoritative institution will send a request to obtain the private key to both the service requester and the model service provider. After obtaining the private keys from both parties, the authoritative institution will retrieve the encrypted model parameters and encrypted data from the blockchain, perform decryption operations, perform plaintext model inference operations locally, and retrieve intermediate results and inference results from the blockchain for comparison to generate a verification report for the service requester to verify.

[0015] Step 3 above: Model-safe reasoning includes:

[0016] Step 3.1: Convolutional Layer Inference: After receiving the inference request, the model service provider obtains the encrypted data to be inferred from the blockchain node and sets the model parameters according to... Figure 3 The encrypted data is packaged using a ciphertext packing algorithm. Simultaneously, the model service provider encodes its model parameters using the public key of the service requester, and then performs a convolution operation. The convolution operation can be viewed as applying a set of filters to the input data; each filter slides across the input data and calculates a weighted sum of local regions. Since the ciphertext has already been arranged in this form during ciphertext packing, this process can be seen as performing multiple multiplications and additions on the ciphertext. Assuming ct... m This represents ciphertext that has been processed by ciphertext packaging of a reasoning image. This indicates that there are M filters arranged in a specific order. This indicates that there are M biases arranged in a specific order. The result represents the computation, ultimately yielding M ciphertext computation results. The following formula illustrates the convolution process of the i-th filter, where Roate represents the ciphertext rotation operation. After multiplication, multiple rotation accumulations are performed, and the variable represents the number of rotation accumulations, following the order 1, 2, 4... up to log. K*K :

[0017]

[0018] Step 3.2: Conversion of RLWE ciphertext to LWE ciphertext: While RLWE ciphertext based on CKKS encryption can encrypt multiple data items into a single ciphertext, improving operational efficiency, it cannot perform logical comparison operations. Therefore, we split the RLWE ciphertext, extract the ciphertext encoded in the polynomial, and obtain a set of LWE ciphertext in FHEW encryption form to perform activation function operations;

[0019] LWE encryption is defined as: in, Represents an integer ring modulo q, key This represents a vector of dimension n, where each component is... Inside, message a is a random vector, e is an error term randomly selected from the error distribution χ, n is the vector dimension, b is the inner product result, m is the message, and T is the matrix transpose. Its decryption is b + a. T s≈m;

[0020] RLWE encryption is defined as: in Indicates the coefficient in Within the range and the polynomial module X n +1 ring, Indicates the coefficient in Within the range and the polynomial module X n +1 ring, key Represents a polynomial key. Let x represent the error polynomial from which coefficients are independently drawn from the error distribution x. It is a random polynomial;

[0021] We denot an RLWE ciphertext encrypted by CKKS as follows: Where Δ refers to the scaling factor in CKKS encryption, v represents the encryption vector, and symbol n, The dimension representing the lattice, and Using keys representing different dimensions, we extract each encoded element from the ciphertext ct to obtain a set of LWE ciphertexts in FHEW encryption form: Where l represents the encryption level, i∈ <l>This indicates that index i is within a certain range, specifically, all values ​​within range l. Then, the lookup table T(x) for each LWE ciphertext is evaluated:

[0022] Step 3.3: Activation Function Layer Inference: We construct a binary comparison circuit for the obtained LWE ciphertext to implement the ReLU operation process of the activation function, and finally we obtain the ciphertext operation result of the activation function;

[0023] Step 3.4: LWE Ciphertext to RLWE Ciphertext Conversion: After activation function operation, when operating on the next linear layer, the ciphertext type needs to be converted. A set of LWE ciphertexts is repackaged into a single RLWE ciphertext to encrypt the encoded vector T(v):

[0024] Step 3.5: Fully connected layer inference: The fully connected layer can be regarded as a special convolutional layer operation. We adopted a packing algorithm similar to the convolutional layer operation to perform the fully connected layer operation.

[0025] A verifiable neural network secure inference system based on homomorphic encryption and blockchain, comprising:

[0026] Initialization module: Used by service requesters (e.g., users, enterprises) and model service providers to generate public and private keys for the CKKS encryption scheme locally; whereby the public and private keys of the service requester are denoted as PK. c SK c The model service provider is denoted as PK. s SK s ;

[0027] Data Processing and Ciphertext Packaging Module: This module handles data processing and ciphertext packaging. It allows service providers to preprocess acquired image data, such as resizing, noise reduction, and normalization, converting images into matrix data. Leveraging the SIMD feature of CKKS, which allows multiple data sets to be packaged into a single ciphertext for computation, we designed a ciphertext packaging algorithm. By arranging the data before encryption, we improve computational efficiency, thereby accelerating model inference. The private key SK is then used... c The rearranged data is encrypted into ciphertext and sent to the blockchain for storage;

[0028] Model-safe inference module: Used for model-safe inference, which includes convolutional layer inference, RLWE ciphertext to LWE ciphertext conversion, activation function layer inference, LWE ciphertext to RLWE ciphertext conversion, and fully connected layer inference;

[0029] Upload Model Inference Results Module: Used by model service providers to upload encrypted inference results to the blockchain;

[0030] The service result acquisition module is used by service requesters to obtain service results. Service requesters obtain encrypted reasoning results from the blockchain and decrypt them using their own private key to obtain the reasoning results.

[0031] Result Verification Module: Used for result verification. When a service requester (user or enterprise) has doubts about the inference results of the model service provider, the service requester can send a verification request to an authoritative institution. The authoritative institution will send a request to both the service requester and the model service provider to obtain their private keys. After obtaining the private keys from both parties, the authoritative institution will retrieve the encrypted model parameters and encrypted data from the blockchain, perform decryption operations, perform plaintext model inference operations locally, and retrieve intermediate results and inference results from the blockchain for comparison to generate a verification report for the service requester to verify.

[0032] The above-mentioned model-safe reasoning module includes the following steps when performing model-safe reasoning:

[0033] Convolutional layer inference: After receiving the inference request, the model service provider obtains the encrypted data to be inferred from the blockchain node and sets the model parameters according to... Figure 3 The encrypted data is packaged using a ciphertext packing algorithm. Simultaneously, the model service provider encodes its model parameters using the public key of the service requester, and then performs a convolution operation. The convolution operation can be viewed as applying a set of filters to the input data; each filter slides across the input data and calculates a weighted sum of local regions. Since the ciphertext has already been arranged in this form during ciphertext packing, this process can be seen as performing multiple multiplications and additions on the ciphertext. Assuming ct... m This represents ciphertext that has been processed by ciphertext packaging of a reasoning image. This indicates that there are M filters arranged in a specific order. This indicates that there are M biases arranged in a specific order. The result represents the computation, ultimately yielding M ciphertext computation results. The following formula illustrates the convolution process of the i-th filter, where Roate represents the ciphertext rotation operation. After multiplication, multiple rotation accumulations are performed, and the variable represents the number of rotation accumulations, following the order 1, 2, 4... up to log. K*K :

[0034]

[0035] RLWE ciphertext to LWE ciphertext conversion: While RLWE ciphertext based on CKKS encryption can encrypt multiple data items into a single ciphertext, improving operational efficiency, it cannot perform logical comparison operations. Therefore, we split the RLWE ciphertext, extract the ciphertext encoded in the polynomial, and obtain a set of FHEW-encrypted LWE ciphertexts for activation function operation.

[0036] LWE encryption is defined as: in, Represents an integer ring modulo q, key This represents a vector of dimension n, where each component is... Inside, message a is a random vector, e is an error term randomly selected from the error distribution χ, n is the vector dimension, b is the inner product result, m is the message, and T is the mathematical transpose. Its decryption is b + a. T s≈m;

[0037] RLWE encryption is defined as: in Indicates the coefficient in Within the range and the polynomial module X n +1 ring, Indicates the coefficient in Within the range and the polynomial module X n +1 ring, key Represents a polynomial key. This represents the error polynomial from which coefficients are independently drawn from the error distribution χ. It is a random polynomial;

[0038] We denot an RLWE ciphertext encrypted by CKKS as follows: Where Δ refers to the scaling factor in CKKS encryption, v represents the encryption vector, and symbol n, The dimension representing the lattice, and s, Using keys representing different dimensions, we extract each encoded element from the ciphertext ct to obtain a set of LWE ciphertexts in FHEW encryption form: Where l represents the encryption level, i∈ <l>This indicates that index i is within a certain range, specifically, all values ​​within range l. Then, the lookup table T(x) for each LWE ciphertext is evaluated:

[0039] Activation function layer inference: We construct a binary comparison circuit for the obtained LWE ciphertext to implement the ReLU operation process of the activation function, and finally we obtain the ciphertext operation result of the activation function;

[0040] LWE ciphertext to RLWE ciphertext conversion: After activation function operation, when operating on the next linear layer, the ciphertext type needs to be converted. A set of LWE ciphertexts is repackaged into a single RLWE ciphertext to encrypt the encoded vector T(v):

[0041] Fully connected layer inference: Fully connected layers can be viewed as special convolutional layer operations. We adopted a packing algorithm similar to that used in convolutional layer operations for fully connected layer operations.

[0042] Our system includes the following entities:

[0043] 1. Service Requester: The service requester refers to an enterprise or individual with a need for neural network inference, responsible for data collection and preprocessing. Afterwards, the data is encrypted using the CKKS scheme and uploaded to the blockchain. The service requester is also responsible for receiving and decrypting the inference results.

[0044] 2. Model Service Provider: The model service provider possesses neural network models and computing resources, and provides inference services. They obtain encrypted data from the blockchain and, during the model inference process, upload model parameters, intermediate inference processes, and inference results to the blockchain in encrypted form.

[0045] 3. Blockchain Nodes: Blockchain nodes are responsible for storing and managing encrypted data, model parameters, and inference results, ensuring the integrity and immutability of all data and providing transparent and verifiable records.

[0046] 4. Authoritative Institution: The authoritative institution is an honest entity. It is responsible for auditing and verifying the inference results. The authoritative institution holds the private keys of both parties, which allows it to decrypt data and model parameters, re-execute the inference process, and provide the audit results back to the service requester to ensure the authenticity and credibility of the inference results.

[0047] A verifiable neural network secure inference device based on homomorphic encryption and blockchain includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the aforementioned verifiable neural network secure inference method based on homomorphic encryption and blockchain.

[0048] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned verifiable neural network secure reasoning method based on homomorphic encryption and blockchain.

[0049] Any techniques not mentioned in this invention are based on existing technologies.

[0050] This invention provides a verifiable secure inference method for neural networks. Addressing the issues of reduced model accuracy and inability to verify inference results caused by polynomial approximation in existing homomorphic encryption schemes when processing activation functions, this invention proposes a solution based on blockchain technology and multiple homomorphic encryption schemes. This invention ensures the reliability of inference results while protecting user privacy and security. We improve the accuracy of model inference by using a method for mutual conversion between RLWE (Ring Learning with Errors) ciphertext and LWE (Learning with Errors) ciphertext.

[0051] This invention presents a verifiable neural network secure inference method based on homomorphic encryption and blockchain. It combines blockchain and various homomorphic encryption technologies, proposing a novel ciphertext packaging algorithm to optimize inference efficiency. Furthermore, it employs a transformation method to avoid the polynomial approximation problem of the activation function, thereby improving the accuracy of model inference. Simultaneously, it protects user privacy and security by providing a verification function to ensure the authenticity of the inference process and results. Attached Figure Description

[0052] Figure 1 This is the overall flowchart of the present invention for secure reasoning in neural networks;

[0053] Figure 2 This is a flowchart illustrating the overall verifiable functionality of the present invention;

[0054] Figure 3 This invention relates to a method for packaging ciphertext in convolutional layers. Detailed Implementation

[0055] To better understand the present invention, the following embodiments further illustrate the content of the present invention, but the content of the present invention is not limited to the following embodiments.

[0056] Example 1

[0057] In this embodiment, we use the MNIST handwritten digit set as the dataset and construct a three-layer neural network architecture, including convolutional layers, activation function layers, and fully connected layers. This model has been trained using plaintext. Figure 1 The entire secure reasoning process is described. The neural network-based secure reasoning method using homomorphic encryption and blockchain includes the following steps:

[0058] Step 1: Initialization: Set the CKKS encryption parameters, where the degree of the polynomial modulus is set to 8192, and the modulus of the polynomial ring is set to {60, 40, 40, 60}. The scaling factor Δ is 40. The user and the model service provider generate their respective public and private keys.

[0059] Step 2: Data Preprocessing and Ciphertext Packaging: For MNIST data, which corresponds to grayscale images of size 28*28, the images are standardized and normalized to make each pixel a floating-point number in the range [0, 255]. A ciphertext packing algorithm is used to reorganize the image's arrangement. Figure 3 This demonstrates our ciphertext packaging process. The specific process is as follows: Given a 28x28 floating-point matrix, we segment the image according to pre-defined convolutional layer parameters, such as kernel size and stride. The resulting image is divided according to the kernel scanning order, ultimately yielding a set of matrices the same size as the kernel, arranged in the kernel sliding order. In the CKKS parameter settings, the polynomial modulus degree is 8192. Before encryption, CKKS encodes multiple data points onto polynomial coefficients; each polynomial coefficient storing the data is called a ciphertext slot. Due to the characteristics of CKKS, a polynomial modulus degree of 8192 means that one ciphertext has 4096 ciphertext slots, and one ciphertext can encrypt 4096 plaintext points. For the segmented matrix group, each matrix is ​​tiled, and the length of each matrix is ​​checked; if it does not meet the 2... n We padded with 0s, where n is a positive integer. This results in a set of tiled vectors arranged in the order of the convolution kernel scans. We encode these vectors into a ciphertext slot corresponding to the ciphertext. This completes the ciphertext packaging. Finally, we store the ciphertext on the blockchain node.

[0060] Step 3: Model-Safe Inference

[0061] Step 3.1: Convolutional Layer Inference: After receiving the inference request, the model service provider encodes its model parameters using the user's public key. This encoding operation changes the arrangement of the model parameters to better adapt to the ciphertext data optimized by the ciphertext packing algorithm. For each convolutional kernel parameter, matching is performed according to the ciphertext length. Based on the parameter settings, each ciphertext contains 4096 data points. The convolutional kernel parameters are then flattened to obtain a one-dimensional vector, and its length is padded with 0 to 2. n Then, this vector is copied and arranged until the length is 4096. During convolutional layer inference, only the multiplication operation between the encrypted data and the encoded convolution kernel parameters needs to be performed, followed by multiple rotation and accumulation operations to obtain the convolution result. Table 1 shows the inference time using a single MNIST image with the same CKKS encryption parameters for both standard secure inference and inference using the ciphertext packing algorithm. This experiment was conducted under the same computer environment, using the SEAL4.0 homomorphic encryption library, with a convolution kernel size of 7*7 and a stride of (2, 2).

[0062] Table 1: Comparison of inference time using the encrypted packetization algorithm and ordinary inference.

[0063]

[0064] Step 3.2: RLWE Ciphertext to LWE Ciphertext Conversion: After the convolutional layer inference is complete, in an RLWE ciphertext, due to a series of rotations and accumulations in the ciphertext slots, only the elements at specific slot positions are the convolution result. We split an RLWE ciphertext into a set of LWE ciphertexts and extract the elements of the correct convolution result according to their specific positions. Some miscellaneous items are discarded, and finally, a set of LWE ciphertexts arranged in the order of the convolution results is formed.

[0065] LWE encryption is defined as: in, Represents an integer ring modulo q, key This represents a vector of dimension n, where each component is... Inside, message a is a random vector, e is an error term randomly selected from the error distribution χ, n is the vector dimension, b is the inner product result, m is the message, and T is the mathematical transpose. Its decryption is b + a. T s≈m;

[0066] RLWE encryption is defined as: in Indicates the coefficient in Within the range and the polynomial module X n +1 ring, Indicates the coefficient in Within the range and the polynomial module X n +1 ring, key Represents a polynomial key. This represents the error polynomial from which coefficients are independently drawn from the error distribution χ. It is a random polynomial;

[0067] We denot an RLWE ciphertext encrypted by CKKS as follows: Where Δ refers to the scaling factor in CKKS encryption, v represents the encryption vector, and symbol n, The dimension representing the lattice, and Using keys representing different dimensions, we extract each encoded element from the ciphertext ct to obtain a set of LWE ciphertexts in FHEW encryption form: Where l represents the encryption level, i∈ <l>This indicates that index i is within a certain range, specifically, all values ​​within range l. Then, the lookup table T(x) for each LWE ciphertext is evaluated:

[0068] Step 3.3: Activation Function Layer Inference: We adopted the ReLU function as the activation function. After converting the CKKS RLWE type ciphertext to the FHEW LWE type ciphertext, based on the characteristics of the ReLU function: F(x) = Max(0, x), we used the FHEW Boolean circuit function to implement the size comparison operation. This allows for the implementation of a true ReLU activation function operation in the encryption environment, avoiding polynomial approximation.

[0069] Step 3.4: LWE Ciphertext to RLWE Ciphertext Conversion: After activation function operation, when operating on the next linear layer, the ciphertext type needs to be converted. A set of LWE ciphertexts is repackaged into a single RLWE ciphertext to encrypt the encoded vector T(v):

[0070] Step 3.5: Fully Connected Layer Inference: The fully connected layer can be viewed as a special type of convolution operation. After activation function operations, the reconstructed RLWE ciphertext is a ciphertext vector flattened according to the result order. The model holder only needs to encode the fully connected parameters to obtain the fully connected layer inference result through one multiplication and multiple rotations. Table 2 shows the plaintext inference accuracy, CKKS encryption scheme inference accuracy, and our proposed hybrid homomorphic encryption scheme inference accuracy comparison on the MNIST dataset using this neural network architecture.

[0071] Table 2 Comparison of Reasoning Accuracy

[0072] MNIST 98.96％ 98.59％ 98.92％

[0073] Step 4: Upload the model inference results: The model service provider sends the inference results to the blockchain for storage.

[0074] Step 5: User obtains service results. The user retrieves the encrypted results from the blockchain and decrypts them using their private key to obtain the reasoning results.

[0075] Step 6: Result Verification. For example... Figure 2 As shown, when a user has doubts about the inference result, they can send a verification request to an authoritative institution. The authoritative institution will request the private keys of both the user and the model service provider, and retrieve the encrypted data, model parameters, intermediate inference result records, and the inference result from the blockchain. Next, the authoritative institution uses the corresponding private key to decrypt the encrypted data and model parameters. It then performs plaintext inference locally using the inherited model parameters, records intermediate results for comparison, and generates an inference information report to verify the authenticity and validity of the inference process and results. After the inference is complete, the authoritative institution compiles the report and sends it to the user.

[0076] Example 2

[0077] A verifiable neural network secure inference system based on homomorphic encryption and blockchain, comprising:

[0078] Initialization module: Used by both the service requester and the model service provider to generate public and private keys for the CKKS encryption scheme locally; the public and private keys of the service requester are denoted as PK. c SK c The model service provider is denoted as PK. s SK s ;

[0079] Data Processing and Ciphertext Packaging Module: This module handles data processing and ciphertext packaging. The service requester preprocesses the acquired images, converting them into matrix data. Leveraging the SIMD feature of CKKS, multiple data points are packaged into a single ciphertext for computation. By arranging the data before encryption, computational efficiency is improved, accelerating model inference. Finally, the private key SK is used. c The rearranged data is encrypted into ciphertext and sent to the blockchain for storage;

[0080] Model-safe inference module: Used for model-safe inference, which includes convolutional layer inference, RLWE ciphertext to LWE ciphertext conversion, activation function layer inference, LWE ciphertext to RLWE ciphertext conversion, and fully connected layer inference;

[0081] Convolutional layer inference: After receiving an inference request, the model service provider obtains the encrypted data to be inferred from the blockchain node and packages the model parameters into encrypted form. Simultaneously, the model service provider encodes its own model parameters using the public key of the service requester, and then performs a convolution operation. The convolution operation is the process of applying a set of filters to the input data. Each filter slides across the input data and calculates a weighted sum of local regions. The encrypted data has already been arranged in this form during the encrypted packaging; this process can be viewed as performing multiple multiplications and additions on the encrypted data. Assuming ct... m This represents ciphertext that has been processed by ciphertext packaging of a reasoning image. This indicates that there are M filters arranged in a specific order. This indicates that there are M biases arranged in a specific order. The result represents the computational output, ultimately yielding M ciphertext computation results. The following formula illustrates the convolution process of the i-th filter, where Roate represents the ciphertext rotation operation. After multiplication, multiple rotation accumulations are performed, and the variable represents the number of rotation accumulations, incremented from 1, 2, 4... up to log. K*K :

[0082]

[0083] RLWE ciphertext to LWE ciphertext conversion: The RLWE ciphertext is split, the ciphertext encoded in the polynomial is extracted, and a set of LWE ciphertext in FHEW encryption form is obtained and the activation function operation is performed.

[0084] LWE encryption is defined as: in, Represents an integer ring modulo q, key This represents a vector of dimension n, where each component is... Inside, message a is a random vector, e is an error term randomly selected from the error distribution χ, n is the vector dimension, b is the inner product result, m is the message, and T is the mathematical transpose. Its decryption is b + a. T s≈m;

[0085] RLWE encryption is defined as: in This indicates that the coefficients are in the range Z and the polynomial modulus X. n +1 ring, Indicates the coefficient in Within the range and the polynomial module X n +1 ring, key Represents a polynomial key. This represents the error polynomial from which coefficients are independently drawn from the error distribution χ. It is a random polynomial;

[0086] An RLWE ciphertext encrypted with CKKS is denoted as Where Δ refers to the scaling factor in CKKS encryption, v represents the encryption vector, and symbol n, The dimension representing the lattice, and s, Using keys representing different dimensions, we extract each encoded element from the ciphertext ct to obtain a set of LWE ciphertexts in FHEW encryption form: Where l represents the encryption level; then the lookup table T(x) for each LWE ciphertext is evaluated:

[0087]

[0088] Activation function layer inference: Construct a binary comparison circuit for the obtained LWE ciphertext to implement the ReLU operation process of the activation function, and finally obtain the ciphertext operation result of the activation function;

[0089] LWE ciphertext to RLWE ciphertext conversion: After activation function operation, when operating on the next linear layer, the ciphertext type needs to be converted. A set of LWE ciphertexts is repackaged into a single RLWE ciphertext to encrypt the encoded vector T(v):

[0090] Fully connected layer inference: Fully connected layers are special convolutional layer operations, and use a packing algorithm similar to that of convolutional layers for fully connected layer operations.

[0091] Upload Model Inference Results Module: Used by model service providers to upload encrypted inference results to the blockchain;

[0092] The service result acquisition module is used by service requesters to obtain service results. Service requesters obtain encrypted reasoning results from the blockchain and decrypt them using their own private key to obtain the reasoning results.

[0093] Result Verification Module: Used for result verification. When a service requester has doubts about the inference results of the model service provider, the service requester sends a verification request to an authoritative institution. The authoritative institution will send a request to both the service requester and the model service provider to obtain their private keys. After obtaining the private keys from both parties, the authoritative institution retrieves the encrypted model parameters and encrypted data from the blockchain, performs decryption operations, performs plaintext model inference operations locally, and retrieves intermediate results and inference results from the blockchain for comparison to generate a verification report for the service requester to verify.

[0094] This invention protects user privacy and security while ensuring the reliability of inference results. By using a conversion method between RLWE (Ring Learning with Errors) ciphertext and LWE (Learning with Errors) ciphertext, we avoid the activation function polynomial approximation problem, improving the accuracy and efficiency of model inference. Simultaneously, to protect user privacy and security, we provide a verification function to ensure the authenticity of the inference process and results.< / l> < / l> < / l>

Claims

1. A verifiable neural network secure reasoning method based on homomorphic encryption and blockchain, characterized in that: Includes the following steps: Step 1: Initialization: The service requester and the model service provider generate a public and private key for the CKKS encryption scheme locally; the public and private keys of the service requester are denoted as follows: The model service provider is denoted as ; Step 2: Data Processing and Ciphertext Packaging: The service requester preprocesses the collected images, converting them into matrix data. Taking advantage of CKKS's SIMD feature, multiple data sets are packaged into a single ciphertext for computation. By arranging the data before encryption, computational efficiency is improved, accelerating model inference. Finally, the private key is used... The rearranged data is encrypted into ciphertext and sent to the blockchain for storage. Step 3: Model-safe inference: including convolutional layer inference, RLWE ciphertext to LWE ciphertext conversion, activation function layer inference, LWE ciphertext to RLWE ciphertext conversion, and fully connected layer inference; Step 4: Upload Model Inference Results: The model service provider uploads the encrypted inference results to the blockchain; Step 5: Service requester obtains service result: The service requester obtains the encrypted reasoning result from the blockchain and decrypts it using their own private key to obtain the reasoning result; Step 6: Result Verification: When the service requester has doubts about the inference results of the model service provider, the service requester sends a verification request to the authoritative institution. The authoritative institution will send a request to obtain the private key to both the service requester and the model service provider. After obtaining the private keys from both parties, the authoritative institution retrieves the encrypted model parameters and encrypted data from the blockchain, performs decryption operations, performs plaintext model inference operations locally, and retrieves intermediate results and inference results from the blockchain for comparison to generate a verification report for the service requester to verify. Step 3: Model-safe reasoning includes: Step 3.1: Convolutional Layer Inference: After receiving the inference request, the model service provider obtains the encrypted data to be inferred from the blockchain node and packages the model parameters into encrypted form. Simultaneously, the model service provider encodes its own model parameters using the public key of the service requester, and then performs a convolution operation. The convolution operation is the process of applying a set of filters to the input data. Each filter slides across the input data and calculates a weighted sum of local regions. The encrypted data has already been arranged in this form during the encrypted packaging; this process can be viewed as performing multiple multiplications and additions on the encrypted data. Assuming... This represents ciphertext that has been processed by ciphertext packaging of a reasoning image. Indicates that there is A filter arranged in a specific order. Indicates that there is A bias arranged in a specific order. The result represents the calculation result, which is the final result. The result of the ciphertext calculation is shown in the following formula; The process of convolving multiple filters involves Roate representing a ciphertext rotation operation. After multiplication, multiple rotations and accumulations are performed, and the variables... This represents the cumulative number of rotations, incremented by 1, 2, 4... up to... : ； Step 3.2: RLWE ciphertext to LWE ciphertext conversion: split the RLWE ciphertext, extract the ciphertext encoded in the polynomial, and obtain a set of LWE ciphertext in FHEW encryption form. Then perform the activation function operation. LWE encryption is defined as: ,in, Modulus Integer ring, key The dimension is The vector, each component is in Inside, message , , For random vectors, To follow the error distribution The random error term, where n is the vector dimension, b is the inner product result, m is the message, and T is the mathematical transpose, is decrypted as follows: ; RLWE encryption is defined as: ,in Indicates the coefficient in Within range and polynomial modulus The ring, Indicates the coefficient in Within range and polynomial modulus ring, key Represents a polynomial key. Represents the error distribution The error polynomial of the independently extracted coefficients. It is a random polynomial; An RLWE ciphertext encrypted by CKKS is denoted as ,in, This refers to the scaling factor in CKKS encryption. Represents the encryption vector, and ,symbol The dimension representing the lattice, and , Keys representing different dimensions, for ciphertext Extracting each encoded element from the data yields a set of LWE ciphertext in FHEW encryption format: ,in This represents the encryption level; followed by a lookup table for each LWE ciphertext. Conduct an assessment: ; Step 3.3: Activation Function Layer Inference: Construct a binary comparison circuit for the obtained LWE ciphertext to implement the ReLU operation process of the activation function, and finally obtain the ciphertext operation result of the activation function; Step 3.4: LWE Ciphertext to RLWE Ciphertext Conversion: After activation function operation, when operating on the next linear layer, the ciphertext type needs to be converted. A group of LWE ciphertexts is repackaged into a single RLWE ciphertext, and the encoding vector is processed accordingly. Encrypt: ; Step 3.5: Fully Connected Layer Inference: The fully connected layer is a special type of convolutional layer operation, and the fully connected layer operation is performed using a packing algorithm that adopts the convolutional layer operation method.

2. A verifiable neural network secure inference system based on homomorphic encryption and blockchain, characterized in that: include: Initialization module: Used by both the service requester and the model service provider to generate public and private keys for the CKKS encryption scheme locally; whereby the public and private keys of the service requester are denoted as... The model service provider is denoted as ; Data Processing and Ciphertext Packaging Module: This module handles data processing and ciphertext packaging. The service requester preprocesses the acquired images, converting them into matrix data. Leveraging the SIMD feature of CKKS, multiple data points are packaged into a single ciphertext for computation. By arranging the data before encryption, computational efficiency is improved, accelerating model inference. The private key is then used... The rearranged data is encrypted into ciphertext and sent to the blockchain for storage; Model-safe inference module: Used for model-safe inference, which includes convolutional layer inference, RLWE ciphertext to LWE ciphertext conversion, activation function layer inference, LWE ciphertext to RLWE ciphertext conversion, and fully connected layer inference; Upload Model Inference Results Module: Used by model service providers to upload encrypted inference results to the blockchain; The service result acquisition module is used by service requesters to obtain service results. Service requesters obtain encrypted reasoning results from the blockchain and decrypt them using their own private key to obtain the reasoning results. Result Verification Module: Used for result verification. When a service requester has doubts about the inference results of the model service provider, the service requester sends a verification request to an authoritative institution. The authoritative institution will send a request to both the service requester and the model service provider to obtain their private keys. After obtaining the private keys from both parties, the authoritative institution retrieves the encrypted model parameters and encrypted data from the blockchain, performs decryption operations, performs plaintext model inference operations locally, and retrieves intermediate results and inference results from the blockchain for comparison to generate a verification report for the service requester to verify. The model-safe reasoning module includes the following steps when performing model-safe reasoning: Convolutional layer inference: After receiving an inference request, the model service provider obtains the encrypted data to be inferred from the blockchain node and packages the model parameters into encrypted form. Simultaneously, the model service provider encodes its own model parameters using the public key of the service requester, and then performs a convolution operation. The convolution operation is the process of applying a set of filters to the input data. Each filter slides across the input data and calculates a weighted sum of a local region. The encrypted data has already been arranged in this form during the encrypted packaging; this process can be viewed as performing multiple multiplications and additions on the encrypted data. Assuming... This represents ciphertext that has been processed by ciphertext packaging of a reasoning image. Indicates that there is A filter arranged in a specific order. Indicates that there is A bias arranged in a specific order. The result represents the calculation result, which is the final result. The result of the ciphertext calculation is shown in the following formula; The process of convolving multiple filters involves Roate representing a ciphertext rotation operation. After multiplication, multiple rotations and accumulations are performed, and the variables... This represents the cumulative number of rotations, incremented by 1, 2, 4... up to... : ； RLWE ciphertext to LWE ciphertext conversion: The RLWE ciphertext is split, the ciphertext encoded in the polynomial is extracted, and a set of LWE ciphertext in FHEW encryption form is obtained and the activation function operation is performed. LWE encryption is defined as: ,in, Modulus Integer ring, key The dimension is The vector, each component is in Inside, message , , For random vectors, To follow the error distribution The random error term, where n is the vector dimension, b is the inner product result, m is the message, and T is the mathematical transpose, is decrypted as follows: ; RLWE encryption is defined as: ,in Indicates the coefficient in Within range and polynomial modulus The ring, Indicates the coefficient in Within range and polynomial modulus ring, key Represents a polynomial key. Represents the error distribution The error polynomial of the independently extracted coefficients. It is a random polynomial; An RLWE ciphertext encrypted by CKKS is denoted as ,in, This refers to the scaling factor in CKKS encryption. Represents the encryption vector, and ,symbol The dimension representing the lattice, and , Keys representing different dimensions, for ciphertext Extracting each encoded element from the data yields a set of LWE ciphertext in FHEW encryption format: ,in This represents the encryption level; followed by a lookup table for each LWE ciphertext. Conduct an assessment: ; Activation function layer inference: Construct a binary comparison circuit for the obtained LWE ciphertext to implement the ReLU operation process of the activation function, and finally obtain the ciphertext operation result of the activation function; LWE ciphertext to RLWE ciphertext conversion: After activation function operation, when operating on the next linear layer, the ciphertext type needs to be converted. A group of LWE ciphertexts is repackaged into a single RLWE ciphertext, and the encoding vector is processed accordingly. Encrypt: ; Fully connected layer inference: Fully connected layers are special convolutional layer operations, and a packing algorithm using convolutional layer operations is used for fully connected layer operations.

3. A verifiable neural network secure inference device based on homomorphic encryption and blockchain, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method as described in claim 1.

4. A computer-readable storage medium, characterized in that: The device contains a computer program that, when executed by a processor, implements the method as described in claim 1.

Images