A verifiable convolutional neural network method and device based on zero-knowledge proof

Abstract

The present invention provides a verifiable convolutional neural network method based on zero-knowledge proof, which belongs to the field of neural network security. The method divides the original convolutional neural network into several convolutional neural network layers, converts the proof task of the original convolutional neural network into sub-proof tasks of each convolutional neural network layer, and decomposes the computational operations in the convolutional neural network into the constraint relationship between the input and output of the neural network, so that the number of constraints required is reduced; the zero-knowledge proof technology is used to prove that each sub-proof task satisfies a specific constraint relationship, and the result of the sub-proof task is reduced to the result of the original convolutional neural network, which significantly improves the proof speed of the convolutional neural network without leaking the model parameters, and is more efficient and feasible in practical applications. The present invention also provides a verifiable convolutional neural network device based on zero-knowledge proof, which realizes a verifiable convolutional neural network method of zero-knowledge proof.

Description

Technical Field

[0001] The present invention relates to the field of neural network security, and in particular to a verifiable convolutional neural network method and device based on zero-knowledge proof. Background Art

[0002] Convolutional Neural Network (CNN) is a neural network structure widely used in the fields of computer vision and pattern recognition. CNN consists of a series of convolutional layers and pooling layers, which can effectively extract the spatial structural features of the input image.

[0003] Through stacked convolution operations, convolutional neural networks achieve the gradual extraction and learning of high-level abstract features from raw image pixels, and have demonstrated outstanding performance in areas such as image recognition and object detection. In the past decade, neural networks in deep learning have received unprecedented attention in various fields such as medical examinations, industrial fault detection, and natural language processing. Therefore, cloud providers provide users with paid services such as Machine-Learning as-a-Service (MLaaS) based on cloud-based data analysis platforms.

[0004] However, in the traditional MLaaS framework, face recognition requests are sent from applications to servers, and the server returns face recognition results to applications. Well-trained models are considered the intellectual property of cloud service providers and are kept strictly confidential. However, users lack tools to verify whether the results provided by cloud vendors are indeed the inference results of specific models, which makes it impossible for users to fully trust the results of cloud service providers. In addition, malicious cloud platforms may provide false recognition results, and applications cannot determine the validity of the recognition results, which also limits the application of MLaaS in safety-critical areas such as medical examinations.

[0005] Zero Knowledge Protocol (ZKP) is an advanced cryptographic technique that allows model owners to prove to users that the returned results are indeed the inference results of the model while protecting the privacy of the model. By leveraging mathematical puzzles, model owners are able to prove the authenticity of an assertion to external verifiers without revealing specific information about the model itself. This approach is particularly suitable for scenarios where user data and model details need to be protected, such as medical diagnosis, financial forecasting, or personalized recommendations.

[0006] For example, a patent application with publication number CN117975543A discloses a blockchain zero-knowledge identity authentication credential interaction method based on optical flow expression. By constructing a convolutional neural network model of multi-dimensional deep optical flow estimation, the user's real face and the expression of the user's randomly selected face are fused. The fused face includes the identity of the user's randomly selected face, the overall head and local key features of the user's randomly selected face and the user's real face, which replaces the user's real face for identity authentication, so that no information about the user's real face will be exposed; after completing the identity verification, the identity provider outputs a verifiable credential. With the verifiable credential, the client obtains the zero-knowledge proof constraint verification and proof key from the service provider, and uses the zero-knowledge proof verification process to generate a zero-knowledge proof of the verifiable credential; the client submits the zero-knowledge proof and the digital signature of the identity provider to the service provider to request zero-knowledge identity authentication and obtain the corresponding service; through ZKP, model owners can ensure that their models are accurate when performing calculations and returning results, while avoiding the leakage of any key information, thereby improving the credibility and security of the service while protecting privacy.

[0007] Although considerable progress has been made in the development of ZKP for neural networks, existing ZKPs rely on multi-party secure computations. These secure computation processes are very complex, computationally intensive, and involve multiple rounds of data interactions. The overall efficiency is low, making it difficult to deploy in practice. Summary of the invention

[0008] The purpose of the present invention is to provide a verifiable convolutional neural network method and device based on zero-knowledge proof, which decomposes the computational operations in the convolutional neural network model into constraint relationships between input and output, and then uses zero-knowledge proof technology to efficiently verify the correctness of the convolutional neural network model reasoning results without leaking model parameters.

[0009] To achieve the above-mentioned purpose of the invention, an embodiment provides a verifiable convolutional neural network method based on zero-knowledge proof, comprising the following steps:

[0010] Divide the original convolutional neural network used for image recognition into several neural network layers, regard each neural network layer as a sub-proof task, calculate the output based on the input of each neural network layer, and store the input and output of each neural network layer;

[0011] Classify the sub-proof tasks according to the type of computational operation of each neural network layer, and convert the sub-proof tasks into constraint relationships between the input and output of each neural network layer based on the type of computational operation;

[0012] The zero-knowledge proof method is used to prove that the input and output of each sub-proof task satisfy specific constraints, and the proof results of each sub-proof task are obtained. The proof results of all sub-proof tasks are reduced to the proof results of the original neural network, completing the verification of the original convolutional neural network reasoning results in image recognition.

[0013] In one embodiment, the original convolutional neural network is divided into several neural network layers according to a hierarchical structure, including: based on the hierarchical structure of linear layer, ReLu activation layer and maximum pooling layer, the original convolutional neural network is divided to obtain several neural network layers.

[0014] In one embodiment, the method of converting the sub-proof tasks into the constraint relationship between the input and output of each neural network layer based on the type of computing operation includes: classifying the sub-proof tasks by linear layer computing operations, ReLu activation layer computing operations, and maximum pooling layer computing operations, and converting the sub-proof tasks into the constraint relationship between the input and output of each neural network layer.

[0015] In one embodiment, the linear layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the linear layer, expressed as ,in, represents the input of the linear layer, represents the output of the linear layer, represents the weight matrix of the linear layer, represents the bias vector;

[0016] The ReLu activation layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the ReLu activation layer, expressed as ,in, represents the input of the ReLu activation layer, Represents the output of the ReLu activation layer;

[0017] The maximum pooling layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the maximum pooling layer, expressed as ,in, is the convolution kernel size, represents the maximum pooling of the convolutional layer, Indicates The input of the convolutional layer, Represents the output of the maximum pooling of the convolutional layer.

[0018] In one embodiment, the zero-knowledge proof method is used to prove that the input and output of each sub-proof task satisfy the constraint relationship, including: using an information-theoretic message authentication code to authenticate the private message of the prover. Carry out certification so that the certification results meet The constraint relationship, where the prover's private message Include the input of the linear layer , the input of the ReLu activation layer , the input of the maximum pooling layer , the weight matrix and bias vector of the linear layer, Held by the certifier The corresponding certification label, Held by the validator, The corresponding local key, Δ is the global key held by the verifier.

[0019] In one embodiment, the authentication result satisfies The constraints include: To authenticate, the certifier obtains , the verifier gets , the prover passes the identity constraint ,prove equal ;right For authentication, the prover obtains 𝑚, the verifier obtains 𝑘, and the prover passes the range constraint , convince the verifier to be within a certain range.

[0020] In one embodiment, reducing the proof results of all sub-proof tasks to the proof results of the original neural network includes: representing the proof results of the sub-proof tasks as , through the formula Reduce the proof result of each sub-proof task to the proof result of the original neural network, where 𝐵 is the proof result of the original convolutional neural network, and & represents the reduction calculation operation.

[0021] In order to clearly demonstrate a verifiable convolutional neural network method based on zero-knowledge proof, a verifiable convolutional neural network device based on zero-knowledge proof is provided, including a memory and a processor, the memory is used to store a computer program, and the processor is used to implement a verifiable convolutional neural network method based on zero-knowledge proof when executing the computer program.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] Compared with the existing methods, the verifiable convolutional neural network method based on zero-knowledge proof provided by the present invention divides the original convolutional neural network into several convolutional neural network layers, converts the proof task of the original convolutional neural network into sub-proof tasks of each convolutional neural network layer, and decomposes the computational operations in the convolutional neural network into the constraint relationship between the input and output of the neural network, so that the number of constraints required is reduced;

[0024] Zero-knowledge proof technology is used to prove that each sub-proof task satisfies a specific constraint relationship, and the results of the sub-proof tasks are reduced to the results of the original convolutional neural network. This significantly improves the proof speed of the convolutional neural network without leaking model parameters, making it more efficient and feasible in practical applications.

[0025] The present invention also provides a verifiable convolutional neural network device based on zero-knowledge proof, and implements a verifiable convolutional neural network method based on zero-knowledge proof. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0027] Figure 1 The flowchart of the verifiable convolutional neural network method based on zero-knowledge proof.

[0028] Figure 2 Schematic diagram of the structure of a convolutional neural network divided into sub-proof tasks according to the hierarchical structure.

[0029] Figure 3 Schematic diagram of the structure of a verifiable convolutional neural network based on zero-knowledge proof. DETAILED DESCRIPTION

[0030] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0031] In order to improve the credibility and security of face recognition services without leaking user privacy, the embodiment provides a verifiable convolutional neural network method based on zero-knowledge proof. The method decomposes the computational operations in the convolutional neural network model into constraint relationships between input and output, and then uses zero-knowledge proof technology to efficiently verify the correctness of the inference results of the convolutional neural network model and protect user privacy.

[0032] This paper takes the face recognition access control system based on MLaaS as an example. Based on the face recognition task, a verifiable convolutional neural network method based on zero-knowledge proof is used. In the traditional framework, such as Figure 1 As shown, the convolutional neural network method can be verified, including the following steps:

[0033] S1. Divide the original convolutional neural network used for image recognition into several neural network layers. Each neural network layer is regarded as a sub-proof task. The output is calculated based on the input of each neural network layer, and the input and output of each neural network layer are stored.

[0034] Convolutional neural networks can be stacked up with various layers, including linear layers (such as matrix cross product, convolution, residual connection, etc.), ReLU activation layers, maximum pooling layers and residual connections, etc. Such neural network architectures can cover a wide range of model types, such as classic LeNet, VGGNet, ResNet, etc. Therefore, the present invention has a wide range of applicability and can be applied to multiple fields such as image recognition and object detection. In the embodiment, the original convolutional neural network is divided into several neural network layers based on the linear layer, ReLU activation layer and maximum pooling layer, and each neural network layer is regarded as a sub-proof task.

[0035] The original convolutional neural network is divided into several neural network layers according to the hierarchical structure. For example, a typical convolutional neural network usually includes: a linear layer (including matrix cross product, convolution, residual connection, etc.), a ReLu activation layer and a maximum pooling layer; for the user's initial input, the output of all convolutional neural network layers is obtained according to the calculation order in the original convolutional neural network. Since the output of the previous convolutional neural network layer is the input of the next convolutional neural network layer, it is only necessary to store the user's initial input and the calculation results of each convolutional neural network layer.

[0036] S2. Classify the sub-proof tasks according to the type of computing operations of each neural network layer, and convert the sub-proof tasks into constraint relationships between the input and output of each neural network layer based on the type of computing operations.

[0037] In the embodiment, Figure 2 As shown, the sub-proof tasks are converted into the constraint relationship between the input and output of each neural network layer according to the computing operation type of the original convolutional neural network, including: the sub-proof tasks are classified by three computing operations, namely, linear layer computing operation, ReLu activation layer computing operation and maximum pooling layer computing operation, and then the sub-proof tasks are converted into the constraint relationship between the input and output of each neural network layer, and the same convolutional neural network layer uses the same constraint relationship.

[0038] The constraints of each layer in the convolutional neural network are as follows: (1) The linear layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the linear layer, expressed as ,in, represents the input of the linear layer, represents the output of the linear layer, represents the weight matrix of the linear layer, represents the bias vector; (2) The ReLu activation layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the ReLu activation layer, expressed as ,in, represents the input of the ReLu activation layer, represents the output of the ReLu activation layer; (3) The maximum pooling layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the maximum pooling layer, expressed as ,in, is the convolution kernel size, represents the maximum pooling of the convolutional layer, Indicates The input of the convolutional layer, Represents the output of the maximum pooling of the convolutional layer.

[0039] S3. Use the zero-knowledge proof method to prove that the input and output of each sub-proof task satisfy specific constraints, obtain the proof result of each sub-proof task, reduce the proof results of all sub-proof tasks to the proof results of the original neural network, and complete the verification of the original convolutional neural network reasoning results in image recognition.

[0040] In one embodiment, the private message of the prover is authenticated by Information Theoretic Message Authentication Codes (IT-MACs). Carry out certification so that the certification results meet The constraint relationship, where the prover's private message Include the input of the linear layer , the input of the ReLu activation layer , the input of the maximum pooling layer , the weight matrix and bias vector of the linear layer, Held by the certifier The corresponding certification label, Held by the validator, The corresponding local key, Δ is the global key held by the verifier.

[0041] Certification mainly includes two categories: (1) To authenticate, the certifier obtains , the verifier gets , the prover passes the identity constraint ,prove equal ; (2) Yes For authentication, the prover obtains 𝑚, the verifier obtains 𝑘, and the prover passes the range constraint , convince the verifier to be within a certain range.

[0042] All sub-proof tasks must use zero-knowledge proof methods to prove that the input and output of the sub-tasks satisfy specific constraints, and then prove that the output of each convolutional network layer is indeed completed by the corresponding calculation operation; then the results of all sub-proof tasks are reduced to the proof results of the original convolutional neural network, where the proof result of each sub-proof task is expressed as a Boolean value result , then the proof result of the original convolutional neural network , For the reduction operation.

[0043] According to the verifiable convolutional neural network method based on zero-knowledge proof proposed in the present invention, combined with the face recognition access control system based on MLaaS, the structure of the verifiable convolutional neural network based on zero-knowledge proof is as follows: Figure 3 As shown, including:

[0044] Server neural network, client application, zero-knowledge proof prover, and zero-knowledge proof verifier.

[0045] Server neural networks are machine learning models of cloud platform providers in the traditional MLaaS (machine learning as a service) framework, responsible for providing a variety of services to client applications, such as natural language processing, face recognition, data visualization, predictive analysis, data modeling APIs, and deep learning tools.

[0046] The client application interacts with the server neural network and provides users with a user-friendly interface, making it easy for non-technical users to use machine learning tools.

[0047] The zero-knowledge prover converts the input, output and weight of the convolutional neural network (i.e., the constraint relationship mentioned in the embodiment) into a proof, which can prove the validity of the reasoning result to the verifier without revealing the weight of the cloud platform provider's neural network, thereby improving the credibility of the service.

[0048] The zero-knowledge verifier checks the proof and returns the request result and the corresponding proof result to the application.

[0049] In order to clearly demonstrate the verifiable convolutional neural network method based on zero-knowledge proof, an embodiment also provides a verifiable convolutional neural network device based on zero-knowledge proof, including a memory and a processor, the memory is used to store a computer program, and the processor is used to implement the verifiable convolutional neural network method based on zero-knowledge proof when executing the computer program.

[0050] Based on the above-mentioned verifiable convolutional neural network method and device based on zero-knowledge proof, the computational operations in the convolutional neural network model are decomposed into the constraint relationship between input and output, and then the zero-knowledge proof technology is used to efficiently verify the correctness of the reasoning results of the convolutional neural network model without leaking the model parameters, and improve the efficiency of zero-knowledge proof, so that it can be widely promoted in practical applications.

[0051] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A verifiable convolutional neural network method based on zero-knowledge proof, characterized in that: The following steps are involved: Divide the original convolutional neural network used for image recognition into several neural network layers, regard each neural network layer as a sub-proof task, calculate the output based on the input of each neural network layer, and store the input and output of each neural network layer; Classify the sub-proof tasks according to the type of computational operation of each neural network layer, and convert the sub-proof tasks into constraint relationships between the input and output of each neural network layer based on the type of computational operation; The zero-knowledge proof method is used to prove that the input and output of each sub-proof task satisfy specific constraints, and the proof results of each sub-proof task are obtained. The proof results of all sub-proof tasks are reduced to the proof results of the original neural network, completing the verification of the original convolutional neural network reasoning results in image recognition.

2. The verifiable convolutional neural network method according to claim 1, characterized in that: The dividing of the original convolutional neural network into several neural network layers includes: dividing the original convolutional neural network into several neural network layers based on the hierarchical structure of linear layer, ReLu activation layer and maximum pooling layer.

3. The verifiable convolutional neural network method according to claim 1, characterized in that: The method of converting the sub-proof tasks into the constraint relationship between the input and output of each neural network layer based on the type of computing operation includes: classifying the sub-proof tasks by linear layer computing operations, ReLu activation layer computing operations and maximum pooling layer computing operations, and converting the sub-proof tasks into the constraint relationship between the input and output of each neural network layer.

4. The verifiable convolutional neural network method according to claim 3, characterized in that: The linear layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the linear layer, expressed as ,in, represents the input of the linear layer, represents the output of the linear layer, represents the weight matrix of the linear layer, represents the bias vector; The ReLu activation layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the ReLu activation layer, expressed as ,in, represents the input of the ReLu activation layer, Represents the output of the ReLu activation layer; The maximum pooling layer calculation operation is , converting the sub-proof task into the constraint relationship between the input and output of the maximum pooling layer, expressed as ,in, is the convolution kernel size, represents the maximum pooling of the convolutional layer, Indicates The input of the convolutional layer, Represents the output of the maximum pooling of the convolutional layer.

5. The verifiable convolutional neural network method according to claim 1, characterized in that: The zero-knowledge proof method is used to prove that the input and output of each sub-proof task satisfy the constraint relationship, including: using information-theoretic message authentication codes to authenticate the private message of the prover. Carry out certification so that the certification results meet The constraint relationship, where the prover's private message Include the input of the linear layer , the input of the ReLu activation layer , the input of the maximum pooling layer , the weight matrix and bias vector of the linear layer, Held by the certifier The corresponding certification label, Held by the validator, The corresponding local key, Δ is the global key held by the verifier.

6. The verifiable convolutional neural network method according to claim 5, characterized in that: The certification results meet the The constraints include: To authenticate, the certifier obtains , the verifier gets , the prover passes the identity constraint ,prove equal ;right For authentication, the prover obtains 𝑚, the verifier obtains 𝑘, and the prover passes the range constraint , convince the verifier to be within a certain range.

7. The verifiable convolutional neural network method according to claim 1, characterized in that: The method of reducing the proof results of all sub-proof tasks to the proof results of the original neural network includes: expressing the proof results of the sub-proof tasks as , through the formula Reduce the proof result of each sub-proof task to the proof result of the original neural network, where 𝐵 is the proof result of the original convolutional neural network, and & represents the reduction calculation operation.

8. A verifiable convolutional neural network device based on zero-knowledge proof, comprising a memory and a processor, wherein the memory is used to store a computer program, characterized in that: The processor is used to implement the verifiable convolutional neural network method based on zero-knowledge proof as described in any one of claims 1 to 7 when executing the computer program.

Images