Function encrypted convolutional neural network privacy reasoning method and system

Abstract

The invention belongs to the technical field of artificial intelligence privacy protection, and discloses a function encrypted convolutional neural network privacy reasoning method and system. The classification accuracy of the scheme of the invention is similar to plaintext reasoning, and the classification accuracy is only slightly reduced. For example, the plaintext accuracy of ResNet-18 on CIFAR-10 is 90.69%, the plaintext accuracy of ResNet-18 on CIFAR-10 is 89.70% after a method I is adopted, the plaintext accuracy of ResNet-18 on CIFAR-10 is 89.55% after a method II is adopted, and the precision loss is controlled within 1%. According to the scheme, only the early layer (features before pooling) of the network is encrypted, and the subsequent layer keeps plaintext calculation, so that accumulative errors caused by layer-by-layer encryption are avoided. After the scheme of the invention is adopted, the similarity of images recovered by model inversion attack is obviously reduced. The similarity recovered under the plaintext condition is 1.0, while the similarity under the method I and the method II is only about 0.16, which is far lower than that of a traditional encryption scheme. Therefore, the input information is effectively blurred through pre-polymerization and blind processing, so that the original image features are difficult to restore even if an attacker masters pooling output.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence privacy protection, and particularly relates to a function encryption convolutional neural network privacy inference method and system. BACKGROUND

[0002] When deploying a convolutional neural network (CNN) model for inference in the cloud, users often need to send original images containing sensitive information to the server in plaintext form. This plaintext upload method has a serious privacy leakage risk, and the server can directly access the user's visual data. Existing privacy protection inference methods mainly use homomorphic encryption (HE), secure multi-party computation (MPC), or differential privacy (DP) technologies. However, these methods have limitations: homomorphic encryption requires performing convolution and nonlinear operations in the ciphertext domain, with huge computational overhead; MPC protocols involve data segmentation and a large number of interactions, resulting in high communication costs; differential privacy protects data by injecting noise, but often causes model accuracy to decrease. As the number of network layers increases, these bottlenecks will further accumulate, making it difficult for traditional solutions to meet practical requirements.

[0003] Recently proposed inner product function encryption (IPFE) technology provides an effective means for neural network inference [1, 2]. IPFE allows the server to calculate the inner product using a pre-distributed function key on encrypted input, and only the first layer of the model needs to be encrypted, while the remaining layers are calculated in the plaintext domain, thereby significantly reducing computational complexity and avoiding accuracy loss. However, directly applying IPFE to the convolutional layer poses a security risk: the receptive field of the convolution kernel is usually small (for example, a $3*3$ convolution corresponds to a 9-dimensional vector), and when the number of convolution filters is greater than the input dimension, the server can use known weights and multiple inner product results to solve linear equations to restore the input image. In other words, directly exposing the inner product results in the convolutional layer poses a serious risk of reverse restoration attacks. Existing research has found that simply encrypting the convolution output using IPFE does not adequately conceal the original features.

[0004] Through the above analysis, the existing problems and defects of the prior art are:

[0005] Directly applying IPFE to the convolutional layer poses a security risk: the receptive field of the convolution kernel is usually small (for example, a $3*3$ convolution corresponds to a 9-dimensional vector), and when the number of convolution filters is greater than the input dimension, the server can use known weights and multiple inner product results to solve linear equations to restore the input image. In other words, directly exposing the inner product results in the convolutional layer poses a serious risk of reverse restoration attacks. Existing research has found that simply encrypting the convolution output using IPFE does not adequately conceal the original features. SUMMARY

[0006] Aiming at the problems existing in the prior art, the application provides a function encryption convolutional neural network privacy reasoning method.

[0007] The application is implemented as follows: a function encryption convolutional neural network privacy reasoning method comprises the following steps:

[0008] Step 1, fully utilizing the aggregation blur characteristics of the pooling layer, the inner product result of the convolution layer is exposed after being postponed to the pooling layer or is subjected to random blinding processing, so that a reasoning process with low calculation overhead and low privacy leakage risk is realized;

[0009] Step 2, combining the information blur brought by the pooling and the blinding disturbance mechanism, the protection ability to the original input is improved under the premise of ensuring the model accuracy.

[0010] Further, the pooling comprises:

[0011] The first stage of the ResNet is modified, the average pooling is used instead of the maximum pooling, and the activation function is moved to the pooling layer.

[0012] Further, the aggregation comprises:

[0013] 1) Symbol definition

[0014] In order for those skilled in the art to understand the application scheme, the important symbols involved are defined as follows in this section:

[0015] The input feature map is denoted as Where H is the height, W is the width, and C is the number of channels.

[0016] The convolution kernel (the tth convolution kernel) is denoted as Where k is the size of the convolution kernel.

[0017] The local patch extracted at the convolution position (a,b) is denoted as ,

[0018] The pooling window size is p x p, and the step is p.

[0019] The set of convolution positions covered by the pooling position (i,j) is denoted as ,

[0020] 2) Standard convolution + average pooling process

[0021] The tth channel convolution result output by the traditional convolution layer is denoted as:

[0022] ,

[0023] Where the symbol ⟨⋅,⋅> represents the vectorized inner product operation; after completing the convolution calculation, the average pooling operation averages the convolution results in the pooling area to obtain

[0024] ,

[0025] 3) The pre-aggregation patch structure of the present application

[0026] In order to avoid the server from seeing all un-aggregated patches , the present application directly constructs an aggregated patch at the client side

[0027] ,

[0028] Where represents the aggregated patch corresponding to the pooling position (i,j), that is, the average of all future convolution patches in this pooling area by element. This step is completely executed at the client side, and the server cannot see the individual ;

[0029] 4) Convolution operation at the server side

[0030] After receiving the encrypted , the server uses the function key to perform inner product calculation

[0031] ,

[0032] This result is equivalent to:

[0033] ,

[0034] This formula shows that pre-aggregation and then convolution = convolution and then average pooling (completely mathematically equivalent), so the pre-aggregation scheme of the present application completely guarantees the model structure consistency and calculation correctness;

[0035] 5) Encryption and inference process

[0036] After obtaining the aggregated patch , the client performs the following steps:

[0037] Vectorize the patch and call the IPFE encryption algorithm to generate ciphertext ,

[0038] Send the ciphertext to the server;

[0039] Server side:

[0040] Use the weight derived function key to perform inner product decryption calculation;

[0041] The decryption result directly obtains the pooling convolution output ,

[0042] Throughout the process, the server never sees , the earliest intermediate output exposed is only the result after pooling, and effectively blocks the leakage of early fine-grained features.

[0043] Further, the blind perturbation mechanism:

[0044] 1) Symbol definition

[0045] In order to facilitate the understanding of the complete process of the blinding mechanism, the related symbols are as follows:

[0046] The vectorized weight of the convolution kernel (the tth kernel) is denoted as , where is the vector dimension, and p is a large prime number;

[0047] The secret vector in the IPFE master key is denoted as ,

[0048] The function key generated by the traditional IPFE for the convolution kernel is ,

[0049] The random blinding vector is defined as , where each element satisfies the constraint ,

[0050] The blinding offset of the tth convolution kernel is denoted as ,

[0051] represents the input patch at convolution position (a, b);

[0052] represents the regular convolution output;

[0053] represents the regular pooling output;

[0054] represents the server-side reasoning result with blinding;

[0055] 2) Construction of blind function key

[0056] In order to hide the real inner product result, the client adds a blinding item when generating the function key, so that the convolution result calculated by the server is always shifted; the blinding function key is defined as:

[0057] ,

[0058] where is the original function key; For random blind offset, generated by the client side; the blind offset remains consistent for the entire convolution channel; the server cannot distinguish between the real inner product and the blind part;

[0059] 3) Server-side blind convolution output

[0060] The convolution result obtained after the server performs IPFE decryption inner product on the encrypted input patch will automatically contain the blind offset.

[0061] ,

[0062] From the formula, we can know:

[0063] Each convolution result is added with the same blind offset ;

[0064] The server can only see the convolution result after being translated as a whole, not the real convolution value.

[0065] 4) Server-side blind pooling output.

[0066] 5) Client-side unblinding to recover the real result.

[0067] Further, the server-side blind pooling output:

[0068] Since the average pooling has linear properties, when the server performs pooling on the blind convolution result, it can obtain ,

[0069] Where:

[0070] p² is the size of the pooling window;

[0071] The blind offset is scaled to after pooling, but still remains recoverable;

[0072] The pooling result returned by the server still does not contain any reversible original convolution information.

[0073] Further, the client unblinds to recover the real result:

[0074] After the client receives the pooling result, it can recover the real pooling output using the previously saved blind offset dtd_tdt. This step is completed locally on the client side, and the server cannot know the real value after unblinding.

[0075] Another object of the present application is to provide a function-encrypted convolutional neural network privacy inference system, which comprises:

[0076] The reasoning module is used for taking advantage of the aggregation blur characteristics of the pooling layer, exposing the inner product result of the convolution layer after the pooling layer, or performing random blinding processing on the inner product result, so as to realize a reasoning process with low calculation overhead and low privacy leakage risk.

[0077] The combination module is used for combining the information blur caused by the pooling and the blind disturbance mechanism, so as to improve the protection ability of the original input on the premise of ensuring the model accuracy.

[0078] Another object of the present application is to provide a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the function-encrypted convolutional neural network privacy reasoning method.

[0079] Another object of the present application is to provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to make the processor execute the steps of the function-encrypted convolutional neural network privacy reasoning method.

[0080] Another object of the present application is to provide an information data processing terminal for implementing the function-encrypted convolutional neural network privacy reasoning system.

[0081] In combination with the above technical solutions and the technical problems solved, the technical solution to be protected by the present application has the following advantages and positive effects:

[0082] The existing privacy protection convolutional neural network (CNN) reasoning technology mainly relies on homomorphic encryption (HE) or secure multi-party computation (MPC), which can protect user input data to some extent, but generally has problems such as high calculation complexity, large communication overhead, and difficulty in adapting to deep CNN structure. For example, the HE scheme needs to complete complex operations such as convolution and activation in the ciphertext domain, which leads to a significant increase in delay; the MPC scheme needs frequent interaction, and the communication cost is unacceptable. In addition, although the existing reasoning method based on inner product function encryption (IPFE) can significantly reduce the overhead, directly using IPFE for the convolution layer will expose the inner product result of the low-dimensional convolution kernel, so that the server can construct a linear equation system to recover the input patch, and there is a serious risk of model inversion. Related research has verified that the leakage of early convolution features will lead to high-precision input reconstruction.

[0083] The present application proposes an IPFE privacy reasoning method that delays early feature exposure and fuses the pooling blur, and the main technical innovations are as follows:

[0084] 1. A pre-aggregation mechanism is proposed to solve the problem of exposing low-dimensional features in convolution-IPFE

[0085] Traditional convolutional layers operate on a very small receptive field (e.g., 3×3, corresponding to a 9-dimensional vector). If the server obtains the inner product of multiple convolutional kernels with the same receptive field, it can use a full-rank matrix to solve for patches and thus recover the input. The pre-aggregation technique proposed in this invention averages multiple convolutional patches belonging to the same pooling window on the client side, ensuring the server only encounters highly ambiguous aggregated features, fundamentally eliminating the possibility of constructing linear equations using convolutional results. This mechanism ensures that: the server never sees fine-grained patches; the aggregated result has significant uncertainty, making model inversion mathematically difficult; and it does not change the accuracy of the original network, with experiments showing only an acceptable loss of about 0.2%.

[0086] 2. A ciphertext blinding mechanism is proposed to ensure that intermediate values ​​of convolutions are never leaked in their exact form. This mechanism adds a random blinding term to the function key, ensuring that all convolution outputs obtained by the server have a random offset. Since pooling is linear, the offset term can be uniformly removed after pooling without affecting the final result. This design has the following technical advantages: it supports maintaining the original CNN structure (without modifying the convolution or pooling positions); the added randomness ensures that no single or multiple convolution outputs are sufficient to recover the input; and the blinding value can be refreshed across rounds, preventing information accumulation even if the server observes multiple times.

[0087] 3. The creative technical effects achieved after solving the above problems

[0088] (1) Significantly improves resistance to model inversion attacks: Experiments show that the method of this invention reduces the cosine similarity of the reconstructed image to about 0.15, which is comparable to strong privacy methods such as HE / MPC, and significantly better than unprotected IPFE convolution (similarity as high as 0.95 or more).

[0089] (2) The computational efficiency is far superior to that of fully homomorphic or fully MPC schemes: Since only the first-layer convolution and pooling are performed in dense state, most of the forward computation is performed in plaintext, which significantly reduces the overall inference latency. Experiments show that IPFE operations can still be completed in a few seconds even at a dimension of 1024, supporting real-time inference.

[0090] (3) Communication overhead is greatly reduced: pre-aggregation reduces the number of patches that need to be encrypted, and the communication cost is significantly better than similar solutions.

[0091] (4) Maintaining almost no decrease in model accuracy: The changes were limited to adjusting the pooling position and linear reversible transformation in the first layer. Experiments showed that the model accuracy decreased by only about 0.2%.

[0092] (5) The solution is scalable to a variety of CNN models and has strong versatility: The paper has proven that it can be directly transferred to various network structures such as ResNet, VGG, and MobileNet, and maintains stable performance on multiple datasets.

[0093] This invention fills a technological gap in the domestic and international industry: While homomorphic encryption (HE) and secure multi-party computation (MPC) have been used to protect deep learning models in existing privacy inference research, a solution that can securely use encryption of inner product functions (IPFE) in the early layers of convolutional neural networks (CNNs) has always been lacking. Convolutional kernels are typically very low-dimensional; once the server obtains multiple convolutional outputs, it can recover the input patch through linear solving, leading to serious privacy leakage risks. Therefore, there is currently no publicly available solution that can securely deploy IPFE convolutional computation without relying on expensive HE / MPC, representing a long-standing technological gap. The "pre-aggregation mechanism" and "ciphertext blinding mechanism" proposed in this invention structurally block the inversion path caused by the exposure of the convolutional inner product for the first time, enabling IPFE to be securely applied to convolutional layers for the first time, achieving efficient execution of ciphertext convolution + plaintext subsequent inference. Experiments further demonstrate that the structure has high accuracy, low overhead, and strong privacy, directly filling the current international gap of lacking a "lightweight privacy inference scheme that can be used for CNN convolutional layers", and providing a new basic route for subsequent related research.

[0094] This invention addresses a long-standing technical challenge that has remained unresolved: For a long time, both academia and industry have sought to achieve efficient privacy-preserving inference on convolutional neural networks (CNNs) without the need for complex HE / MPC methods. However, CNN convolutional operations suffer from several insurmountable bottlenecks: low kernel dimensions make it easy for the server to reverse-engineer input patches using IPFE inner product results; excessively deep convolutional layers result in extremely high HE / MPC inference costs, failing to meet real-time requirements; and existing solutions cannot prevent early feature leakage, leading to a high success rate of model inversion attacks. This invention addresses this challenge through two core technologies—a pre-aggregation mechanism and a ciphertext blinding mechanism—to thoroughly reduce the discernibility of convolutional patches, ensuring the server can never see locally sensitive structures; and to imbue all convolutional results with random offsets, preventing the server from obtaining precise intermediate values ​​and disrupting the inversion equation. These two technologies jointly overcome the core challenge of dense-state inference in convolutional layers, ensuring that early layer outputs cannot reconstruct the original image even when visible to attackers. Related experiments show that the input reconstruction similarity is reduced to below 0.15, achieving security comparable to heavy HE / MPC methods while significantly improving efficiency. Therefore, this invention solves a key problem that has long plagued the industry: how to achieve secure computation and irreversible exposure control of early features in convolutional neural networks while maintaining high efficiency. This problem has long hindered the development of lightweight privacy inference technology, and the realization of this invention is of groundbreaking significance.

[0095] This invention constructs pooled region-level pre-aggregation patches on the client side, compressing the multiple dense convolution inner products that originally needed to be executed on the server side into a single equivalent computation. While ensuring mathematical consistency, it significantly reduces the scale of dense computation, enabling privacy inference to be scalable in high-resolution image and industrial data scenarios, while effectively reducing the risk of fine-grained feature leakage.

[0096] This invention enables pre-aggregation computation to be directly mapped to existing convolutional network inference frameworks by explicitly defining the dimensional relationship between input features, convolutional patches, and pooling windows. This avoids the disconnect between algorithms and engineering implementation, and improves the reproducibility and deployment determinism of privacy inference schemes in real systems.

[0097] This invention moves the complete aggregation of patch-level information to the client side, blocking the server's observation path for individual convolutional position features from an architectural perspective. This ensures that uploaded data naturally meets the principle of minimum necessary information, significantly enhancing privacy compliance and making it suitable for industrial application scenarios with high data sensitivity requirements.

[0098] This invention introduces blinding offset during the function key construction stage, which shifts the overall inference results obtained by the server and suppresses its ability to perform reverse inference through numerical distribution. This improves the privacy and security level of cloud inference without changing the inference process and model accuracy.

[0099] This invention employs a channel-level consistent blinding offset design to eliminate the relative numerical relationships between different spatial locations, thereby preventing the server from recovering structural information through spatial correlation analysis and enhancing the long-term security of the blinding mechanism under real high-structure data distribution.

[0100] This invention utilizes the linear scaling properties of average pooling to ensure that the blinded offset remains in a resolvable form after pooling, allowing the client to recover the true result without loss. This avoids the accuracy degradation problem caused by traditional noise-adding mechanisms and ensures the long-term stable operation of the model.

[0101] This invention clarifies the division of responsibilities between the edge and cloud through a modular system structure, enabling privacy inference capabilities to be integrated into existing inference platforms in the form of functional modules, thereby reducing system transformation costs and improving engineering maintainability.

[0102] This invention limits the server-side to only process the aggregated low-resolution feature representation, compressing the observable information space of the server at the system operation level and strengthening the irreversibility of privacy boundaries.

[0103] This invention reduces intermediate feature resolution, thereby simultaneously decreasing computational load, communication bandwidth, and privacy exposure, meeting the engineering requirements of large-scale cloud inference that prioritize both performance and security.

[0104] This invention completely confines the unblinding process to the client side, forming an end-to-end privacy control closed loop. This prevents the server from having the ability to recover real characteristics in the system design, reducing the dependence on the trustworthiness of the computing party. Attached Figure Description

[0105] Figure 1 This is a flowchart of the privacy inference method for convolutional neural networks with function encryption provided in an embodiment of the present invention.

[0106] Figure 2 This is a block diagram of the privacy inference system for convolutional neural networks with function encryption provided in an embodiment of the present invention.

[0107] Figure 3 This is a system model diagram provided in an embodiment of the present invention.

[0108] Figure 4 This is a graph showing the accuracy performance of Method 1 and Method 2 provided in the embodiments of the present invention on different CNN models and different datasets.

[0109] Figure 5 This is a graph showing the computation time of each stage of the IPFE method provided in this embodiment of the invention.

[0110] Figure 6 This is a diagram illustrating the impact of batch size on the latency and throughput of each batch, as provided in an embodiment of the present invention. Detailed Implementation

[0111] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0112] In existing industrial applications of privacy-preserving convolutional neural networks (CNNs), model inference typically requires exposing the user's original input or fine-grained intermediate features to the computing power provider to complete convolution and feature extraction operations. These intermediate features retain strong structural information in both spatial and channel dimensions, and multiple studies have demonstrated that they can be used to recover key content of the original input through attacks such as inversion, membership inference, or attribute inference, thus causing privacy leaks. Furthermore, directly performing inner product calculations on all convolution positions in dense state is not only computationally complex but also introduces unacceptable communication and computing power overhead under functional encryption or homomorphic encryption frameworks, becoming a major bottleneck for the large-scale deployment of privacy-preserving inference.

[0113] To address the aforementioned technical issues, this method, starting from the structural characteristics of convolutional networks, re-examines the computational dependency between convolution and pooling, and utilizes the linear and aggregation properties of the average pooling operator to equivalently reconstruct the inference process. Specifically, while maintaining the mathematical semantics of the model, the original "convolution first, then pooling" operation sequence is transformed into "first performing region-level pre-aggregation on the input patches, then performing an equivalent convolution." This transformation is based on the linear commutativity of convolutional inner product and summation operations, allowing all convolutional patches within a single pooling unit to be element-wise aggregated on the client side, forming aggregated patches before entering the subsequent dense inner product calculation. Thus, the server only accesses low-resolution feature representations at the pooling scale, eliminating the possibility of observing fine-grained features at individual convolutional locations from the source.

[0114] In terms of workflow, the client first performs element-wise averaging on the input patches within the same pooling region based on the convolution kernel size and pooling window layout to generate corresponding pre-aggregated patches, and completes quantization and function encryption locally. The server only uses the function key derived from the model weights to perform inner product decryption on the encrypted aggregated patches, directly obtaining an output result completely equivalent to the standard "convolution + average pooling". To further prevent the server from inferring the true activation level through numerical distribution, the method introduces a consistent random blinding offset across the entire channel during the function key construction stage, causing a global shift in the inference results visible to all servers. This offset exhibits linear scaling characteristics after average pooling and can be losslessly eliminated locally by the client.

[0115] Through the above design, this method achieves consistency between the inference results and the output of the standard model without introducing additional network structure modifications, while significantly reducing the granularity of intermediate information observable on the server side. Its core advantage lies in moving privacy protection forward to the feature space aggregation stage, combined with the single inner product computation capability of function encryption, achieving an engineering-feasible balance between computational complexity, communication cost, and privacy leakage risk. This provides a practical technical path for privacy protection in industrial applications such as cloud inference and model-as-a-service for convolutional neural networks.

[0116] like Figure 1 As shown, the privacy inference method for convolutional neural networks with function encryption provided in this embodiment of the invention includes the following steps:

[0117] S101 fully utilizes the aggregation and fuzziness characteristics of the pooling layer, and achieves an inference process that balances low computational overhead and low privacy leakage risk by delaying the inner product result of the convolutional layer to be exposed after the pooling layer or by performing random blinding processing on it.

[0118] S102 combines the information ambiguity and blind perturbation mechanism brought about by pooling to improve the protection of the original input while ensuring the accuracy of the model.

[0119] Pooling provided in this embodiment of the invention:

[0120] Preparation phase: Pooling requirements

[0121] Average pooling offers stronger privacy protection than max pooling because it aggregates multiple values ​​into a single output, effectively smoothing fine-grained features and reducing adversaries’ ability to discriminate the original features; however, in the standard ResNet architecture, the ReLU activation function is placed between the first convolutional layer and the pooling layer; this nonlinearity complicates cryptographic computation because previous private inference schemes typically replaced ReLU with a linear proxy (e.g., x^2) or evaluated it through multiple rounds of secure comparisons

[15]

[16] , both of which introduce significant computational costs and accuracy reductions; in order to achieve cryptographic inference while maintaining efficiency, this invention modifies the first stage of ResNet by replacing max pooling with average pooling and moving the activation function after the pooling layer; experiments show that this structural adjustment results in only a small decrease in accuracy (about 0.2%) while significantly improving the robustness of privacy protection in the early layers.

[0122] The aggregation provided in the embodiments of the present invention:

[0123] Privacy Method 1: Pre-aggregation Method

[0124] In an embodiment of this invention, a pre-aggregation mechanism is proposed to aggregate fine-grained features of convolutional layers without significantly reducing model accuracy, thereby preventing the server from accessing the original local pixel structure when performing inner product calculations. The core idea of ​​this mechanism is to average and aggregate multiple convolutional input patches that will be pooled into the same region in the future on the client side, based on the window structure of subsequent pooling operations, to form a blurred patch, and then perform inner product function encryption. In this way, the server can only access the pre-aggregated patch during inference and cannot obtain the original local input pixels, effectively suppressing feature reduction attacks.

[0125] 1) Symbol Definition

[0126] To enable those skilled in the art to understand the present invention, the important symbols involved are defined in this section as follows:

[0127] The input feature map is denoted as Where H is the height, W is the width, and C is the number of channels;

[0128] The convolution kernel (the t-th convolution kernel) is denoted as , where k is the kernel size;

[0129] The local patch extracted at the convolution position (a, b) is denoted as... ,

[0130] The pooling window size is p×p, and the step size is p;

[0131] The set of convolution positions covered by pooling position (i,j) is denoted as ,|·| represents the number of elements in the set.

[0132] 2) Standard convolution + average pooling process

[0133] The convolution result of the t-th channel output by a traditional convolutional layer Represented as:

[0134]

[0135] The symbols ⟨⋅ and ⋅> represent the vectorized inner product operation; after the convolution calculation is completed, the average pooling operation averages the convolution result over the pooling region to obtain...

[0136] ,

[0137] 3) The pre-polymerized patch construction of the present invention

[0138] To avoid the server seeing all unaggregated patches This invention constructs aggregate patches directly on the client side.

[0139] ,

[0140] in This represents the aggregated patch corresponding to the pooling position (i,j), which is obtained by averaging all future convolutional patches in that pooling region element-wise. This step is performed entirely on the client side; the server cannot see the individual patches. ;

[0141] 4) Convolution operations on the server side

[0142] The server received the encrypted Then, the inner product is calculated using the function key.

[0143] ,

[0144] This result is equivalent to:

[0145] ,

[0146] This formula shows that pre-aggregation followed by convolution is equivalent to convolution followed by average pooling (complete mathematical equivalence). Therefore, the pre-aggregation scheme of this invention fully guarantees the consistency of model structure and the correctness of calculation.

[0147] 5) Encryption and Reasoning Process

[0148] After obtaining the aggregate patch Then, the client performs the following steps:

[0149] The patch is vectorized and the IPFE encryption algorithm is invoked to generate ciphertext.

[0150] Send the encrypted message to the server;

[0151] Server-side:

[0152] Use weights Exported function key Perform inner product decryption calculation;

[0153] The decryption result directly yields the pooling convolution output.

[0154] Throughout the entire process, the server never saw... The earliest exposed intermediate output is only the result after pooling, and it effectively blocks the leakage of early fine-grained features.

[0155] The blinding perturbation mechanism provided in this embodiment of the invention:

[0156] Method 2: Add blinding

[0157] The second core technical solution of this invention is a blinding mechanism. Unlike the pre-aggregation of Method 1, the blinding mechanism does not require changing the convolution-pooling structure itself. Instead, it injects a random mask into the IPFE function key, so that the server can never obtain the true inner product result of the convolutional layer during the inference process, thereby effectively avoiding early leakage of convolutional features. The blinding scheme achieves perturbation protection of sensitive features through the method of "key-side blinding - pooling-post-deblinding", maintaining the correctness of calculation while improving privacy.

[0158] 1) Symbol Definition

[0159] To facilitate understanding of the complete process of the blinding mechanism, the relevant symbols are explained as follows:

[0160] The vectorized weights of the convolution kernel (the t-th kernel) are denoted as... ,in Let p be the vector dimension, and p be a large prime number. Represents the Zp domain Dimensional space;

[0161] The secret vector in the IPFE master key is denoted as ,

[0162] The function key generated by the traditional IPFE for the convolution kernel is ,

[0163] The definition of random addition of blind vectors is Each element satisfies the constraints

[0164] The blinding offset for the t-th convolutional kernel is denoted as ,

[0165] This represents the input patch at the convolution position (a, b);

[0166] This represents the output of a regular convolution;

[0167] This indicates the output of a regular pooling operation;

[0168] This represents the server-side inference result with blinding enabled.

[0169] 2) Construction of the blinding function key

[0170] To hide the true inner product result, the client adds a blinding term when generating the function key, ensuring that the convolution result calculated by the server is always shifted; the blinded function key is defined as follows:

[0171] ,

[0172] in The original function key; The blinding offset is a randomized offset generated by the client; this blinding offset is consistent across the entire convolution channel; the server cannot distinguish between the true inner product and the blinded portion.

[0173] 3) Server-side blind convolution output

[0174] After the server performs IPFE decryption inner product on the encrypted input patch, the resulting convolution will automatically include a blinding offset.

[0175] ,

[0176] From this formula, we can see that:

[0177] Each convolution result was given the same blinding offset. ;

[0178] The server can only see the convolution result after it has been completely translated, not the actual convolution value.

[0179] 4) Server-side output with blind pooling.

[0180] 5) Unblind the client and restore the true results.

[0181] The server-side output with blind pooling provided in this embodiment of the invention:

[0182] Because average pooling has a linear property, when the server performs pooling on the results of blind convolutions, it can obtain... ,

[0183] in:

[0184] p² is the pooling window size;

[0185] The blind offset is scaled after pooling. However, it remains recoverable;

[0186] The pooling results returned by the server still do not contain any reversible original convolution information.

[0187] The client-side unblinding and recovery of the true results provided in this embodiment of the invention:

[0188] After receiving the pooling result, the client can recover the actual pooling output using the pre-saved blinding offset d_t. This step is completed locally on the client side, and the server cannot know the true value after unblinding.

[0189] like Figure 2As shown, an embodiment of the present invention provides a function-encrypted convolutional neural network privacy inference system, comprising:

[0190] The inference module is used to fully utilize the aggregation fuzziness characteristics of the pooling layer. By delaying the inner product result of the convolutional layer to be exposed after the pooling layer, or by randomly blinding it, an inference process that balances low computational overhead and low privacy leakage risk is achieved.

[0191] The module combines the information ambiguity and blind perturbation mechanism brought about by pooling to improve the protection of the original input while ensuring model accuracy.

[0192] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the function-encrypted convolutional neural network privacy inference method.

[0193] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the function-encrypted convolutional neural network privacy inference method.

[0194] Another objective of this invention is to provide an information data processing terminal for implementing the convolutional neural network privacy inference system with encrypted function.

[0195] Figure 3 System Model

[0196] Figure 4 The accuracy performance of Method 1 and Method 2 on different CNN models and different datasets.

[0197] Figure 5 The reconstructive performance of Method 1 and Method 2 on different CNN models and different datasets.

[0198] Figure 6 The impact of batch size on latency and throughput for each batch.

[0199] I. Specific application areas or related products of this invention.

[0200] Example: Application of Remote Identity Verification and Anti-Fraud Model in the Financial Industry

[0201] 1. Application Background: In scenarios such as bank account opening, remote loans, facial verification, and document recognition, users need to upload photos of their ID cards or facial images. Uploading in plaintext poses an extremely high risk of data leakage. This invention enables financial institutions to complete high-precision identity verification without seeing the user's original image.

[0202] 2. System Participating Entities

[0203] Client-side: User's mobile app / online banking terminal

[0204] Cloud-based: AI risk control center for banks or payment institutions

[0205] 3. Implementation Process

[0206] Step 1: The client captures the user's face / ID image and performs pre-aggregation.

[0207] Client App Auto-Completion:

[0208] Get image X (such as the front of an ID card or a selfie).

[0209] Patch Pa,b is calculated based on the cloud-based model structure.

[0210] The patches in the same pooling region are averaged to generate an aggregated patch Ai,j.

[0211] Image details such as the edges of the ID card, facial texture, and ID number cannot be reverse-engineered.

[0212] Step 2: The client performs IPFE encryption and uploads the data.

[0213] The client encrypts Ai,j into C(Ai,j), and the uploaded data does not contain any visual information, so the image cannot be recovered even if the data is intercepted.

[0214] Step 3: Perform blinded convolution and deep inference in the cloud.

[0215] Cloud:

[0216] Generate a blinding functional key for W(t): ⟨𝑠,𝑊(𝑡)>+d 𝑡 The inner product of the encrypted patch is applied to obtain a blinded convolutional output. Subsequent network components, such as pooling, ReLU, and residual connections, are then executed to achieve tasks such as facial feature extraction and document region localization. Since convolutional layers are most prone to revealing facial structures, this invention provides complete leakage protection at this stage.

[0217] Step 4: The client completes the deblinding and obtains the verification result.

[0218] The cloud returns the classification results or embedding vectors, and the client removes the blinded terms to obtain:

[0219] Face matching score (e.g., similarity 0.93)

[0220] Document validity assessment (authenticity verification)

[0221] Anti-fraud model output (whether it is an attack sample)

[0222] The entire process is conducted in the cloud without ever touching the user's real face or ID image.

[0223] 4. Summary of Technical Effects

[0224] Banks can offer a risk control service that "completes verification without seeing the image," significantly improving privacy compliance. Because the front-end only performs linear aggregation, the terminal performance overhead is extremely low, making it suitable for all mobile devices such as phones and tablets. It has the same recognition accuracy as plaintext models, meeting bank-level risk control requirements.

[0225] II. Evidence related to the technical effects obtained by the embodiments of the present invention.

[0226] To verify the operational efficiency and engineering feasibility of this invention in practical application scenarios, end-to-end inference performance was experimentally tested under different batch sizes, mainly examining the changes in inference latency and system throughput. As shown in Figure 6, as the batch size increases from 1 to 128, the secure inference framework of this invention exhibits a significant performance improvement. When the batch size is 1, the inference latency per batch is approximately 350 milliseconds; while when the batch size increases to 128, the overall latency decreases to approximately 25 milliseconds, a reduction of over 90%, indicating that this invention can effectively reduce computational overhead while maintaining security. Simultaneously, the system throughput also rapidly increases from approximately 2.86 frames / second under small batch conditions to over 5000 frames / second, achieving an order-of-magnitude performance improvement. This performance improvement is attributed to the batch encryption technology, parallel processing mechanism of IPFE operations, and encrypted inference pipeline on the GPU introduced in this invention during encrypted inference, significantly improving hardware resource utilization. The experimental results of this invention show that it can not only meet the security requirements of convolutional neural network inference under privacy protection, but also achieve low latency and high throughput performance in actual deployment environments, and has good engineering usability and large-scale application value.

[0227] To verify the model accuracy preservation capability of this invention in an encrypted inference environment, the classification accuracy performance of the two schemes of this invention (Method I and Method II) was compared with that of various existing encrypted inference methods on different datasets and various mainstream network structures. The experimental results are as follows: Figure 4As shown in the results, the two schemes of this invention exhibit the smallest accuracy loss across all encrypted baselines. Their accuracy on various networks, including ResNet-18, ResNet-34, VGG-11, and MobileNet, is highly consistent with the plaintext inference results, typically decreasing by only about 0.5%–1.1%, significantly outperforming the controlled encrypted inference methods. The fundamental reason is that this invention only performs encrypted computation on the early convolutional layers of the network, where features are still at a low semantic level, and the impact of encrypted domain computation on the model's expressive power is minimal. Simultaneously, the pre-aggregation mechanism of this invention ensures that convolution and pooling operations maintain mathematical equivalence, thereby maximizing model accuracy. In contrast, CryptoNets suffers a more significant drop in overall accuracy due to the need for polynomial approximation of nonlinear activation functions; while fully encrypted inference schemes such as OPPCNN and OHLC experience more significant accuracy degradation due to deeply accumulated numerical noise. The experimental results above show that the present invention not only has advantages in privacy protection and operating efficiency, but also effectively maintains the discrimination ability of the original neural network, achieves inference performance close to that of the plaintext model, and has good practical value and engineering reliability.

[0228] To further verify the performance of the basic computational unit in the dense-state inference stage of this invention, the runtime of each stage of IPFE (Inner-Product Functional Encryption) under different vector dimensions was tested, including four parts: Setup (system initialization), Encrypt (plaintext vector encryption), KeyDerive (functional key generation), and Decrypt (functional decryption / inner product evaluation). The results are shown in Figure 5. The experimental curves show that when the vector dimension expands from tens to hundreds, the runtime of the Setup and Decrypt stages increases linearly with the dimension, becoming the main source of overall overhead. The Decrypt stage, due to the need to perform exponential operations and discrete logarithmic recovery, shows a more significant increase. The runtime of the Encrypt and KeyDerive stages remains at a low level, accounting for only a small portion of the total computation time even with high-dimensional inputs, indicating good lightweight characteristics. These results demonstrate that the core computational part of this invention still has controllable computational complexity, and its performance growth trend is linearly related to the vector dimension, avoiding exponential computational expansion and meeting the needs of large-scale deployment and real-time inference. Compared with traditional homomorphic encryption schemes, the encryption and key generation process based on IPFE in this invention has a very low burden, and the most time-consuming Decrypt stage can be further optimized through batch processing, parallel acceleration, and other methods. This invention achieves a good balance between security and efficiency, making it suitable for widespread application in cloud inference, edge computing, and resource-constrained scenarios.

[0229] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0230] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A privacy-preserving inference method for convolutional neural networks with function encryption, characterized in that, The method includes the following steps: On the client side, the set of convolution positions corresponding to each pooling position is determined based on the structural parameters of the convolutional layer and its subsequent average pooling layer in the convolutional neural network. For input feature convolution patches belonging to the same pooling position, the pre-aggregated patch is constructed by averaging the elements on the client side. The pre-aggregated patch is vectorized and encrypted using a function before being sent to the server; The server uses the function key derived from the convolution kernel weights to perform inner product decryption on the encrypted pre-aggregation patch, directly obtaining the convolution pooling output result at the corresponding pooling position; The output of the convolution pooling is mathematically consistent with the result obtained by performing convolution operation first and then average pooling operation.

2. The method according to claim 1, characterized in that, The input features are feature maps of dimension H×W×C, the convolution patch is a local feature block of size k×k×C, the average pooling window size is p×p, and the pooling stride is p.

3. The method according to claim 1, characterized in that, The pre-aggregation patch is obtained by averaging p² convolutional patches within the same pooling window according to their corresponding elements, and this pre-aggregation process is completed entirely on the client side.

4. A privacy-preserving inference method for convolutional neural networks with function encryption, characterized in that, The method includes: On the client side, a function key containing a random blinding offset is constructed for each convolution channel, wherein the function key is determined by the convolution kernel weights and the random blinding offset. The server uses the function key containing the blinding offset to perform an inner product decryption operation on the encrypted input to obtain a convolution result containing the blinding offset. The server performs average pooling operation based on the convolution result containing the blinding offset to obtain a pooling result containing the scaled blinding offset. On the client side, based on the pre-saved random blinding offset, the pooling result returned by the server is deblinded to restore the true pooling output result.

5. The method according to claim 4, characterized in that, The random blinding offset remains consistent within the same convolution channel, ensuring that all convolution results obtained by the server undergo the same overall translation.

6. The method according to claim 4, characterized in that, The randomized blind offset is scaled up to the original offset divided by the number of elements in the pooling window after average pooling, and the scaled offset can be completely eliminated by the client.

7. A function-encrypted convolutional neural network privacy inference system, characterized in that, The system includes: The client-side preprocessing module is used to pre-aggregate input feature patches belonging to the same pooling position locally and generate encrypted aggregated patches. The encrypted inference module, deployed on the server side, is used to perform inner product decryption operations on the encrypted aggregate patch based on the function encryption key, and generate intermediate results at the pooling level. The unblinding and recovery module, deployed on the client side, is used to unblind the server inference results containing blinding offsets and restore the true inference output.

8. The system according to claim 7, characterized in that, The client-side preprocessing module aggregates convolutional patches before sending encrypted data, preventing the server from obtaining the original input features corresponding to a single convolutional position throughout the inference process.

9. The system according to claim 7, characterized in that, The dense-state inference module only receives the aggregated feature representation, whose spatial resolution is lower than that of the original convolutional feature resolution.

10. The system according to claim 7, characterized in that, The unblinding recovery module is executed locally on the client side, and the server side does not have the ability to recover the true convolution result or pooling result during the entire inference process.

Images