A privacy-preserving data processing method and prediction system

Abstract

This invention relates to the field of privacy-preserving machine learning, and particularly to a privacy-preserving data processing method and prediction system. The method includes: obtaining an encryption priority value obtained by weighted fusion of at least two complementary encryption priority quantification indices; dividing the original feature vector into a first feature subset and a second feature subset according to the encryption priority value; receiving plaintext data of the first feature subset and homomorphically encrypted data of the second feature subset from a data provider; inputting the plaintext data into a first neural network subnet to extract a first hidden layer representation; inputting the homomorphically encrypted data into a second neural network subnet; fusing the first and second hidden layer representations in the ciphertext domain, and outputting an encryption prediction result. This invention reduces computational overhead while protecting the privacy of sensitive features by quantifying encryption requirements in multiple dimensions and dynamically dividing features, combined with a hybrid neural network of plaintext and homomorphic encryption.

Description

Technical Field

[0001] This invention relates to the field of privacy-preserving machine learning, and in particular to a privacy-preserving data processing method and prediction system. Background Technology

[0002] Against the backdrop of the deep integration of big data and artificial intelligence, data-driven intelligent decision-making systems have been widely applied to various forecasting and evaluation tasks. Taking supply chain finance and other application scenarios as examples, the evaluation process typically requires comprehensive analysis of multi-dimensional data of the evaluated entity, including financial indicators, operational status information, and historical behavior records, some of which constitute privacy-sensitive data requiring protection. Current regulations require data processors to strictly protect the privacy rights of data subjects during collection, transmission, and computation. This poses a substantial challenge to traditional plaintext data processing models, necessitating the development of new technological pathways that can achieve privacy protection while ensuring data utility.

[0003] Privacy-preserving machine learning primarily achieves "data usable but not visible" through four main methods: differential privacy, secure multi-party computation, federated learning, and homomorphic encryption. Differential privacy achieves statistical anonymity by injecting noise into the data or computation results, but the introduction of noise inevitably reduces model accuracy. Secure multi-party computation allows multiple parties to jointly perform function computations without revealing their private inputs, but its communication complexity increases exponentially with the number of participants and the depth of the circuit, making it difficult to scale to large-scale deep learning models. Federated learning performs collaborative training by exchanging model gradients instead of the original data, but existing research shows that attackers can reconstruct some training samples from the shared gradients, still posing a risk of gradient backpropagation. In contrast, homomorphic encryption supports direct arithmetic operations in the ciphertext domain, theoretically achieving a balance between data isolation and computability, and is considered one of the most promising privacy-preserving technologies currently available.

[0004] In recent years, homomorphic encryption has been introduced into predictive analytics tasks for privacy-preserving data. Existing literature documents privacy-preserving credit analysis frameworks based on homomorphic encryption, as well as privacy-preserving architectures that combine functional encryption and homomorphic encryption to support various prediction tasks. However, fully homomorphic encryption schemes still face significant technical bottlenecks: firstly, the computational cost of ciphertext operations is several orders of magnitude higher than that of plaintext operations, making it difficult to meet the latency requirements of real-time or near-real-time prediction; secondly, nonlinear activation functions (such as ReLU and Sigmoid) cannot be directly computed in the ciphertext domain, usually requiring high-order polynomial approximation or interactive protocols, further exacerbating the computational burden. To balance security and computational efficiency, existing technologies have proposed a "partial encryption" strategy: dividing input features into unprotected and protected features, and performing homomorphic encryption only on the latter; or encrypting only a subset of high-risk parameters in the model; or encrypting only communication payloads with high leakage risk in federated learning architectures. However, some existing encryption schemes have the following technical defects: (1) The formulation of encryption strategies depends on predefined static rules and cannot be dynamically adjusted according to the distribution changes of input data or task context; (2) There are often statistical correlations between features. Processing only some features in plaintext may lead to the indirect leakage of privacy information through correlation. Existing schemes have not fully considered this risk; (3) The encryption granularity and strategy lack a linkage mechanism with the importance measurement of downstream prediction tasks, and cannot achieve the optimal allocation of computing resources and privacy protection strength.

[0005] In summary, existing privacy-preserving machine learning techniques suffer from several drawbacks when performing predictive analysis tasks on multidimensional privacy data. These include high computational overhead for fully homomorphic encryption, difficulties in nonlinear operations, and a lack of adaptability in some encryption strategies. Furthermore, existing evaluation methods (such as information entropy and SHAP values) for identifying features requiring encryption suffer from limitations such as noise sensitivity, correlation bias, or computational complexity. Neither purely data-driven nor purely rule-based methods can simultaneously address both regulatory compliance and adaptive distribution evolution. Therefore, there is an urgent need to develop a hybrid encryption framework capable of dynamically quantifying feature protection requirements, adaptively determining encryption strategies, and collaboratively optimizing computational efficiency and prediction accuracy. Summary of the Invention

[0006] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0007] According to a first aspect of the present invention, a privacy-preserving data processing method is provided, comprising:

[0008] The encryption priority value is obtained from the original feature vector provided by the data provider. The original feature vector contains numerical attribute data that needs to be kept confidential. The encryption priority value is obtained by weighted fusion of at least two complementary encryption priority quantification indicators for each feature in the original feature vector. The encryption priority quantification indicators include at least two of the following: a first indicator based on the statistical uncertainty of the data, a second indicator based on the contribution of the feature to the model prediction, and a third indicator based on semantic rule annotation.

[0009] The original feature vector is divided into a first feature subset and a second feature subset according to the encryption priority value, wherein the second feature subset corresponds to features with an encryption priority value higher than a preset condition.

[0010] Receive plaintext data of the first feature subset and homomorphically encrypted data of the second feature subset from the data provider.

[0011] The plaintext data is input into a first neural network subnet to extract a first hidden layer representation, and the homomorphically encrypted data is input into a second neural network subnet to extract a second hidden layer representation, wherein the second neural network subnet uses a polynomial function as an activation function to perform nonlinear computation in the ciphertext domain.

[0012] The first hidden layer representation and the second hidden layer representation are fused in the ciphertext domain to output the encrypted prediction result.

[0013] According to a second aspect of the present invention, a privacy-preserving prediction system is provided, comprising:

[0014] An encryption priority evaluation module is configured to receive an encryption priority value from an original feature vector provided by a data provider, and to divide the original feature vector into a plaintext feature subset and a ciphertext feature subset based on the encryption priority value. The original feature vector contains numerical attribute data that needs to be kept confidential. The encryption priority value is obtained by weighted fusion of at least two complementary encryption priority quantification metrics for each feature in the original feature vector. These quantification metrics include at least two of the following: a first metric based on statistical uncertainty of the data, a second metric based on the feature's contribution to model prediction, and a third metric based on semantic rule annotation. An encryption module is configured to perform homomorphic encryption on the features in the ciphertext feature subset to generate ciphertext feature data.

[0015] Hybrid reasoning module, including:

[0016] The plaintext subnet is configured to process features in the plaintext feature subset and output a first intermediate representation.

[0017] The ciphertext subnet is configured to process the ciphertext feature data and uses a polynomial function as the activation function to perform nonlinear calculations in the ciphertext domain, outputting a second intermediate representation.

[0018] The fusion layer is configured to fuse the first intermediate representation and the second intermediate representation over the ciphertext domain and output the encrypted prediction result.

[0019] The present invention has at least the following beneficial effects:

[0020] This invention achieves prediction accuracy close to that of a plaintext model by using dynamic feature partitioning and a plaintext-ciphertext hybrid neural network, while ensuring full protection of predefined encrypted features. At the same time, it can significantly reduce inference latency compared to fully homomorphic encryption schemes, and the number of encrypted features is linearly controllable with computational overhead, thus balancing data privacy protection and computational efficiency.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a privacy-protecting data processing method provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0027] Example 1

[0028] Accurate identification and classification of sensitive features are crucial prerequisites for achieving differentiated privacy protection. Regarding feature importance assessment, information entropy can quantify the contribution of a single feature to uncertainty, but it is susceptible to noise interference in high-dimensional data environments and cannot distinguish between redundant and synergistic relationships between features. SHAP values, based on Shapley values ​​in game theory, calculate the marginal contribution of features, possessing consistency and local interpretability, but the estimated values ​​are biased when features are highly correlated, and the computational complexity increases exponentially with feature dimensionality. Analysis shows that integrating multi-source assessment metrics can improve the robustness of sensitivity measurement. Furthermore, domain knowledge (such as highly sensitive data types explicitly listed in regulations) plays an irreplaceable role in identifying fields that must be protected. However, purely data-driven assessment methods easily overlook mandatory protection requirements at the business compliance level, while purely domain knowledge rule-based methods suffer from incompleteness (failing to cover potential sensitive patterns implicit in the data) and staticity (unable to adapt to business scenarios or data distribution evolution). How to integrate statistical characteristics, model interpretability metrics, and domain knowledge to achieve dynamic and accurate identification and classification of sensitive features remains an unsolved technical challenge.

[0029] To this end, this embodiment provides a privacy-preserving data processing method that integrates statistical characteristics, model interpretability indicators, and domain knowledge to achieve dynamic identification and classification of sensitive features, and on this basis, uses a plaintext-ciphertext hybrid neural network for efficient inference.

[0030] This embodiment takes the risk prediction scenario of an aviation and tourism supply chain finance platform as an example, involving data providers and data processors.

[0031] The data processor refers to the computing node that holds the original feature data and requests the prediction service. This node is configured with a Trusted Execution Environment (TEE) to generate homomorphic encryption key pairs (public key pk and private key sk), perform homomorphic encryption on the specified features to be encrypted, and decrypt the encrypted prediction results returned by the data processor. The private key never leaves the TEE. For example, the data provider could be a company applying for a risk assessment.

[0032] The data processing unit refers to the computing node that provides predictive models and inference services. This node is equipped with an encryption priority evaluation module, a hybrid neural network inference module, and a key management interface. The data processing unit cannot obtain the plaintext data containing the features to be encrypted; it can only process the plaintext features and the homomorphically encrypted ciphertext features. For example, the data processing unit could be a financial platform. The architecture of the hybrid neural network inference module is as follows:

[0033] Plaintext Subnet (First Neural Network Subnet): Receives data from a subset of plaintext features, uses standard fully connected layers and non-linear activation functions (ReLU or Sigmoid), and extracts the first hidden layer representation.

[0034] The ciphertext subnet (second neural network subnet) receives data from the homomorphically encrypted ciphertext feature subset. Structurally similar to the plaintext subnet, its fully connected layer activation functions employ polynomial approximations (e.g., quadratic polynomial approximations ReLU or Sigmoid), allowing nonlinear operations to be performed directly in the ciphertext domain to extract the second hidden layer representation. The advantage of using polynomial approximations is that they transform the nonlinear activation function into a sequence of addition and multiplication operations supported by homomorphic encryption, avoiding the excessively high multiplication depth caused by directly calculating exponential, logarithmic, or other transcendental functions or performing high-order exponentiation operations in the ciphertext domain. Furthermore, the nonlinear transformation can be completed without interaction with the data provider, thus significantly reducing the multiplication depth and communication overhead of ciphertext computation while maintaining the semantic security of homomorphic encryption.

[0035] Fusion Layer: Connects the outputs of the plaintext subnet and the ciphertext subnet. First, the output of the plaintext subnet is mapped to the output space through a linear transformation (i.e., multiplied by a first linear transformation weight); the output of the ciphertext subnet is mapped to the same space through another linear transformation (i.e., multiplied by a second linear transformation weight); then the two mapping results are added together with a bias; finally, the encrypted prediction result is obtained through the output activation function (a polynomial approximation of the sigmoid). The above hybrid neural network inference module is obtained using a simulation training strategy. Specifically, the first neural network subnet and the second neural network subnet are jointly trained in the following way:

[0036] (1) Obtain the training sample set, wherein each training sample in the training sample set contains a historical feature vector and a corresponding label.

[0037] Historical feature vectors refer to the input features of each sample in the training dataset, with each input feature having a dimension of d. These features originate from accumulated plaintext historical data, such as a company's financial indicators, operating status, and credit records. They have the same feature dimensions and semantics as the raw feature vectors submitted in real time by the data provider during the inference phase, but the data instances are different. Labels represent the true results corresponding to each historical feature vector, such as default status (0 / 1), risk level (high / medium / low), or continuous risk scores. Labels are used for supervised learning, serving as the target for model optimization.

[0038] (2) According to the preset sensitive feature division rules, the historical feature vector of each training sample is divided into simulated plaintext features and simulated ciphertext features.

[0039] Based on the same sensitive feature segmentation rules as in the inference phase (the specific determination method of these rules is described in step S200 below, which describes the feature segmentation method based on encryption priority and threshold), the historical feature vector of each training sample is divided into simulated plaintext features and simulated ciphertext features.

[0040] (3) Construct a hybrid neural network comprising the first neural network subnet, the second neural network subnet and the fusion layer, wherein the second neural network subnet does not use homomorphic encryption during the training phase, but adopts the same polynomial approximation activation function as the inference phase.

[0041] (4) Using the simulated plaintext features and the simulated ciphertext features as inputs and the label as a supervision signal, the network parameters of the first neural network subnet, the second neural network subnet and the fusion layer are jointly updated through backpropagation.

[0042] The entire hybrid neural network (including the first neural network subnetwork, the second neural network subnetwork, and the fusion layer) employs end-to-end supervised learning. The loss function is selected based on the type of prediction task, such as binary classification cross-entropy or mean squared error.

[0043] During training, a batch of training samples is input each time. Each sample consists of simulated plaintext features, simulated ciphertext features (in plaintext form for simulation), and the corresponding ground truth label. Forward propagation sequentially calculates the plaintext subnet output, the ciphertext subnet output (whose activation function is a polynomial approximation), and the fusion layer output. The fusion layer calculation process is as follows: the plaintext subnet output is multiplied by the first linear transformation weights, the ciphertext subnet output is multiplied by the second linear transformation weights, the two are added together, a bias is added, and finally, the predicted value is obtained by applying the polynomial approximation output activation function.

[0044] After calculating the loss, the chain rule is used to backpropagate layer by layer from the output layer to calculate the gradient of the loss with respect to each trainable parameter. These parameters include: the weights and biases of the fully connected layers in the plaintext subnet, the weights and biases of the fully connected layers in the ciphertext subnet, and the weights and biases of the first linear transformation, second linear transformation, and the fusion layer. Since the input to the ciphertext subnet is plaintext and the activation function is fixed in polynomial form during the training phase, the gradient can be calculated normally without the need for homomorphic encryption support.

[0045] Then, the parameters are updated based on the gradient using an optimizer (such as Adam or SGD). The above process (forward propagation, loss calculation, backpropagation, parameter update) is repeated until the loss function converges or the preset number of iterations is reached.

[0046] After training, all parameters are saved and deployed to the inference service for online hybrid neural network inference. Furthermore, since the second subnet in both the training and inference phases uses the same polynomial approximation form, the numerical characteristics simulated by the ciphertext subnet during training are consistent with those in the actual ciphertext computation during inference, thus avoiding performance degradation caused by differences in activation functions.

[0047] Regarding parameter deployment, the network parameters of the encrypted subnet are stored only by the data processor, and the data provider cannot access these parameters; while the parameters of the plaintext subnet are visible to the data provider. This differentiated exposure design achieves partial hiding of the model architecture, effectively preventing attackers from carrying out model extraction attacks by extracting complete model parameters.

[0048] Furthermore, the ciphertext subnet employs a polynomial form independent of the homomorphic encryption key. Specifically, all operations (linear transformations, polynomial activations) in the ciphertext subnet do not depend on any specific key. Therefore, for ciphertext data encrypted with their respective public keys by different data providers, the data processor can use the same set of network parameters for processing, without needing to retrain the model or convert the ciphertext format for each data provider. This feature significantly improves deployment efficiency in multi-user scenarios.

[0049] The aforementioned hybrid neural network is pre-trained offline using historical plaintext data (training dataset). The training dataset is also used to calculate encryption priority quantification metrics such as information entropy and SHAP value, and to determine the optimal fusion weights through a weight optimization process based on TopK coverage constraints.

[0050] During the inference phase, the data provider, based on the feature segmentation results returned by the encryption priority evaluation module, performs CKKS homomorphic encryption only on highly sensitive features, submitting both plaintext and ciphertext features to the data processor. The data processor performs hybrid neural network inference, with all ciphertext operations completed under the CKKS scheme, ultimately returning the encrypted result. The data provider decrypts the result in its local trusted execution environment to obtain the predicted value.

[0051] like Figure 1 As shown, the method includes the following steps:

[0052] S100, obtain the encryption priority value of the original feature vector provided by the data provider.

[0053] The original feature vector is extracted by the data provider from local data sources (such as business systems, financial statements, credit records, etc.), covering multiple dimensions such as financial indicators (e.g., debt-to-equity ratio, cash flow), operating status (e.g., company age, industry category), and credit records (e.g., historical delinquency count, credit score, etc.). The original feature vector contains numerical attribute data that needs to be kept confidential; for example, some information in the aforementioned financial indicators, operating status, and credit records constitutes privacy data that needs protection.

[0054] Specifically, in this embodiment, the data dimensions included in the original feature vector include, but are not limited to: quantitative indicators reflecting the entity's operational and financial status, statistical indicators reflecting the entity's historical behavioral records, and attribute information used for identification and classification. As an example, the above data may include:

[0055] Financial indicators such as debt-to-equity ratio, current ratio, operating revenue, net profit, and cash flow;

[0056] Operational metrics: such as years since establishment, number of employees, industry category, transaction frequency, etc.

[0057] Historical behavioral records: such as number of overdue payments, credit score, loan records, guarantee information, etc.;

[0058] Identity and compliance identifiers: such as legal person information, unified social credit code, registered address, etc. (some of which are predefined encrypted features).

[0059] All data in the above-mentioned data that involves non-public personal or corporate information is considered to be data that requires privacy protection.

[0060] After the data provider performs necessary data cleaning, missing value imputation, and normalization on the original features, a d-dimensional feature vector is formed and sent in plaintext to the encrypted priority evaluation module of the data processing party. In actual deployment, the specific features included can be predetermined based on the business scenario (such as risk prediction in the aviation and tourism supply chain finance) and available data sources.

[0061] The encryption priority value is obtained by weighted fusion of at least two complementary encryption priority quantification metrics for each feature in the original feature vector. The encryption priority quantification metrics include at least two of the following three categories: a first metric based on statistical uncertainty of the data, a second metric based on the contribution of features to model prediction, and a third metric based on semantic rule annotation. In this embodiment, to achieve the best evaluation effect, all three types of metrics are used simultaneously.

[0062] The first indicator: information entropy.

[0063] Information entropy measures the dispersion or uncertainty of feature values ​​and is independent of the specific prediction model. The distribution of each feature's values ​​is statistically analyzed using the training dataset.

[0064] For the i-th feature (where i ranges from 1 to d) in the original feature vector, let M be the total number of features. i M takes different values ​​(discrete values ​​or bin intervals). i The proportion of the j-th value among the different possible values ​​is p. ij Then the original entropy value .

[0065] To eliminate the dimensional influence caused by the difference in the number of values ​​of different features, the entropy values ​​of all features are normalized to the interval [0,1], thus obtaining the first index score S of feature i. entropy，i :

[0066] If M i If S ≥ 2, then S entropy，i =H i / log2M i If M i =1 (i.e., the characteristic is a constant), then S entropy，i =0.

[0067] The score of this first indicator can then be used to calculate the encryption priority value in subsequent weighted fusion calculations.

[0068] The second metric: SHAP value. Based on the Shapley value in cooperative game theory, the SHAP value fairly allocates the marginal contribution of each feature to the model's predictions. For a prediction model f and a feature set F, the SHAP value of feature i is defined as:

[0069] .

[0070] Where S is the feature subset excluding feature i, and f(S) is the model output using only the feature subset S. The SHAP values ​​of all samples in the training set are calculated, and the average of the absolute values ​​is taken as the feature importance score. Where r is the index (serial number) of a sample in the training dataset, and its value ranges from r = 1, 2, ..., N, where N is the total number of training samples. (r) This represents the r-th sample in the training dataset, i.e., a complete feature vector.

[0071] The second metric score reveals the criticality of a feature in the prediction task. Features with high scores are often the core basis for model decisions, and their disclosure may lead to the exposure of model logic or trade secrets. Similarly, normalizing the SHAP score to the [0,1] interval yields the second metric score S. shap .

[0072] The third indicator: Domain knowledge annotation. Both information entropy and SHAP value are data-driven indicators with inherent limitations: information entropy cannot identify the sensitivity of certain fixed values ​​in specific business scenarios; SHAP value depends on the specific model structure, and the importance ranking may change when the model changes; more importantly, purely data-driven methods may ignore industry regulatory requirements, leading to compliance risks.

[0073] To this end, this embodiment introduces a predefined encrypted feature dictionary, denoted as D, built based on relevant regulations, industry standards, and domain expert experience. For example, in a financial risk control scenario, this dictionary can be constructed based on financial industry compliance requirements and business consensus. This dictionary clearly specifies which fields must be encrypted due to their business nature and are not affected by automated scoring. Specifically, for each feature i in the original feature vector, a binary sensitive identifier is assigned based on whether it belongs to dictionary D: if feature i belongs to D, its third indicator score S is... domain，i Set to 1, indicating that the feature is a predefined encryption feature; if feature i does not belong to D, then its third indicator score S is set to 1. domain，i Setting it to 0 indicates that the feature is a non-predefined encryption feature.

[0074] The construction of the predefined cryptographic feature dictionary follows these principles: mandatory regulatory requirements, industry standard requirements, and annotations reached through expert consensus methods (such as the Delphi method). This mechanism ensures both compliance and accuracy.

[0075] For each feature i, calculate the encryption priority value S for that feature. final :

[0076] S final，i =α·S entropy，i +β·S shap，i +γ·S domain，i .

[0077] Where α, β, γ are weighting coefficients, satisfying α + β + γ = 1 and α, β, γ ≥ 0.

[0078] The aforementioned weighting coefficients directly determine the rationality of the encryption priority value.

[0079] To balance the adaptability of data-driven approaches with the mandatory requirements of domain knowledge, this embodiment designs a weight tuning mechanism based on TopK coverage constraints. The core idea of ​​this mechanism is to search the weight space such that the set of the top features, arranged in descending order of encryption priority, must completely contain all predefined encryption features indicated by the third metric (i.e., all features in the dictionary); then, from the weight combinations that satisfy this coverage constraint, the set of weights that optimizes the performance of the hybrid neural network prediction model composed of the first and second neural network subnets is selected. K is a preset positive integer. In this embodiment, the value of K is usually not less than the total number of predefined encryption features. For example, when there are 8 predefined encryption features, K can be set to 8.

[0080] The specific optimization steps are as follows:

[0081] Define a set of candidate weights: In a multidimensional grid search space or random sampling space, generate multiple sets of candidate weights (α, β, γ), each of which satisfies the constraint that they are non-negative and their sum is 1.

[0082] Ranking and Coverage Verification: For each candidate weight group, calculate the encryption priority value of all features and sort them from highest to lowest score. Extract the top K features to form a TopK set. set Determine TopK set Does it fully include all predefined cryptographic features in the domain dictionary? If it includes all predefined cryptographic features in the domain dictionary, then retain the weight set; otherwise, discard it.

[0083] Model performance optimization: For each set of candidate weights that passes the coverage verification, the hybrid neural network prediction model is trained or evaluated using the feature partitions calculated from them (i.e., partitioning plaintext and ciphertext feature subsets according to encryption priority values), and model performance metrics (e.g., recall, F1 score, etc.) are recorded. From all the retained candidate weights, the set that optimizes model performance is selected as the final adopted weight coefficients.

[0084] For the current input original feature vector, the encryption priority values ​​corresponding to each feature are combined into a vector according to the feature order, thus obtaining the encryption priority value vector of the original feature vector. This vector is used for subsequent feature segmentation (S200) and optional anomaly detection steps. The higher the encryption priority value of each feature, the more necessary it is to be encrypted and protected.

[0085] Through the above S100 steps, this embodiment can dynamically and accurately identify sensitive features, and lay the foundation for subsequent differentiated encryption and hybrid neural network inference.

[0086] S200, the original feature vector is divided into a first feature subset and a second feature subset according to the encryption priority value, wherein the second feature subset corresponds to features with encryption priority values ​​higher than preset conditions.

[0087] This step uses the encryption priority value obtained from S100 to perform feature partitioning. A preset threshold τ is set, and features with encryption priority values ​​greater than or equal to τ are assigned to the second feature subset, i.e., the ciphertext feature subset, while the remaining features are assigned to the first feature subset, i.e., the plaintext feature subset.

[0088] The threshold τ can be determined using one of the following strategies:

[0089] Based on the distribution of representation values: τ is set between the lowest representation value of the predefined encryption feature and the highest representation value of the non-predefined feature;

[0090] Based on security requirements: adopt a lower τ value to incorporate more potentially sensitive features (security first);

[0091] Based on TopK boundary: Set τ as the encryption priority value of the Kth feature.

[0092] This embodiment recommends setting τ to a value slightly lower than the minimum representation value of the predefined encryption features (e.g., 0.5) to ensure that all predefined encryption features are included in the ciphertext feature subset, while leaving room for other potential encryption features identified by automated methods, thus achieving the dual goals of compliance assurance and intelligent expansion.

[0093] To reduce the risk that features in the first feature subset may indirectly leak features in the second feature subset through statistical correlation, the above partitioning results must also satisfy the following iterative constraints:

[0094] (1) Based on the current first feature subset and second feature subset, calculate the correlation coefficient between each feature in the first feature subset and each feature in the second feature subset. The initial values ​​of the current first feature subset and second feature subset are the first feature subset and second feature subset without any movement operation (i.e., the result obtained only based on the encryption priority value and the threshold τ). In this embodiment, the Pearson correlation coefficient is used. In practical applications, the correlation coefficient is usually pre-calculated based on the training dataset (as global statistical information of the features) rather than recalculated for each real-time request, thereby reducing online computing overhead while ensuring accuracy.

[0095] (2) If there exists a first candidate feature belonging to the current first feature subset and a second candidate feature belonging to the current second feature subset, such that the absolute value of the correlation coefficient between the first candidate feature and the second candidate feature is greater than the preset correlation coefficient threshold, then the first candidate feature is moved from the current first feature subset to the current second feature subset.

[0096] A preset correlation coefficient threshold is used to determine whether the correlation between two features constitutes a risk of indirect leakage. This threshold can be determined using one of the following strategies:

[0097] Empirical values: These are set based on domain knowledge or industry best practices, for example, by setting a threshold of 0.5 or 0.6. Generally, an absolute value of the correlation coefficient exceeding 0.5 is considered a moderate to high correlation, and exceeding 0.7 is considered a strong correlation.

[0098] Based on statistical distribution: Calculate the absolute value of the correlation coefficient of all cross-feature pairs on the training dataset, and take a high quantile (e.g., the 75th or 90th quantile) as the threshold so that only feature pairs with significant correlation will trigger the move operation.

[0099] Security strategy: Set a strict threshold (e.g., 0.3) based on the expected level of privacy protection, and include more features with weaker correlations in the ciphertext subset to minimize the risk of leakage.

[0100] In this embodiment, it is recommended to use a combination of experience and business requirements: first, set an initial threshold (e.g., 0.5) based on expert knowledge, and then dynamically adjust it according to the actual leakage risk during security assessment or testing.

[0101] (3) Repeat the move operation, i.e. repeat steps (1) and (2) until the absolute value of the correlation coefficient between any feature in the first feature subset and any feature in the second feature subset is not greater than the preset correlation coefficient threshold.

[0102] Specifically, after each feature movement operation is completed, all cross-subset correlation coefficients are recalculated based on the updated first and second feature subsets, and it is checked again whether there are feature pairs that exceed the threshold. If there are, the movement continues until the absolute value of the correlation coefficient between any feature in the current first feature subset and any feature in the current second feature subset is not greater than the preset correlation coefficient threshold.

[0103] If, after the aforementioned iterative movement operations, there still exists a cross-subset feature pair whose absolute correlation coefficient exceeds the preset threshold (i.e., all high correlations cannot be resolved through feature transfer), then random noise is applied to the features in the first feature subset that cause high correlation (i.e., those features located in the first feature subset whose absolute correlation coefficient with a feature in the second feature subset exceeds the preset correlation coefficient threshold) at the input of the plaintext subnet, or differential privacy-based perturbation processing is performed to further cut off the statistical channel for inferring ciphertext features from plaintext features. Each time the original feature vector is received, the data processor dynamically performs the aforementioned partitioning (including iterative correlation transfer) to generate the first and second feature subsets corresponding to the current request. Subsequently, the data processor returns the indication information of the second feature subset (e.g., a feature index list or mask vector) to the data provider in plaintext form. Based on this, the data provider separates the features corresponding to the first feature subset (plaintext features) and the features corresponding to the second feature subset (features to be encrypted) from the original feature vector, and performs homomorphic encryption on the features in the second feature subset using a public key. Finally, the data provider sends the plaintext features of the first feature subset and the ciphertext features of the second feature subset to the data processor for subsequent hybrid neural network inference.

[0104] S300, receive plaintext data of the first feature subset and homomorphically encrypted data of the second feature subset from the data provider.

[0105] The data processor receives plaintext data of the first feature subset and homomorphically encrypted data of the second feature subset from the data provider through a secure communication channel, such as an encrypted channel established based on the TLS / SSL protocol.

[0106] S400, the plaintext data is input into the first neural network subnet to extract the first hidden layer representation, and the homomorphically encrypted data is input into the second neural network subnet to extract the second hidden layer representation.

[0107] Upon receiving both plaintext and homomorphically encrypted data, the data processor performs the following inference operations:

[0108] The plaintext data of the first feature subset is input into the first neural network subnet (plaintext subnet), and the first hidden layer representation is extracted through forward propagation;

[0109] The homomorphic encrypted data of the second feature subset is input into the second neural network subnet (ciphertext subnet). This subnet uses a multinomial approximation activation function to directly perform nonlinear calculations in the ciphertext domain and extract the second hidden layer representation.

[0110] The first neural network subnet and the second neural network subnet employ asynchronous computation: the plaintext subnet begins forward propagation immediately after receiving plaintext data, while the ciphertext subnet waits for homomorphically encrypted data to arrive before starting computation; the two outputs are synchronized through a buffer and then sent to the fusion layer to reduce waiting delays caused by network transmission and encryption / decryption.

[0111] The first neural network subnet and the second neural network subnet employ asynchronous computation: the first neural network subnet starts forward propagation immediately after receiving plaintext data, while the second neural network subnet waits for homomorphic encrypted data to arrive before starting computation; the two outputs are synchronized through a buffer and then sent to the fusion layer to reduce waiting delays caused by network transmission and encryption / decryption.

[0112] S500, the first hidden layer representation and the second hidden layer representation are fused in the ciphertext domain, and the encryption prediction result is output.

[0113] After obtaining the first hidden layer representation and the second hidden layer representation, the data processing party performs the following fusion operation in the ciphertext domain: S501, the first hidden layer representation is encoded and homomorphically encrypted to obtain the first hidden layer representation in ciphertext form.

[0114] S502, the second hidden layer representation is multiplied with the first linear transformation weight in the ciphertext domain to obtain the first product.

[0115] S503, the first hidden layer representation of the ciphertext form is multiplied with the second linear transformation weight in the ciphertext domain to obtain the second product.

[0116] S504, the first product, the second product, and the biased ciphertext domain are added together to obtain the fused ciphertext hidden layer representation.

[0117] S505, apply a polynomial approximation of the output layer activation function to the fused ciphertext hidden layer representation to generate the encryption prediction result.

[0118] The data processor takes the fused ciphertext hidden layer representation as input to the output layer activation function, which is approximated by a polynomial, and calculates the encrypted prediction result in the ciphertext domain. Taking a binary classification task as an example, a polynomial approximation of the sigmoid function (e.g., 0.5 + 0.215x - 0.008x) can be used. 2 The algorithm performs calculations and outputs a ciphertext scalar. All ciphertext operations are performed using a homomorphic encryption scheme (such as CKKS), without requiring the decryption of any intermediate results.

[0119] The plaintext value obtained after decrypting the ciphertext scalar is a probability value between 0 and 1, representing the confidence level of the entity corresponding to the data provider as belonging to the positive class: the higher the probability value, the more likely the entity is to be predicted as positive; conversely, the lower the probability value, the more likely it is to be predicted as negative. In binary classification tasks, the positive class is the target category that the model wants to predict, which is determined by the label definition of the training data. As an example, in the credit status prediction scenario, the positive class can be "default," and the corresponding probability value can be interpreted as the default probability; in the customer status change prediction scenario, the positive class can be "status change," and the corresponding probability value can be interpreted as the status change probability. This embodiment does not limit the specific application field. If the actual task focuses on other categories (such as "normal"), this category can be defined as the positive class, and the classification boundary between the positive class and the negative class (such as "no default" or "no status change") is also determined by the label definition of the training data. Optionally, to facilitate the data provider in verifying the correctness of the ciphertext calculation process, the data processor can attach a hash commitment of the intermediate results or a hash commitment of part of the input data when outputting the encrypted prediction result.

[0120] The data processor returns the encrypted prediction result along with optional verification information to the data provider. The data provider decrypts the prediction using a locally stored private key to obtain the plaintext prediction value for subsequent decision-making. When verification information is attached, the data provider can verify whether the received result matches the commitment by recalculating a portion of the plaintext path (e.g., reproducing the forward propagation of the plaintext subnet) or using a zero-knowledge proof protocol, thereby detecting whether errors or tampering occurred during the encrypted calculation process.

[0121] Example 2

[0122] This embodiment conducts security analysis under a semi-honest adversary model, where each participant strictly adheres to the protocol but may attempt to infer the other party's private data from information observed during protocol execution. The data processing party, as a potential adversary, can observe the following information: plaintext data of the first feature subset, homomorphically encrypted data of the second feature subset, and model parameters of the hybrid neural network (where the ciphertext subnet parameters are not visible). The data processing party cannot obtain the plaintext of the second feature subset features, the data provider's private key, or the decrypted prediction results. External adversaries may eavesdrop on communication channels or attempt to attack the model through query interfaces.

[0123] I. Safety Assumptions

[0124] This embodiment is based on the following security assumptions:

[0125] RLWE difficulty assumption: For the safety parameter λ, define the ring R q =Z q [X] / (X n +1), where Z q Let Z be an integer ring modulo q.q [X] represents all elements of the form a0+a1X+a2X. 2 +…+a n-1 X n-1 The set of polynomials (each coefficient comes from Z) q ), X n +1 is an nth-degree polynomial, where n is usually a power of 2 (e.g., 1024, 2048, 4096). Let χ be the error distribution defined on the ring, which can be a narrow Gaussian distribution or other discrete bounded distributions, used to generate small-coefficient polynomials whose absolute coefficients are much smaller than q. The width of χ affects security and correctness. Let a←Rq denote the error distribution from the ring R. q In the uniform random selection of element a, s←χ represents sampling secret s from error distribution χ, e←χ represents sampling noise e from error distribution χ, u←R q Indicates from ring R q A uniformly randomly selected element u, a·s+e represents the ring R. q The multiplication operation (a multiplied by s) is then added to e. The RLWE hypothesis asserts that it is computationally difficult to distinguish between the distribution {(a, a·s+e)} and the uniform distribution {(a, u)}.

[0126] CKKS Semantic Security: Under the RLWE assumption, the CKKS homomorphic encryption scheme satisfies IND-CPA security, meaning that the advantage of an adversary in polynomial-time with arbitrary probability to distinguish between two ciphertexts is negligible.

[0127] Communication security: All communication channels are protected by the TLS 1.3 protocol to meet confidentiality and integrity requirements.

[0128] TEE isolation: The Trusted Execution Environment (TEE) of the data provider ensures that the private key cannot be accessed from the outside.

[0129] II. Anomaly Detection Before Reasoning

[0130] To further enhance system security, this embodiment includes an optional anomaly detection step before inference. This step, independent of the core prediction process, is used to identify potential poisoning samples or anomalous inputs.

[0131] First, the mean vector and covariance matrix of each feature encryption priority value are calculated based on the training dataset.

[0132] During the offline training phase, for each training sample in the training dataset, its encryption priority value vector is calculated according to the method described in S100, resulting in a set of encryption priority value vectors for all training samples. The mean vector and covariance matrix of this set are then calculated.

[0133] For each training sample, calculate the Mahalanobis distance of its encryption priority value vector relative to the mean vector and covariance matrix. Sort all training samples' Mahalanobis distances in ascending order, and take the Mahalanobis distance value corresponding to the 95th quantile as a preset threshold. This threshold is used consistently in subsequent inference stages.

[0134] Next, for the current input feature vector, i.e., the original feature vector obtained by S100, the Mahalanobis distance of the current input feature vector's encryption priority value vector relative to the mean vector and covariance matrix is ​​calculated. Mahalanobis distance is a distance metric that considers the correlation between features and is used to measure the degree of deviation between the current sample vector and the training set distribution.

[0135] If the calculated Mahalanobis distance exceeds a preset threshold, the current input feature vector is determined to be an anomalous sample, and the prediction process is immediately terminated without returning any prediction results. Otherwise, the subsequent S200 feature segmentation step continues.

[0136] This anomaly detection mechanism can effectively intercept inputs whose distribution deviates significantly from the training dataset, reducing the impact of poisoning attacks or anomalous data on the prediction service.

[0137] III. Poisoning Attack Defense

[0138] Poisoning attacks aim to impair model performance by injecting malicious samples into training data. This embodiment employs the following mechanism for defense:

[0139] Encryption Isolation: During the inference phase, features in the second feature subset are transmitted and processed in ciphertext. When an adversary constructs a poisoning sample, although they can precisely control the plaintext features (the first feature subset), they cannot observe the true distribution of the features in the second feature subset, nor can they obtain the gradient information of the model on the ciphertext features. Therefore, they can only make blind guesses about the features in the second feature subset and cannot construct targeted poisoning samples.

[0140] Anomaly detection: The aforementioned anomaly detection steps based on Mahalanobis distance can identify inputs with abnormal distributions, further reducing the impact of poisoned samples.

[0141] IV. Member-based attack and defense inference

[0142] Membership inference attacks aim to determine whether a specific data point belongs to the model's training set. This embodiment implements defense through the following mechanism:

[0143] Output Encryption: Traditional membership inference attacks rely on the attacker's ability to obtain the confidence distribution of the target model's output. In this embodiment, the model output is returned to the data provider in ciphertext form. The data processor (as a potential attacker) cannot obtain the plaintext prediction results, confidence distribution, or loss value, and therefore cannot perform membership inference.

[0144] Sensitive Feature Encryption: Shadow model attacks require constructing a shadow dataset with a distribution similar to the target model. In this embodiment, the adversary cannot obtain the plaintext of the second subset features, causing the constructed shadow dataset to deviate from the true distribution, thus preventing the training of an effective member inference attack model.

[0145] V. Model Inversion Attack and Defense

[0146] Model inversion attacks aim to reconstruct features of training data from the model output. This embodiment implements defense through the following mechanism:

[0147] Key isolation: The second subset of features always exists in ciphertext form at the data processor's location, while the private key is stored only in the data provider's TEE. The data processor cannot decrypt the plaintext used to obtain the ciphertext features, nor can it calculate the gradient in the plaintext space, thus preventing it from performing inversion optimization.

[0148] Output encryption: The model output is returned in ciphertext to the data provider for decryption. The data processor cannot obtain the plaintext prediction results, cannot calculate the inversion loss, and therefore cannot carry out model inversion attacks.

[0149] VI. Model Extraction of Attack and Defense

[0150] Model extraction attacks aim to construct functionally equivalent alternative models by querying the target model. This embodiment implements defense through the following mechanism:

[0151] Output Ciphertext: Traditional model extraction relies on collecting plaintext query-response pairs. In this embodiment, all responses are in ciphertext form, and the "labels" obtained by the adversary are CKKS ciphertext, which cannot be directly used for supervised learning to train alternative models.

[0152] Partially hidden model architecture: This embodiment employs a hybrid neural network architecture, where the parameters of the ciphertext subnet are visible to the data processor but not to the data provider; the parameters of the plaintext subnet are exposed. Even if an adversary extracts the parameters of the plaintext subnet, the replacement model cannot replicate the complete functionality due to the lack of parameters from the ciphertext subnet and the fusion layer.

[0153] VII. Risks and Mitigation Measures of Plaintext Feature Combination Inference

[0154] Although the features in the second feature subset are encrypted, attackers may still infer sensitive information by exploiting the statistical correlation between the first feature subset (plaintext) and the second feature subset (ciphertext). This embodiment controls this risk through iterative constraints in the feature partitioning stage:

[0155] The correlation coefficients of all cross-subset feature pairs are pre-calculated based on the training dataset;

[0156] During the feature partitioning process, plaintext features that are more than a preset threshold in relation to ciphertext features are iteratively moved into the ciphertext subset;

[0157] If all high correlations cannot be resolved by moving the network, then random noise or differential privacy-based perturbations are applied to the relevant features at the plaintext subnet input.

[0158] Through the above measures, the maximum correlation coefficient between plaintext features and ciphertext features is controlled within an acceptable range, thereby limiting the risk of indirect leakage.

[0159] This application constructs an encryption priority value by fusing information entropy, SHAP value, and domain knowledge dictionary, achieving dynamic quantification of the necessity of feature encryption and a differentiated encryption strategy. Experiments use a corporate rating dataset from a domestic financial institution, containing 603 samples and 96 original features. Results show:

[0160] Accuracy and robustness of sensitive feature identification: When the domain knowledge weight γ=0.5, the recall rate of the Top-K feature set for predefined encrypted features reaches 100%, and the Jaccard consistency coefficient is 1.0, verifying the core role of domain knowledge in maintaining feature stability. Meanwhile, high-entropy, high-SHAP value features such as enterprise type, although not pre-labeled, were correctly identified as features requiring encryption after fusion scoring, demonstrating the adaptive discovery capability of this mechanism.

[0161] Model performance preservation: The hybrid neural network model obtained in this application, when homomorphically encrypting only features with encryption priority values ​​higher than the threshold, achieves an accuracy and AUC of 94.4% and 96.7% of the plaintext model (DNN), respectively, significantly outperforming the fully encrypted model (FHE-DNN). The plaintext model has the highest performance due to its lack of encryption but offers no privacy protection, while the fully encrypted model suffers a significant drop in various metrics due to ciphertext calculation errors. DSEM, on the other hand, achieves the best balance between privacy protection and prediction accuracy.

[0162] Improved computational efficiency: The single-sample inference latency of the hybrid neural network model obtained in this application is 0.94 seconds, which is about 46% lower than the 1.74 seconds of the fully encrypted model. The inference latency has an approximately linear relationship with the number of encrypted features. When the number of encrypted features increases from 1 to 19, the latency increases from 0.77 seconds to 1.41 seconds, indicating that the computational overhead of this scheme is controllable and can flexibly adapt to the needs of different privacy protection levels.

[0163] In summary, the differentiated encryption method proposed in this application achieves near-plaintext model-like high predictive performance while ensuring the protection of core sensitive features in ciphertext, and significantly reduces the computational overhead of homomorphic encryption. It provides a technical solution that balances security and practicality for privacy-sensitive scenarios such as financial risk control.

[0164] Example 3

[0165] This embodiment provides a privacy-preserving prediction system, including:

[0166] The encryption priority evaluation module is configured to receive encryption priority values ​​from the original feature vector provided by the data provider, and to divide the original feature vector into a plaintext feature subset and a ciphertext feature subset according to the encryption priority values. The original feature vector contains numerical attribute data that needs to be kept confidential. The encryption priority value is obtained by weighted fusion of at least two complementary encryption priority quantification indicators for each feature in the original feature vector. These encryption priority quantification indicators include at least two of the following: a first indicator based on statistical uncertainty of the data, a second indicator based on the contribution of the feature to the model prediction, and a third indicator based on semantic rule annotation. The specific calculation, weighting fusion, and weight optimization methods for the quantification indicators are the same as in Embodiment 1, and will not be repeated here.

[0167] The encryption module is configured to perform homomorphic encryption on the features in the second feature subset to generate ciphertext feature data.

[0168] Hybrid reasoning module, including:

[0169] The plaintext subnet is configured to process features in the first feature subset and output a first intermediate representation;

[0170] The ciphertext subnet is configured to process the ciphertext feature data and uses a polynomial function as the activation function to perform nonlinear calculations in the ciphertext domain, outputting a second intermediate representation;

[0171] The fusion layer is configured to fuse the first intermediate representation and the second intermediate representation over the ciphertext domain and output the encrypted prediction result.

[0172] The specific structure, training method, and inference process of the plaintext subnet, ciphertext subnet, and fusion layer are the same as those described in Example 1, and will not be repeated here.

[0173] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0174] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0175] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0176] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A privacy-preserving data processing method, characterized in that, include: Obtain the encryption priority value of the original feature vector provided by the data provider. The original feature vector contains numerical attribute data that needs to be kept confidential. The encryption priority value is obtained by weighted fusion of at least two complementary encryption priority quantification indicators for each feature in the original feature vector. The encryption priority quantification indicators include at least two of the following: a first indicator based on the statistical uncertainty of the data, a second indicator based on the contribution of the feature to the model prediction, and a third indicator based on semantic rule annotation. The original feature vector is divided into a first feature subset and a second feature subset according to the encryption priority value, wherein the second feature subset corresponds to features whose encryption priority value is higher than a preset condition; Receive plaintext data of the first feature subset and homomorphically encrypted data of the second feature subset from the data provider; The plaintext data is input into a first neural network subnet to extract a first hidden layer representation, and the homomorphically encrypted data is input into a second neural network subnet to extract a second hidden layer representation, wherein the second neural network subnet uses a polynomial function as an activation function to perform nonlinear computation in the ciphertext domain; The first hidden layer representation and the second hidden layer representation are fused in the ciphertext domain to output the encrypted prediction result.

2. The method according to claim 1, characterized in that, The weight coefficients used in the weighted fusion are determined by searching in the weight space such that the set of the top K features arranged in descending order of encryption priority includes all the predefined encryption features indicated by the third index, and selecting from the weights that satisfy the inclusion relationship the set of weights that optimizes the performance of the hybrid neural network composed of the first neural network subnet and the second neural network subnet, where K is a preset positive integer.

3. The method according to claim 1, characterized in that, The partitioning of the first feature subset and the second feature subset also satisfies the following constraints: Based on the current first feature subset and second feature subset, calculate the correlation coefficient between each feature in the first feature subset and each feature in the second feature subset; If there exists a first candidate feature belonging to the current first feature subset and a second candidate feature belonging to the current second feature subset, such that the absolute value of the correlation coefficient between the first candidate feature and the second candidate feature is greater than a preset correlation coefficient threshold, then the first candidate feature is moved from the current first feature subset to the current second feature subset. Repeat the movement operation until the absolute value of the correlation coefficient between any feature in the first feature subset and any feature in the second feature subset is not greater than the preset correlation coefficient threshold.

4. The method according to claim 1, characterized in that, The fusion of the first hidden layer representation and the second hidden layer representation over the ciphertext domain includes: The first hidden layer representation is encoded and homomorphically encrypted to obtain the first hidden layer representation in ciphertext form; The second hidden layer representation is multiplied by the first linear transformation weight in the ciphertext domain to obtain the first product; The first hidden layer representation in the ciphertext form is multiplied by the second linear transformation weight in the ciphertext domain to obtain the second product; Add the first product, the second product, and the biased ciphertext domain to obtain the fused ciphertext hidden layer representation; The output layer activation function, which is approximated by a polynomial, is applied to the fused ciphertext hidden layer representation to generate the encryption prediction result.

5. The method according to claim 1, characterized in that, The first neural network subnet and the second neural network subnet are obtained through simulation training: during the training phase, no actual encryption is performed on the second feature subset, but the second neural network subnet uses the same polynomial approximation activation function as the inference phase.

6. The method according to claim 1, characterized in that, Each time an original feature vector is received, the division of the first feature subset and the second feature subset is dynamically generated, and the indication information of the second feature subset is returned to the data provider in plaintext, so that the data provider performs homomorphic encryption only on the features in the current second feature subset.

7. The method according to claim 1, characterized in that, It also includes anomaly detection steps before inference: Calculate the mean vector and covariance matrix of the encryption priority values ​​of each feature on the training dataset; For the current input feature vector, calculate the Mahalanobis distance between the encryption priority value vector of the current input feature vector and the mean vector and covariance matrix; If the calculated Mahalanobis distance exceeds the preset threshold, the current input feature vector is determined to be an abnormal sample, and the prediction process is terminated.

8. The method according to claim 1, characterized in that, The network parameters of the second neural network subnet are stored only by the prediction service provider, and the data provider cannot access the parameters of the second neural network subnet; the parameters of the first neural network subnet are visible to the data provider.

9. The method according to claim 1, characterized in that, The second neural network subnet adopts a polynomial form independent of the homomorphic encryption key, and uses the same network parameters to process ciphertext data encrypted with different public keys from different data providers.

10. A privacy-preserving prediction system, characterized in that, include: An encryption priority evaluation module is configured to receive an encryption priority value of an original feature vector provided by a data provider, and to divide the original feature vector into a plaintext feature subset and a ciphertext feature subset according to the encryption priority value; wherein the original feature vector contains numerical attribute data that needs to be kept confidential, and the encryption priority value is obtained by weighted fusion of at least two complementary encryption priority quantification indicators for each feature in the original feature vector, wherein the encryption priority quantification indicators include at least two of the following: a first indicator based on data statistical uncertainty, a second indicator based on the contribution of features to model prediction, and a third indicator based on semantic rule annotation; The encryption module is configured to perform homomorphic encryption on the features in the subset of ciphertext features to generate ciphertext feature data; Hybrid reasoning module, including: The plaintext subnet is configured to process features in the plaintext feature subset and output a first intermediate representation; The ciphertext subnet is configured to process the ciphertext feature data and uses a polynomial function as the activation function to perform nonlinear calculations in the ciphertext domain, outputting a second intermediate representation; The fusion layer is configured to fuse the first intermediate representation and the second intermediate representation over the ciphertext domain and output the encrypted prediction result.

Images