CKKS three-way decision dynamic hierarchical encryption neural network training method and system

Abstract

The invention provides a CKKS three-way decision dynamic hierarchical encryption neural network training method and system. The method comprises the following steps of performing data coding on original input data, performing CKKS encryption into a polynomial, and adding a noise item to generate a ciphertext; performing homomorphic calculation on the ciphertext through alternate row and column encryption matrix multiplication, processing the intermediate feature by combining an EncryptedPolyReLU activation function and a three-way decision to obtain an encryption feature, and finally outputting an encryption prediction result; the gradient is calculated through an encryption residual sum of squares and a loss function, gradient updating is accelerated by adopting a Nesterov momentum method so as to process an encryption prediction result, and encryption model parameters are obtained by combining a layered noise perception guide mechanism and optimizing encryption model parameters. According to the method, efficient privacy protection training is realized through dynamic coding and row-column alternate encryption, and a safe and efficient deep learning solution is provided for high-sensitivity data in combination with three-way decision screening and cross-layer optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a CKKS three-branch decision dynamic hierarchical encrypted neural network training method and system. BACKGROUND

[0002] The increasingly severe data privacy problem is driving the urgent need for new deep learning methods that must be able to handle sensitive information while strictly protecting data confidentiality. Once such data is misused, it will cause serious economic losses and social trust crises. Therefore, efficient and accurate sensitive data training technology has become a core requirement for data security, and is of great significance for maintaining personal privacy and enterprise compliance.

[0003] Among the many privacy-enhanced technologies, homomorphic encryption stands out due to its unique advantages: it not only provides encryption security in the post-quantum era, but also enables end-to-end data protection, ensuring the security of original data even during the calculation process. Although deep neural networks have been applied to encrypted inference in the homomorphic encryption environment, their application in the more critical model training phase has been severely limited.

[0004] Existing research on encrypted training mostly focuses on relatively simple logistic regression models or relies on multi-party computation techniques to achieve limited model fine-tuning. The root of this limitation lies in: Conducting complex deep neural network training under the homomorphic encryption framework results in unaffordable huge computational overhead and extremely high algorithm complexity. At the same time, existing methods usually statically choose repeated encoding or expanded encoding for input data preprocessing. This fixed mode cannot adapt to the dynamic feature distribution of data in different scenarios, resulting in a large amount of information loss after encryption, which severely restricts the usability of encrypted data.

[0005] At the computational level, most schemes only support single matrix multiplication mode in row encryption or column encryption, which easily produces a large number of invalid calculation operations when processing sparse data such as medical time series, and the serious computational redundancy problem severely limits the training efficiency.

[0006] More critically, existing technologies use a local error propagation mechanism, with each hidden layer independently calculating the loss and updating the parameters, lacking cross-layer information interaction channels, which significantly reduces the model convergence speed and makes it difficult to meet the training needs of deep neural networks. SUMMARY

[0007] The present application provides a CKKS three-branch decision dynamic hierarchical encrypted neural network training method and system, which can solve the technical problems of large homomorphic encryption training computational overhead, poor data adaptability, and insufficient cross-layer gradient coordination in existing technologies.

[0008] The application provides a CKKS three-branch decision dynamic hierarchical encryption neural network training method and system, which can solve the technical problems of large homomorphic encryption training calculation overhead, poor data adaptability and insufficient cross-layer gradient coordination in the prior art.

[0009] In a first aspect, the application provides a CKKS three-branch decision dynamic hierarchical encryption neural network training method, comprising the following steps: Step S1: data encoding is performed on original input data, CKKS encryption is performed to generate ciphertext by polynomial addition and noise term, and the original input data includes input features and labels; Step S2: homomorphic calculation is performed on the ciphertext by alternating row-column encryption matrix multiplication, intermediate features are processed by combining an EncryptedPolyReLU activation function and a three-branch decision, encrypted features are obtained, and encrypted prediction results are finally output, specifically including: Step S21: the fully connected layers of the multi-layer perceptron are divided into an odd layer group and an even layer group according to the parity characteristics of layer indexes; Step S22: the odd layer group and the even layer group are calculated by alternating rows and columns to generate encrypted features; Step S23: the EncryptedPolyReLU activation function is used to perform nonlinear conversion processing on the encrypted features to output optimized encrypted features; Step S24: the optimized encrypted features are predicted and evaluated, and the sample confidence of the current processing stage is obtained by calculating the sum of squares of each element of the prediction vector; Step S25: the confidence is determined by three-branch decision according to a preset upper threshold and a lower threshold, and a determination result of the three-branch decision is obtained, the three-branch decision including receiving, rejecting and delaying decisions; Step S26: corresponding operations are performed according to the determination result of the three-branch decision to output encrypted prediction results; Step S3: the gradient is calculated by an encrypted residual sum of squares loss function, and the Nesterov momentum method is used to accelerate the gradient update to process the encrypted prediction results, and on this basis, the hierarchical noise perception guidance mechanism is combined to optimize the encrypted model parameters, and finally the optimized encrypted model parameters supporting deep network training are obtained.

[0010] Further, the step S1 of data encoding on the original input data, CKKS encryption to polynomial addition and noise term to generate ciphertext specifically includes the following steps: Step S11: receiving original input data; Step S12A: If the original input data is a one-dimensional vector, the adaptive vector encoding mechanism based on dynamic variance determination monitors the variance in real time after L2 normalization processing of the original vector, and according to the comparison result of the variance and the preset threshold, different encoding vector processing strategies are executed to obtain the input matrix. Step S12B: If the original input data is a two-dimensional matrix, the intelligent matrix encoding mechanism based on ring dimension parameters adjusts the fill factor through dynamic zero padding operation to generate the input matrix. Step S13: After converting the input matrix into a polynomial form based on ring homomorphism mapping, controllable noise is added for CKKS encryption processing, and finally the ciphertext satisfying the noise budget constraint is output.

[0011] Further, the step S12A: If the original input data is a one-dimensional vector, the adaptive vector encoding mechanism based on dynamic variance determination monitors the variance in real time after L2 normalization processing of the original vector, and according to the comparison result of the variance and the preset threshold, different encoding vector processing strategies are executed to obtain the input matrix, specifically including the following steps: If the original input data is a one-dimensional vector, according to the preset floating-point encoding specification, vectorization processing is performed on the original input data, and a unit norm encoding vector is output through L2 norm normalization operation, and the variance of the encoding vector is calculated in real time; When it is determined that the variance of the encoding vector exceeds the preset threshold, a loop encryption local loss block copying mechanism is started to process the encoding vector, and a repeated encoding matrix with row and column alignment characteristics is generated; When it is determined that the variance of the encoding vector is lower than the preset threshold, the dimension expansion mode is automatically switched to, and the encoding vector is upgraded using the Kronecker product algorithm, and an expanded encoding matrix with multiplication characteristics is output.

[0012] Further, the step S13: After converting the input matrix into a polynomial form based on ring homomorphism mapping, controllable noise is added for CKKS encryption processing, and finally the ciphertext satisfying the noise budget constraint is output, specifically including the following steps: The input matrix is subjected to standard embedding mapping conversion to obtain a polynomial; Gaussian distribution is used to generate encryption noise, and the polynomial is probabilistically encrypted combined with a private key to generate a ciphertext; The generated ciphertext is subjected to noise budget verification, and the ciphertext satisfying the noise budget constraint is output.

[0013] Further, step S3: calculate the gradient by the encrypted residual sum of squares loss function, and accelerate the gradient update by the Nesterov momentum method to process the encrypted prediction result, on the basis of which, combined with the hierarchical noise perception guiding mechanism, optimize the encrypted model parameters to finally obtain the optimized encrypted model parameters supporting the deep network training, specifically including the following steps: Step S31: perform homomorphic subtraction operation on the encrypted prediction result and the label, generate residual square items by element-by-element multiplication, and construct an encrypted local loss function; Step S32: according to the constructed encrypted local loss function, weightedly fuse the current layer loss and the next layer cross-entropy loss, add a penalty term for the to-be-decided sample, and obtain a total loss function; Step S33: based on the obtained total loss function, perform gradient update in the ciphertext space by using the improved Nesterov algorithm, and obtain updated encrypted weights and momentum; Step S34: perform hierarchical noise perception guiding operation on the updated encrypted weights and momentum respectively, and output the optimized encrypted model parameters supporting the deep network training.

[0014] In a second aspect, the present application provides a CKKS three-branch decision dynamic hierarchical encrypted neural network training system, comprising: A ciphertext acquisition module for data encoding on original input data, CKKS encryption into a polynomial and addition of a noise term to generate a ciphertext; An encrypted prediction result acquisition module in communication connection with the ciphertext acquisition module, configured to perform homomorphic calculation on the ciphertext by alternating row-column encrypted matrix multiplication, combine the EncryptedPolyReLU activation function and the three-branch decision processing intermediate feature, acquire encrypted features, and finally output encrypted prediction results, specifically including: According to the odd-even characteristics of the layer index, the fully connected layers of the multi-layer perception machine are divided into an odd layer group and an even layer group; The odd layer group and the even layer group are calculated by alternating rows and columns to generate encrypted features; The EncryptedPolyReLU activation function is used to perform nonlinear conversion processing on the encrypted features to output optimized encrypted features; The optimized encrypted features are predicted and evaluated, and the sample confidence of the current processing stage is obtained by calculating the sum of squares of each element of the prediction vector; According to the preset upper threshold and lower threshold, the confidence is determined by three-branch decision, and the determination result of the three-branch decision is obtained, the three-branch decision including receiving, rejecting and delaying decision; According to the determination result of the three-branch decision, corresponding operation is performed, and the encrypted prediction result is output; The encryption model parameter acquisition module is connected with the encrypted prediction result acquisition module in communication, is configured to calculate the gradient through an encrypted residual sum of squares loss function, and accelerate the gradient update by using a Nesterov momentum method to process the encrypted prediction result, and on this basis, the optimization of the encrypted model parameters is realized in combination with a hierarchical noise perception guiding mechanism, and finally the optimized encrypted model parameters supporting the deep network training are obtained.

[0015] Further, the ciphertext acquisition module comprises: An original input data acquisition unit configured to receive original input data; A dynamic coding vector processing unit connected with the original input data acquisition unit in communication, configured to, if the original input data is a one-dimensional vector, process the original vector through L2 normalization and monitor the variance in real time based on a dynamic variance determination adaptive vector coding mechanism, execute different coding vector processing strategies according to the comparison result of the variance and a preset threshold, and acquire an input matrix; An intelligent matrix coding processing unit connected with the original input data acquisition unit in communication, configured to, if the original input data is a two-dimensional matrix, generate an input matrix through a dynamic zero padding operation to adjust a fill factor based on an intelligent matrix coding mechanism of ring dimension parameters; A ciphertext acquisition unit connected with the intelligent matrix coding processing unit in communication, configured to convert the input matrix into a polynomial form based on ring homomorphic mapping, add controllable noise to perform CKKS encryption processing, and finally output ciphertext satisfying a noise budget constraint.

[0016] In a third aspect, the present application provides a computer readable storage medium, the computer readable storage medium has a CKKS three-way decision dynamic hierarchical encryption neural network training program stored thereon, wherein the CKKS three-way decision dynamic hierarchical encryption neural network training program is executed by a processor to implement the steps of the CKKS three-way decision dynamic hierarchical encryption neural network training method described above.

[0017] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects: The end-to-end dynamic encryption scheme and the dynamic noise management mechanism ensure the data confidentiality throughout the training process, the hierarchical guiding technology effectively controls the noise accumulation to support the deep network training, the row-column alternating encryption architecture designed innovatively significantly reduces the computational complexity, the intelligent data screening reduces the redundant operation, the momentum acceleration algorithm improves the gradient update efficiency, the adaptive coding mechanism optimizes the feature information retention, the cross-layer error propagation network accelerates the model convergence, and excellent encrypted training accuracy is achieved in the standard test, thereby providing a safe and efficient solution for high-sensitive scenarios such as financial risk control and medical data analysis. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1A flowchart of a method for CKKS three-branch decision dynamic hierarchical encrypted neural network training is provided in the embodiments of the present application. Figure 2 Another flowchart of a method for CKKS three-branch decision dynamic hierarchical encrypted neural network training is provided in the embodiments of the present application. Figure 3 A three-branch decision filter architecture diagram is provided in the embodiments of the present application. DETAILED DESCRIPTION

[0019] In order to enable persons skilled in the art to better understand the schemes of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] The terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above-described drawings are intended to cover the non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device. The terms "first", "second" and "third" and the like descriptions are used to distinguish different objects, and do not represent the order or limit the types of "first", "second" and "third".

[0021] In the description of the embodiments of the present application, "exemplary", "for example", "for instance" or the like are used as an example, illustration or description. Any embodiment or design scheme described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words "exemplary", "for example", "for instance" and the like are intended to present the relevant concept in a specific manner.

[0022] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, in addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0023] In some of the processes described in this embodiment, multiple operations or steps are presented in a particular, the order of which should not be construed as necessarily requiring the operations or steps to be performed in that particular order, nor in sequential order, unless otherwise specified herein. Additionally, some of the processes described in this embodiment can include more, fewer, or other steps. Unless otherwise specified herein, steps can be performed in any order, or in parallel.

[0024] For the purpose of making the purpose, technical scheme and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings.

[0025] In a first aspect, as Figure 1 indicated, the present application provides a CKKS ternary decision dynamic hierarchical encryption neural network training method, comprising the following steps: Step S1: data encoding is performed on original input data, CKKS encryption is performed to generate ciphertext by adding a noise term to a polynomial, and the original input data includes input features and labels; Step S2: homomorphic computation is performed on the ciphertext by alternating row-column encryption matrix multiplication, intermediate features are processed in combination with an EncryptedPolyReLU activation function and a ternary decision, and encrypted features are obtained; Step S3: gradients are calculated by an encrypted residual sum of squares loss function, and gradient updates are accelerated by a Nesterov momentum method to process encrypted prediction results, on the basis of which, an optimization of encrypted model parameters is realized in combination with a hierarchical noise perception guiding mechanism, and finally, optimized encrypted model parameters supporting deep network training are obtained.

[0026] The present application ensures data confidentiality throughout the training by an end-to-end dynamic encryption scheme and a dynamic noise management mechanism, effectively controls noise accumulation to support deep network training by a hierarchical guiding technology, significantly reduces computational complexity by an innovative row-column alternating encryption architecture, reduces redundant operations in combination with intelligent data screening, improves gradient update efficiency in cooperation with a momentum acceleration algorithm, optimizes feature information retention by an adaptive coding mechanism, and accelerates model convergence by a cross-layer error propagation network, thereby achieving excellent encrypted training accuracy in standard tests and providing a safe and efficient solution for high-sensitive scenarios such as financial risk control and medical data analysis.

[0027] In an embodiment, the step S1: data encoding is performed on original input data, CKKS encryption is performed to generate ciphertext by adding a noise term to a polynomial, specifically comprising the following steps: Step S11: receiving original input data; specifically, the original input data is a one-dimensional vector or a two-dimensional matrix , wherein, For the ring dimension of the scheme, denotes a linear array containing real numbers, denotes a real matrix of dimensions; Step S12A: If the original input data is a one-dimensional vector, the adaptive vector encoding mechanism based on dynamic variance determination processes the original vector through L2 normalization and then monitors the variance in real time. According to the comparison result of the variance and the preset threshold, different encoding vector processing strategies are executed to obtain the input matrix. Step S12B: If the original input data is a two-dimensional matrix, the intelligent matrix encoding mechanism based on the ring dimension parameter adjusts the fill factor to generate the input matrix through dynamic zero padding operation. Step S13: After converting the input matrix into a polynomial form based on the ring homomorphism mapping, controllable noise is added for CKKS encryption processing, and finally the ciphertext satisfying the noise budget constraint is output.

[0028] Through the intelligent data preprocessing mechanism, the embodiment can automatically identify the dimension characteristics of the input data and select the corresponding encoding strategy. For a one-dimensional vector, the dynamic variance determination method is used to realize adaptive normalization processing, ensuring the rationality of data distribution. For a two-dimensional matrix, the intelligent padding technology is used to maintain the structural integrity. Then, the standardized data is converted into a polynomial form and integrated with controllable noise to complete encryption, effectively controlling the noise impact while ensuring security. Finally, the ciphertext result meeting the confidentiality requirement is output, realizing seamless conversion from the original form to the encrypted form, and balancing the processing efficiency and security.

[0029] In an embodiment, the step S12A: If the original input data is a one-dimensional vector, the adaptive vector encoding mechanism based on dynamic variance determination processes the original vector through L2 normalization and then monitors the variance in real time. According to the comparison result of the variance and the preset threshold, different encoding vector processing strategies are executed to obtain the input matrix, specifically including the following steps: If the original input data is a one-dimensional vector a, according to the preset floating-point encoding specification, vectorization processing is performed on the original input data, and the L2 norm normalization operation is used to output the unit norm encoding vector. The variance Var(a) of the encoding vector is calculated in real time. According to the comparison result of the variance and the preset variance threshold, the encoding mode is adaptively selected, as shown in the following formula: A: When it is determined that the variance of the encoding vector exceeds the preset threshold, the loop encryption local loss block replication mechanism is started to process the encoding vector, and a repeated encoding matrix with row-column alignment characteristics is generated , i.e., the input matrix, as shown in the following formula: wherein, represents d features of the input one-dimensional vector a, d is the number of features; in the formula, represents the generated repetition encoding matrix, the dimension of the repetition encoding matrix is ; wherein, each row in the formula is a copy of a, the column padding length c is dynamically calculated according to the following formula: wherein, represents the generated repetition encoding matrix, the dimension of the repetition encoding matrix is , represents the floor function, represents the added noise budget.

[0030] B: when it is determined that the variance of the encoding vector is lower than a preset threshold, automatically switching to a dimension expansion mode, using a Kronecker product algorithm to perform dimension upgrading processing on the encoding vector, and outputting an expansion encoding matrix with a multiplication characteristic , that is, an input matrix, as shown in the following formula: The row padding length r is dynamically calculated according to the following formula: wherein, represents the generated expansion encoding matrix, and is hereinafter referred to as , the dimension is , and the structure is: each row is filled with corresponding elements of the input one-dimensional vector repeatedly (for example, the first row is all a1, the second row is all a2, and so on), and the insufficient column number is filled with zeros; represents the row padding length, which is dynamically calculated according to the formula , represents that the dimension of the original vector is rounded up to the nearest power of 2, so as to ensure that the number of rows meets the alignment requirement; d represents the dimension of the original input one-dimensional vector .

[0031] The embodiment intelligently adjusts the encoding strategy of one-dimensional vector through a dynamic variance determination mechanism, ensuring the flexibility and adaptability of data processing. After normalization, the degree of data fluctuation is monitored in real time, and different processing methods are adopted for different variance conditions: when the data fluctuation is large, a cyclic replication method is used to generate a regular matrix to ensure data stability; when the data fluctuation is small, an ascending dimension algorithm is used to enhance the feature expression ability and improve the subsequent calculation efficiency, realizing the adaptive matrix conversion of vector data, preserving the key features of the original information, and optimizing the data structure, providing a high-quality input basis for subsequent encryption processing.

[0032] In an embodiment, the step S12B: if the original input data is a two-dimensional matrix, the intelligent matrix encoding mechanism based on the ring dimension parameter adjusts the fill factor to generate an input matrix through a dynamic zero padding operation, specifically including the following steps: If the original input data is a two-dimensional matrix Perform a zero padding operation on the two-dimensional matrix to generate an input matrix that meets the dimension standard, which is specifically implemented as: Row encoding The format is: Column encoding The format is: Wherein, represents the parameter of the dth row and the kth column of the input matrix , and r and c represent the row dimension and column dimension of the encoded matrix, respectively.

[0033] If row encoding is used, the two-dimensional matrix is directly zero padded in the new row and column to obtain ; if column encoding is used, the transpose of is obtained , and then the new row and column of are zero padded to obtain . For the obtained or , hereinafter it is uniformly denoted as .

[0034] This embodiment achieves secure conversion by mapping matrix data into a polynomial form and combining it with a probabilistic encryption method. During the encryption process, the system automatically generates random noise that conforms to a Gaussian distribution and works with the private key to complete data obfuscation and ensure the security of the ciphertext. At the same time, a noise budget verification mechanism is used to strictly control noise interference during the encryption process, maintaining calculation accuracy while ensuring data confidentiality. The final output is an encryption result that meets both security requirements and is operational, achieving secure and reliable conversion of data from plaintext to ciphertext, laying the foundation for subsequent homomorphic computing.

[0035] In one embodiment, step S13: converting the input matrix into a polynomial form based on a ring homomorphic mapping and then adding controllable noise to perform CKKS encryption processing, and finally outputting a ciphertext that satisfies the noise budget constraint, specifically includes the following steps: For the input matrix or The canonical embedding mapping is converted into a polynomial as shown below: Where, represents the canonical embedding mapping function, which is used to convert the input matrix into a polynomial form. express dimensional complex vector space, where is a set of complex numbers, N is the ring dimension; represents the plaintext target polynomial ring, represents the set of polynomials with integer coefficients, Represents a cyclotomic polynomial, which serves as a reduction polynomial for modular operations, ensuring that the polynomial ring structure supports homomorphic encryption operations.

[0036] Gaussian distribution is used to generate encryption noise, and the polynomial is probabilistically encrypted with the private key to generate ciphertext, as shown in the following formula: Where, Represents the encrypted ciphertext polynomial, which is the input data after canonical embedding mapping, adding Gaussian noise and using the private key The final result after encryption, where , belongs to the polynomial ring Elements in It is the abbreviation of encrypt; Indicates that it is based on a private key Encryption algorithm Represents the polynomial plaintext after mapping, which is represented by the input matrix or Embedding Mapping via Specifications We get, which belongs to the plaintext polynomial ring Elements in .

[0037] The generated ciphertext is subjected to noise budget verification, and the ciphertext satisfying the noise budget constraint is output, as shown in the following formula: In the formula, is the initial noise budget, is the noise growth coefficient, which ensures that the encrypted ciphertext can still support subsequent calculations; represents the noise budget of the input ciphertext; represents the column dimension of the encoded matrix, that is, the number of columns in the matrix, represents the row dimension of the encoded matrix, that is, the number of rows in the matrix.

[0038] The present embodiment converts the input matrix into a polynomial form through a normalized mathematical mapping, laying a foundation for subsequent encryption processing. In the encryption phase, the system uses random noise that meets statistical characteristics combined with a private key to achieve secure and reliable probabilistic encryption, ensuring data confidentiality. At the same time, a noise budget verification mechanism is introduced to accurately control the degree of interference introduced during encryption, maintaining data usability while ensuring security. Ultimately, high-quality ciphertext that meets both privacy requirements and computational feasibility is output, realizing the secure conversion of data from its original form to an encrypted form and providing reliable protection for homomorphic operations.

[0039] In an embodiment, the step S2 of performing homomorphic computation on the ciphertext by alternating row-column encryption matrix multiplication, combining the EncryptedPolyReLU activation function and the three-branch decision processing intermediate features, obtaining encrypted features, and finally outputting encrypted prediction results, specifically includes the following steps: Step S21: According to the parity characteristics of the layer index, the H fully connected layers of the multi-layer perception machine are divided into an odd layer group and an even layer group; Step S22: Implementing row-column alternating calculation on the odd layer group and the even layer group to generate encrypted features; specifically including the following steps: For the odd layer, use RE-FC layer construction, where RE (Row-Encrypted) represents row encryption, and FC (Fully Connected Layer) represents a fully connected layer, using row-encrypted matrix multiplication , the formula is: where, represents homomorphic element-wise multiplication, and are encrypted activation vectors and pre-activation vectors in extended and repeated formats, respectively, the subscript rep of is the abbreviation for repetition, and is an abbreviation for expansion, is an abbreviation for Matrix Multiplication, is an encrypted weight matrix; represents element-wise multiplication followed by summation along the row direction, where represents computation along the row direction; For even layers, CE-FC layers are constructed using column-encrypted matrix multiplication where CE (Col-Encrypted) represents column-encrypted, and FC (Fully Connected Layer) represents a fully connected layer, and the formula is: where, represents homomorphic element-wise multiplication, and are encrypted activation vectors and pre-activation vectors in expansion format and repetition format, respectively, the subscript rep of is an abbreviation for repetition, the subscript is an abbreviation for expansion, is an encrypted weight matrix; column-encrypted matrix multiplication operation is summation along the row; and are collectively denoted as ; represents element-wise multiplication followed by summation along the column direction, where represents computation along the column direction; Step S23: Perform non-linear conversion on the encrypted features by an activation function, and output the optimized encrypted features: The activation function is as follows: where, and represent homomorphic element-wise addition and multiplication, respectively, involving only element-wise operations without changing the encoding structure of the encrypted vector, represents an encrypted pre-activation input, and for odd layers, it is , and for even layers, it is . represents an encrypted activation output, i.e., an encrypted feature, the subscript is an abbreviation for expansion, The subscript rep of the index is the abbreviation of repetition; wherein, The activation function is using as a second order polynomial approximation of the function; Step S24: Perform a prediction quality evaluation on the optimized encrypted features, and obtain the sample confidence of the current processing stage by calculating the sum of squares of each element of the prediction vector; as shown in the following formula: Figure 2 The CKKS three-way decision dynamic hierarchical encrypted neural network training method provided by the embodiment of the application is shown in the flowchart, and specifically includes the following steps: Construct an encrypted local loss block; more specifically, to reduce the multiplication depth and guide operation frequency required for encrypted training, the method divides the MLP into multiple encrypted local loss blocks. For each encrypted fully connected layer (denoted as the h-th layer, h belongs to the set H of all fully connected layers) in the multi-layer perceptron, the encrypted fully connected layer, the EncryptedPolyReLU activation function used with it, and a local classifier with a dimension of (denoted as ) are combined to form an encrypted local loss block, denoted as ; Calculate the information confidence. At the output end of each encrypted local loss block , the data is filtered through a three-way decision module to calculate the confidence of the prediction vector , i.e., the sample confidence of the current processing stage , as shown in the following formula: wherein, represents the length of the prediction vector , i.e., the number of categories, represents the summation index, which is used to traverse each element of the prediction vector; Step S25: Perform a three-way decision on the confidence according to the preset upper threshold and lower threshold, and obtain the three-way decision result, including , and , which represent receive, reject, and delay decisions, respectively; wherein, the three-way decision rule is: wherein, the upper threshold of the three-way decision is , and the lower threshold is (0 ), the upper threshold and the lower threshold are set as learnable parameters, and the decision boundary is optimized through gradient update: in, is the threshold learning rate, To count the current batch The proportion of decision-making is the expected acceptance rate, which represents the long-term average proportion of Accept decisions ultimately reached by the entire model on all data.

[0040] Step S26: Execute corresponding operations according to the judgment results of the three decisions and output the encrypted prediction results; the specific implementation is: , When it is Accept, the current prediction is directly output and the propagation process is terminated, entering the loss calculation and gradient update stage; , If it is Reject, the sample will be discarded and marked as "difficult sample"; , when a Defer decision occurs, the activation vector will be encrypted Passed to subsequent encrypted local loss blocks Continue processing.

[0041] This embodiment optimizes the processing flow of encrypted data through a layered alternating calculation strategy, improving computational efficiency while maintaining homomorphic properties. The system uses an even-odds layering mechanism combined with a row-column alternating calculation method to effectively reduce computational complexity and enhance feature expression capabilities. By introducing a nonlinear activation function to optimize the conversion of encrypted features, and utilizing a three-branch decision-making mechanism to dynamically evaluate and screen the processing results, intelligently selecting subsequent operation paths based on the confidence level to ensure high-quality retention of key features, the entire process achieves efficient computation and intelligent screening of encrypted data, improving the accuracy and reliability of feature extraction while ensuring security.

[0042] In one embodiment, step S2 can be performed as follows: Figure 3 The three decision filters shown are implemented as follows: The process starts with "high-dimensional features" and first calculates the confidence of the sample; The system then makes three types of decisions based on the calculated sample confidence values: Acceptance region: If the confidence level is higher than α, the sample enters this region. The current result is retained and input to the next layer for further processing; Rejection region: If the confidence level is less than or equal to β, the sample enters this region. The sample will be directly abandoned and marked as "difficult data"; Delay / Defer: If the confidence is between β and α (i.e. not reaching the acceptance criterion but also not low enough to be rejected), the sample enters this region. The current result is discarded, but the sample is detached from the current layer and input to the next layer for further processing.

[0043] Finally, the filtered result, i.e. the encrypted prediction result, is output and prepared for loss calculation with the label of the training data, which is the input feature in the original input data, in step S31.

[0044] In an embodiment, step S3: the gradient is calculated by the encrypted residual sum of squares loss function, and the Nesterov momentum method is used to accelerate the gradient update to process the encrypted prediction result, on the basis of which, the optimization of the encrypted model parameters is realized in combination with the hierarchical noise perception guidance mechanism, and finally the optimized encrypted model parameters supporting the training of the deep network are obtained, which specifically includes the following steps: Step S31: Homomorphic subtraction operation is performed on the encrypted prediction result and the label, and the residual square term is generated by element-wise multiplication to construct the encrypted local loss function , as shown in the following formula: wherein, is the encrypted prediction result of the local classifier output, is the encrypted one-hot label, i.e. the label; represents homomorphic element-wise subtraction, represents homomorphic element-wise multiplication, is the expansion (expansion) abbreviation, and rep is the repetition (repetition) abbreviation, is a small classifier matched with the encrypted fully connected layer of the hth layer.

[0045] Step S32: According to the constructed encrypted local loss function, the current layer loss and the next layer cross-entropy loss are fused by weighting, a penalty term is added for the pending decision sample, and the total loss function is obtained, as shown in the following formula: wherein, is the total loss of the hth layer, is the loss weight coefficient, is the delay decision indicator function, represents the L2 norm square of the encrypted activation vector, , is the cross-layer attenuation coefficient, is the cross-entropy loss of the next layer, wherein h represents the hth encrypted fully connected layer, represents the encrypted real label vector of the first element, represents the number of categories of tasks, i.e., the length of the prediction vector; Step S33: Based on the obtained total loss function, gradient update is performed in the ciphertext space by using the improved Nesterov algorithm to obtain updated encrypted weights and momentum; specifically, given the encrypted gradient of the first iteration , the weight decay rate , the weight and momentum are updated by combining the Nesterov accelerated gradient algorithm, as shown in the following formula: wherein, is the encrypted gradient calculated after the tth iteration, is a plaintext matrix filled with a constant , used to match the dimension of the encrypted weight gradient, represents homomorphic element-wise addition, represents homomorphic element-wise multiplication; In the first iteration process, i.e., t = 1, the momentum is initialized as shown in the following formula: wherein, represents the encrypted momentum of the second iteration, represents the encrypted gradient calculated after the first iteration.

[0046] In the iteration process, the encrypted momentum of the tth iteration is used to accumulate gradient information, which plays the role of a momentum term, making the update smoother and accelerating convergence.

[0047] At the same time, the encrypted weight update is as shown in the following formula: wherein, and are plaintext matrices filled with corresponding constants, represents the amount of encrypted weight of the first iteration that needs to be updated, represents the encrypted gradient calculated after the first iteration, which is the gradient value calculated in the ciphertext space, reflecting the direction and original amplitude of the weight that needs to be adjusted.

[0048] Momentum iterative update, specifically, for subsequent iterations , the encrypted momentum update is: wherein, is the encrypted momentum of the tth iteration, is the encrypted momentum of the t-1th iteration, is the encrypted momentum of the first iteration. The encryption momentum of the iteration, It is a plaintext matrix of dimension r×c, filled with momentum parameter μ, which is used to match the dimension of encrypted momentum so that the plaintext parameters can be operated with the encrypted momentum.

[0049] At the same time, the encryption weight is updated as: in, represents the encryption weight update amount of the t-th iteration, represents the learning rate constant plaintext matrix, represents plaintext-ciphertext element-by-element multiplication, represents the constant plaintext matrix represented by the learning rate × momentum coefficient, represents homomorphic element-wise subtraction; Step S34: Perform layered noise-aware guidance operations on the updated encryption weights and momentum, and output optimized encryption model parameters that support deep network training. The specific implementation is as follows: For RE-FC layer and CE-FC local classifier The ciphertext with related weights and speed performs two approximate bootstrapping operations: in, represents the encrypted weight after the t+1th iteration of the hth RE-FC layer stored in rows, represents the encrypted momentum after the t+1th iteration of the hth RE-FC layer stored row by row, Indicates the The encrypted weights after the t+1th iteration stored in columns for each CE-FC station, Indicates the The encrypted momentum after the t+1th iteration stored in columns for each CE-FC bureau; Represents a guided refresh operation process, which receives one or more ciphertexts as input, performs a noise refresh on them, and returns a new ciphertext with the same plaintext value but a lower noise level. "row" represents row-wise expansion, indicating that the parameters are stored in a "row-wise" format and subsequently participate in row-encrypted matrix multiplication. "col" represents column-wise expansion, indicating that the parameters are stored in a "column-wise" format and subsequently participate in column-encrypted matrix multiplication.

[0050] The embodiment realizes efficient update of encrypted model parameters by combining encrypted residual calculation and dynamic gradient optimization. The system adopts homomorphic operation to construct a multi-level loss evaluation system, accurately calculates prediction deviation and fuses cross-entropy information in a ciphertext state, and introduces a decision sample penalty mechanism to enhance model discrimination ability. With the help of an improved momentum acceleration algorithm, parameter update is performed in the encrypted space, effectively improving convergence speed and avoiding local optimum. Through a hierarchical noise perception mechanism, the updated weights are intelligently adjusted to ensure the stability and security of the training process, and the precise optimization of encrypted model parameters is realized under the premise of ensuring data privacy, providing a reliable encrypted learning scheme for deep network training.

[0051] In a second aspect, the application provides a CKKS three-branch decision dynamic hierarchical encrypted neural network training system, comprising: A ciphertext acquisition module is configured to perform data encoding on original input data, encrypt the original input data into a polynomial by CKKS, and add a noise term to generate a ciphertext. An encrypted prediction result acquisition module is in communication connection with the ciphertext acquisition module and is configured to perform homomorphic calculation on the ciphertext by alternating row-column encrypted matrix multiplication, combine an EncryptedPolyReLU activation function and three-branch decision processing intermediate features, and acquire encrypted features. An encrypted model parameter acquisition module is in communication connection with the encrypted prediction result acquisition module and is configured to calculate a gradient by an encrypted residual sum of squares loss function, accelerate gradient update by a Nesterov momentum method, process encrypted prediction results, and on this basis, combine a hierarchical noise perception guidance mechanism to realize optimization of encrypted model parameters, and finally obtain optimized encrypted model parameters supporting deep network training.

[0052] In an embodiment, the ciphertext acquisition module comprises: An original input data acquisition unit is configured to receive original input data. A dynamic encoding vector processing unit is in communication connection with the original input data acquisition unit and is configured to, if the original input data is a one-dimensional vector, monitor variance in real time after processing the original vector by L2 normalization based on an adaptive vector encoding mechanism determined by dynamic variance, execute different encoding vector processing strategies according to a comparison result of the variance and a preset threshold, and acquire an input matrix. An intelligent matrix encoding processing unit is in communication connection with the original input data acquisition unit and is configured to, if the original input data is a two-dimensional matrix, generate an input matrix by adjusting a padding factor through a dynamic zero padding operation based on an intelligent matrix encoding mechanism of ring dimension parameters. The ciphertext acquisition unit is in communication connection with the intelligent matrix encoding processing unit, configured to perform CKKS encryption processing by adding controllable noise after converting an input matrix into a polynomial form based on a ring homomorphism mapping, and finally output ciphertext satisfying a noise budget constraint.

[0053] The functions of each module in the CKKS three-way decision dynamic hierarchical encryption neural network training system correspond to the steps in the CKKS three-way decision dynamic hierarchical encryption neural network training method, and the functions and implementation processes will not be repeated here.

[0054] In a third aspect, the embodiments of the present application provide a CKKS three-way decision dynamic hierarchical encryption neural network training device. The CKKS three-way decision dynamic hierarchical encryption neural network training device can be a personal computer (PC), a notebook computer, a server, or other devices with data processing functions.

[0055] In the embodiments of the present application, the CKKS three-way decision dynamic hierarchical encryption neural network training device can include a processor, a memory, a communication interface, and a communication bus.

[0056] The communication bus can be of any type, used to interconnect the processor, the memory, and the communication interface.

[0057] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces, and other interfaces used to interconnect devices within the CKKS three-way decision dynamic hierarchical encryption neural network training device, and interfaces used to interconnect the CKKS three-way decision dynamic hierarchical encryption neural network training device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber interface, an ATM interface, etc.; the user device can be a display (Display), a keyboard (Keyboard), etc.

[0058] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disks, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0059] The processor can be a general-purpose processor, which can invoke a CKKS three-branch decision dynamic hierarchical encryption neural network training program stored in the memory and execute the CKKS three-branch decision dynamic hierarchical encryption neural network training method provided in the embodiments of the present application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the CKKS three-branch decision dynamic hierarchical encryption neural network training program is invoked can refer to the various embodiments of the CKKS three-branch decision dynamic hierarchical encryption neural network training method of the present application, which will not be described here.

[0060] In a fourth aspect, the embodiments of the present application further provide a readable storage medium.

[0061] The CKKS three-branch decision dynamic hierarchical encryption neural network training program is stored on the readable storage medium of the present application, wherein the CKKS three-branch decision dynamic hierarchical encryption neural network training program is executed by the processor to implement the steps of the CKKS three-branch decision dynamic hierarchical encryption neural network training method as described above.

[0062] The method implemented when the CKKS three-branch decision dynamic hierarchical encryption neural network training program is executed can refer to the various embodiments of the CKKS three-branch decision dynamic hierarchical encryption neural network training method of the present application, which will not be described here.

[0063] It should be noted that the above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0064] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software and the necessary general hardware platform, of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) as described above, and includes a plurality of instructions for causing a terminal device to execute the methods described in the various embodiments of the present application.

[0065] The above is only the preferred embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the contents of the specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A CKKS ternary decision dynamic hierarchical encrypted neural network training method, characterized in that, The method comprises the following steps: Step S1: data encoding on original input data, CKKS encryption into a polynomial and addition of a noise term to generate ciphertext, the original input data comprising a label; Step S2: performing homomorphic computation on the ciphertext by alternating row-column encryption matrix multiplication, combining an EncryptedPolyReLU activation function and a three-branch decision to process intermediate features, obtaining encrypted features, and finally outputting encrypted prediction results, specifically comprising: Step S21: dividing the fully connected layers of the multi-layer perceptron into an odd layer group and an even layer group according to the parity of the layer index; Step S22: implementing row-column alternating computation on the odd layer group and the even layer group to generate encrypted features; Step S23: performing nonlinear conversion processing on the encrypted features using the EncryptedPolyReLU activation function to output optimized encrypted features; Step S24: performing prediction quality evaluation on the optimized encrypted features, obtaining the sample confidence of the current processing stage by calculating the sum of squares of the elements of the prediction vector; Step S25: performing three-branch decision determination on the confidence according to a preset upper threshold and a lower threshold to obtain a three-branch decision result, the three-branch decision comprising a receive, reject, and delay decision; Step S26: performing corresponding operations according to the three-branch decision result to output encrypted prediction results; Step S3: calculating the gradient through an encrypted residual sum of squares loss function and accelerating the gradient update using the Nesterov momentum method to process the encrypted prediction results, and on this basis, combining a hierarchical noise perception guidance mechanism to optimize the encrypted model parameters, and finally obtaining optimized encrypted model parameters supporting deep network training.

2. The CKKS ternary decision dynamic hierarchical encrypted neural network training method of claim 1, wherein, The step S1: data encoding on original input data, CKKS encryption into a polynomial and addition of a noise term to generate ciphertext, specifically comprising the following steps: Step S11: receiving original input data; Step S12A: if the original input data is a one-dimensional vector, an adaptive vector encoding mechanism based on dynamic variance determination processes the original vector through L2 normalization and then monitors the variance in real time, and according to the comparison result of the variance and a preset threshold, different encoding vector processing strategies are executed to obtain an input matrix; Step S12B: if the original input data is a two-dimensional matrix, an intelligent matrix encoding mechanism based on ring dimension parameters generates an input matrix through a dynamic zero padding operation to adjust the padding factor; Step S13: converting the input matrix into a polynomial form based on ring homomorphism mapping, adding controllable noise for CKKS encryption processing, and finally outputting ciphertext satisfying the noise budget constraint.

3. The CKKS ternary decision dynamic hierarchical encrypted neural network training method of claim 2, wherein, The step S12A: if the original input data is a one-dimensional vector, an adaptive vector encoding mechanism based on dynamic variance determination processes the original vector through L2 normalization and then monitors the variance in real time, and according to the comparison result of the variance and a preset threshold, different encoding vector processing strategies are executed to obtain an input matrix, specifically comprising the following steps: If the original input data is a one-dimensional vector, vectorization is performed on the original input data according to a preset floating-point encoding specification, an encoding vector with a unit norm is outputted through an L2 norm normalization operation, and the variance of the encoding vector is calculated in real time; When it is determined that the variance of the code vector exceeds a preset threshold, a cyclic encryption local loss block replication mechanism is initiated to process the code vector to generate a repeated code matrix with row and column alignment characteristics; When it is determined that the variance of the coding vector is lower than a preset threshold, it automatically switches to the dimension expansion mode, uses the Kronecker product algorithm to increase the dimension of the coding vector, and outputs an expanded coding matrix with a multiplication characteristic.

4. The CKKS ternary decision dynamic hierarchical encrypted neural network training method of claim 2, wherein, The step S13: converting the input matrix into a polynomial form based on the ring homomorphic mapping and then adding controllable noise to perform CKKS encryption processing, and finally outputting the ciphertext that meets the noise budget constraint, specifically includes the following steps: Perform canonical embedding mapping transformation on the input matrix to obtain the polynomial; Use Gaussian distribution to generate encryption noise, combine it with the private key to perform probabilistic encryption on the polynomial to generate ciphertext; Perform noise budget verification on the generated ciphertext and output the ciphertext that meets the noise budget constraint.

5. The CKKS ternary decision dynamic hierarchical encrypted neural network training method of claim 1, wherein, Step S3: Gradients are calculated using the encrypted residual sum of squares loss function, and the Nesterov momentum method is used to accelerate gradient updates to process the encrypted prediction results. On this basis, the layered noise perception guidance mechanism is combined to optimize the encryption model parameters, and finally the optimized encryption model parameters that support deep network training are obtained. Specifically, the following steps are included: Step S31: Perform homomorphic subtraction on the encrypted prediction result and the label, generate the residual square term through element-by-element multiplication, and construct the encrypted local loss function; Step S32: Based on the constructed encrypted local loss function, the current layer loss and the next layer cross entropy loss are weightedly fused, and a penalty term is added for the pending decision samples to obtain the total loss function; Step S33: Based on the obtained total loss function, the improved Nesterov algorithm is used to perform gradient update in the ciphertext space to obtain the updated encryption weight and momentum; Step S34: Perform layered noise-aware guidance operations on the updated encryption weights and momentum, and output optimized encryption model parameters that support deep network training. 6.A CKKS ternary decision dynamic hierarchical encrypted neural network training system, characterized in that, include: The ciphertext acquisition module is used to encode the original input data, encrypt it into a polynomial using CKKS, and add a noise term to generate the ciphertext; The encrypted prediction result acquisition module is in communication with the ciphertext acquisition module and is used to perform homomorphic calculations on the ciphertext through alternating row and column encrypted matrix multiplication, combine the EncryptedPolyReLU activation function and three-branch decision processing intermediate features, obtain encryption features, and finally output the encrypted prediction results. Specifically, it includes: The fully connected layers of the multilayer perceptron are divided into odd-numbered layer groups and even-numbered layer groups according to the even-odd characteristics of the layer index; Performing row and column alternating calculations on odd-numbered layer groups and even-numbered layer groups to generate encrypted features; Use the EncryptedPolyReLU activation function to perform nonlinear transformation on the encrypted features and output the optimized encrypted features; The optimized encrypted features are subjected to prediction quality evaluation, and the sample confidence of the current processing stage is obtained by calculating the sum of squares of each element of the prediction vector; According to the preset upper threshold and lower threshold, the confidence is subjected to three-branch decision judgment, and a three-branch decision judgment result is obtained, the three-branch decision including receiving, rejecting and delaying decision; According to the three-branch decision judgment result, corresponding operation is performed, and the encrypted prediction result is outputted; The encrypted model parameter acquisition module is in communication connection with the encrypted prediction result acquisition module, is used for calculating gradient through encrypted residual sum of squares loss function, and is used for accelerating gradient update through Nesterov momentum method, so as to process encrypted prediction result, on the basis of which, combined with hierarchical noise perception guiding mechanism, optimization of encrypted model parameter is realized, and finally optimized encrypted model parameter supporting deep network training is obtained.

7. The system of claim 6, wherein, The ciphertext acquisition module comprises: An original input data acquisition unit is configured to receive original input data; A dynamic encoding vector processing unit is in communication connection with the original input data acquisition unit, and is configured to, if the original input data is a one-dimensional vector, monitor variance in real time after processing the original vector through L2 normalization based on a dynamic variance determination adaptive vector encoding mechanism, execute different encoding vector processing strategies according to a comparison result of the variance and a preset threshold, and acquire an input matrix; An intelligent matrix encoding processing unit is in communication connection with the original input data acquisition unit, and is configured to, if the original input data is a two-dimensional matrix, generate an input matrix through a dynamic zero padding operation to adjust a fill factor based on an intelligent matrix encoding mechanism of ring dimension parameters; A ciphertext acquisition unit is in communication connection with the intelligent matrix encoding processing unit, and is configured to convert the input matrix into a polynomial form based on ring homomorphic mapping, add controllable noise for CKKS encryption processing, and finally output ciphertext satisfying a noise budget constraint.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a CKKS three-branch decision dynamic hierarchical encryption neural network training program, wherein when the CKKS three-branch decision dynamic hierarchical encryption neural network training program is executed by the processor, the steps of the CKKS three-branch decision dynamic hierarchical encryption neural network training method according to any one of claims 1 to 5 are implemented.

Images