Large model confidential fine-tuning method and system for vertical field data mining

Abstract

The application relates to a large model confidential fine-tuning method and system for vertical field data mining. The method comprises the following steps: acquiring low-rank decomposition parameters of a weight matrix to be fine-tuned in a large model and a homomorphic encryption global parameter set; calculating a ciphertext multiplication noise amplification coefficient of an output side matrix and an input side matrix; assigning a noise budget value to the output side matrix and the input side matrix based on the ciphertext multiplication noise amplification coefficient; performing homomorphic encryption on the output side matrix and the input side matrix based on the respective noise budget values to obtain corresponding ciphertext matrix data; performing ciphertext domain multiplication operation on the two ciphertext matrix data to obtain an incremental matrix ciphertext data to complete the large model parameter fine-tuning. By adopting the method, full-link confidentiality protection can be realized under the premise of ensuring the fine-tuning effect, and leakage of model parameters and training data in the calculation process can be effectively prevented.

Description

Technical Field

[0001] This invention belongs to the field of data processing, and in particular relates to a method and system for fine-tuning large models for data mining in vertical domains. Background Technology

[0002] With the rapid development of artificial intelligence technology, large-scale pre-trained models have demonstrated powerful feature extraction and representation learning capabilities in various vertical fields such as natural language processing and computer vision. To address the specific application needs of these vertical fields, it is often necessary to fine-tune the parameters of general-purpose large models to adapt them to the data distribution and task objectives of specific scenarios. Low-rank decomposition fine-tuning methods, as a class of efficient parameter fine-tuning techniques, introduce low-rank matrices to incrementally update the original weight matrix, significantly reducing the number of trainable parameters while maintaining model performance. This has become one of the mainstream solutions for adapting large models to vertical fields.

[0003] In traditional technologies, fine-tuning of large models for specific vertical domains is typically performed directly using plaintext data. Specifically, the weight matrices, gradient information, and intermediate calculation results involved in the fine-tuning process are all stored and processed in plaintext. If the fine-tuning task involves user privacy data or commercially sensitive data, it faces a serious risk of data leakage. To mitigate these issues, some existing solutions employ data anonymization or access control mechanisms to protect the fine-tuning process. However, these solutions still expose the original weight information during model parameter updates, failing to fundamentally guarantee the confidentiality of the model and data. Furthermore, when outsourcing fine-tuning tasks to cloud computing platforms, model parameters and training data are vulnerable to theft or misuse during transmission and computation.

[0004] Therefore, how to ensure the confidentiality of the entire fine-tuning process while guaranteeing the fine-tuning effect of large models in vertical fields is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for fine-tuning large models for data mining in vertical fields to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for fine-tuning large-scale models for data mining in vertical domains, including:

[0007] S1. Obtain the low-rank decomposition parameters and homomorphic encryption global parameter set of the weight matrix to be fine-tuned in the large model; where the low-rank decomposition parameters include the output matrix, the input matrix, and the rank;

[0008] S2. Calculate the first ciphertext multiplication noise amplification factor of the output matrix and the second ciphertext multiplication noise amplification factor of the input matrix;

[0009] S3. Based on the first ciphertext multiplication noise amplification factor, obtain the first noise budget value of the output matrix; based on the second ciphertext multiplication noise amplification factor, obtain the second noise budget value of the input matrix.

[0010] S4. Based on the first noise budget value and the homomorphic encryption global parameter set, homomorphic encryption is performed on the output matrix to obtain the output ciphertext matrix data.

[0011] S5. Based on the second noise budget value and the homomorphic encryption global parameter set, homomorphic encryption is performed on the input matrix to obtain the input ciphertext matrix data.

[0012] S6. Perform ciphertext domain multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain the incremental matrix ciphertext data; the incremental matrix ciphertext data is used to perform parameter fine-tuning on the large model in the ciphertext domain.

[0013] In one embodiment, the homomorphic encryption global parameter set includes the number of ciphertext multiplications, and S2 includes:

[0014] S21. Perform row vector infinite norm operation on the output matrix to obtain the first matrix norm value;

[0015] S22. Perform column vector infinity norm operation on the input matrix to obtain the norm value of the second matrix;

[0016] S23. Multiply the norm of the first matrix and the number of ciphertext multiplications to obtain the first initial magnification factor;

[0017] S24. Multiply the norm of the second matrix and the number of ciphertext multiplications to obtain the second initial magnification factor;

[0018] S25. The first initial amplification factor and the second initial amplification factor are standardized and adjusted to obtain the first ciphertext multiplication noise amplification factor and the second ciphertext multiplication noise amplification factor.

[0019] In one embodiment, S3 includes:

[0020] S31. Extract the ciphertext modulus parameters from the homomorphic encryption global parameter set to obtain the initial noise budget total value;

[0021] S32. Perform a binary logarithmic rounding operation on the first ciphertext multiplication noise amplification coefficient to obtain the first noise consumption value;

[0022] S33. Perform a binary logarithmic rounding operation on the second ciphertext multiplication noise amplification coefficient to obtain the second noise consumption value;

[0023] S34. Perform a difference calculation on the initial noise budget total value and the first noise consumption value to obtain the first noise budget value of the output matrix;

[0024] S35. Perform a difference operation on the initial noise budget total value and the second noise consumption value to obtain the second noise budget value of the input side matrix.

[0025] In one embodiment, the homomorphic encryption global parameter set also includes a scaling factor and a public key, and S4 includes:

[0026] S41. Based on the scaling factor, perform scaling and rounding operations on the output matrix to obtain the integerized output matrix;

[0027] S42. Perform polynomial encoding on the integerized output matrix to obtain the output plaintext polynomial; whereby the expression of the output plaintext polynomial is:

[0028] ;

[0029] in, For the plaintext polynomial on the output side, Let the degree of the polynomial be the polynomial in the global parameter set of homomorphic encryption. This is the index of the polynomial coefficients. The integer-based output matrix, after being expanded in row-major order, corresponds to the [number]th [unit]. The element value at each position;

[0030] S43. Based on the public key, under the constraint of the first noise budget value, perform central binomial distribution sampling operation on the coefficients of the first random polynomial and the coefficients of the first error polynomial to obtain the first random polynomial and the first error polynomial.

[0031] S44. Perform ciphertext generation operation on the plaintext polynomial, the first random polynomial, the first error polynomial and the public key on the output side to obtain the first ciphertext polynomial and the second ciphertext polynomial on the output side.

[0032] S45. Combine the first ciphertext polynomial and the second ciphertext polynomial on the output side to obtain the ciphertext matrix data on the output side.

[0033] In one embodiment, S5 includes:

[0034] S51. Based on the scaling factor, perform scaling and rounding operations on the input matrix to obtain the integerized input matrix;

[0035] S52. Perform polynomial encoding on the integer input matrix to obtain the plaintext polynomial of the input side;

[0036] S53. Based on the public key, under the constraint of the second noise budget value, perform central binomial distribution sampling operation on the coefficients of the second random polynomial and the coefficients of the second error polynomial to obtain the second random polynomial and the second error polynomial.

[0037] S54. Perform ciphertext generation operation on the input plaintext polynomial, the second random polynomial, the second error polynomial and the public key in the homomorphic encryption global parameter set to obtain the input first ciphertext polynomial and the input second ciphertext polynomial.

[0038] S55. Combine the first ciphertext polynomial and the second ciphertext polynomial on the input side to obtain the ciphertext matrix data on the input side.

[0039] In one embodiment, the homomorphic encryption global parameter set further includes a relinearized key, and S6 includes:

[0040] S61. Analyze the first and second ciphertext polynomials of the output side in the ciphertext matrix data to obtain the ciphertext polynomial pair of the output side; Analyze the first and second ciphertext polynomials of the input side in the ciphertext matrix data to obtain the ciphertext polynomial pair of the input side.

[0041] S62. Perform ciphertext-domain polynomial multiplication on the output-side ciphertext polynomial and the input-side ciphertext polynomial over a preset polynomial ring to obtain a first product ciphertext polynomial, a second product ciphertext polynomial, and a third product ciphertext polynomial. The expression for the polynomial ring is:

[0042] ;

[0043] in, Represents a polynomial ring. The ciphertext modulus is given in the global parameter set for homomorphic encryption. Let the degree of the polynomial be the polynomial in the global parameter set of homomorphic encryption. Represents the modulus of an integer The remaining class ring, For variables of a polynomial;

[0044] S63. Based on the relinearization key, perform ciphertext dimension reduction operation on the first product ciphertext polynomial, the second product ciphertext polynomial and the third product ciphertext polynomial to obtain the first relinearized ciphertext polynomial and the second relinearized ciphertext polynomial.

[0045] S64. Based on the scaling factor, perform rescaling operation on the first linearized ciphertext polynomial and the second linearized ciphertext polynomial to obtain the product first ciphertext polynomial and the product second ciphertext polynomial.

[0046] S65. Combine the first ciphertext polynomial and the second ciphertext polynomial to obtain the ciphertext data of the increment matrix.

[0047] Secondly, this application also provides a large-scale model confidentiality fine-tuning system for vertical domain data mining, including:

[0048] The parameter acquisition module is used to obtain the low-rank decomposition parameters of the weight matrix to be fine-tuned in the large model and the homomorphically encrypted global parameter set; among which, the low-rank decomposition parameters include the output side matrix, the input side matrix, and the rank;

[0049] The noise amplification factor calculation module is used to calculate the first ciphertext multiplication noise amplification factor of the output matrix and the second ciphertext multiplication noise amplification factor of the input matrix;

[0050] The noise budget allocation module is used to obtain the first noise budget value of the output matrix based on the first ciphertext multiplication noise amplification factor, and to obtain the second noise budget value of the input matrix based on the second ciphertext multiplication noise amplification factor.

[0051] The output-side matrix encryption module is used to perform homomorphic encryption on the output-side matrix based on the first noise budget value and the homomorphic encryption global parameter set to obtain the output-side ciphertext matrix data.

[0052] The input-side matrix encryption module is used to perform homomorphic encryption on the input-side matrix based on the second noise budget value and the homomorphic encryption global parameter set to obtain the input-side ciphertext matrix data.

[0053] The ciphertext multiplication module is used to perform ciphertext domain multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain incremental matrix ciphertext data; the incremental matrix ciphertext data is used to perform parameter fine-tuning on the large model in the ciphertext domain.

[0054] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0055] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0056] The aforementioned method and system for confidential fine-tuning of large models for vertical domain data mining decomposes the large model weight matrix into output and input matrices for low-rank adaptation fine-tuning. It calculates the ciphertext multiplication noise amplification coefficient based on the norm characteristics of each matrix and the preset number of ciphertext multiplications. Then, it differentially deducts the noise consumption value of each matrix from the total noise capacity of the ciphertext modulus, assigning independent noise budget values ​​to each matrix. Based on their respective noise budget values, it performs homomorphic encryption on the output and input matrices to obtain the corresponding ciphertext matrix data. Finally, it performs ciphertext domain multiplication on the two ciphertext matrix data to obtain the incremental matrix ciphertext data, thus completing the parameter fine-tuning of the large model in the ciphertext domain. This improves the utilization efficiency of the noise budget in homomorphic encryption, reduces the probability of ciphertext operation failure due to uneven noise budget allocation, and enables vertical domain adaptation of large models without exposing the original weight matrix and fine-tuning data. It reduces the security risks of model parameters being stolen or misused during transmission and computation, ensuring the confidentiality of the entire fine-tuning process. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating a method for fine-tuning large models for vertical domain data mining in one embodiment.

[0059] Figure 2 This is a schematic diagram of the structure of a large model confidentiality fine-tuning system for vertical domain data mining in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] In one embodiment, reference Figure 1 The document presents a flowchart illustrating a method for fine-tuning large models for vertical domain data mining, as provided in this application. This embodiment uses the method applied to a fine-tuning terminal (hereinafter referred to as the terminal) as an example. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0062] S1. Obtain the low-rank decomposition parameters of the weight matrix to be fine-tuned in the large model and the homomorphic encryption global parameter set.

[0063] For example, the fine-tuning terminal obtains the low-rank decomposition parameters and homomorphically encrypted global parameter set of the weight matrix to be fine-tuned in the large model. The large model refers to a deep neural network model with hundreds of millions of parameters. The large model contains multiple Transformer layers, each of which may include multi-head self-attention sub-layers and feedforward neural network sub-layers. The weight matrix is ​​distributed across these sub-layers. After the pre-training phase of the large model is completed, the fine-tuning terminal obtains the pre-trained weight matrix, which can be denoted as... , The dimension is ,in, This indicates the row number of the pre-trained weight matrix. This indicates the number of columns in the pre-trained weight matrix. and The specific values ​​can be determined by the network structure of the large model. The fine-tuning terminal can use a low-rank decomposition method to decompose the pre-trained weight matrix to obtain low-rank decomposition parameters. Low-rank decomposition refers to decomposing the original pre-trained weight matrix into the product of two low-rank matrices; the decomposition relationship can be... ,in, This is the incremental weight matrix, which can be used to update the pre-trained weight matrix during fine-tuning. Dimensions and The dimensions are the same. Fine-tuning the terminal can increment the weight matrix. Further decomposed into output-side matrices and input side matrix The product form can be decomposed into the following relations: Output side matrix The dimension is Input side matrix The dimension is , For rank, The value is less than and The minimum value in the output matrix. and input side matrix Initial values ​​can be generated using random initialization, and the output matrix can be... Initialization can be achieved using normal distribution random sampling, with the input matrix... Initialization can be done using a zero matrix. During fine-tuning, the pre-trained weight matrix... Keep the system frozen, and fine-tune the terminal only for the output matrix. and input side matrix Perform parameter updates to reduce the scale of fine-tuning parameters in a large model from... Reduce to .

[0064] For example, the fine-tuning terminal simultaneously acquires the homomorphic encryption global parameter set. The homomorphic encryption global parameter set may contain all common parameters required for homomorphic encryption and ciphertext field operations. The homomorphic encryption global parameter set may include the ciphertext modulus. polynomial degree Scaling factor ciphertext multiplication times Public key and relinearized key Ciphertext Modulus It is an integer, and the ciphertext modulus can be used to limit the range of values ​​for the coefficients of the ciphertext polynomial. The value of can be used to determine the security strength and noise budget of the encrypted data. (Polynomial degree) yes integer powers, The value of can be used to determine the dimension of the polynomial ring and the complexity of encryption operations. Scaling factor It is greater than The scaling factor, a real number, can be used to map floating-point matrix elements to integers to accommodate operations on the integer polynomial ring. The number of ciphertext multiplications. This indicates the upper limit of the number of multiplication operations that can be performed in the ciphertext field. Public key It is a polynomial, and the public key can be used to encrypt plaintext data. Relinearization key. It is a pair of polynomials, consisting of two polynomials. The relinearization key can be used to reduce three ciphertext polynomials to two ciphertext polynomials after ciphertext multiplication. The fine-tuning terminal can store the above parameters in a structured data format in a local cache for later use.

[0065] S2. Calculate the first ciphertext multiplication noise amplification factor of the output matrix and the second ciphertext multiplication noise amplification factor of the input matrix.

[0066] For example, the fine-tuning terminal can calculate the first ciphertext multiplication noise amplification factor of the output matrix and the second ciphertext multiplication noise amplification factor of the input matrix. The ciphertext multiplication noise amplification factor is a numerical indicator that measures the contribution of matrix elements to the growth of ciphertext noise during ciphertext multiplication operations. The fine-tuning terminal can perform row vector infinite norm operations on the output matrix to obtain the maximum absolute cumulative value of the output matrix in the row direction. The fine-tuning terminal can perform column vector infinite norm operations on the input matrix to obtain the maximum absolute cumulative value of the input matrix in the column direction. The fine-tuning terminal can extract the number of ciphertext multiplications from the homomorphic encryption global parameter set, which represents the total number of multiplication operations performed in the ciphertext domain. The fine-tuning terminal can multiply the row vector infinite norm result of the output matrix with the number of ciphertext multiplications to obtain the initial amplification factor corresponding to the output matrix. The fine-tuning terminal multiplies the column vector infinite norm result of the input matrix with the number of ciphertext multiplications to obtain the initial amplification factor corresponding to the input matrix. The fine-tuning terminal can perform a standardized scaling operation on two initial amplification coefficients based on the scaling factor in the homomorphic encryption global parameter set. Dividing the initial amplification coefficients by the scaling factor yields the first and second ciphertext multiplication noise amplification coefficients. The physical significance of this standardized scaling adjustment lies in eliminating the influence of the scaling factor on the magnitude of the noise amplification coefficient, ensuring that the noise amplification coefficient accurately reflects the amplification effect of the matrix element numerical distribution characteristics on the ciphertext noise. A matrix with a larger noise amplification coefficient consumes more noise budget during ciphertext multiplication, requiring a larger noise budget value to ensure the correctness of the ciphertext multiplication.

[0067] S3. Based on the first ciphertext multiplication noise amplification factor, obtain the first noise budget value of the output matrix; based on the second ciphertext multiplication noise amplification factor, obtain the second noise budget value of the input matrix.

[0068] For example, the fine-tuning terminal can obtain a first noise budget value for the output-side matrix based on a first ciphertext multiplication noise amplification factor, and a second noise budget value for the input-side matrix based on a second ciphertext multiplication noise amplification factor. The noise budget value refers to the upper limit of available noise capacity allocated to each matrix ciphertext for homomorphic encryption, measured in bits. The fine-tuning terminal can extract the ciphertext modulus parameter from the homomorphic encryption global parameter set. The ciphertext modulus is an integer, and its binary bit length can be used to determine the total capacity available for carrying noise in the ciphertext data. The fine-tuning terminal can perform a binary logarithmic operation on the ciphertext modulus and round down to obtain an initial total noise budget value, representing the total noise capacity that the ciphertext modulus can provide. The fine-tuning terminal can perform a binary logarithmic rounding operation on the first ciphertext multiplication noise amplification factor to obtain a first noise consumption value corresponding to the output-side matrix, representing the number of noise budget bits consumed by performing one ciphertext multiplication on the matrix elements of the output-side matrix. The fine-tuning terminal can perform a binary logarithmic floor operation on the second ciphertext multiplication noise amplification coefficient to obtain the second noise consumption value corresponding to the input matrix. The fine-tuning terminal can subtract the first noise consumption value from the initial total noise budget value to obtain the first noise budget value for the output matrix. Similarly, the fine-tuning terminal can subtract the second noise consumption value from the initial total noise budget value to obtain the second noise budget value for the input matrix. Noise growth in homomorphic encryption follows an exponential law; the multiplicative noise amplification coefficient is converted to an additive noise consumption value through logarithmic operations, facilitating its deduction from the total noise capacity. Both the first and second noise budget values ​​are in bits. This noise budget allocation ensures differentiated processing of the output and input matrices, allocating larger noise budget values ​​to matrices with greater noise amplification effects, thereby guaranteeing the decryptability of the ciphertext data after multiplication in subsequent homomorphic encryption processes.

[0069] S4. Based on the first noise budget value and the homomorphic encryption global parameter set, homomorphic encryption is performed on the output side matrix to obtain the output side ciphertext matrix data.

[0070] For example, the fine-tuning terminal can perform homomorphic encryption on the output-side matrix based on a first noise budget value and a homomorphic encryption global parameter set to obtain the output-side ciphertext matrix data. Homomorphic encryption is an encryption scheme that allows direct computation on the ciphertext domain and whose decryption result is consistent with the result of performing the same computation on the plaintext. The fine-tuning terminal can read the output-side matrix and the homomorphic encryption global parameter set, which may include a scaling factor, polynomial degree, and public key. The fine-tuning terminal can perform scaling and rounding operations on each floating-point element in the output-side matrix based on the scaling factor, multiplying the floating-point elements by the scaling factor and rounding to the nearest integer to obtain the integerized output-side matrix. The fine-tuning terminal can expand the integerized output-side matrix into a one-dimensional sequence in row-major order, assign each element in the sequence to the corresponding coefficient of the polynomial, and obtain the output-side plaintext polynomial through polynomial encoding. The fine-tuning terminal can perform a central binomial distribution sampling operation based on the public key under the constraint of the first noise budget value to generate a first random polynomial and a first error polynomial. Central binomial sampling can be used to inject randomness into homomorphic encryption schemes, ensuring the security of the encryption process. The first noise budget value can serve as a constraint, limiting the upper bound of the absolute values ​​of the coefficients of the error polynomial, i.e., the absolute values ​​of the error polynomial coefficients do not exceed 2 raised to the power of the first noise budget value. The fine-tuning terminal can perform ciphertext generation operations on the output-side plaintext polynomial, the first random polynomial, the first error polynomial, and the public key on the polynomial ring, converting the plaintext polynomial into the output-side first ciphertext polynomial and the output-side second ciphertext polynomial. The fine-tuning terminal can combine the output-side first ciphertext polynomial and the output-side second ciphertext polynomial into output-side ciphertext matrix data. This encryption process hides the plaintext information of the output-side matrix within the ciphertext, and all subsequent operations are performed on the ciphertext domain, preventing third parties from extracting any valid information from the ciphertext.

[0071] S5. Based on the second noise budget value and the homomorphic encryption global parameter set, homomorphic encryption is performed on the input matrix to obtain the input ciphertext matrix data.

[0072] For example, the fine-tuning terminal can perform homomorphic encryption on the input matrix based on the second noise budget value and the homomorphic encryption global parameter set to obtain the input ciphertext matrix data. The fine-tuning terminal can read the input matrix and the homomorphic encryption global parameter set, which may include a scaling factor, polynomial degree, and public key. The fine-tuning terminal can perform scaling and rounding operations on each floating-point element in the input matrix based on the scaling factor, multiplying the floating-point elements by the scaling factor and rounding to the nearest integer to obtain the integerized input matrix. The fine-tuning terminal can expand the integerized input matrix into a one-dimensional sequence in row-major order, assigning each element in the sequence to the corresponding coefficient of the polynomial, and obtaining the input plaintext polynomial through polynomial encoding. The fine-tuning terminal can perform a central binomial distribution sampling operation based on the public key and the second noise budget value, generating a second random polynomial and a second error polynomial. The second noise budget value can serve as a constraint, limiting the upper bound of the absolute values ​​of the coefficients of the error polynomial, i.e., the absolute values ​​of the error polynomial coefficients do not exceed 2 raised to the power of the second noise budget value. The fine-tuning terminal can perform ciphertext generation operations on the input plaintext polynomial, the second random polynomial, the second error polynomial, and the public key on the polynomial ring, converting the plaintext polynomial into the input first ciphertext polynomial and the input second ciphertext polynomial. The fine-tuning terminal can then combine the input first ciphertext polynomial and the input second ciphertext polynomial into input ciphertext matrix data. The input ciphertext matrix data and the output ciphertext matrix data have the same ciphertext data structure, both being polynomial pairs containing two ciphertext polynomials. This consistent structure facilitates the parallel execution of subsequent ciphertext field multiplication operations. The plaintext information of the input matrix is ​​completely hidden after encryption, ensuring the confidentiality of the input matrix data during the fine-tuning process.

[0073] S6. Perform ciphertext domain multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain the incremental matrix ciphertext data; the incremental matrix ciphertext data is used to perform parameter fine-tuning on the large model in the ciphertext domain.

[0074] For example, the fine-tuning terminal can perform ciphertext field multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain incremental matrix ciphertext data. Ciphertext field multiplication refers to performing a multiplication operation directly on two ciphertext data without decryption, such that the decrypted result is equal to the result of multiplying the corresponding plaintext matrices. The fine-tuning terminal can read the output-side ciphertext matrix data and the input-side ciphertext matrix data. The fine-tuning terminal can parse the first and second ciphertext polynomials from the output-side ciphertext matrix data, and parse the first and second ciphertext polynomials from the input-side ciphertext matrix data. The fine-tuning terminal can perform ciphertext field polynomial multiplication on the two ciphertext polynomial pairs obtained by the ciphertext analysis on the polynomial ring to obtain the first, second, and third product ciphertext polynomials. Ciphertext field polynomial multiplication involves polynomial multiplication and addition. The result requires reducing the degree of the modulo polynomial of the polynomial ring and reducing the coefficients of the ciphertext modulus. The fine-tuning terminal can perform ciphertext dimension reduction operations on three product ciphertext polynomials based on the relinearization key in the homomorphic encryption global parameter set. The relinearization key can be auxiliary data generated beforehand during key generation and can be used to reduce three ciphertext polynomials to two. The fine-tuning terminal can then redistribute the information contained in the third product ciphertext polynomial into the first and second relinearized ciphertext polynomials through multiplication and addition operations with the relinearization key and the third product ciphertext polynomial, restoring the standard bipolynomial ciphertext structure. The fine-tuning terminal can also perform rescaling operations on the first and second relinearized ciphertext polynomials based on a scaling factor. The rescaling operation divides each coefficient of the ciphertext polynomial by the scaling factor and rounds it to the nearest integer, eliminating the squared scaling factor effect introduced by ciphertext multiplication and restoring the scaling factor of the ciphertext to its initial order of magnitude. The fine-tuning terminal can combine the rescaled first and second ciphertext polynomials into an incremental matrix ciphertext data. The incremental matrix ciphertext data corresponds to the ciphertext form of the incremental weight matrix obtained by multiplying the output-side matrix and the input-side matrix in the plaintext domain. The incremental matrix ciphertext data can be used to fine-tune the parameters of a large model in the ciphertext domain. Specifically, this can be achieved by performing ciphertext domain addition operations on the incremental matrix ciphertext data and the ciphertext data of the pre-trained weight matrix of the large model, thereby updating the model weights in the ciphertext domain. Since all operations are performed in the ciphertext domain, the weight parameters of the large model and the intermediate data during the fine-tuning process remain encrypted, enabling end-to-end confidentiality protection for vertical domain fine-tuning of large models.

[0075] The aforementioned method for confidential fine-tuning of large models for vertical domain data mining decomposes the weight matrix of the large model into two low-rank matrices: an output matrix and an input matrix. Based on the matrix norm, the noise amplification coefficient of each matrix in ciphertext multiplication is calculated. Then, noise budget values ​​are allocated to each matrix based on the total noise capacity provided by the ciphertext modulus. Finally, under noise budget constraints, homomorphic encryption is performed on the two matrices, and polynomial multiplication is completed in the ciphertext domain to obtain the incremental matrix ciphertext data. Weight updates are performed in the ciphertext domain, ensuring that the parameters of the large model and intermediate data during fine-tuning always exist in ciphertext form, achieving confidentiality protection throughout the fine-tuning process and improving data security for vertical domain fine-tuning tasks in untrusted computing environments. Simultaneously, low-rank decomposition reduces the number of parameters from the full matrix size to the sum of two smaller matrices, reducing the computational and storage overhead of ciphertext domain operations. Through differentiated noise budget allocation, the encryption strength of each matrix is ​​maximized while ensuring decryption correctness, enabling the adaptation and fine-tuning of the large model to the vertical domain without leaking any plaintext information.

[0076] In an optional embodiment, the homomorphic encryption global parameter set includes the number of ciphertext multiplications, and S2 includes:

[0077] S21. Perform row vector infinity norm operation on the output matrix to obtain the first matrix norm value.

[0078] Optionally, the fine-tuning terminal can perform row vector infinite norm operations on the output matrix to obtain the first matrix norm value. The row vector infinite norm is calculated by summing the absolute values ​​of the elements of each row vector in the matrix and then taking the maximum value among all the sums of the absolute values ​​of the row vectors. The fine-tuning terminal reads the output matrix. , The dimension is ,in, For the output side matrix the number of rows, For the output side matrix The number of columns. The fine-tuning terminal can adjust the output matrix. The norm is calculated for each row in the table. Specifically, for the first row... The fine-tuning terminal calculates the sum of the absolute values ​​of all column elements in that row. The formula for calculating the infinite norm of a row vector can be: ,in, Represents the output side matrix The Middle Line 1 The element values ​​of the column, For row index, The range of values ​​is arrive , For column indexes, The range of values ​​is arrive Fine-tune the terminal traversal output matrix. All Given a set of row vectors, calculate the sum of the absolute values ​​of each row vector. For fine-tuning, the maximum sum of absolute values ​​can be taken as the norm of the first matrix. First matrix norm value The calculation formula can be , Denotes the norm value of the first matrix. The physical meaning is the output side matrix. The maximum absolute cumulative value in the row direction, the first matrix norm, can be used to quantize the output matrix. The contribution of row vector infinite norm operations to noise amplification in ciphertext multiplication. Row vector infinite norm operations ensure that the noise budget allocation process can cover the worst-case noise growth in the matrix, thus guaranteeing the correctness and security of ciphertext field operations.

[0079] S22. Perform column vector infinity norm operation on the input matrix to obtain the norm value of the second matrix.

[0080] Optionally, the fine-tuning terminal can perform column vector infinite norm operations on the input matrix to obtain the second matrix norm value. The column vector infinite norm refers to calculating the sum of the absolute values ​​of the elements of each column vector in the matrix, and then taking the maximum of the sums of the absolute values ​​of all column vectors. The fine-tuning terminal reads the input matrix. , The dimension is ,in, For the input side matrix the number of rows, For the input side matrix The number of columns. The fine-tuning terminal can adjust the input-side matrix. The norm is calculated for each column in the table. Specifically, for the first column... The fine-tuning terminal calculates the sum of the absolute values ​​of all row elements in that column. The formula for calculating the infinite norm of a column vector is as follows: ,in, Represents the input-side matrix The Middle Line 1 The element values ​​of the column, For row index, The range of values ​​is arrive , For column indexes, The range of values ​​is arrive Fine-tuning the terminal to traverse the input-side matrix. All Given a set of column vectors, calculate the sum of the absolute values ​​of each column vector. Then, fine-tune the calculation and take the maximum sum of these absolute values ​​as the norm of the second matrix. The norm of the second matrix. The calculation formula can be , This represents the norm value of the second matrix. The physical meaning is the input-side matrix. The maximum absolute cumulative value in the column direction, the second matrix norm, can be used to quantize the input-side matrix. The degree of contribution to noise amplification in encrypted multiplication operations. The infinite norm operation of column vectors and the infinite norm operation of row vectors are mathematically equivalent in their norm definitions, but they are adapted for different matrix dimensions to ensure that the calculation of the noise amplification factor can reflect the numerical distribution characteristics of each matrix.

[0081] S23. Multiply the norm of the first matrix and the number of ciphertext multiplications to obtain the first initial amplification factor.

[0082] Optionally, the fine-tuning terminal can multiply the norm of the first matrix and the number of ciphertext multiplications to obtain the first initial amplification factor. (Number of ciphertext multiplications) It is a positive integer obtained from the global parameter set of homomorphic encryption. The physical meaning is the total number of multiplication operations performed in the ciphertext field. The fine-tuning terminal reads the norm value of the first matrix. The fine-tuning terminal can adjust the norm value of the first matrix. And the number of times of ciphertext multiplication Multiply to obtain the first initial magnification factor. First initial magnification factor The calculation formula can be The physical meaning of the formula for calculating the first initial amplification factor lies in the fact that each ciphertext multiplication operation amplifies the noise in the matrix elements. The amplification factor is proportional to the sum of the absolute values ​​of the matrix elements. Therefore, it is necessary to multiply the noise amplification effect of a single multiplication by the total number of multiplications to estimate the total cumulative amplification of noise during the entire fine-tuning process. (First initial amplification factor) It can be used to reflect the output matrix. The noise amplification potential during the entire encrypted multiplication process; the larger this value, the greater the noise amplification potential of the output matrix. The more significant the amplification effect of ciphertext noise, the more noise budget space needs to be reserved for this matrix when allocating noise budget in the future.

[0083] S24. Multiply the norm of the second matrix and the number of ciphertext multiplications to obtain the second initial amplification factor.

[0084] Optionally, the fine-tuning terminal can multiply the second matrix norm value and the number of ciphertext multiplications to obtain the second initial amplification factor. The fine-tuning terminal reads the second matrix norm value. The fine-tuning terminal can adjust the norm value of the second matrix. And the number of times of ciphertext multiplication Multiplying them together yields the second initial magnification factor. Second initial magnification factor The calculation formula can be The physical meaning of the formula for calculating the second initial amplification factor lies in the input-side matrix. Each element in the matrix also experiences noise amplification during multiplication in the ciphertext field. The amplification factor is proportional to the sum of the absolute values ​​of the column vector elements. Therefore, the infinite norm of the column vectors needs to be multiplied by the total number of multiplications to estimate the input matrix. The total cumulative noise amplification. Second initial amplification factor. It can be used to reflect the input side matrix Noise amplification potential throughout the entire ciphertext multiplication process. The fine-tuning terminal calculates the first and second initial amplification factors respectively, due to the output-side matrix... and input side matrix The numerical distribution characteristics of the elements are different, and their contribution to noise in encrypted multiplication is also different. Therefore, it is necessary to calculate their respective amplification factors separately in order to achieve differentiated noise budget allocation.

[0085] S25. The first initial amplification factor and the second initial amplification factor are standardized and adjusted to obtain the first ciphertext multiplication noise amplification factor and the second ciphertext multiplication noise amplification factor.

[0086] Optionally, the fine-tuning terminal can perform a standardized scaling adjustment on the first and second initial amplification factors to obtain the first ciphertext multiplication noise amplification factor and the second ciphertext multiplication noise amplification factor. The standardized scaling adjustment refers to dividing the initial amplification factor by a scaling factor in the homomorphic encryption global parameter set. This is done to eliminate the impact of the scaling factor on subsequent noise budget calculations. The fine-tuning terminal reads the first initial amplification factor. Second initial magnification factor and scaling factor Scaling factor The physical meaning is the scaling factor when mapping floating-point numbers to integers. First ciphertext multiplication noise amplification factor. The calculation formula can be Second ciphertext multiplication noise amplification factor The calculation formula can be The physical basis for adjusting the standardization ratio lies in the scaling factor in homomorphic encryption schemes. It will be multiplied into the coefficients of the ciphertext polynomial during the encryption process, resulting in increased noise budget consumption. The value of is relevant. This is to ensure that the subsequent calculation of the noise budget value does not depend on . For specific values, the fine-tuning terminal can divide the initial amplification factor by... This yields the dimensionless noise amplification factor. and All are dimensionless real numbers, representing the output side matrix respectively. and input side matrix The equivalent amplification factor of noise in encrypted domain multiplication. Standardized scaling ensures that the calculation of the noise budget value has a uniform standard, so that the noise budget allocation remains correct under different scaling factor values.

[0087] In an optional embodiment, S3 includes:

[0088] S31. Extract the ciphertext modulus parameters from the homomorphic encryption global parameter set to obtain the initial noise budget total value.

[0089] Optionally, the fine-tuning terminal extracts the ciphertext modulus parameter from the homomorphic encryption global parameter set to obtain the initial noise budget total value. The initial noise budget total value refers to the upper limit of noise that the ciphertext data can tolerate before decryption in the homomorphic encryption scheme. The fine-tuning terminal extracts the ciphertext modulus from the homomorphic encryption global parameter set. , It is a positive integer. Homomorphic encryption schemes are built upon the fault-tolerance learning problem, and their security depends on... The value of , The larger the value, the higher the noise tolerance in the ciphertext, but the computational complexity also increases. Initial noise budget total value The calculation formula is ,in, This indicates the floor function. Indicated by as bottom The logarithm of . The physical meaning is the ciphertext modulus. The length of a binary number is expressed in bits. This represents the total number of bits in the ciphertext that can be used to carry noise. During fine-tuning, ciphertext field operations continuously introduce new noise, and when the accumulated noise bit length exceeds... At that time, the ciphertext will not be able to be decrypted correctly. Therefore, This represents the total resource ceiling for subsequent noise budget allocation. The fine-tuning terminal can extract the total initial noise budget value. Stored in local variables.

[0090] S32. Perform a binary logarithmic rounding operation on the first ciphertext multiplication noise amplification coefficient to obtain the first noise consumption value.

[0091] Optionally, the fine-tuning terminal can perform a binary logarithmic floor operation on the first ciphertext multiplication noise amplification coefficient to obtain the first noise consumption value. The binary logarithmic floor operation refers to... Take the logarithm of the base and round it up to the nearest integer. Fine-tune the noise amplification factor of the multiplication operation when reading the first ciphertext from the terminal. First noise consumption value The calculation formula can be ,in, This indicates the rounding up operation; The physical meaning is the output side matrix. The noise budget bits consumed in a single ciphertext multiplication are expressed in bits. The physical basis of binary logarithmic operations lies in the fact that noise growth in homomorphic encryption follows an exponential law, meaning the noise is amplified by a factor of [number] times per multiplication. The corresponding bit consumption is equal to as bottom The logarithm of the value. Rounding up ensures that the noise consumption value covers the actual noise growth in the worst case, preventing incorrect decryption due to insufficient noise budget allocation caused by truncation. The value of and Related, The larger the value, the larger the output matrix. The stronger the noise amplification effect, the more noise budget is consumed. The fine-tuning terminal can adjust the calculated first noise consumption value. Stored in local variables.

[0092] S33. Perform a binary logarithmic rounding operation on the second ciphertext multiplication noise amplification coefficient to obtain the second noise consumption value.

[0093] Optionally, the fine-tuning terminal can perform a binary logarithmic floor operation on the second ciphertext multiplication noise amplification coefficient to obtain the second noise consumption value. The fine-tuning terminal reads the second ciphertext multiplication noise amplification coefficient. Second noise consumption value The calculation formula can be ,in This indicates the rounding up operation. The physical meaning is the input-side matrix. The number of noise budget bits consumed in one ciphertext multiplication, expressed in bits. The physical basis of this operation is the same as that of S32, using binary logarithmic rounding to amplify multiplicative noise into additive noise for easier subsequent noise budget allocation. The value of and Related, The larger the value, the more likely it is to be an input-side matrix. The stronger the noise amplification effect, the more noise budget is consumed. The fine-tuning terminal can adjust the calculated second noise consumption value. Stored in local variables.

[0094] S34. Perform a difference operation on the initial noise budget total value and the first noise consumption value to obtain the first noise budget value of the output side matrix.

[0095] Optionally, the fine-tuning terminal can perform a difference operation on the initial noise budget total value and the first noise consumption value to obtain the first noise budget value of the output matrix. The difference operation involves subtracting the subtrahend from the minuend to obtain the difference. The fine-tuning terminal reads the initial noise budget total value. The fine-tuning terminal simultaneously reads the first noise consumption value. The first noise budget value of the output-side matrix The calculation formula can be . The physical meaning is the matrix assigned to the output side. The noise budget, expressed in bits, is the available noise budget when homomorphically encrypting ciphertext. The physical meaning of the formula for calculating the first noise budget value lies in the fact that, from the ciphertext modulus... Total bit capacity Subtracting the output matrix Number of noise bits consumed during ciphertext multiplication The remaining part is the output-side matrix. The upper limit of the initial noise allowed when performing homomorphic encryption operations. The value must be greater than zero; otherwise, it indicates that the current homomorphic encryption global parameter configuration cannot support the output side matrix. The encrypted multiplication operation. The fine-tuning terminal can use the first noise budget value as the upper bound of the central binomial distribution sampling constraint. The first noise budget value can be used to limit the range of values ​​for the coefficients of the random polynomial and the error polynomial.

[0096] S35. Perform a difference operation on the initial noise budget total value and the second noise consumption value to obtain the second noise budget value of the input side matrix.

[0097] Optionally, the fine-tuning terminal can perform a difference calculation on the initial noise budget total value and the second noise consumption value to obtain the second noise budget value of the input-side matrix. The fine-tuning terminal reads the initial noise budget total value. The fine-tuning terminal simultaneously reads the second noise consumption value. The second noise budget value of the input-side matrix The calculation formula can be . The physical meaning is the matrix assigned to the input side. The second noise budget is the available noise budget when homomorphically encrypting the ciphertext, expressed in bits. The physical meaning of the formula for calculating the second noise budget value lies in the fact that it is derived from the ciphertext modulus. Total bit capacity Subtracting the input side matrix Number of noise bits consumed during ciphertext multiplication The remaining part is the input-side matrix. The upper limit of the initial noise allowed to be injected when performing homomorphic encryption. Due to the output matrix and input side matrix Their numerical distribution characteristics are different. and The values ​​of are usually different, therefore and The values ​​of are also different, enabling differentiated noise budget allocation for the two matrices. The fine-tuning terminal can use the second noise budget value as a constraint upper bound for central binomial distribution sampling, and the second noise budget value can be used to limit the range of values ​​for the coefficients of the random polynomial and the error polynomial.

[0098] In an optional embodiment, the homomorphic encryption global parameter set further includes a scaling factor and a public key, and S4 includes:

[0099] S41. Based on the scaling factor, perform scaling and rounding operations on the output matrix to obtain the integerized output matrix.

[0100] Optionally, the fine-tuning terminal can perform scaling and rounding operations on the output-side matrix based on a scaling factor to obtain an integer-based output-side matrix. Scaling and rounding involves multiplying each element of the matrix by the scaling factor and taking the nearest integer. The fine-tuning terminal reads the output-side matrix. , The dimension is ,in, For the output side matrix the number of rows, For the output side matrix The number of columns. The fine-tuning terminal simultaneously reads the scaling factor from the homomorphic encryption global parameter set. , For a greater than Real numbers. Integerized output matrix. Dimensions and output matrix The dimensions are the same, for Integerized output side matrix The Middle Line 1 Column elements The calculation formula can be ,in, For the output side matrix The Middle Line 1 The element values ​​of the column, For row index, The range of values ​​is arrive , For column indexes, The range of values ​​is arrive , This represents the rounding operation. The physical basis of the scaling rounding operation is that plaintext data in homomorphic encryption schemes (Cheon-Kim-Kim-Song scheme, or CKKS scheme for short) must be represented in integer form to be encoded as a polynomial, while matrix elements are usually floating-point numbers. This is achieved by multiplying by a scaling factor. And rounding, floating-point numbers are approximated as integers, scaling factor Units and matrix elements The units are consistent, therefore Units and The units must be consistent. Rounding introduces quantization error, the absolute value of which should not exceed [a certain value]. The fine-tuning terminal can integerize the output side matrix. Stored in local cache.

[0101] S42. Perform polynomial encoding on the integer-based output matrix to obtain the plaintext polynomial on the output side.

[0102] Optionally, the fine-tuning terminal can perform polynomial encoding on the integer-side output matrix to obtain the plaintext polynomial on the output side. Polynomial encoding refers to mapping matrix elements to polynomial coefficients in row-major order, constructing a polynomial with a degree less than... Integer polynomials. Fine-tuning the terminal to read the integerized output matrix. , The dimension is ,in, Integerized output side matrix the number of rows, Integerized output side matrix The number of columns. Fine-tuning the terminal to simultaneously read the polynomial degree from the homomorphic encryption global parameter set. , for integer powers, The unit is the cardinality of the dimension. The fine-tuning terminal can integerize the output matrix. Expanding the sequence into a one-dimensional sequence using row-major order means first sorting by row index in ascending order, and then sorting by column index in ascending order within the same row. The length of the expanded one-dimensional sequence is... The fine-tuning terminal can adjust the first element in this one-dimensional sequence. Each element is assigned to the plaintext polynomial on the output side. middle The coefficient, where, This is the index of the polynomial coefficients. The range of values ​​is arrive Output-side plaintext polynomial The expression can be ,in, The integer-based output matrix, after being expanded in row-major order, corresponds to the [number]th [unit]. The element value at each position. For variables of a polynomial. Each coefficient is an integer, and the coefficient value is equal to the value at the corresponding position. The coefficient unit is The unit is polynomial. The physical basis of polynomial encoding is that plaintext data in homomorphic encryption must participate in ring operations in polynomial form. After encoding matrix elements as polynomial coefficients, ciphertext field multiplication operations can process multiple elements of the matrix at once, realizing the parallel computing mode of Single Instruction Multiple Data (SIMD). The fine-tuning terminal can output the plaintext polynomial. Stored in the local cache for use in ciphertext generation operations.

[0103] S43. Based on the public key, under the constraint of the first noise budget value, perform central binomial distribution sampling operation on the coefficients of the first random polynomial and the coefficients of the first error polynomial to obtain the first random polynomial and the first error polynomial.

[0104] Optionally, the fine-tuning terminal can perform central binomial distribution sampling operations on the coefficients of the first random polynomial and the coefficients of the first error polynomial based on the public key and constrained by the first noise budget value, to obtain the first random polynomial and the first error polynomial. Central binomial distribution sampling refers to the random process of independently drawing samples from the central binomial distribution. The fine-tuning terminal reads the public key from the homomorphic encryption global parameter set. , It is a polynomial. Each coefficient is an integer. The fine-tuning terminal simultaneously reads the first noise budget value. The unit is bits. First random polynomial The expression can be ,in, For the first random polynomial The One coefficient, This is the index of the polynomial coefficients. The range of values ​​is arrive , Let the variables be polynomials. First error polynomial. The expression can be ,in, The first error polynomial The Coefficients. Central binomial distribution. The set of values ​​is The probability of each value occurring is as follows: , , The fine-tuning terminal can adjust each coefficient. Independently from the central binomial distribution Mid-sampling, obtained The value of . The sampling range is affected by the first noise budget value. The constraints are as follows: ,Right now The upper limit of absolute value is of The power of. This first noise budget value. The physical basis for this constraint is that the absolute value of the coefficients of the error polynomial directly contributes to the initial noise level in the ciphertext. If the absolute value is too large, the initial noise will exceed the first noise budget value. This causes the accumulated noise after subsequent ciphertext multiplication to exceed the ciphertext modulus. The tolerance limit is exceeded, causing decryption failure. First error polynomial. Each coefficient The upper limit of absolute value is determined by The value determines the magnitude of the value. The unit is bits. Units and The units are consistent. The fine-tuning terminal can adjust the first random polynomial obtained from the sampling. and the first error polynomial Stored in local cache.

[0105] S44. Perform ciphertext generation operation on the plaintext polynomial, the first random polynomial, the first error polynomial and the public key on the output side to obtain the first ciphertext polynomial and the second ciphertext polynomial on the output side.

[0106] Optionally, the fine-tuning terminal can perform ciphertext generation operations on the output-side plaintext polynomial, the first random polynomial, the first error polynomial, and the public key to obtain the output-side first ciphertext polynomial and the output-side second ciphertext polynomial. Ciphertext generation operation refers to the process of converting the plaintext polynomial into two ciphertext polynomials based on a public-key cryptography system. The fine-tuning terminal reads the output-side plaintext polynomial. First stochastic polynomial First error polynomial and public-key polynomials , Public key The polynomial form. The fine-tuning terminal simultaneously reads the ciphertext modulus from the homomorphic encryption global parameter set. , Given a positive integer. Ciphertext generation operations are performed on the polynomial ring. Execute on, where, Represents the modulus of an integer The remaining class ring, For variables of a polynomial, Let the degree be a polynomial. It is a modular polynomial. The first ciphertext polynomial on the output side. The calculation formula can be Output-side second ciphertext polynomial The calculation formula can be Polynomial multiplication in the formula In the polynomial ring The above is executed, and the calculation result needs to be... The number of modulo operations is reduced, and at the same time... The coefficients are reduced by taking the modulus. The physical meaning is the first ciphertext component containing plaintext information. The physical meaning is the second ciphertext component used for decryption. Error polynomial. The introduction of this feature ensures the security of the encryption scheme, preventing attackers from recovering the plaintext from the ciphertext even if they obtain the public key. The fine-tuning terminal can then compute the first ciphertext polynomial on the output side. and the second ciphertext polynomial on the output side Stored in local cache.

[0107] S45. Combine the first ciphertext polynomial and the second ciphertext polynomial on the output side to obtain the ciphertext matrix data on the output side.

[0108] Optionally, the fine-tuning terminal combines the first and second ciphertext polynomials on the output side to obtain the ciphertext matrix data on the output side. Combination refers to encapsulating the two ciphertext polynomials into a single ciphertext data unit according to a ciphertext data structure. The fine-tuning terminal reads the first ciphertext polynomial on the output side. and the second ciphertext polynomial on the output side Output side ciphertext matrix data It is an ordered pair of polynomials, consisting of two ciphertext polynomials. The expression can be . The first element in the output is the first ciphertext polynomial. The second element is the second ciphertext polynomial on the output side. . The physical meaning is the output side matrix. The corresponding ciphertext data can be directly used in subsequent ciphertext field multiplication operations without decryption. The fine-tuning terminal can... and The data is stored in a structured encrypted format in serialized form. This encrypted data format can contain two polynomial fields, with the coefficients of each polynomial stored as an array of integers. The fine-tuning terminal can output the encrypted matrix data. Stored in local cache.

[0109] In an optional embodiment, S5 includes:

[0110] S51. Based on the scaling factor, perform scaling and rounding operations on the input matrix to obtain the integerized input matrix.

[0111] Optionally, the fine-tuning terminal can perform scaling and rounding operations on the input matrix based on a scaling factor to obtain an integer-based input matrix. Scaling and rounding involves multiplying each element of the matrix by the scaling factor and taking the nearest integer. The fine-tuning terminal reads the input matrix. , The dimension is ,in, For the input side matrix the number of rows, For the input side matrix The number of columns. The fine-tuning terminal simultaneously reads the scaling factor from the homomorphic encryption global parameter set. Integerized input matrix Dimensions and input matrix The dimensions are the same, for Integerized input matrix The Middle Line 1 Column elements The calculation formula can be ,in, For the input side matrix The Middle Line 1 The element values ​​of the column, For row index, The range of values ​​is arrive , For column indexes, The range of values ​​is arrive , This indicates rounding to the nearest integer. The physical basis of scaling rounding is multiplying by a scaling factor. And round down, input side matrix The floating-point elements in the array are approximated as integers. Units and matrix elements The units are consistent, therefore Units and The units must be consistent. The absolute value of the quantization error introduced by rounding should not exceed [a certain value]. The fine-tuning terminal can integerize the input-side matrix. Stored in local cache.

[0112] S52. Perform polynomial encoding on the integer input matrix to obtain the plaintext polynomial of the input side.

[0113] Optionally, the fine-tuning terminal performs polynomial encoding on the integer-side input matrix to obtain the plaintext polynomial. Polynomial encoding refers to mapping matrix elements to polynomial coefficients in row-major order. The fine-tuning terminal reads the integer-side input matrix. , The dimension is ,in, Integerized input side matrix the number of rows, Integerized input side matrix The number of columns. Fine-tuning the terminal to simultaneously read the polynomial degree from the homomorphic encryption global parameter set. The fine-tuning terminal can integerize the input-side matrix. Expanding the sequence into a one-dimensional sequence using row-major order means first sorting by row index in ascending order, and then sorting by column index in ascending order within the same row. The length of the expanded one-dimensional sequence is... The fine-tuning terminal can adjust the first element in this one-dimensional sequence. Each element is assigned to the input plaintext polynomial. middle The coefficient, where, This is the index of the polynomial coefficients. The range of values ​​is arrive Input-side plaintext polynomial The expression can be ,in The integer-valued input matrix, after being expanded in row-major order, corresponds to the first... The element value at each position. For variables of a polynomial. Each coefficient is an integer, and the coefficient value is equal to the value at the corresponding position. The coefficient unit is The unit is [unit missing]. The physical basis of polynomial encoding is that after encoding matrix elements as polynomial coefficients, ciphertext field multiplication operations can process multiple elements of the matrix in parallel. The fine-tuning terminal can adjust the input plaintext polynomial... Stored in local cache.

[0114] S53. Based on the public key, under the constraint of the second noise budget value, perform central binomial distribution sampling operation on the coefficients of the second random polynomial and the coefficients of the second error polynomial to obtain the second random polynomial and the second error polynomial.

[0115] Optionally, the fine-tuning terminal can perform central binomial distribution sampling operations on the coefficients of the second random polynomial and the coefficients of the second error polynomial based on the public key and constrained by the second noise budget value, to obtain the second random polynomial and the second error polynomial. The fine-tuning terminal reads the public key from the homomorphic encryption global parameter set. The fine-tuning terminal simultaneously reads the second noise budget value. The unit is bits. Second random polynomial The expression can be ,in, For the second random polynomial The One coefficient, This is the index of the polynomial coefficients. The range of values ​​is arrive , The variables are polynomials. The second error polynomial. The expression can be ,in, The second error polynomial The Each coefficient. The fine-tuning terminal can adjust each coefficient. Independently from the central binomial distribution Mid-sampling, The set of values ​​is The probability of each value occurring is as follows: , , ,get The value of . The sampling range is affected by the second noise budget value. The constraints are as follows: ,Right now The upper limit of absolute value is of The physical basis of this constraint is that the absolute value of the coefficients of the error polynomial directly determines the initial noise level of the ciphertext, and it is necessary to ensure that the initial noise does not exceed the second noise budget value. This ensures that the accumulated noise after subsequent ciphertext multiplication remains within the ciphertext modulus. Within the tolerance range. Units and The units are consistent. The fine-tuning terminal can adjust the sampled second random polynomial. Second error polynomial Stored in local cache.

[0116] S54. Perform ciphertext generation operation on the input plaintext polynomial, the second random polynomial, the second error polynomial, and the public key in the homomorphic encryption global parameter set to obtain the input first ciphertext polynomial and the input second ciphertext polynomial.

[0117] Optionally, the fine-tuning terminal can perform ciphertext generation operations on the input plaintext polynomial, the second random polynomial, the second error polynomial, and the public key to obtain the input first ciphertext polynomial and the input second ciphertext polynomial. The fine-tuning terminal reads the input plaintext polynomial. Second stochastic polynomial Second error polynomial and public-key polynomials The fine-tuning terminal simultaneously reads the ciphertext modulus from the homomorphic encryption global parameter set. Ciphertext generation operations in a polynomial ring Execute on the input side. First ciphertext polynomial The calculation formula can be Input-side second ciphertext polynomial The calculation formula is Polynomial multiplication in the formula In the polynomial ring The above is executed, and the calculation result needs to be... The number of modulo operations is reduced, and at the same time... The coefficients are reduced by taking the modulus. The physical meaning is that it contains the first ciphertext component containing the plaintext information of the input matrix. The physical meaning is the second ciphertext component used for decryption. Error polynomial. The introduction of this ensures the security of the encryption scheme. Fine-tuning the terminal allows the computed input-side first ciphertext polynomial to be used. and the second ciphertext polynomial on the input side Stored in local cache.

[0118] S55. Combine the first ciphertext polynomial and the second ciphertext polynomial on the input side to obtain the ciphertext matrix data on the input side.

[0119] Optionally, the fine-tuning terminal can combine the first ciphertext polynomial and the second ciphertext polynomial on the input side to obtain the ciphertext matrix data on the input side. The fine-tuning terminal reads the first ciphertext polynomial on the input side. and the second ciphertext polynomial on the input side Input side ciphertext matrix data It is an ordered pair of polynomials, consisting of two ciphertext polynomials. The expression can be . The first element in the input is the first ciphertext polynomial. The second element is the second ciphertext polynomial on the input side. . The physical meaning is the input-side matrix. The corresponding encrypted data. The fine-tuning terminal can... and The data is stored in a structured ciphertext format in serialized form, containing two polynomial fields. The fine-tuning terminal can adjust the input ciphertext matrix data. Stored in local cache.

[0120] In an optional embodiment, the homomorphic encryption global parameter set further includes a relinearized key, S6 including:

[0121] S61. Analyze the first and second ciphertext polynomials of the output side in the ciphertext matrix data to obtain the ciphertext polynomial pair of the output side; parse the first and second ciphertext polynomials of the input side in the ciphertext matrix data to obtain the ciphertext polynomial pair of the input side.

[0122] Optionally, the fine-tuning terminal can parse the first and second ciphertext polynomials on the output side of the ciphertext matrix data to obtain a pair of output-side ciphertext polynomials. Similarly, the fine-tuning terminal can parse the first and second ciphertext polynomials on the input side of the ciphertext matrix data to obtain a pair of input-side ciphertext polynomials. Parsing refers to the process of extracting each ciphertext polynomial from the ciphertext matrix data. The fine-tuning terminal reads the output-side ciphertext matrix data. The fine-tuning terminal can be adjusted from The first ciphertext polynomial on the output side is parsed. and the second ciphertext polynomial on the output side Combine these two polynomials into an output-side ciphertext polynomial pair. Fine-tuning the terminal to read the ciphertext matrix data from the input side. The fine-tuning terminal can be adjusted from Parse the first ciphertext polynomial on the input side. and the second ciphertext polynomial on the input side Combine these two polynomials into an input-side ciphertext polynomial pair. The physical basis of the parsing operation lies in the fact that the ciphertext matrix data is stored as ciphertext data units in a structured format. Before performing ciphertext field multiplication, this structured data needs to be unpacked into independent ciphertext polynomial components, and only then can each ciphertext polynomial component participate in the multiplication operation on the polynomial ring. The fine-tuning terminal can store the parsed output-side ciphertext polynomial pair and input-side ciphertext polynomial pair in a local cache.

[0123] S62. Perform ciphertext field polynomial multiplication on the output-side ciphertext polynomial pair and the input-side ciphertext polynomial pair on a preset polynomial ring to obtain the first product ciphertext polynomial, the second product ciphertext polynomial, and the third product ciphertext polynomial.

[0124] Optionally, the fine-tuning terminal performs ciphertext-domain polynomial multiplication on the output-side ciphertext polynomial pair and the input-side ciphertext polynomial pair on the polynomial ring to obtain a first product ciphertext polynomial, a second product ciphertext polynomial, and a third product ciphertext polynomial. Ciphertext-domain polynomial multiplication refers to performing operations on two ciphertext polynomial pairs according to the multiplication rules on the polynomial ring to generate three product ciphertext polynomials. The fine-tuning terminal reads the output-side ciphertext polynomial pair. and input-side ciphertext polynomial pair The fine-tuning terminal simultaneously reads the ciphertext modulus from the homomorphic encryption global parameter set. and polynomial degree Ciphertext field polynomial multiplication operations in polynomial rings Execute on, where, For variables of a polynomial, Let the degree be a polynomial. Modular polynomial. First product ciphertext polynomial The calculation formula can be Second product cryptic polynomial The calculation formula can be Third product ciphertext polynomial The calculation formula can be The polynomial multiplication and addition in the formula both occur within the polynomial ring. The above is executed, and the result of polynomial multiplication needs to be adjusted. The number of modulo operations is reduced, and at the same time... Modulo operation is used to reduce coefficients. The physical basis of ciphertext-domain polynomial multiplication lies in the homomorphic property of homomorphic encryption, which makes polynomial multiplication in the ciphertext domain correspond to element-wise multiplication in the plaintext domain. Multiplying two ciphertext polynomial pairs yields a first-product ciphertext polynomial, a second-product ciphertext polynomial, and a third-product ciphertext polynomial. These three ciphertext polynomials can contain all the information from the plaintext multiplication. The fine-tuning terminal can then calculate the first-product ciphertext polynomial. Second product ciphertext polynomial and the third product ciphertext polynomial Stored in local cache.

[0125] S63. Based on the relinearization key, perform ciphertext dimension reduction operation on the first product ciphertext polynomial, the second product ciphertext polynomial, and the third product ciphertext polynomial to obtain the first and second relinearized ciphertext polynomials.

[0126] Optionally, the fine-tuning terminal can perform ciphertext dimension reduction operations on the first, second, and third product ciphertext polynomials based on the relinearization key, obtaining a first-level linearized ciphertext polynomial and a second-level linearized ciphertext polynomial. Ciphertext dimension reduction refers to using the relinearization key to reduce the three ciphertext polynomials to two ciphertext polynomials, restoring the standard ciphertext data structure. The fine-tuning terminal reads the first product ciphertext polynomial. Second product ciphertext polynomial and the third product ciphertext polynomial The fine-tuning terminal simultaneously reads the relinearized key from the homomorphic encryption global parameter set. , It is a pair of polynomials, consisting of two polynomials, and can be written as , Each coefficient is an integer. The ciphertext dimension reduction operation is performed on the polynomial ring. Execute on. First-level linearized ciphertext polynomial The calculation formula can be Second-level linearized ciphertext polynomial The calculation formula can be The polynomial multiplication and addition in the formula both occur within the polynomial ring. The above is executed. The physical basis of the ciphertext dimension reduction operation is that ciphertext field multiplication produces three ciphertext polynomials, but the standard ciphertext data structure only contains two polynomials. If three polynomials are directly used for subsequent operations, the size of the ciphertext data will increase exponentially with the number of multiplications. Relinearization key. It is auxiliary data generated in advance during the key generation stage, through and The multiplication and addition operation will multiply the third product ciphertext polynomial. The information contained within is redistributed into the first and second linearized ciphertext polynomials, thereby reducing the ciphertext dimension without losing plaintext information. The fine-tuning terminal can then use the calculated first linearized ciphertext polynomial... Second linearized ciphertext polynomial Stored in local cache.

[0127] S64. Based on the scaling factor, perform a rescaling operation on the first-level linearized ciphertext polynomial and the second-level linearized ciphertext polynomial to obtain the product first ciphertext polynomial and the product second ciphertext polynomial.

[0128] Optionally, the fine-tuning terminal can perform rescaling operations on the first and second linearized ciphertext polynomials based on a scaling factor to obtain a product first ciphertext polynomial and a product second ciphertext polynomial. The rescaling operation involves dividing each coefficient of the ciphertext polynomial by the scaling factor and rounding it to the nearest integer to eliminate the squared effect of the scaling factor introduced by ciphertext multiplication. The fine-tuning terminal reads the first linearized ciphertext polynomial. Second linearized ciphertext polynomial The fine-tuning terminal simultaneously reads the scaling factor from the homomorphic encryption global parameter set. First-order ciphertext polynomial of the product The calculation formula can be ,in, This indicates that each coefficient of the polynomial is rounded to the nearest integer. The product is a second-ciphertext polynomial. The calculation formula can be The physical basis of rescaling lies in multiplying by a scaling factor. Encoding the elements of a floating-point matrix as integers, ciphertext-field polynomial multiplication involves each of the two ciphertext polynomials carrying a scaling factor during the multiplication process. The scaling factor implicit in the multiplication result becomes If the scaling factor is not processed, it will grow exponentially with the number of multiplications, eventually causing the numerical range of the ciphertext polynomial coefficients to exceed the ciphertext modulus. The capacity. The rescaling operation is performed by dividing the coefficients of the ciphertext polynomial by... And round down, changing the scaling factor from Restore to This ensures that the scaling factor of the ciphertext remains at the same level as that of the plaintext during encoding. The units of the matrix are consistent with the units of the matrix elements, therefore The result is a dimensionless number, rounded to the nearest integer. and All coefficients are integers. The fine-tuning terminal can calculate the first encrypted polynomial of the product. Sum of the second encrypted polynomial Stored in local cache.

[0129] S65. Combine the first ciphertext polynomial and the second ciphertext polynomial to obtain the ciphertext data of the increment matrix.

[0130] Optionally, the fine-tuning terminal can combine the first and second ciphertext polynomials of the product to obtain the incremental matrix ciphertext data. Combination refers to encapsulating the two ciphertext polynomials into a single incremental matrix ciphertext data unit according to the ciphertext data structure. The fine-tuning terminal reads the first ciphertext polynomial of the product. Sum of the second encrypted polynomial Incremental matrix encrypted data It is an ordered pair of polynomials, consisting of two ciphertext polynomials. The expression can be . The first element in the product is the first ciphertext polynomial. The second element is the product of the second ciphertext polynomial. . The physical meaning is the incremental weight matrix. The corresponding encrypted data, this incremental matrix encrypted data, can be directly used for parameter fine-tuning of large models. In the encrypted domain, the fine-tuning terminal will use the incremental matrix encrypted data... The encrypted data is added to the pre-trained weight matrix to update the encrypted domain of the large model weights. Since all operations are performed within the encrypted domain, the weight parameters of the large model and intermediate data during fine-tuning remain encrypted throughout the process. Third parties cannot obtain any plaintext information from the encrypted data, thus achieving end-to-end confidentiality protection for vertical domain fine-tuning of the large model. The fine-tuning terminal outputs the encrypted incremental matrix data. The incremental matrix ciphertext data can be directly applied to the subsequent large model ciphertext domain parameter update process.

[0131] The aforementioned method for confidential fine-tuning of large models for vertical domain data mining decomposes the weight matrix of the large model into low-rank forms of the output and input matrices. Differential noise budget allocation and homomorphic encryption are then applied to these two matrices. Matrix multiplication is then performed directly in the ciphertext domain to obtain encrypted incremental weights, enabling parameter fine-tuning of the large model without decrypting any intermediate data. By pre-calculating the noise amplification coefficient of each matrix in the ciphertext multiplication based on the matrix norm, and accurately deducting its consumption from the total noise budget accordingly, an independent and matched noise budget is allocated to each matrix. This ensures that the amount of noise injected during encryption meets security requirements without excessively consuming the ciphertext modulus capacity, thereby reducing the ciphertext modulus size and computational overhead required for homomorphic encryption. This method achieves end-to-end confidentiality protection during the fine-tuning process, effectively resisting the risk of data and model theft in cloud environments. Furthermore, since each fine-tuning only processes low-rank matrices instead of the full weights, the computational load and storage consumption in the ciphertext domain are reduced while maintaining model performance, improving the security and execution efficiency of large model fine-tuning tasks in outsourced computing scenarios.

[0132] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0133] Based on the same inventive concept, this application also provides a large model confidentiality fine-tuning system for vertical domain data mining, which implements the above-described method for fine-tuning large models for vertical domain data mining. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the large model confidentiality fine-tuning system for vertical domain data mining provided below can be found in the limitations of the large model confidentiality fine-tuning method for vertical domain data mining described above, and will not be repeated here.

[0134] In one exemplary embodiment, such as Figure 2 As shown, a schematic diagram of the structure of a large model confidentiality fine-tuning system 10 for vertical domain data mining is provided, including:

[0135] The parameter acquisition module 11 is used to acquire the low-rank decomposition parameters of the weight matrix to be fine-tuned in the large model and the homomorphic encryption global parameter set; wherein, the low-rank decomposition parameters include the output side matrix, the input side matrix and the rank;

[0136] The noise amplification factor calculation module 12 is used to calculate the first ciphertext multiplication noise amplification factor of the output matrix and the second ciphertext multiplication noise amplification factor of the input matrix;

[0137] The noise budget allocation module 13 is used to obtain the first noise budget value of the output matrix based on the first ciphertext multiplication noise amplification factor, and to obtain the second noise budget value of the input matrix based on the second ciphertext multiplication noise amplification factor.

[0138] The output-side matrix encryption module 14 is used to perform homomorphic encryption on the output-side matrix based on the first noise budget value and the homomorphic encryption global parameter set to obtain the output-side ciphertext matrix data.

[0139] The input-side matrix encryption module 15 is used to perform homomorphic encryption on the input-side matrix based on the second noise budget value and the homomorphic encryption global parameter set to obtain the input-side ciphertext matrix data.

[0140] The ciphertext multiplication module 16 is used to perform ciphertext domain multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain incremental matrix ciphertext data; wherein, the incremental matrix ciphertext data is used to perform parameter fine-tuning on the large model in the ciphertext domain.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the large model confidentiality fine-tuning method for vertical domain data mining as described above.

[0142] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0143] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0144] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for fine-tuning large models for data mining in vertical domains, characterized in that, The method includes: S1. Obtain the low-rank decomposition parameters and homomorphic encryption global parameter set of the weight matrix to be fine-tuned in the large model; wherein, the low-rank decomposition parameters include the output matrix, the input matrix, and the rank; S2. Calculate the first ciphertext multiplication noise amplification factor of the output matrix and the second ciphertext multiplication noise amplification factor of the input matrix; S3. Based on the first ciphertext multiplication noise amplification factor, obtain the first noise budget value of the output-side matrix; based on the second ciphertext multiplication noise amplification factor, obtain the second noise budget value of the input-side matrix; S4. Based on the first noise budget value and the homomorphic encryption global parameter set, homomorphic encryption is performed on the output side matrix to obtain the output side ciphertext matrix data; S5. Based on the second noise budget value and the homomorphic encryption global parameter set, homomorphic encryption is performed on the input side matrix to obtain the input side ciphertext matrix data; S6. Perform ciphertext domain multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain incremental matrix ciphertext data; wherein, the incremental matrix ciphertext data is used to perform parameter fine-tuning on the large model in the ciphertext domain.

2. The method according to claim 1, characterized in that, The homomorphic encryption global parameter set includes the number of ciphertext multiplications, and S2 includes: S21. Perform row vector infinite norm operation on the output matrix to obtain the first matrix norm value; S22. Perform column vector infinite norm operation on the input matrix to obtain the second matrix norm value; S23. Multiply the first matrix norm value and the number of ciphertext multiplications to obtain the first initial amplification factor; S24. Multiply the norm of the second matrix and the number of ciphertext multiplications to obtain the second initial amplification factor; S25. The first initial amplification factor and the second initial amplification factor are standardized and adjusted to obtain the first ciphertext multiplication noise amplification factor and the second ciphertext multiplication noise amplification factor.

3. The method according to claim 2, characterized in that, S3 includes: S31. Extract the ciphertext modulus parameters from the homomorphic encryption global parameter set to obtain the initial noise budget total value; S32. Perform a binary logarithmic floor operation on the first ciphertext multiplication noise amplification coefficient to obtain the first noise consumption value; S33. Perform a binary logarithmic floor operation on the second ciphertext multiplication noise amplification coefficient to obtain the second noise consumption value; S34. Perform a difference calculation on the initial noise budget total value and the first noise consumption value to obtain the first noise budget value of the output side matrix; S35. Perform a difference operation on the initial noise budget total value and the second noise consumption value to obtain the second noise budget value of the input side matrix.

4. The method according to claim 1, characterized in that, The homomorphic encryption global parameter set also includes a scaling factor and a public key, and S4 includes: S41. Based on the scaling factor, perform scaling and rounding operations on the output-side matrix to obtain an integer-based output-side matrix; S42. Perform polynomial encoding on the integerized output-side matrix to obtain the output-side plaintext polynomial; wherein, the expression of the output-side plaintext polynomial is: ； in, For the plaintext polynomial on the output side, Let the degree be the polynomial in the global parameter set of the homomorphic encryption. This is the index of the polynomial coefficients. The integerized output matrix, after being expanded in row-major order, corresponds to the first... The element value at each position; S43. Based on the public key, under the constraint of the first noise budget value, perform central binomial distribution sampling operation on the coefficients of the first random polynomial and the coefficients of the first error polynomial to obtain the first random polynomial and the first error polynomial. S44. Perform ciphertext generation operation on the output-side plaintext polynomial, the first random polynomial, the first error polynomial and the public key to obtain the output-side first ciphertext polynomial and the output-side second ciphertext polynomial. S45. Combine the first ciphertext polynomial on the output side and the second ciphertext polynomial on the output side to obtain the ciphertext matrix data on the output side.

5. The method according to claim 4, characterized in that, S5 includes: S51. Based on the scaling factor, perform scaling and rounding operations on the input-side matrix to obtain an integerized input-side matrix; S52. Perform polynomial encoding on the integerized input-side matrix to obtain the input-side plaintext polynomial; S53. Based on the public key, under the constraint of the second noise budget value, perform central binomial distribution sampling operation on the coefficients of the second random polynomial and the coefficients of the second error polynomial to obtain the second random polynomial and the second error polynomial. S54. Perform ciphertext generation operation on the input-side plaintext polynomial, the second random polynomial, the second error polynomial, and the public key in the homomorphic encryption global parameter set to obtain the input-side first ciphertext polynomial and the input-side second ciphertext polynomial. S55. Combine the first ciphertext polynomial on the input side and the second ciphertext polynomial on the input side to obtain the ciphertext matrix data on the input side.

6. The method according to claim 5, characterized in that, The homomorphic encryption global parameter set also includes a relinearized key, and S6 includes: S61. The first ciphertext polynomial and the second ciphertext polynomial in the output-side ciphertext matrix data are parsed to obtain a pair of output-side ciphertext polynomials; the first ciphertext polynomial and the second ciphertext polynomial in the input-side ciphertext matrix data are parsed to obtain a pair of input-side ciphertext polynomials. S62. Perform ciphertext-domain polynomial multiplication on the output-side ciphertext polynomial and the input-side ciphertext polynomial over a preset polynomial ring to obtain a first product ciphertext polynomial, a second product ciphertext polynomial, and a third product ciphertext polynomial, wherein the expression for the polynomial ring is: ； in, Represents a polynomial ring. The ciphertext modulus is given in the homomorphic encryption global parameter set. Let the degree be the polynomial in the global parameter set of the homomorphic encryption. Represents the modulus of an integer The remaining class ring, For variables of a polynomial; S63. Based on the relinearization key, perform ciphertext dimension reduction operation on the first product ciphertext polynomial, the second product ciphertext polynomial, and the third product ciphertext polynomial to obtain the first relinearized ciphertext polynomial and the second relinearized ciphertext polynomial. S64. Based on the scaling factor, perform a rescaling operation on the first linearized ciphertext polynomial and the second linearized ciphertext polynomial to obtain a product first ciphertext polynomial and a product second ciphertext polynomial. S65. Combine the first ciphertext polynomial of the product and the second ciphertext polynomial of the product to obtain the ciphertext data of the incremental matrix.

7. A large-scale model fine-tuning system for vertical domain data mining, characterized in that, The system includes: The parameter acquisition module is used to acquire the low-rank decomposition parameters of the weight matrix to be fine-tuned in the large model and the homomorphic encrypted global parameter set; wherein, the low-rank decomposition parameters include the output side matrix, the input side matrix, and the rank; The noise amplification factor calculation module is used to calculate the first ciphertext multiplication noise amplification factor of the output-side matrix and the second ciphertext multiplication noise amplification factor of the input-side matrix; The noise budget allocation module is used to obtain a first noise budget value of the output-side matrix based on the first ciphertext multiplication noise amplification factor; and to obtain a second noise budget value of the input-side matrix based on the second ciphertext multiplication noise amplification factor. The output-side matrix encryption module is used to perform homomorphic encryption on the output-side matrix based on the first noise budget value and the homomorphic encryption global parameter set to obtain output-side ciphertext matrix data. The input-side matrix encryption module is used to perform homomorphic encryption on the input-side matrix based on the second noise budget value and the homomorphic encryption global parameter set to obtain input-side ciphertext matrix data. The ciphertext multiplication module is used to perform ciphertext domain multiplication on the output-side ciphertext matrix data and the input-side ciphertext matrix data to obtain incremental matrix ciphertext data; wherein, the incremental matrix ciphertext data is used to perform parameter fine-tuning on the large model in the ciphertext domain.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images