Compressed local differential privacy protected logistic regression method

Abstract

The application discloses a kind of logic regression methods based on compression local differential privacy protection.In the method, user calculates gradient vector according to the model parameter released by server, and obtains input vector by encoding processing;By random response mechanism disturbance, output vector is obtained, and privacy protection is realized.Server aggregates and corrects output vector, obtains unbiased mean and can be used to update model parameter, and releases update parameter to the user participating in training in next round.After multiple iterations, logic regression model is trained.Finally, the unknown class label of user data can be classified and predicted by using the logic regression model.The application introduces compressed local differential privacy model, which improves the utility and estimation accuracy of data statistics while protecting user privacy, balances privacy protection and data availability, provides classification prediction for users while ensuring that attackers cannot reverse infer individual data in training data, and has high classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of information security, and in particular relates to a logistic regression method based on compression and localized differential privacy protection. Background Technology

[0002] The explosive growth of data has spurred the development of data mining techniques. Internet-connected devices generate massive amounts of data daily, and analyzing and mining this data can effectively improve user experience and service quality. However, when devices handling sensitive personal information connect to the internet and provide access to their data to other devices, there is a risk of privacy breaches, which could cause incalculable losses to users. Therefore, while mining and analyzing data, it is essential to provide privacy protection for personal data.

[0003] Differential privacy (DP) is a rigorously proven privacy protection model with a solid mathematical foundation. Currently, differential privacy is widely used in machine learning, deep neural networks, federated learning, and other fields. Traditional differential privacy models, also known as centralized differential privacy models, typically involve a third-party data collector collecting user data and then releasing privacy-enhanced datasets or related statistical information using differential privacy techniques. However, centralized differential privacy relies on trusted data collectors; therefore, when an untrusted data collector collects data, the privacy of each participant cannot be guaranteed. With the advent of Local Differential Privacy (LDP), LDP inherits the quantitative privacy protection feature of DP, shifting the privacy-enhancing process from the third-party collector to the user's terminal, eliminating the privacy leakage threat that may arise from untrusted third-party servers. Given that satisfying the perturbation mechanism of LDP typically requires assuming a user base on a large dataset of millions, the concept of Compressed Local Differential Privacy (CLDP) was proposed. CLDP introduces a distance metric to construct the indistinguishability between data points, scaling privacy based on the distance (i.e., similarity) between data points. Compared to the LDP model, the CLDP model provides better statistical utility for small datasets or datasets with many dimensions.

[0004] Currently, logistic regression algorithms based on localized differential privacy (LDP) typically require the introduction of a mean estimation algorithm for multidimensional numerical data that satisfies the LDP, combined with gradient descent. However, existing LDP-compliant multidimensional numerical data mean estimation algorithms often address the multidimensionality problem by splitting the privacy budget across different dimensions or randomly sampling data from a single dimension instead of all dimensions. This results in low statistical utility for the algorithm. Furthermore, such mean estimation algorithms usually assume large datasets with millions of users, leading to low estimation accuracy when applied to small datasets. Summary of the Invention

[0005] Objective: The objective of this invention is to provide a compressed local differential privacy-preserving logistic regression method. Addressing the shortcomings of existing local differential privacy-preserving logistic regression methods, this invention introduces a compressed local differential privacy model (CLDP). The CLDP model offers higher statistical utility when dealing with data with high dimensionality and small datasets. Furthermore, this invention eliminates the need for sampling when processing multidimensional gradient vectors, thus effectively avoiding estimation errors introduced by sampling.

[0006] Technical solution: The localized differential privacy-preserving logistic regression method based on compression of the present invention achieves privacy protection by performing the following steps: the user perturbs the gradient vector to achieve privacy protection; the server aggregates the perturbed output vector to restore the mean of all user input vectors and uses the mean to update the parameters of the logistic regression iterative update model; the user uses the decision boundary of the hypothesis function based on the model to predict their label.

[0007] S1: During the training phase, the server initializes the logistic regression model parameters, sets the privacy budget value α, and exposes the initial model parameters and privacy budget value to the user.

[0008] The server is an untrusted entity responsible for aggregating user gradients and calculating model parameters.

[0009] The user group consists of n users and possesses the training data required to participate in the logistic regression model.

[0010] S2: User i Based on the model parameters provided by the server, the d+1 dimensional numerical gradient vector is calculated.

[0011] S3: User i On the user's side, its gradient vector Encoding the input vector s to d+1 dimensions (i) ;

[0012] S4: User iFor input vector s (i) To perform a perturbation, utilize the perturbation mechanism M CLDP-ME Input vector s (i) Perturb to output vector t (i) This allows it to satisfy local differential privacy with alpha compression;

[0013] S5: User i The perturbated output vector t (i) Send to the server;

[0014] S6: The server calculates the output vector t sent by all users. (i) Perform statistical analysis to obtain the mean of the n input vectors.

[0015] S7: The server inputs the obtained mean result into the logistic regression iterative equation to further update the model parameters θ. Repeat steps S2-S7 until the model converges and the iteration ends.

[0016] S8: Finally, we obtain the model parameters θ, which is the logistic regression classifier model. For users with unknown class labels, we substitute θ and its attributes into the hypothesis function of logistic regression, and the user can use the decision boundary of the hypothesis function to predict their label.

[0017] Furthermore, in step S1, the server initializes the logistic regression model parameters θ. Here, θ is a d+1 dimensional vector. Let represent a d+1 dimensional vector of real numbers, where each element is 0. There are n users {User1,...,User...}. i ,...,User n}, each user User has a privacy record (i) =(x (i) ,y (i) ), which includes d-dimensional numerical attributes. Class tag y (i) ∈{0,1}. The server sends the initial logistic regression model parameters θ and privacy budget α to n users {User1,...,User2}. i ,...,User n}

[0018] Furthermore, step S2 specifically involves calculating the gradient of logistic regression. The hypothesis function is: Each user i At the user end, a gradient vector of d+1 dimensions can be obtained. Where i∈[1,n], j∈[0,d].

[0019] Furthermore, step S3 specifically involves: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error Normalization makes right The input vector is obtained after discretization. Make any The discretization method is as follows:

[0020]

[0021] in, User i gradient vector The value at position j.

[0022] Furthermore, step S4 specifically includes the following steps:

[0023] S41: For a vector of dimension d+1, divide the total sample into d+2 sample subspaces. The size of the sample subspace with similarity k is... Calculated to As a normalization factor;

[0024] S42: By simplifying the normalization factor according to the binomial theorem, we get:

[0025] S43: Define the perturbation mechanism M CLDP-ME For any input vector s (i) After perturbation mechanism M CLDP-ME The output vector is then obtained. The probabilities are as follows:

[0026]

[0027] Where: Pr[·] represents the probability distribution value, α is the privacy budget under the CLDP model, and u(s) (i) ,t (i) Let u(s) be the utility function. (i) ,t (i) )=d+1-d(s (i) ,t (i) ), used to define the input vector s (i) and output vector t (i) The similarity between them can be expressed by the function u(s). (i) ,t (i) )∈[0,d+1]. Where, the distance function d(s) (i) ,t (i) Using Hamming distance, i.e., defined The XOR operation reflects the degree of dissimilarity between two vectors, thus providing a basis for the similarity between vectors and satisfying the nonnegativity, identity, symmetry, and trigonometric inequality of the distance function. The higher the similarity between two vectors, the greater the utility value, indicating the output vector t. (i) It has a higher probability of approximating the input vector s (i) ;

[0028] S44: Further simplification of the perturbation mechanism based on the binomial theorem:

[0029]

[0030] The user terminal inputs its vector After perturbation mechanism M CLDP-ME The output vector is then obtained.

[0031] Furthermore, the user terminal inputs its vector After perturbation mechanism M CLDP-ME The output vector is then obtained. Specifically: Initialize the output vector: Generate a uniform random variable r∈[0.0,1.0), if Let t (i) The j-th component if Let t (i) The j-th component t (i) Each component in the vector is calculated according to the above formula to obtain the final output vector.

[0032] Furthermore, step S6 specifically includes the following steps:

[0033] S41: The server collects the output vectors uploaded by n users after perturbation.

[0034] S42: Initialize the mean estimation vector

[0035] S43: Components in The calculation is as follows:

[0036]

[0037] Will Each component in the vector is calculated according to the above formula to obtain the final mean estimate vector.

[0038] Furthermore, step S7 specifically includes the following steps:

[0039] S71: Obtain the mean unbiased estimate of the output vector for the current round based on step S6. The model update is implemented using the following formula:

[0040]

[0041] Where η is the learning rate.

[0042] S72: The updated model parameters θ are sent to n users {User1,...,User...} i ,...,User n}

[0043] Furthermore, the specific steps of step S8 are as follows:

[0044] S81: The server distributes the logistic regression classifier model parameters θ to the users to be classified;

[0045] S82: For users with unknown class labels k User privacy records containing only d-dimensional attribute data (k) =(x (k) ),have Define x (k)′ =[1,x (k) ], thus obtaining a d+1 dimensional attribute vector x (k)′ ,Right now

[0046] S83: For attribute vector x (k)′ Perform normalization processing;

[0047] S84: Combine θ and x (k)′ Substitute into the hypothesis function of logistic regression: in,

[0048] S85: Users can utilize the hypothesis function h θ (x (k)′ Predict its label when h θ (x (i)′ When h ≥ 0.5, the result is classified into category 1; when h θ (x (i)′ When ) < 0.5, the result is classified as class 0.

[0049] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0050] 1. This invention designs a localized differential privacy-preserving logistic regression method based on compression. In this method, a perturbation mechanism M is defined. CLDP-MEA compressed, localized differential privacy-preserving model is introduced, making the output value closer to the original value with a higher probability. This improves the statistical utility on multidimensional datasets while protecting user privacy. The perturbation mechanism M... CLDP-ME It also exhibits higher estimation accuracy on small datasets.

[0051] 2. Under the protection of compressed localized differential privacy, user privacy data can be effectively protected even if an attacker possesses all background knowledge except for the target privacy information.

[0052] 3. The logistic regression method based on compression and localized differential privacy protection of the present invention can achieve higher classification accuracy than existing methods under the same level of privacy protection, and has better practical value. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the logistic regression method based on compression and localized differential privacy protection according to the present invention.

[0054] Figure 2 This is a performance comparison diagram of the present invention. Figure 1 .

[0055] Figure 3 This is a performance comparison diagram of the present invention. Figure 2 .

[0056] Figure 4 This is a performance comparison diagram of the present invention. Figure 3 .

[0057] Figure 5 This is a performance comparison diagram of the present invention. Figure 4 . Detailed Implementation

[0058] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, the overall implementation steps of the compression-based localized differential privacy-preserving logistic regression method in this embodiment are as follows:

[0060] S1: During the training phase, the server initializes the logistic regression model parameters, sets the privacy budget value α, and exposes the initial model parameters and privacy budget value to the user.

[0061] The server is an untrusted entity responsible for aggregating user gradients and calculating model parameters.

[0062] The user group consists of n users and possesses the training data required to participate in the logistic regression model.

[0063] S2: Useri Based on the model parameters provided by the server, the d+1 dimensional numerical gradient vector is calculated.

[0064] S3: The user sets the gradient vector on the user's end. Encoding the input vector s to d+1 dimensions (i) .

[0065] S4: User i For input vector s (i) To perform a perturbation, utilize the perturbation mechanism M CLDP-ME Input vector s (i) Perturb to output vector t (i) This allows it to satisfy local differential privacy with alpha compression.

[0066] S5: The user will perturb the output vector t (i) Send to the server.

[0067] S6: The server performs statistical analysis on the perturbation data sent by all users to restore the mean of these n input vectors as much as possible.

[0068] S7: The server inputs the obtained mean result into the iterative formula of logistic regression to further update the model parameters θ.

[0069] Repeat steps S2-S7 until the model converges, at which point the iteration ends.

[0070] S8: Finally, we obtain the model parameters θ, which is the logistic regression classifier model. For users with unknown class labels, we substitute θ and its attributes into the hypothesis function of logistic regression, and the user can use the decision boundary of the hypothesis function to predict their label.

[0071] In step (S2) of this method, the parameterization of the logistic regression model parameters by the server includes the following process:

[0072] The server initializes the logistic regression model parameters θ. Here, θ is a d+1 dimensional vector. Let represent a d+1 dimensional vector of real numbers, where each element is 0. There are n users {User1,...,User...}. i ,...,User n Each user i User has a privacy record (i) =(x (i) ,y (i) ), which includes d-dimensional numerical attributes. Class tag y (i)∈{0,1}. The server sends the initial logistic regression model parameters θ and privacy budget α to n users {User1,...,User2}. i ,...,User n}

[0073] In step (S2) of this method, the user terminal calculates the gradient vector for the original data, including the following process:

[0074] According to the gradient calculation formula of logistic regression The hypothesis function is: Each user i At the user end, a gradient vector of d+1 dimensions can be obtained. Where i∈[1,n], j∈[0,d].

[0075] In step (S3) of this method, the encoding of the gradient vector by the user end includes the following process:

[0076] S31: Yes Normalization makes For any dimension The normalization method is as follows:

[0077]

[0078] in, This represents the maximum value appearing at the j-th position in the original gradient vectors of all users who participated in the training. This represents the minimum value appearing in the j-th position of the original gradient vector of all users participating in the training.

[0079] S32: Yes The input vector is obtained after discretization. Make any The discretization method is as follows:

[0080]

[0081] in, User i gradient vector The value at position j.

[0082] because It can be seen that the discretization process ensures the unbiasedness of the data.

[0083] In step (S4) of the method, the disturbance mechanism M CLDP-ME The specific steps are as follows:

[0084] S41: The total sample can be divided into d+2 sample subspaces, and the size of the sample subspace with similarity k is... Therefore, it can be calculated to As a normalization factor.

[0085] S42: According to the binomial theorem, the normalization factor can be simplified to obtain:

[0086] S43: Define the perturbation mechanism M CLDP-ME For any input vector s (i) After perturbation mechanism M CLDP-ME The output vector is then obtained. The probabilities are as follows:

[0087]

[0088] Where: Pr[·] represents the probability distribution value, and α is the privacy budget under the CLDP model. (i) ,t (i) Let u(s) be the utility function. (i) ,t (i) )=d+1-d(s (i) ,t (i) ), used to define the input vector s (i) and output vector t (i) The similarity between them can be effectively expressed by the function u(s). (i) ,t (i) )∈[0,d+1]. Where the distance function d(s) (i) ,t (i) Using Hamming distance, i.e., defined The XOR operation reflects the degree of dissimilarity between two vectors, thus providing a basis for the similarity between them and satisfying the nonnegativity, identity, symmetry, and trigonometric inequality of the distance function. The higher the similarity between two vectors, the greater the utility value, indicating the output vector t. (i) It has a higher probability of approximating the input vector s (i) .

[0089] S44: The perturbation mechanism M can be further simplified according to the binomial theorem. CLDP-ME :

[0090]

[0091] The user terminal inputs its vector After perturbation mechanism M CLDP-ME The output vector is then obtained. The specific steps are as follows:

[0092] 1) Initialize the output vector:

[0093] 2) Generate a uniform random variable r∈[0.0,1.0).

[0094] 3) If Let t (i) The j-th component if Let t (i) The j-th component

[0095] t (i) Each component in the vector is calculated according to the above formula to obtain the final output vector.

[0096] In step (S6) of this method, the specific steps for server-side mean estimation are as follows:

[0097] S61: The server collects the output vectors uploaded by n users after perturbation.

[0098] S62: Initialize the mean estimation vector

[0099] S63: Components in The calculation is as follows:

[0100]

[0101] Will Each component in the vector is calculated according to the above formula to obtain the final mean estimate vector.

[0102] In step (S7) of this method, the specific steps for updating the model parameters on the server side are as follows:

[0103] S71: Obtain the mean unbiased estimate of the output vector for the current round based on step (S6). The model update is implemented using the following formula:

[0104]

[0105] Where η is the learning rate.

[0106] S72: The updated model parameters θ are sent to n users {User1,...,User...} i ,...,User n}

[0107] In step (S8) of this method, the specific steps for the user terminal to classify and predict user attribute data with unknown class labels are as follows:

[0108] S81: The server distributes the logistic regression classifier model parameters θ to the users to be classified.

[0109] S82: For users with unknown class labels k User privacy records containing only d-dimensional attribute data (k) =(x (k) ),have Define x (k)′ =[1,x (k) ], thus obtaining a d+1 dimensional attribute vector x (k)′ ,Right now

[0110] S83: For attribute vector x (k)′ Maintain the normalization process consistent with step (S31).

[0111] S84: Combine θ and x (k)′ Substitute into the hypothesis function of logistic regression: in,

[0112] S85: Users can utilize the hypothesis function h θ (x (k)′ Predict its label. When h θ (x (i)′ When h ≥ 0.5, the result is classified into category 1; when h θ (x (i)′ When ) < 0.5, the result is classified as class 0.

[0113] The above describes the specific processing of the compression-based localized differential privacy-preserving logistic regression method. To illustrate the perturbation mechanism M defined in this application... CLDP-ME The local differential privacy that can be compressed is provided below, along with a theoretical proof.

[0114] I. Disturbance Mechanism M CLDP-ME Local differential privacy that satisfies alpha compression.

[0115] Proof: For any input vector s of length d+1 (i) and s′ (i) Through the perturbation mechanism M CLDP-ME (*) outputs t (i) have:

[0116]

[0117] Because the distance function d(s) (i) ,t (i) Since the triangle property is satisfied, therefore:

[0118] d(s′ (i) ,t (i) )-d(s (i) ,t(i) )≤d(s′ (i) ,s (i) )

[0119] Therefore:

[0120]

[0121] According to the definition of local differential privacy in α-compression, Theorem 1 holds. Q.E.D.

[0122] To illustrate the perturbation mechanism M in this application CLDP-ME The availability of the mean estimate is demonstrated below, along with proofs of its unbiasedness and estimation error.

[0123] II. Let the true mean vector and the estimated mean vector be represented as z = [z0,...,z...]. j ,...,z d ]and For any z j Let j be the true mean of the j-th component. For the unbiased mean of the j-th component, is there an unbiased estimate? Established.

[0124] Proof: According to the perturbation mechanism M CLDP-ME have:

[0125]

[0126] Therefore, the expected value of the mean can be estimated as follows:

[0127]

[0128] Q.E.D.

[0129] III. For set up z is an unbiased estimate of the mean. j The true mean, i.e. Algorithm estimation error: It holds with a probability of 1-β. Where j∈[0,d], n is the number of users, and α is the privacy budget.

[0130] Proof: According to Hofding's inequality, we have the upper probability limit of the deviation between the mean and the expected value of a random variable: Where S is n independent random variables x j The mean, x j ∈[a j ,b j ]. Because z j The mean of n independent random variables, each with a value of 1 or -1, can be obtained as follows:

[0131]

[0132] Where b j =1,a j =-1, therefore:

[0133]

[0134] because Therefore:

[0135]

[0136]

[0137] set up Will Substituting into the above equation, we get:

[0138]

[0139]

[0140] but It holds true with at least a probability of 1-β. Q.E.D.

[0141] The following is the perturbation mechanism M defined in the compressed, localized differential privacy-preserving logistic regression method. CLDP-ME The experimental results are presented below. The experimental environment consisted of an Intel(R) Core(TM) i7-4770HQ, 2.20GHz, 16GB of RAM, and a Windows 10 operating system. Python was used as the programming language.

[0142] To verify the practicality of this perturbation mechanism, the Mean Square Error (MSE) was used to measure the mean estimation accuracy of this mechanism and the most representative mean estimation algorithms, Harmony, PM, and three-outputs. The Mean Square Error (MSE) is: T represents the number of runs, and z represents the true mean. The mean of the estimated MSE is denoted as MSE. A larger MSE indicates more noise introduced, resulting in lower data usability. To mitigate the randomness of the algorithms and eliminate the influence of errors, each algorithm is run 50 times on the dataset, and the mean of the MSE is taken.

[0143] To ensure the reliability of the results, the experiment used three synthetic datasets and one real dataset for simulation. Each synthetic dataset contained 50,000 records, and each record consisted of six attributes, satisfying the following conditions:

[0144] 1) Uniform datasets that follow a uniform distribution;

[0145] 2) The Normal-1 dataset follows a normal distribution with a mean of 0 and a standard deviation of 1;

[0146] 3) The Normal-2 dataset follows a normal distribution with a mean of 1 and a standard deviation of 2.

[0147] The real dataset used is the Adult dataset from the UCI Machine Learning Library, which uses 6 numerical attributes and normalizes them.

[0148] Figure 2 The effect of changes in the privacy budget α or ε on the MSE of four algorithms is shown. When the privacy budget α or ε increases from 0.1 to 2, the degree of privacy protection decreases, the error in the data collected by the third-party server decreases, and therefore the MSE of all four algorithms decreases. Under the same privacy budget, the perturbation mechanism M... CLDP-ME Its accuracy is superior to Harmony, PM, and three-outputs, due to the perturbation mechanism M. CLDP-ME The CLDP model was introduced, which, compared to LDP, incorporates a distance metric, making the output value closer to the true value with a higher probability, thereby improving the utility of the data in mean estimation.

[0149] Figure 3 This demonstrates the relationship between MSE and the number of users (dataset size). To study the impact of user number on MSE, the Uniform dataset was used for mean estimation, with the privacy budget α or ε set to 1, the number of attributes set to 6, and the number of records n in the dataset set to:

[0150] n={50000,40000,30000,20000,10000,5000,1000,500}. Figure 3 This demonstrates that MSE decreases as the number of users in the dataset increases; the more user samples collected by a third party, the more accurate the unbiased estimation of the original data. (Perturbation mechanism M) CLDP-ME On datasets with varying numbers of users, the MSE value is lower than that of Harmony, PM, and three-outputs. Experimental results demonstrate the effectiveness of the perturbation mechanism M. CLDP-ME Compared to existing LDP multidimensional numerical data mean estimation algorithms, it has higher estimation accuracy on small datasets.

[0151] Figure 4This demonstrates the relationship between MSE and attribute dimension (data dimension). To study the impact of the number of attributes on MSE, the Uniform dataset was used for mean estimation, with the privacy budget α or ε set to 1, the number of users set to 50,000, and the attribute dimension of the dataset set to d = {1, 5, 10, 15, 20}. Figure 4 This demonstrates that the MSE of the Harmony, PM, and three-outputs methods increases with the attribute dimension d, because the upper bound error of the algorithm is positively correlated with the attribute dimension. The perturbation mechanism M... CLDP-ME The error is not related to the dimension and does not change with the increase of the attribute dimension, so it is more suitable for situations with high attribute dimension.

[0152] Below are the experimental results of a compressed, localized, differentially privacy-preserving logistic regression method. The experimental environment consisted of an Intel(R) Core(TM) i7-4770HQ, 2.20GHz, 16GB RAM, and Windows 10 operating system. Python was used as the programming language. Kaggle's 2019 Airline Delays and Google Merchandise Sale Prediction datasets were employed. The 2019 Airline Delays data, sourced from the U.S. Bureau of Transportation Statistics, predicted whether flights would be delayed based on airport and weather conditions. Google Merchandise Sale Prediction, derived from Google's BigQuery data warehouse, predicted whether each session would lead to a visitor adding items to their shopping cart.

[0153] The experiment used the accuracy F of the classification prediction results. acc To measure the utility of this invention in classification tasks, i.e., F acc = Total number of correctly predicted data points / Total number of data points in the test set. A five-fold cross-validation method is used, dividing the dataset into five mutually exclusive subsets. The training and testing processes for the logistic regression classifier are each run five times, maintaining consistency in subset distribution through stratified sampling. Four subsets are selected alternately as the training set, and the remaining subsets are used as the test set. Considering the randomness of the method, 50 experiments are performed for each dataset during training and testing, and then the F-value is taken. acc The mean of the values ​​is used as the final classification accuracy.

[0154] Figure 5The experiment demonstrates the classification accuracy of each method on two datasets with different privacy budgets, where the privacy budget α or ε takes values ​​of 0.1, 0.2, 0.5, 1, 1.5, and 2. For the same dataset, as the privacy budget increases, the degree of privacy protection decreases, and the classification accuracy gradually increases. With the same number of iterations and the same privacy budget, the method of this invention achieves higher classification accuracy than Harmony, PM, and three-outputs in logistic regression tasks. This is because the perturbation gradient submitted to the server uses M... CLDP-ME Compared to Harmony, PM, and three-outputs, it has higher statistical utility, making the model training more stable.

[0155] The above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention; any substitutions and improvements made without departing from the concept of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A localized differential privacy-preserving logistic regression method based on compression, characterized in that, By performing the following steps, users encode gradient vectors into input vectors and perturb them to protect privacy. The server aggregates the perturbed output vectors to restore the mean of all user input vectors and uses this mean to iteratively update the parameters of the logistic regression model. Users then use the decision boundary of the hypothesis function based on this model to predict their labels. S1: During the training phase, the server initializes the logistic regression model parameters, sets the privacy budget value, and exposes the initial model parameters and privacy budget value to the user. The server is responsible for aggregating user gradients and calculating model parameters. Each user Having the training data required to participate in the logistic regression model, by Composed of individual users; S2: User Based on the model parameters provided by the server, the following calculations were performed: 3D numerical gradient vector In step S2, the gradient calculation formula for logistic regression is used. The hypothesis function is , Each user The user can obtain from 3D numerical gradient vector ,in ; S3: User On the user's side, its gradient vector Encoding to dimensional input vector ; Step S3 specifically includes the following steps: S31: Yes Normalization makes ; S32: Yes The input vector is obtained after discretization. , making any The discretization method is as follows: in, Indicates user gradient vector No. The value of the bit; S4: User For the input vector To perform perturbation, and utilize perturbation mechanisms input vector Perturb to output vector Make it satisfy Compressed local differential privacy; step S4 specifically includes the following steps: S41: For dimension... The vector divides the total sample into The sample subspaces of the group have a similarity of The sample subspace size is Calculated to As a normalization factor; S42: By simplifying the normalization factor according to the binomial theorem, we get: ; S43: Define the perturbation mechanism For any input vector After perturbation mechanism The output vector is then obtained. The probabilities are as follows: in: Represents the probability distribution value. For privacy budgeting under the CLDP model, For utility function, , used to define the input vector and output vector The similarity between them, a function that can be used. ; where, distance function Using Hamming distance, i.e., defined , The XOR operation reflects the degree of dissimilarity between two vectors, thus providing a basis for the similarity between them, and satisfying the nonnegativity, identity, symmetry, and trigonometric inequality of the distance function; the higher the similarity between two vectors, the greater the utility value, indicating the output vector. It has a higher probability of approximating the input vector. ; S44: Further simplification of the perturbation mechanism based on the binomial theorem: The user terminal inputs its vector After perturbation mechanism The output vector is then obtained. ; The user terminal inputs its vector After perturbation mechanism The output vector is then obtained. Specifically: Initialize the output vector: Generate a uniform random variable ,if Then let If the j-th component in the matrix is... Then let The j-th component ;Will Each component in the vector is calculated according to the above formula to obtain the final output vector; S5: User The perturbated output vector Send to the server; S6: The server bases the output vectors sent by all users. Statistical analysis was performed to obtain... The mean of the input vectors; S7: The server inputs the obtained mean result into the logistic regression iterative formula to update the model parameters. Repeat steps S2-S7 until the model converges, the iteration ends, and proceed to step 8. S8: Obtain the parameters of the model That is, a logistic regression classifier model; for users with unknown class labels, The user then substitutes the user's attributes into the hypothesis function of logistic regression, and uses the decision boundary of the hypothesis function to predict the user's label.

2. The logistic regression method based on compression and localized differential privacy protection according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11: Server Initialization Logistic Regression Model Parameters ; in, It is dimensional vector, , Represents a set of real numbers A 3D vector, where each element is 0; individual users Each user Have a privacy record , which includes 3D numerical properties Class tags ; S12: The server will initialize the logistic regression model parameters. and privacy budget Send to individual users .

3. The logistic regression method based on compression and localized differential privacy protection according to claim 1, characterized in that, Step S6 specifically includes the following steps: S61: The server collected... The output vector uploaded by each user after perturbation ; S62: Initialize the mean estimation vector ; S63: Components in The calculation is as follows: Will Each component in the vector is calculated according to the above formula to obtain the final mean estimate vector.

4. The logistic regression method based on compression and localized differential privacy protection according to claim 1, characterized in that, Step S7 specifically includes the following steps: S71: Obtain the mean unbiased estimate of the output vector for the current round based on step S6. The model is then updated, and the specific update formula is as follows: in, The learning rate; S72: Updated model parameters Send to individual users .

5. The logistic regression method based on compression and localized differential privacy protection according to claim 1, characterized in that, The specific steps of step S8 are as follows: S81: The server distributes logistic regression classifier model parameters to the users to be classified. ; S82: For users with unknown class tags , possessing only Privacy records of dimensional attribute data ,have ;definition ,get dimensional attribute vector ,Right now ; S83: For attribute vectors Perform normalization processing; S84: Will and Substitute into the hypothesis function of logistic regression: ,in, ; S85: Users can utilize hypothesis functions Predict its label, when When, the result is classified into category 1; when When the result is in the zero category, it will be classified as 0.

Images