A privacy protection method for software defect prediction based on federated learning

Abstract

The application relates to a software defect prediction privacy protection method based on federal learning, which is used for software defect prediction based on a random forest model constructed based on a decision tree, and solves the problem of privacy leakage caused by model training depending on centralized storage of enterprise private data. First, a software defect prediction model is constructed, is decomposed into data sets for preprocessing, a Bootstrap sampling strategy is adopted to create a training set for each decision tree, and a random forest model part is formed based on the decision trees; second, the model is used for prediction and training on each department in the internal, loss and local gradient are calculated; then, the local gradient is gradient clipped and local differential privacy technology is added to obtain protected gradient, which is uploaded to the server; finally, the server aggregates the gradient, updates the global model parameters and distributes them to each department. In a privacy protection manner, the application trains an accurate and unbiased software defect prediction model by using a large amount of data.

Description

Technical Field

[0001] This invention relates to a privacy-preserving method for software defect prediction based on federated learning, belonging to the technical field of privacy-oriented software defect prediction in software engineering. Background Technology

[0002] As software systems play an increasingly important role in our lives, their complexity continues to grow. This increasing complexity makes it difficult to guarantee their quality. Consequently, much recent research has focused on prioritizing software quality assurance efforts. To improve software quality, the field of software defect prediction has emerged. Its novel approach of isolating defective software units can eliminate defects and better utilize resources in software development and maintenance activities. Currently, software defect prediction is considering using high-precision search-based techniques. The practical application of software defect prediction methods has significant benefits for the software production process. For example, it can lead to more reliable software systems; improve the accuracy and speed of software testing by focusing on error-prone modules; make designs more rational and identify refactorable modules using object-oriented methods; and improve overall project quality by allocating resources based on results. Patents related to software defect prediction include: a software defect prediction model based on deep neural networks and probabilistic decision forests (publication number CN 109446090); and a class-imbalanced software defect prediction method based on data resampling (CN 113204481 B), etc.

[0003] In recent years, numerous machine learning-based methods and various software metrics have been proposed and applied to software defect prediction. For example, Kim et al. used machine learning classifiers to predict potential software defects; Jing et al. introduced dictionary learning techniques into defect prediction. In a recent study, most empirical research only used datasets provided by the Promise repository, which may threaten external validity; software defect prediction model sharing is a potential solution to mitigate this problem, encouraging researchers to share more models. However, directly sharing models may lead to privacy breaches; for example, Fredrikson et al. successfully performed model inversion attacks on decision trees for a lifestyle survey and neural networks for face recognition. Privacy and security issues are becoming increasingly important in current software defect prediction research. In practice, numerous external projects exist that often provide more information; however, the prerequisite for using this external information is the availability of the external source. Due to privacy and security concerns, most users or businesses are unwilling to share their data. To promote data sharing, it is necessary to study how to protect the privacy of data owners before they publish their data. In this invention, machine learning classifiers and model sharing are combined, utilizing a federated learning privacy protection framework and local differential privacy techniques for software defect prediction, addressing the privacy concerns of users or businesses. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a privacy-preserving method for software defect prediction based on federated learning. By leveraging the privacy-preserving framework of federated learning, information barriers are broken down, and multiple sets of data are fully utilized for training in a privacy-preserving manner, thereby effectively improving the accuracy of the software defect prediction model.

[0005] To address the aforementioned technical issues, the following technical solution is adopted: This solution protects the privacy of data from various departments while utilizing a large amount of data for model training; it includes a method for constructing a software defect prediction model, and the application of federated learning techniques to train the software defect prediction model, thereby achieving defect prediction based on the data; the method for constructing the software defect prediction model and the model training process include the following steps:

[0006] Step A. Select a group of clients consisting of various departments within the enterprise to participate in collaborative model training. For each training session, a department from the client is randomly selected. This department inputs locally stored source code data, which is then used by the software defect prediction model for prediction and training. The process then proceeds to Step B.

[0007] Step B. On the department's local device, preprocess the input source code, use the Bootstrap sampling strategy to create a training set for each decision tree, and build a software defect prediction model based on the random forest algorithm generated by the decision tree classifier, and then proceed to Step C.

[0008] Step C. Based on local departmental data, perform software defect prediction and model training to obtain the prediction result y. i Calculate the probability P1 that a sample label is predicted as 1, train to obtain the binary classification cross-entropy loss L(u), and calculate the local model gradient G based on the loss. u Then proceed to step D;

[0009] Step D. Apply the local model gradient G u Gradient clipping is performed, followed by the application of local differential privacy techniques to obtain protected gradients. Then proceed to step E;

[0010] Step E. Numerous departmental clients will protect the model gradients. The gradients of all uploaded models are uploaded to the server, where federated learning technology is used to aggregate the gradients of all uploaded models, resulting in the aggregated gradient. The global model parameters w maintained in the server are updated using aggregated gradients. The updated global model is then distributed to departmental devices to update their local model parameters. This process is repeated until model training converges.

[0011] As a preferred technical solution of the present invention: In step B, a software defect prediction model is constructed based on the random forest algorithm generated by the decision tree classifier, and the following steps B1 to B3 are performed to obtain the software defect prediction model.

[0012] Step B1. Preprocess the input source code; the preprocessing mainly includes two steps: oversampling a few instances and discretizing continuous features.

[0013] In practical applications, step B1 addresses the issues of imbalanced datasets and hidden defects in the datasets, and is performed according to steps B1.1 to B1.2 to obtain the preprocessed dataset.

[0014] Step B1.1 uses the SMOTE algorithm to oversample the minority instances. First, for each sample i in the minority class, calculate its distance to all samples in the minority class sample set using Euclidean distance to obtain its k nearest neighbors. Then, set a sampling ratio based on the sample imbalance ratio to determine the sampling multiplier N. For each minority class sample i, randomly select several samples from its k nearest neighbors, assuming the selected nearest neighbors are ik and ik. n Finally, for each randomly selected nearest neighbor i n Construct new samples according to the following formula:

[0015] i new =i+rand(0,1)*|ii n |

[0016] The rand function is used to randomly select an element from the list sequence for output, and then proceed to step B1.2.

[0017] Step B1.2 uses an entropy-based feature discretization method. First, the continuous feature is divided into two parts, and the sum of the entropies of the two parts is calculated. Then, the feature is divided at the point where the entropy is minimum. Finally, step B1.2 is repeated for the part with the maximum entropy. The division stops when the entropy value is less than a given threshold. The formula for calculating entropy is as follows:

[0018] E=-∑P x log2P x

[0019] Where P x This represents the probability of class x appearing in the data interval, and then proceeds to step B2.

[0020] Step B2. Use the Bootstrap sampling strategy to create a training set for each decision tree. First, randomly divide the dataset into two parts. Then, use the Bootstrap sampling strategy to sample from the dataset to generate multiple different datasets. Based on each dataset, build a decision tree classifier and then proceed to step B3.

[0021] Step B3. Generate a random forest algorithm from the generated decision tree classifiers, construct a software defect prediction model, and determine the final classification result by voting among the multiple tree classifiers.

[0022] As a preferred embodiment of the present invention, step C includes steps C1 to C3 as follows:

[0023] Step C1. Predict the input data. Sample true label y i The probability of being predicted as defective, i.e., taking the value 1, is:

[0024] P1=P(y i =1|x i ,w)

[0025] Where sample i is composed of feature vector x i The prediction function is composed of the weight vector w, and then proceeds to step C2;

[0026] Step C2. Train using binary cross-entropy loss. For a dataset of m samples, the loss function over all samples is:

[0027]

[0028] Where u represents the selected department and m represents the number of samples in the dataset, then proceed to step C3.

[0029] Step C3. Calculate the model gradient based on the loss function. The obtained local model gradient is:

[0030]

[0031] Where w represents the current model parameters.

[0032] As a preferred embodiment of the present invention, step D includes steps D1 to D2 as follows:

[0033] Step D1. Clip the gradient using the following clipping function:

[0034] clip(G u ,μ)

[0035] Here, μ is a parameter that limits the range of the gradient, and then proceed to step D2;

[0036] Step D2. In gradient clipping, local differential privacy is used to obtain protected gradients.

[0037]

[0038] Where M(·) is a random algorithm, n is Laplace noise, and the parameter λ can control the intensity of the Laplace noise.

[0039] As a preferred embodiment of the present invention, step E includes steps E1 to E2 as follows:

[0040] Step E1. The server aggregates all uploaded protected gradients to obtain the aggregated gradient.

[0041]

[0042] Where u represents the selected department, then proceed to step E2;

[0043] Step E2. The server uses aggregated gradients. Update the global model parameters w maintained on the server, and then distribute them to departmental devices to update their local models:

[0044]

[0045] Where η is the learning rate. Repeat this process until the model training converges.

[0046] As a preferred embodiment of the present invention, the information includes loss calculation, local gradient calculation, adding noise to the gradient, aggregated gradient calculation, and global model parameter update.

[0047] Compared with the prior art, the technical solution of the present invention has the following technical effects:

[0048] This invention leverages a random forest model built on decision trees for software defect prediction, addressing the privacy concerns arising from relying on centralized storage of enterprise private data for model training. It employs a local differential privacy algorithm, adding noise to the gradients before uploading local model gradients to protect their privacy, thus eliminating reliance on trusted servers for enterprise departments. Simultaneously, it utilizes a federated learning privacy framework, enabling multiple clients to collaboratively train the model without sharing their private data, overcoming the limitation of insufficient single-data sets for training accurate and unbiased software defect prediction models. Through continuous updates to global model parameters via federated learning, a software defect prediction model with strong learning capabilities and high predictive performance is trained. Attached Figure Description

[0049] Figure 1 This is a flowchart of the privacy protection method for software defect prediction based on federated learning designed in this invention.

[0050] Figure 2 This is a diagram of the random forest local defect prediction model designed in this invention after sample oversampling and feature discretization.

[0051] Figure 3 This is a framework diagram of the software defect federated learning model designed in this invention, which combines differential privacy protection and gradient aggregation. Detailed Implementation

[0052] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0053] Embodiments of this invention utilize data from a large number of clients to train a software defect prediction model, without requiring centralized storage of these models. Client data is not directly uploaded to the server, thus achieving software defect prediction model training in a privacy-preserving manner. This includes a defect prediction model construction method, and the application of a local differential privacy algorithm and a federated learning framework for secure gradient aggregation, such as... Figure 1 As shown, the specific steps are as follows: A through E.

[0054] Step A. Select a group of clients consisting of various departments within the enterprise to participate in collaborative model training. For each training session, a department from the client is randomly selected. This department inputs locally stored source code data, which is then used by the software defect prediction model for prediction and training. The process then proceeds to Step B.

[0055] In practical applications, departmental data is stored locally on their own devices and is not uploaded to servers, which can prevent the risk of data leakage.

[0056] Since the data from a single department is far from sufficient to train an accurate and unbiased model, we coordinate a large number of enterprise departments to jointly train the software defect prediction model and proceed to step B below.

[0057] Step B. On the department's local equipment, preprocess the input source code, create a training set for each decision tree using the Bootstrap sampling strategy, and construct a software defect prediction model based on the random forest algorithm generated by the decision tree classifier, such as... Figure 2 As shown, proceed to step C.

[0058] In practical application, step B is specifically executed as follows: steps B1 to B3.

[0059] Step B1. Preprocess the input source code; the preprocessing mainly includes two steps: oversampling a few instances and discretizing continuous features.

[0060] In practical applications, step B1 addresses the issues of imbalanced datasets and hidden defects in the datasets, and is performed according to steps B1.1 to B1.2 to obtain the preprocessed dataset.

[0061] Step B1.1 uses the SMOTE algorithm to oversample the minority instances. First, for each sample i in the minority class, calculate its distance to all samples in the minority class sample set using Euclidean distance to obtain its k nearest neighbors. Then, set a sampling ratio based on the sample imbalance ratio to determine the sampling multiplier N. For each minority class sample i, randomly select several samples from its k nearest neighbors, assuming the selected nearest neighbors are ik and ik. n Finally, for each randomly selected nearest neighbor i n Construct new samples according to the following formula:

[0062] i new =i+rand(0,1)*|ii n |

[0063] The rand function is used to randomly select an element from the list sequence for output; the SMOTE algorithm is used to analyze the minority class samples and artificially synthesize new samples based on the minority class samples to add to the dataset, and then proceed to step B1.2.

[0064] Step B1.2 uses an entropy-based feature discretization method. First, the continuous feature is divided into two parts, and the sum of the entropies of the two parts is calculated. Then, the feature is divided at the point where the entropy is minimum. Finally, step B1.2 is repeated for the part with the maximum entropy. The division stops when the entropy value is less than a given threshold. The formula for calculating entropy is as follows:

[0065] E=-∑Px log2P x

[0066] Where P x This represents the probability of class x appearing in the data interval. Discretizing continuous features can improve model performance, and then proceed to step B2.

[0067] In practical applications, after step B1, the class imbalance problem between samples will be improved, and continuous features will be discretized to obtain a dataset suitable for model training.

[0068] Step B2. Use the Bootstrap sampling strategy to create a training set for each decision tree. First, randomly divide the dataset into two parts. Then, use the Bootstrap sampling strategy to sample from the dataset to generate multiple different datasets. Based on each dataset, build a decision tree classifier and then proceed to step B3.

[0069] The Bootstrap sampling strategy uses a limited sample to build up a sufficient sample through multiple repeated sampling, thus solving the problem of insufficient samples.

[0070] Step B3. Generate a random forest algorithm from the generated decision tree classifiers, construct a software defect prediction model, determine the final classification result by voting among the multiple tree classifiers, and then proceed to step C.

[0071] Random forest algorithms have high accuracy among all current binary classification algorithms and can handle input samples with high-dimensional features without the need for additional dimensionality reduction and feature selection. On the other hand, the randomness of random forests is mainly reflected in the randomness of data selection and feature selection, which makes random forests less prone to overfitting.

[0072] Step C. Based on local departmental data, perform software defect prediction and model training to obtain the prediction result y. i Calculate the probability P1 that a sample label is predicted as 1, train to obtain the binary classification cross-entropy loss L(u), and calculate the local model gradient G based on the loss. u Then proceed to step D;

[0073] In actual application, step C is specifically executed as follows: steps C1 to C3.

[0074] Step C1. Predict the input data. Sample i, true label y i The probability of being predicted as defective, i.e., taking the value 1, is:

[0075] P1=P(y i =1|x i ,w)

[0076] Where sample i is composed of feature vector x i The prediction function is composed of the weight vector w, and then proceeds to step C2.

[0077] Step C2. Train using binary cross-entropy loss. For a dataset of m samples, the loss function over all samples is:

[0078]

[0079] Where u represents the selected department and m represents the number of samples in the dataset, then proceed to step C3.

[0080] Step C3. Calculate the model gradient based on the loss function. The resulting local model gradient G u for:

[0081]

[0082] Where w represents the current model parameters, then proceed to step D.

[0083] Step D. Apply the local model gradient G u Gradient clipping is performed, followed by the application of local differential privacy techniques to obtain protected gradients. Then proceed to step E;

[0084] In practical applications, local gradients from a randomly selected set of departments are uploaded to a server for further aggregation to update the global model on the server. However, since local model gradients may contain some implicit private information, this invention applies local differential privacy technology to these local model gradients before uploading them to the server to better protect departmental privacy.

[0085] In practical applications, the gradient can become too large during model training, making it impossible to obtain a stable model from the training data. Therefore, this invention uses a gradient clipping function to ensure the maximum value of the gradient. By using gradient clipping, the gradient change is limited to a reasonable range, thereby avoiding the gradient explosion problem.

[0086] Gradient clipping is a process of trimming the gradient during gradient propagation so that the gradient never reaches a set threshold.

[0087] Differential privacy is a privacy protection technique based on data distortion, which distorts sensitive data by adding noise. Local differential privacy is achieved by adding Laplace noise to the values ​​that require privacy protection using a random algorithm M(·).

[0088] In this invention, local differential privacy technology is used to operate locally in each department without going through a third party, so as to better protect departmental privacy and make enterprise departments independent of trusted servers.

[0089] In actual application, step D is specifically executed as follows: steps D1 to D2.

[0090] Step D1. Clip the gradient using the following clipping function:

[0091] clip(G u ,μ)

[0092] Here, μ is a parameter that limits the range of the gradient, and then proceeds to step D2.

[0093] Step D2. In gradient clipping, local differential privacy is used to obtain protected gradients.

[0094]

[0095] Where M(·) is a random algorithm, n is Laplace noise, and the parameter λ can control the intensity of the Laplace noise, and then proceed to step E.

[0096] Step E. Numerous departmental clients will protect the model gradients. The gradients of all uploaded models are uploaded to the server, where federated learning technology is used to aggregate the gradients of all uploaded models, resulting in the aggregated gradient. The global model parameters w maintained in the server are updated using aggregated gradients, and then the updated global model is distributed to departmental devices to update their local model parameters. The framework of federated learning is as follows: Figure 3 As shown, repeat this process until the model training converges.

[0097] Federated learning is a privacy-preserving machine learning technique that uses data from a large number of clients to train a global model without centrally storing the client data. In federated learning, client data does not need to be uploaded to a server; it only needs to be stored locally on the client. Instead, each client device computes the model gradient based on its local data, and the local model gradients from multiple clients are aggregated to update the global model. Because model updates typically contain far less information than the original user data, the risk of privacy breaches is effectively reduced.

[0098] In actual application, step E is specifically executed as follows: steps E1 to E2.

[0099] Step E1. The server aggregates all uploaded protected gradients to obtain the aggregated gradient.

[0100]

[0101] Where u represents the selected department, then proceed to step E2.

[0102] Step E2. The server uses aggregated gradients. Update the global model parameters w maintained on the server, and then distribute them to departmental devices to update their local models:

[0103]

[0104] Where η is the learning rate. Repeat this process until the model training converges.

[0105] The present invention has been described in detail above with reference to the accompanying drawings and specific embodiments. However, the present invention is not limited to the above-described embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made based on the present invention.

Claims

1. A privacy-preserving method for software defect prediction based on federated learning, characterized in that: This includes a method for constructing a software defect prediction model, and the application of federated learning technology to train the software defect prediction model to achieve defect prediction based on departmental data. It includes the following steps: Step A. Select a group of clients consisting of various departments of the enterprise to participate in collaborative model training; each training session randomly selects a department from the clients, which inputs the source code data stored locally, and then proceeds to Step B; Step B. On the department's local device, preprocess the input source code, use the Bootstrap sampling strategy to create a training set for each decision tree, build a software defect prediction model based on the random forest algorithm generated by the decision tree classifier, and then proceed to step C; Step C. Based on local departmental data, perform software defect prediction and model training to obtain the prediction result y. i Calculate the probability P1 that a sample label is predicted as 1, train to obtain the binary classification cross-entropy loss L(u), and calculate the local model gradient G based on the loss. u Then proceed to step D; Step C includes steps C1 to C3 as follows: Step C1. Predict the input data: the true label y of the sample. i The probability of being predicted as defective, i.e., taking the value 1, is: P1=P(y i =1|x i ,w) Where sample i is composed of feature vector x i The prediction function is composed of the weight vector w, and then proceeds to step C2; Step C2. Train using binary cross-entropy loss. For a dataset of m samples, the loss function over all samples is: Where u represents the selected department and m represents the number of samples in the dataset, then proceed to step C3; Step C3. Calculate the model gradient based on the loss function. The resulting local model gradient is: Where w represents the current model parameters; Step D. Apply gradient G to the local model. u Gradient clipping is performed, followed by the application of local differential privacy techniques to obtain protected gradients. Then proceed to step E; Step E. Numerous departmental clients will protect the model gradients. The models are uploaded to a server, where federated learning technology is used to aggregate the gradients of all uploaded models, resulting in the aggregated gradient. The global model parameters w maintained in the server are updated using aggregated gradients. The updated global model is then distributed to departmental devices to update their local model parameters. This process is repeated until the model training converges.

2. The privacy-preserving method for software defect prediction based on federated learning according to claim 1, characterized in that: In step B, a software defect prediction model is constructed based on the random forest algorithm generated by the decision tree classifier, and is carried out according to steps B1 to B3 to obtain the software defect prediction model. Step B1. Preprocess the input source code; the preprocessing mainly includes two steps: oversampling a few instances and discretizing continuous features; Proceed to steps B1.1 and B1.2 below to obtain the preprocessed dataset: Step B1.1 Use the SMOTE algorithm to oversample the minority instances; first, for each sample i in the minority class, calculate its distance to all samples in the minority class sample set using Euclidean distance as the standard, and obtain its k nearest neighbors; Then, a sampling ratio is set according to the imbalance ratio to determine the sampling multiplier N. For each minority class sample i, several samples are randomly selected from its k nearest neighbors. Let's assume the selected nearest neighbors are i. n Finally, for each randomly selected nearest neighbor i n Construct new samples according to the following formula: i new =i+rand(0,1)*|i-i n | The rand function is used to randomly select an element from the list sequence for output, and then proceed to step B1.2; Step B1.2 uses an entropy-based feature discretization method. First, the continuous feature is divided into two parts, and the sum of the entropies of the two parts is calculated. The division occurs at the point where the entropy is minimum. Finally, step B1.2 is repeated for the part with the maximum entropy. The division stops when the entropy value is less than a given threshold. The formula for calculating entropy is as follows: E =-∑P x log2P x Where P x This represents the probability of class x appearing in the data interval, and then proceeds to step B2; Step B2. Use the Bootstrap sampling strategy to create a training set for each decision tree; first, randomly divide the dataset into two parts, then use the Bootstrap sampling strategy to sample from the dataset to generate multiple different datasets. Based on each dataset, build a decision tree classifier, and then proceed to step B3. Step B3. Generate a random forest algorithm from the generated decision tree classifiers, construct a software defect prediction model, and determine the final classification result by voting among the multiple tree classifiers.

3. The privacy-preserving method for software defect prediction based on federated learning according to claim 1, characterized in that, Step D includes steps D1 to D2 as follows: Step D1. Clip the gradient using the following clipping function: clip(G u ,m) Here, μ is a parameter that limits the range of the gradient, and then proceed to step D2; Step D2. In gradient clipping, local differential privacy is used to obtain protected gradients. : Where M(·) is a random algorithm, n is Laplace noise, and the parameter λ is used to control the intensity of the Laplace noise.

4. The privacy-preserving method for software defect prediction based on federated learning according to claim 3, characterized in that: Step E includes steps E1 to E2 as follows: Step E1. The server aggregates all uploaded protected gradients to obtain the aggregated gradient. : Where u represents the selected department, then proceed to step E2; Step E2. The server uses aggregated gradients. Update the global model parameters w maintained on the server, and then distribute them to departmental devices to update their local models: Where η is the learning rate; repeat this process until the model training converges.

Images