Federal self-distillation software defect prediction method for privacy protection

Abstract

The invention relates to a federal self-distillation software defect prediction method for privacy protection, which is used for performing local software defect prediction based on a deep learning MLP model and solving the problem of privacy disclosure in model training. A local differential privacy algorithm is applied, and before local model parameters are uploaded, privacy protection is carried out on the model parameters through a noise adding technology, so that a client does not depend on a server; meanwhile, a federal learning privacy protection framework is used for collaboratively training the model under the condition that multiple clients do not share private data of the clients, and the problem that single data is insufficient to train a software defect prediction model with excellent performance is solved; and a knowledge self-distillation technology is introduced, so that the personalization of a local model is improved through a historical model of the client while the knowledge of other client models is learned through collaborative training. Global model parameters are continuously updated through federal learning, and a software defect prediction model with high learning ability and good prediction performance is trained.

Description

Technical Field

[0001] The present invention relates to a privacy-protection-oriented federated self-distillation software defect prediction method, belonging to the technical field of privacy-protection-oriented software defect prediction in software engineering. Background Art

[0002] As software systems become more important in people's lives, their complexity also increases. As software systems become more complex, it becomes increasingly difficult to ensure their quality. Therefore, a large amount of research has recently focused on dealing with software quality assurance. In order to improve software quality, the field of software defect prediction has emerged. This field identifies defective software units, attempts to eliminate defects, and makes better use of resources during software development and maintenance. Its practical application is of great significance to the software production process. For example, through software defect prediction, more reliable software systems can be obtained; by focusing on error-prone modules, the accuracy and speed of software testing can be improved; using object-oriented methods, the design can be more reasonable and reconfigurable modules can be identified; and resources can be allocated based on the prediction results, thereby improving the overall quality of the project.

[0003] In recent years, many deep learning-based methods and various feature representation methods have been proposed and applied to software defect prediction. For example, Samoaa et al. used deep learning models to predict potential software defects; Giray et al. extracted abstract syntax trees and various types of graph technologies from source code to introduce defect prediction. In recent studies, it is pointed out that multiple client models sharing model data can train a model that is suitable for all clients and has excellent performance. However, directly sharing models may lead to privacy leakage. In current software defect prediction research, privacy and security issues are very important, and most clients are reluctant to share their data. In order to promote data sharing and protect user privacy, FedAVG, a traditional federated learning model with strong generalization ability, has been proposed in recent years. However, the data distribution of each client is different. In this case of data heterogeneity, the performance of the local model after collaborative training by traditional federated learning models such as FedAVG may be worse than that of the local model without collaborative training. The shared global model learned by the traditional federated learning solution has a consistent training goal and excellent generalization ability. In the field of software defects, it is hoped that each client model can not only learn the knowledge of other clients through the generalization ability of federated learning, but also train a well-performing local personalized model on its own specific data distribution to retain local specific knowledge.

[0004] In order to adapt to heterogeneous learning scenarios and protect user privacy, under the federated learning framework, it is necessary to study how to adapt to local data distribution while maintaining personalization, and also generalize to leverage collaboration between clients and learn shared knowledge. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a privacy-protected federated self-distillation software defect prediction method, which uses a federated learning privacy protection framework to realize knowledge sharing between clients, fully utilizes multiple sets of data for training in a privacy-protected manner, and integrates knowledge self-distillation technology into the training to effectively improve the accuracy of the software defect prediction model.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions: the present invention designs a privacy-preserving federated self-distillation software defect prediction method, initializes the number of iterations k=1, and obtains a software defect prediction model according to the following steps, which is used to predict whether the corresponding code in the software has defects;

[0007] Step A. Randomly select a preset number N of local clients as each local client in the kth iteration, and use each local client to initialize each parameter in the MLP multi-layer perceptron model, and then enter step B;

[0008] Step B. In the kth iteration, each local client creates a preset number of code strings including a corresponding defective label 1 and a non-defective label 0, and each code string corresponds to a feature value of each preset target type feature, and a single sample is formed by the label corresponding to a single code string and the feature value of each preset target type feature corresponding to the code string, forming a sample set corresponding to each local client, and then proceeding to step C;

[0009] Step C. In the kth iteration, each local client performs self-distillation training on the MLP multi-layer perceptron model based on its corresponding sample set and the initial values ​​of each parameter in the MLP multi-layer perceptron model, combines the binary cross entropy loss and the KL divergence loss, takes the feature values ​​of each preset target type feature corresponding to the code string in the sample as input, and the label corresponding to the code string in the sample as output, and obtains the software defect prediction local model and its corresponding model performance parameter prediction results, and then proceeds to step D;

[0010] Step D. In the kth iteration, each local client predicts the parameter group of the local model obtained by its training, applies local differential privacy technology to protect it, obtains the protection parameter group, and uploads it to the server in combination with the corresponding model performance parameter prediction result, and then proceeds to step E;

[0011] Step E. The server receives the protection parameter groups and model performance parameter prediction results uploaded by each local client in the kth iteration, fuses each protection parameter group to obtain a fused protection parameter group, and obtains the average prediction result of the model performance parameter, and determines whether the average prediction result of the model performance parameter converges. If so, the obtained fused protection parameter group is the target parameter group, and the MLP multi-layer perceptron model is connected to form a software defect prediction model; otherwise, it goes to step F;

[0012] Step F. The server randomly selects a preset number N of local clients as the local clients in the k+1th iteration, and sends the fused protection parameter group to each local client to constitute the initial value of each parameter in the MLP multi-layer perceptron model for each local client, and then updates k with the value of k+1, and returns to step B.

[0013] As a preferred technical solution of the present invention: in the kth iteration in step B, each local client performs the following steps B1 to B7 respectively to obtain a sample set corresponding to each local client;

[0014] Step B1. Create a preset quantity A code string containing a defective label 1 and a non-defective label 0, and each code string corresponds to a feature value of each preset target type feature, and a single sample is formed by a label corresponding to a single code string and a feature value of each preset target type feature corresponding to the code string, wherein: represents the number of samples corresponding to the nth local client in the kth iteration, and then proceeds to step B2;

[0015] Step B2: Determine whether the proportion of samples with defective label 1 and the proportion of samples without defective label 0 is less than the preset proportion threshold b. If so, take the samples corresponding to the proportion as minority class samples and select the samples based on the number of minority class samples. according to Obtain the sample increment corresponding to the minority class samples to achieve the ratio of the number of minority class samples to the total number of samples equal to the preset proportion threshold b Then proceed to step B3; otherwise, the obtained samples form a sample set corresponding to the local client, and proceed to step C; wherein, represents the number of minority class samples corresponding to the nth local client in the kth iteration, Indicates the sample increment corresponding to the minority class samples corresponding to the nth local client in the kth iteration;

[0016] Step B3. Based on the vectors composed of the feature values ​​and labels of the preset target type features corresponding to each minority class sample, the distance between the two vectors is obtained, and the k nearest neighbors are applied to obtain the nearest minority class samples corresponding to each minority class sample, and then proceed to step B4;

[0017] Step B4. For each minority class sample, Get the weights corresponding to each minority class sample, represents the weight corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, k represents the k value of the kth neighbor, Indicates the number of neighboring minority class samples corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, and then proceeds to step B5;

[0018] Step B5. For each minority class sample, Get the sample increments corresponding to each minority class sample, where: represents the sample increment corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, and then proceeds to step B6;

[0019] Step B6. For each minority class sample, according to the sample increment corresponding to the minority class sample, Create a feature vector constructed by the new target type features, and combine it with the same label as the minority class sample to construct a new sample, and then obtain the sample increment corresponding to the minority class sample. New samples, where represents the feature vector constructed based on the new target type features created based on the i'th minority class sample corresponding to the nth local client in the kth iteration, represents the feature vector constructed by the features of each target type in the i'th minority class sample corresponding to the nth local client in the kth iteration, λ1 represents a random value from the uniform distribution of values ​​in [0, 1], Indicates the feature vector constructed by randomly selecting each target type feature of a minority class sample from each neighboring minority class sample corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, and then proceeds to step B7;

[0020] Step B7: The sample set corresponding to the local client is formed by each sample and each new sample obtained based on each minority class sample.

[0021] As a preferred technical solution of the present invention: in the step C, each local client performs the following steps C1 to C4 respectively in the kth iteration, constructs a binary cross entropy loss and a KL divergence loss, and a comprehensive loss, and participates in self-distillation training for the MLP multi-layer perceptron model;

[0022] Step C1. According to the following formula:

[0023]

[0024] Based on the preset temperature parameter T', is the input, predict the output The probability of being equal to 1 And further press Based on the preset temperature parameter T', is the input, predict the output The probability of being equal to 0 in, represents the feature vector constructed by the features of each target type in the i-th sample corresponding to the n-th local client in the k-th iteration, express The corresponding predicted output is represents the weight vector in the MLP multi-layer perceptron model obtained by training the nth local client at the kth iteration, represents the bias term in the MLP multi-layer perceptron model obtained by training the nth local client in the kth iteration, and T represents transposition; then proceed to step C2;

[0025] Step C2. According to the following formula:

[0026]

[0027] Construct binary cross entropy loss l CE (n k ),in, represents the number of samples corresponding to the nth local client in the kth iteration, l CE (n k ) represents the binary cross entropy loss corresponding to the nth local client at the kth iteration, and then proceeds to step C3;

[0028] Step C3.

[0029] According to the following formula:

[0030]

[0031] Construct KL divergence loss l KL (n k ),in, Indicates the number of samples corresponding to the nth local client in the k-1th iteration, represents the feature vector constructed by the features of each target type in the i-th sample corresponding to the n-th local client in the k-1-th iteration, express The corresponding predicted output, l KL (n k ) represents the KL divergence loss corresponding to the nth local client in the kth iteration, and then proceeds to step C4;

[0032] Step C4. According to the following formula:

[0033] l(n k )=l CE (n k )+λl KL (n k )

[0034] Construct comprehensive loss l(n k ), l(n k ) represents the comprehensive loss obtained by the n-th local client performing training at the k-th iteration.

[0035] As a preferred technical solution of the present invention: in the step D, each local client in the kth iteration respectively predicts the parameter set of the local model of software defect prediction obtained by its training According to the following formula:

[0036]

[0037] Apply local differential privacy technology for protection and obtain the protection parameter group in, represents the parameter group of the local model for software defect prediction obtained by training the nth local client at the kth iteration, represents the protection parameter group corresponding to the local model for software defect prediction obtained by training the nth local client at the kth iteration, Represents a mean of 0 and a covariance matrix of σ 2 I is the Gaussian noise distribution, σ is the standard deviation of the noise, and I represents the identity matrix.

[0038] As a preferred technical solution of the present invention: in the step E, the server executes the following steps E1 to E5 to merge the protection parameter groups uploaded by each local client in the kth iteration to obtain a merged protection parameter group;

[0039] Step E1. According to the following formula:

[0040]

[0041] Obtain the data balance of the sample set corresponding to each local client in the kth iteration in, Indicates the data balance of the sample set corresponding to the nth local client in the kth iteration, represents the Shannon entropy corresponding to the nth local client at the kth iteration, represents the number of samples corresponding to the nth local client in the kth iteration, Q represents the number of labels, that is, Q = 2, represents the number of samples corresponding to the qth label in the samples corresponding to the nth local client in the kth iteration, the first label is a non-defective label, the second label is a defective label, and then proceeds to step E2;

[0042] Step E2. According to the following formula:

[0043]

[0044] Get the sample size corresponding to each local client in the kth iteration in, Indicates the sample size corresponding to the nth local client in the kth iteration, represents the number of samples initially created by the nth local client in step B at the kth iteration, D k represents the sum of the number of samples initially created in step B corresponding to each local client selected in the kth iteration, and then proceeds to step E3;

[0045] Step E3. According to the following formula:

[0046]

[0047] Obtain the learnability of the minority class samples corresponding to each local client under the kth iteration, where represents the learnability of the minority class samples corresponding to the nth local client at the kth iteration, represents the number of minority samples in the samples initially created in step B corresponding to the nth local client in the kth iteration, a1,k represents the sum of the number of minority samples in the samples initially created in step B corresponding to each selected local client in the kth iteration, and then proceeds to step E4;

[0048] Step E4. According to the following formula:

[0049]

[0050] Get the model parameter weights corresponding to each local client in the kth iteration in, represents the model parameter weight corresponding to the nth local client in the kth iteration, and then proceeds to step E5;

[0051] Step E5. The server receives the protection parameter groups uploaded by each local client in the kth iteration. The fusion is performed according to the following formula:

[0052]

[0053] Get the fused protection parameter group W under the kth iteration c，k ,in, represents the protection parameter group corresponding to the local model for software defect prediction obtained by training the nth local client at the kth iteration, and β represents the sum of the model parameter weights corresponding to each local client at the kth iteration.

[0054] The privacy-preserving federated self-distillation software defect prediction method described in the present invention adopts the above technical solution and has the following technical effects compared with the prior art:

[0055] The present invention designs a privacy-oriented federated self-distillation software defect prediction method, which performs local software defect prediction based on the deep learning MLP model, solving the problem of privacy leakage in model training; and applies the local differential privacy algorithm, and protects the privacy of model parameters by adding noise technology before uploading local model parameters, so that the client does not rely on the server; at the same time, the federated learning privacy protection framework is used for multiple clients to collaborate in training models without sharing their private data, solving the problem that single data is not enough to train a software defect prediction model with excellent performance; and introduces knowledge self-distillation technology, while learning the model knowledge of other clients through collaborative training, the personalization of the local model is improved through the historical model of this client. The global model parameters are continuously updated through federated learning, and a software defect prediction model with strong learning ability and good prediction performance is trained. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of the federal self-distillation software defect prediction method designed by the present invention for privacy protection;

[0057] Figure 2 This is a model architecture diagram of the federal self-distillation software defect prediction method designed for privacy protection in the present invention. DETAILED DESCRIPTION

[0058] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0059] The present invention designs a privacy-oriented federated self-distillation software defect prediction method. In practical applications, the number of iterations k=1 is initialized, and the software defect prediction model is obtained according to the following steps to predict whether the corresponding code in the software has defects.

[0060] Step A: Randomly select a preset number N of local clients as local clients in the kth iteration, and use each local client to initialize each parameter in the MLP multi-layer perceptron model, and then proceed to step B.

[0061] Step B. In the kth iteration, each local client creates a preset number of code strings containing corresponding defective labels 1 and non-defective labels 0, and each code string corresponds to a feature value of each preset target type feature, and a single sample is formed by the label corresponding to a single code string and the feature value of each preset target type feature corresponding to the code string, which constitutes a sample set corresponding to each local client, and then enters step C.

[0062] In practical application, the above step B specifically performs the following steps B1 to B7.

[0063] Step B1. Create a preset quantity A code string containing a defective label 1 and a non-defective label 0, and each code string corresponds to a feature value of each preset target type feature, and a single sample is formed by a label corresponding to a single code string and a feature value of each preset target type feature corresponding to the code string, wherein: It represents the number of samples corresponding to the nth local client in the kth iteration, and then goes to step B2.

[0064] Step B2: Determine whether the proportion of samples with defective label 1 and the proportion of samples without defective label 0 is less than the preset proportion threshold b. If so, take the samples corresponding to the proportion as minority class samples and select the samples based on the number of minority class samples. according to Obtain the sample increment corresponding to the minority class samples to achieve the ratio of the number of minority class samples to the total number of samples equal to the preset proportion threshold b Then proceed to step B3; otherwise, the obtained samples form a sample set corresponding to the local client, and proceed to step C; wherein, represents the number of minority class samples corresponding to the nth local client in the kth iteration, Indicates the sample increment corresponding to the minority class samples corresponding to the nth local client in the kth iteration.

[0065] Step B3. Based on the vectors composed of the feature values ​​and labels of the preset target type features corresponding to each minority class sample, the distance between the two vectors is obtained, and the k nearest neighbors are applied to obtain the nearest neighbor minority class samples corresponding to each minority class sample, and then enter step B4.

[0066] Step B4. For each minority class sample, Obtaining the weights corresponding to each minority class sample reflects the density information of the minority class samples. The larger the weight, the sparser the area where the minority class samples are located. Indicates the i-th local client corresponding to the n-th local client in the k-th iteration ’ The weight corresponding to the minority class samples, k represents the k value of the k nearest neighbor, Indicates the i-th local client corresponding to the n-th local client in the k-th iteration ’ The number of minority class samples corresponding to the neighboring minority class samples is calculated, and then the process goes to step B5.

[0067] Step B5. For each minority class sample, Get the sample increments corresponding to each minority class sample, where: Indicates the i-th local client corresponding to the n-th local client in the k-th iteration ’ The sample increment corresponding to the minority class samples is calculated, and then the process goes to step B6.

[0068] Step B6. For each minority class sample, according to the sample increment corresponding to the minority class sample, Create a feature vector constructed by the new target type features, and combine it with the same label as the minority class sample to construct a new sample, and then obtain the sample increment corresponding to the minority class sample. New samples, where Indicates the i-th local client corresponding to the n-th local client in the k-th iteration ’ The feature vectors constructed by the new target type features created by the minority class samples are Indicates the i-th local client corresponding to the n-th local client in the k-th iteration ’ The feature vector constructed by the features of each target type in the minority class samples, λ1 represents a random value from the uniform distribution between 0 and 1, Indicates the i-th local client corresponding to the n-th local client in the k-th iteration ’ A feature vector is constructed by randomly selecting each target type feature of a minority class sample from each neighboring minority class sample corresponding to each minority class sample, and then entering step B7.

[0069] Step B7: The sample set corresponding to the local client is formed by each sample and each new sample obtained based on each minority class sample.

[0070] In the application, since the data of the local client is stored locally in its own device and will not be uploaded to the server, the risk of data leakage can be prevented. And because the data of a single local client is far from sufficient to train a high-performance model, a large number of local clients are coordinated to jointly train the software defect prediction model.

[0071] Step C. In the kth iteration, each local client respectively performs self-distillation training on the MLP multi-layer perceptron model based on its corresponding sample set and the initial values ​​of each parameter in the MLP multi-layer perceptron model, combines the binary cross entropy loss and the Kullback-Leibler (KL) divergence loss, takes the feature values ​​of each preset target type feature corresponding to the code string in the sample as input, and the label corresponding to the code string in the sample as output, obtains the software defect prediction local model and its corresponding model performance parameter prediction results, and then enters step D.

[0072] Multilayer Perceptron (MLP) is widely used in current binary classification algorithms and demonstrates high accuracy. MLP is able to process input samples with high-dimensional features without the need for additional dimensionality reduction and feature selection. On the other hand, the advantage of MLP lies in its flexibility and nonlinear modeling capabilities, which enable it to capture complex data patterns. In addition, MLP can effectively prevent overfitting problems by using appropriate regularization techniques such as Dropout and L2 regularization. The multi-layer structure of MLP enables it to extract features layer by layer, thereby achieving better classification performance and generalization capabilities.

[0073] In practical applications, in the above step C, each local client executes the following steps C1 to C4 respectively in the kth iteration to construct the binary cross entropy loss and Kullback-Leibler (KL) divergence loss, as well as the comprehensive loss, and participate in the self-distillation training for the MLP multi-layer perceptron model.

[0074] Step C1. According to the following formula:

[0075]

[0076] Get the preset temperature parameter T ’ , when T ’ When >1, the output probability distribution will be smoother. is the input, predict the output The probability of being equal to 1 And further press Get the preset temperature parameter T ’ ,by is the input, predict the output The probability of being equal to 0 in, represents the feature vector constructed by the features of each target type in the i-th sample corresponding to the n-th local client in the k-th iteration, express The corresponding predicted output is represents the weight vector in the MLP multi-layer perceptron model obtained by training the nth local client at the kth iteration, represents the bias term in the MLP multi-layer perceptron model obtained by training the n-th local client in the k-th iteration, and T represents transposition; then proceed to step C2.

[0077] Step C2. According to the following formula:

[0078]

[0079] Construct binary cross entropy loss l CE (n k ),in, represents the number of samples corresponding to the nth local client in the kth iteration, l CE (n k ) represents the binary cross entropy loss corresponding to the nth local client in the kth iteration, and then proceeds to step C3.

[0080] Step C3. According to the following formula:

[0081]

[0082] Construct KL divergence loss l KL (n k ),in, Indicates the number of samples corresponding to the nth local client in the k-1th iteration, represents the feature vector constructed by the features of each target type in the i-th sample corresponding to the n-th local client in the k-1-th iteration, express The corresponding predicted output, l KL (n k ) represents the KL divergence loss corresponding to the nth local client in the kth iteration, and then proceeds to step C4.

[0083] Step C4. According to the following formula:

[0084] l(n k )=l CE (n k )+λl KL (n k )

[0085] Construct comprehensive loss l(n k ), l(n k ) represents the comprehensive loss obtained by the n-th local client performing training at the k-th iteration.

[0086] Step D. In the kth iteration, each local client predicts the parameter set of the local model for the software defect obtained through training. According to the following formula:

[0087]

[0088] Apply local differential privacy technology for protection and obtain the protection parameter group And combined with the corresponding model performance parameter prediction results, upload to the server, and then enter step E. Among them, represents the parameter group of the local model for software defect prediction obtained by training the nth local client at the kth iteration, represents the protection parameter group corresponding to the local model for software defect prediction obtained by training the nth local client at the kth iteration, Represents a mean of 0 and a covariance matrix of σ 2 I is the Gaussian noise distribution, σ is the standard deviation of the noise, and I represents the identity matrix.

[0089] In actual applications, local model parameters from a group of randomly selected local clients are uploaded to the server and further aggregated to update the global model in the server. However, since the local model parameters may contain some privacy information, the present invention is designed to apply local differential privacy technology to these local model parameters before uploading these local model parameters to the server to better protect client privacy. Differential privacy technology distorts sensitive data by adding noise, and is a privacy protection technology based on data distortion.

[0090] Step E. The server receives the protection parameter groups and model performance parameter prediction results uploaded by each local client in the kth iteration, fuses each protection parameter group to obtain a fused protection parameter group, and simultaneously obtains the average prediction results of the model performance parameters, and determines whether the average prediction results of the model performance parameters converge. If so, the obtained fused protection parameter group is the target parameter group, and the MLP multi-layer perceptron model is connected to form a software defect prediction model; otherwise, step F is entered.

[0091] Federated learning is a privacy-preserving machine learning technology that can use data from a large number of clients to aggregate global models without centrally storing client data, thereby achieving the effect of generalizing the model. Traditional aggregation methods only consider the data size of each client, and now also introduce minority class learnability and data balance.

[0092] In actual applications, in the above step E, the server executes the following steps E1 to E5 to merge the protection parameter groups uploaded by each local client in the kth iteration to obtain a merged protection parameter group.

[0093] Step E1. According to the following formula:

[0094]

[0095] Obtain the data balance of the sample set corresponding to each local client in the kth iteration The value range is from 0 to 1, where 0 means the data set is extremely unbalanced and 1 means the data set is relatively balanced. The larger the value of, the more balanced the data set is, where Indicates the data balance of the sample set corresponding to the nth local client in the kth iteration, represents the Shannon entropy corresponding to the nth local client at the kth iteration, represents the number of samples corresponding to the nth local client in the kth iteration, Q represents the number of labels, that is, Q = 2, It represents the number of samples corresponding to the qth label in the samples corresponding to the nth local client in the kth iteration, the first label is a non-defective label, and the second label is a defective label, and then enters step E2.

[0096] Step E2. According to the following formula:

[0097]

[0098] Get the sample size corresponding to each local client in the kth iteration in, Indicates the sample size corresponding to the nth local client in the kth iteration, represents the number of samples initially created by the nth local client in step B at the kth iteration, D k It represents the sum of the number of samples initially created in step B corresponding to each local client selected in the kth iteration, and then enters step E3.

[0099] Step E3. Convert the measure of the learnability of the minority class of each local client into the ratio of the minority class instances in each dataset to the total number of such instances in all client datasets, as follows:

[0100]

[0101] Obtain the learnability of the minority class samples corresponding to each local client under the kth iteration, where represents the learnability of the minority class samples corresponding to the nth local client at the kth iteration, represents the number of minority samples in the samples initially created by step B corresponding to the nth local client in the kth iteration, a 1,k It represents the sum of the number of minority class samples in the samples initially created in step B corresponding to each local client selected in the kth iteration, and then enters step E4.

[0102] Step E4. According to the following formula:

[0103]

[0104] Get the model parameter weights corresponding to each local client in the kth iteration in, It represents the model parameter weight corresponding to the nth local client in the kth iteration, and then goes to step E5.

[0105] Step E5. The server receives the protection parameter groups uploaded by each local client in the kth iteration. The fusion is performed according to the following formula:

[0106]

[0107] Get the fused protection parameter group W under the kth iteration c，k ,in, represents the protection parameter group corresponding to the local model for software defect prediction obtained by training the nth local client at the kth iteration, and β represents the sum of the model parameter weights corresponding to each local client at the kth iteration.

[0108] Step F. The server randomly selects a preset number N of local clients as the local clients in the k+1th iteration, and sends the fused protection parameter group to each local client to constitute the initial value of each parameter in the MLP multi-layer perceptron model for each local client, and then updates k with the value of k+1, and returns to step B.

[0109] The above technical solution design performs local software defect prediction based on the deep learning MLP model, solving the problem of privacy leakage in model training; and applies the local differential privacy algorithm to protect the privacy of model parameters through noise addition technology before uploading local model parameters, so that the client does not rely on the server; at the same time, the federated learning privacy protection framework is used for multiple clients to collaborate in training models without sharing their private data, solving the problem that a single data is not enough to train a software defect prediction model with excellent performance; and introduces knowledge self-distillation technology, while learning the knowledge of other client models through collaborative training, the personalization of the local model is improved through the historical model of this client. Through federated learning, the global model parameters are continuously updated to train a software defect prediction model with strong learning ability and good prediction performance.

[0110] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A privacy-preserving federated self-distillation software defect prediction method, characterized by: Initialize the number of iterations k=1, and follow the following steps to obtain a software defect prediction model for predicting whether the corresponding code in the software has defects; Step A. Randomly select a preset number N of local clients as each local client in the kth iteration, and use each local client to initialize each parameter in the MLP multi-layer perceptron model, and then enter step B; Step B. In the kth iteration, each local client creates a preset number of code strings including a corresponding defective label 1 and a non-defective label 0, and each code string corresponds to a feature value of each preset target type feature, and a single sample is formed by the label corresponding to a single code string and the feature value of each preset target type feature corresponding to the code string, forming a sample set corresponding to each local client, and then proceeding to step C; Step C. In the kth iteration, each local client performs self-distillation training on the MLP multi-layer perceptron model based on its corresponding sample set and the initial values ​​of each parameter in the MLP multi-layer perceptron model, combines the binary cross entropy loss and the KL divergence loss, takes the feature values ​​of each preset target type feature corresponding to the code string in the sample as input, and the label corresponding to the code string in the sample as output, and obtains the software defect prediction local model and its corresponding model performance parameter prediction results, and then proceeds to step D; Step D. In the kth iteration, each local client predicts the parameter group of the local model obtained by its training, applies local differential privacy technology to protect it, obtains the protection parameter group, and uploads it to the server in combination with the corresponding model performance parameter prediction result, and then proceeds to step E; Step E. The server receives the protection parameter groups and model performance parameter prediction results uploaded by each local client in the kth iteration, fuses each protection parameter group to obtain a fused protection parameter group, and obtains the average prediction result of the model performance parameter, and determines whether the average prediction result of the model performance parameter converges. If so, the obtained fused protection parameter group is the target parameter group, and the MLP multi-layer perceptron model is connected to form a software defect prediction model; otherwise, it goes to step F; Step F. The server randomly selects a preset number N of local clients as the local clients in the k+1th iteration, and sends the fused protection parameter group to each local client to constitute the initial value of each parameter in the MLP multi-layer perceptron model for each local client, and then updates k with the value of k+1, and returns to step B.

2. According to claim 1, a privacy-preserving federated self-distillation software defect prediction method is characterized by: In the kth iteration of step B, each local client performs the following steps B1 to B7 respectively to obtain a sample set corresponding to each local client; Step B1. Create a preset quantity A code string containing a defective label 1 and a non-defective label 0, and each code string corresponds to a feature value of each preset target type feature, and a single sample is formed by a label corresponding to a single code string and a feature value of each preset target type feature corresponding to the code string, wherein: represents the number of samples corresponding to the nth local client in the kth iteration, and then proceeds to step B2; Step B2: Determine whether the proportion of samples with defective label 1 and the proportion of samples without defective label 0 is less than the preset proportion threshold b. If so, take the samples corresponding to the proportion as minority class samples and select the samples based on the number of minority class samples. according to Obtain the sample increment corresponding to the minority class samples to achieve the ratio of the number of minority class samples to the total number of samples equal to the preset proportion threshold b Then proceed to step B3; otherwise, the obtained samples form a sample set corresponding to the local client, and proceed to step C; wherein, represents the number of minority class samples corresponding to the nth local client in the kth iteration, Indicates the sample increment corresponding to the minority class samples corresponding to the nth local client in the kth iteration; Step B3. Based on the vectors composed of the feature values ​​and labels of the preset target type features corresponding to each minority class sample, the distance between the two vectors is obtained, and the k nearest neighbors are applied to obtain the nearest minority class samples corresponding to each minority class sample, and then proceed to step B4; Step B4. For each minority class sample, Get the weights corresponding to each minority class sample, represents the weight corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, k represents the k value of the kth neighbor, Indicates the number of neighboring minority class samples corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, and then proceeds to step B5; Step B5. For each minority class sample, Get the sample increments corresponding to each minority class sample, where: represents the sample increment corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, and then proceeds to step B6; Step B6. For each minority class sample, according to the sample increment corresponding to the minority class sample, Create a feature vector constructed by the new target type features, and combine it with the same label as the minority class sample to construct a new sample, and then obtain the sample increment corresponding to the minority class sample. New samples, where represents the feature vector constructed based on the new target type features created based on the i'th minority class sample corresponding to the nth local client in the kth iteration, represents the feature vector constructed by the features of each target type in the i'th minority class sample corresponding to the nth local client in the kth iteration, λ1 represents a random value from the uniform distribution in [0,1], Indicates the feature vector constructed by randomly selecting each target type feature of a minority class sample from each neighboring minority class sample corresponding to the i'th minority class sample corresponding to the nth local client in the kth iteration, and then proceeds to step B7; Step B7: The sample set corresponding to the local client is formed by each sample and each new sample obtained based on each minority class sample.

3. According to the privacy protection-oriented federated self-distillation software defect prediction method of claim 1, it is characterized by: In the step C, each local client performs the following steps C1 to C4 respectively in the kth iteration, constructs a binary cross entropy loss and a KL divergence loss, and a comprehensive loss, and participates in self-distillation training for the MLP multi-layer perceptron model; Step C1. According to the following formula: Based on the preset temperature parameter T', is the input, predict the output The probability of being equal to 1 And further press Based on the preset temperature parameter T', is the input, predict the output The probability of being equal to 0 in, represents the feature vector constructed by the features of each target type in the i-th sample corresponding to the n-th local client in the k-th iteration, express The corresponding predicted output is represents the weight vector in the MLP multi-layer perceptron model obtained by training the nth local client at the kth iteration, represents the bias term in the MLP multi-layer perceptron model obtained by training the nth local client in the kth iteration, and T represents transposition; then proceed to step C2; Step C2. According to the following formula: Construct binary cross entropy loss l CE (n k ),in, represents the number of samples corresponding to the nth local client in the kth iteration, l CE (n k ) represents the binary cross entropy loss corresponding to the nth local client at the kth iteration, and then proceeds to step C3; Step C3. According to the following formula: Construct KL divergence loss l KL (n k ),in, Indicates the number of samples corresponding to the nth local client in the k-1th iteration, represents the feature vector constructed by the features of each target type in the i-th sample corresponding to the n-th local client in the k-1-th iteration, express The corresponding predicted output, l KL (n k ) represents the KL divergence loss corresponding to the nth local client in the kth iteration, and then proceeds to step C4; Step C4. According to the following formula: l(n k )=l CE (n k )+λl KL (n k ) Construct comprehensive loss l(n k ), l(n k ) represents the comprehensive loss obtained by the n-th local client performing training at the k-th iteration.

4. According to the privacy protection-oriented federated self-distillation software defect prediction method of claim 1, it is characterized by: In step D, each local client in the kth iteration respectively sets the parameter set of the local model for software defect prediction obtained through training. According to the following formula: Apply local differential privacy technology for protection and obtain the protection parameter group in, represents the parameter group of the local model for software defect prediction obtained by training the nth local client at the kth iteration, represents the protection parameter group corresponding to the local model for software defect prediction obtained by training the nth local client at the kth iteration, Represents a mean of 0 and a covariance matrix of σ 2 Ι is the Gaussian noise distribution, σ is the standard deviation of the noise, and I represents the identity matrix.

5. According to the privacy protection-oriented federated self-distillation software defect prediction method of claim 1, it is characterized by: In the step E, the server executes the following steps E1 to E5 to merge the protection parameter groups uploaded by each local client in the kth iteration to obtain a merged protection parameter group; Step E1. According to the following formula: Obtain the data balance of the sample set corresponding to each local client in the kth iteration in, Indicates the data balance of the sample set corresponding to the nth local client in the kth iteration, represents the Shannon entropy corresponding to the nth local client at the kth iteration, represents the number of samples corresponding to the nth local client in the kth iteration, Q represents the number of labels, that is, Q = 2, represents the number of samples corresponding to the qth label in the samples corresponding to the nth local client in the kth iteration, the first label is a non-defective label, the second label is a defective label, and then proceeds to step E2; Step E2. According to the following formula: Get the sample size corresponding to each local client in the kth iteration in, Indicates the sample size corresponding to the nth local client in the kth iteration, represents the number of samples initially created by the nth local client in step B at the kth iteration, D k represents the sum of the number of samples initially created in step B corresponding to each local client selected in the kth iteration, and then proceeds to step E3; Step E3. According to the following formula: Obtain the learnability of the minority class samples corresponding to each local client under the kth iteration, where represents the learnability of the minority class samples corresponding to the nth local client at the kth iteration, represents the number of minority samples in the samples initially created by step B corresponding to the nth local client in the kth iteration, a 1,k represents the sum of the number of minority class samples in the samples initially created in step B corresponding to each local client selected in the kth iteration, and then proceeds to step E4; Step E4. According to the following formula: Get the model parameter weights corresponding to each local client in the kth iteration in, represents the model parameter weight corresponding to the nth local client in the kth iteration, and then proceeds to step E5; Step E5. The server receives the protection parameter groups uploaded by each local client in the kth iteration. The fusion is performed according to the following formula: Get the fused protection parameter group W under the kth iteration c,k ,in, represents the protection parameter group corresponding to the local model for software defect prediction obtained by training the nth local client at the kth iteration, and β represents the sum of the model parameter weights corresponding to each local client at the kth iteration.

Images