Personalized federated learning method for software defect prediction fusion adaptive screening

Abstract

The invention relates to a software defect prediction fusion adaptive screening oriented personalized federated learning method, which is characterized in that a'feature extractor + classifier 'layered architecture is constructed based on a deep learning MLP model, and multi-client collaborative training is realized by relying on a personalized federated learning framework; similar clients are screened out through a double-screening mechanism, and irrelevant cooperation is avoided; weighting and aggregating classifier parameters by adopting cosine similarity, averaging and aggregating feature extractor parameters, and balancing the universality and personalization of the model; the customized classifier generated by the server is locally finely adjusted to serve as a teacher model, local training is guided in combination with cross entropy and KL divergence loss, and distillation fusion of cooperation and local knowledge is achieved. According to the method, on the premise that private data of the clients are not leaked, the software defect prediction accuracy and stability are effectively improved, the model has high learning ability, high prediction performance and good local adaptability, and the method is suitable for a software defect detection scene with multi-client cooperation.

Description

TECHNICAL FIELD

[0001] The application belongs to the cross field of software engineering and federated learning, and particularly relates to a personalized federated learning method for software defect prediction. The method decomposes the client model into a feature extractor and a classifier, selects a collaboration cluster through classifier parameter distance measurement in collaborative training, and adopts a personalized aggregation and knowledge distillation strategy. In the software defect prediction scene, the method realizes the personalized optimization of each client on its own project data while protecting data privacy. BACKGROUND

[0002] With the continuous expansion of software systems, software defect prediction (SDP) has become an important means to ensure software quality. It predicts whether a code module has potential defects by analyzing its static features (such as complexity, cyclomatic complexity, and code lines), thereby locating high-risk areas in advance.

[0003] However, existing software defect prediction mainly relies on centralized model training, i.e., training on multiple project data sets. This approach has two problems in cross-project scenarios:

[0004] (1) The data distribution of different projects is significantly different, and the centralized model is difficult to balance the characteristics of each project, resulting in decreased prediction performance;

[0005] (2) Software project code and defect information often involve enterprise internal privacy and are difficult to share, resulting in "data silos" and limiting the feasibility of large-scale joint modeling.

[0006] Federated learning (FL) provides a new solution to this problem, allowing multiple clients to collaboratively train a global model without sharing raw data, thereby protecting data privacy. However, the unified global model obtained by the traditional federated averaging (FedAvg) method has significantly decreased performance under non-independent and identically distributed (Non-IID) data, making it difficult to adapt to the heterogeneity between projects.

[0007] To solve this problem, personalized federated learning (PFL) has become a research hotspot. This method builds a personalized model for each client that is more suitable for its data distribution based on global collaboration. In the PFL field, it is a consensus design to divide a multilayer perceptron (MLP) model into "body" and "head": the body (lower / middle layers) is responsible for learning public knowledge across projects, and the head (top layer) is responsible for learning individualized knowledge for a single project.

[0008] Therefore, there is a need for a personalized federated learning method for software defect prediction tasks, which enables the model to fully absorb similar client knowledge while protecting data privacy, and optimizes the aggregation process by combining sample size and performance factors, thereby obtaining a more optimal and adaptive personalized software defect prediction model. SUMMARY

[0009] The present application proposes a personalized federated learning method for software defect prediction with adaptive screening to solve the problems of poor model generalization ability, uneven data distribution, and privacy leakage in existing software defect prediction.

[0010] The method separates the client model structure, realizes personalized collaborative aggregation based on classifier similarity calculation and clustering strategy, and combines the knowledge distillation mechanism for adaptive update, thereby improving the local prediction accuracy and global model stability.

[0011] Initialize the number of iterations k = 1, and obtain the software defect prediction model for predicting whether the code in the software has defects according to the following steps;

[0012] Step A. Select a preset number n of local clients as participating clients in the kth iteration. Each local client corresponds to a software project or a set of code modules. Each client initializes the model parameters based on the MLP (Multi-Layer Perceptron) structure, and divides the model into feature extractor parameter groups, which refer to the first L-1 layers of MLP parameters (used to extract deep representations of code features), and classifier parameter groups, which refer to the Lth layer of MLP parameters (used for binary classification output), then enter step B;

[0013] Step B. In the kth iteration, each local client constructs its training sample set, which contains software module code strings with defect label 1 and non-defect label 0. The label corresponding to a single code string and the feature values corresponding to the preset target type features of the code string constitute a single sample, and finally constitute the sample set corresponding to each local client, then enter step C;

[0014] Step C. In the kth iteration, the client uploads the feature extractor parameters and classifier parameters to the server, which are then vectorized by the server. Then the server calculates the distance matrix of the classifier parameters between the clients. To capture the feature distribution difference between the clients, the server adaptively determines the similar group and the non-similar group by dynamically calculating the mean and standard deviation of the distance distribution, and obtains the collaboration subset of each client after the initial screening in the kth round. Then enter step D;

[0015] Step D. The server further refines the final collaborator set for each client n's classifier parameter set using fine screening on its collaboration subset. Then personalized aggregation is performed for each client, the cosine similarity method is used to calculate the weight of each client for its final collaborator set, and the personalized classifier parameters are obtained after aggregation according to the weight. The feature extractor parameters are the average of the feature extractor parameters of the clients uploaded to the server. Then go to step E;

[0016] Step E. In the kth iteration, each client n downloads the feature extractor parameters and personalized classifier parameters obtained after the processing in step D in the k-1th iteration, combines the binary cross-entropy loss and the knowledge distillation loss, and minimizes the loss by back propagation to obtain updated local model parameters, so as to train the client model and enhance the model's ability to retain and migrate its own historical knowledge, and return to step C until the model converges.

[0017] As a preferred technical solution of the present application: in step A, in the kth iteration, each local client performs steps A1 to A2 as follows, and the client selects and initializes the MLP model parameters;

[0018] Step A1. Select a preset number n of local clients as participating clients in the kth iteration. Each local client uniquely corresponds to a software project or a set of independent code modules, ensuring the business scene independence of the data of each client and avoiding cross-scene data interference. Then go to step A2;

[0019] Step A2. Each participating client initializes the model parameters based on the MLP (Multi-Layer Perceptron) structure, sets the total number of MLP model layers as L, and divides the model parameters into a feature extractor parameter set and a classifier parameter set wherein, represents the first L-1 layer parameters of the MLP model corresponding to the nth local client in the kth iteration, represents the Lth layer parameters of the MLP model corresponding to the nth local client in the kth iteration.

[0020] As a preferred technical solution of the present application: in step C, in the kth iteration, each local client performs steps C1 to C5 as follows, and the server receives the classifier parameters uploaded by the client and performs clustering analysis to obtain the preliminary screening collaboration subset of each client;

[0021] Step C1. Each local client uploads its feature extractor parameter set and classifier parameter set to the server, and the server receives the classifier weight matrix Flattened to 1-dimensional vector The parameter vectorization is completed. Then go to step C2;

[0022] Step C2. According to the following formula:

[0023]

[0024] Calculate the distance of the classifier parameters between any two clients i and n, where, represents the classifier weight vector of the i-th local client at the k-th iteration, represents the square of the L2 norm, represents the maximum distance of the i-th client from all other clients, used to normalize the distance value to the [0, 1] interval, eliminating the dimension effect. Construct the client classifier parameter distance matrix at the k-th iteration Then go to step C3;

[0025] Step C3. According to the following formula:

[0026]

[0027] Calculate the mean μ of the non-diagonal elements of the distance matrix at the k-th iteration k , reflecting the average level of classifier parameter distance between all clients. Where the denominator n(n-1) is the total number of non-diagonal elements of the distance matrix, n is the number of participating clients, and the total number of elements (n*n) is reduced by the diagonal n elements. The numerator is the cumulative sum of all non-diagonal elements .

[0028] According to the following formula:

[0029]

[0030] Calculate the standard deviation σ of the non-diagonal elements of the distance matrix at the k-th iteration k , reflecting the dispersion of the classifier parameter distance between all clients, is the square of the deviation of a single non-diagonal element from the mean. Then go to step C4;

[0031] Step C4. According to the following formula:

[0032] τ k =μ k +σ k

[0033] Calculate the non-similarity group threshold τ at the k-th iteration k , used to exclude clients with significant differences in data distribution from the target client. Then go to step C5;

[0034] Step C5. For the nth client in the kth iteration, if the distance between client i and the classifier parameter of the nth client is then client i is included in the preliminary collaboration subset of the nth client. The final preliminary collaboration subset can be expressed as:

[0035] As a preferred technical solution of the present application: in step D, the server performs the following steps D1 to D6 to further optimize the collaboration range and perform personalized parameter aggregation;

[0036] Step D1. Calculate the distance mean within the preliminary collaboration subset according to the following formula:

[0037]

[0038] reflects the average similarity level of the clients in the preliminary subset to the target client. Wherein, represents the number of clients in the preliminary subset.

[0039] Step D2. Calculate the distance standard deviation within the preliminary collaboration subset according to the following formula:

[0040]

[0041] reflects the dispersion degree of the distance in the preliminary subset. Then go to step D2;

[0042] Step D2. Calculate the distance mean within the preliminary collaboration subset according to the following formula:

[0043]

[0044]

[0045] Step D3. For the nth client in the kth iteration, if the distance between client i and the classifier parameter of the nth client is where T is the total number of iterations. As k increases (in the later iterations), γ k tends to 1, only retaining the most similar clients in the preliminary subset. Then go to step D3;

[0046] Step D3. For the nth client in the kth iteration, if the distance between client i and the classifier parameter of the nth client is then client i is included in the final collaboration subset of the nth client. The final collaboration subset can be expressed as: wherein, in the early iterations the size is large, and more potential collaborators are included to accumulate global knowledge; in the later iterations the size is small, only retaining high-similarity clients to avoid interference from low-similarity clients in personalized aggregation. Then go to step D4;​​

[0047] Step D4. According to the following formula:

[0048]

[0049] For each client i in the final collaborator set Calculate its vectorized classifier weight vector The cosine similarity of the n-th client Measures the similarity of the two parameters. Where, represents the classifier weight vector of the i-th local client at the k-th iteration, represents the classifier weight vector of the n-th local client at the k-th iteration. Then go to step D5;

[0050] Step D5. According to the following formula:

[0051]

[0052] The cosine similarity of each collaborator of each client n is normalized to obtain the weight of each collaborator i Ensure that the sum of the weights is 1. Then according to the weight Weighted aggregation of the classifier parameter group of all clients in the final collaborator set Get the personalized classifier parameters of the n-th client The calculation formula is:

[0053]

[0054] Then go to step D6;

[0055] Step D6. According to the following formula:

[0056]

[0057] The server calculates the global feature extractor parameters using the average aggregation method Where, represents the first L-1 layer parameters of the MLP model corresponding to the n-th local client at the k-th iteration. Ensure that all clients use a unified basic module with general feature extraction capability.

[0058] As a preferred technical solution of the present application: in step E, the server performs the following steps E1 to E5, and each local client completes model training and parameter updating until the model converges.

[0059] Step E1. At the k-th iteration, the n-th client downloads the global feature extractor parameters And personalized classifier parameters The local model parameter set is divided into feature extractor parameters and classifier parameters, for The client first updates the feature extractor parameters of the local model To Complete the synchronization of the global general feature extraction capability. At the same time, the local classifier parameters Prepare for subsequent training. Then go to step E2;

[0060] Step E2. Freeze the feature extractor parameters of the updated local model Only the downloaded customized classifier personalized classifier parameters Set to trainable state. For any sample in the local training sample set Where x is the sample representation, y is the label of the sample, The local data set of client k. First, the input x is encoded by the frozen feature extractor To get the feature vector Then input z into the customized classifier Output the predicted probability distribution Where g is the classifier mapping function, including Softmax normalization.

[0061] Then according to the following formula:

[0062]

[0063] The binary classification cross-entropy loss As the optimization goal, the ρ rounds of stochastic gradient descent (SGD) update fine-tuning are realized, and then the customized classifier parameter is updated to Ensure that the customized classifier gradually adapts to the local data distribution, and this parameter is used as a "teacher model" to guide the local model training. Wherein, Indicates the gradient of the cross-entropy loss to η v Control the fine-tuning step size. Then go to step E3;

[0064] Step E3. The local model parameter set of the nth client at the kth iteration is denoted as Unfreeze the feature extractor parameters Allow it to adapt to the local feature distribution. And freeze the teacher model Fixed guide knowledge to avoid deviation.

[0065] Then according to the following formula:

[0066] L CE (p w , y) = -[y·log(p w) + (1 - y) - log (1 - p w )]

[0067] Build binary cross-entropy loss, where the local feature extractor Encode the input x to get the feature vector Then input z into the customized classifier Output the predicted probability distribution y represents the true label of the sample, y = 1 indicates that the sample corresponds to a code defect, and y = 0 indicates that the sample corresponds to a code without defect. Then go to step E4;

[0068] Step E4. According to the following formula:

[0069]

[0070] Build KL divergence loss for knowledge distillation, where p v is the prediction probability distribution of the fine-tuned teacher model, that is, the customized classifier. p w is the prediction probability distribution of the student model, that is, the local model. p v (0) represents the probability that the sample is predicted to be "defect-free", and p v (1) represents the probability that the sample is predicted to be "defect". This loss guides the local model to absorb the integrated knowledge of the teacher model. Then go to step E5;

[0071] Step E5. According to the following formula:

[0072] L EN = L CE (p w , y) + L KL (p v || p w )

[0073] Build a comprehensive loss that combines cross-entropy loss and KL divergence loss. Take L EN as the optimization goal, and update the local model parameters by SGD:

[0074]

[0075] Where, represents the gradient of the comprehensive loss with respect to , η w controls the local training step size, and finally gets the updated local model parameters

[0076] The software defect prediction fusion adaptive screening oriented personalized federated learning method has the following technical effects compared with the prior art by adopting the above technical scheme:

[0077] The application constructs a "feature extractor + classifier" hierarchical architecture based on a deep learning MLP model, realizes multi-client collaborative training relying on a federated learning framework, and simultaneously fuses adaptive collaborative screening, personalized parameter aggregation, and a knowledge distillation mechanism guided by a customized classifier, thereby significantly improving software defect prediction performance while ensuring privacy security. Through client classifier parameter distance matrix calculation and clustering screening, irrelevant clients are avoided from participating in collaboration, effectively resolving the contradiction between privacy protection and collaboration efficiency; meanwhile, a double threshold is constructed by using the mean and standard deviation of the distance matrix, and the fine screening threshold is shrunk with the iteration round, thereby accurately capturing the feature distribution difference of the client under non-independent and identically distributed data, ensuring that the data distribution similarity of the collaborative client is high, solving the problem of performance decline caused by "blind aggregation" in traditional federated learning, and making the aggregated model more adaptive to the local data characteristics. In addition, the cosine similarity weighted aggregation of the classifier parameters of the final collaborator (the higher the similarity, the greater the weight) is combined with the hierarchical strategy of global feature extractor average aggregation (ensuring general feature capability) and classifier personalized aggregation (retaining client differences), balancing model universality and personalization, and overcoming the pain point of insufficient generalization ability of a single data training model. On this basis, the customized classifier generated by the server is locally fine-tuned to adapt to the local task, and the fine-tuned classifier is used as a teacher model to guide local model training with the help of cross-entropy loss and KL divergence loss, realizing the distillation and fusion of collaborative knowledge and local knowledge, absorbing similar client integrated knowledge, retaining local model historical experience, solving the training shock problem caused by the inconsistency between the global model and the local task target, and significantly improving the prediction accuracy and stability.

[0078] In summary, the application realizes the deep fusion of adaptive screening, personalized aggregation, and knowledge distillation, and makes the trained software defect prediction model have strong learning ability, high prediction performance, and good local adaptability without revealing the private data of the client, which is suitable for the software defect detection scene of multi-client collaboration. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 is a flowchart of the software defect prediction fusion adaptive screening oriented personalized federated learning method designed by the application;

[0080] Figure 2 is a server model architecture diagram of the software defect prediction fusion adaptive screening oriented personalized federated learning method designed by the application.

[0081] Figure 3It is a client model architecture diagram of a personalized federated learning method designed for software defect prediction fusion adaptive screening. DETAILED DESCRIPTION

[0082] The specific embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings of the specification.

[0083] Initialize the number of iterations k = 1, and obtain the software defect prediction model for predicting whether the code in the software has defects according to the following steps;

[0084] Step A. Select a preset number n of local clients as participating clients in the kth iteration. Each local client corresponds to a software project or a set of code modules. Each client initializes the model parameters based on the MLP (Multi-Layer Perceptron) structure, and divides the model into a feature extractor parameter group, which refers to the first L-1 layers of MLP parameters (used to extract deep representations of code features), and a classifier parameter group, which refers to the Lth layer of MLP parameters (used for binary classification output), and then proceeds to step B;

[0085] In actual application, the above step A is implemented as follows: step A1 to step A2.

[0086] Step A1. Select a preset number n of local clients as participating clients in the kth iteration. Each local client uniquely corresponds to a software project or a set of independent code modules, ensuring the independence of the business scenarios of the data of each client and avoiding cross-scenario data interference. Then proceed to step A2;

[0087] Step A2. Each participating client initializes the model parameters based on the MLP (Multi-Layer Perceptron) structure, with the total number of MLP model layers being L, and divides the model parameters into a feature extractor parameter group and a classifier parameter group wherein, represents the first L-1 layers of parameters of the MLP model corresponding to the nth local client in the kth iteration, represents the Lth layer of parameters of the MLP model corresponding to the nth local client in the kth iteration.

[0088] Step B. In the kth iteration, each local client constructs its training sample set, which contains software module code strings with defect label 1 and non-defect label 0. A single sample is composed of the label corresponding to a single code string and the feature values of the preset target type features corresponding to the code string. Finally, a sample set corresponding to each local client is constructed, and then step C is entered.

[0089] Step C. In the kth iteration, the client uploads the feature extractor parameters and the classifier parameters to the server, and the server vectorizes them after receiving. Then the server calculates the distance matrix of the classifier parameters between clients. To capture the differences in feature distribution between clients, the server adaptively determines the similar group and the dissimilar group by dynamically calculating the mean and standard deviation of the distance distribution, and obtains the cooperation subset of each client after the preliminary screening in the kth round. Then go to step D;

[0090] The above step C is specifically implemented as follows in actual application: steps C1 to C5.

[0091] Step C1. Each local client uploads its feature extractor parameter group and the classifier parameter group to the server, and the server flattens the classifier weight matrix of each client into a one-dimensional vector to complete parameter vectorization. Then go to step C2;

[0092] Step C2. Calculate the distance of the classifier parameters between any two clients i and n according to the following formula:

[0093]

[0094] wherein represents the classifier weight vector of the ith local client in the kth iteration, represents the square of the L2 norm, represents the maximum distance between the ith client and all other clients, which is used to normalize the distance value to the interval [0, 1] and eliminate the dimension effect. The distance matrix of the classifier parameters between clients in the kth iteration is constructed as follows: Then go to step C3;

[0095] Step C3. Calculate the mean μ of the non-diagonal elements of the distance matrix in the kth iteration according to the following formula:

[0096]

[0097] k , which reflects the average level of the distance of the classifier parameters between all clients. The denominator n(n-1) is the total number of non-diagonal elements of the distance matrix, n is the number of participating clients, and the total number of elements (n*n) is reduced by the diagonal n elements. The numerator is the cumulative sum of all non-diagonal elements .

[0098] According to the following formula:

[0099]

[0100] ​Calculate the standard deviation σ of the distance matrix off-diagonal elements at the kth iteration k , reflecting the dispersion of the distance of the classifier parameters between all clients, is the square of the deviation of the single off-diagonal element from the mean. Then go to step C4;

[0101] Step C4. According to the following formula:

[0102] τ k = μ k + σ k

[0103] Calculate the threshold τ of the dissimilar group at the kth iteration k , used to exclude clients with significant differences in data distribution from the target client. Then go to step C5;

[0104] Step C5. For the nth client at the kth iteration, if the distance of the classifier parameters between client i and the nth client then client i is included in the preliminary screening collaboration subset of the nth client. The final preliminary screening collaboration subset can be represented as:

[0105] Step D. The server further refines the preliminary screening collaboration subset of the classifier parameter group of each client n to the final collaborator set. Then perform personalized aggregation for each client, calculate the weight of each final collaborator set of the client using the cosine similarity method, and aggregate the personalized classifier parameters according to the weight. The feature extractor parameter is the average of the feature extractor parameters uploaded to the server by the client. Then go to step E;

[0106] The above step D is specifically implemented as follows in actual application: steps D1 to D6.

[0107] Step D1. According to the following formula:

[0108]

[0109] Calculate the distance mean within the preliminary screening collaboration subset , reflecting the average similarity level of the clients in the preliminary screening subset and the target client. Wherein, represents the number of clients in the preliminary screening subset.

[0110] According to the following formula:

[0111]

[0112] Calculate the distance standard deviation within the preliminary screening collaboration subset , reflecting the dispersion of the distance in the preliminary screening subset. Then go to step D2;

[0113] Step D2. According to the following formula:

[0114]

[0115] Constructing the fine screening threshold in the nth client under the kth iteration Where T is the total iteration round. As k increases (later iteration), γ k Approaches 1, only the most similar clients in the primary screening set are retained. Then go to step D3;

[0116] Step D3. For the nth client under the kth iteration, if the distance between client i and the classifier parameter of the nth client Client i is included in the final collaboration subset of the nth client. The final collaboration subset can be expressed as: Where, at the beginning of iteration The scale is large, and more potential collaborators are included to accumulate global knowledge; at the later stage of iteration The scale is small, and only high-similarity clients are retained to avoid interference from low-similarity clients. Then go to step D4;

[0117] Step D4. According to the following formula:

[0118]

[0119] For each client i in the final collaborator set Calculate the cosine similarity between its vectorized classifier weight vector And the of the nth client, which measures the parameter similarity between the two. Where, Indicates the classifier weight vector of the ith local client under the kth iteration, Indicates the classifier weight vector of the nth local client under the kth iteration. Then go to step D5;

[0120] Step D5. According to the following formula:

[0121]

[0122] The cosine similarity of each client n's collaborator is normalized to obtain the weight Of each collaborator i, ensuring that the sum of the weights is 1. Then according to the weight The classifier parameter group of all clients in the final collaborator set is weighted and aggregated to obtain the personalized classifier parameter Of the nth client. The calculation formula is:

[0123] ​

[0124] Then go to step D6;

[0125] Step D6. According to the following formula:

[0126]

[0127] The server calculates the global feature extractor parameters in an average aggregation manner Wherein, Indicates the first L-1 layer parameters of the MLP model corresponding to the nth local client under the kth iteration. Ensure that all clients use a unified basic module with general feature extraction capability.

[0128] Step E. Under the kth iteration, each client n downloads the feature extractor parameters and personalized classifier parameters obtained after the processing in step D under the k-1th iteration, combines the binary cross-entropy loss and the knowledge distillation loss, minimizes the loss through back propagation, obtains the updated local model parameter set, to achieve the training of the client model and enhance the model's ability to retain and migrate historical knowledge, and returns to step C until the model converges.

[0129] The above step E is specifically implemented as follows in actual application: steps E1 to E5.

[0130] Step E1. Under the kth iteration, the nth client downloads the global feature extractor parameters And the personalized classifier parameters The original local model parameter set is divided into feature extraction parameters and classifier parameters, which are The client first updates the local model's feature extractor parameters To Complete the synchronization of global general feature extraction capability. At the same time, keep the local classifier parameters Prepare for subsequent training. Then go to step E2;

[0131] Step E2. Freeze the updated feature extractor parameters of the local model Only set the downloaded customized classifier personalized classifier parameters To a trainable state. For any sample Wherein, x is the sample representation, y is the label of the sample, Is the local data set of client k. First, encode the input x through the frozen feature extractor To get the feature vector Then input z into the customized classifier Output the predicted probability distribution where g is the classifier mapping function with Softmax normalization.

[0132] Then, the following formula is used:

[0133]

[0134] The binary classification cross-entropy loss is used as the optimization objective, and the ρ-rounds of stochastic gradient descent (SGD) update fine-tuning is implemented, and then the customized classifier parameter update is to ensure that the customized classifier gradually adapts to the local data distribution, and the parameter is used as a “teacher model” to guide the local model training. Wherein, represents the gradient of the cross-entropy loss with respect to , and η v controls the fine-tuning step size. Then go to step E3;

[0135] Step E3. At the kth iteration, the local model parameter set of the nth client is denoted as The feature extractor parameters are unfrozen to allow them to adapt to the local feature distribution. And the teacher model is fixed to guide the knowledge and avoid drift.

[0136] Then, the following formula is used:

[0137] L CE (p w , y) = -[y·log(p w ) + (1-y)·log(1-p w )]

[0138] The binary classification cross-entropy loss is constructed, where the local feature extractor encodes the input x to obtain the feature vector Then, the z is input into the customized classifier to output the predicted probability distribution y represents the true label of the sample, and y = 1 indicates that the sample corresponds to a code defect, and y = 0 indicates that the sample corresponds to a code without defect. Then go to step E4;

[0139] Step E4. The following formula is used:

[0140]

[0141] The KL divergence loss is constructed for knowledge distillation, where p v is the fine-tuned teacher model, that is, the predicted probability distribution of the customized classifier. p w is the student model, that is, the predicted probability distribution of the local model. p v(0) represents the probability that the sample is predicted as "non-defective", p v (1) represents the probability that the sample is predicted as "defective". This loss guides the local model to absorb the integrated knowledge of the teacher model. Then go to step E5;

[0142] Step E5. According to the following formula:

[0143] L EN = L CE (p w , y) + L KL (p v || p w )

[0144] Construct a comprehensive loss that combines cross-entropy loss and KL divergence loss. Take L EN as the optimization goal, update the local model parameters by SGD:

[0145]

[0146] Where, represents the gradient of the comprehensive loss with respect to , η w controls the local training step size, and finally obtains the updated local model parameters

[0147] The technical scheme is designed based on a deep learning MLP model to construct a "feature extractor + classifier" hierarchical architecture, relying on a federated learning framework to realize multi-client collaborative training, while combining adaptive collaborative screening, personalized parameter aggregation and customized classifier guided knowledge distillation mechanisms to significantly improve software defect prediction performance while ensuring privacy security. Through client classifier parameter distance matrix calculation and clustering screening, irrelevant clients are avoided from participating in collaboration, effectively resolving the contradiction between privacy protection and collaboration efficiency; at the same time, the mean and standard deviation of the distance matrix are used to construct a double threshold, and the fine screening threshold is contracted with the iteration round, accurately capturing the feature distribution difference of the client under the condition of non-independent and identically distributed data, ensuring that the data distribution similarity of the collaborative client is high, solving the problem of "performance decline caused by blind aggregation" in traditional federated learning, making the aggregated model more suitable for local data characteristics; in addition, the cosine similarity weighted aggregation of the final collaborator's classifier parameters is combined with the hierarchical strategy of global feature extractor average aggregation and classifier personalized aggregation to balance the model's generality and personalization, and to overcome the pain point of insufficient generalization ability of single data training model; on this basis, the customized classifier generated by the server is fine-tuned locally to adapt to the local task, and the fine-tuned classifier is used as the teacher model to guide the local model training with the help of cross-entropy loss and KL divergence loss, realizing the distillation and fusion of collaborative knowledge and local knowledge, absorbing similar client integrated knowledge and retaining local model historical experience, solving the training shock problem caused by the inconsistency between the global model and the local task target, and significantly improving the prediction accuracy and stability.

Claims

1. A personalized federated learning method for software defect prediction fusion adaptive screening, characterized in that: Initialize the iteration number k = 1, and obtain the software defect prediction model for predicting whether the code in the software has defects according to the following steps; Step A. Select a preset number n of local clients as participating clients in the kth iteration. Each local client corresponds to a software project or a set of code modules. Each client initializes the model parameters based on the MLP (Multi-Layer Perceptron) structure, divides the model into a feature extractor parameter group, which refers to the first L-1 layer parameters of the MLP (used to extract the deep representation of the code features), and a classifier parameter group, which refers to the Lth layer parameters of the MLP (used for binary classification output), and then proceeds to Step B; Step B. In the kth iteration, each local client constructs its training sample set, which includes software module code strings with defect label 1 and non-defect label 0. A single sample is composed of the label corresponding to a single code string and the feature values of the preset target type features corresponding to the code string. Finally, a sample set corresponding to each local client is constructed, and then Step C is entered; Step C. In the kth iteration, the client uploads the feature extractor parameters and the classifier parameters to the server, which then vectorizes them. The server then calculates the classifier parameter distance matrix between the clients. To capture the differences in feature distribution between the clients, the server adaptively determines the similar group and the non-similar group by dynamically calculating the mean and standard deviation of the distance distribution, and obtains the preliminary cooperation subset of each client in the kth round. Then Step D is entered; Step D. The server further refines the classifier parameter group of each client n on its cooperation subset to obtain the final collaborator set. Then, individualized aggregation is performed for each client, and the cosine similarity method is used to calculate the weight of each final collaborator set for the client. The individualized classifier parameters are obtained after aggregation according to the weight. The feature extractor parameters are the average of the feature extractor parameters uploaded by the clients to the server. Then Step E is entered; Step E. In the kth iteration, each client n downloads the feature extractor parameters and individualized classifier parameters obtained after the processing in Step D in the k-1th iteration, combines the binary cross-entropy loss and the knowledge distillation loss, and minimizes the loss through backpropagation to obtain updated local model parameters, so as to train the client model and enhance the model's ability to retain and transfer historical knowledge, and return to Step C until the model converges.

2. The personalized federated learning method for software defect prediction fusion adaptive screening according to claim 1, characterized in that: In the kth iteration of Step A, each local client performs the following steps A1 and A2 to select and initialize the MLP model parameters; Step A1. Select a preset number n of local clients as participating clients in the kth iteration. Each local client uniquely corresponds to a software project or a set of independent code modules, ensuring the independence of the business scenarios of the data of each client and avoiding cross-scene data interference. Then Step A2 is entered; Step A2. Each participating client initializes the model parameters based on the MLP (Multi-Layer Perceptron) structure, assuming the total number of layers of the MLP model is L, and divides the model parameters into a feature extractor parameter group and a classifier parameter group wherein, represents the first L-1 layer parameters of the MLP model corresponding to the nth local client at the kth iteration, represents the Lth layer parameters of the MLP model corresponding to the nth local client at the kth iteration.

3. The personalized federated learning method for software defect prediction fusion adaptive screening according to claim 1, characterized in that: The respective local clients in the kth iteration of step C perform steps C1-C5 as follows, the server receives the uploaded classifier parameters from the clients and performs clustering analysis to obtain the preliminary screening collaboration subset of each client; Step Cl. Each local client uploads its feature extractor parameter set and classifier parameter set to the server, which upon receipt flattens each client's classifier weight matrix into a 1 -dimensional vector completes parameter vectorization. Then proceeds to Step C2; Step C2. According to the following formula: Compute the distance of the classifier parameters of any two clients i and n, where, denotes the classifier weight vector of the i-th local client at the k-th iteration, denotes the square of the L2 norm, denotes the maximum distance of the i-th client to all other clients, used to normalize the distance values to the interval [0, 1] and to eliminate the dimension effect. Construct the distance matrix of the classifier parameters of the clients at the k-th iteration Then go to step C3; Step C3. According to the following formula: Computing the mean of the off-diagonal elements of the distance matrix at the kth iteration μ k , reflecting the average level of the classifier parameter distances among all clients. Here, the denominator n(n-1) is the total number of off-diagonal elements of the distance matrix, n is the number of participating clients, and the total number of elements (n*n) is reduced by the n diagonal elements. The numerator is the cumulative sum of all off-diagonal elements . According to the following formula: calculating the standard deviation σ of the off-diagonal elements of the distance matrix at the kth iteration k reflecting the degree of dispersion of the classifier parameter distances between all clients, is the square of the deviation of the individual off-diagonal element from the mean. Then step C4 is entered. Step C4. According to the following formula: τ k = μ k + σ k Compute the dissimilarity group threshold τ at the kth iteration k for excluding clients with a significant difference in data distribution from the target client. Then go to step C5; Step C5. For the nth client at the kth iteration, if the client i is within the classifier parameter distance of the nth client then client i is included in the screening collaboration subset for the nth client. The final screening collaboration subset can be represented as:

4. The personalized federated learning method for software defect prediction fusion adaptive screening according to claim 1, characterized in that: In step D, the server performs steps D1-D6 as follows to further optimize the collaboration range and perform personalized parameter aggregation; Step D1. According to the following formula: Calculate the initial screening of collaborative subsets The mean distance within the initial subset reflects the average similarity level between the client and the target client within the initial subset. This indicates the number of clients within the initial screening subset. According to the following formula: Compute the screening collaboration subset The distance standard deviation within the screening subset, reflecting the dispersion of distances within the screening subset. Then go to step D2; Step D2. According to the following formula: Constructing the precision threshold in the nth client at the kth iteration where T is the total number of iteration rounds. As k increases (later iterations), γ k approaches 1, leaving only the most similar clients in the pre-screening set. Then go to step D3; Step D3. For the nth client in the kth iteration, if the client i is within the classifier parameter distance of the nth client then the client i is included in the final collaboration subset of the nth client. The final collaboration subset can be represented as: where the iteration is in the early stage the scale is large, and more potential collaborators are included to accumulate global knowledge; the iteration is in the later stage the scale is small, and only high-similar clients are reserved to avoid interference of low-similar clients with personalized aggregation. Then go to Step D4; Step D4. According to the following formula: For each client i within the final set of collaborators Compute its vectorized classifier weight vector Cosine similarity to the n-th client's measure of parameter similarity. Where, denotes the classifier weight vector of the i-th local client at the k-th iteration, denotes the classifier weight vector of the n-th local client at the k-th iteration. Then go to step D5; Step D5. According to the following formula: The cosine similarity of the collaborators of each client n is normalized to obtain the weight of each collaborator i Ensure that the sum of the weights is 1. Then, according to the weights The classifier parameter groups of all clients in the final collaborator set are weighted and aggregated to obtain the personalized classifier parameters of the nth client The calculation formula is: ​ Then go to step D6; Step D6. According to the following formula: The server calculates the global feature extractor parameters in an average aggregation manner wherein, denotes the parameters of the first L-1 layers of the MLP model corresponding to the nth local client at the kth iteration. It is ensured that all clients use a unified base module with general feature extraction capability.

5. The personalized federated learning method for software defect prediction fusion adaptive screening according to claim 1, characterized in that: In step E, the server performs steps E1-E5 as follows, each local client completes model training and parameter updating until the model converges. Step E1. At the kth iteration, the nth client downloads the global feature extractor parameters from the server with the personalized classifier parameters The local model parameters set is originally split into feature extractor parameters and classifier parameters, for The client first updates the feature extractor parameters of the local model with the global feature extractor parameters The synchronization of the global common feature extraction capability is completed. At the same time, the local classifier parameters are retained Prepare for subsequent training. Then go to Step E2; Step E2. Freeze the feature extractor parameters of the updated local model Downloaded custom classifier personalizes only the classifier parameters Set to trainable state. For any sample in the local training sample set where x is the sample representation, y is the label of the sample, is the local dataset of client k. First, encode the input x through the frozen feature extractor to get the feature vector z Then input z to the custom classifier Output the predicted probability distribution where g is the classifier mapping function, which contains the Softmax normalization. Then according to the following formula: with binary cross-entropy loss To optimize the objective, implement a ρ-rounds of stochastic gradient descent (SGD) update fine-tuning, followed by customizing the classifier parameter update as Ensuring the customized classifier gradually adapts to the local data distribution, and the parameter serves as a "teacher model" to guide the local model training. Wherein, denotes the gradient of the cross-entropy loss with respect to , η v controls the fine-tuning step size. Then go to step E3; Step E3. At the k-th iteration, the local model parameter set of the n-th client is denoted as Unfreeze the feature extractor parameters Allow it to adapt to the local feature distribution. And freeze the teacher model Fix the guiding knowledge and avoid drift. Then according to the following formula: L CE (p w , y) = -[y · log(p w ) + (1 - y) · log(1 - p w )] A binary cross-entropy loss is constructed, where the local feature extractor Encode the input x to get a feature vector Then input z into the customized classifier Output the predicted probability distribution y represents the true label of the sample, y = 1 indicates that the sample corresponds to a code defect, and y = 0 indicates that the sample corresponds to a code without defect. Then go to step E4; Step E4. According to the following formula: The KL divergence loss is constructed for knowledge distillation, where p v is the prediction probability distribution of the fine-tuned teacher model, i.e., the customized classifier. w is the prediction probability distribution of the student model, i.e., the local model. v (0) represents the probability that the sample is predicted as "defect-free", p v (1) represents the probability that the sample is predicted as "defective". This loss guides the local model to absorb the integrated knowledge of the teacher model. Then go to step E5; Step E5. According to the following formula: L EN = L CE (p w , y) + L KL (p v || p w ) The comprehensive loss is constructed by combining the cross-entropy loss and the KL divergence loss. To minimize L EN The local model parameters are updated by SGD for the optimization objective: in, Representing the overall loss The gradient, η w By controlling the local training step size, the updated local model parameters are finally obtained.

Images