A protein phosphorylation site prediction method based on federated learning

Abstract

The present invention discloses a protein phosphorylation site prediction method based on federated learning, which relates to the field of bioinformatics analysis technology, including: each client participating in federated learning pre-processes its protein sequence data and establishes its own training set and test set; each client uses a local data set to train a model and outputs the local model, prediction accuracy and local data set feature description to the committee; the committee tests the local models of each client, selects a local model that meets the standards, and optimizes the aggregated global model based on the feature description of each client data set; uses the fixed members of the federated learning committee to test the aggregated global model and determine whether to end federated learning. Therefore, the above-mentioned protein phosphorylation site prediction method based on federated learning can overcome the problem of data imbalance, effectively protect data privacy, and realize the sharing and utilization of multi-party data.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics analysis technology, and mainly relates to a protein phosphorylation site prediction method based on federated learning. Background Art

[0002] With the completion of the Human Genome Project and the advent of the post-genomic era, researchers have gained a deeper understanding of the importance of protein phosphorylation in biological regulation. Protein phosphorylation is a key post-translational modification of proteins, catalyzed by kinases that add phosphate groups to specific amino acid residues, such as serine, threonine, and tyrosine. When external stimuli reach the cell surface, they trigger a series of signaling events, ultimately affecting biological responses within the cell.

[0003] Phosphorylation, as a key regulatory mechanism of signal transduction, can transform external signals into intracellular biological effects such as cell proliferation, apoptosis, and differentiation. By controlling the phosphorylation state of proteins, cells can respond promptly and precisely to changes in the external environment. Phosphorylation also participates in the regulation of protein interactions, altering the conformation and charge state of proteins, affecting their binding ability and specificity with other proteins, and thereby regulating the establishment and stability of protein interaction networks. During cell proliferation and differentiation, phosphorylation ensures that cells divide and proliferate at the appropriate time and under appropriate conditions, regulates the activity of cell cycle proteins, and controls the progression of the cell cycle, thereby maintaining the balance and stability of cell proliferation. Therefore, the accurate prediction of protein phosphorylation sites is of great significance for understanding cellular regulatory mechanisms, disease diagnosis, and drug design.

[0004] Currently, the identification of phosphorylation sites relies primarily on experimental and computational methods. Experimental methods, such as mass spectrometry and antibody detection, can directly quantify and localize phosphorylation sites, but are limited by resources and costs, making them unsuitable for large-scale application. Computational methods offer advantages in prediction speed and cost-efficiency, but traditional computational methods are often limited by the scale and quality of data.

[0005] With the development of deep learning technology, deep learning based on multi-layer neural networks has made significant progress in the field of bioinformatics. Deep learning can automatically generate complex patterns and capture high-level abstractions, providing new insights into protein phosphorylation site prediction. However, deep learning typically requires large amounts of data for training, and data in the medical and disease fields is often subject to privacy protection restrictions, making data acquisition difficult. These methods are often trained on independent databases, which can lead to small datasets, low data heterogeneity, and poor model generalization.

[0006] Therefore, it is necessary to provide a method suitable for the task of multi-dimensional imbalanced phosphorylation site prediction, which can overcome the imbalance problems of data and scale, effectively protect data privacy, and realize the sharing and utilization of multi-party biomedical data. Summary of the Invention

[0007] The purpose of the present invention is to provide a protein phosphorylation site prediction method based on federated learning. This method comprehensively utilizes biomedical data from different medical institutions and laboratories, overcomes the imbalance problem of data among various institutions, and realizes efficient prediction of protein phosphorylation sites based on deep learning, while effectively protecting the data privacy of various medical institutions and laboratories.

[0008] To achieve the above objectives, the present invention provides a method for predicting protein phosphorylation sites based on federated learning, comprising the following steps:

[0009] S1. Preprocess the protein sequence data to obtain positive and negative samples. Random sampling is used to make the positive and negative samples independent and identically distributed. The training and test sets are divided. At the same time, the size, label ratio, and site ratio of the client's local data are statistically analyzed to form a feature description of the local data set.

[0010] S2. Establish a federated learning committee consisting of fixed members and temporary members. Fixed members own the global model test dataset and are responsible for generating the initial global model. Temporary members are selected by clients participating in federated learning and are responsible for testing the prediction accuracy of local models.

[0011] S3. Each client participates in federated learning, obtains the initial global model, and trains the model using the local dataset to obtain the local model and local prediction accuracy of each client;

[0012] S4. The local models, prediction accuracy, and dataset feature descriptions trained by each client are passed to the Federated Learning Committee for testing. The selection criteria are determined based on the prediction accuracy of the Federated Learning Committee. Local models that meet the selection criteria are selected for aggregation to generate a global model.

[0013] S5. The federated learning committee uses the global model test dataset to test the generated global model and decides to terminate federated learning or start the next round of federated learning based on the test results until convergence.

[0014] Preferably, the selection criteria for temporary members of the Federal Learning Committee include:

[0015] During the first round of federated learning, fixed members test the local model of each client to obtain the prediction accuracy of the federated learning committee. The difference between the prediction accuracy of the committee and the local model is calculated. The absolute value of the difference is used as the reliability of this round. The temporary members of the federated learning committee for this round are then selected from the clients based on the reliability.

[0016] In subsequent rounds of federated learning, the local models of each client are tested using the temporary members selected in the previous round to obtain the prediction accuracy of each temporary member. The difference between the prediction accuracy and the local model prediction accuracy is calculated, and the absolute values ​​of the differences are summed as the reliability of subsequent rounds. The temporary members of the federated learning committee for subsequent rounds are selected based on this reliability.

[0017] Preferably, local model training includes determining the window size, performing feature extraction using multiple feature blocks, concatenating feature maps along the feature dimension, generating a phosphorylation prediction score through a fully connected layer, and outputting the prediction accuracy of the local model.

[0018] Preferably, local models that meet the selection criteria are selected to participate in the aggregation, including:

[0019] First, the local models of non-Federated Learning Committee clients are tested using the local test set through the temporary members of the FLC to obtain the prediction accuracy of each temporary member.

[0020] Secondly, the difference between the prediction accuracy of the temporary member and the prediction accuracy of the local model is calculated, and the absolute value of each difference is summed as the reliability of the corresponding local model;

[0021] Then, the selection standard line is determined according to the reliability distribution, and local models that are below the standard are eliminated, and then the local models that meet the selection criteria are globally aggregated.

[0022] Preferably, determining the selection standard line includes sorting the reliability of each client local model, calculating the upper and lower quartiles and the interquartile range, and determining the selection standard line k as follows:

[0023] k=q1-t*iqr;

[0024] Where q1 is the upper quartile, iqr is the interquartile range, and t is the ratio of iqr.

[0025] Preferably, the local models that meet the selection criteria are globally aggregated, including assigning corresponding weights to the local models participating in the aggregation according to the feature description of the dataset of each client, and then obtaining the global model as follows:

[0026]

[0027] Where, is the parameter of the i-th local model in the l-th round of federated learning, n is the number of clients participating in the global model aggregation, m i is the weight of the i-th local model.

[0028] Therefore, the present invention adopts the above-mentioned protein phosphorylation site prediction method based on federated learning, which has the following technical effects:

[0029] (1) The federated learning method solves the data “island” problem caused by only analyzing and training highly private data locally.

[0030] (2) Multiple local models are screened for performance and then aggregated to improve training efficiency, save time and computing power costs, achieve more accurate prediction of phosphorylation sites, and provide a more reliable data prediction tool.

[0031] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a general flow chart of the model federated learning training in an embodiment of a protein phosphorylation site prediction method based on federated learning;

[0033] Figure 2 The present invention is a data processing flow chart in an embodiment of a protein phosphorylation site prediction method based on federated learning. DETAILED DESCRIPTION

[0034] The present invention can be explained in more detail by the following examples. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following examples.

[0035] like Figure 1 As shown, the present invention provides a protein phosphorylation site prediction method based on federated learning, comprising the following steps:

[0036] S1. Each client preprocesses the local protein sequence data, such as Figure 2 shown.

[0037] First, for the prediction of general sites, all repeated sequences were deleted, and then the protein data with the number of amino acid residue sequences lower than the window size were removed.

[0038] Secondly, from the filtered protein site data, the phosphorylation sites and nearby fragment data of all verified amino acid sites are extracted as positive samples; for negative samples, a subset of a specific number of other sites and their nearby fragment information are randomly selected and fused with the positive samples to form a client local dataset, which is then divided into a training dataset and a test dataset.

[0039] In order to construct independent and identically distributed data, for a data set with m samples, it is necessary to mix its positive and negative samples and use a random algorithm to permute the positions of the sample data in the data set.

[0040] Analyze the imbalance of the client's local dataset itself (covering label imbalance and site imbalance) and the imbalance between clients (covering data scale imbalance, label imbalance and site imbalance) to form a data imbalance feature description.

[0041] S2. To support federated learning, a federated learning committee must be established, consisting of one permanent member and several temporary members to evaluate the effectiveness of federated learning. Specifically, if 20 clients participate in federated learning and temporary members are selected at a ratio of 30%, a seven-member federated learning committee will be formed.

[0042] The selection rules for temporary members are as follows: a reliability scoring criterion, e, is defined to filter out unreliable clients. Specifically, after the client undergoes federated learning training, each client is tested to obtain a local prediction accuracy. During the initial selection process, the local model and the locally measured accuracy are transmitted to the committee. A fixed member then tests each client's local model to obtain the prediction accuracy. The local accuracy of each local model is subtracted from the accuracy measured by the fixed member, and the absolute value is taken as the reliability for this round. Six temporary members are then selected from the lowest to highest reliability level. In subsequent rounds, the six temporary members selected in the previous round will be responsible for testing the local models of all other clients. In this way, each local model obtains six prediction accuracy rates, which are subtracted from the local model's phosphorylation prediction accuracy to obtain the absolute value. The sum is then used to obtain the reliability level. The next six temporary members are selected from the lowest to highest reliability level.

[0043] The six temporary members selected in each round will form a seven-member federated learning committee together with the permanent members, which will determine the selection criteria for each client's local model participating in the global model aggregation.

[0044] S3. For each round of federated learning, the federated learning committee will provide the starting global model. In the first round, fixed members will generate the initial global model. In other rounds, the global model gathered by the committee will serve as the starting global model for the next round. Each client will then use local data to train the starting global model and test its prediction accuracy.

[0045] First, the local protein sequence is input into the model with a total of k DC-CNN blocks to form sequence feature clusters Among them, I and L k Denotes the size of the amino acid symbol dictionary and the local window size of the corresponding phosphorylation site, respectively. In this embodiment, a one-hot encoding scheme for encoding protein sequences is adopted, so I is set to 21.

[0046] Secondly, by analyzing the window size configurations of different DC-CNN blocks, three effective window sizes were determined, namely 15, 33, and 51. Each DC-CNN block contains a one-dimensional convolutional layer that performs convolution operations along the length of the protein sequence and applies activation functions such as ReLU to achieve nonlinear transformations.

[0047] For the kth DC-CNN block, the first convolutional layer generates the feature map for the block for:

[0048]

[0049] Where, f k is the activation function, W k is the weight matrix, is the bias term.

[0050] In addition, in order to enhance the generalization ability of the model and reduce the threat of overfitting during training, the dropout algorithm is used after the convolutional layer to randomly discard a part of the neurons.

[0051] Afterwards, Intra-BCL is introduced to enhance the propagation of phosphorylation information in the DC-CNN block. This connects all preceding convolutional layers with the following convolutional layers, i.e., concatenating feature maps along the feature dimension, thus enhancing the model's ability to capture phosphorylation information. Accordingly, the sequence feature is formed by concatenating the feature map and the output of all previous layers as the input to the i-th convolutional layer, as follows:

[0052]

[0053] Where, is the feature map generated by the i-th convolutional layer, C is the number of convolutional layers in each DC-CNN block, is the bias term of the i-th convolutional layer. When the network depth increases, the abstract features of the previous layer at different levels are more easily transferred to the current layer.

[0054] The protein phosphorylation site sequences created by each DC-CNN block are further integrated along the first dimension through intra-BCL:

[0055]

[0056] Where h f The concatenation of the feature maps generated by the C-th convolutional layer, is the bias term of the C-th convolutional layer. In this way, multiple feature maps are connected to each other through the plane layer and converted into a one-dimensional tensor.

[0057] Subsequently, a fully connected neural network is used to create a softmax function to obtain the phosphorylation prediction score and output the prediction accuracy of the trained model.

[0058] S4. The Federated Learning Committee evaluates the reliability of the local models trained by each client and selects local models that meet the standards to participate in global aggregation. The details are as follows:

[0059] First, each client packages the trained local model, prediction accuracy, and dataset feature description and passes it to the Federated Learning Committee.

[0060] Second, temporary members of the Federated Learning Committee test the local models of the remaining clients using their local test datasets. Each client's local model receives six test values, which are then combined with the client's own measured prediction accuracy to determine the reliability of the client's local model. Based on this reliability distribution, the committee will establish criteria for selecting client local models for participation in the global model aggregation process.

[0061] Next, the reliability of each client's local model is ranked, the upper and lower quartiles and the interquartile range are calculated, and the selection standard line k is determined as follows:

[0062] k=q1-t*iqr;

[0063] Wherein, q1 is the upper quartile, iqr is the interquartile range, and t is the ratio of iqr, which is 0.5 in this embodiment.

[0064] Afterwards, the client local models with accuracy lower than k are eliminated, and the parameters of the remaining local models are aggregated. During the aggregation process, the imbalance of the dataset size, label distribution, and site type distribution between the clients is weighted and calculated to obtain the global model as follows:

[0065]

[0066] Where, is the i-th model parameter of the l-th round of federated learning, n is the number of clients, m i is the weight of the i-th model.

[0067] S5. The committee tests the global model based on the global model test dataset to obtain the prediction accuracy. If the target is achieved, federated learning will end and the global model will be passed to the model demander. If the target is not achieved, a new round of federated learning will continue until the global model achieves the target.

[0068] Therefore, the protein phosphorylation site prediction method based on federated learning provided by the present invention can overcome the problem of data imbalance, effectively protect data privacy, and realize the sharing and utilization of multi-party data.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A protein phosphorylation site prediction method based on federated learning, characterized in that: The following steps are involved: S1. Preprocess the protein sequence data to obtain positive and negative samples. Random sampling is used to make the positive and negative samples independent and identically distributed. The training and test sets are divided. At the same time, the size, label ratio, and site ratio of the client's local data are statistically analyzed to form a feature description of the local data set. S2. Establish a federated learning committee consisting of fixed members and temporary members. Fixed members own the global model test dataset and are responsible for generating the initial global model. Temporary members are selected by clients participating in federated learning and are responsible for testing the prediction accuracy of local models. Among them, the selection criteria for temporary members of the Federal Learning Committee include: During the first round of federated learning, fixed members test the local model of each client to obtain the prediction accuracy of the federated learning committee. The difference between the prediction accuracy of the committee and the local model is calculated. The absolute value of the difference is used as the reliability of this round. The temporary members of the federated learning committee for this round are then selected from the clients based on the reliability. In subsequent rounds of federated learning, the local models of each client are tested using the temporary members selected in the previous round to obtain the prediction accuracy of each temporary member. The difference between the temporary members and the local model prediction accuracy is calculated, and the absolute value of the difference is summed as the reliability of the subsequent rounds. The temporary members of the federated learning committee in each subsequent round are selected based on the reliability. S3. Each client participates in federated learning, obtains the initial global model, and trains the model using the local dataset to obtain the local model and local prediction accuracy of each client; S4. The local models, prediction accuracy, and dataset feature descriptions trained by each client are passed to the Federated Learning Committee for testing. The selection criteria are determined based on the prediction accuracy of the Federated Learning Committee. Local models that meet the selection criteria are selected for aggregation to generate a global model. Among them, local models that meet the selection criteria are selected to participate in the aggregation, including: First, the local models of non-Federated Learning Committee clients are tested using the local test set through the temporary members of the FLC to obtain the prediction accuracy of each temporary member. Secondly, the difference between the prediction accuracy of each temporary member and the prediction accuracy of the local model is calculated, and the absolute value of each difference is summed as the reliability of the corresponding local model; Then, the selection criteria are determined based on the reliability distribution, and local models that are below the criteria are eliminated. Finally, the local models that meet the selection criteria are globally aggregated. S5. The federated learning committee uses the global model test dataset to test the generated global model and decides to terminate federated learning or start the next round of federated learning based on the test results until convergence.

2. A protein phosphorylation site prediction method based on federated learning according to claim 1, characterized in that: Local model training includes determining the window size, using multiple feature blocks to extract features, and concatenating feature maps along the feature dimension. Then, a fully connected layer is used to generate phosphorylation prediction scores and output the prediction accuracy of the local model.

3. The protein phosphorylation site prediction method based on federated learning according to claim 1, characterized in that: Determine the selection criteria, including sorting the reliability of each client's local model, calculating the upper and lower quartiles and the interquartile range, and determining the selection criteria k as follows: k=q1-t*iqr; Where q1 is the upper quartile, iqr is the interquartile range, and t is the ratio of iqr.

4. A protein phosphorylation site prediction method based on federated learning according to claim 1, characterized in that: Global aggregation is performed on local models that meet the selection criteria, including assigning corresponding weights to the local models participating in the aggregation based on the feature description of the dataset of each client, and then obtaining the global model as follows: Where, is the parameter of the i-th local model in the l-th round of federated learning, n is the number of clients participating in the global model aggregation, m i is the weight of the i-th local model.

Images