Data set construction method for B cell epitope prediction

Abstract

The invention relates to a data set construction method for B-cell antigen epitope prediction, which is an invention made by using a natural science fund of Hebei Province, and the project name of the fund is B-cell antigen epitope prediction research based on a self-learning mechanism. The method comprises the following steps: a, establishing an epitope sample set, a non-epitope sample set and a candidate sequence set; b, extracting a positive sample seed set and a negative sample seed set from the epitope sample set and the non-epitope sample set; c, training an algorithm based on deep Qlearning by using the candidate sequence set, and optimizing a Q network to obtain a non-epitope automatic filter; and d, selecting non-epitope sequences and sequences in the IEDB database as a sample data set, and selecting epitopes and non-epitope sequences with the same number to form a data set. According to the method, non-epitope data is learned in protein sequence fragment data, a high-quality sample data set can be obtained, and the accuracy of antigen epitope classification is improved.

Description

Technical Field

[0001] The invention relates to a data set construction method for predicting B cell antigen epitopes, and belongs to the technical field of data processing. Background Art

[0002] The present invention is an invention made using the Natural Science Foundation of Hebei Province. The project name of the fund is “Research on prediction of B cell antigen epitopes based on self-learning mechanism, project number: F2022302001”.

[0003] B cell antigen epitope prediction based on machine learning has important applications in vaccine development and disease test kit development. It can quickly identify antigen epitopes, speed up R&D progress, and save R&D investment.

[0004] Based on machine learning, the prediction of B cell antigen epitopes mostly adopts supervised learning, that is, classification is performed on the data set. Classification model training must be based on a high-quality sample data set. The Chinese invention patent with publication number CN114242169A discloses a method for predicting antigen epitopes for B cells. The method first forms a pre-training set PT; in each episode of the Q_learning algorithm, the Q agent takes any 8 consecutive amino acid residues in the primary sequence of the protein as the state, and selects k residues from the 12 consecutive residues behind each state and incorporates them into the state as the first action; selects one of the n complementary classifiers as the second action option, searches in PT according to the continuous action search method, and gives an immediate reward to the amino acid sequence searched by the tendency reward rule, calculates the Q value and updates it until the change in the value function is less than 1% to end the training; then uses the trained strategy to search for amino acid sequences in the primary sequence of the protein, and the selected classifier is used for classification. The invention greatly enhances the prediction ability of B cell antigen epitopes through automatic iteration and improves the accuracy of antigen epitope classification. B cell antigen epitope model training usually requires two types of samples: epitope and non-epitope. Most epitope samples used for B cell antigen epitope prediction come from the IEDB (http: / / www.iedb.org / ) database of the immune epitope database, which has high reliability. However, there is no reliable source for non-epitope samples. In the literature, they are generally randomly selected from the remaining sequences after removing the experimentally verified epitopes from the primary sequence of the protein. The random selection method is unscientific and imprecise. On the one hand, the literature only gives the epitope sequences that the experimenters think are most likely, missing many possible epitope results, and the random selection method may cause those omitted epitopes to be selected as non-epitope samples. On the other hand, our scientific understanding of protein sequences makes it difficult to randomly select high-quality non-epitope samples. In fact, many literatures have also pointed out that this non-epitope sample selection method is unreliable. Up to now, there is a lack of scientific and effective methods for obtaining non-epitope samples, and the low quality of non-epitope samples leads to low quality of data sets used for classification model training, which seriously affects the accuracy of antigen epitope classification models. Summary of the invention

[0005] The purpose of the present invention is to address the shortcomings of the prior art and provide a method for constructing a data set for predicting B cell antigen epitopes, so as to obtain a sample data set and improve the accuracy of antigen epitope classification.

[0006] The problem described in the present invention is solved by the following technical solution:

[0007] A method for constructing a data set for predicting B cell antigen epitopes, the method comprising the following steps:

[0008] a. Establish epitope sample set EPT using IEDB database, epitope prediction literature and uniport database L , non-epitope sample set NEPT L and the candidate sequence set CPT L ;

[0009] b. Using the CD-hit redundancy removal method to perform epitope sample set EPT L and non-epitope sample set NEPT L De-redundancy operations are performed respectively, and the epitope sample set EPT after de-redundancy is L and non-epitope sample set NEPT L Randomly extract the same number of samples from the set to form a high-quality positive sample seed set Seed P And negative sample seed set Seed NP ;

[0010] c. Set the candidate sequence CPT L As a training data set, training is performed based on the deep Q_learning algorithm, and the epitope sample set EPT is used L and non-epitope sample set NEPT L The Q network is optimized, and after training is completed, a non-epitope automated filter PC is obtained;

[0011] d. Extract the primary sequence of antigen protein from the Uniport database, use the non-epitope automatic filter PC to automatically screen the sequence according to the set sequence length L, and use the screened non-epitope sequences and the sequences of sequence length L in the IEDB database as a sample data set. Use the cd-hit de-redundancy method to deduplicate the sample data set, select the same number of epitope and non-epitope sequences, and form a data set for B cell antigen epitope prediction.

[0012] The above-mentioned method for constructing a data set for predicting B cell antigen epitopes, wherein the epitope sample set EPT is established L , non-epitope sample set NEPT L and the candidate sequence set CPT L The specific method is as follows:

[0013] Download B cell epitope sequences from the IEDB database, retrieve B cell epitope sequence data with more than 5 entries, and form epitope sample sets EPT according to sequence length L. L ; Collect non-epitope data from epitope prediction literature, extract non-epitope data that appear more than 5 times from the collected data as negative samples, and form non-epitope sample sets NEPT according to sequence length L L; Collect several protein primary sequences from the uniport database and divide them into sequence fragments according to the length L to form the candidate sequence set CPT L .

[0014] The above-mentioned method for constructing a dataset for predicting B cell antigen epitopes is used to establish an epitope sample set EPT L , non-epitope sample set NEPT L and the candidate sequence set CPT L When, the sequence length L is: 7 <L<25。

[0015] The above-mentioned method for constructing a data set for predicting B cell antigen epitopes includes the following specific methods for training the deep Q_learning algorithm:

[0016] Each time, training is carried out according to a fixed length L. During training, the candidate sequence set CPT L As the state space, a sequence is randomly selected as the initial state. The action space contains two actions, "yes" and "no". The batch value is k. The training process is as follows:

[0017] Use s t represents the state at time step t, a t represents the action at time step t; θ i represents the learning network parameters at the i-th iteration, Q(s t ,a t θ i ) represents the state s at the i-th iteration t Next, perform action a t At the beginning of training, initialize a learning rate r between 0 and 1, a discount factor γ between 0 and 1, a state list, a set number of rounds, and learning network parameters θ i ;

[0018] At time step t=1, the initial action a0, initial state s0, and batch value k are input into a convolutional neural network containing 3 convolutional layers and 2 linear layers to obtain an initial state and action corresponding to Q(s0, a0; θ0), where θ0 represents the initial value of the learning network parameters;

[0019] At each time step t, t≥1, the agent observes the current state s t , and select an action a from the action space t , then the environment calculates the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i ), and then the state st The corresponding action a t The reward value is updated to the state list. When the set number of time steps is reached, it is called a round. When the entire training reaches the set number of rounds, a training is completed.

[0020] The above-mentioned method for constructing a data set for predicting B cell antigen epitopes, wherein the environment calculates the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i ) are as follows:

[0021] The agent selects m based on each state and the reward value of the action at that time. a QUR t (s t ,a t θ i ) action, the sequence set P1 that executes the "yes" action is merged into Seed P Create a new collection Seed P ', the sequence set P2 that performs the "No" action is merged into Seed NP Create a new collection Seed NP ', extract the collection Seed P ' and Seed NP 'Based on the physical and chemical characteristics of amino acids in the sequence, the support vector machine SVM is used to train the classifier C', and then the classification scores of the sets P1 and P2 are normalized by C', and the classifier scores are divided into four levels from small to large, and the corresponding weights P are assigned to each level. t : 1, 0.8, 0.8, 1, and these weights are combined into a vector in sequence order and multiplied with the calculated reward value, and then iterative optimization learning is performed in the Q network. In the learning network, the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i ) is calculated as:

[0022] Q(s t ,a t θ i )=P t R t +γQ(s t+1 ,a t+1 θ i )

[0023] Where R t Is to perform action a tThe immediate reward after t+1 ,a t+1 θ i ) is the next state s t+1 Next, perform action a t+1 The maximum value reward;

[0024] At the i-1th iteration, the state s t The value reward V(θ i-1 ) is calculated as:

[0025]

[0026] in, is the expectation, V(s t+1 ,a t+1 θ i-1 ) is the next state s in the learning network t+1 The value reward below;

[0027] The formula for minimizing the loss of gradient descent of network parameters at the i-th iteration is:

[0028]

[0029] The above-mentioned method for constructing a dataset for predicting B cell antigen epitopes is used to construct a dataset for any state sequence s t The maximum consecutive coincidence matching reward is calculated according to the formula:

[0030]

[0031] in and For state s t EPT L and non-epitope sample set NEPT L Sequence matching reward, and The states s t In EPT L and NEPT L The maximum number of consecutive coincidences in , and The states s t In EPT L and NEPT L The maximum continuous overlap ratio in state s, L is the maximum continuous overlap ratio in state s t The number of amino acids in the corresponding sequence;

[0032] At time step t, perform action a t The immediate reward R t The calculation formula is:

[0033]

[0034] in Beneficial Effects

[0035] The present invention adopts an integrated learning framework based on deep Q_learning to learn non-epitope data in noisy protein sequence fragment data. Compared with traditional methods, it can effectively improve the quality of non-epitope samples, thereby obtaining high-quality sample data sets and improving the accuracy of antigen epitope classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with the accompanying drawings.

[0037] Figure 1 It is a schematic diagram of the composition principle of this method.

[0038] Figure 2 This is a diagram of the dataset construction framework.

[0039] The symbols in the text represent the following: IEDB database: Immune Epitope Database (IEDB) funded by the National Institute of Allergy and Infectious Diseases (NIAID);

[0040] Uniport database: It is the protein database with the richest information and the broadest resources, which is formed by integrating the data of three major databases: Swiss-Prot, TrEMBL and PIR-PSD;

[0041] EPT L : epitope sample collection;

[0042] NEPT L : Non-epitope sample collection;

[0043] CPT L : Candidate sequence set;

[0044] cd-hit: a redundancy removal method;

[0045] Seed P : positive sample seed set;

[0046] Seed NP : Negative sample seed set;

[0047] Q network: refers to the network trained with the deep Q_learning algorithm;

[0048] PC: non-epitope automated filter;

[0049] Sequence length L: refers to the number of L amino acids in the sequence (from the primary sequence of the protein) Batch value k: refers to the number of samples in each batch of batch training;

[0050] Time step t: refers to the state value of a time series;

[0051] s t : represents the state at time step t;

[0052] a t : represents the action at time step t;

[0053] θ i : represents the learning network parameters at the i-th iteration;

[0054] Q(s t ,a t θ i ): indicates the state s at the i-th iteration t Next, perform action a t Value rewards;

[0055] Learning rate r: is a number between 0 and 1;

[0056] Discount factor γ: is a number between 0 and 1;

[0057] a0: is the initial action;

[0058] s0: is the initial state;

[0059] Q(s0,a0;θ0): is a set of network parameters obtained by training the convolutional neural network;

[0060] θ0: represents the initial value of the learning network parameters;

[0061] Q t (s t ,a t θ i ) The corresponding action with the largest action value;

[0062] C': is a classifier obtained through training;

[0063] Pt: refers to weight;

[0064] Refers to expectations;

[0065] For state s t EPT L The sequence matching reward in ;

[0066] For state st Divided into the non-epitope sample set NEPT L Sequence matching reward in;

[0067] For state s t In EPT L The maximum number of consecutive coincidences that occur;

[0068] For state s t In NEPT L The maximum number of consecutive coincidences that occur;

[0069] For state s t In EPT L The proportion of the maximum number of consecutive coincidences that occur;

[0070] For state s t In NEPT L The proportion of the maximum number of consecutive coincidences that occur;

[0071] L: For state s t The number of amino acids in the corresponding sequence. Specific implementation manner

[0072] The present invention provides a method for constructing a data set for B-cell epitope prediction. The method adopts an integrated learning framework based on deep Q-learning to learn non-epitope data in noisy protein sequence fragment data, including a protein sequence fragment candidate set, a policy learner based on deep Q-learning, and a non-epitope sample filter. Through this method, non-epitope samples can be automatically labeled. The method includes the following steps:

[0073] a. Download B-cell epitope sequences from the IEDB database, and then retrieve B-cell epitope sequence data with more than 5 included entries, and form an epitope sample set EPT according to the sequence length L L , generally taking 7 < L < 25; collect non-epitope data from epitope prediction literature, extract non-epitope data that appears more than 5 times as negative samples, and form a non-epitope sample set NEPT according to the sequence length L L , generally taking 7 < L < 25; collect several protein primary sequences from the uniport database, and uniformly divide them into sequence fragments according to the length L to form a candidate sequence set CPT L ;

[0074] b. Use the cd-hit redundancy removal method for the epitope sample set EPT obtained in step a LAnd the non-epitope sample set NEPT L Perform redundancy removal operations on them respectively to find samples with high similarity. Then delete the samples with high similarity and update the epitope sample set EPT L And the non-epitope sample set NEPT L . Randomly select the same number of samples from the redundancy-removed epitope sample set EPT L And the non-epitope sample set NEPT L to form the positive sample seed set Seed P and the negative sample seed set Seed NP .

[0075] c. Use the candidate sequence set CPT L as the training data set to train the deep Q-learning algorithm. Each time, train according to a fixed length L, where 7 < L < 25. During training, use the candidate sequence set CPT L as the state space, randomly select a sequence from it as the initial state. The action space contains two actions, "yes" and "no", and the batch value is k, that is, input k candidate sequences each time, where 1 < k < 100. The training process of the deep Q-learning algorithm is as follows:

[0076] Use s t to represent the state at time step t, and a t to represent the action at time step t; θ i represents the learning network parameters at the i-th iteration, and Q(s t , a t ; θ i ) represents the value reward for performing action a t in state s t at the i-th iteration. At the beginning of training, initialize a learning rate r that is any number between 0 and 1, a discount factor γ that is any number between 0 and 1, a list of states, a set number of rounds, and the learning network parameters θ i .

[0077] The deep Q-learning algorithm is a new algorithm formed by combining Q-learning and convolutional neural networks. At time step t = 1, input the initial action a0, the initial state s0, and the batch value k into a convolutional neural network containing 3 convolutions and 2 linear layers to obtain a Q(s0, a0; θ0) corresponding to the initial state and action, where θ0 represents the initial value of the learning network parameters.

[0078] At each time step t, t ≥ 1, the agent observes the current state s t , and selects an action a t, then the environment calculates the state s at the i-th iteration according to the "synchronous optimization strategy" and "matching reward rule" t Next, perform action a t The value reward Q(s t ,a t θ i ), and then the state s t The corresponding action a t The reward value is updated to the state list, and the agent updates the state by executing actions. In a given training data set, reaching the set number of time steps according to this method is called an episode. When the entire training reaches the set number of episodes, a training is completed, and the trained deep Q_learning algorithm becomes an excellent non-epitope automatic filter PC.

[0079] d. Extract several primary sequences of antigen proteins from the Uniport database, and use the non-epitope automatic filter PC to automatically filter the sequences according to the set sequence length L. The non-epitope sequences that have been filtered out are taken together with the sequences of sequence length L in the IEDB database as a sample data set, and the sample data set is de-redundanted using the cd-hit de-redundancy method. Samples with high similarity are found and deleted, and an excellent data set can be formed by selecting the same number of epitope and non-epitope sequences.

[0080] The "synchronous optimization strategy" in step c uses the "action selection strategy" to select actions, and the sequence set P1 of the agent executing the "yes" action is incorporated into the Seed P Create a new collection Seed P ', the sequence set P2 that performs the "No" action is merged into Seed NP Create a new collection Seed NP '. Extract the collection Seed P ' and Seed NP 'Based on the physical and chemical characteristics of amino acids in the sequence, the support vector machine SVM is used to train the classifier C', and then the classification scores of the sets P1 and P2 are normalized by C', and the classifier scores are divided into four levels from small to large, and the corresponding weights P are assigned to each level. t : 1, 0.8, 0.8, 1, and organize these weights into a vector in sequence order, perform dot multiplication with the reward value calculated in the "matching reward rule", and then perform iterative optimization learning in the Q network.

[0081] In the learning network, the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i) is calculated as:

[0082] Q(s t ,a t θ i )=P t R t +γQ(s t+1 ,a t+1 θ i ) (1)

[0083] Where R t Is to perform action a t The immediate reward after t+1 ,a t+1 θ i ) is the next state s t+1 Next, perform action a t+1 The maximum value reward.

[0084] In the learning network, the state s at the i-1th iteration t The value reward V(θ i-1 ) is calculated as:

[0085]

[0086] in, is the expectation, V(s t+1 ,a t+1 θ i-1 ) is the next state s in the learning network t+1 The value reward below.

[0087] In the learning network, the formula for minimizing the loss of the gradient descent of the network parameters at the i-th iteration is:

[0088]

[0089] The "matching reward rule" in step c follows the state sequence s t EPT with epitope sample set L and non-epitope sample set NEPT L The proportion and number of the maximum continuous overlaps are calculated.

[0090] For any state sequence s t The maximum consecutive coincidence matching reward is calculated according to the formula:

[0091]

[0092] in and For state s t EPT Land non-epitope sample set NEPT L Sequence matching reward, and The states s t In EPT L and NEPT L The maximum number of consecutive coincidences in , and The states s t In EPT L and NEPT L The maximum continuous overlap ratio in state s, L is the maximum continuous overlap ratio in state s t The number of amino acids in the corresponding sequence.

[0093] At time step t, perform action a t The immediate reward R t The calculation formula is:

[0094]

[0095] in

[0096] The "action selection strategy" in the above steps is: the agent selects the action with the largest reward value based on each state and the action reward value at that time. Action as a result.

[0097] Advantages of the present invention:

[0098] The present invention adopts an epitope prediction method based on continuous action search, which, on the one hand, breaks the epitope prediction scheme based on the "window method" and realizes the autonomous selection of epitope sequences; on the other hand, the classification accuracy is improved by introducing complementary classifiers.

[0099] This method is suitable for the construction of data sets of different lengths required for epitope prediction using the window method, and develops the seeds required for strategy learning of deep Q_learning.

[0100] Considering the huge number of sequences as states, the present invention adopts a deep learning strategy, which makes training easy and ensures excellent learning effect.

[0101] This method adopts the "synchronous optimization strategy" and "matching reward rule" to ensure the iterative optimization of the learning strategy, while also using the respective characteristics of non-epitope and epitope samples in learning.

[0102] The method involves automated screening of non-epitope samples, which is rapid, effective and accurate.

[0103] Introduction to Q-Learning Algorithm:

[0104] (1) In the Q-Learning algorithm, each state-action has a corresponding Q value. Therefore, the learning process of the Q-Learning algorithm is to repeatedly iteratively calculate the Q value of the learning state-action pair. Finally, the optimal action strategy obtained by the learner is to choose the action corresponding to the maximum Q value under state s. The Q value Q(s, a) of action a under state s is defined as the cumulative reward value obtained after the learner executes action a under state s and continues to execute according to a certain action strategy. The basic equation for updating its Q value is:

[0105] Q(s t ,a t )=Q(s t ,a t )+α[Rs t +γmaxQ(s t+1 , a)-Q(s t ,at)] (6-1)

[0106] In formula (6-1): a is the optional action under the state; Rs t is the immediate reward given by the environment in state s at time t; α is the learning rate; Q(s t ,a t ), the evaluation value of the state-action (s, a) at time t.

[0107] (2) The steps of the Q-Learning algorithm are shown in Table 1:

[0108] Table 1. Pseudocode of Q_learning algorithm

[0109]

[0110] (3) Deep Q-Learning Algorithm

[0111] In deep Q-Learning, due to the powerful expressive power of neural networks, the traditional Q table is replaced by a deep neural network, which is used to approximate the Q function. The input of the network is the state of the environment, and the output is the expected reward of each action. The problem of labels and loss functions. The goal of reinforcement learning is to get a better strategy, which requires a more appropriate value estimation function.

[0112] One implementation step of the deep Q-Learning algorithm:

[0113] 1) Network initialization: Initialize the Q network Q(s,a;θ) and the target network Q(s,a;θ-). The two networks have the same structure but independent parameters. The parameters θ- of the target network are copied and updated from the Q network regularly.

[0114] 2) Observation initialization: Obtain the initial state information s_1 by randomly selecting actions in the environment.

[0115] 3) For each step t, perform the following operations:

[0116] Strategy execution: select action a according to the ε-greedy strategy, randomly select an action with probability ε, and select argmax with probability 1-ε a Q(st,a;θ).

[0117] Take action: Perform action a in the environment and observe reward r and new state s_{t+1}.

[0118] Calculate the target: For each sample drawn, calculate the corresponding target Q value: Q(s,a)=r+γmaxa′Q(s′,a′). Where r is the reward, s′ is the next state, γ is the discount factor, and a′ is the optimal action for the next state. Using the target network to calculate the target Q value can improve the stability of the algorithm.

[0119] Gradient Descent: Perform gradient descent on the network parameters to minimize the loss function:

[0120] L(θ)=(1 / N)*Σj(yj-Q(sj,aj;θ)) 2 , where N is the number of samples drawn.

[0121] 4) Target network update: After a certain number of steps, the parameters θ of the Q network are used to update the parameters θ- of the target network.

[0122] 5) Loop: Repeat step 4 until the agent shows stable learning performance or reaches a predetermined number of iterations.

[0123] 6) Termination: When the agent’s performance meets certain criteria or reaches the iteration limit, the training process ends.

[0124] (4) CD-hit redundancy removal method

[0125] cd-hit is a data set de-redundancy method that can calculate the data with high similarity in the data set and delete it to ensure the specificity of the data. The working principle of cd-hit is: cluster all sequences according to the parameter settings, and output the longest sequence in each group of clusters as the representative sequence, and give the name of each sequence under each group of clusters for similarity analysis. Let's briefly introduce its usage below.

[0126] cd-hit needs to be operated under Linux system. After decompressing the compressed package, enter the software path and directly enter the command: make to compile.

[0127] The input file of cd-hit is only one fasta format file. Generally speaking, cd-hit clusters the gene or protein sequences of several samples, so the sequences of these samples need to be aggregated together as input files. This can be achieved through the cat command in Linux system:

[0128] cat a.fasta b.fasta c.fasta>all.fasta

[0129] Among them, a.fasta, b.fasta, and c.fasta are three sample protein sequences in fasta format, and all.fasta is the summarized sequence, which is used as the input sequence of cd-hit in the analysis.

[0130] It is worth noting that there cannot be any sequence with the same sequence name in the three sample sequences, otherwise an error will occur. Therefore, the sample name is generally added before each sample sequence name during analysis to avoid duplication. The sequence name is the content before the space at the beginning of the line with ">" in the fasta file.

[0131] There are many parameters that can be adjusted when running cd-hit. The running command is (parameters are only examples):

[0132] / path-to-cdhit-4.6.8 / cd-hit-i all.fasta-o new.fa-c 0.8-aS 0.8-d 0

[0133] Important parameters of cd-hit:

[0134] -i: input file, fasta format

[0135] -o: output file prefix. There are two output files, namely fasta format sequence file and clustering information file ending with .clstr

[0136] -c: If the ratio of the bp of the shorter sequence aligned to the longer sequence to the bp of the shorter sequence itself exceeds this value, the shorter sequence will be clustered into one group. The default value is 0.9

[0137] -d: The length of the sequence name in each cluster group in the cluster information file. If it is set to 0, the full sequence name will be used.

[0138] -aL: controls the stringency of the sequence alignment. The default value is 0. If it is set to 0.8, it means that the alignment interval should account for 80% of the representative (long) sequence.

[0139] -AL: Parameter that controls the stringency of sequence alignment. The default value is 99999999. If it is set to 40, it means that the non-aligned interval of the sequence must be shorter than 40bp.

[0140] -aS: Parameter that controls the strictness of short sequence alignment. The default value is 0. If it is set to 0.8, it means that the alignment interval must account for 80% of the short sequence.

[0141] -AS: Parameter that controls the strictness of short sequence alignment. The default value is 99999999. If it is set to 40, it means that the non-alignment interval of the short sequence must be shorter than 40bp.

[0142] Professional term explanation

[0143] (1) Protein primary sequence, a sequence consisting of 20 amino acids, such as ADFCEGHIKLST

[0144] (2) B cell epitope, which is part of the primary sequence of a protein and can be composed of partial sequences

[0145] (3) An “episode” is the process from the beginning to the end of an agent executing a strategy in an environment.

[0146] Agent: A software or hardware mechanism that takes corresponding measures by interacting with the surrounding environment. Agent is also called intelligent body in Chinese. It is often called Q agent in Q_learning algorithm. It is the subject that explores and learns in the environment.

[0147] Action: The various possible actions that the agent can take. Although actions are somewhat self-explanatory, we still need to allow the agent to choose from a set of discrete, possible actions.

[0148] Environment: There is an interaction and response relationship between the external environment and the agent. The environment takes the agent's current state and action as input, and the agent's reward and next state as output. The environment is everything outside the agent.

[0149] State: A state is a specific, immediate situation that an agent finds itself in, including a specific place, time, and instantaneous configuration that relates the agent to other important things.

[0150] Reward: A reward is a type of feedback that we use to measure the success or failure of the agent's various actions in a given state.

[0151] Discount factor: The discount factor is a multiplier. Future rewards discovered by the agent are multiplied by this factor to weaken the cumulative effect of such rewards on the agent's current action selection. This is the core of reinforcement learning, which is to gradually reduce the value of future rewards in order to give more weight to recent actions. This is crucial for the paradigm based on the "delayed action" principle.

[0152] Policy: A function that takes a state observation as input and outputs an action. It is the strategy that the agent uses to determine the next action based on the current state. It can map different states to various actions to promise the highest reward.

[0153] Value: It is defined as the discounted long-term expected reward (not short-term reward) for the current state under a specific policy. Short-term reward is the instantaneous reward that the agent gets for being in a certain state and taking a specific action. Value is the total amount of reward that the agent expects to get from a certain state to the future.

[0154] Q-value or action-value: The difference from "value" is that Q-value requires an additional parameter, which is the current action. It refers to the long-term reward generated by the current state under a specific strategy for an action.

[0155] Bellman equations: It is a set of equations that decompose the value function into an immediate reward plus a discounted future value.

[0156] Value iteration: This is an algorithm that iteratively improves the estimate of the value to calculate the function with the best state value. The algorithm first initializes the value function to any random value, and then repeatedly updates the Q value and the value function until they converge.

[0157] Policy iteration: Since the agent is only concerned with finding the optimal policy, which sometimes converges before the value function, policy iteration should not repeatedly improve the estimate of the value function, but redefine the policy at each step and calculate the value based on the new policy until the policy converges.

[0158] Episode: Reinforcement learning describes the interaction between the agent and the environment. At a certain moment, the agent t Given an action a t , the environment gives feedback r based on the actions taken by the agent t , and the environment is also updated t= 1, the agent makes new actions based on the feedback. This cycle repeats from the beginning to the end, s1, a1, r1, s2, a2, r2, ... is called an episode.

[0159] Timestep: In reinforcement learning, each interaction between the agent and the environment constitutes a timestep.

[0160] Protein primary sequence in uniport:

[0161] For example, A0A0A0MQN9:

[0162] MEQPPPLAPEPASARSRRRREPESPPAPIPLFGARTVVQRSPDEPALSKAEFVEKVRQSNQACHDGDF

[0163] HTAIVLYNEALAVDPQNCILYSNRSAAYMKTQQYHKALDDAIKARLLNPKWPKAYFRQGVALQYLGRH

[0164] ADALAAFASGLAQDPKSLQLLVGMVEAAMKSPMRDTLEPTYQQLQKMKLDKSPFVVVSVVGQELLTAG

[0165] HHGASVVVLEAALKIGTCSLKLRGSVFSALSSAHWSLGNTEKSTGYMQQDLDVAKTLGDQTGECRAHG

[0166] NLGSAFFSKGNYREALTNHRHQLVLAMKLKDREETIVCVSRGRYTATSSQLHTGWGETSQGLPSAASS

[0167] ALSSLGHVYTAIGDYPNALASHKQCVLLAKQSKDDLSEARELGNMGAVYIAMGDFENAVQCHEQHLRI

[0168] AKDLGSKREEARAYSNLGSAYHYRRNFDKAMSYHNCVLELAQELMEKPIEMRAYAGLGHAARCMQDLE

[0169] RAKQYHEQQLGIAEDLKDRAAEGRASSNLGIIHQMKGDYDTALKLHKTHLCIAQELSDYAAQGRAYGN

[0170] MGNAYNALGMYDQAVKYRHQELQISMEVNDRASQASTHGNLAVAYQALGAHDRALQHYQNHNLIAREL

[0171] RDIQSEARALSNLGNFHCSRGEYVQAAPYYEQYLRLAPDLQDMEGGKVCHNLGYAHYCLGNYQEAVK

[0172] YYEQDLALAKDLHDKLSQAKAYCNLGLAFKALLNFAKAEECQKYLLSLAQSLDNSQAKFRALGNLGDI

[0173] FICKKDINGAIKFYEQQLGLSHHVKDRRLEASAYAALGTAYRMVQKYDKALGYHTQELEVYQELSDLP

[0174] GECRAHGHLAAVYMALGKYTMAFKCYQEQLELGRKLKEPSLEAQVYGNMGITKMNMNVMEDAIGYFEQ

[0175] QLAMLQQLSGNESVLDRGRAYGNLGDCYEALGDYEEAIKYYEQYLSVAQSLNRMQDQAKAYRGLGNGH

[0176] RATGSLQQALVCFEKRLVVAHELGEASNKAQAYGELGSLHSQLGNYEQAISCLERQLNIARDMKDRAL

[0177] ESDAACGLGGVYQQMGEYDTALQYHQLDLQIAEETDNPTCQGRAYGNLGLTYESLGTFERAVVYQEQH

[0178] LSIAAQMNDLVAKTVSYSSLGRTHHALQNYSQAVMYLQEGLRLAEQLGRREDEAKIRHGLGLSLWASG

[0179] NLEEAQHQLYRASALFETIRHEAQLSTDYKLSLFDLQTSSYQALQRVLVSLGHHDEALAVAERGRTRA

[0180] FADLLVERQTGQQDSDPYSPITIDQILEMVNAQRGLVLYYSLAAGYLYSWLLAPGAGILKFHEHYLGD

[0181] NSVESSSDFQAGSSAALPVATNSTLEQHIASVREALGVESYYSRACASSETESEAGDIMEQQLEEMNK

[0182] QLNSVTDPTGFLRMVRHNNLLHRSCQSMTSLFSGTVSPSKDGTSSLPRRQNSLAKPPLRALYDLLIAP

[0183] MEGGLMHSSGPVGRHRQLVLVLEGELYFVPFALLKGSASNEYLYERFTLIAVPAVRSLGPHSKCHLRK

[0184] TPPTYSSSTTMAAVIGNPKLPSAVMDRWLWGPMPSAEEEAFMVSELLGCQPLVGSMATKERVMSALTQ

[0185] AECVHFATHVSWKLSALVLTPNTEGNPAGSKSSFGHPYTIPESLRVQDDASDVESISDCPPLRELLLT

[0186] AADLLDLRLSVKLVVLSSSQEANGRVTADGLVALTRAFLAAGAQCVLVALWPVPVAASKMFVHAFYSS

[0187] LLNGLKASASLGEAMKVVQSSKAFSHPSNWAGFTLIGSDVKLNSPSSLIGQALTEILQHPERARDALR

[0188] VLLHLVEKSLQRIQNGQRNAMYTSQQSVENKVGGIPGWQALLTAVGFRLDPAASGLPAAVFFPTSDPG

[0189] DRLQQCSSTLQALLGLPNPALQALCKLITASETGEQLISRAVKNMVGMLHQVLVQLQACEKEQDFASA

[0190] PIPVSLSVQLWRLPGCHEFLAALGFDLCEVGQEEVILKTGKQASRRTTHFALQSLLSLFDSTELPKRL

[0191] SLDSSSSLESLASAQSVSNALPLGYQHPPFSPTGADSIASDAISVYSLSSIASSMSFVSKPEGGLEGG

[0192] GPRGRQDYDRSKSTHPQRATLPRRQTSPQARRGASKEEEEYEGFSIISMEPLATYQGEGKTRFSPDPK

[0193] QPCVKAPGGVRLSVSSKGSVSTPNSPVKMTLIPSPNSPFQKVGKLASSDTGESDQSSTETDSTVKSQE

[0194] ESTPKLDPQELAQRILEETKSHLLAVERLQRSGGPAGPDREDSVVAPSSTTVFRASETSAFSKPILSH

[0195] QRSQLSPLTVKPQPPARSSSLPKVSSPATSEVSGKDGLSPPGSSHPSPGRDTPVSPADPPLFRLKYPS

[0196] SPYSAHISKSPRNTSPACSAPSPALSYSSAGSARSSPADAPDEKVQAVHSLKMLWQSTPQPPRGPRKT

[0197] CRGAPGTLTSKRDVLSLLNLSPRHGKEEGGADRLELKELSVQRHDEVPPKVPTNGHWCTDTATLTTAG

[0198] GRSTTAAPRPLRLPLANGYKFLSPGRLFPSSKC

[0199] Epitope sequence sample:

[0200] For example: LASSDTGESDQSSTE

[0201] Non - epitope sequence sample:

[0202] For example: GTLTSKRDVLSLLNLSPR.

Claims

1. A method for constructing a data set for predicting B cell antigen epitopes, characterized in that: The method comprises the following steps: a. Establish epitope sample set EPT using IEDB database, epitope prediction literature and uniport database L , non-epitope sample set NEPT L and the candidate sequence set CPT L ; b. Using the CD-hit redundancy removal method to perform epitope sample set EPT L and non-epitope sample set NEPT L De-redundancy operations are performed respectively, and the epitope sample set EPT after de-redundancy is L and non-epitope sample set NEPT L Randomly select the same number of samples from the set to form the positive sample seed set Seed P And negative sample seed set Seed NP ; c. Set the candidate sequence CPT L As a training data set, the deep Q_learning algorithm is trained, and the epitope sample set EPT is used L and non-epitope sample set NEPT L Optimize the Q network to make it a non-epitope automated screener PC based on the deep Q_learning algorithm; d. Extract the primary sequence of antigen protein from the Uniport database, use the non-epitope automatic filter PC to automatically screen the sequence according to the set sequence length L, and use the screened non-epitope sequences and the sequences of sequence length L in the IEDB database as a sample data set. Use the cd-hit de-redundancy method to deduplicate the sample data set, select the same number of epitope and non-epitope sequences, and form a data set for B cell antigen epitope prediction.

2. A method for constructing a data set for predicting B cell antigen epitopes according to claim 1, characterized in that: Establishing epitope sample set EPT L , non-epitope sample set NEPT L and the candidate sequence set CPT L The specific method is as follows: Download B cell epitope sequences from the IEDB database, retrieve B cell epitope sequence data with more than 5 entries, and form epitope sample sets EPT according to sequence length L. L ; Collect non-epitope data from epitope prediction literature, extract non-epitope data that appear more than 5 times from the collected data as negative samples, and form non-epitope sample sets NEPT according to sequence length L L ; Several protein primary sequences were collected from the uniport database and divided into sequence fragments according to the length L to form the candidate sequence set CPT. L .

3. A method for constructing a data set for predicting B cell antigen epitopes according to claim 2, characterized in that: Epitope Sample Set EPT L , non-epitope sample set NEPT L and the candidate sequence set CPT L When, the sequence length L is: 7 <L<25。 4. The method for constructing a data set for predicting B cell antigen epitopes according to claim 1, characterized in that: The specific method for training based on the deep Q_learning algorithm is: Each time, training is carried out according to a fixed length L. During training, the candidate sequence set CPT L As the state space, a sequence is randomly selected as the initial state. The action space contains two actions, "yes" and "no". The batch value is k. The training process is as follows: Use s t represents the state at time step t, a t represents the action at time step t; θ i represents the learning network parameters at the i-th iteration, Q(s t ,a t θ i ) represents the state s at the i-th iteration t Next, perform action a t At the beginning of training, initialize a learning rate r between 0 and 1, a discount factor γ between 0 and 1, a state list, a set number of rounds, and learning network parameters θ i ; At time step t=1, the initial action a0, initial state s0, and batch value k are input into a convolutional neural network containing 3 convolutional layers and 2 linear layers to obtain an initial state and action corresponding to Q(s0, a0; θ0), where θ0 represents the learning network parameters at the 0th iteration; At each time step t, t≥1, the agent observes the current state s t , and select an action a from the action space t , then the environment calculates the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i ), and then the state s t The corresponding action a t The reward value is updated to the state list. When the set number of time steps is reached, it is called a round. When the entire training reaches the set number of rounds, a training is completed.

5. The method for constructing a data set for predicting B cell antigen epitopes according to claim 1, characterized in that: The environment calculates the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i ) are as follows: The agent selects m based on each state and the reward value of the action at that time. a QUR t (s t ,a t θ i ) action, the sequence set P1 that executes the "yes" action is merged into Seed P Create a new collection Seed P ', the sequence set P2 that performs the "no" action is merged into Seed NP Create a new collection Seed NP ', extract the collection Seed P ' and Seed NP 'Based on the physical and chemical characteristics of amino acids in the sequence, the support vector machine SVM is used to train the classifier C', and then the classification scores of the sets P1 and P2 are normalized by C', and the classifier scores are divided into four levels from small to large, and the corresponding weights P are assigned to each level. t : 1, 0.8, 0.8, 1, and these weights are combined into a vector in sequence order and multiplied with the calculated reward value, and then iterative optimization learning is performed in the Q network. In the learning network, the state s at the i-th iteration t Next, perform action a t The value reward Q(s t ,a t θ i ) is calculated as: Q(s t ,a t ;θ i )=P t R t +γQ(s t+1 ,a t+1 ;θ i ) Where R t Is to perform action a t The immediate reward after t+1 ,a t+1 θ i ) is the next state s t+1 Next, perform action a t+1 The maximum value reward; At the i-1th iteration, the state s t The value reward V(θ i-1 ) is calculated as: in, is the expectation, V(s t+1 ,a t+1 θ i-1 ) is the next state st in the learning network +1 The value reward below; The formula for minimizing the loss of gradient descent of network parameters at the i-th iteration is:

6. The method for constructing a data set for predicting B cell antigen epitopes according to claim 1, characterized in that: For any state sequence s t The maximum consecutive coincidence matching reward is calculated according to the formula: in and For state s t EPT L and non-epitope sample set NEPT L Sequence matching reward, and The states s t In EPT L and NEPT L The maximum number of consecutive coincidences in , and The states s t In EPT L and NEPT L The maximum continuous overlap ratio in state s, L is the maximum continuous overlap ratio in state s t The number of amino acids in the corresponding sequence; At time step t, perform action a t The immediate reward R t The calculation formula is: in

Images