Phage construction model training method, phage construction method, and terminal device

Abstract

The application discloses a phage construction model training method, a phage construction method and a terminal device. The training method trains a phage construction model through a phage sample, so that the phage construction model learns a mapping relationship between a sequence state of the phage and next element information of the phage, thereby enabling, in actual application, for a phage with missing tail proteins, the element sequence of the tail proteins of the phage to be filled through the trained phage construction model, so that a phage with a complete sequence is constructed, and efficient construction of the phage is realized.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a method for training a phage construction model, a method for constructing phages, and a terminal device. Background Technology

[0002] Phage construction is a technique commonly used in biomedical research and applications. Its basic principle is to utilize the special parasitic nature of bacteriophages and modify them through methods such as genome editing to give them a wider host range and stronger infectivity.

[0003] Most existing methods for constructing bacteriophages are based on modifying the natural parasitic nature of bacteriophages. These methods include natural selection and artificial selection. Natural selection utilizes the random mutation and selection mechanisms inherent in the natural evolution of bacteriophages, selecting phage strains with stronger infectivity and a wider host range through multiple passages and screenings. Artificial selection, on the other hand, utilizes the genetic characteristics of specific host strains, selecting phage strains with high infectivity for that host strain through propagation and screening.

[0004] However, the above-mentioned phage construction methods require extensive phage culture and screening to find phage strains with ideal infectivity and host range, which is time-consuming and labor-intensive. Summary of the Invention

[0005] The purpose of this application is to provide a phage construction model training method, a phage construction method, a terminal device, and a storage medium, aiming to solve the problem of low efficiency in existing phage construction methods. The embodiments of this application construct a phage construction model through a machine learning algorithm, thereby efficiently constructing phages using the phage construction model.

[0006] The first aspect of this application provides a method for training a bacteriophage construction model, including:

[0007] Obtain phage samples and phage construction models to be trained; wherein, the phage samples include the sequence state of the phage samples; the initial sequence state of the phage samples is a phage element sequence in which the tail protein is hollowed out and the element sequence to be filled is a phage element sequence.

[0008] The phage sample is input into the phage construction model, so that the phage construction model determines the elements to be filled based on the current sequence state of the phage sample, until a complete phage sequence is constructed.

[0009] Based on the constructed complete phage sequence, the parameters of the phage construction model are updated until the update stop condition is met, thus completing the training of the phage construction model.

[0010] By training a phage construction model using phage samples, the model learns the mapping relationship between the current sequence state of the phage and the information of the next element of the phage. In practical applications, for phages lacking tail proteins, the trained phage construction model can fill in the element sequence of the tail protein of the phage to construct a complete phage sequence, thus achieving efficient phage construction.

[0011] In one possible implementation, the cell construction model is a phage construction model based on a reinforcement learning network; the step of inputting the phage sample into the phage construction model, so that the phage construction model determines the elements to be filled based on the current sequence state of the phage sample, until a complete phage sequence is constructed, includes:

[0012] The phage sample is input into the phage construction model to determine the element to be filled at each time step in the current strategy parameter training round based on the initial sequence state of the phage sample and the current strategy parameters of the phage construction model, so as to construct a complete phage sequence.

[0013] In one possible implementation, updating the parameters of the phage construction model based on the constructed complete phage sequence until an update stopping condition is met includes:

[0014] Determine the cumulative reward obtained after performing component filling actions at multiple time steps in the current policy parameter training round;

[0015] Based on the sequence state of the complete phage and the accumulated reward, the strategy parameters of the phage construction model are updated until the update stop condition is met.

[0016] In one possible implementation, determining the cumulative reward obtained after performing component-filling actions at multiple time steps in the current policy parameter training round includes:

[0017] Based on the preset action value function, determine the reward value obtained after performing the action in the last time step of the current strategy parameter training round;

[0018] The Monte Carlo search algorithm is used to sample the reward value obtained after the action is executed in the last time step of multiple policy parameter training rounds, so as to determine the reward value obtained after the action is executed in other time steps of each policy parameter training round based on the sampling results; wherein, the other time steps refer to the time steps other than the last time step in each policy parameter training round.

[0019] The cumulative reward obtained by the current policy parameter is determined based on the reward values ​​obtained after performing actions at all time steps in the training round of the current policy parameter.

[0020] In one possible implementation, prior to inputting the phage sample into the phage construction model, the method further includes:

[0021] Obtain the network parameters of the pre-trained phage fitting model;

[0022] The network parameters are used as the initial values ​​for the strategy parameters of the phage construction model.

[0023] By assigning the network parameters of the phage fitting model to the initial values ​​of the policy parameters of the phage construction model to pre-train the phage construction model, the problem of reward sparsity faced by the phage construction model in the reinforcement learning policy due to the large number of phage elements that can be selected by the policy model in reinforcement learning, which leads to an excessively large action space for phage element selection, and consequently makes it difficult for the phage construction model to converge, can be solved.

[0024] In one possible implementation, the phage fitting model is trained as follows:

[0025] A generative adversarial network (GAN) model is constructed, and the generative model of the GAN model generates a phage fitting sequence based on the input random vector, so as to use the phage fitting sequence as a fitted phage sample.

[0026] Obtain natural phage samples, and use the discriminative model of the generative adversarial network model to distinguish between the natural phage samples and the fitted phage samples;

[0027] Based on the identification results, the generative model and the discriminative model are trained alternately to obtain the phage fitting model.

[0028] In one possible implementation, the generative model of the generative adversarial network model generates a phage fitting sequence based on an input random vector, including:

[0029] The randomly generated vector is input into the generation model to obtain the first phage element;

[0030] The i-th phage element is input into the generation model to obtain the (i+1)-th phage element;

[0031] Repeat the steps of inputting the i-th phage element into the generation model to obtain the (i+1)-th phage element until the number of phage elements obtained is equal to T; where i and T are integers greater than or equal to 1.

[0032] A second aspect of this application provides a method for constructing a bacteriophage, comprising:

[0033] A phage with a missing tail protein and a sequence of elements to be filled is input into the phage construction model to obtain a complete phage sequence; wherein, the phage construction model is a phage construction model trained using the phage construction model training method provided in the first aspect above.

[0034] A third aspect of this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the phage construction model training method provided in the first aspect above, or implements the phage construction method provided in the second aspect above.

[0035] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the phage construction model training method as provided in the first aspect above, or implements the phage construction method as provided in the second aspect above.

[0036] It is understood that the beneficial effects of the second, third and fourth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating a phage construction model training method provided in an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of multiple element action filling trajectories obtained by the phage construction model provided in this application embodiment in a reinforcement learning environment when multiple training rounds are executed with the same strategy;

[0039] Figure 3 This is a structural block diagram of a terminal device provided in an embodiment of this application;

[0040] Figure 4 This is a structural block diagram of a terminal device provided in another embodiment of this application. Detailed Implementation

[0041] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0042] In the embodiments of this application, the entity executing the process can be a terminal device. This terminal device includes, but is not limited to, servers, computers, smartphones, and tablets, as well as other devices capable of executing the phage construction model training method and / or phage construction method provided in this application.

[0043] As described in the background section, existing phage construction methods are time-consuming and labor-intensive due to the characteristics of traditional biological experiments. Furthermore, existing phage construction methods require avoiding unpredictable mutations during phage construction, which could lead to unexpected changes in the activity and biological characteristics of the phage.

[0044] Among other existing phage construction methods, there are artificial construction methods using genome editing and molecular cloning techniques. The most commonly used method utilizes CRISPR / Cas gene editing technology, which uses precise gene editing and cloning techniques to construct phage lines with specific functions and characteristics. However, the scope of modification using this method is limited by genome structure and restriction nuclease sites, making it difficult to meet certain specific modification needs.

[0045] In summary, existing phage construction methods suffer from drawbacks due to the characteristics of traditional biological experiments, such as being time-consuming and labor-intensive, having unpredictable modification results, and being unable to meet specific needs, thus failing to achieve the effect of efficiently constructing phages.

[0046] Based on this, this application provides a phage construction model training method. This method trains a phage construction model using machine learning, which is used to construct the element sequences of phage tail proteins. In this method, phage samples with their tail proteins removed are first input into the phage construction model for training. The phage construction model learns the mapping relationship between the current sequence state of the phage and the next element to be filled in the tail protein, thereby predicting the complete phage sequence. The phage construction model is then iteratively trained based on the predicted complete phage sequence until the iteration stops, ultimately obtaining a phage construction model capable of constructing phages lacking tail proteins. This allows for the efficient construction of phages with tail proteins using this phage construction model.

[0047] The following explains the relevant technical terms used in this application:

[0048] Machine learning: Machine learning is the science of artificial intelligence. Its main research focus is on artificial intelligence, particularly how to improve the performance of specific algorithms through experiential learning. Machine learning uses data or past experience to optimize the performance of computer programs.

[0049] Generative models: In machine learning, generative models can be used to directly model data (e.g., sample data based on the probability density function of a variable), or to establish conditional probability distributions between variables. In other words, a generative model can obtain the probability distribution of a given dataset and then generate new data based on probability.

[0050] Reinforcement Learning (RL) is a learning method where an agent learns through trial and error. Rewards gained through interaction with the environment guide behavior, with the goal of maximizing the agent's reward. Reinforcement learning differs from supervised learning in connectionist learning, primarily in the reinforcement signal. In reinforcement learning, the reinforcement signal provided by the environment evaluates the quality of the action (usually a scalar signal), rather than telling the reinforcement learning system (RLS) how to produce the correct action. Because the external environment provides limited information, the reinforcement learning system must learn through its own experience. In this way, the reinforcement learning system acquires knowledge in the action-evaluation environment and improves its action plans to adapt to the environment.

[0051] Agent: The problem context of reinforcement learning is a binary structure: environment and agent. An agent is primarily an entity capable of interacting with the environment and altering its state, mainly through actions. In short, an agent is a neural network model consisting of multiple neural network layers that maps observations to actions.

[0052] Generative Adversarial Networks (GANs): A GAN consists of a generative model and a discriminative model. The generative model is responsible for capturing the distribution of the sample data, while the discriminative model is typically a binary classifier that distinguishes between real data and generated samples. The optimization process of this model is a binary minimax game problem. During training, one side (the discriminative network or the generative network) is fixed, and the parameters of the other model are updated. This process is repeated iteratively until the generative model is able to estimate the distribution of the sample data.

[0053] See Figure 1 , Figure 1 This is a flowchart illustrating the phage construction model training method provided in the first embodiment of this application. In this embodiment, the phage construction model training method includes steps S11 to S13:

[0054] S11, Obtain phage samples and phage construction models to be trained.

[0055] The phage sample includes the sequence state of the phage sample; the initial sequence state of the phage sample is a phage element sequence in which the tail protein is hollowed out and the element sequence to be filled is phage element sequence.

[0056] Most bacteriophages consist of a head and a tail, but some exist without a tail. Larger bacteriophage heads are often hexagonal prisms, containing nucleic acid and surrounded by a protein coat. A few bacteriophages also possess an envelope. The tail of a bacteriophage is the organ that contacts the bacteria; it is primarily composed of protein and contains multiple elements. Each element is a sequence of amino acids.

[0057] Before implementing the method of this embodiment, a batch of phages with tail structures is first obtained. The staff removes all the tail proteins from the phages with tail structures to obtain phages with all tail proteins removed. At this time, there is no component information in the tail of the phage, and the phage with the tail proteins removed is used as a phage sample.

[0058] In this embodiment, the phage sample used to train the phage construction model includes the phage sequence state. The phage sequence state is a sequence composed of multiple elements. During each training iteration of the phage construction model, the initial sequence state of the phage sample is a phage sequence with the tail protein removed and the element sequence to be filled in.

[0059] In some embodiments, the terminal device may respond to a sample entry instruction input by the user and read the phage sample uploaded by the user through the sample upload interface. The terminal device may also obtain the phage sample from a local database or a cloud database.

[0060] In some embodiments, the phage construction model can be constructed using a Long Short-Term Memory (LSTM) network, a Recurrent Neural Network (RNN), a reinforcement learning network, or other network structures. The phage construction model can be an untrained phage construction model or a pre-trained phage construction model whose network parameters still need optimization.

[0061] S12, the phage sample is input into the phage construction model, so that the phage construction model determines the element to be filled based on the current sequence state of the phage sample, until a complete phage sequence is constructed. This complete phage sequence is a phage with a sequence of elements padded at the end. In one embodiment, after the phage sample is input into the phage construction model, the phage construction model can learn the next element of the phage based on the current sequence state of the phage sample.

[0062] For example, for a phage with its tail protein hollowed out, the number of elements to be filled in the tail is n, where n is an integer greater than or equal to 1. After the phage sample is input into the phage construction model, the phage construction model predicts the first element T1 to be filled into the tail based on the initial sequence state S0 of the phage sample. The model outputs a second sequence state S1, which is the next sequence state that the phage sample migrates to from the initial sequence state S0. The obtained second sequence state S1 is then input into the phage construction model for processing to obtain a third sequence state S3... This process of repeatedly outputting the next sequence state of the phage based on its current sequence state continues until a complete phage sequence is constructed.

[0063] In another embodiment, after the phage sample is input into the phage construction model, the phage construction model can directly predict the multiple elements to be filled in the phage tail based on the current sequence state of the phage sample. For example, for a phage with its tail protein hollowed out, the number of elements to be filled in its tail is n, where n is an integer greater than or equal to 1. After the phage sample is input into the phage construction model, the phage construction model predicts the multiple elements to be filled as (T1, T2, ..., T...) based on the initial sequence state S0 of the phage sample. n This outputs the complete phage sequence (S0, T1, T2, ..., T...). n ).

[0064] In one possible scenario, the phage construction model can be trained using a supervised training method. In this training method, the terminal device acquires phage samples, which are samples carrying filler element sequence tags. The phage samples can be (S0, (T1', T2', ..., T...). n The sequence is given by (T1`, T2`, ..., T)), where S0 is the initial sequence state of the phage sample, and (T1`, T2`, ..., T) is the sequence of the phage sample. n `) represents the fill element sequence label, T1`, T2`, ..., T n` represents the first element label, the second element label, ..., the nth element label filled in the tail of the phage, respectively. The terminal device inputs the phage sample into the phage construction model, and the phage construction model outputs the predicted result (T1, T2, ..., Tn) of the filled element sequence based on the initial sequence state S0 of the phage sample. n ), and based on the labels (T1`, T2`, ..., T n `) and the predicted results output by the model (T1, T2, ..., T) n The phage construction model is iteratively trained.

[0065] In another possible scenario, the phage construction model can be trained using an unsupervised training method. In this method, the terminal device acquires unlabeled phage samples and inputs the initial sequence state S0 of the phage samples into the phage construction model. The phage construction model then generates a phage (S0, T1) filled with the first element based on the initial sequence state S0, and feeds this information back to the phage construction model. The phage construction model then generates a phage (S0, T1, T2) filled with the second element based on the phage filled with the first element, and so on, until a complete phage sequence is generated. This embodiment can be implemented using other unsupervised network models, such as generative models.

[0066] S13, Based on the constructed complete phage sequence, update the parameters of the phage construction model until the update stop condition is met, thus completing the training of the phage construction model.

[0067] In this embodiment, the phage construction model determines whether the current model parameters meet the model optimization objective based on the constructed complete phage sequence, and then completes the model training process based on the judgment result.

[0068] In some embodiments, the update stopping condition may be: the number of iterations during model training reaches a preset iteration threshold, or the difference between the objective function value of the model in two training iterations is less than a preset difference threshold. The iteration threshold and the difference threshold can be set based on experience or actual needs.

[0069] This application embodiment utilizes phage samples to train a phage construction model. The phage construction model learns the mapping relationship between the current sequence state and the elements to be filled, thereby obtaining a phage construction model capable of constructing phages lacking tail proteins. This model enables efficient phage construction. Because this construction method is based on dataset learning and model training, it can efficiently discover and optimize key factors and parameters in phage construction. Furthermore, due to the predictability of artificial intelligence methods, compared to traditional phage construction methods, the phage construction method of this embodiment can avoid unpredictable mutations and accurately meet different application requirements.

[0070] In one possible implementation, the phage construction model is a phage construction model based on a reinforcement learning network; then S12 may include:

[0071] The phage sample is input into the phage construction model to determine the element to be filled at each time step in the current strategy parameter training round based on the initial sequence state of the phage sample and the current strategy parameters of the phage construction model, so as to construct a complete phage sequence.

[0072] Reinforcement learning is a type of Markov Decision Process (MDP). MDPs are built upon a set of interacting objects, namely agents and the environment, and include elements such as states, actions, policies, and rewards.

[0073] In this embodiment, a Markov decision process model M is defined in the phage construction model.<S,A,P,r,p0> In this model, M represents the Markov decision process model, S represents the set of states (composed of sequences of phage elements), A represents the set of actions (composed of selected phage elements), P represents the state transition probability, and P(s,a,s^') represents the probability of reaching state s^' after performing action a in state s. In this model, when the current state is s, the action is a, and the next state is s^', the state transition probability P(s,a,s^') = 1, and for other next states s^", P(s,a,s^") = 0. r represents the reward function. Since the quality and activity of the phage can only be determined when the phage sequence is complete, a reward is only given when the last action is performed. p0 represents the initial state distribution, and p0(s) represents the probability of the initial state being s. In this embodiment, the initial state of the samples is the same for each training iteration.

[0074] When training a phage construction model based on a reinforcement learning network, the terminal device typically includes phage samples.<s,a,r,s’> The sequence is given by s, where s represents the sequence state of the phage, a represents the action performed when the sequence state of the phage is s, r represents the reward obtained after performing action a, and s' represents the new sequence state to which the phage migrates after performing action a.

[0075] In one embodiment, the terminal device can pre-build a sample pool for the phage construction model to store raw phage samples. These phage samples can be obtained from a cloud database or generated during the training of agents in the phage construction model.

[0076] In one possible implementation, S13 above may include:

[0077] S131, determine the cumulative reward obtained after performing component filling actions at multiple time steps in the current policy parameter training round.

[0078] S132, based on the sequence state of the complete phage and the accumulated reward, update the strategy parameters of the phage construction model until the update stop condition is met.

[0079] In this embodiment, the phage construction model is based on the initial sequence state of the phage sample, executes a policy according to the policy parameters of the current network, obtains a component filling action trajectory T = {S1, a1, S2, a2...}, obtains the cumulative reward after executing the component filling action trajectory, and trains the optimal policy parameters of the phage construction model with the goal of maximizing the cumulative reward.

[0080] Accumulated rewards J(θ) represents the cumulative reward, E[] represents the expected function, and R... T The reward represents the complete phage sequence, T represents the number of filling elements, and s0 represents the initial sequence state. Let E[R] represent the action-value function in reinforcement learning, π represent the policy, θ represent the policy parameters, and E[R] represent the action-value function in reinforcement learning. T [s0,θ] represents the expected cumulative reward obtained after performing T element filling actions based on the initial sequence state s0 with policy parameter θ, and a1 represents one of the actions in the action set A under the current policy.

[0081] In practical implementation, when the phage construction model is trained on the policy parameters, the policy parameters are updated using the gradient ascent method, i.e.:

[0082]

[0083] in, Let J(θ) represent the gradient, J(θ) represent the cumulative reward, and A represent the cumulative reward. 1:t-1 Let π represent the first to the (t-1)th actions in action set A, π represent the policy, and θ represent the policy parameters. Indicated by strategy π θ The expected cumulative reward obtained from performing actions 1 through t-1, a t Let S represent the t-th action. 1:t-1 This represents the state transitioned to after performing the first action, ..., the state transitioned to after performing the (t-1)th action. This represents the action-value function in reinforcement learning.

[0084] The update formula for the policy parameter θ can be:

[0085]

[0086] Where, α h Let h be the learning rate.

[0087] In this embodiment, a phage construction model based on a reinforcement learning system is established. This model, based on the sequence state environment of the phage, outputs the selected element to be filled according to the policy, and trains the optimal policy parameters by maximizing the cumulative reward obtained from executing a series of actions. The model training does not require a large number of labeled samples, reducing the workload of manual sample labeling. Simultaneously, the model can define a reward evaluation system within the reinforcement learning system based on different construction requirements to evaluate the quality of the element-filling actions and return corresponding rewards to the agent in the reinforcement learning system, enabling the agent to learn a reasonable policy based on the rewards. In implementation, different reward evaluation systems can be defined to construct phage construction models for specific needs.

[0088] In one possible implementation, S131 may include:

[0089] Based on the preset action value function, determine the reward value obtained after performing the action in the last time step of the current strategy parameter training round;

[0090] The Monte Carlo search algorithm is used to sample the reward value obtained after the action is executed in the last time step of multiple policy parameter training rounds, so as to determine the reward value obtained after the action is executed in other time steps of each policy parameter training round based on the sampling results; wherein, the other time steps refer to the time steps other than the last time step in each policy parameter training round.

[0091] The cumulative reward obtained by the current policy parameter is determined based on the reward values ​​obtained after performing actions at all time steps in the training round of the current policy parameter.

[0092] In this embodiment, since the quality and activity of the phage can only be determined when the phage sequence is fully constructed, there is only a reward after the last element filling action is performed. That is, the action value function only provides a reward for the complete phage sequence.

[0093] Assuming there are t components to be filled, where t is an integer greater than or equal to 1, this embodiment uses a Monte Carlo search algorithm to obtain the reward value for intermediate actions (or intermediate states) in order to obtain the reward value for intermediate actions, i.e., the reward value from the first action to the (t-1)th action. The idea behind the Monte Carlo search algorithm is that for a random event, if we want to obtain the probability of its occurrence, we can approximate the probability by repeating the experiment, using the frequency of the event's occurrence. Therefore, in this embodiment, to obtain the reward value for intermediate actions, we start from this state, follow the current strategy, complete multiple component filling trajectories (i.e., execute multiple training rounds, each with actions executed according to the current strategy), and obtain the reward values ​​for multiple final states. Then, we calculate the average of these multiple reward values ​​as the reward value for the current state, i.e., using sampling to approximate the true value.

[0094] For example, see Figure 2 Taking element sequence state S2 as an example, starting from state S2 until the final state, according to the strategy... By executing multiple training rounds, multiple component filling trajectories can be obtained, namely: S2→a3, S3→a3, S7→a5, S... 11 S2→a3, S3→a 4` S4→a 5` ,S 12 and S2→a3, S3→a 4`` S7→a 5`` ,S 12 Among them, S i (i = 2, 3, 4, 7, 11, 12) represent different sequence states, a i (i = 3, 4, 4', 4'', 5, 5', 5'') represent different actions. Each component filling trajectory can obtain a final state reward value after performing the last action, which are -0.5, 2, and 1 respectively. Then, the reward value Q (s = S2) obtained by the current sequence state S2 can be determined by the reward values ​​obtained by all component filling trajectories after performing the last component filling action, that is:

[0095]

[0096] Therefore, the reward values ​​for actions 1 through t-1 are obtained through N Monte Carlo searches, i.e.:

[0097]

[0098] in, Let N represent the reward obtained by transitioning to the first state after performing the first action, ..., the reward obtained by transitioning to the (t-1)th state after performing the (t-1)th action, where N represents the number of Monte Carlo searches. Let represent the reward obtained by the agent after performing the 1st action, ..., the Tth action in the nth Monte Carlo search. For N Monte Carlo searches, S 1:t Indicates the initial sequence state. and Let T represent the reward value obtained after performing the last action in one of the component filling action trajectories corresponding to the current policy parameters obtained in the first Monte Carlo search and the reward value obtained after performing the last action in one of the component filling action trajectories corresponding to the current policy parameters obtained in the nth Monte Carlo search, respectively, where T is the total number of components to be filled.

[0099] After obtaining the reward values ​​for all actions, the cumulative reward obtained after executing all actions with the current policy parameters can be obtained. In one embodiment, the cumulative reward G(t) can be calculated using a discount factor, i.e.:

[0100]

[0101] in, r represents the discount factor corresponding to the it-th action. i This represents the reward value obtained after performing the i-th action.

[0102] In this embodiment, the reward value obtained by the phage construction model in each policy parameter training is obtained by the Monte Carlo search algorithm, which solves the problem that the phage construction model cannot evaluate the reward of incomplete phage sequence states when training in a reinforcement learning environment.

[0103] In one possible implementation, before step S11, which involves inputting the phage sample into the phage construction model, step S10 is further included:

[0104] S10, obtain the network parameters of the pre-trained phage fitting model, and use the network parameters as the initial values ​​of the strategy parameters of the phage construction model.

[0105] In some embodiments, the phage fitting model can be constructed using a Long Short-Term Memory (LSTM) network, a Recurrent Neural Network (RNN), a Generative Adversarial Network, or other network structures.

[0106] In this embodiment, the phage fitting model is a model with a similar task to the phage construction model, such as fitting a phage sequence. Since the phage construction model and the phage construction model have similar tasks, the network parameters of the trained phage fitting model can be used as initial values ​​for the policy parameters of the phage construction model to pre-train the phage construction model.

[0107] This embodiment pre-trains the phage construction model by assigning the network parameters of the phage fitting model to the initial values ​​of the policy parameters of the phage construction model. This solves the problem that the phage construction model faces reward sparsity in reinforcement learning policies due to the large number of phage elements that can be selected by the policy model in reinforcement learning, which leads to an excessively large action space for phage element selection and thus makes it difficult for the phage construction model to converge.

[0108] In one possible implementation, the phage fitting model is trained as follows:

[0109] A generative adversarial network (GAN) model is constructed, and the generative model of the GAN model generates a phage fitting sequence based on the input random vector, so as to use the phage fitting sequence as a fitted phage sample.

[0110] Obtain natural phage samples, and use the discriminative model of the generative adversarial network model to distinguish between the natural phage samples and the fitted phage samples;

[0111] Based on the identification results, the generative model and the discriminative model are trained alternately to obtain the phage fitting model.

[0112] In this embodiment, natural bacteriophages are used as the sample dataset. The terminal device can obtain the natural bacteriophage samples from a local database or a cloud database, or it can obtain the natural bacteriophage samples in response to a sample entry command input by the user.

[0113] In a Generative Adversarial Network (GAN), the generative model acts as a sample generator, outputting a realistic sample based on an input noise or sample. The discriminative model acts as a binary classifier, determining whether an input sample is true or false (e.g., a value greater than 0.5 indicates true, and less than 0.5 indicates false). In practice, the network parameters of both the generative and discriminative models are first initialized. Training the GAN involves two phases: training the discriminative model and training the generative model.

[0114] Phase 1: Keeping the network parameter values ​​of the discriminant model unchanged, train the generative model. The generative model generates a fake fitted phage sample and inputs it into the discriminant model for discrimination.

[0115] The second stage involves training the discriminative model while keeping the network parameter values ​​of the generative model unchanged. The natural phage samples are input into the discriminative model, and the natural phage samples are marked as true. The discriminative model is then trained. Conversely, the fitted phage samples generated by the generative model are input into the discriminative model, and the fitted phage samples are marked as false. The discriminative model is then trained again.

[0116] The training in the first and second stages is repeated to alternately train the discriminative model and the generative model until the iteration stopping condition is met, such as reaching a preset number of iterations. This number of iterations can be set according to experience or actual needs to obtain the trained generative model. The trained generative model is then used to generate a fitted phage sequence.

[0117] In one embodiment, the generative model may use a recursive neural network (RNN) as the generative model, which takes the input random vector x1,...,x as the generative model. T The hidden state sequence is recursively transformed by the update function g into a sequence of hidden states h1,...,h. T , i.e. h t =g(h t-1 ,x t The hidden state sequence h1,...,h is then passed through the output layer. T Mapping this to the output token distribution, we have p(y) t |x1,...,x T )=z(h t ) = softmax(c + Vh t ), where softmax() is the activation function, c and V are the bias and weight matrices, respectively, and y t Let represent the output of the generative model. Then, the generative model is trained using maximum likelihood, i.e.: Where G represents the generative model, and θ represents the parameters of the generative model.

[0118] In one embodiment, the discriminant model may use a convolutional neural network, which represents the input phage sequence as... Where x1, x2...x T ∈R k It is a k-dimensional token embedding. It is to establish matrix ε 1:T ∈R T×k The cascaded operators are then used to train the discriminant model, i.e., ... in, This represents the cross-entropy loss function, where y is the true label. It is the predicted probability of the discriminant model.

[0119] Of course, in other embodiments, the generative model and the discriminative model may also adopt generative models with other network structures, and this application does not limit them.

[0120] In one possible implementation, the generative model of the generative adversarial network model generates a phage fitting sequence based on an input random vector, including:

[0121] The randomly generated vector is input into the generation model to obtain the first phage element;

[0122] The i-th phage element is input into the generation model to obtain the (i+1)-th phage element;

[0123] Repeat the steps of inputting the i-th phage element into the generation model to obtain the (i+1)-th phage element until the number of phage elements obtained is equal to T; where i and T are integers greater than or equal to 1.

[0124] In this embodiment, the generative model can infer the next element in a sequence of elements using contextual information. For example, after generating the first element T1 based on a random vector, the generative model inputs the first element T1 into the model and outputs the second element T2, ..., and so on, until the (n-1)th element T... n-1 The input is given to the generative model, and the nth element T is output. n Thus, based on T1, T2, ..., T n Obtain the fitted phage sequences (T1, T2, ..., T...). n ).

[0125] Based on the phage construction model training method provided in the above embodiments, the second embodiment of this application also provides a phage construction method, including:

[0126] A phage with a missing tail protein and a sequence of elements to be filled is input into the phage construction model to obtain a complete phage sequence; wherein, the phage construction model is a phage construction model trained using the phage construction model training method provided in the above embodiments.

[0127] This embodiment constructs phages using the phage construction model, which is a model trained on a dataset. Therefore, it can efficiently discover and optimize key factors and parameters for phage construction. At the same time, due to the predictability of artificial intelligence algorithms, this construction method can avoid unpredictable mutations present in traditional phage construction methods and can achieve comprehensive and multi-objective optimization of phage construction, thus more accurately meeting different application needs than traditional phage construction methods.

[0128] Accordingly, see Figure 3 , Figure 3 This is a structural block diagram of the terminal device provided in the third embodiment of this application. The terminal device 3 in this embodiment includes:

[0129] The acquisition module 31 is used to acquire phage samples and phage construction models to be trained; wherein, the phage sample includes the initial sequence state of the phage sample; the initial sequence state of the phage sample is a phage element sequence in which the tail protein is hollowed out and the element sequence to be filled.

[0130] The first construction module 32 is used to input the phage sample into the phage construction model, so that the phage construction model can determine the elements to be filled based on the current sequence state of the phage sample, until a complete phage sequence is constructed.

[0131] Training module 33 is used to update the parameters of the phage construction model based on the constructed complete phage sequence until the update stop condition is met, thereby completing the training of the phage construction model.

[0132] In one possible implementation, the phage construction model is a phage construction model based on a reinforcement learning network; the first construction module 32 is specifically used for:

[0133] The phage sample is input into the phage construction model, which then determines the elements to be filled based on the current sequence state of the phage sample, until a complete phage sequence is constructed, including:

[0134] The phage sample is input into the phage construction model to determine the element to be filled at each time step in the current strategy parameter training round based on the initial sequence state of the phage sample and the current strategy parameters of the phage construction model, so as to construct a complete phage sequence.

[0135] In one possible implementation, the first construction module 32 updates the parameters of the phage construction model based on the constructed complete phage sequence until an update stop condition is met, including:

[0136] Determine the cumulative reward obtained after performing component filling actions at multiple time steps in the current policy parameter training round;

[0137] Based on the sequence state of the complete phage and the accumulated reward, the strategy parameters of the phage construction model are updated until the update stop condition is met.

[0138] In one possible implementation, the first construction module 32 determines the cumulative reward obtained after performing component filling actions at multiple time steps in the current policy parameter training round, including:

[0139] Based on the preset action value function, determine the reward value obtained after performing the action in the last time step of the current strategy parameter training round;

[0140] The Monte Carlo search algorithm is used to sample the reward value obtained after the action is executed in the last time step of multiple policy parameter training rounds, so as to determine the reward value obtained after the action is executed in other time steps of each policy parameter training round based on the sampling results; wherein, the other time steps refer to the time steps other than the last time step in each policy parameter training round.

[0141] The cumulative reward obtained by the current policy parameter is determined based on the reward values ​​obtained after performing actions at all time steps in the training round of the current policy parameter.

[0142] In one possible implementation, the terminal device further includes:

[0143] The pre-training module is used to obtain the network parameters of the pre-trained phage fitting model and use the network parameters as the initial values ​​of the strategy parameters of the phage construction model.

[0144] In one possible implementation, the phage fitting model is trained as follows:

[0145] A generative adversarial network (GAN) model is constructed, and the generative model of the GAN model generates a phage fitting sequence based on the input random vector, so as to use the phage fitting sequence as a fitted phage sample.

[0146] Obtain natural phage samples, and use the discriminative model of the generative adversarial network model to distinguish between the natural phage samples and the fitted phage samples;

[0147] Based on the identification results, the generative model and the discriminative model are trained alternately to obtain the phage fitting model.

[0148] In one possible implementation, the generative model of the generative adversarial network model generates a phage fitting sequence based on an input random vector, including:

[0149] The randomly generated vector is input into the generation model to obtain the first phage element;

[0150] The i-th phage element is input into the generation model to obtain the (i+1)-th phage element;

[0151] Repeat the steps of inputting the i-th phage element into the generation model to obtain the (i+1)-th phage element until the number of phage elements obtained is equal to T; where i and T are integers greater than or equal to 1.

[0152] In one possible implementation, the terminal device further includes a second construction model, wherein the second construction module is used to input a phage with tail protein missing and element sequence to be filled into the phage construction model to obtain a complete phage sequence.

[0153] It should be noted that the information interaction and execution process between the above-mentioned devices are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0154] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0155] Accordingly, see Figure 4 , Figure 4 This is a structural block diagram of a terminal device provided in the third embodiment of this application. In this embodiment, the terminal device 4 includes a memory 41, a processor 40, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, it implements all the steps of the phage construction model training method provided in the first embodiment above, or implements all the steps of the phage construction method provided in the second embodiment above.

[0156] The terminal device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This terminal device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that... Figure 4 This is merely an example of terminal device 4 and does not constitute a limitation on terminal device 4. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0157] The processor 40 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0158] In some embodiments, the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. In other embodiments, the memory 41 may be an external storage device of the terminal device 4, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 4. Furthermore, the memory 41 may include both internal and external storage units of the terminal device 4. The memory 41 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0159] Accordingly, the fourth embodiment of this application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements all the steps of the phage construction model training method provided in the first embodiment above, or implements all the steps of the phage construction method provided in the second embodiment above.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium.

[0161] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training a bacteriophage construction model, characterized in that, include: Obtain phage samples and a phage construction model to be trained; wherein, the phage samples include the sequence state of the phage samples; the initial sequence state of the phage samples is a phage element sequence in which the tail protein is hollowed out and the element sequence to be filled; the phage construction model is a phage construction model based on a reinforcement learning network; The process of inputting the phage sample into the phage construction model, so that the phage construction model determines the element to be filled based on the current sequence state of the phage sample, until a complete phage sequence is constructed, includes: inputting the phage sample into the phage construction model, so that the element to be filled at each time step in the current strategy parameter training round is determined based on the initial sequence state of the phage sample and the current strategy parameters of the phage construction model, so as to construct a complete phage sequence; Based on the constructed complete phage sequence, the parameters of the phage construction model are updated until the update stopping condition is met, thus completing the training of the phage construction model. This includes: determining the reward value obtained after performing an action at the last time step in the current policy parameter training round according to a preset action value function; sampling the reward values ​​obtained after performing an action at the last time step in multiple policy parameter training rounds using a Monte Carlo search algorithm to determine the reward values ​​obtained for performing actions at other time steps in each policy parameter training round based on the sampling results; wherein, the other time steps refer to all time steps in each policy parameter training round except the last time step; determining the cumulative reward obtained by the current policy parameter based on the reward values ​​obtained after performing actions at all time steps in the current policy parameter training round; and updating the policy parameters of the phage construction model based on the complete phage sequence state and the cumulative reward until the update stopping condition is met.

2. The phage construction model training method as described in claim 1, characterized in that, Before inputting the phage sample into the phage construction model, the method further includes: Obtain the network parameters of the pre-trained phage fitting model; The network parameters are used as the initial values ​​for the strategy parameters of the phage construction model.

3. The phage construction model training method as described in claim 2, characterized in that, The phage fitting model was trained in the following manner: A generative adversarial network (GAN) model is constructed, and the generative model of the GAN model generates a phage fitting sequence based on the input random vector, so as to use the phage fitting sequence as a fitted phage sample. Obtain natural phage samples, and use the discriminative model of the generative adversarial network model to distinguish between the natural phage samples and the fitted phage samples; Based on the identification results, the generative model and the discriminative model are trained alternately to obtain the phage fitting model.

4. The phage construction model training method as described in claim 3, characterized in that, The generative model of the generative adversarial network model generates a phage fitting sequence based on an input random vector, including: The randomly generated vector is input into the generation model to obtain the first phage element; The first i The first phage element is input into the generation model to obtain the second phage element. i +1 phage element; Repeat the above to the first i The first phage element is input into the generation model to obtain the second phage element. i Continue adding one phage element at a time until the total number of phage elements obtained is equal to the given number. T ;in, i and T It is an integer greater than or equal to 1.

5. A method for constructing bacteriophages, characterized in that, include: Phages with missing tail proteins and missing element sequences are input into a phage construction model to obtain complete phage sequences. The phage construction model is a phage construction model trained using the phage construction model training method as described in any one of claims 1 to 4.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the phage construction model training method as described in any one of claims 1 to 4, or the phage construction method as described in claim 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the phage construction model training method as described in any one of claims 1 to 4, or the phage construction method as described in claim 5.

Images