Protein safety and controllable generation method and device based on adversarial reinforcement learning based on protein safety knowledge graph

Abstract

The present invention discloses a protein safe and controllable generation method and device based on adversarial reinforcement learning of protein safety knowledge graph, comprising: constructing a protein safety knowledge graph; extracting an initial embedding representation of a harmful protein sequence and an embedding vector of external knowledge; embedding knowledge based on the initial embedding representation of the harmful protein sequence and the embedding vector of external knowledge and in combination with the protein safety knowledge graph to obtain an enhanced embedding representation; constructing a generative adversarial network including a generator and a penalty model, and performing a first-stage adversarial penalty learning; fixing the parameters of the penalty model after the first-stage adversarial penalty learning, and performing a second-stage reinforcement learning; and generating safe and controllable new proteins using the generator after the second-stage parameter optimization. The method and device are based on a language model, use the knowledge graph as external knowledge, and use adversarial reinforcement learning to guide the safe and controllable generation of proteins.

Description

Technical Field

[0001] The present invention belongs to the technical field of the combination of protein safety generation, knowledge graph, adversarial learning and reinforcement learning, and specifically relates to a method and device for controllable protein safety generation based on adversarial reinforcement learning of protein safety knowledge graph. Background Art

[0002] In recent years, the application of large language models (LLMs) in the field of bioinformatics has made significant progress, especially in protein sequence generation and function prediction. These models usually capture complex patterns and functional relationships in protein sequences by learning a large amount of biological sequence data. By imitating natural language processing technology, such as Transformer and BERT model structures, large language models can process and generate high-quality protein sequences with biological functions. For example, the patent application with publication number CN117809749A discloses a method for generating functional polypeptide sequences, which utilizes the feature extraction capabilities of a pre-trained protein large language model and the generation capabilities of a generative model, and effectively extracts the sequence features of a specific category of functional polypeptides to achieve the purpose of synthesizing a variety of functional polypeptides or protein sequences from scratch without a template and with an unknown target structure.

[0003] For example, a patent application with publication number CN117727365A discloses a protein reverse folding method based on a multimodal pre-trained large model. Based on the protein structure, the structural representation of the protein is obtained through a pre-trained protein structure encoder; the structural representation of the protein is converted into structural guidance for sequence generation through a protein structure adapter; the protein large language model is pre-trained using an autoregressive method, and the structural guidance generated by the sequence is spliced ​​with the protein language start symbol and then input into the pre-trained protein large language model to generate a protein sequence paired with the protein structure; and the generated protein sequence is evaluated using evaluation indicators.

[0004] The application of large language models in protein design can not only accelerate the discovery of new proteins, but also help scientists understand how proteins can mutate without affecting their functions, or how to change their functions to adapt to new applications, such as in drug development and bioengineering. In addition, proteins generated by these models can be used in a variety of fields such as disease treatment, biomanufacturing and environmental protection.

[0005] Although large language models have performed well in the field of protein generation, current technologies still face some challenges, especially in ensuring the safety of generated proteins. Current models generally focus on improving generation efficiency and sequence quality, but do not adequately consider the potential toxicity or adverse biological activity of generated proteins. This situation may cause the generated proteins to cause unforeseen biosafety issues in practical applications, such as triggering allergic reactions, toxic reactions, or other health problems.

[0006] In addition, the functional complexity of proteins and the unpredictability of their biological activities make it difficult to comprehensively evaluate the safety of generated proteins using traditional bioinformatics methods alone. Therefore, it is particularly important to develop a method that can instantly evaluate and ensure the safety of proteins during their production process. Summary of the invention

[0007] In view of the above, the purpose of the present invention is to provide a method and device for safe and controllable protein generation based on adversarial reinforcement learning of protein safety knowledge graph, which is based on a language model, takes the knowledge graph as external knowledge, and uses adversarial reinforcement learning to guide the safe and controllable generation of proteins.

[0008] To achieve the above-mentioned purpose of the invention, the embodiment provides a method for generating protein safety and controllability based on adversarial reinforcement learning of protein safety knowledge graph, comprising the following steps:

[0009] Construct a protein safety knowledge graph, which includes gene ontology knowledge, harmful protein knowledge, and the relationship between knowledge;

[0010] A masked language model is used to extract the initial embedding representation of the harmful protein sequence, and an encoder is used to encode the embedding vectors of gene ontology terms contained in gene ontology knowledge and the embedding vectors of protein function descriptions contained in harmful protein knowledge;

[0011] Based on the protein safety knowledge graph, the embedding vectors of gene ontology terms and protein function descriptions are fused into the initial embedding representation of the protein sequence to obtain an enhanced embedding representation.

[0012] Construct a generative adversarial network including a generator and a penalty model, where the generator is used to generate a new protein embedding representation based on the input data, and the penalty model evaluates the security of the new protein embedding representation and the true protein embedding representation respectively. Sample data is sampled from the enhanced embedding representation, and the sample data is used to perform a one-stage adversarial penalty learning on the generative adversarial network, and the parameters of the generator and the penalty model are optimized in one stage.

[0013] The parameters of the penalty model after the first phase of adversarial penalty learning are fixed, and the reward value is constructed based on the security assessment results output by the penalty model. The reward function is constructed based on the reward value to conduct the second phase of reinforcement learning, and the parameters of the generator are optimized in the second phase.

[0014] The generator with two-stage parameter optimization is used to generate safe and controllable new proteins.

[0015] Preferably, the construction of a protein safety knowledge graph comprises:

[0016] Collect and organize toxic or antigenic proteins and their functional descriptions to build a database of harmful proteins;

[0017] Prepare gene ontology and gene annotation data and extract ontology structure data from them, extract a type of triple describing the association between protein and gene ontology from gene annotation data, and extract another type of triple describing the association between two gene ontologies from ontology structure data;

[0018] Protein names, protein sequences and protein function descriptions are extracted from the harmful protein database to form harmful protein knowledge. Gene ontology terms and gene ontology types are extracted from gene ontology to form gene ontology knowledge. The protein safety knowledge graph is constructed based on the association relationship between the above two types of triples.

[0019] Preferably, the method for constructing the masked language model is:

[0020] Based on the language model, some amino acids in the real protein sequence are masked to obtain a masked sequence, and the loss function is constructed based on the embedding representation generated by the large language model based on the masked sequence and the real protein sequence to optimize the language model parameters and obtain a masked language model.

[0021] Preferably, the method of fusing the embedding vectors of gene ontology terms and the embedding vectors of protein function descriptions into the initial embedding representation of the protein sequence based on the protein safety knowledge graph to obtain an enhanced embedding representation comprises:

[0022] Find triplets containing protein sequences and gene ontology terms or / and protein function descriptions from the protein safety knowledge graph to form a local graph, and use the graph model to learn the embedding representation of the triplets in the local graph. During learning, the graph model parameters and the embedding representation of the triplets are updated based on the interval loss function. After learning, the embedding representation corresponding to the protein sequence in the triplet is extracted as its enhanced embedding representation;

[0023] Among them, the loss function of the interval is expressed as:

[0024] L=∑ (h,r,t)∈S ∑ (h′,r,t′)∈S′max(0,γ+d(h+r,t)-d(h′+r,t′)

[0025] where (h, r, t) is the correct triple extracted from the protein safety knowledge graph, (h′, r, t′) is the incorrect triple generated by replacing the head entity or the tail entity, S is the set of correct triplets corresponding to the local graph, S′ is the set of incorrect triplets, d(·) represents the distance metric between the head and tail entities, and γ represents the safety interval γ.

[0026] Preferably, when performing one-stage adversarial penalty learning on the generative adversarial network, the loss function used includes the generator loss function L G And the loss function L of the penalty model D ;

[0027]

[0028] Where z represents the distribution p from the user-enhanced embedding representation z (z), G(z) represents the new protein embedding representation generated by the generator G according to the sampled data z, D(·) represents the discrimination result of the input, and x represents the distribution p of the real protein embedding representation from the harmless d (x) is the sample data, and E represents the expected value.

[0029] Preferably, the reward value R(G(z;θ)) is constructed based on the safety assessment result output by the penalty model, which is expressed as:

[0030] R(G(z;θ))=-log(1-D(G(z;θ)))

[0031] The reward function J(θ) is constructed based on the reward value:

[0032]

[0033] Where z represents the distribution p from the user-enhanced embedding representation z In (z), G(z; θ) represents the new protein embedding representation generated by the generator G with parameter θ according to the sampled data z, D(·) represents the discrimination result of the input, and E represents the expected value.

[0034] Preferably, when constructing a reward function based on the reward value for two-stage reinforcement learning training, the REINFORCE policy gradient algorithm is used to calculate the gradient of the reward function to update the parameters of the generator.

[0035] To achieve the above-mentioned purpose of the invention, the embodiment further provides a protein safety controllable generation device of adversarial reinforcement learning of a protein safety knowledge graph, comprising:

[0036] A graph construction module, which is used to construct a protein safety knowledge graph, which includes gene ontology knowledge, harmful protein knowledge, and the association relationship between knowledge;

[0037] An embedding representation module, which is used to extract an initial embedding representation of the harmful protein sequence using a masked language model, and at the same time uses an encoder to encode the embedding vectors of gene ontology terms contained in gene ontology knowledge and the embedding vectors of protein function descriptions contained in the harmful protein knowledge;

[0038] A knowledge embedding module, which is used to fuse the embedding vectors of gene ontology terms and protein function descriptions into the initial embedding representation of the protein sequence based on the protein safety knowledge graph to obtain an enhanced embedding representation;

[0039] An adversarial penalty learning module is used to construct a generative adversarial network including a generator and a penalty model, wherein the generator is used to generate a new protein embedding representation based on the input data, the penalty model evaluates the security of the new protein embedding representation and the true protein embedding representation respectively, samples the sample data from the enhanced embedding representation, and uses the sample data to perform a one-stage adversarial training on the generative adversarial network, optimizing the parameters of the generator and the penalty model in one stage;

[0040] The reinforcement learning module is used to fix the parameters of the penalty model after the first stage of adversarial training, and to construct the reward value based on the security assessment results output by the penalty model. The reward function is constructed based on the reward value to conduct the second stage of reinforcement learning training, and the parameters of the generator are optimized in the second stage.

[0041] The inference application module is used to generate safe and controllable new proteins using the generator after two-stage parameter optimization.

[0042] To achieve the above-mentioned purpose of the invention, an embodiment also provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned protein safety and controllable generation method based on adversarial reinforcement learning based on protein safety knowledge graph.

[0043] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-mentioned protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] (1) Construct a protein safety knowledge graph of domain knowledge, which covers gene ontology knowledge, harmful protein knowledge, and the relationship between knowledge, providing a basic knowledge description for the safe generation of proteins.

[0046] (2) By combining protein safety knowledge graphs and adversarial reinforcement learning, it is possible to evaluate and ensure the safety of proteins during the protein generation process. This approach can effectively identify and reduce potential toxic or harmful biological activities, thereby improving the safety of biological products.

[0047] (3) By combining adversarial penalty learning with reinforcement learning, the learning process of the generator is optimized, which improves learning efficiency and generation accuracy. This approach enables the generator to reduce generation errors and outputs that do not meet the target while maintaining a high degree of innovation.

[0048] (4) The method of the present invention is not only applicable to protein production, but can also be extended to the design and optimization of other biomacromolecules. Its flexibility and adaptability enable it to meet different biological and medical needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 It is a flow chart of a method for generating safe and controllable proteins based on adversarial reinforcement learning of a protein safety knowledge graph provided in an embodiment;

[0051] Figure 2 is an example diagram of a protein safety knowledge graph constructed in the embodiment;

[0052] Figure 3 It is a schematic diagram of one-stage adversarial penalty learning and two-stage reinforcement learning provided by the embodiment;

[0053] Figure 4 It is a schematic diagram of the structure of a protein safety and controllable generation device based on adversarial reinforcement learning of protein safety knowledge graph provided in an embodiment. DETAILED DESCRIPTION

[0054] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0055] Figure 1 A protein safety controllable generation method based on adversarial reinforcement learning of a protein safety knowledge graph provided by an embodiment of the present invention, as Figure 1 shown, the protein safety controllable generation method provided by the embodiment includes the following steps:

[0056] S1, construct a protein safety knowledge graph.

[0057] In the embodiment, the constructed protein safety knowledge graph includes gene ontology knowledge, harmful protein knowledge, and the association relationships between the knowledge. The specific construction process includes:

[0058] Collect and organize proteins with toxicity or antigenicity and their functional descriptions, and construct a harmful protein database after duplicate deletion;

[0059] Prepare Gene Ontology (GO) and gene annotation data and extract ontology structure data therefrom. Among them, the gene annotation data encapsulates the external structure of the gene ontology in the form of annotations. Extract a type of triple that describes the association relationship between proteins and gene ontology, that is, protein-gene ontology (GO) triples, in the format of <protein (h), relationship (r), GO term (t)>, which consists of proteins (h) connected to GO terms (t) through a specific relationship (r) (such as binding function), explaining the complex interactions within the bioinformatics framework. Extract another type of triple that describes the association relationship between two gene ontologies from the ontology structure data, that is, GO-GO triples, in the format of <GO term 1 (h), relationship (r), GO term 2 (t)>. These triples describe the internal connections within GO, where each relationship (r) (such as is_a or part_of) connects GO terms (h and t), representing the hierarchical relationships and association relationships crucial for understanding complex biological functions.

[0060] Extract the initial embedding representation of the harmful protein sequence and the embedding vector of external knowledge from the harmful protein database. Among them, the entire protein sequence data is used as the protein representation in the subsequent graph learning process, and the functional description is used as a supplementary information resource to enhance understanding and characterization. At the same time, extract gene ontology terms and gene ontology types (including MF, BP, and CC) from the gene ontology to form gene ontology knowledge, and construct a protein safety knowledge graph according to the above two types of triples to form an association relationship, as Figure 2 shown, the black circles are all protein knowledge, Q6GZW6 is the protein name, the white circles are gene ontology knowledge, and GO-BP is the gene ontology of type BP.

[0061] S2, extract the initial embedding representation of the harmful protein sequence and the embedding vector of external knowledge.

[0062] In an embodiment, a masked language model is used to extract an initial embedding representation of a harmful protein sequence, including: based on the language model, masking some amino acids in the real protein sequence to obtain a masked sequence, and using a large language model to generate an embedding representation based on the masked sequence and the real protein sequence to construct a loss function to optimize the language model parameters, thereby obtaining a masked language model.

[0063] Specifically, a language model such as Bert may be used, and during masking, 15% of the amino acid markers in the protein sequence may be randomly masked.

[0064] In the embodiment, gene ontology knowledge and harmful protein knowledge are used as external knowledge. Specifically, an encoder is used to encode low-dimensional embedding vectors of gene ontology terms contained in gene ontology knowledge and low-dimensional embedding vectors of protein function descriptions contained in harmful protein knowledge. These low-dimensional embedding vectors of external knowledge are used for subsequent enhanced learning of protein sequence embedding representation.

[0065] S3, based on the initial embedding representation of harmful protein sequences and the embedding vector of external knowledge, combined with the protein safety knowledge graph, performs knowledge embedding to obtain an enhanced embedding representation.

[0066] In an embodiment, based on a protein safety knowledge graph, embedding vectors of gene ontology terms and embedding vectors of protein function descriptions are fused into an initial embedding representation of a protein sequence to obtain an enhanced embedding representation.

[0067] The triplets in the protein safety knowledge graph can be divided into two categories, GO-GO triplets and protein-GO triplets. Knowledge embedding represents relations by simple vector addition. These vectors are learned by minimizing the distance between the head entity, relation, and tail entity in the relation triplet, so that for each correct triple (h, r, t), there is:

[0068] h+r≈t

[0069] In order to learn these representations, we find triplets containing protein sequences and gene ontology terms or / and protein function descriptions from the protein safety knowledge graph to form a local graph, and use the graph model to learn the embedded representation of the triplets in the local graph. During learning, we use an interval-based loss function to make the score of the correct triplet lower than the score of the wrong triplet by a safety interval γ. We update the graph model parameters and the embedded representation of the triplet based on the interval loss function. After learning, we extract the embedded representation corresponding to the protein sequence in the triplet as its enhanced embedded representation. The loss function of the interval is expressed as:

[0070] L=∑ (h,r,t)∈S ∑ (h′,r,t′)∈S′max(0,γ+d(h+r,t)-d(h′+r,t′)

[0071] where (h, r, t) is the correct triple extracted from the protein safety knowledge graph, (h′, r, t′) is the incorrect triple generated by replacing the head entity or the tail entity, S is the set of correct triplets corresponding to the local graph, S′ is the set of incorrect triplets, d(·) represents the distance metric between the head and tail entities, d(h+r,t)=||h+rt||, d(h′+r,t′)=||h′+rt′||.

[0072] S4, construct a generative adversarial network consisting of a generator and a penalty model, and perform a one-stage adversarial penalty learning.

[0073] In the embodiment, Figure 3 As shown, the generative adversarial network includes a generator G and a penalty model D, wherein the generator G is used to generate a new protein embedding representation based on the input data, trying to imitate the real, harmless protein characteristics as much as possible; the penalty model D acts as a discriminator to evaluate the harmful possibility of a given protein, that is, to evaluate the safety of the new protein embedding representation and the real protein embedding representation respectively, and outputs a score indicating the level of potential danger. The generator G and the penalty model D are trained with opposite goals. The generator G can adopt a large language model.

[0074] Specifically, sample data is sampled from the enhanced embedding representation, and the sample data is used to perform a one-stage adversarial penalty learning on the generative adversarial network, optimizing the parameters of the generator and penalty model in one stage. The loss functions used include the generator loss function L G And the loss function L of the penalty model D , through the generator loss function L G Trying to minimize this loss involves tricking the penalized model into accepting that synthetic proteins are harmless, by penalizing the model's loss function L D Refers to the correct classification of real proteins (harmless proteins) and newly generated proteins (potentially harmful proteins).

[0075]

[0076] Where z represents the distribution p from the user-enhanced embedding representation z (z), G(z) represents the new protein embedding representation generated by the generator G according to the sampled data z, D(·) represents the discrimination result of the input, and x represents the distribution p from the real protein embedding representation d (x) is the sample data, and E represents the expected value.

[0077] S5, fix the parameters of the penalty model after the first stage of adversarial penalty learning, and perform the second stage of reinforcement learning.

[0078] In the embodiment, reinforcement learning (RL) technology is used to improve the strategy of generator G according to the feedback of penalty model D. The goal is to minimize the harmfulness of the generated protein determined by the adversarial penalty learning in the previous step. Generator G acts as a strategy in the RL context, where the role is to generate protein embedding. The state can be represented by a vector z or a combination of z and previous feedback from D. Specifically, the parameters of the penalty model D after the first stage of adversarial penalty learning are fixed, and the reward value is constructed based on the safety assessment result output by the penalty model D, and the reward function is constructed based on the reward value to perform two-stage reinforcement learning, and the parameters of the generator G are optimized in the two stages.

[0079] The reward value R(G(z;θ)) is constructed based on the safety evaluation results output by the penalty model, aiming to prevent the production of harmful proteins:

[0080] R(G(z;θ))=-log(1-D(G(z;θ)))

[0081] The reward value constructed in this way increases as the penalty model D increases its predictions of G(z;θ) that are harmless.

[0082] In order to optimize the generator G, a reward function J(θ) is constructed based on the reward value, and the REINFORCE policy gradient algorithm is used to adjust the parameters θ and G to maximize the expected reward:

[0083]

[0084] Among them, G(z;θ) represents the new protein embedding representation generated by the generator G with parameter θ according to the sampled data z.

[0085] S6, Generate safe and controllable new proteins using the generator after two-stage parameter optimization.

[0086] In the embodiment, when a new protein is generated, a requirement text is input into the generator, and the generator generates the new protein based on the embedding distribution corresponding to the requirement text.

[0087] like Figure 4 As shown, the embodiment also provides a protein safety controllable generation device 40 of adversarial reinforcement learning of a protein safety knowledge graph, including a graph construction module 41, an embedding representation module 42, a knowledge embedding module 43, an adversarial penalty learning module 44, a reinforcement learning module 45, and a reasoning application module 46.

[0088] Among them, the graph construction module 41 is used to construct a protein safety knowledge graph, which includes gene ontology knowledge, harmful protein knowledge, and the association between knowledge; the embedding representation module 42 is used to extract the initial embedding representation of the harmful protein sequence using a masked language model, and at the same time use an encoder to encode the embedding vectors of gene ontology terms contained in the gene ontology knowledge and the embedding vectors of protein function descriptions contained in the harmful protein knowledge; the knowledge embedding module 43 is used to fuse the embedding vectors of gene ontology terms and the embedding vectors of protein function descriptions into the initial embedding representation of the protein sequence based on the protein safety knowledge graph to obtain an enhanced embedding representation; the adversarial penalty learning module 44 is used to construct a generative adversarial model including a generator and a penalty model. The invention relates to an anti-network, wherein the generator is used to generate a new protein embedding representation according to the input data, the penalty model evaluates the security of the new protein embedding representation and the real protein embedding representation respectively, the sample data is sampled from the enhanced embedding representation, and the sample data is used to perform a first-stage adversarial training on the generative adversarial network, and the parameters of the generator and the penalty model are optimized in the first stage; the reinforcement learning module 45 is used to fix the parameters of the penalty model after the first-stage adversarial training, and construct a reward value based on the security assessment result output by the penalty model, and construct a reward function based on the reward value to perform a second-stage reinforcement learning training, and the parameters of the generator are optimized in the second stage; the reasoning application module 46 is used to generate a safe and controllable new protein using the generator after the second-stage parameter optimization.

[0089] It should be noted that the protein safe and controllable production device provided in the above embodiment should be illustrated by the division of the above functional modules when performing safe and controllable production of proteins. The above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the protein safe and controllable production device provided in the above embodiment belongs to the same concept as the protein safe and controllable production method embodiment. The specific implementation process is detailed in the protein safe and controllable production method embodiment, which will not be repeated here.

[0090] Based on the same inventive concept, an embodiment further provides a computing device, including a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, the method for safely and controllably generating a protein is implemented, specifically including the following steps:

[0091] S1, constructing a protein safety knowledge graph;

[0092] S2, extracting the initial embedding representation of harmful protein sequences and the embedding vector of external knowledge;

[0093] S3, based on the initial embedding representation of the harmful protein sequence and the embedding vector of the external knowledge and combined with the protein safety knowledge graph, knowledge embedding is performed to obtain an enhanced embedding representation;

[0094] S4, construct a generative adversarial network including a generator and a penalty model, and perform a one-stage adversarial penalty learning;

[0095] S5, fix the parameters of the penalty model after the first stage of adversarial penalty learning and conduct the second stage of reinforcement learning;

[0096] S6, Generate safe and controllable new proteins using the generator after two-stage parameter optimization.

[0097] The computing device provided in the embodiment, in addition to the processor and memory, also includes hardware required for other services such as internal bus, network interface, memory, etc. at the hardware level. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph described in S1-S6 above. Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0098] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph is implemented, which specifically includes the following steps:

[0099] S1, constructing a protein safety knowledge graph;

[0100] S2, extracting the initial embedding representation of harmful protein sequences and the embedding vector of external knowledge;

[0101] S3, based on the initial embedding representation of the harmful protein sequence and the embedding vector of the external knowledge and combined with the protein safety knowledge graph, knowledge embedding is performed to obtain an enhanced embedding representation;

[0102] S4, construct a generative adversarial network including a generator and a penalty model, and perform a one-stage adversarial penalty learning;

[0103] S5, fix the parameters of the penalty model after the first stage of adversarial penalty learning and conduct the second stage of reinforcement learning;

[0104] S6, Generate safe and controllable new proteins using the generator after two-stage parameter optimization.

[0105] In the embodiment, computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data.

[0106] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A protein safety controllable generation method based on adversarial reinforcement learning based on protein safety knowledge graph, characterized in that: The following steps are involved: Construct a protein safety knowledge graph, which includes gene ontology knowledge, harmful protein knowledge, and the relationship between knowledge; A masked language model is used to extract the initial embedding representation of the harmful protein sequence, and an encoder is used to encode the embedding vectors of gene ontology terms contained in gene ontology knowledge and the embedding vectors of protein function descriptions contained in harmful protein knowledge; Based on the protein safety knowledge graph, the embedding vectors of gene ontology terms and protein function descriptions are fused into the initial embedding representation of the protein sequence to obtain an enhanced embedding representation. Construct a generative adversarial network including a generator and a penalty model, where the generator is used to generate a new protein embedding representation based on the input data, and the penalty model evaluates the security of the new protein embedding representation and the true protein embedding representation respectively. Sample data is sampled from the enhanced embedding representation, and the sample data is used to perform a one-stage adversarial penalty learning on the generative adversarial network, and the parameters of the generator and the penalty model are optimized in one stage. The parameters of the penalty model after the first phase of adversarial penalty learning are fixed, and the reward value is constructed based on the security assessment results output by the penalty model. The reward function is constructed based on the reward value to conduct the second phase of reinforcement learning, and the parameters of the generator are optimized in the second phase. Use the generator after two-stage parameter optimization to generate safe and controllable new proteins.

2. The protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to claim 1 is characterized in that: The construction of the protein safety knowledge graph comprises: Collect and organize toxic or antigenic proteins and their functional descriptions to build a database of harmful proteins; Prepare gene ontology and gene annotation data and extract ontology structure data from them, extract a type of triple describing the association between protein and gene ontology from gene annotation data, and extract another type of triple describing the association between two gene ontologies from ontology structure data; Protein names, protein sequences and protein function descriptions are extracted from the harmful protein database to form harmful protein knowledge. Gene ontology terms and gene ontology types are extracted from gene ontology to form gene ontology knowledge. The protein safety knowledge graph is constructed based on the association relationship between the above two types of triples.

3. The protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to claim 1 is characterized in that: The method for constructing the masked language model is: Based on the language model, some amino acids in the real protein sequence are masked to obtain a masked sequence, and the loss function is constructed based on the embedding representation generated by the large language model based on the masked sequence and the real protein sequence to optimize the language model parameters and obtain a masked language model.

4. The protein safety controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to claim 1, characterized in that: The method of fusing the embedding vectors of gene ontology terms and protein function descriptions into the initial embedding representation of the protein sequence based on the protein safety knowledge graph to obtain an enhanced embedding representation includes: Find triplets containing protein sequences and gene ontology terms or / and protein function descriptions from the protein safety knowledge graph to form a local graph, and use the graph model to learn the embedding representation of the triplets in the local graph. During learning, the graph model parameters and the embedding representation of the triplets are updated based on the interval loss function. After learning, the embedding representation corresponding to the protein sequence in the triplet is extracted as its enhanced embedding representation; Among them, the loss function of the interval is expressed as: L=∑ (h,r,t)∈S ∑ (h′,r,t′)∈S′ max(0,γ+d(h+r,t)-d(h′+r,t′) where (h, r, t) is the correct triple extracted from the protein safety knowledge graph, (h′, r, t′) is the incorrect triple generated by replacing the head entity or the tail entity, S is the set of correct triplets corresponding to the local graph, S′ is the set of incorrect triplets, d(·) represents the distance metric between the head and tail entities, and γ represents the safety interval γ.

5. The protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to claim 1 is characterized in that: When performing one-stage adversarial penalty learning on the generative adversarial network, the loss function used includes the generator loss function L G And the loss function L of the penalty model D ; Where z represents the distribution p from the user-enhanced embedding representation z (z), G(z) represents the new protein embedding representation generated by the generator G according to the sampled data z, D(·) represents the discriminant result of the input, and x represents the distribution p of the real protein embedding representation from the harmless d (x) is the sample data, and E represents the expected value.

6. The protein safety controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to claim 1, characterized in that: The reward value R(G(z;θ)) is constructed based on the safety assessment result output by the penalty model, which is expressed as: R(G(z;θ))=-log(1-D(G(z;θ))) The reward function J(θ) is constructed based on the reward value: Where z represents the distribution p from the user-enhanced embedding representation z In (z), G(z; θ) represents the new protein embedding representation generated by the generator G with parameter θ according to the sampled data z, D(·) represents the discrimination result of the input, and E represents the expected value.

7. The protein safety controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to claim 1 or 6, characterized in that: When constructing a reward function based on the reward value for two-stage reinforcement learning training, the REINFORCE policy gradient algorithm is used to calculate the gradient of the reward function to update the parameters of the generator.

8. A protein safety controllable generation device based on adversarial reinforcement learning of protein safety knowledge graph, characterized in that: include: A graph construction module, which is used to construct a protein safety knowledge graph, which includes gene ontology knowledge, harmful protein knowledge, and the association relationship between knowledge; An embedding representation module, which is used to extract an initial embedding representation of the harmful protein sequence using a masked language model, and at the same time uses an encoder to encode the embedding vectors of gene ontology terms contained in gene ontology knowledge and the embedding vectors of protein function descriptions contained in the harmful protein knowledge; A knowledge embedding module, which is used to fuse the embedding vectors of gene ontology terms and protein function descriptions into the initial embedding representation of the protein sequence based on the protein safety knowledge graph to obtain an enhanced embedding representation; An adversarial penalty learning module is used to construct a generative adversarial network including a generator and a penalty model, wherein the generator is used to generate a new protein embedding representation based on the input data, the penalty model evaluates the security of the new protein embedding representation and the true protein embedding representation respectively, samples the sample data from the enhanced embedding representation, and uses the sample data to perform a one-stage adversarial training on the generative adversarial network, optimizing the parameters of the generator and the penalty model in one stage; The reinforcement learning module is used to fix the parameters of the penalty model after the first stage of adversarial training, and to construct the reward value based on the security assessment results output by the penalty model. The reward function is constructed based on the reward value to conduct the second stage of reinforcement learning training, and the parameters of the generator are optimized in the second stage. The inference application module is used to generate safe and controllable new proteins using the generator after two-stage parameter optimization.

9. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the protein safety and controllable generation method based on adversarial reinforcement learning based on protein safety knowledge graph according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the protein safety and controllable generation method based on adversarial reinforcement learning of protein safety knowledge graph according to any one of claims 1 to 7 is implemented.

Images