Training method of antibody structure prediction model, antibody structure prediction method and equipment

Abstract

The invention discloses a training method of an antibody structure prediction model, an antibody structure prediction method and equipment. The training method specifically comprises the following steps: acquiring high-dimensional vector representation of an amino acid sequence of an antibody sample; performing feature extraction on the high-dimensional vector through a feature extraction module in the initial antibody structure prediction model to obtain a first low-dimensional representation vector and a second low-dimensional representation vector; and training an initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain an antibody structure prediction model. The feature extraction module adopts the linear unit to perform feature extraction, and model parameters of the antibody structure prediction model are reduced, so that on one hand, the storage space required by the antibody structure prediction model can be reduced, and on the other hand, the parameter quantity required by training of the antibody structure prediction model can be reduced; and the training speed of the antibody structure prediction model is improved.

Description

Technical Field

[0001] The present application relates to the field of biopharmaceutical technology, and in particular to a training method for an antibody structure prediction model, an antibody structure prediction method and a device. Background Art

[0002] Antibody structure prediction is an important research direction in the field of bioinformatics and computational biology. It is of great significance for understanding antibody functions, designing new therapeutic antibodies, and developing vaccines. In recent years, with the development of machine learning technology, especially the application of deep learning methods, significant progress has been made in the field of antibody structure prediction. Existing antibody structure prediction models generally use large-scale neural network models, which means that the neural network has a large number of network parameters that need to be trained, resulting in a large amount of GPU resources and a lot of time required for training during the training process, which seriously affects the research efficiency and model iteration speed. In addition, the trained antibody structure prediction model requires a large amount of storage space to save model parameters and intermediate calculation results, which makes its use limited by hardware resources.

[0003] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a training method for an antibody structure prediction model, an antibody structure prediction method and a device in view of the deficiencies in the prior art.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for training an antibody structure prediction model, wherein the training method for the antibody structure prediction model specifically includes:

[0006] Obtaining an amino acid coding sequence of the amino acid sequence of the antibody sample, and determining a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence;

[0007] Inputting the high-dimensional vector representation into a feature extraction module in an initial antibody structure prediction model, and determining an attention encoding matrix through an attention unit in the feature extraction module;

[0008] Determine a first representation vector by a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix;

[0009] Determine a second representation vector by a second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector;

[0010] Based on the first representation vector and the second representation vector, the isovariant attention module in the initial antibody structure prediction model is trained to obtain the antibody structure prediction model.

[0011] The training method of the antibody structure prediction model, wherein the step of obtaining the amino acid coding sequence of the amino acid sequence of the antibody sample specifically comprises:

[0012] Each amino acid in the amino acid sequence of the antibody sample is encoded to obtain an amino acid coding sequence.

[0013] The training method of the antibody structure prediction model, wherein the high-dimensional vector representation of the amino acid sequence of the antibody sample determined based on the amino acid coding sequence is specifically:

[0014] The amino acid coding sequence is input into a language model in the initial antibody structure prediction model, and a high-dimensional vector representation of the amino acid sequence of the antibody sample is output through the language model.

[0015] The training method of the antibody structure prediction model, wherein the vector dimension of the first representation vector and the vector dimension of the second representation vector are both smaller than the vector dimension of the high-dimensional vector representation.

[0016] The training method of the antibody structure prediction model, wherein the inputting of the high-dimensional vector representation into a feature extraction module in the initial antibody structure prediction model and determining the attention encoding matrix through the attention unit in the feature extraction module specifically comprises:

[0017] Input the high-dimensional vector representation into a first attention encoder in the attention unit, and output an intermediate attention encoding matrix through the first attention encoder;

[0018] The intermediate attention coding matrix is ​​input into the second attention encoder in the attention unit, and the attention coding matrix is ​​output through the second attention encoder.

[0019] The training method of the antibody structure prediction model, wherein the determining the first representation vector by the first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix specifically includes:

[0020] Fusing the high-dimensional vector representation with the attention encoding matrix to obtain a first fused matrix;

[0021] Inputting the first fusion matrix into a first linear layer in the first linear unit, and outputting an intermediate high-dimensional vector representation through the first linear layer;

[0022] The intermediate high-dimensional vector representation is input into a second linear layer in the first linear unit, and a first representation vector is output through the second linear layer.

[0023] The training method of the antibody structure prediction model, wherein the determining the second representation vector by the second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector specifically includes:

[0024] Fusing the attention encoding matrix and the first representation vector to obtain a second fused matrix;

[0025] The second fusion matrix is ​​input into a third linear layer in the second linear unit, and a second representation vector is output through the third linear layer.

[0026] The training method of the antibody structure prediction model, wherein the step of training the isovariant attention module in the initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain the antibody structure prediction model specifically comprises:

[0027] Inputting the first representation vector and the second representation vector into an isovariant attention module in the initial antibody structure prediction model, and determining the predicted antibody structure of the antibody sample through the isovariant attention module;

[0028] The isovariant attention module in the initial antibody structure prediction model is trained based on the predicted antibody structure and the actual antibody structure of the antibody sample to obtain an antibody structure prediction model.

[0029] The training method of the antibody structure prediction model, wherein the inputting the first representation vector and the second representation vector into the isovariant attention module in the initial antibody structure prediction model, and determining the predicted antibody structure of the antibody sample by the isovariant attention module specifically comprises:

[0030] Acquiring an initial antibody structure of the antibody sample, wherein the initial antibody structure includes an initial three-dimensional coordinate of each amino acid in the amino acid sequence of the antibody sample;

[0031] Inputting the initial antibody structure, the first representation vector and the second representation vector into an isovariant attention unit in the isovariant attention module, and determining an attention weight matrix through the isovariant attention unit;

[0032] Based on the attention weight matrix and the initial antibody structure, the predicted three-dimensional coordinates of each amino acid are determined by the third linear unit in the isovariant attention module to obtain the predicted antibody structure of the antibody sample.

[0033] The training method of the antibody structure prediction model, wherein the step of training the isovariant attention module in the initial antibody structure prediction model based on the predicted antibody structure and the real antibody structure of the antibody sample to obtain the antibody structure prediction model specifically comprises:

[0034] Constructing a frame alignment point error loss function based on the predicted antibody structure and the real antibody structure of the antibody sample;

[0035] The equivariant attention module in the initial antibody structure prediction model is trained based on the frame alignment point error loss function to obtain an antibody structure prediction model.

[0036] The second aspect of the present application provides an antibody structure prediction method, which uses an antibody structure prediction model constructed based on the training method of the above-mentioned antibody structure prediction model, and the antibody structure prediction method specifically includes:

[0037] Obtaining the amino acid sequence of the antibody to be predicted, and determining the amino acid coding sequence of the amino acid sequence;

[0038] The amino acid coding sequence is input into the antibody structure prediction model, and the predicted antibody structure of the antibody to be predicted is determined by the antibody structure prediction model, wherein the predicted antibody structure includes the predicted three-dimensional coordinates of each amino acid in the amino acid sequence of the predicted antibody.

[0039] The third aspect of the present application provides a device for constructing an antibody structure prediction model, wherein the device for constructing an antibody structure prediction model specifically comprises:

[0040] An acquisition module, used to acquire an amino acid coding sequence of the amino acid sequence of the antibody sample, and determine a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence;

[0041] A control module, configured to input the high-dimensional vector representation into a feature extraction module in an initial antibody structure prediction model, determine an attention encoding matrix through an attention unit in the feature extraction module; determine a first representation vector through a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix; determine a second representation vector through a second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector;

[0042] A training module is used to train an isovariant attention module in an initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain an antibody structure prediction model.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the training method of the antibody structure prediction model as described above.

[0044] A fifth aspect of the present application provides a terminal device, comprising: a processor and a memory;

[0045] The memory stores a computer-readable program executable by the processor;

[0046] When the processor executes the computer-readable program, the steps in any of the above-described methods for training an antibody structure prediction model are implemented.

[0047] Beneficial effects:

[0048] (1) The present application uses linear units to perform feature extraction. Since the model parameters of the antibody structure prediction model are reduced, on the one hand, the training speed of the antibody structure prediction model is improved, and on the other hand, the storage space required for the antibody structure prediction model can be reduced.

[0049] (2) The present application converts a high-dimensional vector representation into two low-dimensional first representation vectors and second representation vectors, thereby reducing the amount of subsequent calculations for determining the predicted antibody structure based on the first representation vector and the second representation vector, thereby improving the training speed of the antibody structure prediction model.

[0050] (3) The present application reduces the storage space and computing resources required by the antibody structure prediction model in the inference stage by reducing model parameters and calculation amount, thereby reducing the requirements of the antibody structure prediction model for hardware resources and avoiding the limitation of hardware resources on the use of the antibody structure prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A flowchart of a method for training an antibody structure prediction model provided in an embodiment of the present application.

[0053] Figure 2 This is a principle flow chart of a specific example of a feature extraction module in the antibody structure prediction model provided in an embodiment of the present application.

[0054] Figure 3A principle flow chart of a specific example of the training method of the antibody structure prediction model provided in the embodiments of the present application.

[0055] Figure 4 This is a schematic diagram of the structure of the device for constructing the antibody structure prediction model provided in the embodiments of the present application.

[0056] Figure 5 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The present application embodiment provides a training method for an antibody structure prediction model, an antibody structure prediction method and an apparatus. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0059] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0060] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0061] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0062] This embodiment provides a method for training an antibody structure prediction model. Figure 1 As shown, the method includes:

[0063] S10. Obtaining an amino acid coding sequence of the amino acid sequence of the antibody sample.

[0064] Specifically, the amino acid coding sequence is obtained by encoding the amino acid sequence of the antibody sample, the sequence length of the amino acid coding sequence is equal to the sequence length of the amino acid sequence, and the amino acid coding in the amino acid coding sequence corresponds to the amino acid in the amino acid sequence. That is to say, the amino acid coding sequence is obtained by encoding each amino acid in the amino acid sequence, wherein each amino acid can be encoded as a unique code representation, for example, using a single letter code as the code representation of the amino acid (such as A represents alanine, R represents arginine, etc.), and the amino acid coding sequence is a one-dimensional sequence with a length of L, wherein L is the sequence length of the amino acid sequence. Of course, in practical applications, other methods can also be used to represent the code representation of amino acids, for example, using numbers as code representations, or using two-dimensional letter vectors as amino acid code representations, etc.

[0065] S20. Determine a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence.

[0066] Specifically, the high-dimensional vector representation is a representation matrix, and it includes a high-dimensional vector representation of each amino acid in the amino acid sequence, and the high-dimensional vector representation of each amino acid is a vector. That is, the high-dimensional vector representation of the amino acid sequence is formed by arranging the high-dimensional vector representation of each amino acid according to the amino acid position of each amino acid in the amino acid sequence, so that the high-dimensional vector representation of the amino acid sequence of the antibody sample contains rich information of each amino acid in the context of the entire amino acid sequence, providing rich knowledge information for subsequent antibody structure prediction.

[0067] Exemplarily, the high-dimensional vector representation of the amino acid sequence of the antibody sample determined based on the amino acid coding sequence is specifically:

[0068] The amino acid coding sequence is input into the language model in the initial antibody structure prediction model, and a high-dimensional vector representation of the amino acid sequence of the antibody sample is output through the language model.

[0069] Specifically, the initial antibody structure prediction model also includes a language model, which is used to determine a high-dimensional vector representation, wherein the input item of the language model is an amino acid coding sequence, and the output item is a high-dimensional vector representation of the amino acid sequence of the antibody sample, that is, the context representation of each amino acid is learned from the amino acid coding sequence by the language model to obtain a high-dimensional vector representation of the amino acid sequence of the antibody sample. For example, the sequence length of the amino acid sequence is L, and the dimension of the high-dimensional vector representation of each amino acid is N, then the dimension of the high-dimensional vector representation of the amino acid sequence of the antibody sample is L*N, wherein L and N are both positive integers. Preferably, N is a positive integer greater than 1. Of course, in practical applications, the language model may not be included in the initial antibody structure prediction model, but is two independent models with the initial antibody structure prediction model, and the high-dimensional vector representation can be obtained by the language model first, and then the high-dimensional vector representation is used as the input item of the initial antibody structure prediction model.

[0070] Further, the language model is a pre-trained language model, and the language model may include an encoder, and the encoder may include a multi-layer self-attention mechanism and a feed-forward neural network, etc., and the complex pattern and long-range dependency in the amino acid sequence are captured by the language model, and the context-related feature representation (i.e., high-dimensional vector representation) is generated for the amino acid sequence of the antibody sample. For example, the language model can adopt an ESM model or a ProtBERT model, etc. The embodiment of the present application, by adopting a pre-trained language model, can directly extract high-dimensional vector representations through a language model, so that the model parameters that need to be trained in the initial antibody structure prediction model can be reduced, so that the training speed of the antibody structure prediction model can be improved. Of course, in practical applications, the language model can also adopt a language model that has not been pre-trained, or, other network models can be used to extract high-dimensional vector representations, such as, trained deep learning models, etc.

[0071] S30, inputting the high-dimensional vector representation into a feature extraction module in an initial antibody structure prediction model, determining an attention coding matrix through an attention unit in the feature extraction module, determining a first representation vector through a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention coding matrix, and determining a second representation vector through a second linear unit in the feature extraction module according to the attention coding matrix and the first representation vector.

[0072] Specifically, the input item of the feature extraction module is a high-dimensional vector representation, wherein, when the high-dimensional vector representation is extracted by a language model, and the language model is included in the initial antibody structure prediction model, such as Figure 2As shown, the language model is connected to the feature extraction module, and the high-dimensional vector representation extracted by the language model is input into the feature extraction module. The high-dimensional vector representation is learned by the feature extraction module to obtain a first representation vector and a second representation vector of the amino acid sequence, wherein the first representation vector and the second representation vector are both representation matrices, and both contain contextual information of the amino acid in the amino acid sequence.

[0073] like Figure 3 As shown, the feature extraction module includes an attention unit, a first linear unit and a second linear unit, wherein the attention unit is connected to the first linear unit and the second linear unit respectively, the input item of the attention unit is a high-dimensional vector representation, and the output item of the attention unit is an attention coding matrix; the input item of the first linear unit includes a high-dimensional vector representation and an attention coding matrix, and the output item of the first linear unit is a first representation vector; the input item of the second linear unit includes an attention coding matrix and a first representation vector, and the output item of the second linear unit is a second representation vector. The first linear unit and the second linear unit in the embodiment of the present application both have a simple network structure and carry a small amount of model parameters, and can well learn the knowledge information carried by the high-dimensional vector representation, and well reduce the model parameters of the antibody structure prediction model under the premise of ensuring the performance of the model, and can also reduce the computing resources and time required in the training process from the initial antibody structure prediction model to the antibody structure prediction model, thereby improving the training speed of the antibody structure prediction model.

[0074] Exemplarily, inputting the high-dimensional vector representation into a feature extraction module in the initial antibody structure prediction model, and determining the attention encoding matrix through the attention unit in the feature extraction module specifically includes:

[0075] Input the high-dimensional vector representation into a first attention encoder in the attention unit, and output an intermediate attention encoding matrix through the first attention encoder;

[0076] The intermediate attention coding matrix is ​​input into the second attention encoder in the attention unit, and the attention coding matrix is ​​output through the second attention encoder.

[0077] Specifically, Figure 3As shown, the attention unit includes a first attention encoder and a second attention encoder, the first attention encoder and the second attention encoder are cascaded, and the first attention encoder is respectively connected to the first linear unit and the second linear unit, wherein the input item of the first attention encoder is a high-dimensional vector representation, the input item of the second attention encoder is an intermediate attention coding matrix output by the first attention encoder, and the output item of the second attention encoder is an attention coding matrix. The present application can better learn the amino acid information contained in the high-dimensional vector representation and the contextual information of the amino acid in the amino acid sequence by combining the first attention encoder and the second attention encoder, such as the combination in Alphafo ld. Of course, in practical applications, the attention unit may also include only one attention encoder, or more than two attention encoders, for example, including 3 attention encoders, etc.

[0078] Further, the intermediate attention encoding matrix is ​​obtained by encoding the high-dimensional vector representation through the first attention encoder, wherein the matrix dimension of the intermediate attention encoding matrix is ​​the same as the matrix dimension of the high-dimensional vector representation, for example, if the matrix dimension of the high-dimensional vector representation is L*N, then the matrix dimension of the intermediate attention encoding matrix is ​​L*N. The attention encoding matrix is ​​obtained by encoding the intermediate attention encoding matrix through the second attention encoder, wherein the matrix dimension of the attention encoding matrix is ​​smaller than the matrix dimension of the intermediate attention encoding matrix, for example, the matrix dimension of the intermediate attention encoding matrix is ​​L*N, and the matrix dimension of the attention encoding matrix is ​​L*N1, wherein N>N1, and N1 and N are both positive integers.

[0079] It should be noted that, in practical applications, the matrix dimension of the intermediate attention coding matrix may be different from the matrix dimension of the high-dimensional vector representation, and the matrix dimension of the attention coding matrix may be equal to or greater than the matrix dimension of the intermediate attention coding matrix. In the implementation of this application, the intermediate attention coding matrix is ​​encoded as an attention coding matrix with a dimension greater than that of the intermediate attention coding matrix, which can reduce the amount of calculation for determining the first representation vector and the second representation vector based on the attention coding matrix, thereby further reducing the amount of calculation of the antibody structure prediction model.

[0080] Exemplarily, determining the first representation vector by a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix specifically includes:

[0081] Fusing the high-dimensional vector representation with the attention encoding matrix to obtain a first fused matrix;

[0082] Inputting the first fusion matrix into a first linear layer in the first linear unit, and outputting an intermediate high-dimensional vector representation through the first linear layer;

[0083] Input the intermediate high-dimensional vector representation into the second linear layer in the first linear unit, and output a first representation vector through the second linear layer.

[0084] Specifically, the first fusion matrix is obtained by fusing the high-dimensional vector representation and the attention encoding matrix. The corresponding fusion method of the first fusion matrix can adopt matrix addition, matrix splicing, or matrix weighting methods, etc. In the embodiments of the present application, the corresponding fusion method of the first fusion matrix adopts the matrix addition method. That is to say, the high-dimensional vector representation and the attention encoding matrix are fused by matrix addition to obtain the first fusion matrix. In addition, in the embodiments of the present application, since the matrix dimension of the high-dimensional vector representation is greater than the matrix dimension of the attention encoding matrix, when fusing by the matrix addition method, the attention encoding matrix can be padded first so that the matrix dimension of the padded attention encoding matrix is equal to the matrix dimension of the high-dimensional vector representation, and then the attention encoding matrix and the high-dimensional vector representation are fused by matrix addition. The matrix dimension of the fused first fusion matrix is equal to the matrix dimension higher than the vector representation. Of course, in practical applications, when the matrix dimension of the attention encoding matrix is equal to the high-dimensional vector representation, fusion can be directly performed by matrix addition; when the matrix dimension of the attention encoding matrix is greater than the high-dimensional vector representation, the high-dimensional vector representation can be padded first, and the matrix dimension of the fused first fusion matrix is equal to the matrix dimension of the attention encoding matrix.

[0085] The first linear unit is used to perform feature extraction based on the attention encoding matrix, and a first representation vector for reflecting the feature information of the amino acid sequence is extracted through the first linear unit. The first representation vector is a feature matrix, and the matrix dimension of the first representation vector is smaller than the matrix dimension of the high-dimensional vector representation. For example, the matrix dimension of the high-dimensional vector is represented as L*N, and the matrix dimension of the first representation vector is L*N1, where N1 < N, and both N1 and N are positive integers. Specifically, as Figure 3 shown, the first linear unit may include a first linear layer and a second linear layer. The first linear layer is cascaded with the second linear layer, and the first linear layer is connected to the second attention encoder. Among them, the input item of the first linear layer is the first fusion matrix obtained by fusing the high-dimensional vector representation and the attention encoding matrix, and the input item of the second linear layer is the intermediate high-dimensional vector representation output through the first linear layer. In the embodiments of the present application, by using the first linear layer and the second linear layer for feature extraction, the model parameters included in the feature extraction module are reduced. On the one hand, this can reduce the storage space required for the antibody structure prediction model, and on the other hand, it can reduce the number of parameters that need to be trained in the antibody structure prediction model, improving the training speed of the antibody structure prediction model.

[0086] It should be noted that the first linear layer and the second linear layer in the first linear unit have the same structure, and the parameters of the two may be the same or different. For example, the first linear layer and the second linear layer are two identical linear layers, or the first linear layer and the second linear layer are two linear layers used in combination, such as the combination used in AlexNet, that is, the parameters of the first linear layer and the second linear layer are different. In addition, in practical applications, the first linear unit may also include only one linear layer, or more than two linear layers, for example, three layers, four layers, etc. In the embodiment of the present application, the first linear unit includes two linear layers, and feature extraction is performed through the two linear layers, so that limited feature information can be extracted while minimizing model parameters.

[0087] Exemplarily, the determining the second representation vector by the second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector specifically includes:

[0088] Fusing the attention encoding matrix and the first representation vector to obtain a second fused matrix;

[0089] The second fusion matrix is ​​input into a third linear layer in the second linear unit, and a second representation vector is output through the third linear layer.

[0090] Specifically, the second fusion matrix is ​​obtained by fusing the first representation vector and the attention coding matrix, and the corresponding fusion method of the second fusion matrix can be a matrix addition method, a matrix splicing method, or a matrix weighting method. In an embodiment of the present application, the corresponding fusion method of the second fusion matrix adopts a matrix addition method, that is, the first representation vector and the attention coding matrix are fused by matrix addition to obtain a second fusion matrix. In addition, in an embodiment of the present application, the matrix dimension of the first representation vector is equal to the matrix dimension of the attention coding matrix, both of which are L*N1, and the first representation vector and the attention coding matrix can be directly fused by matrix addition. Of course, in actual applications, when the matrix dimension of the first representation vector is different from the matrix dimension of the attention coding matrix, the first representation vector and the attention coding matrix can be filled first and then the first representation vector and the attention coding matrix can be fused by matrix addition.

[0091] The second linear unit is used to perform feature extraction based on the attention encoding matrix, and a second representation vector for reflecting the feature information of the amino acid sequence is extracted through the second linear unit. Among them, the second representation vector is a feature matrix, and the matrix dimensions of the second representation vector are all smaller than the matrix dimensions of the high-dimensional vector representation. For example, the matrix dimension of the high-dimensional vector is represented as L*N, and the matrix dimension of the first representation vector is L*N2, where N2 < N, and both N2 and N are positive integers. Specifically, the second linear unit may include a third linear layer, and the third linear layer is respectively connected to the second linear layer and the second attention encoder. Among them, the input item of the third linear layer is the second fusion matrix obtained by fusing the attention encoding matrix and the first representation vector. By adopting the third linear layer to perform feature extraction in the embodiment of the present application, the model parameters included in the feature extraction module can be reduced. On the one hand, this can reduce the storage space required by the antibody structure prediction model, and on the other hand, it can reduce the number of parameters that need to be trained in the antibody structure prediction model, thereby improving the training speed of the antibody structure prediction model.

[0092] It should be noted that the third linear layer in the second linear unit may be the same as the first linear layer or the second linear layer in the first linear unit, or may be different from both the first linear layer and the second linear layer in the first linear unit. Among them, "different" means the same structure but different parameters, and "same" means the same structure and parameters. For example, the third linear layer may be AlexNet. In addition, in practical applications, the second linear unit may also include multiple linear layers, such as 2 layers, 3 layers, etc. In the embodiment of the present application, the second linear unit includes 1 layer of linear layer, and 1 layer of linear layer is used to perform feature extraction, which can minimize the model parameters while extracting limited feature information.

[0093] S40. Based on the first representation vector and the second representation vector, train the equivariant attention module in the initial antibody structure prediction model to obtain the antibody structure prediction model.

[0094] Specifically, the antibody structure prediction model is obtained by training the equivariant attention module in the initial antibody structure prediction model. The model structure of the antibody structure prediction model is the same as that of the initial antibody structure prediction model. The difference between the two is that the model parameters of the initial antibody structure prediction model are initial parameters, and the model parameters of the antibody structure prediction model are model parameters after training.

[0095] The initial antibody structure prediction model may include a feature extraction module and an equivariant attention module. In addition, when performing feature extraction through the feature extraction module, a high-dimensional vector representation of the amino acid sequence of the antibody sample will be obtained first, and the high-dimensional vector representation can be extracted by a pre-trained language model. For this reason, such as Figure 2As shown, the initial antibody structure prediction model may also include a language model, that is, the initial antibody structure prediction model may include a language model, a feature extraction module and an isovariant attention module, wherein the language model is connected to the feature extraction module, and the feature extraction module is connected to the isovariant attention model. Of course, in practical applications, the language model and the initial antibody structure prediction model may be two independent models, and when performing antibody structure prediction, the language model and the initial antibody structure prediction model may be used in conjunction with each other. In addition, it should be noted that when obtaining the antibody structure prediction model, in addition to training the isovariant attention module, the feature extraction module may also be trained, and when the initial antibody structure prediction model includes a language model, the language model, the feature extraction module and the isovariant attention module may be trained.

[0096] The isovariant attention module is used to predict the predicted antibody structure of the antibody sample based on the feature information provided by the first representation vector and the second representation vector, wherein the isovariant attention module is configured with an isovariant attention mechanism, and the predicted antibody structure includes the predicted three-dimensional coordinates of each amino acid in the amino acid sequence of the antibody sample. That is to say, the isovariant attention module predicts the predicted three-dimensional coordinates of each amino acid in the amino acid sequence of the antibody sample through the isovariant attention mechanism to obtain the predicted antibody structure, and then the isovariant attention module is trained based on the predicted antibody structure to obtain the antibody structure prediction model. Among them, the isovariant attention mechanism is a special neural network structure that uses the principle of the attention mechanism to process data in three-dimensional space while maintaining rotation and translation invariance, so as to avoid the influence of the direction and position of the protein structure in space on the predicted antibody structure during the antibody structure prediction process, so as to improve the accuracy of the predicted antibody structure.

[0097] In one implementation, the training of the isovariant attention module in the initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain the antibody structure prediction model specifically includes:

[0098] S41, inputting the first representation vector and the second representation vector into an isovariant attention module in the initial antibody structure prediction model, and determining the predicted antibody structure of the antibody sample through the isovariant attention module;

[0099] S42. Training the isovariant attention module in the initial antibody structure prediction model based on the predicted antibody structure and the actual antibody structure of the antibody sample to obtain an antibody structure prediction model.

[0100] Specifically, in step S41, the isovariant attention module processes the initial antibody structure of the antibody sample through the isovariant attention mechanism, and maintains rotation and translation invariance to obtain a predicted antibody structure. That is to say, the contextual representation between each amino acid in the amino acid sequence of the antibody sample is learned from the first representation vector and the second representation vector by the isovariant attention module to determine the predicted antibody structure of the antibody sample. Among them, when determining the predicted antibody structure through the isovariant attention module, the isovariant attention module can first learn the contextual representation between each amino acid in the amino acid sequence based on the first representation vector and the second representation vector, and then update the initial antibody structure of the antibody sample based on the learned contextual representation and maintain the isovariant property to obtain the predicted antibody structure.

[0101] Exemplarily, the step of inputting the first representation vector and the second representation vector into an isovariant attention module in the initial antibody structure prediction model, and determining the predicted antibody structure of the antibody sample through the isovariant attention module specifically includes:

[0102] S411, obtaining an initial antibody structure of the antibody sample, wherein the initial antibody structure includes an initial three-dimensional coordinate of each amino acid in the amino acid sequence of the antibody sample;

[0103] S412, inputting the initial antibody structure, the first representation vector and the second representation vector into an isovariant attention unit in the isovariant attention module, and determining an attention weight matrix through the isovariant attention unit;

[0104] S413. Based on the attention weight matrix and the initial antibody structure, the predicted three-dimensional coordinates of each amino acid are determined by the third linear unit in the isovariant attention module to obtain the predicted antibody structure of the antibody sample.

[0105] Specifically, the initial antibody structure includes the initial three-dimensional coordinates of each amino acid in the amino acid sequence of the antibody sample, and the initial antibody structure can be obtained by initializing the three-dimensional coordinate representation of each amino acid in the amino acid sequence. The initialization method can be a random coordinate method or a default coordinate method.

[0106] After obtaining the initial antibody structure, the initial antibody structure, the first representation vector and the second representation vector are input into the equivariant attention mechanism in the equivariant attention module. For each amino acid in the amino acid sequence, the equivariant attention mechanism is used to first calculate the relationship information between the three-dimensional coordinates of the amino acid and other amino acids in the amino acid sequence except the amino acid, wherein the relationship information includes relative distance and direction information. Then, the attention weights between the amino acid and other amino acids in the amino acid sequence except the amino acid are calculated based on the calculated relationship information, and the attention weight matrix of the amino acid sequence is obtained based on the attention weights between each amino acid and other amino acids in the amino acid sequence except the amino acid.

[0107] The third linear unit is used to update the initial antibody structure based on the attention weight to obtain the predicted three-dimensional coordinates of the amino acids, wherein the third linear unit maintains the equivariant property while updating the initial antibody structure, that is, the initial antibody structure is rotated or translated as a whole, and the relative relationship between the amino acids remains unchanged. The third linear unit may include one or more linear layers. In the embodiment of the present application, the third linear unit includes one linear layer, which is recorded as the fourth linear layer such as AlexNet. The attention weight matrix and the initial antibody structure are input into the fourth linear layer, and the predicted antibody structure is output through the fourth linear layer.

[0108] Further, in step S42, the predicted antibody structure includes the predicted three-dimensional coordinates of each amino acid in the amino acid sequence of the antibody sample, and the real antibody structure is used as the real label of the antibody sample, so as to supervise the training process of the antibody sample prediction model as the supervision data of the antibody structure prediction model, wherein the real antibody structure includes the real three-dimensional coordinates of each amino acid in the amino acid sequence of the antibody sample. In other words, the predicted three-dimensional coordinates of each amino acid in the predicted antibody structure are supervised based on the real three-dimensional coordinates of each amino acid to optimize the model parameters in the initial antibody structure prediction model, so that the predicted antibody structure determined by the antibody structure prediction model obtained by training the initial antibody structure prediction model can be closer to the real antibody structure.

[0109] Exemplarily, the training of the initial antibody structure prediction model based on the predicted antibody structure and the actual antibody structure of the antibody sample to obtain the antibody structure prediction model specifically includes:

[0110] Constructing a frame alignment point error loss function based on the predicted antibody structure and the real antibody structure of the antibody sample;

[0111] The equivariant attention module in the initial antibody structure prediction model is trained based on the frame alignment point error loss function to obtain an antibody structure prediction model.

[0112] Specifically, the frame alignment point error loss function reflects the error between the predicted antibody structure and the real antibody structure, and the training structure of the antibody structure prediction model is supervised by controlling the frame alignment point error loss function. That is, when the equivariant attention module in the initial antibody structure prediction model is trained based on the frame alignment point error loss function, the end condition of the training is that the frame alignment point error loss function converges.

[0113] Further, when the equivariant attention module in the initial antibody structure prediction model is trained based on the frame alignment point error loss function, if the frame alignment point error loss function does not meet the preset convergence condition, the initial antibody structure is reacquired, and the new predicted antibody structure is re-determined based on the re-acquired initial antibody structure, the first representation vector and the second representation vector by the equivariant attention module, and so on, when the frame alignment point error loss function meets the preset convergence condition, the antibody structure prediction model is obtained. Of course, in practical applications, when the training data set corresponding to the antibody structure prediction model includes multiple antibody samples, the above steps S10-S40 can be performed for each antibody sample to obtain the antibody structure prediction model.

[0114] In summary, the embodiment of the present application provides a training method for an antibody structure prediction model, and the training method for the antibody structure prediction model specifically includes: obtaining an amino acid coding sequence of an amino acid sequence of an antibody sample, and determining a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence; inputting the high-dimensional vector representation into a feature extraction module in the initial antibody structure prediction model, determining an attention coding matrix through an attention unit in the feature extraction module, determining a first representation vector through a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention coding matrix, and determining a second representation vector through a second linear unit in the feature extraction module according to the attention coding matrix and the first representation vector; based on the first representation vector and the second representation vector, training the equivariant attention module in the initial antibody structure prediction model to obtain an antibody structure prediction model. The present application uses a linear unit to perform feature extraction to obtain a first representation vector and a second representation vector, thereby reducing the model parameters of the antibody structure prediction model, which can reduce the storage space required for the antibody structure prediction model on the one hand, and reduce the amount of parameters required to train the antibody structure prediction model on the other hand, thereby improving the training speed of the antibody structure prediction model. At the same time, the present application reduces the storage space and computing resources required by the antibody structure prediction model in the inference stage by reducing model parameters and calculation amount, thereby reducing the requirements of the antibody structure prediction model for hardware resources and avoiding the limitation of hardware resources on the use of the antibody structure prediction model.

[0115] Based on the above-mentioned antibody structure prediction model training method, this embodiment provides a device for constructing an antibody structure prediction model, such as Figure 4 As shown, the device for constructing the antibody structure prediction model specifically includes:

[0116] An acquisition module 100 is used to acquire an amino acid coding sequence of an amino acid sequence of an antibody sample, and determine a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence;

[0117] The control module 200 is used to input the high-dimensional vector representation into a feature extraction module in the initial antibody structure prediction model, and determine an attention encoding matrix through an attention unit in the feature extraction module; determine a first representation vector through a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix; and determine a second representation vector through a second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector;

[0118] The training module 300 is used to train the isovariant attention module in the initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain the antibody structure prediction model.

[0119] Based on the above-mentioned antibody structure prediction model training method, this embodiment provides an antibody structure prediction method, using the antibody structure prediction model constructed by the antibody structure prediction model training method based on the above-mentioned embodiment, the antibody structure prediction method specifically includes:

[0120] Obtaining the amino acid sequence of the antibody to be predicted, and determining the amino acid coding sequence of the amino acid sequence;

[0121] The amino acid coding sequence is input into the antibody structure prediction model, and the predicted antibody structure of the antibody to be predicted is determined by the antibody structure prediction model, wherein the predicted antibody structure includes the predicted coordinates of each amino acid in the amino acid sequence of the predicted antibody.

[0122] Based on the above-mentioned antibody structure prediction model training method, this embodiment provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the antibody structure prediction model training method as described in the above-mentioned embodiment.

[0123] Based on the above-mentioned antibody structure prediction model training method, the present application also provides a terminal device, such as Figure 5As shown, it includes at least one processor 20, a display screen 21, and a memory 22, and may also include a communication interface 23 and a bus 24. The processor 20, the display screen 21, the memory 22, and the communication interface 23 may communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0124] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0125] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0126] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0127] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for training an antibody structure prediction model, characterized in that: The training method of the antibody structure prediction model specifically includes: Obtaining an amino acid coding sequence of the amino acid sequence of the antibody sample, and determining a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence; Inputting the high-dimensional vector representation into a feature extraction module in an initial antibody structure prediction model, and determining an attention encoding matrix through an attention unit in the feature extraction module; Determine a first representation vector by a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix; Determine a second representation vector by a second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector; Based on the first representation vector and the second representation vector, the isovariant attention module in the initial antibody structure prediction model is trained to obtain the antibody structure prediction model.

2. The method for training an antibody structure prediction model according to claim 1, characterized in that: The amino acid coding sequence of the amino acid sequence of the antibody sample obtained specifically includes: Each amino acid in the amino acid sequence of the antibody sample is encoded to obtain an amino acid coding sequence.

3. The method for training an antibody structure prediction model according to claim 1, characterized in that: The high-dimensional vector representation of the amino acid sequence of the antibody sample determined based on the amino acid coding sequence is specifically: The amino acid coding sequence is input into a language model in the initial antibody structure prediction model, and a high-dimensional vector representation of the amino acid sequence of the antibody sample is output through the language model.

4. The method for training an antibody structure prediction model according to claim 1, characterized in that: The vector dimension of the first representation vector and the vector dimension of the second representation vector are both smaller than the vector dimension of the high-dimensional vector representation.

5. The method for training an antibody structure prediction model according to any one of claims 1 to 4, characterized in that: The step of inputting the high-dimensional vector representation into a feature extraction module in the initial antibody structure prediction model and determining an attention encoding matrix through an attention unit in the feature extraction module specifically includes: Input the high-dimensional vector representation into a first attention encoder in the attention unit, and output an intermediate attention encoding matrix through the first attention encoder; The intermediate attention coding matrix is ​​input into the second attention encoder in the attention unit, and the attention coding matrix is ​​output through the second attention encoder.

6. The method for training an antibody structure prediction model according to any one of claims 1 to 4, characterized in that: Determining a first representation vector by a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix specifically includes: Fusing the high-dimensional vector representation with the attention encoding matrix to obtain a first fused matrix; Inputting the first fusion matrix into a first linear layer in the first linear unit, and outputting an intermediate high-dimensional vector representation through the first linear layer; The intermediate high-dimensional vector representation is input into a second linear layer in the first linear unit, and a first representation vector is output through the second linear layer.

7. The method for training an antibody structure prediction model according to any one of claims 1 to 4, characterized in that: Determining a second representation vector by a second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector specifically includes: Fusing the attention encoding matrix and the first representation vector to obtain a second fused matrix; The second fusion matrix is ​​input into a third linear layer in the second linear unit, and a second representation vector is output through the third linear layer.

8. The method for training an antibody structure prediction model according to any one of claims 1 to 4, characterized in that: The step of training the isovariant attention module in the initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain the antibody structure prediction model specifically includes: Inputting the first representation vector and the second representation vector into an isovariant attention module in the initial antibody structure prediction model, and determining a predicted antibody structure of the antibody sample through the isovariant attention module, wherein the predicted antibody structure includes a predicted three-dimensional coordinate of each amino acid in the amino acid sequence of the antibody sample; The isovariant attention module in the initial antibody structure prediction model is trained based on the predicted antibody structure and the real antibody structure of the antibody sample to obtain an antibody structure prediction model, wherein the real antibody structure includes the real three-dimensional coordinates of each amino acid in the amino acid sequence of the antibody sample.

9. The method for training an antibody structure prediction model according to claim 8, characterized in that: The step of inputting the first representation vector and the second representation vector into an isovariant attention module in the initial antibody structure prediction model, and determining the predicted antibody structure of the antibody sample by the isovariant attention module specifically includes: Acquiring an initial antibody structure of the antibody sample, wherein the initial antibody structure includes an initial three-dimensional coordinate of each amino acid in the amino acid sequence of the antibody sample; Inputting the initial antibody structure, the first representation vector and the second representation vector into an isovariant attention unit in the isovariant attention module, and determining an attention weight matrix through the isovariant attention unit; Based on the attention weight matrix and the initial antibody structure, the predicted three-dimensional coordinates of each amino acid are determined by the third linear unit in the isovariant attention module to obtain the predicted antibody structure of the antibody sample.

10. The method for training an antibody structure prediction model according to claim 8, characterized in that: The step of training the isovariant attention module in the initial antibody structure prediction model based on the predicted antibody structure and the real antibody structure of the antibody sample to obtain the antibody structure prediction model specifically includes: Constructing a frame alignment point error loss function based on the predicted antibody structure and the real antibody structure of the antibody sample; The equivariant attention module in the initial antibody structure prediction model is trained based on the frame alignment point error loss function to obtain an antibody structure prediction model.

11. A method for predicting antibody structure, characterized in that: An antibody structure prediction model constructed using the training method of the antibody structure prediction model according to any one of claims 1 to 9, wherein the antibody structure prediction method specifically comprises: Obtaining the amino acid sequence of the antibody to be predicted, and determining the amino acid coding sequence of the amino acid sequence; The amino acid coding sequence is input into the antibody structure prediction model, and the predicted antibody structure of the antibody to be predicted is determined by the antibody structure prediction model, wherein the predicted antibody structure includes the predicted three-dimensional coordinates of each amino acid in the amino acid sequence of the predicted antibody.

12. A device for constructing an antibody structure prediction model, characterized in that: The antibody structure prediction model construction device specifically includes: An acquisition module, used to acquire an amino acid coding sequence of the amino acid sequence of the antibody sample, and determine a high-dimensional vector representation of the amino acid sequence of the antibody sample based on the amino acid coding sequence; A control module, configured to input the high-dimensional vector representation into a feature extraction module in an initial antibody structure prediction model, determine an attention encoding matrix through an attention unit in the feature extraction module; determine a first representation vector through a first linear unit in the feature extraction module according to the high-dimensional vector representation and the attention encoding matrix; determine a second representation vector through a second linear unit in the feature extraction module according to the attention encoding matrix and the first representation vector; A training module is used to train an isovariant attention module in an initial antibody structure prediction model based on the first representation vector and the second representation vector to obtain an antibody structure prediction model.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the training method of the antibody structure prediction model as described in any one of claims 1 to 10, and / or to implement the steps in the method for predicting the antibody structure as described in claim 11.

14. A terminal device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, it implements the steps in the method for training an antibody structure prediction model as described in any one of claims 1 to 10, and / or implements the steps in the method for predicting antibody structure as described in claim 11.

Images