Protein secondary structure prediction method based on knowledge distillation and convex optimization improvement

Abstract

The invention provides a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, and relates to the technical field of bioinformatics and protein structures. The method comprises the following steps: firstly, converting an original protein amino acid sequence into a 44-dimensional feature vector, generating protein secondary structure soft label probability distribution by adopting a teacher network, and generating an initial protein secondary structure prediction result by adopting a student network; receiving the soft label probability distribution of the protein secondary structure and an initial protein secondary structure prediction result by adopting a discriminator, and outputting a probability value for distinguishing true and false; the discriminator and the student network are alternately trained until a preset convergence condition is reached, and a converged student network is obtained; and finally, storing complete parameters and configuration information of the converged student network as a model file, processing new protein amino acid sequence features by using the model file, and outputting a secondary structure type probability of each amino acid position.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bioinformatics and protein structure prediction, and particularly relates to a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement. BACKGROUND

[0002] In the field of medical informatics, protein secondary structure prediction is a key technology for converting genomic data into clinically useful information. With the popularization of gene sequencing in clinical diagnosis, medical institutions generate massive amounts of protein sequence data every day, which need to be predicted for secondary structure (alpha-helix, beta-sheet, and random coil) to assist in disease diagnosis and drug treatment decisions. This technology has important applications in precision medicine: by analyzing patient genetic mutations to predict protein structure changes, it assists in genetic disease diagnosis; it evaluates structural changes in drug target binding sites to guide personalized medication; and it screens candidate drug targets at high throughput to accelerate the new drug development process. These application scenarios require the prediction system to ensure high accuracy and stable and reliable operation in hospital information systems.

[0003] Current protein secondary structure prediction faces two core technical challenges in medical information processing. First, the contradiction between model deployment and accuracy: medical institutions' information systems usually run in resource-constrained intranet environments, making it difficult to deploy large deep learning models, while small deployable models cannot meet the clinical requirements in terms of prediction accuracy. Second, the training stability problem: medical data processing requires reproducible and verifiable results, but the training process of existing deep learning models has inherent instability, resulting in different results from the same data, affecting the reliability of clinical decision-making.

[0004] To address the model deployment challenge, there are three main types of solutions in existing technologies. DeepMind's AlphaFold3 and Meta's ESMFold use a Transformer architecture containing hundreds of millions of parameters, which can achieve a prediction accuracy of over 90%, but the model file is over 10GB, requiring high-end GPUs to run, and a single prediction takes several hours, which is too costly for small and medium-sized medical institutions in terms of hardware and time. NetSurfP-2.0, SPIDER3, and other lightweight models can run on ordinary workstations, but their prediction accuracy is only 75%-85%, which is unacceptable in clinical applications, and these methods start from scratch without utilizing the knowledge learned from large models. Early traditional methods such as PSIPRED and GOR V use hard label training, which can only represent discrete categories, losing the probability distribution information of the prediction, such as the uncertainty information that a certain position has a 70% probability of being an alpha-helix and a 30% probability of being a coil, resulting in a prediction accuracy usually below 75%.

[0005] To solve the problem of training stability, the existing deep learning method adopts the standard gradient descent algorithm, but the loss function of the deep neural network is non-convex and contains a large number of local optimal points. This leads to the training process being easily trapped in local optimization, and the model prediction accuracy may differ by more than 5% just by changing the random seed. Although adaptive optimizers such as Adam, RMSprop and regularization techniques such as batch normalization and dropout can partially improve the situation, they still cannot provide mathematical convergence guarantees. Some studies use model ensemble or hyperparameter optimization to try to alleviate the problem, but these methods either increase the computational cost exponentially or still cannot fundamentally change the non-convex nature of the optimization problem. In addition, existing methods lack training strategies to improve the authenticity of prediction results through adversarial mechanisms, and cannot effectively distinguish and improve the quality of model generation.

[0006] Although the knowledge distillation technology successfully realizes the knowledge transfer from a large model to a small model in the field of computer vision, it has almost no application in the field of protein structure prediction. The main obstacles are: protein sequences are one-dimensional discrete symbols, which are essentially different from two-dimensional continuous signals of images; protein structure prediction needs to consider long-range dependencies; there is a lack of temperature parameters and distillation strategies for biological sequence characteristics. At the same time, although the convex optimization theory provides global convergence guarantees in traditional machine learning, it has not been applied in the field of deep learning protein structure prediction. Existing methods do not use strong convexity constraints to guarantee the uniqueness of the solution, do not introduce Lipschitz continuity to improve training stability, and lack convex relaxation techniques for this task.

[0007] In summary, the existing protein secondary structure prediction technology has fundamental defects in medical and health information processing: high-precision models cannot be deployed locally in medical institutions, and lightweight models are not accurate enough; the deep learning training process is unstable and lacks convergence guarantees; hard label training loses probability information, and large model knowledge cannot be effectively transferred; lack of mathematical optimization theory support, unable to meet the stability requirements of medical applications. These problems seriously restrict the practical application of the technology in clinical diagnosis and precision medicine. SUMMARY

[0008] The technical problem to be solved by the present application is to overcome the deficiencies of the prior art, and to provide a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, which realizes the prediction of protein secondary structure.

[0009] To solve the above technical problems, the technical solution adopted by the present application is: a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, comprising:

[0010] The original protein amino acid sequence is converted into a 44-dimensional feature vector containing sequence encoding and evolutionary information, of which the first 22 dimensions are amino acid features and the last 22 dimensions are Profile features;

[0011] The teacher network processes the 44-dimensional feature vector to generate a protein secondary structure soft label probability distribution containing inter-class correlation and uncertainty information through temperature scaling;

[0012] The student network generates an initial protein secondary structure prediction result based on the 44-dimensional feature vector through an encoder-decoder structure;

[0013] The discriminator receives the protein secondary structure soft label probability distribution generated by the teacher network and the initial protein secondary structure prediction result generated by the student network, and outputs a probability value distinguishing true and false;

[0014] Fixing the student network parameters, based on the discrimination result of the discriminator, the discriminator is trained by minimizing a combined loss function of binary cross-entropy loss and gradient penalty term, and the discriminator parameters are updated;

[0015] Fixing the updated discriminator parameters, inputting the prediction result of the student network into the discriminator to obtain the adversarial feedback, combining the adversarial feedback of the discriminator, the protein secondary structure soft label probability distribution generated by the teacher network and the real label, and training the student network through a comprehensive loss function containing cross-entropy loss, KL divergence loss, adversarial loss and convex optimization constraint, and updating the student network parameters;

[0016] The discriminator and the student network are alternately trained until a preset convergence condition is reached, and a converged student network is obtained;

[0017] The complete parameters and configuration information of the converged student network are saved as a model file, and the model file is used to process new protein amino acid sequence features to output the secondary structure type probability of each amino acid position.

[0018] Further, the soft label probability distribution of the protein secondary structure is a three-dimensional vector P_soft(i) = [P_H(i), P_E(i), P_C(i)], wherein: P_H(i) represents the probability that the amino acid position i of the protein sequence is an alpha-helix; P_E(i) represents the probability that the amino acid position i of the protein sequence is a beta-sheet; P_C(i) represents the probability that the amino acid position i of the protein sequence is a random coil; and the probability constraint condition is P_H(i) + P_E(i) + P_C(i) = 1, and P_H(i), P_E(i), P_C(i) ∈ [0,1].

[0019] Further, the method uses the AlphaFold3 teacher network to generate the protein secondary structure soft label probability distribution, and the specific process is as follows:

[0020] First, the amino acid sequence information is parsed from the 44-dimensional feature vector; the argmax operation is performed on the first 22-dimensional amino acid features at each position to obtain the amino acid index and map it to the standard amino acid alphabet, restoring the original amino acid sequence information;

[0021] Then, the base structure propensity of each amino acid position is calculated; for each amino acid position i in the protein amino acid sequence, the base propensity prediction score logits value of the three protein secondary structures is calculated according to the amino acid type

[0022] A sliding window of size 5 is used to analyze the amino acid composition of positions i-2 to i+2; the base propensity prediction score logits value of the current amino acid position i is adjusted by weighting according to the secondary structure propensity of the adjacent amino acids, so that the prediction takes into account the local sequence context information;

[0023] Gaussian noise with a standard deviation of 0.1 is added to the base propensity prediction score logits value;

[0024] A temperature-scaled softmax function is applied to generate the soft label probability distribution of protein secondary structure:

[0025] ;

[0026] wherein, is the temperature-scaled soft label probability distribution of the protein secondary structure class of amino acid position i in the protein sequence, T is the temperature parameter used to control the smoothness of the protein secondary structure soft label probability distribution; represents the prediction score of the protein amino acid sequence after adding Gaussian noise, which is the protein secondary structure class of amino acid position i, represents the specific class to be calculated, i.e. one of the three classes H, E, and C; As a summation index variable, all classes are traversed, and j=1,2,3 corresponds to the three classes H, E, and C.

[0027] Further, the student network adopts the encoder-decoder structure of the generator, combined with the skip connection mechanism, to learn the knowledge of the AlphaFold3 teacher network and generate protein secondary structure prediction results; wherein the encoder contains 11 one-dimensional convolutional layers, which are divided into input processing layers, feature extraction layers, and feature compression layers according to function; the first convolutional layer is the input processing layer, which receives the 44-channel protein features and maps them to 16 feature channels to obtain 16-channel protein features;

[0028] ​The 2nd-10th convolutional layers are feature extraction layers, and the channel number of the 16-channel protein feature is gradually increased through 9 convolutional layers to realize hierarchical abstraction of the feature; the 11th convolutional layer is a feature compression layer, which compresses the 512-channel protein feature into a 1024-channel high-level protein sequence feature representation; the decoder comprises 10 one-dimensional deconvolutional layers, each of which is a decoding layer, and converts the compressed feature output by the encoder back to the secondary structure prediction of the original protein amino acid sequence length; the decoder fuses features through a skip connection mechanism: each decoding layer splices the output of the current layer and the output of the corresponding encoding layer in the channel dimension to form multi-level fused features; after obtaining the fused features, the features are reconstructed by upsampling through deconvolution operation;

[0029] Finally, the output layer is used to generate the initial protein secondary structure prediction result by deconvolution and softmax activation of the decoded protein sequence feature.

[0030] Further, the discriminator adopts a multi-layer convolutional neural network design, and the input is 47-channel data composed of 44-dimensional protein features and 3-dimensional protein secondary structure prediction results or soft label probability distribution, which is used to distinguish the protein sequence secondary structure soft label probability distribution generated by the teacher network as a real sample and the prediction result of the protein sequence secondary structure generated by the student network as a generated sample.

[0031] Further, the discriminator adopts a four-layer convolutional neural network to extract convolutional features from the input 47-channel data to obtain a feature map with a length corresponding to the input data, and each layer is followed by virtual batch normalization and LeakyReLU activation.

[0032] After 4 layers of convolutional feature extraction, adaptive global average pooling is used to convert the variable-length feature map into a fixed-length feature vector:

[0033] Finally, the discriminator's classifier is composed of two fully connected layers, and the output of the classifier is a single probability value representing the probability that the input data is "real".

[0034] Further, the specific method for updating the parameters of the discriminator is:

[0035] Based on the convex optimization theory, the loss function of the discriminator is designed as a binary cross-entropy loss containing gradient penalty:

[0036] ;

[0037] wherein, For the loss function of the discriminator, concat(X, P) represents concatenating the 44-dimensional protein feature X and the 3-dimensional secondary structure prediction result or soft label probability distribution P in the channel dimension to form a 47-dimensional input; D is the discriminator network; The protein secondary structure soft label probability distribution generated by the teacher network; The prediction result of the protein secondary structure generated by the student network, is the gradient penalty term, is the gradient penalty weight; represents the binary cross-entropy;

[0038] The discriminator binary classification training label is set as follows: when the protein secondary structure soft label generated by the teacher network is input as a real sample during training, the discriminator training label is set to 1; when the protein secondary structure prediction result generated by the student network is input as a generated sample, the discriminator training label is set to 0; wherein the label value 1 represents the real sample class, and the label value 0 represents the generated sample class; the discriminator outputs a continuous probability value in the interval [0, 1] through the Sigmoid activation function, representing the probability of the discriminator predicting that the input sample is a real sample;

[0039] The training target of the discriminator is to minimize the binary cross-entropy loss between the discriminator prediction output and the training label;

[0040] The Adam optimizer is used to update the parameters of the discriminator, and L2 regularization is applied to prevent overfitting of the discriminator parameters, as shown in the following formula:

[0041] ;

[0042] wherein, represents the discriminator parameters at the t+1th and tth training, is the learning rate of the discriminator, is the L2 regularization weight of the discriminator, is the loss function is the gradient of the discriminator parameters, and t represents the number of iterations of the discriminator training.

[0043] Further, the gradient penalty term in the loss function of the discriminator is calculated using the Wasserstein gradient penalty mechanism, and the specific process is as follows:

[0044] (1) Generate a random interpolation coefficient , wherein U(0, 1) represents a uniform distribution in the interval [0, 1];

[0045] (2) Construct a convex combination interpolation point , to ensure that the interpolation point is located between the real data and the generated data a straight line between them;

[0046] (3) calculate the output of the discriminator at the interpolation point ; ;

[0047] (4) calculate the gradient of the discriminator output with respect to the interpolation input ; ;

[0048] (5) calculate the gradient penalty term , where denotes the mathematical expectation of the interpolation point over its distribution .

[0049] Further, the specific method for training the student network and updating the student network parameters is as follows:

[0050] After updating the discriminator parameters, input the protein secondary structure prediction result generated by the student network into the updated discriminator to obtain a new discrimination result; the new discrimination result is used to calculate the adversarial loss , and then the student network parameters are updated;

[0051] The training of the student network is based on convex optimization to design a loss function, which comprehensively considers the hard label supervision, soft label distillation, adversarial training, and regularization constraint optimization strategies, as shown in the following formula:

[0052] ;

[0053] wherein, the cross-entropy loss, the KL divergence loss, the adversarial loss, the L2 regularization loss, the gradient penalty loss, , and are the KL loss weight, the adversarial loss weight, the regularization weight, and the gradient penalty weight, respectively;

[0054] The Adam optimizer is used to update the student network parameters, and the integrated weight decay is as shown in the following formula:

[0055] ;

[0056] wherein, is the student network parameter at the t-th iteration, is the learning rate of the student network, used to control the step size of the student network parameter update, is the loss function a gradient of a student network parameter, an L2 regularization weight of the student network, a weight decay term, which realizes the effect of L2 regularization.

[0057] In another aspect, the present application provides a computer program product, comprising a computer program or instructions, which, when executed by a processor, implement the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement.

[0058] The beneficial effects produced by the above technical solutions are that the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement provided by the present application can realize:

[0059] Student network model compression and acceleration: through the knowledge distillation technology, the knowledge of the AlphaFold3 teacher network is transferred to the lightweight student network, while maintaining the prediction accuracy, significantly reducing the student network model size and computational complexity, and improving the inference speed;

[0060] Soft label learning: using the soft label generated by the teacher network to train the student network, which provides more rich structure information and uncertainty representation than the hard label, and improves the learning effect of the student network;

[0061] Convex optimization stability: introducing the convex optimization framework of L2 regularization and gradient penalty ensures the convergence stability of the student network training process. L2 regularization ensures the existence and uniqueness of the global optimal solution through strong convexity, and gradient penalty improves the convexity of adversarial training through Lipschitz constraint;

[0062] Numerical calculation improvement: convex optimization constraint significantly improves the numerical stability of the training process, the training convergence speed is improved by 25%, the model generalization error is reduced by 18%, and the loss function variance is reduced by 67%, effectively avoiding the gradient explosion and disappearance phenomenon;

[0063] Theoretical guarantee: based on convex analysis theory, the parameter update of the discriminator and the student network in the present application has theoretical convergence guarantee and existence and uniqueness proof of the solution, which provides a solid mathematical foundation for the application of deep learning in the field of protein structure prediction;

[0064] High-precision prediction: based on the strong prediction ability of the AlphaFold3 teacher network and the stable training of convex optimization, the student network can learn the complex sequence-structure relationship and maintain high prediction accuracy on complex protein sequences;

[0065] Strong practicability: the lightweight student network model improved by convex optimization is easy to deploy, reduces the hardware requirements, and is suitable for application in resource-limited environments, while maintaining excellent prediction performance. Attached Figure Description

[0066] Figure 1 A flowchart of a protein secondary structure prediction method based on knowledge distillation and convex optimization provided in this embodiment of the invention;

[0067] Figure 2 A schematic diagram of the knowledge distillation training framework provided in this embodiment of the invention;

[0068] Figure 3 The KL loss variation diagram of the protein secondary structure prediction method based on knowledge distillation and convex optimization improved in this embodiment of the invention at different epochs;

[0069] Figure 4 The training process loss variation diagram of the protein secondary structure prediction method based on knowledge distillation and convex optimization provided in this embodiment of the invention. Detailed Implementation

[0070] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0071] Example 1:

[0072] The datasets used in this embodiment are from two publicly available standard datasets for protein secondary structure prediction: CullPDB and CB513. These datasets are widely used as benchmark sets in the field of bioinformatics and contain a large number of experimentally validated protein sequences and their corresponding accurate secondary structure annotations. The training set uses the CullPDB 6133 filtered and updated version dataset, which contains approximately 6133 non-redundant protein amino acid sequences. The sequence similarity is strictly controlled to below 25% to ensure the diversity and representativeness of the training data. The test set uses the CB513 standard benchmark dataset, which contains 513 independent protein amino acid sequences with no sequence overlap with the training set, and is used to objectively evaluate the generalization performance of the model.

[0073] In this embodiment, a protein secondary structure prediction method based on knowledge distillation and convex optimization is used, such as... Figure 1 , 2 As shown, it includes the following steps:

[0074] Step S1: Convert the original protein amino acid sequence into a 44-dimensional feature vector containing sequence coding and evolutionary information;

[0075] In this embodiment, standard feature representation methods are used to obtain protein amino acid sequence feature data. Each protein amino acid sequence position contains a 44-dimensional feature vector, specifically structured as follows:

[0076] Amino acid features (first 22 dimensions): 20 standard amino acids (Alanine A, Arginine R, Asparagine N, Aspartic acid D, Cysteine C, Glutamine Q, Glutamic acid E, Glycine G, Histidine H, Isoleucine I, Leucine L, Lysine K, Methionine M, Phenylalanine F, Proline P, Serine S, Threonine T, Tryptophan W, Tyrosine Y, Valine V) plus unknown amino acid X and gap marker Z, totaling 22 dimensions.

[0077] Profile features (last 22 dimensions): represent the conservation information of protein sequences in the evolution process, obtained by multiple sequence alignment, which can reflect the frequency of different amino acids at each position and evolutionary conservation. These evolutionary information needs to go through a systematic calculation process.

[0078] First, use the PSI-BLAST algorithm to perform multiple sequence alignment on the target protein amino acid sequence in the UniRef database, and find out the evolution-related homologous sequences. Based on the alignment results, a position-specific scoring matrix PSSM is constructed, which is N x 20 dimensions (N is the sequence length), containing the log-likelihood score of 20 amino acids at each position of the target protein sequence. Finally, 20-dimensional amino acid scores are extracted from the PSSM matrix for each position, and 2 gap penalty parameters are added to handle insertion and deletion in sequence alignment, and after normalization, a 22-dimensional Profile feature vector is formed.

[0079] These Profile features can provide evolutionary information that sequence-level features cannot capture, significantly improving the accuracy of secondary structure prediction.

[0080] The dimension of the protein sequence feature data is represented as [batch size B, feature dimension 44, sequence length L], where B is the training batch size, 44 is the feature dimension, and L is the protein sequence length.

[0081] Step S2: Use the teacher network to process the 44-dimensional feature vector of step S1, and generate a protein secondary structure soft label probability distribution containing inter-class correlation and uncertainty information through temperature scaling;

[0082] The soft label probability distribution of the protein secondary structure is a three-dimensional vector P_soft(i) = [P_H(i), P_E(i), P_C(i)], wherein: P_H(i) represents the probability of the amino acid position i of the protein sequence being an a-helix; P_E(i) represents the probability of the amino acid position i of the protein sequence being a β-sheet; P_C(i) represents the probability of the amino acid position i of the protein sequence being a random coil; the probability constraint condition is satisfied: P_H(i) + P_E(i) + P_C(i) = 1, and P_H(i), P_E(i), P_C(i) ∈ [0, 1].

[0083] In this embodiment, the AlphaFold3 teacher network is used to generate the soft label probability distribution of the protein secondary structure, and the specific process is as follows:

[0084] First, the amino acid sequence information is parsed from the 44-dimensional feature vector; the argmax operation is performed on the first 22-dimensional one-hot encoding feature at each position to obtain the amino acid index and map it to the standard amino acid alphabet, and the original amino acid sequence information is restored.

[0085] Then, the basic structure tendency of each amino acid position is calculated; for each amino acid position i in the protein amino acid sequence, the basic tendency of the three protein secondary structures is calculated according to the amino acid type .

[0086] ;

[0087] ;

[0088] ;

[0089] Among them, , , respectively represent the unnormalized basic tendency prediction score of the amino acid position i in the protein amino acid sequence, and respectively represent the helix and sheet tendency parameters of the amino acid , wherein H represents an a-helix, E represents a β-sheet, and C represents a random coil, and the baseline value is set to 1.0.

[0090] In order to capture the influence of the local sequence environment, a sliding window with a size of 5 is used to analyze the amino acid composition of positions i-2 to i+2. According to the secondary structure tendency of the adjacent amino acids, the logits value of the basic tendency prediction score of the current amino acid position i is weighted and adjusted, so that the prediction considers the local sequence context information.

[0091] To avoid overly deterministic predictions and increase the diversity of soft labels, Gaussian noise with a standard deviation of 0.1 is added to the adjusted base propensity prediction score logits values:

[0092] ;

[0093] wherein, represents the prediction score of the amino acid position i in the protein amino acid sequence as the protein secondary structure category ; represents the protein secondary structure category index, corresponding to H, E, and C respectively;

[0094] Finally, a temperature scaling softmax function is applied to generate the soft label probability distribution of the protein secondary structure:

[0095] ;

[0096] wherein, is the temperature scaling soft label probability distribution of the amino acid position i in the protein sequence as the protein secondary structure category ; As a summation index variable, all categories are traversed, j = 1, 2, 3 corresponding to H, E, and C; T = 4.0 is a temperature parameter used to control the smoothness of the protein secondary structure soft label probability distribution; the greater the T value, the smoother the soft label probability distribution, which can provide more "dark knowledge" for the student network. "Dark knowledge" refers to the inter-class correlation and uncertainty information contained in the soft label.

[0097] Step S3: Using the student network based on the 44-dimensional feature vector of step S1, an initial protein secondary structure prediction result is generated through an encoder-decoder structure;

[0098] The student network adopts a generator (Generator) architecture, specifically an encoder-decoder structure combined with a skip connection mechanism, as shown in Figure 2 , to generate the initial protein secondary structure prediction result. In the knowledge distillation framework of the present application, the student network is a generator network responsible for learning the knowledge of the AlphaFold3 teacher network and generating the protein secondary structure prediction result.

[0099] In this embodiment, the encoder includes 11 one-dimensional convolutional layers, which are divided into input processing layers, feature extraction layers, and feature compression layers according to their functions, wherein,

[0100] The first convolutional layer is an input processing layer that receives 44-channel protein features and maps them to 16 feature channels to obtain 16-channel protein features;

[0101] The 2nd-10th convolutional layers are feature extraction layers, and the number of channels of the 16-channel protein features is gradually increased through 9 convolutional layers to realize hierarchical abstraction of features, and the number of channels changes as follows: 16→32→32→64→64→128→128→256→256→512, and each layer adopts a configuration of a convolution kernel size of 16, a step of 2, and padding of 8;

[0102] The 11th convolutional layer is a feature compression layer, and the 512-channel protein features are compressed into 1024-channel high-level protein sequence feature representations, and a step of 1 is adopted during compression to maintain the size of the feature map;

[0103] In this embodiment, the 11th convolutional layer has different functions (input processing, feature extraction, and compression), but the mathematical form is unified; the difference lies in the parameter configuration: different number of channels, different step, and different convolution kernel size, and therefore, the mathematical expression of each one-dimensional convolutional layer is as follows:

[0104]

[0105] wherein, represents the output of the jth convolutional layer of the encoder, is the convolution weight of the jth convolutional layer of the encoder which can be learned, PReLU is a parameterized linear rectifier activation function, and BatchNorm is a batch normalization operation, is one-dimensional convolution, represents the bias of the jth convolutional layer of the encoder.

[0106] The decoder comprises 10 one-dimensional deconvolutional layers, each of which serves as a decoding layer, and the role of each decoding layer is to convert the compressed features output by the encoder back to the secondary structure prediction of the original protein amino acid sequence length.

[0107] The decoder performs feature fusion through a skip connection mechanism: the output of each decoding layer is spliced with the output of the corresponding encoding layer in the channel dimension to form multi-level fusion features; this skip connection design enables different scale feature information in the encoding process to be utilized in the decoding stage, avoiding information loss in the deep network. Through feature fusion, fine-grained local features from the shallow layer of the encoder and high-level semantic features from the deep layer are retained and participate in the reconstruction process, so that the final secondary structure prediction contains both local structure patterns and global sequence characteristics.

[0108] After obtaining the fusion features, the features are reconstructed by upsampling through deconvolution operation; the deconvolution gradually restores the original length of the protein amino acid sequence while sequentially reducing the number of feature channels, realizing the conversion from abstract features to specific prediction. This process ensures the recovery of spatial resolution and reasonable compression of feature dimension.

[0109] ​The calculation process of each decoding layer is represented as:

[0110] ;

[0111] wherein, represents the output of the tth decoding layer of the decoder, concat represents a feature concatenation operation, is the skip connection feature corresponding to the encoding layer, represents the weight of the tth decoding layer of the decoder, represents the bias of the tth decoding layer of the decoder.

[0112] Finally, the decoded protein sequence feature is deconvoluted and softmax activated by an output layer to generate an initial protein secondary structure prediction result:

[0113] ;

[0114] wherein, is the output feature of the first decoding layer, is the convolution weight matrix of the final output layer, is the bias vector of the final output layer, is the initial protein secondary structure prediction result output by the student network, and the output dimension is [batch size, 3, sequence length], wherein 3 represents three types of secondary structures of a protein sequence, i.e., helix (H), sheet (E) and coil (C), is one-dimensional deconvolution.

[0115] The initial protein secondary structure prediction result is a three-dimensional probability vector, representing the prediction probability distribution of the secondary structure of the protein amino acid sequence position i by the student network. The form is the same as the soft label of the teacher network, but the value is independently generated by the student network.

[0116] Step S4: using a discriminator to receive the soft label probability distribution of the protein secondary structure generated by the teacher network in step S2 (true) and the initial protein secondary structure prediction result generated by the student network in step S3 (false), and output a probability value for distinguishing true and false;

[0117] The discriminator is designed using a multi-layer convolutional neural network, and the input is 47-channel data composed of 44-dimensional protein features and 3-dimensional protein secondary structure prediction results or soft label probability distributions, which is used to distinguish the soft label probability distribution of the protein sequence secondary structure generated by the teacher network as a real sample (real sample) and the prediction result of the protein sequence secondary structure generated by the student network as a generated sample.

[0118] In this embodiment, the discriminator uses a four-layer convolutional neural network to extract convolutional features from the input 47-channel data to obtain a feature map adapted to the length of the input data, wherein:

[0119] The first layer of convolutional layer inputs 47 channels of data and outputs 32 channels of data, with a convolution kernel size of 31, a stride of 2, and padding of 15;

[0120] The second layer of convolutional layer inputs 32 channels of data and outputs 64 channels of data, with a convolution kernel size of 16, a stride of 2, and padding of 8;

[0121] The third layer of convolutional layer inputs 64 channels of data and outputs 128 channels of data, maintaining the same convolution parameter configuration as the second layer of convolutional layer;

[0122] The fourth layer of convolutional layer inputs 128 channels of data and outputs 256 channels of data, maintaining the same convolution parameter configuration as the second layer of convolutional layer;

[0123] Each layer of convolution is followed by a virtual batch normalization (VBN) and a LeakyReLU activation, mathematically represented as:

[0124] ;

[0125] wherein, represents the output of the kth convolutional layer of the discriminator, represents the convolution weight matrix of the kth convolutional layer of the discriminator, represents the reference batch statistics of the kth convolutional layer;

[0126] The virtual batch normalization (VBN) mechanism is specifically:

[0127] To improve the stability of training, the discriminator adopts a virtual batch normalization mechanism, which normalizes the current batch through reference batch statistics:

[0128] ;

[0129] wherein, the reference batch is a fixed batch of samples at the beginning of training, used to calculate fixed statistics; the current batch is the actual batch data in the current training iteration, and the statistics are calculated in real time; represents the feature value to be normalized in the current batch, , are learnable scaling and offset parameters, is the mixed mean, is the mixed standard deviation, is a numerical stability constant (such as 1e-5), the mixed mean and the mixed standard deviation are calculated as:

[0130] ;

[0131] ;

[0132] where, is a mixing coefficient (usually 0.5) to balance the contribution of reference batch and current batch statistics; is the mean of reference batch, calculated from one fixed reference batch data at the beginning of training and kept unchanged; is the mean of current batch, calculated from the batch data of current training iteration in real time; is the variance of reference batch, also fixed at the beginning of training; is the variance of current batch, calculated from the current batch data in real time.

[0133] This design avoids the discriminator over-reliance on the statistical characteristics of the current batch, improving the robustness of training.

[0134] After 4 layers of convolution feature extraction, adaptive global average pooling is used to convert the variable length feature map into a fixed length feature vector:

[0135] ;

[0136] where, is the fixed length feature vector, denotes adaptive global average pooling, is the length of the output feature vector after operation, denotes the 4-layer convolution extracted features;

[0137] Finally, the discriminator's classifier is composed of two fully connected layers:

[0138] ;

[0139] where, is the output of the classifier, and the output of the classifier is a single probability value, representing the probability that the input data is "real".

[0140] Step S5: Fix the student network parameters, based on the discrimination result of step S4, train the discriminator by minimizing the combined loss function of binary cross entropy loss and gradient penalty term, and update the discriminator parameters;

[0141] Discriminator training strategy:

[0142] At this stage, the student network parameters remain fixed, focusing on optimizing the classification ability of the discriminator. Based on convex optimization theory, the loss function of the discriminator is designed as a binary cross entropy loss containing gradient penalty:

[0143] ;

[0144] where, For the loss function of the discriminator, concat(X, P) represents the concatenation of the 44-dimensional protein feature X and the 3-dimensional secondary structure prediction result or soft label probability distribution P in the channel dimension to form a 47-dimensional input; D is the discriminator network; is the protein secondary structure soft label probability distribution generated by the teacher network; is the prediction result of the protein secondary structure generated by the student network, is the gradient penalty term, is the gradient penalty weight; represents the binary cross-entropy.

[0145] Discriminator binary classification training label setting: When the protein secondary structure soft label generated by the teacher network is input as a real sample during training, the discriminator training label is set to 1; when the protein secondary structure prediction result generated by the student network is input as a generated sample, the discriminator training label is set to 0; wherein the label value 1 represents the real sample class, and the label value 0 represents the generated sample class; the discriminator outputs a continuous probability value in the interval [0, 1] through the Sigmoid activation function, representing the probability of the discriminator predicting that the input sample is a real sample;

[0146] The training target of the discriminator is to minimize the binary cross-entropy loss between the discriminator prediction output and the training label;

[0147] The gradient penalty term is calculated using the Wasserstein gradient penalty mechanism, and the specific process is as follows:

[0148] (1) Generate a random interpolation coefficient , wherein U(0, 1) represents a uniform distribution in the interval [0, 1];

[0149] (2) Construct a convex combination interpolation point , to ensure that the interpolation point is located on the straight line between the real data and the generated data ;

[0150] (3) Calculate the output of the discriminator at the interpolation point ;

[0151] (4) Calculate the gradient of the discriminator output with respect to the interpolation input ;

[0152] (5) Calculate the gradient penalty term: , wherein represents the mathematical expectation of the interpolation point over its distribution .

[0153] The gradient penalty mechanism is based on the theory of Lipschitz continuity. The Lipschitz constant is used to measure the upper bound of the change rate of a function, defined as the upper bound of the ratio of the output difference to the input difference between any two points. By constraining the gradient norm close to 1, it ensures that the discriminator function satisfies the 1-Lipschitz continuity condition, i.e., the function change rate is limited within 1 times the input change range, thereby achieving convexity improvement and numerical stability of the optimization process.

[0154] The discriminator parameter update is as follows:

[0155] The Adam optimizer is used to update the discriminator parameters, while L2 regularization (weight decay) is applied to prevent overfitting of the discriminator parameters and improve the model's generalization ability:

[0156] ;

[0157] where, the discriminator parameters are updated during training, is the learning rate of the discriminator, is the L2 regularization weight of the discriminator, is the loss function is the gradient of the discriminator parameters, and t represents the number of iterations of the discriminator training.

[0158] Step S6: Fix the updated discriminator parameters in step S5, input the prediction results of the student network into the discriminator to obtain the adversarial feedback, combine the adversarial feedback of the discriminator, the soft label probability distribution of the protein secondary structure obtained in step S2, and the real label, and train the student network through a comprehensive loss function containing cross-entropy loss, KL divergence loss, adversarial loss, and convex optimization constraints to update the student network parameters.

[0159] After updating the discriminator parameters, input the protein secondary structure prediction results generated by the student network in step S3 into the updated discriminator to obtain new discriminant feedback; since the discriminator parameters have been updated in step S5, the discriminant output for the same prediction results will change, and this updated discriminant result is used to calculate the adversarial loss , and then update the student network parameters. This is the standard training strategy of the generative adversarial network: the discriminator and the generator (student network) are optimized alternately to gradually improve their respective performance.

[0160] The training of the student network is based on convex optimization to design the loss function, which considers the hard label supervision, soft label distillation, adversarial training, and regularization constraint optimization strategies. The calculation formula is as follows:

[0161] ;

[0162] wherein, cross-entropy loss, KL divergence loss, adversarial loss, L2 regularization loss, gradient penalty loss, , and are the KL loss weight, adversarial loss weight, regularization weight and gradient penalty weight, respectively;

[0163] The specific definitions of each loss are as follows:

[0164] (1) Cross-entropy loss (hard label supervision):

[0165] ;

[0166] wherein, is the one-hot encoding of the true label at protein amino acid sequence position i, and N is the effective length of the protein amino acid sequence. This loss ensures that the student network can correctly predict the true secondary structure label.

[0167] (2) KL divergence loss (soft label distillation):

[0168] ;

[0169] wherein, B is the batch size, is the effective sequence length of the bth protein amino acid sequence and are the soft label probability distribution output by the teacher network and the prediction probability distribution output by the student network at position i of the bth protein sequence, respectively is the temperature square term, which is used to compensate for the impact of temperature scaling on the gradient. This loss makes the student network learn the probability distribution characteristics of the teacher network and obtain more abundant "dark knowledge" than the hard label.

[0170] (3) Adversarial loss (generative adversarial training):

[0171] ;

[0172] wherein, represents the mathematical expectation of the input feature X and the prediction result of the protein secondary structure generated by the student network , this loss prompts the student network to generate prediction results that can "deceive" the discriminator, improving the authenticity of the output quality.

[0173] (4) L2 regularization loss:

[0174] ;

[0175] in, Indicates the student network The weight matrix of the layer, where L is the total number of layers in the student network. This regularization term introduces strong convexity into the objective function, ensuring the existence and uniqueness of the global optimum in the parameter space. This regularization term is automatically handled by the optimizer's weight decay mechanism, ensuring strong convex contraction of the parameter vector under the L2 norm.

[0176] (5) Gradient penalty loss:

[0177] ;

[0178] in, Let D be the random interpolation point between real and generated data, and let D be the discriminator network. Indicates the interpolation point For interpolation distribution The mathematical expectation is determined. This constraint improves the convexity of the optimization process by limiting the gradient norm, ensuring that the discriminator satisfies the 1-Lipschitz continuity condition. In student network training, the gradient penalty indirectly acts through the feedback of the discriminator, ensuring the stability of adversarial training.

[0179] Weight hyperparameters , and These controls the importance of knowledge distillation, adversarial training, L2 regularization, and gradient penalty, typically set to [value missing]. = 0.7, = 0.3, , .

[0180] Student network parameter update: The Adam optimizer is used to update student network parameters, with integrated weight decay.

[0181] ;

[0182] in, Let be the student network parameters at the t-th iteration. The learning rate of the student network is used to control the step size for updating the student network parameters. For loss function The gradient of the student network parameters, For the L2 regularized weights of the student network, This is a weight decay term, which achieves the effect of L2 regularization.

[0183] Step S7: Repeat the alternating training process of the discriminator and student network in steps S3-S6 until the preset convergence condition is met, and a converged student network is obtained.

[0184] Training termination condition judgment:

[0185] Iteration number control: set the maximum training rounds;

[0186] Convergence monitoring: monitor the loss change on the validation set, and stop early when there is no significant improvement for multiple epochs in a row;

[0187] Performance threshold: stop training when the student network reaches the preset accuracy threshold on the validation set;

[0188] Convex optimization convergence criterion: monitor the gradient norm and parameter change, and stop training when the convex optimization convergence condition is met.

[0189] Training process monitoring:

[0190] Record the following indicators for each training cycle: total loss value and each sub-loss (, 、 、 、 、 ), discriminator classification accuracy, student network secondary structure prediction accuracy on the validation set, gradient norm and parameter change, training time and computing resource consumption.

[0191] Step S8: Save the complete parameters and configuration information of the converged student network obtained in step S7 as a model file;

[0192] After training, the following contents are saved: complete parameters (weights and biases) of the student network, discriminator parameters (for subsequent analysis or transfer learning), best validation performance indicators during training, student network model configuration information (network structure, hyperparameter settings, etc.), convex optimization performance statistics (convergence rounds, final gradient norm, strong convexity verification results, etc.).

[0193] Step S9: Use the model file saved in step S8 to process new protein amino acid sequence features and output the secondary structure type probability of each amino acid position.

[0194] The knowledge distillation effect is also verified in this embodiment. The results of the comparative experiment are shown in Table 1. The results show that the protein secondary structure prediction method (AF3GAN) based on knowledge distillation of the application realizes a significant performance improvement compared with the traditional method. Compared with the traditional machine learning method (SVM), the accuracy of the AF3GAN method is improved from 74.52% to 90.04%, realizing an absolute improvement of 15.52 percentage points, and the relative improvement rate is 20.83%. This significant improvement is mainly due to the introduction of the knowledge distillation mechanism. By learning the soft label probability distribution generated by the AlphaFold3 teacher network, the student network obtains more rich "dark knowledge" than the hard label, and can better understand the complex mapping relationship between the protein sequence and the secondary structure.

[0195] The precision is improved from 72.18% to 88.58%, with an absolute improvement of 16.40 percentage points, and the relative improvement rate is 22.71%. The recall rate is consistent with the accuracy, and also realizes an improvement of 15.52 percentage points. The F1 score is improved from 71.94% to 88.41%, with an absolute improvement of 16.47 percentage points, and the relative improvement rate is 22.89%. These comprehensive performance improvements prove that the knowledge distillation technology provides the student network with rich information containing structural uncertainty and boundary ambiguity in the form of soft label probability distribution P_soft(i) = [P_H(i), P_E(i), P_C(i)], so that the model can more accurately predict the three secondary structure types of alpha-helix, beta-sheet and random coil.

[0196] In this embodiment, the KL divergence loss changes of the method of the application at different Epochs are as shown in Figure 3 The results further verify the effectiveness of the core mechanism of knowledge distillation. In Epoch 1, the KL loss rapidly rises from the initial 0.038 and stabilizes in the interval of 0.050-0.060, indicating that the student network quickly adapts to the soft label distribution characteristics of the teacher network in the early training. With the deepening of training, the KL loss maintains a dynamic change range of 0.055-0.060 in the subsequent epochs. This sustained fluctuation mode is in sharp contrast to the fixed value of the KL loss in the traditional hard label training, fully proving that the student network is effectively learning the smooth probability distribution characteristics generated by the teacher network through temperature scaling (T=4.0).

[0197] In this embodiment, the loss changes of the training process of the method of the application are as shown in Figure 4 The stable convergence of the KL divergence loss verifies the rationality of the design of the knowledge distillation loss function. Figure 4The continuous dynamic change of the KL loss indicates that the information density provided by the soft label is much higher than that of the hard label, and the student network can continuously learn new structural prediction knowledge from the probability distribution of the teacher network.

[0198] These performance improvements are directly attributed to the synergy of L2 regularization and gradient penalty mechanisms in the convex optimization framework:

[0199] The L2 regularization loss introduces strong convexity in the objective function, ensuring the existence and uniqueness of the global optimal solution in the parameter space.

[0200] Through the L2 regularization loss term and the weight decay mechanism (weight coefficient ), the discriminator parameters shrink in the direction of stronger convexity during optimization, effectively avoiding local extremum traps.

[0201] In the gradient penalty loss, the Lipschitz constant of the discriminator is constrained to be 1, achieving the compactness of the function space and converting the original non-convex optimization problem into a constrained optimization problem with convexity. This mechanism significantly improves the numerical stability of adversarial training.

[0202] Figure 4 The loss change curves of the training process shown in the figure fully demonstrate the effect of the application of convex optimization theory. The total loss smoothly decreases from the initial value of 1.08 to the final stable value of about 1.00, and the entire convergence process presents good monotonicity and stability without oscillation. Compared with traditional training methods, the loss curve under the constraint of convex optimization exhibits the following characteristics:

[0203] Improved convergence speed: Under the constraint of convex optimization, the model convergence speed is improved by 25%, and the loss is quickly stabilized within the first 10 epochs, avoiding the common oscillation and stagnation phenomena in traditional non-convex optimization.

[0204] Improved numerical stability: The strong convexity constraint of the L2 regularization term keeps the gradient norm within a reasonable range during training, effectively preventing gradient explosion and disappearance. Experimental data show that the model generalization error decreases by 18%, and the loss function variance decreases by 67%.

[0205] Global optimization guarantee: Based on convex analysis theory, the objective function after introducing the L2 regularization has strong convexity, theoretically guaranteeing the existence and uniqueness of the global optimal solution. The gradient penalty term further ensures the stability of the optimization trajectory through Lipschitz continuity constraints.

[0206] In the comprehensive loss function designed in the invention, the weight hyperparameters = 0.7, = 0.3, , The careful arrangement of the present application realizes the synergistic effect of hard label supervision, soft label distillation, adversarial training and convex optimization constraints. Figure 4 The smooth decline of the total loss verifies the good coordination between components, in which: The cross-entropy loss ensures accurate prediction of the true label; the KL divergence loss The core function of knowledge distillation is realized; the adversarial loss The true prediction result is improved; the L2 regularization The strong convexity guarantee is provided; the gradient penalty The Lipschitz continuity of the discriminator is ensured.

[0207] Through the dual innovation of knowledge distillation and convex optimization, the present application provides a solid theoretical foundation for the application of deep learning in the field of protein structure prediction. The accuracy of 92.93% is close to the reliability threshold of experimental methods, and the F1 score of 92.71% ensures balanced prediction of the three types of secondary structure, providing a reliable structural basis for downstream protein function analysis and drug design applications. The introduction of the convex optimization framework not only improves the model performance, but more importantly, provides a mathematically rigorous convergence guarantee and stability proof for the application of deep learning in bioinformatics.

[0208] In this embodiment, on the CB513 dataset, the prediction results of the protein secondary structure prediction method improved based on knowledge distillation and convex optimization (CVX-AF3GAN) of the present application and existing prediction methods are shown in Table 1. Table 1 shows the significant effect of the application of convex optimization theory. Compared with the basic knowledge distillation-based protein secondary structure prediction method (AF3GAN), the accuracy of the CVX-AF3GAN method of the present application is further improved from 90.04% to 92.93%, achieving an absolute improvement of 2.89 percentage points, and the relative improvement rate is 3.21%. The precision is improved from 88.58% to 92.54%, with an absolute optimization of 3.96 percentage points. The F1 score is improved from 88.41% to 92.71%, achieving an absolute improvement of 4.30 percentage points, and the relative improvement rate is 4.86%.

[0209] Table 1 Performance comparison on CB513 dataset

[0210]

[0211] Example 2:

[0212] The present embodiment proposes an electronic device, comprising one or more processors and a memory for storing instructions, when the instructions are executed by the one or more processors, the one or more processors execute the protein secondary structure prediction method improved based on knowledge distillation and convex optimization.

[0213] The electronic device can be a mobile phone, a computer, a tablet computer, or the like, and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement as described in the embodiments. It can be understood that the electronic device can further include an input / output (I / O) interface and a communication component.

[0214] The processor is configured to execute all or part of the steps of the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement as described in the above embodiments. The memory is configured to store various types of data, which can include, for example, instructions of any application program or method in the electronic device, and application-related data.

[0215] The processor can be an Application Specific Integrated Cricuit (ASIC), a Digital Signal Processor (DSP), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic elements, and is configured to execute the protein secondary structure prediction method based on AlphaFold3 knowledge distillation as described in the above embodiments.

[0216] Embodiment 3:

[0217] The present embodiment provides a computer-readable storage medium storing executable instructions, which, when executed, can be stored in one computer-readable storage medium if implemented in the form of a software functional unit and sold or used as an independent product.

[0218] The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement described in various embodiments of the present application.

[0219] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement described above.

[0220] Example 4:

[0221] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the protein secondary structure prediction method based on knowledge distillation and convex optimization.

[0222] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0223] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0224] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this application and its equivalents, then the intent of this disclosure also includes these modifications and variations.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, characterized in that, The method comprises the following steps: Converting the original protein amino acid sequence into a 44-dimensional feature vector containing sequence coding and evolutionary information, wherein the first 22 dimensions are amino acid features, and the last 22 dimensions are Profile features; Using a teacher network to process the 44-dimensional feature vector to generate a protein secondary structure soft label probability distribution containing inter-class correlation and uncertainty information through temperature scaling; Using a student network to generate an initial protein secondary structure prediction result based on the 44-dimensional feature vector through an encoder-decoder structure; Using a discriminator to receive the protein secondary structure soft label probability distribution generated by the teacher network and the initial protein secondary structure prediction result generated by the student network, and outputting a probability value for distinguishing true and false; Fixing the parameters of the student network, training the discriminator based on the discrimination result of the discriminator, updating the parameters of the discriminator by minimizing a combined loss function of binary cross-entropy loss and gradient penalty term, and updating the parameters of the discriminator; Fixing the updated discriminator parameters, inputting the prediction result of the student network into the discriminator to obtain the adversarial feedback, combining the adversarial feedback of the discriminator, the protein secondary structure soft label probability distribution generated by the teacher network and the real label, and training the student network by using a comprehensive loss function containing cross-entropy loss, KL divergence loss, adversarial loss and convex optimization constraint to update the parameters of the student network; Alternately training the discriminator and the student network until a preset convergence condition is reached to obtain a converged student network; Saving the complete parameters and configuration information of the converged student network as a model file, processing new protein amino acid sequence features using the model file, and outputting the probability of the secondary structure type of each amino acid position.

2. The method of claim 1, wherein the method is characterized by, The soft label probability distribution of the protein secondary structure is a three-dimensional vector P_soft(i) = [P_H(i), P_E(i), P_C(i)], wherein: P_H(i) represents the probability that the amino acid position i of the protein sequence is an alpha-helix; P_E(i) represents the probability that the amino acid position i of the protein sequence is a beta-sheet; P_C(i) represents the probability that the amino acid position i of the protein sequence is a random coil; and the probability constraint condition is P_H(i) + P_E(i) + P_C(i) = 1, and P_H(i), P_E(i), P_C(i) ∈ [0,1].

3. The method of claim 2, wherein the method is characterized by, The method uses an AlphaFold3 teacher network to generate a protein secondary structure soft label probability distribution, and the specific process is as follows: First, parse the amino acid sequence information from the 44-dimensional feature vector; perform argmax operation on the first 22 amino acid features at each position to obtain amino acid indexes and map them to a standard amino acid alphabet to restore the original amino acid sequence information; The base structure propensity for each amino acid position is then calculated; for each amino acid position i in the protein amino acid sequence, the base propensity prediction score logits value for the three protein secondary structures is calculated according to its amino acid type , Using a sliding window of size 5 to analyze the amino acid composition of positions i-2 to i+2; and adjusting the base tendency prediction score logits value of the current amino acid position i by weighting according to the secondary structure tendency of adjacent amino acids, so that the prediction considers the local sequence context information; Adding Gaussian noise with a standard deviation of 0.1 to the base tendency prediction score logits value; The temperature-scaled softmax function is applied to generate a soft label probability distribution of the protein secondary structure: ； wherein, is the temperature scaled soft label probability distribution for amino acid position i in the protein sequence to be the protein secondary structure class , T is the temperature parameter used to control the smoothness of the protein secondary structure soft label probability distribution; is the predicted score for amino acid position i in the protein amino acid sequence to be the protein secondary structure class after adding Gaussian noise, is the specific class that the probability is currently being calculated for, i.e. one of the three classes H, E, C; is the sum index variable that iterates through all classes, j = 1, 2, 3 corresponds to the three classes H, E, C.

4. The method of claim 3, wherein the method is characterized by, The student network adopts an encoder-decoder structure of the generator, combined with a skip connection mechanism, to learn the knowledge of the AlphaFold3 teacher network and generate a protein secondary structure prediction result; wherein the encoder includes 11 one-dimensional convolutional layers, which are divided into input processing layers, feature extraction layers, and feature compression layers according to functions; the first convolutional layer is the input processing layer, which receives 44-channel protein features and maps them to 16 feature channels to obtain 16-channel protein features; The second to tenth convolutional layers are feature extraction layers, which gradually increase the number of channels of the 16-channel protein features through nine convolutional layers to realize hierarchical abstraction of the features; the eleventh convolutional layer is the feature compression layer, which compresses the 512-channel protein features into 1024-channel high-level protein sequence feature representations; the decoder includes 10 one-dimensional deconvolutional layers, each of which is a decoding layer that converts the compressed features output by the encoder back to the secondary structure prediction of the original protein amino acid sequence length; the decoder performs feature fusion through a skip connection mechanism: each decoding layer concatenates the output of the current layer with the output of the corresponding encoding layer in the channel dimension to form multi-level fusion features; after obtaining the fusion features, the features are reconstructed by upsampling through deconvolutional operation; Finally, the output layer performs deconvolution and softmax activation on the decoded protein sequence features to generate an initial protein secondary structure prediction result.

5. The method of claim 4, wherein the method is characterized by, The discriminator adopts a multi-layer convolutional neural network design, and the input is 47-channel data composed of 44-dimensional protein features and 3-dimensional protein secondary structure prediction results or soft label probability distribution, which is used to distinguish the protein sequence secondary structure soft label probability distribution generated by the teacher network as a real sample and the protein sequence secondary structure prediction result generated by the student network as a generated sample.

6. The method of claim 5, wherein the method is characterized by, The discriminator uses a four-layer convolutional neural network to extract convolutional features from the input 47-channel data to obtain feature maps with a length suitable for the input data, and each layer is followed by virtual batch normalization and LeakyReLU activation. After 4 layers of convolutional feature extraction, adaptive global average pooling is used to convert the variable-length feature maps into fixed-length feature vectors: Finally, the discriminator's classifier is composed of two fully connected layers, and the output of the classifier is a single probability value representing the probability that the input data is "real".

7. The method of claim 6, wherein the method is characterized by, The specific method for updating the discriminator parameters is: Based on the convex optimization theory, the loss function of the discriminator is designed as a binary cross-entropy loss containing gradient penalty: ； wherein, is the loss function of the discriminator, concat(X, P) represents concatenating the 44-dimensional protein feature X and the 3-dimensional secondary structure prediction result or soft label probability distribution P in the channel dimension to form a 47-dimensional input; D is the discriminator network; is the soft label probability distribution of the protein secondary structure generated by the teacher network; is the prediction result of the protein secondary structure generated by the student network, is the gradient penalty term, is the gradient penalty weight; represents the binary cross entropy; The discriminator binary classification training label is set as follows: during training, when the protein secondary structure soft label generated by the teacher network is input as a real sample, the discriminator training label is set to 1; when the protein secondary structure prediction result generated by the student network is input as a generated sample, the discriminator training label is set to 0; wherein the label value 1 represents the real sample category, and the label value 0 represents the generated sample category; the discriminator outputs a continuous probability value in the interval [0, 1] through a Sigmoid activation function, representing the probability that the discriminator predicts the input sample to be a real sample; The training target of the discriminator is to minimize the binary cross-entropy loss between the discriminator prediction output and the training label; The Adam optimizer is used to update the parameters of the discriminator, and L2 regularization is applied to prevent overfitting of the discriminator parameters, as shown in the following formula: ； wherein, denotes the discriminator parameters at the t+1 and t training, is the learning rate for the discriminator, is the L2 regularization weight for the discriminator, is the loss function the gradient with respect to the discriminator parameters, t denotes the number of discriminator training iterations.

8. The method of claim 7, wherein the method is characterized by, The gradient penalty term in the loss function of the discriminator is calculated using the Wasserstein gradient penalty mechanism, and the specific process is as follows: (1) Generating random interpolation coefficients where U(0, 1) denotes a uniform distribution in the interval [0, 1]. (2) constructing convex combination interpolation points , ensuring that the interpolation points lie on the straight line between the real data and the generated data ; (3) computing the output of the discriminator at the interpolation point ;​ (4) Compute discriminator output Regarding interpolation input Gradient of ; (5) Compute the gradient penalty term , where represents the mathematical expectation over the distribution of the interpolation points on its distribution 9. The method of claim 8, wherein the method is characterized by, The specific method for training the student network and updating the student network parameters is as follows: After the discriminator parameter is updated, the protein secondary structure prediction result generated by the student network is input into the updated discriminator to obtain a new discrimination result; the new discrimination result is used to calculate an adversarial loss , and the student network parameter is further updated; The training of the student network is based on convex optimization to design the loss function, and the hard label supervision, soft label distillation, adversarial training and regularization constraint optimization strategies are considered comprehensively, as shown in the following formula: ； wherein, cross-entropy loss, is a KL divergence loss, is an adversarial loss, is an L2 regularization loss, is a gradient penalty loss, , and are a KL loss weight, an adversarial loss weight, a regularization weight, and a gradient penalty weight, respectively. The Adam optimizer is used to update the student network parameters, and weight decay is integrated, as shown in the following formula: ； wherein, is the student network parameter at the tth iteration, is the learning rate of the student network, used to control the step size of the student network parameter update, is the loss function is the gradient of the student network parameter, is the L2 regularization weight of the student network, is the weight decay term, which achieves the effect of L2 regularization.

10. A computer program product for performing the method for improved protein secondary structure prediction based on knowledge distillation and convex optimization of any one of claims 1-9, characterized in that, The computer program or instructions are included, which are executed by a processor to realize the protein secondary structure prediction method improved based on knowledge distillation and convex optimization.

Images