Protein secondary structure prediction method based on knowledge distillation and convex optimization improvement

Abstract

The application provides a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, and relates to the technical field of bioinformatics and protein structure. The method first converts an original protein amino acid sequence into a 44-dimensional feature vector, generates a protein secondary structure soft label probability distribution by using a teacher network, and generates an initial protein secondary structure prediction result by using a student network; then a discriminator receives the soft label probability distribution of the protein secondary structure and the initial protein secondary structure prediction result, and outputs a probability value for distinguishing the true and false; the discriminator and the student network are alternately trained until a preset convergence condition is reached, and a converged student network is obtained; finally, complete parameters and configuration information of the converged student network are saved as a model file, a new protein amino acid sequence feature is processed by using the model file, and a secondary structure type probability of each amino acid position is output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bioinformatics and protein structure prediction, and particularly relates to a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement. BACKGROUND

[0002] In the field of medical informatics, protein secondary structure prediction is a key technology for converting genomic data into clinically useful information. With the popularization of gene sequencing in clinical diagnosis, medical institutions generate massive amounts of protein sequence data every day, which need to be predicted for secondary structure (alpha-helix, beta-sheet, and random coil) to assist in disease diagnosis and drug treatment decisions. This technology has important applications in precision medicine: by analyzing patient genetic mutations to predict protein structure changes, it assists in genetic disease diagnosis; it evaluates structural changes in drug target binding sites to guide personalized medication; and it screens candidate drug targets at high throughput to accelerate the new drug development process. These application scenarios require the prediction system to ensure high accuracy and stable and reliable operation in hospital information systems.

[0003] Current protein secondary structure prediction faces two core technical challenges in medical information processing. First, the contradiction between model deployment and accuracy: medical institutions' information systems usually run in resource-constrained intranet environments, making it difficult to deploy large deep learning models, while small deployable models cannot meet the clinical requirements in terms of prediction accuracy. Second, the training stability problem: medical data processing requires reproducible and verifiable results, but the training process of existing deep learning models has inherent instability, resulting in different results from the same data, affecting the reliability of clinical decision-making.

[0004] To address the model deployment challenge, there are three main types of solutions in existing technologies. DeepMind's AlphaFold3 and Meta's ESMFold use a Transformer architecture containing hundreds of millions of parameters, which can achieve a prediction accuracy of over 90%, but the model file is over 10GB, requiring high-end GPUs to run, and a single prediction takes several hours, which is too costly for small and medium-sized medical institutions in terms of hardware and time. NetSurfP-2.0, SPIDER3, and other lightweight models can run on ordinary workstations, but their prediction accuracy is only 75%-85%, which is unacceptable in clinical applications, and these methods start from scratch without utilizing the knowledge learned from large models. Early traditional methods such as PSIPRED and GOR V use hard label training, which can only represent discrete categories, losing the probability distribution information of the prediction, such as the uncertainty information that a certain position has a 70% probability of being an alpha-helix and a 30% probability of being a coil, resulting in a prediction accuracy usually below 75%.

[0005] To solve the problem of training stability, the existing deep learning method adopts the standard gradient descent algorithm, but the loss function of the deep neural network is non-convex and contains a large number of local optimal points. This leads to the training process being easily trapped in local optimization, and the model prediction accuracy may differ by more than 5% just by changing the random seed. Although adaptive optimizers such as Adam, RMSprop and regularization techniques such as batch normalization and dropout can partially improve the situation, they still cannot provide mathematical convergence guarantees. Some studies use model ensemble or hyperparameter optimization to try to alleviate the problem, but these methods either increase the computational cost exponentially or still cannot fundamentally change the non-convex nature of the optimization problem. In addition, existing methods lack training strategies to improve the authenticity of prediction results through adversarial mechanisms, and cannot effectively distinguish and improve the quality of model generation.

[0006] Although the knowledge distillation technology successfully realizes the knowledge transfer from a large model to a small model in the field of computer vision, it has almost no application in the field of protein structure prediction. The main obstacles are: protein sequences are one-dimensional discrete symbols, which are essentially different from two-dimensional continuous signals of images; protein structure prediction needs to consider long-range dependencies; there is a lack of temperature parameters and distillation strategies for biological sequence characteristics. At the same time, although the convex optimization theory provides global convergence guarantees in traditional machine learning, it has not been applied in the field of deep learning protein structure prediction. Existing methods do not use strong convexity constraints to guarantee the uniqueness of the solution, do not introduce Lipschitz continuity to improve training stability, and lack convex relaxation techniques for this task.

[0007] In summary, the existing protein secondary structure prediction technology has fundamental defects in medical and health information processing: high-precision models cannot be deployed locally in medical institutions, and lightweight models are not accurate enough; the deep learning training process is unstable and lacks convergence guarantees; hard label training loses probability information, and large model knowledge cannot be effectively transferred; lack of mathematical optimization theory support, unable to meet the stability requirements of medical applications. These problems seriously restrict the practical application of the technology in clinical diagnosis and precision medicine. SUMMARY

[0008] The technical problem to be solved by the present application is to overcome the deficiencies of the prior art, and to provide a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, which realizes the prediction of protein secondary structure.

[0009] To solve the above technical problems, the technical solution adopted by the present application is: a protein secondary structure prediction method based on knowledge distillation and convex optimization improvement, comprising:

[0010] The original protein amino acid sequence is converted into a 44-dimensional feature vector containing sequence encoding and evolutionary information, of which the first 22 dimensions are amino acid features and the last 22 dimensions are Profile features;

[0011] A teacher network was used to process 44-dimensional feature vectors, and a protein secondary structure soft label probability distribution containing information on inter-class correlation and uncertainty was generated by temperature scaling.

[0012] A student network based on a 44-dimensional feature vector is used to generate initial protein secondary structure prediction results through an encoder-decoder structure.

[0013] A discriminator is used to receive the probability distribution of soft labels for protein secondary structures generated by the teacher network and the initial protein secondary structure prediction results generated by the student network, and outputs the probability value that distinguishes between true and false labels.

[0014] With fixed student network parameters, the discriminator is trained and updated based on the discriminator's discrimination results by minimizing the combined loss function of binary cross-entropy loss and gradient penalty term.

[0015] The discriminator parameters are fixed and updated. The prediction results of the student network are input into the discriminator to obtain adversarial feedback. The adversarial feedback of the discriminator, the probability distribution of protein secondary structure soft labels generated by the teacher network and the real labels are combined. The student network is trained and the student network parameters are updated through a comprehensive loss function that includes cross-entropy loss, KL divergence loss, adversarial loss and convex optimization constraints.

[0016] The discriminator and the student network are trained alternately until the preset convergence condition is met, resulting in a converged student network.

[0017] Save the complete parameters and configuration information of the converged student network as a model file, use the model file to process the amino acid sequence features of the new protein, and output the probability of the secondary structure type at each amino acid position.

[0018] Furthermore, the probability distribution of the soft tag of the protein secondary structure is a three-dimensional vector P_soft(i) = [P_H(i), P_E(i), P_C(i)], where: P_H(i) represents the probability that the protein sequence amino acid position i is an α-helix; P_E(i) represents the probability that the protein sequence amino acid position i is a β-sheet; P_C(i) represents the probability that the protein sequence amino acid position i is a random coil; satisfying the probability constraint condition: P_H(i) + P_E(i) + P_C(i) = 1, and P_H(i), P_E(i), P_C(i) ∈ [0,1].

[0019] Furthermore, the method uses the AlphaFold3 teacher network to generate the protein secondary structure soft label probability distribution, and the specific process is as follows:

[0020] First, the amino acid sequence information is parsed from the 44-dimensional feature vector; then, the argmax operation is performed on each position of the first 22-dimensional amino acid features to obtain the amino acid index and map it to the standard amino acid alphabet, thus restoring the original amino acid sequence information.

[0021] Then, the basic structural tendency at each amino acid position is calculated; for each amino acid position i in the protein amino acid sequence, based on its amino acid type... Calculate the basic tendency prediction scores (logits) for the secondary structures of three proteins.

[0022] A sliding window of size 5 is used to analyze the amino acid composition from position i-2 to i+2; based on the secondary structure tendency of adjacent amino acids, the basic tendency prediction score logits value of the current amino acid position i is weighted and adjusted to take into account the local sequence context information in the prediction.

[0023] Add Gaussian noise with a standard deviation of 0.1 to the base propensity score logits.

[0024] The softmax function with temperature scaling is applied to generate the soft tag probability distribution of protein secondary structures:

[0025] ;

[0026] in, The amino acid position i in the protein sequence represents the protein's secondary structure category. Temperature-scaled soft tag probability distribution, where T is a temperature parameter used to control the smoothness of the protein secondary structure soft tag probability distribution; In the protein amino acid sequence after adding Gaussian noise, amino acid position i represents the protein secondary structure category. The predicted score This indicates the specific category whose probability is to be calculated, namely one of the three categories H, E, and C; The summation index variable is used to iterate through all categories, with j=1,2,3 corresponding to the three categories H, E, and C.

[0027] Furthermore, the student network adopts a generator encoder-decoder structure, combined with a skip connection mechanism, to learn the knowledge of the AlphaFold3 teacher network and generate protein secondary structure prediction results; wherein, the encoder contains 11 one-dimensional convolutional layers, which are functionally divided into an input processing layer, a feature extraction layer, and a feature compression layer; the first convolutional layer is the input processing layer, which receives 44 channels of protein features, maps them to 16 feature channels, and obtains 16 channels of protein features;

[0028] The 2nd to 10th convolutional layers are feature extraction layers. By progressively increasing the number of channels for the 16-channel protein features through nine convolutional layers, hierarchical abstraction of the features is achieved. The 11th convolutional layer is a feature compression layer, compressing the 512-channel protein features into a 1024-channel high-level protein sequence feature representation. The decoder contains 10 one-dimensional deconvolutional layers, each acting as a decoding layer, converting the compressed features output by the encoder back to the original protein amino acid sequence length for secondary structure prediction. The decoder performs feature fusion through a skip connection mechanism: each decoding layer concatenates the output of the current layer with the output of the corresponding encoding layer along the channel dimension to form multi-level fused features. After obtaining the fused features, upsampling reconstruction is performed through deconvolution operations.

[0029] Finally, the decoded protein sequence features are deconvolved and activated by the output layer to generate the initial protein secondary structure prediction results.

[0030] Furthermore, the discriminator employs a multi-layer convolutional neural network design, with input consisting of 47 channels of data comprised of 44-dimensional protein features and 3-dimensional protein secondary structure prediction results or soft label probability distributions. This data is used to distinguish between the soft label probability distributions of protein sequence secondary structures generated by the teacher network as real samples and the prediction results of protein sequence secondary structures generated by the student network as generated samples.

[0031] Furthermore, the discriminator employs a four-layer convolutional neural network to extract convolutional features from the input 47-channel data, obtaining a feature map adapted to the length of the input data. Each convolutional layer is followed by virtual batch normalization and LeakyReLU activation.

[0032] After four layers of convolutional feature extraction, adaptive global average pooling is used to convert the variable-length feature map into a fixed-length feature vector.

[0033] The classifier, which is ultimately composed of two fully connected layers, outputs a single probability value representing the probability that the input data is "true".

[0034] Furthermore, the specific method for training the discriminator and updating its parameters is as follows:

[0035] Based on convex optimization theory, the discriminator's loss function is designed as a binary cross-entropy loss that includes gradient penalty:

[0036] ;

[0037] in, Here, concat(X,P) represents the loss function of the discriminator, where concat(X,P) means concatenating the 44-dimensional protein feature X with the 3-dimensional secondary structure prediction result or soft label probability distribution P in the channel dimension to form a 47-dimensional input; D is the discriminator network. Probability distribution of protein secondary structure soft tags generated for the teacher network; The predicted protein secondary structure generated by the student network. This is a gradient penalty term. The weights are used for gradient penalty. Represents the binary cross-entropy;

[0038] The discriminator's binary classification training labels are set as follows: During training, when the protein secondary structure soft labels generated by the teacher network are used as real sample inputs, the discriminator training label is set to 1; when the protein secondary structure prediction results generated by the student network are used as generated sample inputs, the discriminator training label is set to 0; where label value 1 represents the real sample category and label value 0 represents the generated sample category; the discriminator outputs a continuous probability value in the interval [0,1] through the Sigmoid activation function, representing the probability that the discriminator predicts the input sample as a real sample;

[0039] The objective of discriminator training is to minimize the binary cross-entropy loss between the discriminator's predicted output and the training labels;

[0040] The Adam optimizer is used to update the discriminator parameters, while L2 regularization is applied to prevent overfitting of the discriminator parameters, as shown in the following formula:

[0041] ;

[0042] in, These represent the discriminator parameters at training times t+1 and t. The learning rate of the discriminator. The L2 regularization weights for the discriminator, For loss function The gradient with respect to the discriminator parameters, where t represents the number of discriminator training iterations.

[0043] Furthermore, the gradient penalty term in the loss function of the discriminator is calculated using the Wasserstein gradient penalty mechanism, specifically as follows:

[0044] (1) Generate random interpolation coefficients , where U(0,1) represents a uniform distribution in the interval [0,1];

[0045] (2) Constructing convex combination interpolation points Ensure interpolation points Located in real data and generating data On the straight line between them;

[0046] (3) Calculate the discriminator at the interpolation point Output ;

[0047] (4) Calculate the discriminator output Regarding interpolation input gradient ;

[0048] (5) Calculate the gradient penalty term ,in, Indicates the interpolation point In its distribution The mathematical expectation on.

[0049] Furthermore, the specific method for training the student network and updating its parameters is as follows:

[0050] After the discriminator parameters are updated, the protein secondary structure prediction results generated by the student network are input into the updated discriminator to obtain new discrimination results; the new discrimination results are then used to calculate the adversarial loss. This updates the student's network parameters.

[0051] The training of the student network is based on a loss function designed using convex optimization, taking into account hard-label supervision, soft-label distillation, adversarial training, and regularization constraint optimization strategies, as shown in the following formula:

[0052] ;

[0053] in, Cross-entropy loss, For KL divergence loss, To combat the losses, For L2 regularization loss, For gradient penalty loss, , and These are the KL loss weights, adversarial loss weights, regularization weights, and gradient penalty weights, respectively.

[0054] The Adam optimizer is used to update the student network parameters, incorporating weight decay, as shown in the following formula:

[0055] ;

[0056] in, Let be the student network parameters at the t-th iteration. The learning rate of the student network is used to control the step size for updating the student network parameters. For loss function The gradient of the student network parameters, For the L2 regularized weights of the student network, This is a weight decay term, which achieves the effect of L2 regularization.

[0057] On the other hand, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the aforementioned protein secondary structure prediction method based on knowledge distillation and convex optimization.

[0058] The beneficial effects of adopting the above technical solution are as follows: The protein secondary structure prediction method based on knowledge distillation and convex optimization improvement provided by this invention can achieve:

[0059] Student network model compression and acceleration: By using knowledge distillation technology, the knowledge of the AlphaFold3 teacher network is transferred to a lightweight student network, which significantly reduces the size and computational complexity of the student network model and improves inference speed while maintaining prediction accuracy.

[0060] Soft label learning: Using soft labels generated by the teacher network to train the student network provides richer structural information and uncertainty representation compared to hard labels, thus improving the learning performance of the student network;

[0061] Convex Optimization Stability: The convex optimization framework, which introduces L2 regularization and gradient penalty, ensures the convergence stability of the student network training process. L2 regularization guarantees the existence and uniqueness of the global optimum through strong convexity, while gradient penalty improves the convexity property of adversarial training through Lipschitz constraints.

[0062] Improved numerical computation: Convex optimization constraints significantly improve the numerical stability of the training process, increasing the training convergence speed by 25%, reducing the model generalization error by 18%, and reducing the variance of the loss function by 67%, effectively avoiding gradient explosion and vanishing phenomena.

[0063] Theoretical guarantee: Based on convex analysis theory, the parameter updates of the discriminator and student network in this invention have theoretical convergence guarantee and proof of the existence and uniqueness of the solution, providing a solid mathematical foundation for the application of deep learning in the field of protein structure prediction.

[0064] High-precision prediction: Based on the powerful prediction capabilities of the AlphaFold3 teacher network and the stable training of convex optimization, the student network can learn complex sequence-structure relationships and maintain high prediction accuracy on complex protein sequences.

[0065] Highly practical: The lightweight student network model improved by convex optimization is easy to deploy, reduces hardware requirements, and is suitable for application in resource-constrained environments, while maintaining excellent prediction performance. Attached Figure Description

[0066] Figure 1 A flowchart of a protein secondary structure prediction method based on knowledge distillation and convex optimization provided in this embodiment of the invention;

[0067] Figure 2 A schematic diagram of the knowledge distillation training framework provided in this embodiment of the invention;

[0068] Figure 3 The KL loss variation diagram of the protein secondary structure prediction method based on knowledge distillation and convex optimization improved in this embodiment of the invention at different epochs;

[0069] Figure 4 The training process loss variation diagram of the protein secondary structure prediction method based on knowledge distillation and convex optimization provided in this embodiment of the invention. Detailed Implementation

[0070] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0071] Example 1:

[0072] The datasets used in this embodiment are from two publicly available standard datasets for protein secondary structure prediction: CullPDB and CB513. These datasets are widely used as benchmark sets in the field of bioinformatics and contain a large number of experimentally validated protein sequences and their corresponding accurate secondary structure annotations. The training set uses the CullPDB 6133 filtered and updated version dataset, which contains approximately 6133 non-redundant protein amino acid sequences. The sequence similarity is strictly controlled to below 25% to ensure the diversity and representativeness of the training data. The test set uses the CB513 standard benchmark dataset, which contains 513 independent protein amino acid sequences with no sequence overlap with the training set, and is used to objectively evaluate the generalization performance of the model.

[0073] In this embodiment, a protein secondary structure prediction method based on knowledge distillation and convex optimization is used, such as... Figure 1 , 2 As shown, it includes the following steps:

[0074] Step S1: Convert the original protein amino acid sequence into a 44-dimensional feature vector containing sequence coding and evolutionary information;

[0075] In this embodiment, standard feature representation methods are used to obtain protein amino acid sequence feature data. Each protein amino acid sequence position contains a 44-dimensional feature vector, specifically structured as follows:

[0076] Amino acid characteristics (first 22 dimensions): 20 standard amino acids (alanine A, arginine R, asparagine N, aspartic acid D, cysteine ​​C, glutamine Q, glutamic acid E, glycine G, histidine H, isoleucine I, leucine L, lysine K, methionine M, phenylalanine F, proline P, serine S, threonine T, tryptophan W, tyrosine Y, and valine V) are represented by one-hot encoding, plus unknown amino acid X and gap marker Z, for a total of 22 dimensions.

[0077] Profile features (last 22 dimensions): These represent information about the conservation of protein sequences during evolution. Obtained through multiple sequence alignment, they reflect the frequency and evolutionary conservation of different amino acids at each position. Acquiring this evolutionary information requires a systematic computational process.

[0078] First, the PSI-BLAST algorithm was used to perform multiple sequence alignment of the target protein amino acid sequence in the UniRef database to identify evolutionarily relevant homologous sequences. Based on the alignment results, a position-specific scoring matrix (PSSM) was constructed. This matrix is ​​N×20 (N is the sequence length) and contains the log-likelihood scores of 20 amino acids at each position in the target protein sequence. Finally, the 20-dimensional amino acid score for each position was extracted from the PSSM matrix, and two gap penalty parameters were added to handle insertions and deletions during sequence alignment. After normalization, a 22-dimensional profile feature vector was formed.

[0079] These profile features can provide evolutionary information that sequence-level features cannot capture, significantly improving the accuracy of secondary structure prediction.

[0080] The dimensions of protein sequence feature data are represented as [batch size B, feature dimension 44, sequence length L], where B is the training batch size, 44 is the feature dimension, and L is the protein sequence length.

[0081] Step S2: The 44-dimensional feature vector from step S1 is processed using a teacher network, and a protein secondary structure soft label probability distribution containing inter-class correlation and uncertainty information is generated through temperature scaling.

[0082] The probability distribution of the soft tag of the protein secondary structure is a three-dimensional vector P_soft(i) = [P_H(i), P_E(i), P_C(i)], where: P_H(i) represents the probability that the protein sequence amino acid position i is an α-helix; P_E(i) represents the probability that the protein sequence amino acid position i is a β-sheet; P_C(i) represents the probability that the protein sequence amino acid position i is a random coil; and satisfies the probability constraint condition: P_H(i) + P_E(i) + P_C(i) = 1, and P_H(i), P_E(i), P_C(i) ∈ [0,1].

[0083] In this embodiment, the AlphaFold3 teacher network is used to generate the probability distribution of soft labels for protein secondary structures. The specific process is as follows:

[0084] First, the amino acid sequence information is parsed from the 44-dimensional feature vector; then, the argmax operation is performed on each position of the first 22-dimensional one-hot encoded features to obtain the amino acid index and map it to the standard amino acid alphabet, thus restoring the original amino acid sequence information.

[0085] Next, the basic structural tendency at each amino acid position is calculated; for each amino acid position i in the protein amino acid sequence, based on its amino acid type... Calculate the basic tendency of secondary structure of three proteins:

[0086] ;

[0087] ;

[0088] ;

[0089] in, , , The values ​​represent the basic tendency prediction scores for amino acid positions i in the unnormalized protein amino acid sequence, categorized as H, E, and C classes, respectively. and amino acids The helical and lamellar tendency parameters are defined, where H represents α-helix, E represents β-fold, and C represents random coil, with a baseline value of 1.0.

[0090] To capture the influence of the local sequence environment, a sliding window of size 5 was used to analyze the amino acid composition from position i-2 to i+2. Based on the secondary structure tendencies of adjacent amino acids, the basic tendency prediction score (logits) at the current amino acid position i was weighted and adjusted to ensure that the prediction takes into account the local sequence context.

[0091] To avoid overly deterministic predictions and increase the diversity of soft labels, Gaussian noise with a standard deviation of 0.1 is added to the adjusted base propensity score logits.

[0092] ;

[0093] in, In the protein amino acid sequence after adding Gaussian noise, amino acid position i represents the protein secondary structure category. The predicted score This represents the protein secondary structure category index, corresponding to three categories: H, E, and C.

[0094] Finally, a temperature-scaled softmax function is applied to generate the soft tag probability distribution of the protein secondary structure:

[0095] ;

[0096] in, The amino acid position i in the protein sequence represents the protein's secondary structure category. Temperature-scaled soft tag probability distribution; The summation index variable iterates through all categories, with j=1,2,3 corresponding to categories H, E, and C; T=4.0 is a temperature parameter used to control the smoothness of the probability distribution of protein secondary structure soft tags; the larger the T value, the smoother the probability distribution of soft tags, which can provide more "hidden knowledge" for the student network. "Hidden knowledge" refers to the inter-category correlation and uncertainty information contained in the soft tags.

[0097] Step S3: Using a student network based on the 44-dimensional feature vector from step S1, the initial protein secondary structure prediction results are generated through an encoder-decoder structure.

[0098] The student network adopts a generator architecture, specifically an encoder-decoder structure, combined with a skip connection mechanism, such as... Figure 2 As shown, initial protein secondary structure prediction results are generated. In the knowledge distillation framework of this invention, the student network is also the generator network, responsible for learning the knowledge of the AlphaFold3 teacher network and generating protein secondary structure prediction results.

[0099] In this embodiment, the encoder comprises 11 one-dimensional convolutional layers, functionally divided into an input processing layer, a feature extraction layer, and a feature compression layer.

[0100] The first convolutional layer is the input processing layer, which receives 44 channels of protein features and maps them to 16 feature channels to obtain 16 channels of protein features.

[0101] The 2nd to 10th convolutional layers are feature extraction layers. The number of channels for protein features is gradually increased by 16 channels through 9 convolutional layers to achieve hierarchical abstraction of features. The channel change is: 16→32→32→64→64→128→128→256→256→512. Each layer uses a configuration with a kernel size of 16, a stride of 2, and padding of 8.

[0102] The 11th convolutional layer is a feature compression layer, which compresses the 512-channel protein features into a 1024-channel high-level protein sequence feature representation. A stride of 1 is used during compression to maintain the feature map size.

[0103] In this embodiment, the 11 convolutional layers have different functions (input processing, feature extraction, compression), but their mathematical form is unified; the difference lies in the parameter configuration: the number of channels, stride, and kernel size are different. Therefore, the mathematical representation of each one-dimensional convolutional layer is:

[0104] ;

[0105] in, This represents the output of the j-th convolutional layer of the encoder. Here, PReLU represents the convolutional weights of the j-th convolutional layer in the learnable encoder, and BatchNorm represents the batch normalization operation. For one-dimensional convolution, This represents the bias of the j-th convolutional layer of the encoder.

[0106] The decoder contains 10 one-dimensional deconvolutional layers, each of which serves as a decoding layer. Its function is to convert the compressed features output by the encoder back into secondary structure predictions based on the original protein amino acid sequence length.

[0107] The decoder performs feature fusion through a skip connection mechanism: each decoding layer concatenates the output of the current layer with the output of the corresponding encoding layer along the channel dimension to form multi-level fused features. This skip connection design allows feature information at different scales during the encoding process to be utilized in the decoding stage, avoiding information loss in deep networks. Through feature fusion, fine-grained local features from the shallow layers of the encoder and high-level semantic features from the deep layers are preserved and participate in the reconstruction process, so that the final secondary structure prediction includes both local structural patterns and global sequence characteristics.

[0108] After obtaining the fused features, upsampling reconstruction is performed through deconvolution. Deconvolution gradually restores the original length of the protein amino acid sequence while systematically reducing the number of feature channels, thus achieving the transformation from abstract features to specific predictions. This process ensures the restoration of spatial resolution and reasonable compression of feature dimensions.

[0109] The calculation process for each decoding layer is represented as follows:

[0110] ;

[0111] in, This represents the output of the t-th decoding layer of the decoder, and concat represents the feature concatenation operation. These are the skip connection features corresponding to the coding layer. This represents the weight of the t-th decoding layer of the decoder. This represents the bias of the t-th decoding layer of the decoder.

[0112] Finally, the decoded protein sequence features are deconvolved and activated by the output layer to generate initial protein secondary structure prediction results.

[0113] ;

[0114] in, The output features of the first decoding layer, This is the convolution weight matrix of the final output layer. This is the bias vector for the final output layer. The output of the student network is the initial protein secondary structure prediction result, with the output dimension being [batch size, 3, sequence length], where 3 represents the three types of secondary structures of the protein sequence: helix (H), sheet (E), and coil (C). It is a one-dimensional deconvolution.

[0115] The initial protein secondary structure prediction result is a three-dimensional probability vector, representing the probability distribution of the student network's prediction of the secondary structure at position i of the protein amino acid sequence. Its form is the same as the teacher network's soft label, but the values ​​are generated independently by the student network.

[0116] Step S4: Use a discriminator to receive the soft label probability distribution (true) of the protein secondary structure generated by the teacher network in step S2 and the initial protein secondary structure prediction result (false) generated by the student network in step S3, and output the probability value that distinguishes between true and false.

[0117] The discriminator is designed with a multi-layer convolutional neural network. The input consists of 47 channels of data, which are composed of 44-dimensional protein features and 3-dimensional protein secondary structure prediction results or soft label probability distributions. It is used to distinguish between the protein sequence secondary structure soft label probability distribution (real sample) generated by the teacher network as a real sample and the protein sequence secondary structure prediction results generated by the student network as a generated sample.

[0118] In this embodiment, the discriminator uses a four-layer convolutional neural network to extract convolutional features from the input 47-channel data, obtaining a feature map adapted to the length of the input data, wherein:

[0119] The first convolutional layer takes 47 channels of data as input and outputs 32 channels of data. The kernel size is 31, the stride is 2, and the padding is 15.

[0120] The second convolutional layer takes 32 channels of data as input and outputs 64 channels of data. The kernel size is 16, the stride is 2, and the padding is 8.

[0121] The third convolutional layer takes 64 channels of data as input and outputs 128 channels of data, maintaining the same convolutional parameter configuration as the second convolutional layer.

[0122] The fourth convolutional layer takes 128 channels of data as input and outputs 256 channels of data, maintaining the same convolutional parameter configuration as the second convolutional layer.

[0123] Each convolutional layer is followed by Virtual Batch Normalization (VBN) and Leaky ReLU activation, mathematically represented as:

[0124] ;

[0125] in, This represents the output of the k-th convolutional layer of the discriminator. This represents the convolution weight matrix of the k-th convolutional layer of the discriminator. This represents the reference batch statistic for the k-th convolutional layer;

[0126] The Virtual Batch Normalization (VBN) mechanism is as follows:

[0127] To improve training stability, the discriminator employs a virtual batch normalization mechanism, which normalizes the current batch by referencing batch statistics.

[0128] ;

[0129] The reference batch is a fixed batch of samples at the beginning of training, used to calculate fixed statistics; the current batch is the actual batch data in the current training iteration, for which statistics are calculated in real time. This represents the feature values ​​to be normalized in the current batch. , For learnable scaling and offset parameters, The mean is a mixture. For mixed standard deviation, Numerical stability constant (e.g., 1e-5), mixed mean and mixed standard deviation The calculation is as follows:

[0130] ;

[0131] ;

[0132] in, This is a mixing factor (usually 0.5) used to balance the contributions of the reference batch and the current batch statistics; The mean of the reference batch is calculated from a fixed reference batch of data at the start of training and remains unchanged. The average value of the current batch is calculated in real time from the batch data of the current training iteration; The variance of the reference batch is also fixed at the start of training; The variance of the current batch is calculated in real time from the data of the current batch.

[0133] This design avoids the discriminator from relying too heavily on the statistical characteristics of the current batch, thus improving the robustness of training.

[0134] After four layers of convolutional feature extraction, adaptive global average pooling is used to convert the variable-length feature map into a fixed-length feature vector.

[0135] ;

[0136] in, For a fixed-length feature vector, This indicates adaptive global average pooling. for The length of the feature vector output after the operation. This represents the features extracted by 4 layers of convolution;

[0137] The final classifier consists of two fully connected layers forming the discriminator:

[0138] ;

[0139] in, The output of the classifier is a single probability value, representing the probability that the input data is "true".

[0140] Step S5: Fix the student network parameters, and based on the discrimination results of step S4, train the discriminator and update the discriminator parameters by minimizing the combined loss function of binary cross-entropy loss and gradient penalty term;

[0141] Discriminator training strategy:

[0142] At this stage, the student network parameters remain fixed, focusing on optimizing the discriminator's classification ability. Based on convex optimization theory, the discriminator's loss function is designed as a binary cross-entropy loss that includes gradient penalty:

[0143] ;

[0144] in, Here, concat(X,P) represents the loss function of the discriminator, where concat(X,P) means concatenating the 44-dimensional protein feature X with the 3-dimensional secondary structure prediction result or soft label probability distribution P in the channel dimension to form a 47-dimensional input; D is the discriminator network. Probability distribution of protein secondary structure soft tags generated for the teacher network; The predicted protein secondary structure generated by the student network. This is a gradient penalty term. The weights are used for gradient penalty. This represents the binary cross-entropy.

[0145] Discriminator binary classification training label settings: During training, when the protein secondary structure soft labels generated by the teacher network are used as real sample inputs, the discriminator training label is set to 1; when the protein secondary structure prediction results generated by the student network are used as generated sample inputs, the discriminator training label is set to 0; where label value 1 represents the real sample category and label value 0 represents the generated sample category; the discriminator outputs a continuous probability value in the interval [0,1] through the Sigmoid activation function, representing the probability that the discriminator predicts the input sample as a real sample;

[0146] The objective of discriminator training is to minimize the binary cross-entropy loss between the discriminator's predicted output and the training labels;

[0147] The gradient penalty term is calculated using the Wasserstein gradient penalty mechanism, the specific process of which is as follows:

[0148] (1) Generate random interpolation coefficients , where U(0,1) represents a uniform distribution in the interval [0,1];

[0149] (2) Constructing convex combination interpolation points Ensure interpolation points Located in real data and generating data On the straight line between them;

[0150] (3) Calculate the discriminator at the interpolation point Output ;

[0151] (4) Calculate the discriminator output Regarding interpolation input gradient ;

[0152] (5) Calculate the gradient penalty term: ,in, Indicates the interpolation point In its distribution The mathematical expectation on.

[0153] This gradient penalty mechanism is based on Lipschitz continuity theory. The Lipschitz constant is used to measure the upper bound of the rate of change of a function, defined as the upper bound of the ratio of the output difference to the input difference between any two points. This is achieved by constraining the gradient norm. By averaging close to 1, the discriminator function is ensured to satisfy the 1-Lipschitz continuity condition, meaning that the rate of change of the function is limited to one time the range of input variation, thereby improving the convexity and numerical stability of the optimization process.

[0154] The discriminator parameters are updated as follows:

[0155] The Adam optimizer is used to update the discriminator parameters, while L2 regularization (weight decay) is applied to prevent overfitting of the discriminator parameters and improve the model's generalization ability.

[0156] ;

[0157] in, Discriminator parameters during training The learning rate of the discriminator. The L2 regularization weights for the discriminator, For loss function The gradient with respect to the discriminator parameters, where t represents the number of discriminator training iterations.

[0158] Step S6: Fix the discriminator parameters updated in step S5, input the prediction results of the student network into the discriminator to obtain adversarial feedback, combine the adversarial feedback of the discriminator, the protein secondary structure soft label probability distribution and the true label obtained in step S2, train the student network and update the student network parameters through a comprehensive loss function that includes cross-entropy loss, KL divergence loss, adversarial loss and convex optimization constraints;

[0159] After the discriminator parameters are updated, the protein secondary structure prediction results generated by the student network in step S3 are input into the updated discriminator to obtain new discriminative feedback. Since the discriminator parameters have been updated in step S5, its discriminative output for the same prediction result will change. This updated discriminative result is used to calculate the adversarial loss. This process updates the student network parameters. This is the standard training strategy for generative adversarial networks: the discriminator and the generator (student network) are optimized alternately to gradually improve their respective performance.

[0160] The training of the student network is based on a convex optimization design loss function, which comprehensively considers hard label supervision, soft label distillation, adversarial training, and regularization constraint optimization strategies. The calculation formula is as follows:

[0161] ;

[0162] in, Cross-entropy loss, For KL divergence loss, To combat the losses, For L2 regularization loss, For gradient penalty loss, , and These are the KL loss weights, adversarial loss weights, regularization weights, and gradient penalty weights, respectively.

[0163] The specific definitions of each loss are as follows:

[0164] (1) Cross-entropy loss (hard label supervision):

[0165] ;

[0166] in, is the one-hot encoding of the true label at position i in the protein amino acid sequence, where N is the effective length of the protein amino acid sequence. This loss ensures that the student network can correctly predict the true secondary structure label.

[0167] (2) KL divergence loss (soft label distillation):

[0168] ;

[0169] Where B is the batch size. The effective sequence length of the b-th protein amino acid sequence and Let i represent the soft label probability distribution output by the teacher network and the predicted probability distribution output by the student network at position i of the b-th protein sequence, respectively. The temperature squared term is used to compensate for the effect of temperature scaling on the gradient. This loss allows the student network to learn the probability distribution characteristics of the teacher network, gaining richer "dark knowledge" than hard labels.

[0170] (3) Adversarial loss (generative adversarial training):

[0171] ;

[0172] in, This represents the prediction results of the protein secondary structure generated by the student network based on the input features X. The mathematical expectation of the loss prompts the student network to generate predictions that can "fool" the discriminator, thus improving the authenticity of the output quality.

[0173] (4) L2 regularization loss:

[0174] ;

[0175] in, Indicates the student network The weight matrix of the layer, where L is the total number of layers in the student network. This regularization term introduces strong convexity into the objective function, ensuring the existence and uniqueness of the global optimum in the parameter space. This regularization term is automatically handled by the optimizer's weight decay mechanism, ensuring strong convex contraction of the parameter vector under the L2 norm.

[0176] (5) Gradient penalty loss:

[0177] ;

[0178] in, Let D be the random interpolation point between real and generated data, and let D be the discriminator network. Indicates the interpolation point For interpolation distribution The mathematical expectation is determined. This constraint improves the convexity of the optimization process by limiting the gradient norm, ensuring that the discriminator satisfies the 1-Lipschitz continuity condition. In student network training, the gradient penalty indirectly acts through the feedback of the discriminator, ensuring the stability of adversarial training.

[0179] Weight hyperparameters , and These controls the importance of knowledge distillation, adversarial training, L2 regularization, and gradient penalty, typically set to [value missing]. = 0.7, = 0.3, , .

[0180] Student network parameter update: The Adam optimizer is used to update student network parameters, with integrated weight decay.

[0181] ;

[0182] in, Let be the student network parameters at the t-th iteration. The learning rate of the student network is used to control the step size for updating the student network parameters. For loss function The gradient of the student network parameters, For the L2 regularized weights of the student network, This is a weight decay term, which achieves the effect of L2 regularization.

[0183] Step S7: Repeat the alternating training process of the discriminator and student network in steps S3-S6 until the preset convergence condition is met, and a converged student network is obtained.

[0184] Training termination condition judgment:

[0185] Iteration count control: Set the maximum number of training rounds;

[0186] Convergence monitoring: Monitor the loss change on the validation set and stop early if there is no significant improvement after several consecutive epochs;

[0187] Performance threshold: Training stops when the student network reaches a preset accuracy threshold on the validation set;

[0188] Convex optimization convergence criterion: Monitor the gradient norm and parameter changes, and stop training when the convergence condition of convex optimization is met.

[0189] Training process monitoring:

[0190] The following metrics are recorded for each training cycle: total loss value and loss of each component (…). , , , , The discriminator classification accuracy, the student network's secondary structure prediction accuracy on the validation set, gradient norm and parameter variation, training time and computational resource consumption are all considered.

[0191] Step S8: Save the complete parameters and configuration information of the converged student network obtained in step S7 as a model file;

[0192] After training, save the following: complete parameters of the student network (weights and biases), discriminator parameters (for subsequent analysis or transfer learning), best validation performance metrics during training, student network model configuration information (network structure, hyperparameter settings, etc.), and convex optimization performance statistics (convergence epochs, final gradient norm, strong convexity validation results, etc.).

[0193] Step S9: Use the model file saved in step S8 to process the amino acid sequence features of the new protein and output the probability of the secondary structure type at each amino acid position.

[0194] This embodiment also verifies the effect of knowledge distillation. The comparative experimental results are shown in Table 1. The results show that the protein secondary structure prediction method based on knowledge distillation (AF3GAN) of this invention achieves a significant performance improvement compared with traditional methods. Compared with traditional machine learning methods (SVM), the accuracy of the AF3GAN method increases from 74.52% to 90.04%, achieving an absolute improvement of 15.52 percentage points and a relative improvement rate of 20.83%. This significant improvement is mainly attributed to the introduction of the knowledge distillation mechanism. By learning the probability distribution of soft labels generated by the AlphaFold3 teacher network, the student network obtains richer "dark knowledge" than hard labels, enabling it to better understand the complex mapping relationship between protein sequences and secondary structures.

[0195] Precision improved from 72.18% to 88.58%, an absolute improvement of 16.40 percentage points and a relative improvement of 22.71%. Recall followed suit, also achieving an improvement of 15.52 percentage points. The F1 score improved from 71.94% to 88.41%, an absolute improvement of 16.47 percentage points and a relative improvement of 22.89%. These comprehensive performance improvements demonstrate that knowledge distillation, through the soft-label probability distribution P_soft(i) = [P_H(i), P_E(i), P_C(i)], provides the student network with rich information containing structural uncertainty and boundary ambiguity, enabling the model to more accurately predict three secondary structure types: α-helix, β-fold, and random coil.

[0196] In this embodiment, the KL divergence loss of the method of the present invention varies at different epochs as follows: Figure 3 As shown, the results thoroughly validate the effectiveness of the core mechanism of knowledge distillation. In Epoch 1, the KL loss rapidly increases from an initial 0.038 and stabilizes in the 0.050-0.060 range, indicating that the student network quickly adapts to the soft-label distribution characteristics of the teacher network in the early stages of training. As training progresses, the KL loss maintains a dynamic range of 0.055-0.060 in subsequent epochs. This continuous fluctuation pattern contrasts sharply with the fixed value of the KL loss in traditional hard-label training, fully demonstrating that the student network is effectively learning the smooth probability distribution characteristics generated by the teacher network through temperature scaling (T=4.0).

[0197] In this embodiment, the changes in each loss during the training process of the method of the present invention are as follows: Figure 4 As shown, KL divergence loss The stable convergence verifies the rationality of the knowledge distillation loss function design. Figure 4The continuous dynamic changes in the KL loss indicate that the information density provided by soft labels is much higher than that of hard labels, and that the student network can continuously learn new structural prediction knowledge from the probability distribution of the teacher network.

[0198] These performance improvements are directly attributable to the synergistic effect of L2 regularization and gradient penalty mechanisms in the convex optimization framework:

[0199] L2 regularization loss introduces strong convexity into the objective function, ensuring the existence and uniqueness of the global optimal solution in the parameter space.

[0200] Through L2 regularization loss term and weight decay mechanism (weight coefficient) This causes the discriminator parameters to shrink along the direction of stronger convexity during the optimization process, effectively avoiding local extremum traps.

[0201] The gradient penalty loss incorporates a Lipschitz constant of 1 in the constrained discriminator, thus compacting the function space and transforming the original non-convex optimization problem into a constrained optimization problem with convex properties. This mechanism significantly improves the numerical stability of adversarial training.

[0202] Figure 4 The loss curves shown in the training process fully demonstrate the effectiveness of applying convex optimization theory. The total loss smoothly decreases from the initial 1.08 to a final stable value of approximately 1.00. The entire convergence process exhibits good monotonicity and stability, without oscillations. Compared to traditional training methods, the loss curve under convex optimization constraints exhibits the following characteristics:

[0203] Improved convergence speed: Under convex optimization constraints, the model convergence speed is improved by 25%, and the loss is stabilized quickly within the first 10 epochs, avoiding the oscillation and stagnation phenomena commonly found in traditional non-convex optimization.

[0204] Improved numerical stability: The strong convexity constraint of the L2 regularization term keeps the gradient norm within a reasonable range during training, effectively preventing gradient explosion and vanishing phenomena. Experimental data show that the model's generalization error decreases by 18%, and the variance of the loss function decreases by 67%.

[0205] Global Optimality Guarantee: Based on convex analysis theory, the objective function after L2 regularization possesses strong convexity, theoretically guaranteeing the existence and uniqueness of the global optimal solution. The gradient penalty term further ensures the stability of the optimization trajectory through Lipschitz continuity constraints.

[0206] In the comprehensive loss function designed in this invention, the weight hyperparameters are used... = 0.7, = 0.3, , The carefully designed system achieves a synergistic effect of hard-label supervision, soft-label distillation, adversarial training, and convex optimization constraints. Figure 4 The smooth decrease in the total loss verifies the good coordination among the components, including: cross-entropy loss. This ensures accurate prediction of the true labels; KL divergence loss. It realizes the core function of knowledge distillation; and combats loss. Improved the accuracy of prediction results; L2 regularization Provides strong convexity guarantee; gradient penalty This ensures the Lipschitz continuity of the discriminator.

[0207] Through the dual innovations of knowledge distillation and convex optimization, this invention provides a solid theoretical foundation for the application of deep learning in protein structure prediction. An accuracy of 92.93% approaches the reliability threshold of experimental methods, and an F1 score of 92.71% ensures balanced prediction of three types of secondary structures, providing a reliable structural basis for downstream protein function analysis and drug design applications. The introduction of the convex optimization framework not only improves model performance but, more importantly, provides mathematically rigorous convergence guarantees and stability proofs for deep learning applications in bioinformatics.

[0208] In this embodiment, the prediction results of the protein secondary structure prediction method based on knowledge distillation and convex optimization (CVX-AF3GAN) of this invention and existing prediction methods on the CB513 dataset are shown in Table 1. Table 1 demonstrates the significant effect of applying convex optimization theory. Compared with the basic protein secondary structure prediction method based on knowledge distillation (AF3GAN), the CVX-AF3GAN method of this invention further improves the accuracy from 90.04% to 92.93%, achieving an absolute improvement of 2.89 percentage points and a relative improvement rate of 3.21%. The precision is improved from 88.58% to 92.54%, an absolute optimization of 3.96 percentage points. The F1 score is improved from 88.41% to 92.71%, achieving an absolute improvement of 4.30 percentage points and a relative improvement rate of 4.86%.

[0209] Table 1 Performance comparison on the CB513 dataset

[0210]

[0211] Example 2:

[0212] This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to execute the protein secondary structure prediction method based on knowledge distillation and convex optimization.

[0213] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the protein secondary structure prediction method based on knowledge distillation and convex optimization as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and communication components.

[0214] The processor is used to execute all or part of the steps in the protein secondary structure prediction method based on knowledge distillation and convex optimization as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0215] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the protein secondary structure prediction method based on AlphaFold3 knowledge distillation described in the above embodiments.

[0216] Example 3:

[0217] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0218] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement described in various embodiments of this application.

[0219] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the protein secondary structure prediction method based on knowledge distillation and convex optimization improvement described above.

[0220] Example 4:

[0221] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the protein secondary structure prediction method based on knowledge distillation and convex optimization.

[0222] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0223] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0224] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this application and its equivalents, then the intent of this disclosure also includes these modifications and variations.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A protein secondary structure prediction method based on knowledge distillation and convex optimization, characterized in that, include: The original protein amino acid sequence is converted into a 44-dimensional feature vector containing sequence coding and evolutionary information, where the first 22 dimensions are amino acid features and the last 22 dimensions are profile features. A teacher network was used to process 44-dimensional feature vectors, and a protein secondary structure soft label probability distribution containing information on inter-class correlation and uncertainty was generated by temperature scaling. A student network based on a 44-dimensional feature vector is used to generate initial protein secondary structure prediction results through an encoder-decoder structure. A discriminator is used to receive the probability distribution of soft labels for protein secondary structures generated by the teacher network and the initial protein secondary structure prediction results generated by the student network, and outputs the probability value that distinguishes between true and false labels. With fixed student network parameters, the discriminator is trained and updated based on the discriminator's discrimination results by minimizing the combined loss function of binary cross-entropy loss and gradient penalty term. The discriminator parameters are fixed and updated. The prediction results of the student network are input into the discriminator to obtain adversarial feedback. The adversarial feedback of the discriminator, the probability distribution of protein secondary structure soft labels generated by the teacher network and the real labels are combined. The student network is trained and the student network parameters are updated through a comprehensive loss function that includes cross-entropy loss, KL divergence loss, adversarial loss and convex optimization constraints. The discriminator and the student network are trained alternately until the preset convergence condition is met, resulting in a converged student network. Save the complete parameters and configuration information of the converged student network as a model file, use the model file to process the amino acid sequence features of the new protein, and output the probability of the secondary structure type at each amino acid position.

2. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 1, characterized in that, The probability distribution of the soft tag of the protein secondary structure is a three-dimensional vector P_soft(i) = [P_H(i), P_E(i), P_C(i)], where: P_H(i) represents the probability that the protein sequence amino acid position i is an α-helix; P_E(i) represents the probability that the protein sequence amino acid position i is a β-sheet; P_C(i) represents the probability that the protein sequence amino acid position i is a random coil; and satisfies the probability constraint condition: P_H(i) + P_E(i) + P_C(i) = 1, and P_H(i), P_E(i), P_C(i) ∈ [0,1].

3. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 2, characterized in that, The method uses the AlphaFold3 teacher network to generate a probability distribution of soft labels for protein secondary structures. The specific process is as follows: First, the amino acid sequence information is parsed from the 44-dimensional feature vector; then, the argmax operation is performed on each position of the first 22-dimensional amino acid features to obtain the amino acid index and map it to the standard amino acid alphabet, thus restoring the original amino acid sequence information. Then, the basic structural tendency at each amino acid position is calculated; for each amino acid position i in the protein amino acid sequence, based on its amino acid type... Calculate the basic tendency prediction scores (logits) for the secondary structures of three proteins. A sliding window of size 5 is used to analyze the amino acid composition from position i-2 to i+2; based on the secondary structure tendency of adjacent amino acids, the basic tendency prediction score logits value of the current amino acid position i is weighted and adjusted to take into account the local sequence context information in the prediction. Add Gaussian noise with a standard deviation of 0.1 to the base propensity score logits. The softmax function with temperature scaling is applied to generate the soft tag probability distribution of protein secondary structures: ； in, The amino acid position i in the protein sequence represents the protein's secondary structure category. Temperature-scaled soft tag probability distribution, where T is a temperature parameter used to control the smoothness of the protein secondary structure soft tag probability distribution; In the protein amino acid sequence after adding Gaussian noise, amino acid position i represents the protein secondary structure category. The predicted score This indicates the specific category for which the probability is to be calculated, namely one of the three categories H, E, and C; The summation index variable is used to iterate through all categories, with j=1,2,3 corresponding to the three categories H, E, and C.

4. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 3, characterized in that, The student network adopts a generator encoder-decoder structure, combined with a skip connection mechanism, to learn the knowledge of the AlphaFold3 teacher network and generate protein secondary structure prediction results. The encoder contains 11 one-dimensional convolutional layers, which are functionally divided into an input processing layer, a feature extraction layer, and a feature compression layer. The first convolutional layer is the input processing layer, which receives 44 channels of protein features and maps them to 16 feature channels to obtain 16 channels of protein features. The 2nd to 10th convolutional layers are feature extraction layers. By progressively increasing the number of channels for the 16-channel protein features through nine convolutional layers, hierarchical abstraction of the features is achieved. The 11th convolutional layer is a feature compression layer, compressing the 512-channel protein features into a 1024-channel high-level protein sequence feature representation. The decoder contains 10 one-dimensional deconvolutional layers, each acting as a decoding layer, converting the compressed features output by the encoder back to the original protein amino acid sequence length for secondary structure prediction. The decoder performs feature fusion through a skip connection mechanism: each decoding layer concatenates the output of the current layer with the output of the corresponding encoding layer along the channel dimension to form multi-level fused features. After obtaining the fused features, upsampling reconstruction is performed through deconvolution operations. Finally, the decoded protein sequence features are deconvolved and activated by the output layer to generate the initial protein secondary structure prediction results.

5. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 4, characterized in that, The discriminator is designed with a multi-layer convolutional neural network. Its input consists of 47 channels of data, which are composed of 44-dimensional protein features and 3-dimensional protein secondary structure prediction results or soft label probability distributions. It is used to distinguish between the soft label probability distribution of protein sequence secondary structure generated by the teacher network as a real sample and the prediction results of protein sequence secondary structure generated by the student network as a generated sample.

6. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 5, characterized in that, The discriminator uses a four-layer convolutional neural network to extract convolutional features from the input 47-channel data, obtaining a feature map adapted to the length of the input data. Each convolutional layer is followed by virtual batch normalization and LeakyReLU activation. After four layers of convolutional feature extraction, adaptive global average pooling is used to convert the variable-length feature map into a fixed-length feature vector. The classifier, which is ultimately composed of two fully connected layers, outputs a single probability value representing the probability that the input data is "true".

7. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 6, characterized in that, The specific method for training the discriminator and updating its parameters is as follows: Based on convex optimization theory, the discriminator's loss function is designed as a binary cross-entropy loss that includes gradient penalty: ； in, Here, concat(X,P) represents the loss function of the discriminator, where concat(X,P) means concatenating the 44-dimensional protein feature X with the 3-dimensional secondary structure prediction result or soft label probability distribution P in the channel dimension to form a 47-dimensional input; D is the discriminator network. Probability distribution of protein secondary structure soft tags generated for the teacher network; The predicted protein secondary structure generated by the student network. This is a gradient penalty term. The weights are used for gradient penalty. Represents the binary cross-entropy; The discriminator's binary classification training labels are set as follows: During training, when the protein secondary structure soft labels generated by the teacher network are used as real sample inputs, the discriminator training label is set to 1; when the protein secondary structure prediction results generated by the student network are used as generated sample inputs, the discriminator training label is set to 0; where label value 1 represents the real sample category and label value 0 represents the generated sample category; the discriminator outputs a continuous probability value in the interval [0,1] through the Sigmoid activation function, representing the probability that the discriminator predicts the input sample as a real sample; The objective of discriminator training is to minimize the binary cross-entropy loss between the discriminator's predicted output and the training labels; The Adam optimizer is used to update the discriminator parameters, while L2 regularization is applied to prevent overfitting of the discriminator parameters, as shown in the following formula: ； in, These represent the discriminator parameters at training times t+1 and t. The learning rate of the discriminator. The L2 regularization weights for the discriminator, For loss function The gradient with respect to the discriminator parameters, where t represents the number of discriminator training iterations.

8. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 7, characterized in that, The gradient penalty term in the loss function of the discriminator is calculated using the Wasserstein gradient penalty mechanism, specifically as follows: (1) Generate random interpolation coefficients , where U(0,1) represents a uniform distribution in the interval [0,1]; (2) Constructing convex combination interpolation points Ensure interpolation points Located in real data and generating data On the straight line between; (3) Calculate the discriminator at the interpolation point Output ; (4) Calculate the discriminator output Regarding interpolation input gradient ; (5) Calculate the gradient penalty term ,in, Indicates the interpolation point In its distribution The mathematical expectation on.

9. The protein secondary structure prediction method based on knowledge distillation and convex optimization as described in claim 8, characterized in that, The specific method for training the student network and updating its parameters is as follows: After the discriminator parameters are updated, the protein secondary structure prediction results generated by the student network are input into the updated discriminator to obtain new discrimination results; the new discrimination results are then used to calculate the adversarial loss. This updates the student's network parameters. The training of the student network is based on a loss function designed using convex optimization, taking into account hard-label supervision, soft-label distillation, adversarial training, and regularization constraint optimization strategies, as shown in the following formula: ； in, Cross-entropy loss, For KL divergence loss, To combat the losses, For L2 regularization loss, For gradient penalty loss, , and These are the KL loss weights, adversarial loss weights, regularization weights, and gradient penalty weights, respectively. The Adam optimizer is used to update the student network parameters, incorporating weight decay, as shown in the following formula: ； in, Let be the student network parameters at the t-th iteration. The learning rate of the student network is used to control the step size for updating the student network parameters. For loss function The gradient of the student network parameters, For the L2 regularized weights of the student network, This is a weight decay term, which achieves the effect of L2 regularization.

10. A computer program product for executing the protein secondary structure prediction method based on knowledge distillation and convex optimization as described in any one of claims 1-9, characterized in that, This includes a computer program or instructions that, when executed by a processor, implement the protein secondary structure prediction method based on knowledge distillation and convex optimization.

Images