Protein function mining method and system based on attention mechanism and active learning

Abstract

The invention belongs to the technical field of function mining, and discloses a protein function mining method and system based on an attention mechanism and active learning, and the method comprises the steps: carrying out the initial training of obtained initial protein labeling data through a pre-constructed protein sequence task classification model, and obtaining an initial protein sequence task classification model; obtaining protein unlabeled data, carrying out data labeling grade sorting by utilizing the initial protein sequence task classification model, and carrying out labeling screening according to a data labeling grade sorting result; performing data integration on the screened protein unlabeled data and the initial protein labeled data to obtain updated protein labeled data, and performing iterative training by using the initial protein sequence task classification model to obtain a final protein sequence task classification model; and performing protein function mining on the obtained protein sequence data. According to the method, the generalization ability of the model under long-tail distribution is remarkably improved, and meanwhile, the sampling efficiency in large-scale data is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of function mining, in particular to a protein function mining method and system based on an attention mechanism and active learning. BACKGROUND

[0002] Protein function mining is a key link in the field of protein engineering, especially the identification of enzyme protein functions, which often relies on multiple biological experiments for verification, and the cost of confirming the function of a single protein is extremely high. Moreover, protein function prediction is also a core task of decoding the molecular mechanism of life activities, especially in the context of the rapid development of genomics and synthetic biology, its importance is increasingly prominent; with the breakthrough of sequencing technology, nearly 200 million protein sequences have been accumulated in databases such as UniProt, but only 0.3% have been experimentally verified, and traditional computational methods rely on homology alignment or manual features such as sequence conserved motifs, so there are two major bottlenecks; one is that the sequence similarity of distantly related homologous proteins is inconsistent with the functional relevance; the second is that the complexity of alignment-based algorithms makes it difficult to cope with the exponential growth of databases, resulting in a high error rate of up to 40% in the function mining of newly discovered proteins. In recent years, self-supervised learning technology has provided a new way to break through the above limitations; by pre-training language models such as ESM and ProtTrans, implicit physical and chemical and evolutionary features can be extracted directly from amino acid sequences without relying on multiple sequence alignment. However, the performance of such models is limited when there is a lack of labeled data, especially for low homology new enzyme prediction accuracy significantly decreased. In this context, the active learning strategy is the key, which can dynamically optimize the model training set by intelligently selecting high-information unlabelled samples for experimental verification, to improve the prediction reliability with the minimum labeling cost.

[0003] However, in protein function prediction, although traditional active learning methods perform well on small-scale, class-balanced datasets, in actual scenarios, they face challenges such as large data size, extreme class distribution imbalance, and significant labeling noise, resulting in a significant decline in performance. Uncertainty-based sampling methods are easily affected by the performance of the initial model, and diversity-based strategies are difficult to capture the non-linear relationships between deep features. Partial combination strategies alleviate the above problems to some extent, but there are still new bottlenecks such as the need for manual setting of the size of the candidate sample set and high computational overhead, which limit their widespread application in protein engineering tasks.

[0004] In summary, the deployment of active learning methods faces double challenges. On the one hand, the high parameter scale of protein large language models and the massive sequence data lead to exponential growth in training computing costs, making it difficult for active learning frameworks based on variational autoencoders (VAE) and adversarial generative networks (VAAL) to be practically applied. On the other hand, although traditional uncertainty sampling methods have computational efficiency advantages, their core defect is over-reliance on the prediction results of the initial classification model. When the model's early performance is insufficient, the reliability of uncertainty estimation is significantly reduced. In addition, such methods fail to effectively handle inherent noise in the data, for example, the presence of remotely homologous proteins with sequence similarity exceeding 50% in enzyme protein families. The redundant features of these proteins interfere with the identification of key functional residues by the classifier, leading to the failure of sample value evaluation based on latent space similarity, such as ALFA-Mix and CoreGCN.

[0005] Therefore, how to provide a protein function mining method and system based on attention mechanism and active learning is a problem to be solved at present. SUMMARY

[0006] The embodiments of the present application provide a protein function mining method and system based on attention mechanism and active learning to solve the above technical problems in the prior art.

[0007] To have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not a general overview, nor is it intended to identify key / important elements or delineate the scope of the embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0008] According to a first aspect of the embodiments of the present application, a protein function mining method based on attention mechanism and active learning is provided.

[0009] In one embodiment, the protein function mining method based on attention mechanism and active learning comprises:

[0010] initially training the obtained initial protein annotation data using a pre-constructed protein sequence task classification model to obtain an initial protein sequence task classification model;

[0011] obtaining protein unlabeled data and performing data annotation level sorting using the initial protein sequence task classification model to obtain a data annotation level sorting result, and performing annotation screening on the protein unlabeled data according to the data annotation level sorting result to obtain screened protein unlabeled data;

[0012] The unlabeled protein data selected from the screening process is integrated with the initial labeled protein data to obtain updated labeled protein data. The updated labeled protein data is then iteratively trained using the initial protein sequence task classification model to obtain the final protein sequence task classification model.

[0013] Based on the final protein sequence task classification model, protein function mining is performed on the acquired protein sequence data to obtain protein function mining results.

[0014] In one embodiment, the initial protein sequence task classification model is initially trained on the acquired initial protein annotation data using a pre-built protein sequence task classification model, resulting in the following initial protein sequence task classification model:

[0015] The pre-built protein sequence task classification model is initialized based on the acquired initial protein annotation data to obtain the initialized protein sequence task classification model.

[0016] The initial protein sequence task classification model was used to make preliminary predictions on the initial protein annotation data, and preliminary prediction results were obtained.

[0017] Based on the preliminary prediction results, the initial protein sequence task classification model was optimized using the binary cross-entropy loss function to obtain the initial protein sequence task classification model.

[0018] In one embodiment, the expression for the binary cross-entropy loss function is:

[0019]

[0020] In the formula, BCE Loss N represents the number of groups of objects predicted by the model; y i Indicates a binary label 0 or 1; p(y i ) indicates the probability of outputting a label.

[0021] In one embodiment, unlabeled protein data is acquired, and the data is ranked by labeling level using an initial protein sequence task classification model to obtain the data labeling level ranking result. Based on the data labeling level ranking result, the unlabeled protein data is then filtered for labeling, resulting in the following filtered unlabeled protein data:

[0022] The initial protein sequence task classification model is used to perform initial classification prediction on the acquired unlabeled protein data, and the self-attention mechanism is bypassed to perform secondary classification prediction on the unlabeled protein data.

[0023] The initial protein sequence classification model includes: protein language model, recurrent neural network, self-attention mechanism, and classifier;

[0024] The initial classification prediction results and the secondary classification prediction results are probabilistically mapped based on the sigmoid function, and the sample information content score is calculated based on the probability mapping results. The unlabeled protein data are sorted based on the sample information content score to obtain the data labeling level ranking results.

[0025] The unlabeled protein data were filtered and labeled based on the data labeling level sorting results, resulting in the filtered unlabeled protein data.

[0026] In one embodiment, the initial classification prediction of the acquired unlabeled protein data is performed using an initial protein sequence task classification model, and a secondary classification prediction of the unlabeled protein data is performed by bypassing the self-attention mechanism, including:

[0027] Protein sequence features were extracted from the acquired unlabeled protein data using a protein language model, and the protein sequence feature extraction results were obtained.

[0028] Based on the protein sequence feature extraction results, a recurrent neural network is used for gradient processing to obtain the processed protein sequence feature extraction results.

[0029] The self-attention mechanism is used to correct the features of the extracted protein sequence, and the protein sequence features are corrected. Based on the protein sequence features corrected, a classifier is used to make an initial classification prediction, and the initial classification prediction result is obtained.

[0030] The initial protein sequence task classification model bypasses the self-attention mechanism to perform secondary classification prediction on unlabeled protein data, resulting in secondary classification prediction results.

[0031] In one embodiment, a protein language model is used to extract protein sequence features from the acquired unlabeled protein data, and the protein sequence feature extraction results include:

[0032] Relative position encoding is introduced into the protein language model, and position information is encoded in the acquired unlabeled protein data to obtain amino acid residue data;

[0033] The amino acid residue data is masked using a masking language model, and context prediction is performed based on the masking results to obtain the context prediction results.

[0034] Based on the context prediction results, co-evolutionary signals between residues and global structural constraints are learned to obtain preliminary protein sequence features. Then, the preliminary protein sequence features are used to generate folded backbones and three-dimensional structures to obtain protein sequence feature extraction results.

[0035] In one embodiment, a self-attention mechanism is used to correct the extracted protein sequence features, resulting in a corrected protein sequence feature result. Based on this corrected result, a classifier is used for initial classification prediction, yielding the following initial classification prediction result:

[0036] The processed protein sequence feature extraction results are projected into the latent space using the weight matrix and bias of the self-attention mechanism to obtain the protein sequence feature projection results.

[0037] The protein sequence feature projection results are scalar scored using the hidden state of the parameter vector, and the scalar scoring results are normalized to obtain normalized scalar scoring results. The normalized scalar scoring results are then filled with noise and suppressed using a dynamic masking mechanism to obtain attention weights.

[0038] The processed protein sequence feature extraction results are weighted and summed according to the attention weights to obtain the weighted sum result. The weighted sum result is then dimensionally compressed to obtain the protein sequence feature correction result.

[0039] The protein sequence feature correction results are used to perform initial classification prediction using a classifier to obtain the initial classification prediction results.

[0040] In one embodiment, a probability mapping is performed on the initial classification prediction results and the secondary classification prediction results based on a sigmoid function, and a sample information content score is calculated based on the probability mapping results. The unlabeled protein data are then ranked based on these sample information content scores to obtain the data labeling level ranking results, including:

[0041] The initial classification prediction is mapped using the sigmoid function to obtain the initial classification probability, and the secondary classification prediction result is mapped using the probability to obtain the secondary classification prediction probability.

[0042] The initial classification probability and the secondary classification prediction probability are input into the sample value evaluation function, and the sample information score is calculated through the sample value evaluation function.

[0043] Based on the numerical value of the sample information content score, the unlabeled protein data are sorted according to preset rules to obtain the data labeling level ranking results.

[0044] In one embodiment, the expression for the sample value evaluation function is:

[0045]

[0046] In the formula, S(x) represents the sample value evaluation function; C represents the total number of categories; p1 represents the probability of secondary classification; p2 represents the probability of primary classification prediction; y represents the prediction result; and x represents the input sample.

[0047] According to a second aspect of the present invention, a protein function mining system based on attention mechanisms and active learning is provided.

[0048] In one embodiment, a protein function mining system based on attention mechanisms and active learning includes: a classification model initial training module, a labeled data screening module, an iterative training module, and a protein function mining module.

[0049] The initial training module for the classification model is used to perform initial training on the acquired initial protein annotation data using a pre-built protein sequence task classification model to obtain the initial protein sequence task classification model.

[0050] The labeled data filtering module is used to obtain unlabeled protein data, sort the data labeling levels using the initial protein sequence task classification model, obtain the data labeling level sorting results, and filter the unlabeled protein data according to the data labeling level sorting results to obtain the filtered unlabeled protein data.

[0051] The iterative training module is used to integrate the selected unlabeled protein data with the initial labeled protein data to obtain updated labeled protein data, and then use the initial protein sequence task classification model to iteratively train the updated labeled protein data to obtain the final protein sequence task classification model.

[0052] The protein function mining module is used to perform protein function mining on the acquired protein sequence data based on the final protein sequence task classification model, and obtain the protein function mining results.

[0053] According to a third aspect of the present invention, a computer device is provided.

[0054] In some embodiments, the computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the protein function mining method based on attention mechanism and active learning described above.

[0055] According to a fourth aspect of the present invention, a computer-readable storage medium is provided.

[0056] In one embodiment, a computer program is stored on a computer-readable storage medium, which, when executed by a processor, implements the steps of the protein function mining method based on attention mechanisms and active learning described above.

[0057] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0058] 1. This invention, through a self-attention mechanism and a sample value evaluation function, can measure the sample value before and after feature correction from the perspective of data features, thereby quantifying the amount of sample information, and prioritizing the selection of samples with weak model feature expression capabilities, strengthening the feature extraction of rare enzyme catalytic sites, significantly improving the model's generalization ability under long-tailed distributions while ensuring sampling efficiency in large-scale data, and alleviating the problems of traditional uncertain active learning relying too much on model classification performance and failing in noisy scenarios.

[0059] 2. This invention utilizes a self-attention mechanism to effectively mitigate the class bias problem under long-tailed distributions in enzyme protein function mining tasks. It maintains linear time complexity on large-scale datasets and significantly reduces annotation costs in protein function tasks. At the same time, it alleviates the problem of traditional active learning relying too much on the classification performance of the initial model to some extent.

[0060] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0062] Figure 1 This is a flowchart illustrating a protein function mining method based on attention mechanisms and active learning, according to an exemplary embodiment.

[0063] Figure 2 This is a schematic diagram of the system structure according to an exemplary embodiment;

[0064] Figure 3 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment;

[0065] Figure 4 This is a schematic diagram of an active learning framework in a protein function mining method based on attention mechanisms and active learning, according to an exemplary embodiment.

[0066] Figure 5 This is a schematic diagram of a protein language model in a protein function mining method based on attention mechanism and active learning, according to an exemplary embodiment.

[0067] Figure 6 This is a flowchart illustrating a classification model in a protein function mining method based on attention mechanisms and active learning, according to an exemplary embodiment.

[0068] Figure 7This is a flowchart illustrating the self-attention mechanism in a protein function mining method based on attention mechanism and active learning, according to an exemplary embodiment.

[0069] Figure 8 This is a schematic diagram of active sampling with self-attention divergence in a protein function mining method based on attention mechanism and active learning, according to an exemplary embodiment.

[0070] Figure 9 This is a flowchart illustrating a protein function mining method based on attention mechanisms and active learning, according to an exemplary embodiment.

[0071] Figure 10 This is a flowchart illustrating the active learning process based on self-attention divergence in a protein function mining method based on attention mechanisms and active learning, according to an exemplary embodiment.

[0072] Figure label:

[0073] 201. Initial training module for classification model; 202. Labeled data selection module; 203. Iterative training module; 204. Protein function mining module. Detailed Implementation

[0074] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some portions and features of certain embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents thereof. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0075] The modules in the apparatus or system of this application can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0076] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0077] Figure 1 An embodiment of the protein function mining method based on attention mechanism and active learning of the present invention is shown.

[0078] In this optional embodiment, the protein function mining method based on attention mechanisms and active learning includes:

[0079] Step S101: Initial training is performed on the acquired initial protein annotation data using a pre-built protein sequence task classification model to obtain the initial protein sequence task classification model.

[0080] Step S102: Obtain unlabeled protein data, sort the data labeling levels using the initial protein sequence task classification model, obtain the data labeling level sorting results, and filter the unlabeled protein data according to the data labeling level sorting results to obtain the filtered unlabeled protein data.

[0081] Step S103: The screened unlabeled protein data is integrated with the initial labeled protein data to obtain updated labeled protein data. The updated labeled protein data is then iteratively trained using the initial protein sequence task classification model to obtain the final protein sequence task classification model.

[0082] Step S104: Based on the final protein sequence task classification model, protein function mining is performed on the acquired protein sequence data to obtain protein function mining results.

[0083] In this optional embodiment, the initial protein sequence task classification model is initially trained on the acquired initial protein annotation data using a pre-built protein sequence task classification model, resulting in the following initial protein sequence task classification model:

[0084] The pre-built protein sequence task classification model is initialized based on the acquired initial protein annotation data to obtain the initialized protein sequence task classification model.

[0085] The initial protein sequence task classification model was used to make preliminary predictions on the initial protein annotation data, and preliminary prediction results were obtained.

[0086] Based on the preliminary prediction results, the initial protein sequence task classification model was optimized using the binary cross-entropy loss function to obtain the initial protein sequence task classification model.

[0087] In this optional embodiment, the expression for the binary cross-entropy loss function is:

[0088]

[0089] In the formula, BCE Loss N represents the number of groups of objects predicted by the model; y i Indicates a binary label 0 or 1; p(y i ) indicates the probability of outputting a label.

[0090] In this optional embodiment, unlabeled protein data is acquired, and the data annotation levels are ranked using an initial protein sequence task classification model to obtain the data annotation level ranking results. Based on the data annotation level ranking results, the unlabeled protein data is then labeled and filtered to obtain the filtered unlabeled protein data, which includes:

[0091] The initial protein sequence task classification model is used to perform initial classification prediction on the acquired unlabeled protein data, and the self-attention mechanism is bypassed to perform secondary classification prediction on the unlabeled protein data.

[0092] The initial protein sequence classification model includes: protein language model, recurrent neural network, self-attention mechanism, and classifier;

[0093] The initial classification prediction results and the secondary classification prediction results are probabilistically mapped based on the sigmoid function, and the sample information content score is calculated based on the probability mapping results. The unlabeled protein data are sorted based on the sample information content score to obtain the data labeling level ranking results.

[0094] The unlabeled protein data were filtered and labeled based on the data labeling level sorting results, resulting in the filtered unlabeled protein data.

[0095] In this optional embodiment, the initial classification prediction of the acquired unlabeled protein data is performed using an initial protein sequence task classification model, and a secondary classification prediction of the unlabeled protein data is performed by bypassing the self-attention mechanism, including:

[0096] Protein sequence features were extracted from the acquired unlabeled protein data using a protein language model, and the protein sequence feature extraction results were obtained.

[0097] Based on the protein sequence feature extraction results, a recurrent neural network is used for gradient processing to obtain the processed protein sequence feature extraction results.

[0098] The self-attention mechanism is used to correct the features of the extracted protein sequence, and the protein sequence features are corrected. Based on the protein sequence features corrected, a classifier is used to make an initial classification prediction, and the initial classification prediction result is obtained.

[0099] The initial protein sequence task classification model bypasses the self-attention mechanism to perform secondary classification prediction on unlabeled protein data, resulting in secondary classification prediction results.

[0100] In this optional embodiment, a protein language model is used to extract protein sequence features from the acquired unlabeled protein data, and the protein sequence feature extraction results include:

[0101] Relative position encoding is introduced into the protein language model, and position information is encoded in the acquired unlabeled protein data to obtain amino acid residue data;

[0102] The amino acid residue data is masked using a masking language model, and context prediction is performed based on the masking results to obtain the context prediction results.

[0103] Based on the context prediction results, co-evolutionary signals between residues and global structural constraints are learned to obtain preliminary protein sequence features. Then, the preliminary protein sequence features are used to generate folded backbones and three-dimensional structures to obtain protein sequence feature extraction results.

[0104] In this optional embodiment, a self-attention mechanism is used to correct the extracted protein sequence features, resulting in a corrected protein sequence feature result. Based on this corrected result, a classifier is used for initial classification prediction, yielding the following initial classification prediction result:

[0105] The processed protein sequence feature extraction results are projected into the latent space using the weight matrix and bias of the self-attention mechanism to obtain the protein sequence feature projection results.

[0106] The protein sequence feature projection results are scalar scored using the hidden state of the parameter vector, and the scalar scoring results are normalized to obtain normalized scalar scoring results. The normalized scalar scoring results are then filled with noise and suppressed using a dynamic masking mechanism to obtain attention weights.

[0107] The processed protein sequence feature extraction results are weighted and summed according to the attention weights to obtain the weighted sum result. The weighted sum result is then dimensionally compressed to obtain the protein sequence feature correction result.

[0108] The protein sequence feature correction results are used to perform initial classification prediction using a classifier to obtain the initial classification prediction results.

[0109] In this optional embodiment, a probability mapping is performed on the initial classification prediction results and the secondary classification prediction results based on the sigmoid function, and a sample information content score is calculated based on the probability mapping results. The unlabeled protein data are then sorted based on the sample information content score to obtain the data labeling level ranking results, including:

[0110] The initial classification prediction is mapped using the sigmoid function to obtain the initial classification probability, and the secondary classification prediction result is mapped using the probability to obtain the secondary classification prediction probability.

[0111] The initial classification probability and the secondary classification prediction probability are input into the sample value evaluation function, and the sample information score is calculated through the sample value evaluation function.

[0112] Based on the numerical value of the sample information content score, the unlabeled protein data are sorted according to preset rules to obtain the data labeling level ranking results.

[0113] In this optional embodiment, the expression for the sample value evaluation function is:

[0114]

[0115] In the formula, S(x) represents the sample value evaluation function; C represents the total number of categories; p1 represents the probability of secondary classification; p2 represents the probability of primary classification prediction; y represents the prediction result; and x represents the input sample.

[0116] Figure 2 An embodiment of the protein function mining system based on attention mechanism and active learning of the present invention is shown.

[0117] In this optional embodiment, the protein function mining system based on attention mechanism and active learning includes: a classification model initial training module 201, a labeled data screening module 202, an iterative training module 203, and a protein function mining module 204.

[0118] The classification model initial training module 201 is used to perform initial training on the acquired initial protein annotation data using a pre-built protein sequence task classification model to obtain an initial protein sequence task classification model.

[0119] The labeled data filtering module 202 is used to obtain unlabeled protein data, sort the data labeling levels using the initial protein sequence task classification model, obtain the data labeling level sorting results, and filter the unlabeled protein data according to the data labeling level sorting results to obtain the filtered unlabeled protein data.

[0120] The iterative training module 203 is used to integrate the selected unlabeled protein data with the initial labeled protein data to obtain updated labeled protein data, and to iteratively train the updated labeled protein data using the initial protein sequence task classification model to obtain the final protein sequence task classification model.

[0121] The protein function mining module 204 is used to perform protein function mining on the acquired protein sequence data based on the final protein sequence task classification model, and obtain the protein function mining results.

[0122] To facilitate understanding of the above technical solutions of this invention, the following further explains the technical solutions from the perspectives of architecture and principles: Active learning is a branch of machine learning dedicated to researching how to obtain the greatest possible performance improvement by labeling as few samples as possible. More specifically, its goal is to select the most useful samples from unlabeled datasets and then hand them over to an oracle (such as human annotators) for labeling, minimizing labeling costs while maintaining performance. By designing sampling strategies, samples that maximize the model's information gain are dynamically selected in each training cycle. Specifically, its core lies in constructing a sample value evaluation function, which must simultaneously consider the model's current knowledge blind spots and data distribution characteristics. High-value samples are selected from a large pool of unlabeled samples using a certain sampling strategy and handed over to human annotation experts for labeling. Active learning frameworks include... Figure 4 As shown, this is a learning algorithm for continuously iterating and optimizing a model. Its core consists of five parts, including an unlabeled dataset D. U This represents the entire unlabeled data pool. Active learning extracts high-value data from this entire unlabeled data pool to improve the model's generalization ability. Labeled dataset D L This is the manually labeled dataset. In active learning, a small, randomly selected dataset needs to be labeled to initialize the model and give it a certain degree of generalization performance. The sampling strategy S is the core of active learning; it helps to quickly select the most valuable batch of data from the unlabeled data pool, reducing the labeling cost of redundant data and improving model performance more quickly. The target model M is the final goal. Active learning selectively selects samples for training the target model, thereby obtaining sufficient model generalization ability within a limited labeling budget. The entire active learning process is as follows: In the initial state, one or more samples can be randomly selected from the unlabeled sample pool U. These samples are then submitted to an Oracle to query labels, resulting in a labeled dataset L. Then, supervised learning is used on D... L The model is trained. Next, using this new knowledge, the next sample to be queried is selected, added to L, and then trained. This process is repeated until the label budget is exhausted or a predefined termination condition is met. The sampling strategy in active learning is crucial, and the key to selecting unlabeled samples is establishing a reasonable and efficient sample value evaluation system. The constructed value evaluation system is then used to evaluate the unlabeled data D. U Each unlabeled data point is assigned a value score, and the data are then sorted in descending order of value scores. The top k samples are selected for expert labeling before being added to the model training set. The ultimate goal of constructing the sampling strategy is to select samples that maximize model performance.

[0123] It's worth noting that the attention mechanism in deep learning draws inspiration from the operating principles of the human visual and cognitive systems, guiding neural networks to dynamically filter key information when processing data. This mechanism assigns differentiated weights to different regions of the input data, enabling the model to autonomously focus on feature regions that significantly contribute to the current task during computation. This characteristic not only effectively enhances model performance but also improves its generalization potential to adapt to diverse data scenarios. In the field of sequence data processing, such as text parsing, speech recognition, or image temporal analysis, the introduction of the attention mechanism significantly improves the limitations of traditional models. Specifically, self-attention architecture achieves efficient modeling of long-distance dependencies by capturing the correlations between elements within a sequence; spatial attention focuses on analyzing the spatial distribution characteristics of visual features, thereby optimizing the processing effect of image understanding tasks; and attention models designed for temporal signals can accurately identify the differences in importance at different time points. This differentiated weight allocation strategy allows the model to adaptively adjust its focus when processing complex data, thereby improving overall computational efficiency and decision quality.

[0124] It should be added that, such as Figure 5 As shown, protein language models, such as ESM-2 (Evolutionary Scale Modeling), are large-scale protein language models. Their core idea is to implicitly learn atomic-level three-dimensional structural information from billions of protein sequences through unsupervised scaling, breaking through the dependence of traditional methods on multiple sequence alignment (MSA). The model is based on the Transformer architecture, introducing relative position encoding (RoPE) to support arbitrary sequence length processing, and is trained through a masked language model (MLM) task. This involves randomly masking 15% of the amino acids in the input sequence, and the model predicts the masked positions based on context, forcing the learning of co-evolutionary signals between residues and global structural constraints. Training data comes from billions of evolutionarily diverse sequences in the UniRef database, and a dynamic sampling strategy enhances data diversity. As the model parameters scale from 8 million to 15 billion, its structure prediction capability exhibits a non-linear leap. For example, when the parameters increase from 35 million to 150 million, the RMSD of a specific protein, such as 7QQA, drops sharply from 7.0 to 3.2, showing that the increase in model scale directly drives the implicit mapping capability from sequence to structure. The Active Protein Learning Framework (APLF), based on self-attention divergence (i.e., self-attention mechanism), is used in protein sequence classification tasks (i.e., protein function mining). It measures the amount of information in samples at the data feature level, ensuring sampling efficiency while mitigating the problems of traditional uncertain active learning models, such as over-reliance on model classification performance and failure in noisy environments. Protein sequence classification models, such as... Figure 6As shown, this classification model mainly consists of ESM, GRU, a self-attention mechanism, and a classifier. ESM is an unsupervised protein language model based on Transformer, capable of directly predicting the structure, function, and other properties of a protein from its amino acid sequence. In practical applications, a pre-trained ESM-2650M model with a limited number of parameters is used as the feature extractor for the amino acid sequence. Next, a variant of the recurrent neural network, GRU, is used to process the features, preventing the gradient vanishing and gradient exploding problems of traditional RNNs. Then, a simplified self-attention layer is used to correct the final features, weakening unimportant features and strengthening important ones. Finally, a linear layer serves as the classifier.

[0125] It should be further explained that the self-attention mechanism, specifically the self-attention layer, is a lightweight self-attention mechanism optimized for efficiently handling long protein sequence classification tasks. For example... Figure 7 As shown, this mechanism consists of three stages of computation. The first stage is feature projection, which is achieved through a learnable weight matrix W∈R. d×h and bias b∈R d Given the input sequence X∈R n×d The sequence is projected into the latent space E = tanh(X·W+b), where n is the sequence length, d is the embedding dimension, and h is the attention hidden layer dimension. The second stage is the generation of attention weights, i.e., using the parameter vector u∈R. d The hidden state is scored by a scalar S = Eu, and then normalized by Softmax to obtain the attention distribution A = Softmax(S). A dynamic masking mechanism is introduced to suppress padding noise. The third stage is context aggregation, which is to perform a weighted sum O = A on the original input according to the weight A. T X, the output dimension is compressed to R. d Compared to standard Transformer self-attention, this design reduces computational complexity to linear complexity through a single-layer projection plus scalar scoring strategy, and uses tanh nonlinear activation to replace multi-head interaction, which significantly improves the processing efficiency of ultra-long sequences while ensuring the selectivity of key biometric features.

[0126] It should be further explained that the entire process of active sampling based on the self-attention mechanism is as follows: Figure 8As shown, firstly, a classification model incorporating a self-attention mechanism is trained using a labeled dataset (i.e., the initial protein annotation data). Then, the trained classification model (i.e., the initial protein sequence task classification model) is used to process samples in the unlabeled dataset (i.e., the acquired unlabeled protein data), obtaining the features of each unlabeled sample before and after self-attention correction. These features are then passed through a model classifier to obtain different prediction results. Finally, the difference in prediction results is used to measure the information content of the unlabeled samples (i.e., sample information content score). Samples with greater prediction differences contain richer important information. The information content is ranked, and the top-ranked samples are submitted to human experts for annotation. After annotation, these samples are expanded into the labeled dataset to train the classification model. This method considers data features, aiming to select data samples with poor feature extraction in the task model for annotation, thereby improving the task model's feature extraction ability and indirectly enhancing its generalization ability. Utilizing the self-attention mechanism, an attention layer embedded before the task model's classification layer weights the data features extracted by the task model. The processed features enhance the features that determine the classification result, while weakening other less important features. Therefore, if the prediction results of the features extracted by the task model from the data sample differ significantly before and after attention processing, it indicates that the task classification model has a weak feature extraction ability for that data sample. Selecting this sample can effectively enhance the feature extraction ability of the task model. Furthermore, measuring the value of the data sample at the feature level can alleviate the problem of traditional uncertain active learning over-reliance on the classification performance of the task model, while maintaining high sampling efficiency and not adding additional training costs to the overall task.

[0127] It's worth noting that active learning is being considered in a supervised multi-label classification task. At the start of active learning, the model is on an initial small-scale labeled dataset. Initialize on; where D L This refers to the initial protein annotation data obtained, where x i ∈X represents the input sample data, y i ∈0,1 C Where C is the multi-label independent-hot encoding of the sample, indicating that the sample may belong to multiple categories, i.e., C is the total number of categories. The model simultaneously accesses the unlabeled dataset D. U This involves acquiring unlabeled protein data, selecting b sample instances based on the uncertainty of multi-label prediction, and then having human experts perform full-label annotation, meaning labeling all relevant categories. The newly labeled samples will be added to D. L Used for retraining the model.

[0128] It should be noted that, to ensure the effectiveness of the self-attention mechanism for sequence correction, it is added as a layer before the classifier in the classification model, participating in the training process. Since the biological enzyme protein sequence classification task is a multi-label classification task, a binary cross-entropy loss function is used.

[0129] It should be noted that the sampling strategy first uses a labeled set to train an initial model, giving it some classification performance. Then, the model is used to obtain p1 and p2 respectively; the formulas for calculating p1 and p2 are:

[0130] p1 = FC(GRU(ESM(x) i )));

[0131] p2=FC(Self-Attention(GRU(ESM(x i )))).

[0132] It should be added that x i For unlabeled samples, ESM is a protein sequence feature extractor, a type of recurrent neural network in GRU, used here to further process the features. The following are the prediction results p1 and p2, which are directly output by the linear layer classifier and then output by the classification layer after correction by the self-attention layer.

[0133] It should be noted that the difference in prediction between p1 and p2, calculated according to the sample value evaluation function, is the sample information content score. The sample with the highest score, i.e., the sample with the greatest prediction difference, is selected. First, input the labeled sample set D. L Unlabeled sample set D U The initial model and parameters θ, the number of samples per round b, the number of training arguments Epochs, and then the output of the trained model, i.e., using the labeled set D. L Initialize model parameters, then Forx i ∈D U Use the model to obtain x i Prediction results y after and without the self-attention layer i y i After activation by the sigmoid function (i.e., the S-shaped function), probabilities p1 and p2 are obtained; the value score S(x) of each unlabeled sample is calculated using the sample value evaluation function. i Select the top b samples {x} with the highest value scores. s}, and then have human experts obtain the true labels {y} s}; Update D L and D U D L '←D L ∪{(x s ,y s)},D U '←D U -(x s ,y s ); Use the updated D L Train the model parameters θ; continue repeating the above steps until the model reaches the expected performance or the annotation budget is exhausted. The workflow of the active learning method based on self-attention divergence is as follows: Figure 9 As shown, the data is divided into labeled dataset D. L and D U In active learning, using D L Train the task model, and then, based on the task model and a sampling strategy, measure the divergence of the unlabeled data (i.e., calculate the sample information content score). Select the b samples with the highest divergence and have them manually labeled. Repeat the active learning steps until the pre-set number of active learning iterations is reached.

[0134] It should be noted that the experimental evaluation of this invention was conducted using an NVIDIA A800 GPU and the PyTorch 2.1.2 deep learning framework. After each round of sampling, the AdamW optimizer was used to train the model 10 times with a learning rate of 0.0001 and a batch size of 64. Due to the abnormally imbalanced distribution of the enzyme protein sequence dataset, multiple evaluation metrics were used in this experiment to comprehensively evaluate the performance of the final classification model. If N and P replace 0 and 1, and T and F represent correct and incorrect predictions, then by combining them in pairs, four classification results can be obtained: TP represents successfully predicting a positive sample as positive, TN represents successfully predicting a negative sample as negative, FP represents incorrectly predicting a negative sample as positive, and FN represents incorrectly predicting a positive sample as negative. To evaluate the performance of the algorithm, multiple evaluation metrics were used, including accuracy, precision, and recall. Among them, accuracy is the most common and basic evaluation metric, which refers to the proportion of correctly predicted samples out of the total number of samples. Its formula is as follows:

[0135]

[0136] Accuracy provides a more intuitive representation of a model's learning performance. However, in classification tasks with an imbalance of positive and negative examples, accuracy is particularly unfavorable for the minority class. Even when the classifier misclassifies the minority class as the majority class, the accuracy remains high—this is the accuracy paradox. That is, a model may have high overall accuracy but low recognition rate for the minority class. Therefore, precision and recall are often used together to evaluate a model's classification performance.

[0137]

[0138] Precision refers to the proportion of positive samples that are actually positive, while recall refers to the proportion of correctly predicted positive samples out of the total number of true positive samples. For multi-class classification tasks, evaluation metrics include mACC (macro-average precision), mPR (macro-average precision), mRecall (macro-average recall), and mF1 (macro-average F1 score), with the following formulas:

[0139]

[0140]

[0141] It should be noted that the biological protease dataset was provided by the Institute of Technology, containing 468,891 amino acid sequences of proteases in the training set and 11,477 amino acid sequences in the test set. The training set contains 222,445 enzyme protein sequences and 246,446 non-enzyme protein sequences, while the test set contains 6,627 non-enzyme protein sequences and 4,850 enzyme protein sequences. The total number of reaction categories is 9,887, and for ease of description, it is named Protein1. Based on Protein1, the protein sequences were expanded, increasing the training set to 508,578 protein sequences, including 282,920 non-enzyme protein sequences and 225,667 enzyme protein sequences. The test set was expanded to 13,515 protein sequences, including 10,310 non-enzyme protein sequences and 3,205 enzyme protein sequences. The total number of reaction categories was expanded to 10,478. For ease of description, it is named Protein2.

[0142] It should be noted that the experimental comparisons included predictions of the chemical reactions catalyzed by enzyme proteins based on EC numbers, unsupervised training, supervised training, and APLF. EC numbers are a numbering and classification system for enzymes developed by the Enzyme Commission, based on the chemical reactions catalyzed by each enzyme. This classification system also assigns a suggested name to each enzyme, hence it is also known as the Enzyme Commission nomenclature system. For example, EC1 represents an oxidoreductase. The specific method is as follows... Figure 10First, sequences 1, 2, 3, and 4 are mapped to EC numbers A, B, C, and D, respectively. Then, specific reactions are associated with these EC numbers. Unsupervised training utilizes a protein big language model pre-trained on UniRef50 to extract features from enzyme protein sequences, and then directly calculates similarity using the extracted feature vectors. Supervised training uses ESM as the feature embedding layer, combined with an RNN recurrent neural network for classification. APLF (Active Protein Learning Framework) is used to improve the supervised model using active learning. This experiment compares various methods on the Protein1 dataset, including supervised, unsupervised, and methods that associate reactions catalyzed by proteins based on EC number prediction. The experimental results of predicting enzyme protein-catalyzed chemical reactions based on EC numbers are shown in Table 1 below.

[0143] Table 1 shows chemical reactions catalyzed by enzyme proteins based on EC predictions in Protein 1.

[0144] Method name Accuracy Precision Recall Blast 6.1％ 75.1％ 38.2％ DeepEC 76.2％ 81.7％ 25.3％ CLEAN 7.3％ 24.2％ 79.8％ ECRECer 68.1％ 75.8％ 53.3％ ECPred 42.0％ 95.2％ 6.5％ CatFam 77.0％ 90.2％ 17.1％ PRIMA 1.7％ 18.4％ 79.6％

[0145] DeepEC is a deep learning computational framework that predicts enzyme identifiers (ECs) with high throughput and high accuracy. Clean is a machine learning algorithm that enables enzyme annotation through contrastive learning. ECRECer is a deep learning framework for enzyme function annotation. ECPred is a tool for predicting enzyme function based on the EC nomenclature. CatFam is a novel method called CatFam (catalysis family) for automatically inferring the function of catalytic proteins; it is a sequence-based method that generates sequence profiles to represent and infer protein catalytic functions. PRIMA is a method for automatically detecting enzymes across the entire genome based on enzyme classification in the ENZYME database.

[0146] It should be noted that unsupervised learning utilizes a protein big language model pre-trained on a large-scale protein dataset to extract features from enzyme protein sequences, and then directly calculates Euclidean distance or cosine similarity to classify the enzyme protein sequences. This experiment uses the protein big language models ESM, UniRep, and Prost_T5, as shown in Table 2 below.

[0147] Table 2 Unsupervised learning in Protein 1

[0148]

[0149]

[0150] As shown in Table 2, the direct sequence alignment reaction outperforms the reaction that associates after encoding the EC number. The unsupervised method using protein language models can also significantly improve the performance of downstream applications.

[0151] Supervised learning was employed, using a Protein Large Language Model (ESM, Prost_T5) as the feature extraction layer, combined with an RNN classifier for training. Reaction labels were directly predicted from sequences. Since one enzyme protein corresponds to multiple reactions, this is a multi-label classification task. In this experiment, reaction labels were encoded using independent-hot encoding. For supervised active learning, the dataset was randomly divided into 90,485 data points as the initial labeled dataset. In each round, a designed sampling strategy was used to select the 20,000 most informative data points from the remaining 378,406 unlabeled data points, obtain their true labels, and add them to the labeled dataset. To accelerate the sampling process, half of the entire unlabeled dataset was randomly selected as the unlabeled sample pool for sampling, as shown in Table 3 below.

[0152] Table 3 Active learning in Protein 1

[0153] Method name Accuracy Precision Recall Baseline method 61.7％ 82.7％ 75.8％ ours 81.1％ 86.6％ 96.5％

[0154] The results above show that the classification model trained using supervised active learning exhibits the best performance in accuracy, precision, and recall. Compared to the baseline method using non-active learning, the classification performance improvement achieved by selecting the most informative samples through sampling strategies is greater than that achieved by training with all samples. There are two main reasons for this: First, in the enzyme protein dataset, the class distribution varies greatly. For example, non-enzyme protein data accounts for 52% of the entire dataset, while some enzyme proteins, particularly a minority of classes, account for only 1% or even fewer than 5 data points in the entire training set. Without active learning, training the classification model on the entire dataset would easily lead to overfitting, resulting in misclassification of the minority classes. Active learning, however, samples only 20,000 highly informative samples in each round, avoiding a severe imbalance between the majority and minority classes. Second, in protein sequence data, redundant information constitutes a large proportion. In enzyme protein sequences, the key active site residues often determine the enzymatic reaction, while other parts do not directly participate in the enzymatic reaction. Attention mechanisms can weaken redundant data information and enhance the data information that determines the classification result. By measuring the value of samples before and after correction by attention mechanisms, we can select those samples that the model itself overemphasizes redundant information. Using these samples, we can enhance the model's feature extraction ability and thus improve the model's generalization performance.

[0155] It should be noted that experimental validation was conducted on a biological enzyme protein dataset. The results demonstrate that for protein sequence data with severely imbalanced class distributions and a large amount of redundant information, active learning based on self-attention divergence significantly improves the generalization performance of the classification model. This effectively addresses the inherent shortcomings of traditional methods in terms of annotation efficiency, model generalization ability, and computational resource consumption. Furthermore, while maintaining linear time complexity, it significantly alleviates the problem of traditional active learning methods over-relying on task model classification performance, providing a feasible solution for rapid annotation of large-scale sequence data. In the enzyme protein function classification task, the self-attention divergence strategy improved the classification accuracy to 81.1%, far exceeding traditional methods based on EC and sequence similarity for predicting enzyme protein function.

[0156] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores static and dynamic information data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps described in the embodiments of the protein function mining method based on attention mechanisms and active learning.

[0157] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0158] Furthermore, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiments of the protein function mining method based on attention mechanism and active learning.

[0159] In addition, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described embodiments of the protein function mining method based on attention mechanisms and active learning.

[0160] Those skilled in the art will understand that implementing all or part of the processes in the protein function mining methods based on attention mechanisms and active learning described in the above embodiments can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the protein function mining methods based on attention mechanisms and active learning described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0161] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. A protein function mining method based on attention mechanisms and active learning, characterized in that, include: The initial protein sequence task classification model is obtained by initially training the acquired initial protein annotation data using a pre-built protein sequence task classification model. Unlabeled protein data is obtained, and the data is sorted by labeling level using the initial protein sequence task classification model. The unlabeled protein data is then filtered based on the labeling level ranking results to obtain the filtered unlabeled protein data. The unlabeled protein data selected from the screening process is integrated with the initial labeled protein data to obtain updated labeled protein data. The updated labeled protein data is then iteratively trained using the initial protein sequence task classification model to obtain the final protein sequence task classification model. Based on the final protein sequence task classification model, protein function mining is performed on the acquired protein sequence data to obtain protein function mining results.

2. The protein function mining method based on attention mechanism and active learning according to claim 1, characterized in that, The initial protein sequence task classification model is obtained by initially training the acquired initial protein annotation data using a pre-built protein sequence task classification model, including: The pre-built protein sequence task classification model is initialized based on the acquired initial protein annotation data to obtain the initialized protein sequence task classification model. The initial protein sequence task classification model was used to make preliminary predictions on the initial protein annotation data, and preliminary prediction results were obtained. Based on the preliminary prediction results, the initial protein sequence task classification model was optimized using the binary cross-entropy loss function to obtain the initial protein sequence task classification model.

3. The protein function mining method based on attention mechanism and active learning according to claim 2, characterized in that, The expression for the binary cross-entropy loss function is: In the formula, BCE Loss N represents the number of groups of objects predicted by the model; y i Indicates a binary label 0 or 1; p(y i ) indicates the probability of outputting a label.

4. The protein function mining method based on attention mechanism and active learning according to claim 1, characterized in that, The process involves acquiring unlabeled protein data, ranking the data by annotation level using an initial protein sequence task classification model, obtaining a ranking result, and then filtering the unlabeled protein data based on the ranking result. The filtered unlabeled protein data includes: The initial protein sequence task classification model is used to perform initial classification prediction on the acquired unlabeled protein data, and the self-attention mechanism is bypassed to perform secondary classification prediction on the unlabeled protein data. The initial protein sequence task classification model includes: a protein language model, a recurrent neural network, a self-attention mechanism, and a classifier; The initial classification prediction results and the secondary classification prediction results are probabilistically mapped based on the sigmoid function, and the sample information content score is calculated based on the probability mapping results. The unlabeled protein data are sorted based on the sample information content score to obtain the data labeling level ranking results. The unlabeled protein data were filtered and labeled based on the data labeling level sorting results, resulting in the filtered unlabeled protein data.

5. The protein function mining method based on attention mechanism and active learning according to claim 4, characterized in that, The process of using an initial protein sequence task classification model to perform initial classification prediction on the acquired unlabeled protein data, and then bypassing the self-attention mechanism to perform secondary classification prediction on the unlabeled protein data, includes: Protein sequence features were extracted from the acquired unlabeled protein data using a protein language model, and the protein sequence feature extraction results were obtained. Based on the protein sequence feature extraction results, a recurrent neural network is used for gradient processing to obtain the processed protein sequence feature extraction results. The self-attention mechanism is used to correct the features of the extracted protein sequence, and the protein sequence features are corrected. Based on the protein sequence features corrected, a classifier is used to make an initial classification prediction, and the initial classification prediction result is obtained. The initial protein sequence task classification model bypasses the self-attention mechanism to perform secondary classification prediction on unlabeled protein data, resulting in secondary classification prediction results.

6. The protein function mining method based on attention mechanism and active learning according to claim 5, characterized in that, The protein sequence feature extraction process, which utilizes a protein language model to extract protein sequence features from the acquired unlabeled protein data, yields the following results: Relative position encoding is introduced into the protein language model, and position information is encoded in the acquired unlabeled protein data to obtain amino acid residue data; The amino acid residue data is masked using a masking language model, and context prediction is performed based on the masking results to obtain the context prediction results. Based on the context prediction results, co-evolutionary signals between residues and global structural constraints are learned to obtain preliminary protein sequence features. Then, the preliminary protein sequence features are used to generate folded backbones and three-dimensional structures to obtain protein sequence feature extraction results.

7. The protein function mining method based on attention mechanism and active learning according to claim 5, characterized in that, The process involves using a self-attention mechanism to correct the extracted protein sequence features, resulting in corrected protein sequence features. Based on these corrected features, a classifier is used for initial classification prediction, yielding the following initial classification prediction results: The processed protein sequence feature extraction results are projected into the latent space using the weight matrix and bias of the self-attention mechanism to obtain the protein sequence feature projection results. The protein sequence feature projection results are scalar scored using the hidden state of the parameter vector, and the scalar scoring results are normalized to obtain normalized scalar scoring results. The normalized scalar scoring results are then filled with noise and suppressed using a dynamic masking mechanism to obtain attention weights. The processed protein sequence feature extraction results are weighted and summed according to the attention weights to obtain the weighted sum result. The weighted sum result is then dimensionally compressed to obtain the protein sequence feature correction result. The protein sequence feature correction results are used to perform initial classification prediction using a classifier to obtain the initial classification prediction results.

8. The protein function mining method based on attention mechanism and active learning according to claim 4, characterized in that, The method involves performing a probability mapping between the initial and secondary classification prediction results based on the sigmoid function, calculating the sample information content score based on the probability mapping result, and sorting the unlabeled protein data based on the sample information content score to obtain the data labeling level ranking result, including: The initial classification prediction is mapped using the sigmoid function to obtain the initial classification probability, and the secondary classification prediction result is mapped using the probability to obtain the secondary classification prediction probability. The initial classification probability and the secondary classification prediction probability are input into the sample value evaluation function, and the sample information score is calculated through the sample value evaluation function. Based on the numerical value of the sample information content score, the unlabeled protein data are sorted according to preset rules to obtain the data labeling level ranking results.

9. The protein function mining method based on attention mechanism and active learning according to claim 8, characterized in that, The expression for the sample value assessment function is: In the formula, S(x) represents the sample value evaluation function; C represents the total number of categories; p1 represents the probability of secondary classification; p2 represents the probability of primary classification prediction; y represents the prediction result; and x represents the input sample.

10. A protein function mining system based on attention mechanisms and active learning, characterized in that, The protein function mining system based on attention mechanism and active learning includes: a classification model initial training module, a labeled data screening module, an iterative training module, and a protein function mining module; The initial training module of the classification model is used to perform initial training on the acquired initial protein annotation data using a pre-built protein sequence task classification model to obtain an initial protein sequence task classification model. The labeled data filtering module is used to obtain unlabeled protein data, sort the data labeling levels using the initial protein sequence task classification model, obtain the data labeling level sorting results, and filter the unlabeled protein data according to the data labeling level sorting results to obtain the filtered unlabeled protein data. The iterative training module is used to integrate the selected unlabeled protein data with the initial labeled protein data to obtain updated labeled protein data, and to iteratively train the updated labeled protein data using the initial protein sequence task classification model to obtain the final protein sequence task classification model. The protein function mining module is used to perform protein function mining on the acquired protein sequence data based on the final protein sequence task classification model, and obtain the protein function mining results.

Images