Microbial strain screening method based on self-distillation deep learning

Abstract

The invention relates to a microbial strain screening method based on self-distillation deep learning, and belongs to the crossing field of synthetic biology and artificial intelligence. According to the method, lightweight self-distillation network modeling is carried out on DNA sequences of microbial strains, the microbial strains with high protein expression quantity are efficiently screened, and a screening tool is provided for cost and mass balanced genotype-phenotype design. The method comprises the following steps: carrying out One-hot two-dimensional matrix coding and high-dimensional feature extraction on a DNA sequence of a microbial strain; constructing a super network topology architecture by using the encoded DNA sequence, and performing architecture weight optimization and sampling according to the distillability of the current dominant architecture; in order to simplify a super network structure and reduce calculation complexity, a filtering module is introduced for global optimization and candidate operation filtering, so that a potential architecture space is compressed; finally, strategy knowledge distillation is carried out on the compressed network architecture and the network architecture of the previous generation, so that knowledge in the architecture of the previous generation is migrated to the current architecture, and the prediction precision and efficiency of the protein expression quantity of the microbial strain are improved. According to the technical scheme, the problems of protein expression quantity prediction and microbial strain DNA prototype design and screening can be solved by effectively utilizing a lightweight deep learning model, and the screening cost is effectively reduced compared with a traditional genetic engineering method.

Description

Technical Field

[0001] The present invention relates to the fields of biotechnology and artificial intelligence, and in particular to a method for predicting protein expression levels of microbial strains based on lightweight deep learning. Background Art

[0002] Synthetic biology, one of the fastest-growing areas in biotechnology, aims to engineer microbial strains capable of efficiently expressing high-value proteins through the design and modification of biological systems. Modifying host cells to achieve high-yield, efficient expression of target proteins is a key step in the production process. Accurately predicting protein expression levels in strains and enabling efficient screening based on these levels have been key challenges hindering industrialization.

[0003] Traditional strain design and optimization relies heavily on researcher experience. Faced with the vast gene sequence space, this often requires the construction and validation of a vast number of mutation models. This process is time-consuming, labor-intensive, and resource-inefficient, making it difficult to meet the urgent needs of developing high-throughput strain screening and expression systems.

[0004] With the rapid advancement of DNA synthesis and sequencing technologies, the application of high-throughput technologies such as deep mutation scanning (DMS) has made it possible to systematically obtain data linking gene mutations with various key biological phenotypes, such as protein production, ribosome binding efficiency, and translational status. In recent years, machine learning (ML), particularly deep learning (DL), based on such data, has demonstrated great potential in predicting the function of regulatory elements (such as ribosome binding sites (RBSs), promoter strength, and RNA secondary structure) and ultimately protein expression phenotypes. Deep learning, with its powerful nonlinear modeling capabilities, can automatically capture the highly nonlinear dependencies between sequence features and complex phenotypes. It can uncover underlying patterns from massive amounts of data without requiring extensive prior assumptions, significantly improving prediction accuracy and providing a new path for intelligent strain screening within the "design-build-test-learn" (DBTL) cycle of synthetic biology.

[0005] However, the effective application of deep learning to large-scale microbial strain screening still faces severe challenges: 1. The contradiction between model complexity and computational cost: Expanding the network size in pursuit of higher prediction accuracy will lead to an exponential increase in training time. The huge consumption of computing resources significantly increases the application threshold and experimental costs. 2. Data efficiency bottleneck: Obtaining high-quality DNA sequence data with accurate phenotypic labels (especially protein production) is expensive and limited in scale. Complex, large-scale deep learning models are prone to overfitting in such scenarios with small samples (or limited labeled data), resulting in reduced generalization ability and difficulty in steadily improving prediction accuracy. Prediction accuracy may even decline due to excessive model complexity. 3. Lightweight Requirements and Accuracy Guarantee: Developing lightweight deep learning models that can be trained efficiently with limited labeled data while maintaining high prediction accuracy has become crucial for improving the efficiency and success rate of microbial strain screening. Knowledge distillation, as a model compression and knowledge transfer technique, offers a solution to this problem. However, it typically relies on complex pre-trained teacher models, and obtaining high-performance teacher models in scenarios with limited biological data is inherently challenging. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for screening microbial strains based on self-distillation deep learning. This method uses lightweight self-distillation network modeling of the DNA sequences of microbial strains to efficiently screen microbial strains with high protein expression, providing a screening tool for selecting microbial strains with high protein expression. By using a deep learning method combined with self-distillation network architecture optimization to capture the high-dimensional features of DNA sequences, the present invention reduces the complexity of the deep network, enhances the generalization ability of the model, and significantly improves the efficiency of microbial strain selection.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions: S1. Matrix Representation and Feature Extraction of DNA Sequences. The DNA sequence of the target microbial strain is converted into a binary two-dimensional matrix representation. One-hot encoding is used to map each nucleotide base (A, T, C, G) into a one-hot vector, which is then stacked along the sequence to form an initial two-dimensional feature map. The feature map is further abstracted using a Residual Convolutional Neural Network (Res-CNN) to generate a high-dimensional semantic feature vector, which serves as the input for the deep learning model. S2. Initialize and jointly optimize the self-distillation network architecture space. Based on the high-dimensional feature vector generated in step S1, initialize the architecture search space of the lightweight self-distillation deep learning network. By constructing a two-level optimization problem for network architecture parameters (such as layer types, connection methods, hyperparameters) and network connection weight parameters, the network architecture structure and model weight parameters are optimized simultaneously in an iterative manner. S3. Dynamic evaluation of network architecture distillability. In each iteration of step S2, an architecture distillability evaluation function is established to quantitatively evaluate the adaptability of the current network architecture to a given DNA sequence data sample from the following dimensions: (a) Architecture similarity: Calculate the topological similarity between the current architecture and the parent architecture; (b) Feature clustering ability: evaluates the degree of cluster separation of high / low expression phenotype samples in the latent space by the features extracted by the current architecture. S4: Real-time compression and acceleration of the architecture space. Based on the distillability evaluation results obtained in step S3, the potential network architecture space is dynamically pruned and compressed: highly adaptable architecture branches are retained, while inefficient and redundant structures are eliminated. This process significantly accelerates architecture search efficiency while reducing network complexity. S5, cross-generational knowledge distillation and phenotype prediction. The streamlined network architecture compressed in step S4 is used as the student model, and its previous generation complete architecture is used as the teacher model. Through a policy-based knowledge transfer mechanism (including soft label distillation and feature map alignment), the high-level knowledge in the teacher model is distilled into the student model. Finally, the student model outputs the predicted protein expression level of the target strain.

[0008] Furthermore, a preferred implementation method is provided, wherein step S1 comprises: S1.1. Use one-hot encoding to map adenine A, thymine T, cytosine C, and guanine G in the DNA sequence into one-hot vectors, which are stacked along the sequence direction to form an initial DNA two-dimensional feature map with 4 rows and n columns, where n is the length of the DNA sequence intercepted in the experiment. S1.2. A residual convolutional network (Res-CNN) is used to process the DNA two-dimensional feature map (x). The residual network consists of the following modules: the first convolutional layer uses a small convolution kernel (3×3) to extract local features of the sequence; batch normalization then standardizes the convolution output to speed up training and reduce sensitivity to initial parameters; ReLU is then used for nonlinear activation to help the model better handle complex patterns; a second convolutional layer is then stacked to further process features and enhance deep expression capabilities; ReLU is used for nonlinear feature activation; a dropout layer is then added to prevent overfitting of the model; a jump connection is used to pass the input initial DNA two-dimensional feature map (x) to the output F(x) of the dropout layer. Finally, F(x) + x is used to generate a high-dimensional semantic feature vector output of the DNA sequence, which serves as the input for step S2.

[0009] The step S2 is specifically as follows: S2.1. Construct a hypernetwork containing eight candidate operations as the initial network architecture: no connection, 3×3 max pooling, 3×3 average pooling, residual connection, 3×3 separable convolution, 5×5 separable convolution, 3×3 dilated convolution, and 5×5 dilated convolution. These candidate operations are iteratively updated by minimizing the loss function, ultimately generating a self-distillation network architecture. S2.2. Construct the loss function for the two-layer optimization problem of network architecture parameters and network connection weight parameters: in , represent the training loss and validation loss respectively, Represents the optimized network connection weight parameters; the optimization process can be converted to minimize the validation loss To find the best architectural parameters , then fix the architecture parameters and minimize the training loss To get the network connection weight parameters corresponding to the architecture ; When optimizing architecture parameters, use the network connection weight parameters of the previous iteration , and when optimizing the network connection weight parameters, the architecture parameters of the previous iteration are also used .

[0010] The step S3 is specifically as follows: S3.1: Build architecture similarity metrics using attention feature maps and prediction distribution maps ( ), which quantifies the degree of difference between the current architecture and its predecessor, is calculated as: in is the attention map of the current network, is the attention map of the previous network, is the prediction distribution matrix of the current network, is the prediction distribution matrix of the previous network, and N is the number of DNA two-dimensional feature maps. S3.2: Use the DAN sequence prediction distribution map and the true label distribution map to establish the clustering ability index ( ), evaluates the adaptability of the current architecture to a given data sample, and its calculation expression is: in is the label value distribution matrix. The above indicators are input into the loss function of step S2, and the architecture parameters are continuously optimized and improved until convergence, resulting in the optimal sub-network architecture.

[0011] The step S4 is specifically as follows: Dynamically prune and compress the potential network architecture space, that is, delete or adjust each candidate operation of the network architecture parameters. Introduce a mask matrix To filter the weight of each candidate operation of the network architecture parameters. After obtaining the evaluation output, adjust their respective weights according to the impact of each candidate operation on the distillable index in the existing architecture. The calculation formula for each element of the mask matrix is: in is the parameter weight of the j-th candidate operation in the i-th neural network unit, is the average weight of all candidate operations of the ith neural network unit, It is the threshold used to determine whether to compress or filter network architecture parameters.

[0012] The step S5 is specifically as follows: Knowledge distillation is used to transfer knowledge from the teacher network (the previous generation architecture) to the student network, which has been compressed in step S4. Specifically, a distillation branch is added to each unit output node in the network to serve as a knowledge source. During training, the student network is guided by supervisory signals generated by the teacher network, bringing its performance closer to that of the teacher network. The primary sources of knowledge transfer include the intermediate layer feature maps of the teacher network and its final output predictions. These supervisory signals are compared with the outputs of the corresponding layers of the student network to calculate an additional distillation loss term. This distillation loss term is combined with the original loss function to optimize the student network training process.

[0013] The beneficial effects of the present invention are: 1. DNA Sequence Two-Dimensional Feature Map: By using a matrix representation of strain DNA sequences and a residual CNN, this method can learn the high-dimensional features of the DNA sequence two-dimensional feature map layer by layer through stacking convolutional layers. Compared to traditional machine learning, the skip connections of residual deep networks overcome the difficulties of training deep networks, making it feasible to build and train networks with hundreds or thousands of layers, and capable of learning more complex, abstract, and discriminative features of DNA sequences. 2. Self-distillation deep network architecture optimization: This framework can dynamically adjust the network architecture based on the distillability of the current architecture, and simultaneously guide the optimization direction of the deep network architecture from the perspectives of architectural similarity and clustering ability, thereby more efficiently predicting the protein expression level of the strain and saving the time and cost of strain screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 Flowchart of a method for screening microbial strains based on self-distillation deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0017] like Figure 1 As shown in the embodiment of the present application, an embodiment of a microbial strain screening method based on self-distillation deep learning is shown, and the specific steps are as follows: S1. The present invention uses one-hot encoding to map each nucleotide base (A, T, C, G) in the DNA sequence of the target microbial strain into a one-hot vector, which is stacked along the sequence direction to form an initial two-dimensional feature map. The feature map is further abstracted through a residual convolutional neural network to generate a high-dimensional semantic feature vector as the input of the deep learning model. S2. Initialize the architecture search space of the lightweight self-distillation deep learning network based on the high-dimensional feature vector generated in step S1; by constructing a two-layer optimization problem of network architecture parameters and network connection weight parameters, iteratively optimize the network architecture structure and model weight parameters simultaneously; S3. The present invention implements dynamic evaluation using network architecture distillability. In each iteration of step S2, an architecture distillability evaluation function is established to quantitatively evaluate the adaptability of the current network architecture to a given DNA sequence data sample based on architecture similarity and feature clustering ability. S4. The present invention implements real-time compression and acceleration of the architecture space. Based on the distillability evaluation results obtained in step S3, the potential network architecture space is dynamically pruned and compressed: highly adaptable architecture branches are retained, and inefficient redundant structures are eliminated; S5. The present invention implements cross-generational knowledge distillation and phenotype prediction. The streamlined network architecture compressed in step S4 serves as the student model, and its previous generation complete architecture serves as the teacher model. Through a policy-based knowledge transfer mechanism (including soft label distillation and feature map alignment), high-level knowledge from the teacher model is distilled into the student model. Ultimately, the student model outputs predicted protein expression levels for the target strain.

[0018] Furthermore, a preferred implementation method is provided, wherein step S1 comprises: S1.1. Use one-hot encoding to map adenine A, thymine T, cytosine C, and guanine G in the DNA sequence into one-hot vectors, which are stacked along the sequence direction to form an initial DNA two-dimensional feature map with 4 rows and n columns, where n is the length of the DNA sequence intercepted in the experiment. S1.2. A residual convolutional network (Res-CNN) is used to process the DNA two-dimensional feature map (x). The residual network consists of the following modules: the first convolutional layer uses a small convolution kernel (3×3) to extract local features of the sequence; batch normalization then standardizes the convolution output to speed up training and reduce sensitivity to initial parameters; ReLU is then used for nonlinear activation to help the model better handle complex patterns; a second convolutional layer is then stacked to further process features and enhance deep expression capabilities; ReLU is used for nonlinear feature activation; a dropout layer is then added to prevent overfitting of the model; a jump connection is used to pass the input initial DNA two-dimensional feature map (x) to the output F(x) of the dropout layer. Finally, F(x) + x is used to generate a high-dimensional semantic feature vector output of the DNA sequence, which serves as the input for step S2.

[0019] The step S2 is specifically as follows: S2.1. Construct a hypernetwork containing eight candidate operations as the initial network architecture: no connection, 3×3 max pooling, 3×3 average pooling, residual connection, 3×3 separable convolution, 5×5 separable convolution, 3×3 dilated convolution, and 5×5 dilated convolution. These candidate operations are iteratively updated by minimizing the loss function, ultimately generating a self-distillation network architecture. S2.2. Construct the loss function for the two-layer optimization problem of network architecture parameters and network connection weight parameters: in , represent the training loss and validation loss respectively, Represents the optimized network connection weight parameters; the optimization process can be converted to minimize the validation loss To find the best architectural parameters , then fix the architecture parameters and minimize the training loss To get the network connection weight parameters corresponding to the architecture ; When optimizing architecture parameters, use the network connection weight parameters of the previous iteration , and when optimizing the network connection weight parameters, the architecture parameters of the previous iteration are also used .

[0020] The step S3 is specifically as follows: S3.1: Build architecture similarity metrics using attention feature maps and prediction distribution maps ( ), which quantifies the degree of difference between the current architecture and its predecessor, is calculated as: wherein is the attention map of the current network, is the attention map of the previous network, is the prediction distribution matrix of the current network, is the prediction distribution matrix of the previous network, and N is the number of DNA two-dimensional feature maps. S3.2: A clustering capability index is established by using the DAN sequence prediction distribution map and the real label distribution map (C), which evaluates the adaptability of the current architecture to the given data sample, and the calculation expression is: wherein is the label value distribution matrix. The above index inputs the loss function of step S2, and iteratively optimizes and improves the architecture parameters until convergence, and finally obtains the optimal sub-network architecture.

[0021] The step S4 is specifically: Dynamic pruning and compression of the potential network architecture space, i.e. deleting or adjusting each candidate operation of the network architecture parameters. A mask matrix is introduced to filter each candidate operation weight of the network architecture parameters. After obtaining the evaluation output, the weight of each candidate operation in the existing architecture is adjusted according to its influence on the distillable index. The mask matrix The calculation formula is: wherein is the parameter weight of the jth candidate operation in the ith neural network unit, is the average weight of all candidate operations of the ith neural network unit, is a threshold value for whether the network architecture parameters need to be compressed or filtered.

[0022] The step S5 is specifically: The knowledge distillation technology is used to migrate the knowledge of the teacher network (the previous generation architecture) to the student network compressed by step S4. Specifically, a distillation branch is added at each unit output node in the network as a knowledge source. During the training process, the student network is guided by the supervision signal generated by the teacher network to make its performance approach the teacher network. The main sources of knowledge migration include the intermediate layer feature maps and the final output prediction of the teacher network. These supervision signals are compared with the outputs of the corresponding layers of the student network to calculate an additional distillation loss term. The distillation loss term is combined with the original loss function to optimize the training process of the student network. Finally, the protein expression level of the microbial strain is predicted by the distilled student network.

[0023] ​The above describes the embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. After knowing the contents described in the present invention, ordinary technicians in this technical field can make several equivalent changes and substitutions without departing from the principles of the present invention. These equivalent changes and substitutions should also be regarded as falling within the scope of protection of the present invention.

Claims

1. A method for screening microbial strains based on self-distillation deep learning, characterized in that: The following steps are involved: S1: The strain DNA sequence is represented as a two-dimensional matrix. One-hot encoding is used to convert each nucleotide (A, T, C, G) into a binary-encoded two-dimensional matrix. The residual convolutional neural network is then used to represent the matrix into a high-dimensional feature map that can be understood by the deep learning model. S2: Read the feature map of step S1, initialize the architecture space of the self-distillation deep learning network, and iteratively optimize the architecture and network parameters by constructing a two-layer optimization problem of network architecture parameters and connection weight parameters; S3: Establish an evaluation function for the distillability of the network architecture to evaluate the adaptability of the current architecture in step S2 to a given DNA sequence sample, and perform a quantitative evaluation from two dimensions: architecture similarity and clustering ability; S4: During each iteration of step S2, the distillation performance of the current architecture is evaluated and the potential network architecture space is optimized and compressed based on the evaluation results to speed up the overall architecture training and reduce network complexity. S5: The compressed network architecture performs strategic knowledge distillation with the previous generation network architecture, thereby migrating the knowledge in the previous generation architecture to the current architecture, and finally outputs the predicted value of the strain protein expression level.

2. The method for screening microbial strains based on self-distillation deep learning according to claim 1, characterized in that: In step S1, the DNA sequence is expressed as a binary two-dimensional matrix based on one-hot encoding; the DNA sequence high-dimensional feature extraction network includes a convolutional layer, a batch normalization layer, a dropout layer, a pooling layer, a skip connection and a ReLU activation function. Step S1 is specifically as follows: S1-1: After the input strain DNA sequence is one-hot encoded, a small-size convolution kernel is used to extract local features of the sequence. Batch normalization then standardizes the convolution output, thereby speeding up training and reducing sensitivity to initial parameters. Finally, ReLU is used for nonlinear activation, enabling the deep network to learn complex features. S1-2: The second convolutional layer further processes features and enhances the deep expression capability; the dropout layer is introduced, which randomly disables some neurons during training, thereby reducing the co-adaptation relationship between neurons and enhancing the generalization ability of the network; S1-3: Skip connection adds the input 𝑥 directly to the residual function On, the output ; Then nonlinear activation is performed through ReLU.

3. The method for screening microbial strains based on self-distillation deep learning according to claim 1, characterized in that: The optimization problem of the self-distillation deep learning network architecture described in step S2 is expressed as a two-layer optimization problem of network architecture parameters and connection weight parameters. The optimal deep network architecture can be obtained by randomizing initial parameters and heuristic search iterations; Step S2 is specifically as follows: S2-1: By introducing an architecture space containing candidate operations such as convolution, maximum pooling, average pooling, separable convolution, and dilated convolution, a hypernetwork is constructed as the initial network architecture; S2-2: Initialize the hypernetwork, and set the weight of the connection candidate operation between each node to a random value; S2-3: Construct a two-layer optimization loss function. The upper layer optimizes the architectural parameters of the candidate weights, and the lower layer optimizes the parameters of the network neuron connection weights. An evolutionary algorithm is used to iteratively optimize the upper and lower layers to obtain the optimal network architecture.

4. The method for screening microbial strains based on self-distillation deep learning according to claim 1, characterized in that: The evaluation index of the network architecture distillability described in step S3 consists of two parts: architecture similarity and clustering ability; Step S3 is specifically as follows: S3-1: Use attention feature maps and prediction distribution maps to build an architecture similarity metric to quantify the degree of difference between the current architecture and its predecessor; S3-2: Use the DAN sequence prediction distribution map and the true label distribution map to establish a clustering ability indicator to evaluate the adaptability of the current architecture to a given data sample.

5. The method for screening microbial strains based on self-distillation deep learning according to claim 1, characterized in that: In step S4, in order to simplify the hypernetwork architecture and speed up the construction process, the architecture space is compressed while optimizing the architecture parameters; Step S4 specifically involves introducing a two-dimensional mask matrix to filter the weight of each candidate operation. After obtaining the evaluation output, the weight of each candidate operation is adjusted according to the impact of each candidate operation on the distillable evaluation index in the existing architecture, thereby compressing the architecture space.

Images