Single-cell RNA sequencing data classification method and system based on deep learning

Abstract

The present invention discloses a single-cell RNA sequencing data classification method and system based on deep learning. The present invention proposes a multi-scale neural network based on an improved Transformer for classifying preprocessed single-cell RNA sequencing data. The multi-scale neural network integrates a multi-scale attention mechanism, relative position encoding, and a gated feedforward network, which can not only capture the multi-level dependencies of the data, but also effectively reduce noise interference through feature screening. Compared with traditional convolutional neural networks or classic Transformers, the multi-scale neural network proposed in the present invention shows stronger robustness and generalization ability in sequence modeling and classification tasks. Through extensive testing on multiple public data sets, the proposed multi-scale neural network shows better performance in cell category classification tasks.

Description

Technical Field

[0001] The present invention relates to a classification method for single-cell RNA sequencing data in the field of bioinformatics, and in particular to a single-cell RNA sequencing data classification method and system based on deep learning. Background Art

[0002] Single-cell RNA sequencing technology (scRNA-seq) allows researchers to perform high-throughput analysis of gene expression at the single-cell level, revealing cell heterogeneity and changes in cell states in complex biological processes. With the advancement of sequencing technology, the scale and complexity of single-cell RNA sequencing data continue to increase, providing unprecedented data resources for biomedical research.

[0003] However, the processing and analysis of single-cell RNA sequencing data face many challenges, especially when dealing with complex biological data. First, single-cell RNA sequencing data are usually extremely high-dimensional and sparse. The gene expression data of a single cell often involves thousands of genes, while the expression profile of each cell usually has only a few genes with non-zero values, which leads to a large number of zero or missing values ​​in the data. At the same time, the high noise of the data also makes it more difficult to extract useful biological information from the raw data. In order to solve these problems, new methods need to be proposed to effectively process these high-dimensional sparse data and reduce the interference of noise on the analysis results.

[0004] Secondly, single-cell RNA sequencing data is also subject to interference from factors such as batch effects, technical variation, and biological variation. Batch effects refer to systematic differences between different experimental batches, which may lead to different sequencing results for the same biological sample in different experiments. Technical variation refers to random errors generated during the experiment, while biological variation stems from natural differences between different cells in the sample. These factors may affect the quality of the data, thereby reducing the accuracy of the classification results. Therefore, in the process of data analysis, how to effectively remove or reduce these interferences and ensure the reliability and stability of the analysis results has become a major challenge in single-cell RNA sequencing data processing.

[0005] At present, common methods for processing single-cell RNA sequencing data include Seurat, Scanpy, etc. These methods provide a series of analysis processes such as data preprocessing, standardization, dimensionality reduction, classification and visualization. Although these tools have promoted the progress of single-cell data analysis to a certain extent, they still have limitations when processing large-scale and complex data sets. Summary of the invention

[0006] In order to solve the problem of complex data processing caused by high dimensionality, high sparsity and complex feature patterns in single-cell RNA sequencing data analysis, the present invention proposes a single-cell RNA sequencing data classification method and system based on deep learning. The present invention proposes an improved Transformer model, which captures the multi-level feature relationship of gene expression data through a multi-scale attention mechanism, introduces relative position encoding to enhance the modeling ability of sequence data, and combines the gated feedforward network (Gated FFN) to optimize feature screening and expression. The present invention has made systematic improvements in both feature learning and sequence modeling, thereby achieving more efficient, robust and accurate results in the classification task of single-cell RNA sequencing data.

[0007] The technical solution of the present invention is as follows:

[0008] 1. A single-cell RNA sequencing data classification method based on deep learning

[0009] Step 1: After preprocessing the single-cell RNA sequencing data, obtain the training data set;

[0010] Step 2: Build a multi-scale neural network based on the improved Transformer, and use the training data set to train the multi-scale neural network based on the improved Transformer to obtain the RNA sequencing data classification model;

[0011] Step 3: After preprocessing, the single-cell RNA sequencing data to be tested is input into the RNA sequencing data classification model, and the model outputs the corresponding classification results.

[0012] The preprocessing includes data cleaning and standardization.

[0013] The multi-scale neural network based on the improved Transformer includes a first linear layer, an improved Transformer model, an average pooling layer, a first normalization layer and a second linear layer connected in sequence. The input of the multi-scale neural network based on the improved Transformer serves as the input of the first linear layer, and the output of the second linear layer serves as the output of the multi-scale neural network based on the improved Transformer.

[0014] The improved Transformer model includes an encoder and a gated feedforward network. The input of the improved Transformer model is used as the input of the encoder. The output of the encoder is added to the learnable position code and then input into the gated feedforward network. The output of the gated feedforward network is used as the output of the improved Transformer model.

[0015] The encoder includes a multi-scale multi-head attention mechanism, a random dropout layer and a second normalization layer. Multiple input sequences of different scales are generated according to the input of the encoder, and then the input sequence of each scale is used as the input of the multi-head attention mechanism of the corresponding scale. The input of the encoder is then spliced ​​with the output of the multi-head attention mechanism of all scales and then input into the random dropout layer. The random dropout layer is connected to the second normalization layer, and the output of the second normalization layer is used as the output of the encoder.

[0016] The gated feedforward network is used to perform nonlinear processing on the input of the gated feedforward network.

[0017] 2. A single-cell RNA sequencing data classification system based on deep learning

[0018] A data preprocessing unit, used to preprocess single-cell RNA sequencing data;

[0019] A training data set generating unit, used to generate a training data set according to the preprocessed data;

[0020] A data classification unit, used to store the multi-scale neural network based on the improved Transformer and the corresponding network parameters, and to classify the input pre-processed sequencing data using the multi-scale neural network based on the improved Transformer, and output the corresponding classification results;

[0021] The network training unit is used to train the multi-scale neural network based on the improved Transformer using the training data set until the training is completed and the final network parameters are sent to the data classification unit.

[0022] 3. A computer device

[0023] The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the single-cell RNA sequencing data classification method based on deep learning when executing the computer program.

[0024] 4. A computer-readable storage medium

[0025] The storage medium stores a computer program, which, when executed by a processor, implements the steps of the single-cell RNA sequencing data classification method based on deep learning.

[0026] 5. A computer program product

[0027] The product includes a computer program / instruction, which, when executed by a processor, implements the steps of the single-cell RNA sequencing data classification method based on deep learning.

[0028] This paper proposes a multi-scale neural network based on an improved Transformer, which achieves efficient classification of single-cell gene expression data. The multi-scale multi-head attention mechanism in the multi-scale neural network improves the feature modeling capability and data discriminability, and combines the gated feedforward network (Gated FFN) to optimize feature screening and expression, thereby significantly improving feature learning and sequence modeling capabilities. Experiments on multiple public data sets show that the method proposed in this paper shows good performance in cell classification tasks and is superior to existing technologies in terms of evaluation indicators.

[0029] The beneficial effects of the present invention are:

[0030] 1. The present invention uses a multi-scale multi-head attention mechanism to capture the cross-scale characteristics of gene expression patterns, significantly improving feature modeling capabilities and data discriminability.

[0031] 2. The improved Transformer model proposed in the present invention includes relative position encoding and gated feedforward network (Gated FFN), which optimizes the feature extraction efficiency and enhances the adaptability of the model to sparse sequence data.

[0032] 3. The multi-scale neural network based on the improved Transformer proposed in the present invention can effectively capture the differences in gene expression patterns at different scales.

[0033] 4. The present invention improves the adaptability of the model to unseen data by preprocessing the gene expression data, thereby significantly enhancing the robustness of the classification task.

[0034] 5. The present invention optimizes the classification head by adding a global pooling layer and a normalization layer before the classification head, thereby improving the classification accuracy and the stability of the model output.

[0035] Therefore, the multi-scale neural network based on the improved Transformer proposed in the present invention shows efficient performance and accuracy in the single-cell RNA sequencing data classification task, providing a new perspective and solution for the field of single-cell data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of data processing in the multi-scale multi-head attention mechanism.

[0037] Figure 2 Schematic diagram of the structure of the gated feedforward network.

[0038] Figure 3 Schematic diagram of the structure of the improved Transformer model.

[0039] Figure 4Schematic diagram of the structure of a multi-scale neural network based on the improved Transformer.

[0040] Figure 5 The figure is an overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the present invention, and thus to more clearly define the scope of protection claimed in the present invention, the present invention is described in detail with respect to certain specific embodiments and drawings of the present invention. It should be noted that the following are only certain specific implementation methods of the present invention, which are only part of the embodiments of the present invention, wherein the specific and direct description of the relevant structures is only for the convenience of understanding the present invention, and each specific feature does not naturally and directly limit the scope of implementation of the present invention. The conventional selections and substitutions made by those skilled in the art under the guidance of the present invention, as well as the reasonable arrangement and combination of several technical features under the guidance of the present invention, should all be deemed to be within the scope of protection claimed in the present invention.

[0042] The present invention proposes a single-cell RNA sequencing data classification method based on deep learning, such as Figure 5 As shown, the method comprises the following steps:

[0043] Step 1: Collect raw single-cell RNA sequencing data from public databases or laboratory resources. These data usually exist in the form of gene expression matrices, containing gene expression information from thousands of single cells. After preprocessing the single-cell RNA sequencing data, obtain the training data set;

[0044] Preprocessing includes data cleaning and standardization. Data cleaning is to control the quality of data and remove low-quality cells and genes. Standardization is used to ensure the consistency and comparability of data. The formula for standardization is as follows:

[0045] x'=log((x+c u ) / c v )- log((c r +c u ) / c v )

[0046] Among them, x is the original expression value, x' is the normalized expression value, c u and c v are the variable and invariant components estimated by the Seurat toolkit, c r is the total expression level per cell.

[0047] Step 2: Build a multi-scale neural network based on the improved Transformer, and use the training data set to train the multi-scale neural network based on the improved Transformer to obtain the RNA sequencing data classification model;

[0048] Among them, Figure 4 As shown, the multi-scale neural network based on the improved Transformer includes a first linear layer, an improved Transformer model, an average pooling layer, a first normalization layer, and a second linear layer connected in sequence. The input of the multi-scale neural network based on the improved Transformer is used as the input of the first linear layer, and the output of the second linear layer is used as the output of the multi-scale neural network based on the improved Transformer.

[0049] like Figure 3 As shown in the figure, the improved Transformer model includes an encoder and a gated feedforward network (Gated FFN). The input of the improved Transformer model is used as the input of the encoder. The output of the encoder is added to the learnable position encoding and then input into the gated feedforward network (Gated FFN). The output of the gated feedforward network (Gated FFN) is used as the output of the improved Transformer model. Among them, the position encoding adds a relative position encoding obtained by learning to each input position, thereby introducing position information into the model. This enables the model to understand and process the relative position relationship in the input sequence without relying on the traditional absolute position encoding method. The gated feedforward network (Gated FFN) is a feedforward network that controls the flow of information through a gating mechanism. The network structure is shown in the figure. Figure 2 As shown. It uses the Sigmoid activation function to weight the input, selectively activates the features, and finally outputs the weighted features. The present invention combines position encoding with a gated feedforward network. Position encoding provides the model with necessary position information to help the model understand the structure of the data, while the gated feedforward network optimizes the process of feature selection and nonlinear transformation, thereby improving the performance of the model in complex tasks. The synergy of these two modules enables the model to be more efficient and accurate in processing high-dimensional sparse data, especially in the analysis of biological data with long-range dependencies and high complexity such as single-cell RNA sequencing.

[0050] The encoder includes a multi-head attention mechanism of multiple scales, a random dropout layer and a second normalization layer. Multiple input sequences of different scales are generated according to the input of the encoder. In this embodiment, the number of scales of the input sequence is 3. Then, the input sequence of each scale is used as the input of the multi-head attention mechanism of the corresponding scale, and then the input of the encoder is spliced ​​with the output of the multi-head attention mechanism of all scales and then input into the random dropout layer. The random dropout layer is connected to the second normalization layer, and the output of the second normalization layer is used as the output of the encoder. Figure 1 As shown in the figure, the multi-scale multi-head attention mechanism in the encoder can extract features of the input sequence from different scales and capture global and local dependencies at multiple scales, thereby enhancing the model's understanding of complex data structures. The output of each scale is concatenated and input to the random dropout layer. Random dropout effectively prevents overfitting and improves the generalization ability of the model. Finally, the second normalization layer standardizes the concatenated features to ensure the stable flow of gradients during training, thereby improving the convergence speed and stability of the model. Overall, the encoder can effectively fuse multi-scale information, enhance feature learning capabilities, and improve the robustness of the model, especially for processing high-dimensional complex data.

[0051] In the encoder, the multi-scale multi-head attention mechanism introduced in the present invention plays an important role, and its advantage lies in the multi-scale mechanism. The multi-scale mechanism can capture both short-term and long-term dependencies in the input sequence. Specifically, the multi-scale multi-head attention mechanism uses multiple multi-head attention layers in its implementation, and the input sequence of each layer is downsampled at different scales. For example, the input sequence is downsampled proportionally, so that different local and global information in the sequence can be captured at different scales. After being processed by the attention mechanism, these outputs of different scales are adjusted back to the length of the original sequence by interpolation (usually linear interpolation) to ensure that information of different scales can be fused in the same dimension. Finally, the outputs of these scales are spliced ​​together and projected through a linear transformation layer to obtain the final output. The traditional single-scale self-attention mechanism may not be able to fully understand the different levels of features in a long sequence, while the multi-scale mechanism can learn more levels of contextual information through multiple downsampling and feature fusion, thereby improving the expressiveness of the model. In this way, the model can process complex sequence data more flexibly, and the model can effectively reduce the feature dimension through pooling operation, complete the dimensionality reduction and abstraction of features, and better capture the complex information in gene expression patterns.

[0052] The present invention introduces residual connections, random dropout layers and normalization layers in the encoder. The introduction of residual connections is to alleviate the gradient vanishing problem in deep neural networks. In traditional deep networks, the gradient may gradually decrease as the number of layers increases in multiple iterations, making the network difficult to train. The residual connection ensures that the gradient can be directly transmitted through multiple layers of the network by directly adding the input of each layer to the output of the layer, thereby enhancing the fluidity of the gradient. Random dropout is a regularization technique that aims to prevent the model from overfitting during training. It forces the model to rely on different combinations of neurons each time it is trained by randomly shutting down a certain proportion of neurons during training, thereby reducing excessive reliance on certain features or neurons. In the encoder module, the random dropout layer is usually applied to the multi-head attention layer and the feedforward network layer to reduce overfitting, especially in high-dimensional sparse data and complex tasks, to prevent the model from performing well only on the training data and failing to generalize to new data. The normalization layer is a commonly used standardization method in neural networks, which is usually used to improve the convergence speed and stability during training. In each layer, the normalization layer normalizes the input, that is, the activation value of each neuron is subtracted from the mean and divided by the standard deviation, which makes the distribution of each feature more stable. The combination of these three enables the model to learn and classify more efficiently and accurately when processing high-dimensional sparse data.

[0053] The present invention utilizes the powerful self-attention mechanism in the encoder of the Transformer model to capture cross-scale and global dependencies by calculating the attention weights between the elements in the gene expression sequence. The model can effectively learn local and global features in gene expression data by introducing a multi-scale attention mechanism and modeling at different feature scales. This multi-scale processing enables the model to extract information from different scales, thereby optimizing the feature learning process and significantly improving the sequence modeling performance of the model when processing high-dimensional sparse data. In single-cell RNA sequencing data, gene expression often presents a sparse distribution and the data dimension is extremely high. Traditional sequence modeling methods are difficult to capture the complex dependencies therein, and the multi-scale multi-head attention mechanism effectively solves this problem. During the model training process, a random inactivation layer is applied to randomly close some neurons in the network to reduce overfitting and enhance the generalization ability of the model. The encoder module combines a multi-scale attention mechanism with relative position encoding to capture long-range dependencies and multi-level features in single-cell RNA sequencing data.

[0054] Gated FFN replaces the traditional activation function and is used to perform nonlinear processing on the input of the gated feedforward network. The gated feedforward network not only accelerates the training process of the neural network, but also improves the convergence effect and classification accuracy of the model by screening important features. Through the gating structure, the network can select important features more flexibly, thereby enhancing the nonlinear expression ability of the model and improving the overall performance.

[0055] Step 3: After preprocessing, the single-cell RNA sequencing data to be tested is input into the RNA sequencing data classification model, and the model outputs the corresponding classification results.

[0056] Model training and cross-validation were performed on multiple public single-cell RNA sequencing datasets, using accuracy (ACC) as the main evaluation indicator to ensure that the RNA sequencing data classification model has good generalization ability and stability. In order to comprehensively evaluate the performance of the model on different datasets, the accuracy on the training set and validation set was calculated respectively, and then the performance of the model at different stages was analyzed. In the experiment, different Dropout rates were also tested. The results showed that when the Dropout rate was set to 0.5, the classification accuracy of the model achieved the best performance, indicating that an appropriate Dropout rate can effectively prevent overfitting and improve the generalization ability of the model.

[0057] During the model training process, the Adam optimizer was used to update the parameters, and the optimization step size was set to 0.001 to ensure that the model converged efficiently and stably in each training cycle. The training cycle was set to 50 cycles to ensure that the model was fully trained and could capture complex patterns in the data. In terms of loss function, cross entropy was selected as the classification loss function, which is suitable for multi-classification tasks and can effectively measure the difference between the model prediction and the true label. The ratio of the training set and the validation set is 7:3 to ensure that the model can fully learn the characteristics of the data during training and evaluate its generalization ability through the validation set. The number of batches processed per batch is set to 64 to effectively improve the computational efficiency and accelerate the training process under limited computing resources.

[0058] In order to further improve the computational efficiency of the model, the present invention uses multithreading and GPU acceleration technology to optimize the model training and prediction process. Through parallel processing technology, multiple tasks can be processed simultaneously, thereby significantly reducing the time consumption of model training and prediction, and improving the overall computational efficiency. These optimization measures not only improve the model training speed, but also ensure efficient execution on large-scale data sets, so that the model can cope with the analysis tasks of complex biological data such as single-cell RNA sequencing.

[0059] In the model evaluation phase, the accuracy rate (ACC) is used as an evaluation indicator to calculate the accuracy rate on the training set and the validation set respectively to evaluate the specific performance of the model. The accuracy rate (ACC) is calculated by dividing the number of correctly classified samples by the total number of samples. The accuracy rate ranges from 0 to 1, and the larger the value, the better the classification result.

[0060] Finally, the method of the present invention was compared in detail with the existing single-cell RNA data analysis methods. The experimental results show that on multiple public data sets, including PBMC, Klein, and Mouse, the method of the present invention shows a significant improvement in the accuracy (ACC) index compared with existing models such as DCA and scSemiAAE. Specifically, the accuracy of the present invention on these data sets is about 10% higher than that of traditional methods, indicating its advantage in classification tasks. This improvement not only reflects the generalization ability of the model on different data sets, but also verifies the effectiveness of the method in processing high-dimensional sparse data, and further proves its application potential and superiority in single-cell RNA sequencing data analysis.

[0061] The present invention also proposes a single-cell RNA sequencing data classification system based on deep learning, the system comprising:

[0062] A data preprocessing unit, used to preprocess single-cell RNA sequencing data;

[0063] A training data set generating unit, used to generate a training data set according to the preprocessed data;

[0064] A data classification unit, used to store the multi-scale neural network based on the improved Transformer and the corresponding network parameters, and to classify the input pre-processed sequencing data using the multi-scale neural network based on the improved Transformer, and output the corresponding classification results;

[0065] The network training unit is used to train the multi-scale neural network based on the improved Transformer using the training data set until the training is completed and the final network parameters are sent to the data classification unit.

[0066] The present invention also proposes a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a single-cell RNA sequencing data classification method based on deep learning.

[0067] The present invention also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a single-cell RNA sequencing data classification method based on deep learning are implemented.

[0068] The present invention also proposes a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of a single-cell RNA sequencing data classification method based on deep learning.

[0069] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A single-cell RNA sequencing data classification method based on deep learning, characterized in that: The following steps are involved: Step 1: After preprocessing the single-cell RNA sequencing data, obtain the training data set; Step 2: Build a multi-scale neural network based on the improved Transformer, and use the training data set to train the multi-scale neural network based on the improved Transformer to obtain the RNA sequencing data classification model; Step 3: Preprocess the single-cell RNA sequencing data to be detected and then input it into the RNA sequencing data classification model, and the model outputs the corresponding classification results; The multi-scale neural network based on the improved Transformer includes a first linear layer, an improved Transformer model, an average pooling layer, a first normalization layer and a second linear layer connected in sequence, the input of the multi-scale neural network based on the improved Transformer is used as the input of the first linear layer, and the output of the second linear layer is used as the output of the multi-scale neural network based on the improved Transformer; The improved Transformer model includes an encoder and a gated feedforward network, the input of the improved Transformer model is used as the input of the encoder, the output of the encoder is added to the learnable position code and then input into the gated feedforward network, and the output of the gated feedforward network is used as the output of the improved Transformer model; The encoder includes a multi-scale multi-head attention mechanism, a random dropout layer and a second normalization layer. Multiple input sequences of different scales are generated according to the input of the encoder, and then the input sequence of each scale is used as the input of the multi-head attention mechanism of the corresponding scale. The input of the encoder is then spliced ​​with the output of the multi-head attention mechanism of all scales and then input into the random dropout layer. The random dropout layer is connected to the second normalization layer, and the output of the second normalization layer is used as the output of the encoder.

2. A single-cell RNA sequencing data classification method based on deep learning according to claim 1, characterized in that: The preprocessing includes data cleaning and standardization.

3. The single-cell RNA sequencing data classification method based on deep learning according to claim 1, characterized in that: The gated feedforward network is used to perform nonlinear processing on the input of the gated feedforward network.

4. A single-cell RNA sequencing data classification system based on deep learning, characterized in that: include: A data preprocessing unit, used to preprocess single-cell RNA sequencing data; A training data set generating unit, used to generate a training data set according to the preprocessed data; A data classification unit, used to store the multi-scale neural network based on the improved Transformer and the corresponding network parameters, and to classify the input pre-processed sequencing data using the multi-scale neural network based on the improved Transformer, and output the corresponding classification results; A network training unit is used to train a multi-scale neural network based on an improved Transformer using a training data set until the training is completed and the final network parameters are sent to a data classification unit; The multi-scale neural network based on the improved Transformer includes a first linear layer, an improved Transformer model, an average pooling layer, a first normalization layer and a second linear layer connected in sequence, the input of the multi-scale neural network based on the improved Transformer is used as the input of the first linear layer, and the output of the second linear layer is used as the output of the multi-scale neural network based on the improved Transformer; The improved Transformer model includes an encoder and a gated feedforward network, the input of the improved Transformer model is used as the input of the encoder, the output of the encoder is added to the learnable position code and then input into the gated feedforward network, and the output of the gated feedforward network is used as the output of the improved Transformer model; The encoder includes a multi-scale multi-head attention mechanism, a random dropout layer and a second normalization layer. Multiple input sequences of different scales are generated according to the input of the encoder, and then the input sequence of each scale is used as the input of the multi-head attention mechanism of the corresponding scale. The input of the encoder is then spliced ​​with the output of the multi-head attention mechanism of all scales and then input into the random dropout layer. The random dropout layer is connected to the second normalization layer, and the output of the second normalization layer is used as the output of the encoder.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the single-cell RNA sequencing data classification method based on deep learning described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the single-cell RNA sequencing data classification method based on deep learning according to any one of claims 1 to 3 are implemented.

7. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the single-cell RNA sequencing data classification method based on deep learning as described in any one of claims 1 to 3 are implemented.

Images