Full-length gene sequence modeling method and system based on neural network

Abstract

The invention provides a full-length gene sequence modeling method and system based on a neural network, and the method comprises the steps: constructing a first expression matrix for initial single-cell RNA sequencing data, and carrying out the quality control transformation of the first expression matrix to obtain a second expression matrix; inputting the second expression matrix into a preset binning embedding module to obtain a binning embedding matrix; maintaining and loading a gene pathway set through a knowledge base and a mapping module to obtain a binary mask matrix, and performing mask processing on the binning embedded matrix based on the binary mask matrix to obtain a pathway mask matrix; the path mask matrix is input into a preset attention state space model, the attention state space model comprises an encoder module, a jump connection module and a decoder module which are arranged in sequence, and a reconstruction tensor is output through the decoder module. According to the scheme, an efficient and extensible whole-gene annotation method is provided, and whole-gene expression input can be processed while the calculation efficiency is kept.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bioinformatics, and in particular to a full-length gene sequence modeling method and system based on a neural network. BACKGROUND

[0002] Single-cell RNA sequencing (scRNA-seq) is one of the most breakthrough technologies in the field of life sciences. It captures the transcriptome information of individual cells and accurately analyzes cell heterogeneity for the first time. Traditional RNA sequencing technology can only reflect the average expression characteristics of cell populations, while scRNA-seq can reveal the unique gene expression patterns of each cell by separating single cells and performing reverse transcription, amplification and sequencing of mRNA.

[0003] The emergence of single-cell RNA sequencing (scRNA-seq) technology enables researchers to analyze gene expression heterogeneity at the single-cell level. This technology plays an important role in tumor microenvironment analysis, immune cell subpopulation division and development process tracking. However, scRNA-seq data is large in scale and high in dimension, and is also affected by sequencing read depth (Read Depth) differences, noise interference and data sparsity.

[0004] In order to accurately identify cell types from large-scale scRNA-seq data, a large number of methods have attempted to apply machine learning and deep learning to single-cell annotation in recent years. However, the full gene information of the prior art is difficult to utilize efficiently: existing models often reduce the input dimension through gene screening or dimension reduction, which may result in the loss of biological signals of low-expression or low-variance genes. SUMMARY

[0005] In view of this, the embodiments of the present application provide a full-length gene sequence modeling method and system based on a neural network to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present application provides a full-length gene sequence modeling method based on a neural network, the steps of which include: constructing a first expression matrix from initial single-cell RNA sequencing data, and performing quality control transformation on the first expression matrix to obtain a second expression matrix; inputting the second expression matrix into a preset binning embedding module to obtain a binning embedding matrix; maintaining and loading a gene pathway set through a knowledge base and a mapping module to obtain a binary mask matrix, and performing mask processing on the binning embedding matrix based on the binary mask matrix to obtain a pathway mask matrix; The path mask matrix is ​​input into a preset attention state space model, which includes an encoder module, a skip connection module, and a decoder module arranged in sequence. The decoder module outputs a reconstructed tensor.

[0007] Using the above approach, this scheme provides an efficient and scalable whole-genome annotation method that can handle whole-genome expression input while maintaining computational efficiency; it provides a pathway structure masking strategy that introduces gene pathway priors during the self-supervised learning stage to improve the biological interpretability of the model; it provides a read depth-aware modeling mechanism that explicitly preserves read depth information through skip connections to improve the robustness of the model in cross-platform and low read depth scenarios; and it provides an efficient deep learning framework to achieve low-complexity long-range dependency modeling.

[0008] In some embodiments of the present invention, the method further includes inputting the reconstructed tensor into a preset classification model, wherein the classification model outputs cell classification results.

[0009] In some embodiments of the present invention, the encoding module employs a Performer encoder, the jump connection module employs a U-Net network structure, and the decoding module employs a Mamba decoder.

[0010] In some embodiments of the present invention, in the step of inputting the second expression matrix into a preset binning embedding module to obtain a binning embedding matrix, the binning embedding module uses Bin Embedding to discretize each value of the second expression matrix into integer bins to obtain the binning embedding matrix.

[0011] In some embodiments of the present invention, in the step of performing quality control transformation on the first expression matrix to obtain the second expression matrix, for any row or column in the second expression matrix, the number of outliers is screened. If the number of outliers is greater than a preset threshold, the row or column is deleted.

[0012] In the specific implementation process, the behavioral cells of the first expression matrix are listed as genes.

[0013] In some embodiments of the present invention, the step of performing a quality control transformation on the first expression matrix to obtain a second expression matrix further includes normalizing and logarithmically transforming the values ​​in the second expression matrix.

[0014] In some embodiments of the present invention, in the step of maintaining and loading the gene pathway set through the knowledge base and mapping module to obtain a binary mask matrix, the gene pathway set is loaded, and each pathway in the gene pathway set is defined as a gene set. When a new gene expression dataset is input, the gene names in the gene set are matched with the gene names in the first expression matrix. If the gene exists in the gene pathway set, the value is 1; otherwise, the value is 0, and a binary association matrix is ​​generated.

[0015] In some embodiments of the present invention, the number of rows and columns in the binary mask matrix and the bin embedding matrix are equal. In the step of masking the bin embedding matrix based on the binary mask matrix to obtain the path mask matrix, each position in the bin embedding matrix is ​​mapped to a position in the binary mask matrix. If the value in the corresponding binary mask matrix is ​​0, the original value is retained; if the value in the corresponding binary mask matrix is ​​1, masking is performed.

[0016] In some embodiments of the present invention, the method further includes model pre-training, wherein the model is pre-trained using a cross-entropy loss function.

[0017] A second aspect of the present invention also provides a full-length gene sequence modeling system based on a neural network. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0018] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned neural network-based full-length gene sequence modeling method.

[0019] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0020] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0021] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0022] Figure 1 This is a schematic diagram illustrating one implementation of the neural network-based full-length gene sequence modeling method of this scheme; Figure 2 This is a schematic diagram illustrating another implementation of the neural network-based full-length gene sequence modeling method of this scheme; Figure 3 This is a schematic diagram of the overall architecture of this solution. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0024] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0025] Introduction to existing technologies: The advent of single-cell RNA sequencing (scRNA-seq) technology has enabled researchers to analyze gene expression heterogeneity at the single-cell level. This technology has played an important role in tumor microenvironment analysis, immune cell subset segmentation, and developmental process tracking. However, scRNA-seq data are massive in scale and extremely high in dimensionality, and are also affected by variations in sequencing read depth, noise interference, and data sparsity.

[0026] The closest solutions to this invention currently include the following categories: 1. Transformer-based methods (such as scBERT and scGPT): These methods borrow from the self-supervised pre-training approach in natural language processing, perform random masking prediction on the gene expression matrix, model global dependencies, are suitable for large-scale data, have high computational costs, and are difficult to extend to full-gene input; the masking strategy has no biological priors.

[0027] 2. Large-scale pre-trained models (such as scFoundation): By pre-training on ultra-large datasets, general single-cell feature representations are obtained, which have strong transfer capabilities. However, the training cost is extremely high, and performance still degrades when crossing reading depths and platforms.

[0028] 3. Solutions combining state-space models (such as GeneMamba): Using state-space models (SSM) to model long-range dependencies improves the ability to process long sequences and is more efficient than Transformer, but lacks pathway knowledge guidance; it does not explicitly handle read depth differences.

[0029] 4. Traditional dimensionality reduction + classification methods (such as PCA + classifier): Dimensionality reduction is achieved through PCA or autoencoders, and then the data is input into a classifier to identify cell types. These methods are computationally simple and easy to implement, but their accuracy is limited in complex heterogeneous and cross-batch tasks, and they lose information about low-expressed genes.

[0030] Deficiencies of existing technology: To accurately identify cell types from large-scale scRNA-seq data, numerous methods have recently attempted to apply machine learning and deep learning to single-cell annotation. Traditional reference map-based alignment methods are effective with small datasets but struggle to scale to cross-species and cross-platform scenarios. While deep learning models (such as those based on Transformers or variational autoencoders) have improved performance to some extent, they still face the following challenges: 1. Difficulty in efficiently utilizing whole genome information: Existing models often reduce input dimensions through gene screening or dimensionality reduction, which may lead to the loss of biological signals from genes with low expression or low variance.

[0031] 2. Read depth differences in sequencing introduce systematic bias: Read depth differences under different experimental conditions may lead to inconsistent expression distribution of the same cell type in different datasets, and existing models lack explicit mechanisms to eliminate this bias.

[0032] 3. Self-supervised pre-training lacks biological priors: Existing random masking strategies often ignore the pathway structural relationships between genes, making it difficult for the model to capture real biological dependency patterns.

[0033] 4. Poor robustness: Existing models are sensitive to different sequencing read depths, and their performance drops significantly when crossing datasets and species.

[0034] To address the aforementioned shortcomings, the present invention aims to: provide an efficient and scalable whole-genome annotation method that can handle whole-genome expression input while maintaining computational efficiency; provide a pathway structure masking strategy that introduces gene pathway priors during the self-supervised learning stage to improve the biological interpretability of the model; provide a read depth-aware modeling mechanism that explicitly preserves read depth information through skip-connections to improve the robustness of the model in cross-platform and low read depth scenarios; and provide an efficient deep learning framework that combines the Performer encoder and the Mamba decoder to achieve low-complexity long-range dependency modeling.

[0035] likeFigure 1 and 3 As shown, this invention proposes a method for modeling full-length gene sequences based on neural networks. The steps of this method include: Step S100: Construct a first expression matrix from the initial single-cell RNA sequencing data, and perform quality control transformation on the first expression matrix to obtain a second expression matrix; Step S200: Input the second expression matrix into the preset binning embedding module to obtain the binning embedding matrix; In practice, the binning embedding module discretizes continuous gene expression values ​​into several bins, such as low, medium, and high expression. Each bin is mapped to a vector embedding to preserve information about low-expression / low-variance genes.

[0036] Step S300: Maintain and load the gene pathway set through the knowledge base and mapping module to obtain a binary mask matrix. Perform masking processing on the binning embedding matrix based on the binary mask matrix to obtain a pathway mask matrix. Specifically, a preset set of gene pathways is loaded through the gene pathway knowledge base module, and a binary mask matrix is ​​generated based on the set.

[0037] In practice, genes are divided into multiple functional pathway sets based on MSigDB or other pathway databases. During the self-supervised training phase, block masking is performed on a pathway-by-pathway basis, simultaneously masking the expression values ​​of several genes within that pathway. The training objective is to reconstruct the masked pathway expression, thereby capturing pathway-level dependencies between genes.

[0038] Step S400: Input the path mask matrix into a preset attention state space model. The attention state space model includes an encoder module, a skip connection module, and a decoder module arranged in sequence. The decoder module outputs the reconstructed tensor.

[0039] Using the above approach, this scheme provides an efficient and scalable whole-genome annotation method that can handle whole-genome expression input while maintaining computational efficiency; it provides a pathway structure masking strategy that introduces gene pathway priors during the self-supervised learning stage to improve the biological interpretability of the model; it provides a read depth-aware modeling mechanism that explicitly preserves read depth information through skip-connections to improve the robustness of the model in cross-platform and low read depth scenarios; and it provides an efficient deep learning framework to achieve low-complexity long-range dependency modeling.

[0040] like Figure 2 As shown, in some embodiments of the present invention, the method further includes step S500, inputting the reconstructed tensor into a preset classification model, wherein the classification model outputs cell classification results.

[0041] In practice, the classification model can employ recurrent neural networks, long short-term memory networks, or Transformer networks.

[0042] In the specific implementation process, the cell classification results can be output through the final classification layer. The classification layer can use a softmax classifier; or it can use a multilayer perceptron (MLP) classification head and perform fine-tuning training to map cell features to specific cell type labels.

[0043] The above scheme integrates biological priors and self-supervised learning into an end-to-end training process, providing a training system for single-cell annotation models. Its key feature is a two-stage training process. The first stage (pre-training) uses a self-supervised task, such as pathway masking, to train the model on unlabeled data, optimizing reconstruction losses (e.g., cross-entropy loss from masked genes). The second stage (fine-tuning) adds a classification head to the pre-trained model, using labeled cell data, targeting cell type classification, and jointly fine-tunes all parameters of the entire model (including the backbone network and the classification head).

[0044] In some embodiments of the present invention, the encoding module employs a Performer encoder, the jump connection module employs a U-Net network structure, and the decoding module employs a Mamba decoder.

[0045] Specifically, the Performer encoder employs an efficient attention mechanism. The Performer encoder takes the embedded gene sequence as input, and this scheme utilizes the kernel function approximation mechanism of the Performer to calculate attention weights and extract global features. Compared to the traditional Transformer, this structure exhibits linear complexity in whole-genome sequence modeling, significantly reducing computational and memory overhead.

[0046] Furthermore, before inputting into the Performer encoder, a subset of pathways is sampled at a scale of p, and a pathway-based gene-level mask is calculated. This mask is then replaced with a preserved "mask token." The Performer encoder module adds positional encoding to the output of the preceding module and uses FAVOR+^++ linear attention to extract global dependencies, outputting a multi-layered encoding.

[0047] Specifically, the Performer encoder takes the embedded gene sequence as input, calculates attention weights using the Performer kernel function approximation mechanism, and extracts global features. Compared to the traditional Transformer, this structure has linear complexity in whole-genome sequence modeling, significantly reducing computational and memory overhead.

[0048] The Mamba decoder is a state-space model. The Mamba structure based on the State-Space Model (SSM) is used in the decoding stage to reconstruct the masked gene expression. SSM excels at modeling long sequence dependencies, compensating for the shortcomings of attention mechanisms in sparse scenarios.

[0049] Furthermore, the Mamba decoder module performs long-range modeling and reconstruction on the state-space model (SSM), using inter-layer residuals and normalized outputs to finally obtain the reconstructed tensor.

[0050] A read depth-aware skip connection is used between the encoder and decoder modules, and a U-Net-style symmetric skip connection is introduced between the encoder and decoder. Each connection transmits the total read depth information of the cell, enabling the model to consider the impact of different sequencing depths when recovering masked pathways.

[0051] Using the above approach, the sequencing depth-aware U-Net-style skip connection mechanism introduces symmetric skip connections between the encoder and decoder to deliver and enhance low-abundance transcript signals and align features at different depth levels to implicitly model sequencing depth variations.

[0052] A read depth scalar / vector for each cell is computed using read depth-aware skip connections (which can be obtained from the original counts or the total counts before normalization), and generated by a small MLP. This vector is then used for gated fusion of the encoded features of the symmetric layer during the decoding stage.

[0053] A U-Net-style symmetric skip connection is introduced between the encoder and decoder. Each connection transmits the total read depth information of the cell, allowing the model to account for the effects of different sequencing depths when reconstructing masked pathways.

[0054] The above approach, based on a hybrid architecture of Performer and Mamba, is used to model full-length gene sequences. This method employs a hybrid neural network architecture that combines a Performer encoder (linear complexity attention mechanism) with a Mamba decoder (selective state-space model) to achieve efficient modeling of full-length gene expression sequences in single cells (without gene truncation or screening).

[0055] In some embodiments of the present invention, in the step of inputting the second expression matrix into a preset binning embedding module to obtain a binning embedding matrix, the binning embedding module uses Bin Embedding to discretize each value of the second expression matrix into integer bins to obtain the binning embedding matrix.

[0056] Specifically, the bin embedding module discretizes each value of the input matrix into integer bins.

[0057] In some embodiments of the present invention, in the step of performing quality control transformation on the first expression matrix to obtain the second expression matrix, for any row or column in the second expression matrix, the number of outliers is screened. If the number of outliers is greater than a preset threshold, the row or column is deleted.

[0058] In the specific implementation process, the behavioral cells of the first expression matrix are listed as genes.

[0059] Using the above approach, single-cell RNA sequencing data is standardized (e.g., logarithmic normalization, batch effect correction) to obtain a cell × gene expression matrix, filtering out low-quality cells or abnormal genes to ensure the validity of the input data.

[0060] In some embodiments of the present invention, the step of performing a quality control transformation on the first expression matrix to obtain a second expression matrix further includes normalizing and logarithmically transforming the values ​​in the second expression matrix.

[0061] Specifically, in the quality control transformation step, the original scRNA-seq expression matrix (i.e., the first expression matrix) is received, and quality control (i.e., removal of low-quality cells and zero-expression genes), total normalization, and logarithmic transformation are performed. The minimum gene count threshold is 200; the target count for normalization is 10,000; and log1p transformation is used.

[0062] In some embodiments of the present invention, in the step of maintaining and loading the gene pathway set through the knowledge base and mapping module to obtain a binary mask matrix, the gene pathway set is loaded, and each pathway in the gene pathway set is defined as a gene set. When a new gene expression dataset is input, the gene names in the gene set are matched with the gene names in the first expression matrix. If the gene exists in the gene pathway set, the value is 1; otherwise, the value is 0, and a binary association matrix is ​​generated.

[0063] The above approach, based on the structured masking self-supervised pre-training method of gene program, uses gene sets in biological knowledge bases (such as MSigDB) to perform pathway-level masking. The pre-training goal is to reconstruct the masked gene program, thereby guiding the model to learn the functional synergistic expression patterns between genes.

[0064] In some embodiments of the present invention, the number of rows and columns in the binary mask matrix and the bin embedding matrix are equal. In the step of masking the bin embedding matrix based on the binary mask matrix to obtain the path mask matrix, each position in the bin embedding matrix is ​​mapped to a position in the binary mask matrix. If the value in the corresponding binary mask matrix is ​​0, the original value is retained; if the value in the corresponding binary mask matrix is ​​1, masking is performed.

[0065] Specifically, bin embedding and whole-genome vocabulary input representation discretize continuous gene expression values ​​into bin tokens, retaining all genes (including low-expression genes) as input, without the need for gene screening or dimensionality reduction.

[0066] In some embodiments of the present invention, the method further includes model pre-training, wherein the model is pre-trained using a cross-entropy loss function.

[0067] In some embodiments of the present invention, the loss of the cross-entropy loss function is calculated for the masked gene location.

[0068] In summary, in single-cell RNA sequencing (scRNA-seq) data annotation tasks, existing methods often reduce modeling complexity by using random masking, dimensionality reduction, or gene screening, but this results in the loss of low-expressed genes and pathway-level information, and lacks robustness under different sequencing read depths.

[0069] To address this, this invention proposes a method combining pathway structure masking and attention-state space hybrid modeling. The basic idea is to: preserve whole-genome information by replacing gene screening with expression value binning to avoid losing low-expression signals; introduce biological priors by masking the entire gene pathway structure during self-supervised training to improve the biological interpretability of the features learned by the model; and employ depth-aware modeling by combining the Performer encoder and Mamba decoder, incorporating skip connections to enhance the model's robustness and generalization ability under different sequencing depths.

[0070] This principle ensures that the model can effectively annotate cell types on large-scale, batch-wise, and platform-wise data, while maintaining both efficiency and scalability.

[0071] The beneficial effects of this plan include: 1. Using the Performer encoder (linear complexity attention mechanism) and Mamba decoder (selective state space model), we achieve efficient modeling of full-length gene sequences (up to 20,000+ genes); 2. A U-Net-style skip connection is introduced to align encoder features with the decoder at the same depth, enhancing the recovery capability for low-abundance transcripts. A depth-sensitivity minimization objective is proposed, and depth-invariant representations are learned through implicit modeling of sequencing depth variations via architectural design. Skip connections deliver depth-related fine-grained signals, enhancing the model's ability to perceive low-expression genes. Theoretical guarantees demonstrate that the model can simultaneously optimize reconstruction accuracy and robustness, avoiding the trade-offs between the two in traditional methods. 3. Pathway-level masking is performed using the MSigDB gene set, masking entire functionally related gene programs. The pre-training goal is to reconstruct the masked pathways, encouraging the model to learn co-expression patterns between genes. By introducing biological priors, the representations learned by the model are more interpretable and generalizable. Structured priors reduce model complexity and improve generalization ability with limited data.

[0072] This invention also provides a full-length gene sequence modeling system based on neural networks. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0073] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned neural network-based full-length gene sequence modeling method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0074] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0075] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0076] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for modeling full-length gene sequences based on neural networks, characterized in that, The steps of this method include: A first expression matrix was constructed from the initial single-cell RNA sequencing data, and a quality control transformation was performed on the first expression matrix to obtain a second expression matrix; The second expression matrix is ​​input into a preset binning embedding module to obtain a binning embedding matrix; By maintaining and loading the gene pathway set through the knowledge base and mapping module, a binary mask matrix is ​​obtained. The binning embedding matrix is ​​then masked based on the binary mask matrix to obtain the pathway mask matrix. The path mask matrix is ​​input into a preset attention state space model, which includes an encoder module, a skip connection module, and a decoder module arranged in sequence. The decoder module outputs a reconstructed tensor.

2. The method for modeling full-length gene sequences based on neural networks according to claim 1, characterized in that, The method further includes inputting the reconstructed tensor into a preset classification model, wherein the classification model outputs cell classification results.

3. The method for modeling full-length gene sequences based on neural networks according to claim 1, characterized in that, The encoding module uses a Performer encoder, the jump connection module uses a U-Net network structure, and the decoding module uses a Mamba decoder.

4. The method for modeling full-length gene sequences based on neural networks according to claim 3, characterized in that, In the step of inputting the second expression matrix into a preset binning embedding module to obtain a binning embedding matrix, the binning embedding module uses Bin Embedding to discretize each value of the second expression matrix into integer bins to obtain the binning embedding matrix.

5. The method for modeling full-length gene sequences based on neural networks according to claim 4, characterized in that, In the step of performing quality control transformation on the first expression matrix to obtain the second expression matrix, for any row or column in the second expression matrix, the number of outliers is screened. If the number of outliers is greater than a preset threshold, the row or column is deleted.

6. The method for modeling full-length gene sequences based on neural networks according to claim 1, characterized in that, The step of performing a quality control transformation on the first expression matrix to obtain the second expression matrix further includes normalizing and performing a logarithmic transformation on the values ​​in the second expression matrix.

7. The method for modeling full-length gene sequences based on neural networks according to claim 1, characterized in that, In the step of maintaining and loading the gene pathway set through the knowledge base and mapping module to obtain the binary mask matrix, the gene pathway set is loaded. Each pathway in the gene pathway set is defined as a gene set. When a new gene expression dataset is input, the gene names in the gene set are matched with the gene names in the first expression matrix. If the gene exists in the gene pathway set, the value is 1; otherwise, the value is 0, generating a binary association matrix.

8. The method for modeling full-length gene sequences based on neural networks according to any one of claims 1 to 7, characterized in that, The binary mask matrix and the bin embedding matrix have the same number of rows and columns. In the step of masking the bin embedding matrix based on the binary mask matrix to obtain the path mask matrix, each position in the bin embedding matrix is ​​mapped to a position in the binary mask matrix. If the value in the corresponding binary mask matrix is ​​0, the original value is retained; if the value in the corresponding binary mask matrix is ​​1, masking is performed.

9. The method for modeling full-length gene sequences based on neural networks according to claim 1, characterized in that, The method further includes model pre-training, in which the model is pre-trained using a cross-entropy loss function.

10. A full-length gene sequence modeling system based on neural networks, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 9.

Images