A method for serialization extraction of highly variable exons

Abstract

This invention discloses an efficient RNA data preprocessing method to address the problems of low processing efficiency and low information density in high-throughput sequencing data. Its core steps include: (1) introducing a parallel processing scheme for high-throughput sequence data, rapidly mapping RNA-seq data to a reference genome to generate a BAM file; (2) extracting base sequences and expression levels and storing them as compact PKL format files; (3) extracting all exon position information by parsing the genome annotation file; (4) combining multi-sample expression level data to screen for highly variable exons and constructing a high-information-density feature list based on the sample set; and (5) accurately extracting target sequences from the preprocessed file based on this list. Compared to traditional methods, this innovative approach achieves triple optimization: full-process parallel processing for accelerated computation, high-compression data storage, and adaptive feature selection. Processing speed is increased by 3-5 times, and data volume is reduced by more than 90%, making it suitable for high-throughput RNA-seq data analysis with large sample sizes.

Description

Technical Field

[0001] This invention relates to the field of high-throughput biological data processing, specifically to an efficient preprocessing method for RNA sequencing data, particularly for the rapid extraction and serialization storage of highly variable exons, aiming to improve data processing speed and reduce computational resource consumption. Background Technology

[0002] High-throughput RNA sequencing (RNA-seq) technology is widely used in gene expression analysis and transcriptional regulation research, but the massive amounts of data it generates pose a serious challenge to traditional preprocessing methods. While existing tools (such as STAR and Hisat2) can perform preprocessing, they are inefficient when handling large-scale datasets, and their workflows are fragmented with significant data redundancy. Traditional methods typically require sequential execution of quality control, alignment, and expression level estimation steps, which are time-consuming and generate large file sizes, thus limiting the efficiency of subsequent analysis.

[0003] Furthermore, existing technologies largely focus on differential analysis at the gene level, while exons, as the core units of gene function, contain more refined biological information through cross-sample expression variations. However, traditional workflows lack efficient screening and compressed storage mechanisms for exons, resulting in insufficient extraction of high-information features and difficulty in supporting advanced applications such as tumor heterogeneity analysis. Therefore, there is an urgent need for a preprocessing method that integrates parallel computing acceleration, exon-level feature screening, and highly compressed storage. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention proposes a high-speed, high-throughput RNA data preprocessing and extraction method, aiming to solve the problems of long processing time, complex operation, and information redundancy in traditional RNA-seq preprocessing workflows. This invention is achieved through the following technical solution: This invention provides a high-speed, exon-level, high-throughput RNA data preprocessing and extraction method.

[0005] The method includes the following steps: (1) inputting RNA sequencing data (fastq format), reference genome sequence (fasta format), and annotation file (gtf format); (2) using GPU parallel computing technology to quickly align the sequencing data to the reference genome and generate a bam file containing base alignment information; (3) extracting the exon base sequence and expression level of the sample from the bam file and storing it as a highly compressed binary file; (4) parsing the genome annotation file and constructing a whole-genome exon position index; (5) extracting the original sequence information of exons in the reference genome based on the index; (6) integrating exon expression data from multiple samples, screening highly variable exons by calculating the coefficient of variation, and generating a target list; (7) according to the target list, extracting the sequence and expression level of highly variable exons from the preprocessed file and outputting a simplified dataset.

[0006] The core innovation of this invention lies in: (1) Exon-level processing: directly using exons as the analysis unit to avoid redundancy at the gene or transcript level; (2) Use exons directly as the analysis unit to avoid redundancy at the gene or transcript level; (3) Adaptive feature selection: Accurately identify high-information exons based on cross-sample expression variation coefficient; (4) The process is implemented through a full-process script, including data comparison, feature extraction, high variation screening and result output modules. The process is fast and modular. Attached Figure Description

[0007] Figure 1 This is an overall flowchart of the present invention; Detailed Implementation

[0008] The preprocessing process is implemented using modular scripts, and the specific steps are as follows: (1) Data alignment: GPU parallel computing is called to accelerate the alignment module, and RNA sequencing data (fastq format) is quickly mapped to the reference genome to generate a bam file containing base alignment information, which reduces the time consumption by 3-5 times compared with the traditional CPU method.

[0009] (2) Sequence and expression level extraction: The exon base sequence and its expression level are parsed from the bam file and stored as a highly compressed binary format file.

[0010] (3) Construction of exon position index: Parse the genome annotation file (gtf format), extract the start and end site information of the whole genome exons, and generate a standardized position index list.

[0011] (4) Exon sequence extraction: Based on position indexing, the original base sequence of all exons is extracted from the reference genome (fasta format) for subsequent variation analysis.

[0012] (5) Screening of highly variable exons: Integrating exon expression data from multiple samples to construct Expression matrix ( For the sample size, (Exon number), through local regression (LOESS) modeling and standardized variance calculation, a preset number of highly variable exons are selected.

[0013] (6) Target sequence extraction: Based on the list of highly variable exons, the sequence and expression level of the corresponding exons are accurately extracted from the preprocessed file to generate the final exon-level simplified dataset.

[0014] The screening of highly variable exons with high information density is achieved through the following refined process: (1) Exon expression statistics: For the expression matrix P (dimension) , For the sample size, (where the number of exons is 0), calculate the mean of each exon across samples (where the number of exons is 0). ) and variance ( 2), The formula is: in, These are the sample index and the exon index, respectively.

[0015] (2) Mean-variance relationship modeling: Locally weighted regression (LOESS) is used to fit the non-linear relationship between exon mean and variance, eliminating technical noise (such as sequencing depth differences). The expected variance of each exon is predicted using the LOESS model. ; (3) Outlier truncation: To reduce the interference of extreme values ​​on the screening results, the expression matrix P was truncated, and the expression level of each exon was limited to the 99th percentile of its distribution.

[0016] (4) Standardized variance calculation: Based on the LOESS model predictions, calculate the standardized variance of the exons: This metric measures the level at which exon expression variation exceeds technical noise; a higher value indicates more significant biological variation.

[0017] (5) Screening of highly variable exons: based on standardized variance Sort all exons in descending order and select the top-ranked ones. (or a pre-defined percentage) of exons, generating a list of highly variable exons.

[0018] This invention is not limited to the above-described embodiments. Any modifications or improvements based on the technical solutions of this invention should be included within the scope of protection of this invention.

Claims

1. A method for the serialization and extraction of highly variable exons, characterized in that, The process includes the following steps: (1) Inputting RNA sequencing data (fastq format), a reference genome sequence (fas format), and an annotation file (gtf format); (2) Introducing a parallel computing process into the sequencing data mapping to the reference genome to generate a bam file containing base alignment information; (3) Extracting the exon base sequences and expression levels of the samples from the bam file and storing them as a compressed serialized file; (4) Parsing the genome annotation file and constructing a whole-genome exon position index list; (5) Extracting the original sequence information of all exons from the reference genome based on the exon position index. (6) Integrate exon expression data from multiple samples, screen for highly variable exons through differential expression analysis, and generate a list of highly variable exons; (7) Based on the list of highly variable exons, extract the base sequence and expression level of the target exons from the serialization file and output concise, high-information preprocessed data.

2. The method for serialization extraction of highly variable exons as described in claim 1, characterized in that: In step (2), parallel computing technology is used to accelerate the comparison process. Specifically, the comparison tool driven by the GPU is called to achieve efficient conversion from fastq to bam format.

3. The method as described in claim 1, characterized in that, The serialized format file described in step (3) is stored using binary compression. Combined with the filtering of the high-information-content exon information list, its data volume is reduced by more than 95% compared with the format evaluation before preprocessing, and it supports fast reading and subsequent analysis.

4. The method for serialization extraction of highly variable exons as described in claim 1, characterized in that: The differential expression analysis algorithm described in step (6) includes data standardization, high-variability feature screening and data dimensionality reduction steps. High-information exons are screened by calculating the coefficient of variation or dispersion index of exon expression.

Images