Whole genome map splicing method for structural variation detection

Abstract

The invention relates to the technical field of genome data analysis, in particular to a whole genome map splicing method for structural variation detection, which comprises the steps of data preprocessing, feature map generation, multi-scale splicing frame construction, local optimization module design, global consistency correction, structural variation marking and result output and storage. According to the method, the data processing efficiency is improved through a distributed computing framework and a Hash index technology, the splicing precision and efficiency are improved by adopting deep learning to extract features and combining a multi-scale splicing framework, global consistency correction is realized by utilizing a graph theory method, and structural variation is detected through a sliding window scanning algorithm. According to the method, the splicing efficiency and accuracy of large-scale genome data can be remarkably improved, and reliable support is provided for structural variation detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of bioinformatics and genomics, and specifically relates to a whole genome mapping splicing method for structural variation detection. BACKGROUND

[0002] Genomics research has developed rapidly in recent years, and the importance of structural variation detection in whole genome mapping splicing is increasingly prominent. Existing splicing methods usually face problems such as high data complexity, insufficient splicing accuracy, and low computational efficiency when dealing with large-scale genomic data. Therefore, developing an efficient and accurate whole genome mapping splicing method has become an urgent need in the field of genomics.

[0003] After searching, a path adaptive knowledge graph automatic generation method based on teaching material directory and directed hypergraph with publication number CN110287327B was found, and the publication date was April 12, 2022. This patent generates a knowledge graph with path adaptability and knowledge relevance by constructing a knowledge expression hypergraph and a knowledge expression chain graph, and splices multiple knowledge graphs to construct a subject knowledge graph. However, this technical solution is mainly aimed at the generation of knowledge graphs for teaching material directories and teaching paths, and its splicing method focuses on path correlation analysis, which has certain limitations in extracting and splicing the complex characteristics of structural variations in genomic data. In addition, this method does not involve efficient processing mechanisms for large-scale data, and may face performance bottlenecks when applied to whole genome data.

[0004] After searching, a whole field quantitative statistical distribution characterization method for precipitated phase particles in metal materials with publication number CN108226159B was found, and the publication date was November 22, 2019. This patent realizes the whole field quantitative distribution characterization of precipitated phase particles by automatically collecting and splicing metallographic structure feature maps. Although this technical solution shows high precision and efficiency in image splicing and feature extraction, its application scenario is limited to the analysis of microstructure of metal materials, and does not fully consider the sequence complexity and variation type diversity specific to genomic data. In addition, this method relies on a specific mathematical model for image fusion, which may lack flexibility and adaptability when dealing with non-imaging genomic data.

[0005] The above problems show that the existing splicing methods have the following shortcomings when processing whole genome data: first, the splicing algorithm designed for a specific field is difficult to be directly migrated to the field of genomics; second, the calculation efficiency and adaptability of the existing methods need to be improved when processing large-scale and high-complexity data; third, the in-depth analysis and targeted optimization of the characteristics of the genomic structure variation are insufficient. Therefore, the present application provides a whole genome map splicing method for structural variation detection, which aims to combine the characteristics of genomic data, optimize the accuracy, efficiency and adaptability of the splicing algorithm, and thus meet the demand of genomics research for efficient and accurate splicing methods. SUMMARY

[0006] The present application aims to provide a whole genome map splicing method for structural variation detection to solve the problems raised in the background art.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solution: a whole genome map splicing method for structural variation detection, comprising the following steps:

[0008] T1: data preprocessing;

[0009] T2: feature map generation;

[0010] T3: multi-scale splicing framework construction;

[0011] T4: local optimization module design;

[0012] T5: global consistency correction;

[0013] T6: structural variation labeling;

[0014] T7: result output and storage.

[0015] Further, in T1, a distributed computing framework is used to clean and filter the original genomic sequencing data, remove low-quality sequence fragments and background noise, and use hash index technology to quickly classify and store high-quality sequence fragments. A dynamic window segmentation algorithm is used to divide long sequences into short fragments, and a unique identifier is assigned to each short fragment.

[0016] Further, in T2, the feature information of each short fragment is extracted based on a deep learning model. Specifically, a convolutional neural network combined with a long short-term memory network is used, where the convolutional neural network is used to capture local features of the sequence, and the long short-term memory network is used to model long-range dependencies between sequences. The extracted feature information is mapped to a high-dimensional space to form a feature map, and the resolution of the feature map is dynamically adjusted according to the complexity of the input data.

[0017] Further, in the T3, a multi-scale splicing framework is designed, which is composed of multiple sub-modules, each of which is responsible for processing short fragments within a specific length range. The sub-modules work collaboratively through a shared parameter mechanism to avoid resource waste caused by repeated calculations. The core of the multi-scale splicing framework lies in the introduction of an adaptive weight distribution strategy, which dynamically adjusts the splicing priority according to the similarity between short fragments, thereby improving the splicing efficiency.

[0018] Further, in the T4, a local optimization module is designed based on the multi-scale splicing framework to further improve the splicing accuracy. The local optimization module adjusts the splicing boundaries of adjacent short fragments through an iterative matching algorithm to ensure the continuity and accuracy of the splicing results. The matching algorithm uses a bidirectional comparison verification mechanism to verify the rationality of the splicing boundaries from two directions and fine-tunes the boundary positions according to the verification results. The local optimization module also introduces an error correction mechanism based on Bayesian inference to repair errors that may occur during the splicing process.

[0019] Further, in the T5, global consistency correction is performed on the spliced whole genome map to eliminate global deviations caused by the accumulation of local splicing errors. The global consistency correction uses a graph-based method to represent the splicing results as a weighted undirected graph, where nodes represent short fragments and edge weights reflect the similarity between short fragments. By minimizing the total weight of the graph, the optimal global splicing path is found to achieve consistency correction. The correction process supports incremental updates.

[0020] Further, in the T6, based on the spliced whole genome map, a sliding window scanning algorithm is used to detect potential structural variation regions. The size of the sliding window is dynamically adjusted according to the complexity of the genomic data to balance detection accuracy and computational efficiency. The detected structural variation regions are verified by statistical testing methods to exclude false positive results. The verified structural variation regions are marked as candidate variation sites and their positions, types, and possible biological significance are recorded.

[0021] Further, in the T7, the spliced whole genome map and its annotated structural variation information are output to a designated storage medium, supporting multiple formats of file export, including FASTA, GFF3, and other common formats. The output file contains detailed metadata information, such as splicing parameter settings, splicing timestamps, and splicing quality evaluation indicators. The output file can be protected through encryption algorithms.

[0022] Further, in the T1, the distributed computing framework is composed of multiple nodes, each of which is responsible for processing a part of data to achieve parallel computing, and the removal of low-quality sequence fragments and background noise depends on a set quality threshold, data below the threshold will be marked as invalid and removed, and the window size of the dynamic window segmentation algorithm is dynamically adjusted according to the sequence complexity.

[0023] Further, in the T3, the core of the adaptive weight allocation strategy lies in the similarity calculation module, which evaluates the similarity between short fragments based on the cosine similarity algorithm and dynamically adjusts the splicing priority according to the similarity, and the adjustment of the splicing priority is realized through the weighted average algorithm, and the weight value is allocated after the similarity score is normalized.

[0024] Compared with the prior art, the beneficial effects of the structural variation detection whole genome mapping splicing method are as follows: the structural variation detection whole genome mapping splicing method adopts a distributed computing framework to clean and filter original genomic sequencing data, removes low-quality sequence fragments and background noise, simultaneously utilizes a hash index technology to quickly classify and store high-quality sequence fragments, divides long sequences into a plurality of short fragments through a dynamic window segmentation algorithm, and allocates a unique identifier to each short fragment, facilitating accurate positioning in subsequent splicing operations. Based on a deep learning model, feature information of each short fragment is extracted, and a method combining a convolutional neural network (CNN) and a long short-term memory network (LSTM) is specifically adopted, wherein the CNN is used to capture local features of the sequence, and the LSTM is used to model long-range dependence between sequences. The extracted feature information is mapped to a high-dimensional space to form a feature map for describing biological characteristics of each short fragment. A multi-scale splicing framework is designed, the framework is composed of a plurality of sub-modules, each sub-module is responsible for processing short fragments within a specific length range, the sub-modules work cooperatively through a shared parameter mechanism to avoid resource waste caused by repeated calculation, and the core of the multi-scale splicing framework lies in introducing an adaptive weight distribution strategy to dynamically adjust splicing priorities according to the similarity between short fragments, so as to improve splicing efficiency. On the basis of the multi-scale splicing framework, a local optimization module is designed to further improve splicing accuracy. The local optimization module adjusts the splicing boundary of adjacent short fragments through an iterative matching algorithm to ensure the continuity and accuracy of the splicing result. The matching algorithm adopts a bidirectional comparison verification mechanism, that is, the rationality of the splicing boundary is verified from two directions, and the boundary position is fine-tuned according to the verification result. The local optimization module also introduces an error correction mechanism based on Bayesian inference to repair errors that may occur in the splicing process. Global consistency correction is performed on the spliced whole genome map to eliminate global deviation caused by accumulation of local splicing errors. The global consistency correction adopts a graph theory-based method to represent the splicing result as a weighted undirected graph, wherein the nodes represent short fragments, and the weight of the edge reflects the similarity between short fragments. By minimizing the total weight of the graph, the optimal global splicing path is found to realize consistency correction. Based on the spliced whole genome map, a sliding window scanning algorithm is used to detect potential structural variation regions. The size of the sliding window is dynamically adjusted according to the complexity of the genomic data to balance detection accuracy and computational efficiency. The detected structural variation regions are verified by a statistical test method to exclude false positive results. The verified structural variation regions are marked as candidate variation sites, and their positions, types and possible biological significance are recorded.The spliced whole genome map and its annotated structural variation information are output to a designated storage medium, supporting file export in multiple formats, including FASTA, GFF3 and other common formats, and the output file contains detailed metadata information, such as splicing parameter settings, splicing timestamp and splicing quality evaluation index, facilitating subsequent analysis and tracing, and the output file can be protected by encryption algorithm to prevent unauthorized access. DETAILED DESCRIPTION

[0025] The present application provides a whole genome map splicing method for structural variation detection, and the specific implementation is as follows. First, in the data preprocessing stage, i.e. step T1, the original genome sequencing data is cleaned and filtered through a distributed computing framework. The distributed computing framework is composed of multiple nodes, each node is responsible for processing a part of data to realize parallel computing. The removal of low-quality sequence fragments and background noise depends on the set quality threshold, and the data below the threshold will be marked as invalid and removed. High-quality sequence fragments are quickly classified and stored through hash index technology, and the design of hash table solves the conflict problem by using open address method, ensuring efficient access to each sequence. The dynamic window segmentation algorithm is used to divide long sequences into short fragments, and the window size is dynamically adjusted according to the sequence complexity, for example, a smaller window is used for regions with more repetitive sequences, and a larger window is used for simple sequence regions. The process of assigning a unique identifier to each short fragment is based on a global counter, which generates a new identifier each time the counter is incremented, ensuring the uniqueness and continuity of the identifier.

[0026] In the feature map generation stage, i.e. step T2, the specific implementation of the deep learning model includes the combined use of convolutional neural network (CNN) and long short-term memory network (LSTM). The CNN part is composed of multiple convolutional layers and pooling layers, and the convolution kernel size is set to 3x3 to capture local features, and the pooling layer uses the maximum pooling operation to reduce the feature dimension. The LSTM part is composed of two hidden layers, each containing 128 units, which are used to model the long-range dependence between sequences. In the feature extraction process, the input sequence is first converted to a fixed-length vector representation through the embedding layer, and then sequentially passes through the CNN and LSTM modules. The generation of high-dimensional feature maps is completed through fully connected layers, and the feature map resolution is dynamically adjusted according to the complexity of the input data, for example, higher resolution is used for high complexity data to retain more detailed information. The storage of feature maps uses sparse matrix format to save memory space, while supporting fast access.

[0027] In the multi-scale stitching framework construction phase, i.e., step T3, the framework is composed of multiple sub-modules, each responsible for processing short fragments within a specific length range. The sharing parameter mechanism between sub-modules is implemented through global variables, and all related sub-modules are updated synchronously to avoid repeated calculations. The core of the adaptive weight allocation strategy is the similarity calculation module, which evaluates the similarity between short fragments based on the cosine similarity algorithm and dynamically adjusts the stitching priority according to the similarity. The adjustment of the stitching priority is realized through the weighted average algorithm, and the weight value is allocated after the similarity score is normalized. The multi-scale stitching framework supports parallel processing, with each sub-module running in an independent thread, and data exchange is realized through a message queue, significantly shortening the stitching time.

[0028] In the local optimization module design phase, i.e., step T4, the iterative matching algorithm adjusts the stitching boundaries of adjacent short fragments to improve the stitching accuracy. The specific implementation of the bidirectional comparison verification mechanism includes two processes: forward verification and reverse verification. Forward verification checks the rationality of the stitching boundary from left to right, and reverse verification performs the same operation from right to left. The fine-tuning of the boundary position is based on the interpolation algorithm, and the interpolation step is dynamically adjusted according to the verification result. The Bayesian inference error correction mechanism evaluates the likelihood of stitching errors through a probability model, and the error correction process is based on the maximum a posteriori probability criterion. The reliability of the correction result is verified through the confidence score.

[0029] In the global consistency correction phase, i.e., step T5, the graph-based method represents the stitching result as a weighted undirected graph, where nodes represent short fragments and edge weights reflect the similarity between short fragments. The construction process of the weighted undirected graph includes three steps: node initialization, edge weight calculation, and graph structure generation. Minimizing the total weight of the graph is realized through the shortest path algorithm, such as Dijkstra's algorithm or Floyd-Warshall's algorithm, to find the optimal global stitching path. The incremental update mechanism is realized through differential calculation, and the newly added data only affects the weights of related nodes and edges, without the need to recalculate the entire graph structure, thereby quickly adjusting the stitching result.

[0030] In the structural variant marker phase, i.e., step T6, the specific implementation of the sliding window scanning algorithm includes three steps: window initialization, feature extraction, and variant detection. The window size is dynamically adjusted according to the complexity of the genomic data, for example, a smaller window is used for high-complexity regions to improve detection accuracy, while a larger window is used for low-complexity regions to improve computational efficiency. The detection of potential structural variant regions is completed through statistical test methods, such as t-test or chi-square test, and the exclusion of false positive results is realized based on multiple hypothesis testing correction methods. The marking process of candidate variant sites includes position recording, type annotation, and biological significance analysis, and the relevant information is stored in a structured database for subsequent query and analysis.

[0031] In the result output and storage stage, i.e., step T7, the completed whole genome map and its annotated structural variation information are output to the designated storage medium through the file export module. The file export module supports multiple formats of file export, such as FASTA and GFF3 formats, and the export process of each format includes three steps of data parsing, format conversion, and file writing. The output file contains detailed metadata information, such as splicing parameter settings, splicing timestamps, and splicing quality evaluation indicators, etc. These information are generated by the log recording module and attached to the output file. The encryption algorithm uses the AES-256 bit encryption standard to protect the output file, and the key management is realized through the hardware security module to ensure that unauthorized access cannot decrypt the file content.

[0032] In order to better enable relevant persons in the technical field to fully understand and implement the present application, the specific implementation principles of the present application are further supplemented below in conjunction with a specific application scenario.

[0033] In genomics research, a research team needs to splice the whole genome map of a certain complex plant and detect possible structural variations. The plant genome has high complexity, containing a large number of repetitive sequences and a high diversity of variation types. The researchers decide to use the method provided by the present application to complete the task.

[0034] First, in the data preprocessing stage, i.e., step T1, the original sequencing data is loaded onto multiple computing nodes through a distributed computing framework. Each node independently runs cleaning and filtering algorithms to mark and remove low-quality sequence fragments as invalid. For example, for some sequence fragments with sequencing quality lower than Q20, the system automatically considers them as noise and removes them from the data set. High-quality sequence fragments are quickly stored through hash indexing technology. The hash table is designed using the open address method to solve the conflict problem, ensuring efficient access to each sequence. The dynamic window segmentation algorithm adjusts the window size according to the sequence complexity, for example, in areas with more repetitive sequences, the window size is set to 500 bp, while in simple sequence areas, it is expanded to 1000 bp. In addition, a global counter assigns a unique identifier to each short fragment, and this process is implemented through an incremental mechanism to ensure the continuity and uniqueness of the identifier.

[0035] Subsequently, the input sequence is converted into a fixed-length vector representation by an embedding layer, and then passed through a CNN and an LSTM module to extract feature information. The 3x3 convolution kernel of the CNN part captures local features, and the max-pooling operation reduces the feature dimension to improve computational efficiency; the LSTM part models the long-range dependencies between sequences, which is completed by two hidden layers. The resolution of the feature map is dynamically adjusted according to the complexity of the input data, for example, for high complexity regions, the system generates higher resolution feature maps to retain more detailed information. These high-dimensional feature maps are stored in sparse matrix format, not only saving memory space, but also supporting fast access.

[0036] In the multi-scale splicing framework construction stage, multiple sub-modules work together, each sub-module is responsible for processing short fragments within a certain length range. For example, a sub-module is dedicated to processing short fragments between 500bp and 1000bp, while another sub-module processes 1000bp to 2000bp fragments. Parameters are shared between sub-modules through global variables, avoiding resource waste caused by repeated calculations. The adaptive weight allocation strategy evaluates the similarity between short fragments based on the cosine similarity algorithm, and dynamically adjusts the splicing priority according to the similarity score. The adjustment of splicing priority is realized through weighted average algorithm, and the weight value is normalized and allocated to each short fragment. The multi-scale splicing framework supports parallel processing, each sub-module runs in an independent thread and exchanges data through a message queue, thereby significantly shortening the splicing time.

[0037] Next, in the local optimization module design stage, the iterative matching algorithm adjusts the splicing boundary of adjacent short fragments to improve the splicing accuracy. The bidirectional comparison verification mechanism verifies the rationality of the splicing boundary from two directions, for example, the forward verification checks whether the boundary meets the expected value from left to right, and the reverse verification performs the same operation from right to left. The fine-tuning of the boundary position is based on the interpolation algorithm, and the interpolation step is dynamically adjusted according to the verification result. The Bayesian inference error correction mechanism evaluates the possibility of splicing errors through a probability model, and the error correction process is based on the maximum a posteriori probability criterion, and the correction result is verified by the confidence score for its reliability.

[0038] In the global consistency correction stage, the splicing result is represented as a weighted undirected graph, where the nodes represent short fragments and the weights of the edges reflect the similarity between short fragments. For example, for two highly similar short fragments, the edge weight will be set to a lower value to reflect their higher matching priority. Minimizing the total weight of the graph is achieved through Dijkstra's algorithm, which finds the optimal global splicing path. The incremental update mechanism is realized through differential calculation, and the new data only affects the weights of the relevant nodes and edges, without the need to recalculate the entire graph structure, thereby quickly adjusting the splicing result.

[0039] In the structural variation marker stage, i.e., step T6, the sliding window scanning algorithm dynamically adjusts the window size according to the complexity of the genomic data. For example, in high complexity regions, the window size is set to 200 bp to improve detection accuracy, while in low complexity regions, it is expanded to 500 bp to improve computational efficiency. The detection of potential structural variation regions is completed by t-test or chi-square test, and false positive results are excluded by multiple hypothesis testing correction methods. The location, type and biological significance of the candidate variation sites are recorded in the structured database for subsequent query and analysis.

[0040] Finally, in the result output and storage stage, i.e., step T7, the completed whole genome map and its annotated structural variation information are output to the designated storage medium through the file export module. For example, researchers choose FASTA format to export the spliced results, and the export process of each format includes three steps of data parsing, format conversion and file writing. The output file contains detailed metadata information, such as splicing parameter settings, splicing timestamp and splicing quality evaluation indicators, etc. These information are generated by the log recording module and attached to the output file. The encryption algorithm uses AES-256 bit encryption standard to protect the output file, and the key management is realized through the hardware security module, ensuring that unauthorized access cannot decrypt the file content.

[0041] Through the implementation of the above specific application scenarios, the method of the present application successfully solves the problems of complexity, insufficient precision and low efficiency in large-scale genomic data splicing, realizes efficient and accurate whole genome map splicing, and meets the actual needs of genomics research.

Claims

1. A whole genome map assembly method for structural variant detection, characterized by, The method comprises the following steps: T1: data preprocessing; T2: feature map generation; T3: multi-scale splicing framework construction; T4: local optimization module design; T5: global consistency correction; T6: structural variation marking; T7: result output and storage.

2. The whole genome map assembly method for structural variation detection according to claim 1, wherein: In T1, a distributed computing framework is used to clean and filter the original genomic sequencing data, remove low-quality sequence fragments and background noise, and use a hash index technology to quickly classify and store high-quality sequence fragments. A dynamic window segmentation algorithm is used to divide long sequences into short fragments, and each short fragment is assigned a unique identifier.

3. The whole genome haplotyping method for structural variation detection according to claim 2, wherein: In T2, the feature information of each short fragment is extracted based on a deep learning model. Specifically, a convolutional neural network combined with a long short-term memory network is used. The convolutional neural network is used to capture local features, and the long short-term memory network is used to model long-range dependencies between sequences. The extracted feature information is mapped to a high-dimensional space to form a feature map, and the resolution of the feature map is dynamically adjusted according to the complexity of the input data.

4. The method according to claim 3, wherein: In T3, a multi-scale splicing framework is designed, which consists of multiple sub-modules. Each sub-module is responsible for processing short fragments within a specific length range. The sub-modules work together through a shared parameter mechanism to avoid resource waste caused by repeated calculations. The core of the multi-scale splicing framework is the introduction of an adaptive weight distribution strategy, which dynamically adjusts the splicing priority based on the similarity between short fragments, thereby improving splicing efficiency.

5. The method of claim 4, wherein the method is a whole genome structural variation detection de novo assembly method. In T4, a local optimization module is designed based on the multi-scale splicing framework to further improve splicing accuracy. The local optimization module uses an iterative matching algorithm to fine-tune the splicing boundaries of adjacent short fragments to ensure the continuity and accuracy of the splicing results. The matching algorithm uses a bidirectional comparison verification mechanism to verify the rationality of the splicing boundaries from two directions and adjusts the boundary positions based on the verification results. The local optimization module also introduces a Bayesian inference-based error correction mechanism to repair errors that may occur during splicing.

6. The method of claim 5, wherein: In T5, global consistency correction is performed on the spliced whole genome map to eliminate global deviations caused by the accumulation of local splicing errors. The global consistency correction uses a graph theory-based method to represent the splicing results as a weighted undirected graph, where nodes represent short fragments and edge weights reflect the similarity between short fragments. The optimal global splicing path is found by minimizing the total weight of the graph, thereby achieving consistency correction. The correction process supports incremental updates.

7. The method of claim 6, wherein the method further comprises: In T6, based on the completed whole genome map, a sliding window scanning algorithm is used to detect potential structural variation regions. The size of the sliding window is dynamically adjusted according to the complexity of the genomic data to balance detection accuracy and computational efficiency. The detected structural variation regions are verified by statistical testing methods to exclude false positive results. The verified structural variation regions are marked as candidate variation sites and their positions, types, and possible biological significance are recorded.

8. The whole genome haplotyping method of structural variation detection according to claim 7, wherein: In the T7, the completed whole genome map and its annotated structural variation information are output to a designated storage medium, supporting file export in multiple formats, including FASTA, GFF3, and other commonly used formats. The output file contains detailed metadata information, such as splicing parameter settings, splicing timestamps, and splicing quality evaluation indicators. The output file can be protected by encryption algorithms.

9. The whole genome haplotyping method of structural variation detection according to claim 8, wherein: In the T1, the distributed computing framework is composed of multiple nodes, each responsible for processing a portion of data to achieve parallel computing. The removal of low-quality sequence fragments and background noise relies on a set quality threshold. Data below this threshold is marked as invalid and removed. The window size of the dynamic window segmentation algorithm is dynamically adjusted according to the sequence complexity.

10. The whole genome structural variation detection method of mapping contigs according to claim 9, wherein: In the T3, the core of the adaptive weight allocation strategy is the similarity calculation module. This module evaluates the similarity between short fragments based on the cosine similarity algorithm and dynamically adjusts the splicing priority according to the similarity. The adjustment of splicing priority is achieved through a weighted average algorithm, and the weight value is allocated after normalizing the similarity score.