Analysis method and device based on sequencing data dimension reduction
By using two sample* sample matrix and multi-threaded processing methods in sequencing data analysis, the memory usage and computing efficiency are optimized, and the problems of low efficiency and large memory usage in the existing technology are solved, and the problem of low efficiency and large memory usage is realized. Rapid analysis of group grouping and evolutionary relationships.
Patent Information
- Application Number
- CN202310587343.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
The existing dimensionality reduction method based on sequencing data is inefficient, has too much memory usage and cannot be visualized, making it difficult to quickly and efficiently analyze group grouping and evolutionary relationships.
Use the matrix storage information of two samples* samples, and release the site information while reading, only the array of two samples* samples is retained in memory. The matrix operation is optimized through multi-threaded parallel processing and the difference value mapping table to reduce memory Requirements and calculation time.
It significantly reduces memory usage and user waiting time, improves data dimensionality reduction and analysis efficiency, supports fast group grouping and evolutionary relationship analysis, and can handle large data volume on ordinary devices.
Smart Images

Figure BDA0004244293080000031 
Figure BDA0004244293080000032 
Figure BDA0004244293080000081
Abstract
Description
Technical Field
[0001] The present invention relates to the field of bioinformatics, and more specifically, to an analysis method based on dimensionality reduction of sequencing data. More specifically, the present invention relates to a method and apparatus for determining the kinship between samples, a method and apparatus for analyzing sequencing results of multiple samples, a computing device, and a computer-readable storage medium. Background Art
[0002] Whole genome resequencing is the resequencing of individuals of species with complete reference genomes, and on this basis, the difference analysis of individuals or groups is performed. Through whole genome resequencing, a large amount of genomic variation information can be obtained, including single nucleotide polymorphism sites (SNP), insertion / deletion sites (InDel), structure variation sites (SV) and copy number variation sites (CNV). In the population evolution relationship analysis of resequencing, it is necessary to distinguish different subpopulations in complex samples through data dimensionality reduction and infer the evolutionary relationship between subpopulations.
[0003] To solve the problem of data dimensionality reduction, many tools have been developed, including GCTA, GAPIT, and plink. These tools use principal component analysis to reduce the dimensionality of data to facilitate subsequent analysis. However, these tools have some disadvantages, such as low efficiency and excessive memory usage.
[0004] Therefore, there is an urgent need in this field to develop an analysis method based on dimensionality reduction of sequencing data that can perform population clustering and evolutionary relationship analysis quickly, efficiently and with low memory. Summary of the invention
[0005] This application is filed by the inventor based on the discovery of the following problems and facts:
[0006] In view of the problems existing in the current traditional dimensionality reduction methods, such as high memory requirements, slow matrix operation speed, inability to visualize (such as GCTA) and insufficient analysis (such as GAPIT), further improvement and perfection are still needed. Therefore, the inventor innovatively used the data of two matrices (sample * sample) to store information. When reading each site, some information is recorded at the same time, and then the information of the site in the row is released in time to read the information of the new row. By reading and calculating while releasing the site information, the influence of the number of sites on the memory is stripped, that is, only two (sample * sample) array data reside in the memory, which significantly reduces the amount of memory required for operation and reduces the waiting time for users.
[0007] The present invention aims to solve one of the above technical problems at least to a certain extent.
[0008] To this end, in a first aspect of the present invention, the present invention proposes a method for determining the kinship between samples. According to an embodiment of the present invention, the method comprises: obtaining gene information of multiple samples at multiple predetermined sites; for at least a portion of the multiple predetermined sites, based on the gene information differences between the samples at each predetermined site, obtaining a first matrix for each predetermined site; and determining the kinship between the multiple samples based on the first matrix of the multiple predetermined sites.
[0009] It should be noted that the multiple predetermined sites include one or more predetermined sites in multiple samples.
[0010] The predetermined site refers to a specific position on the genome, which may be associated with a known gene or other biological function. By obtaining the gene information at this position, it is possible to better understand whether there is a mutation or other change at this position.
[0011] The genetic information refers to the inference of the kinship between multiple samples by comparing the gene sequences or variations between samples. In this analysis, researchers usually select some specific gene loci that are known to be related to kinship, such as STR (short string repeat) markers, single nucleotide polymorphisms (SNPs), mitochondrial DNA, etc., which are used to determine the relationship between parents and children. By comparing the genotypes or sequences of these sites, the genetic similarity between the research objects can be obtained, thereby judging the kinship between them. For example, two people with the same STR marker are likely to be father and son or siblings, while two people with different STR markers are less likely to be related. The source of the genetic information can come from population resequencing data or spatiotemporal single-cell expression data, etc.
[0012] At least a portion of the plurality of predetermined sites indicates that one predetermined site, a plurality of predetermined sites or all predetermined sites may be used to calculate the gene information difference between each predetermined site and obtain an initial kinship matrix.
[0013] The first matrix of the plurality of predetermined sites is obtained based on the differences in sequencing results between samples at each predetermined site.
[0014] According to the embodiment of the present invention, the above method is used to construct a kinship matrix for massive gene information, reduce the dimension of the data, and facilitate result analysis. In addition, the memory usage can be significantly reduced, and the time for users to wait for calculations can be reduced.
[0015] According to an embodiment of the present invention, the above method for determining the kinship between samples may further include at least one of the following additional technical features:
[0016] According to an embodiment of the present invention, the first matrix of each predetermined site is obtained by the following steps: writing the gene information of the multiple samples at the predetermined site into a computer-readable storage medium; and calculating the first matrix based on the predetermined site based on the gene information.
[0017] According to an embodiment of the present invention, the kinship is determined by superimposing the first matrices of the plurality of predetermined sites. According to an embodiment of the present invention, superimposing the first matrices comprises superimposing each first matrix obtained for the plurality of predetermined sites.
[0018] For example, assuming that the genotypes are AA, CC and heterozygous AC, 0 represents AA, 2 represents CC and 1 represents AC. If the genotypes of four samples (one locus) are [0, 1, 2, 0], the degree of difference is defined as the absolute value of the difference between the two genes. Then, the genotypes of the first and second samples are 0 (AA) and 1 (AC), the difference degree is abs (0-1), recorded as 1; the genotypes of the first and third samples are 0 (AA) and 2 (CC), the difference degree is abs (0-2), recorded as 2; the genotypes of the first and fourth samples are 0 (AA) and 0 (AA), the difference degree is abs (0-0), recorded as 0; the genotypes of the second and third samples are 1 (AC) and 2 (CC), the difference degree is abs (1-2), recorded as 1; the genotypes of the second and fourth samples are 1 (AC) and 0 (AA), the difference degree is abs (1-0), recorded as 1; the genotypes of the third and fourth samples are 2 (CC) and 0 (AA), the difference degree is abs (2-0), recorded as 2; among them, if the genotypes are the same, that is, there is no difference, it is recorded as 0.
[0019] You can get the matrix IBS kinship, 4*4 (difference matrix p_dis)
[0020]
[0021] Assuming there are 5 loci and the genotypes are consistent, the final matrix obtained is as follows:
[0022]
[0023] According to an embodiment of the present invention, the data of the first matrix of each predetermined site is stored in the second matrix in the form of two-dimensional data.
[0024] According to an embodiment of the present invention, the second matrix is selected from at least one of a two-dimensional data table, an array or a data frame.
[0025] According to an embodiment of the present invention, the second matrix is a two-dimensional data table.
[0026] According to an embodiment of the present invention, the second matrix is determined based on genotype difference values between multiple samples.
[0027] Exemplarily, the first matrix is converted into a data format that can be processed by the program, and operations and calculations can be performed in the program through the second matrix (such as using a temporary two-dimensional data table data structure), thereby reducing the need to record results that need to be repeatedly calculated, mapping related results in one step, reducing the amount of calculations, improving efficiency, and reducing CPU time.
[0028] For example, when calculating the data of one hundred samples (i.e. A1, A2...A50, B1, B2...B50), it is necessary to calculate the difference between any two samples when using matrix calculation, and the two-dimensional data table can store the completed calculation results in the second matrix (difference matrix). For example, when calculating the difference between A1 and A3 (calculating the difference between two samples, that is, whether the genotypes are the same, where 0 represents AA, 2 represents CC and 1 represents AC), it is found that A3 and A2 are the same, then the calculation results of A1 and A2 can be used directly.
[0029] According to an embodiment of the present invention, the first matrix of the plurality of predetermined sites is obtained by multi-thread or multi-process parallel processing.
[0030] It should be noted that the multithreading is based on the idle threads of the current system, not the maximum number of threads. In other words, the user does not feel the computing node stuck, and the maximum number of threads is used as much as possible, rather than the user specifying how many threads (CPUs) to run this program. The multi-process parallel processing refers to the technology of using multiple CPU cores of a computer to process different tasks simultaneously. In this way, each task is assigned to an independent process to run, and each process can access system resources and memory space independently. In this way, each process has its own independent address space and state.
[0031] According to an embodiment of the present invention, obtaining the first matrix of each predetermined site includes: for one predetermined site, calculating the genotype differences between the multiple samples in parallel through multi-threading or multi-process; and determining the initial kinship of the predetermined site based on the genotype differences between the multiple samples.
[0032] According to an embodiment of the present invention, the determining the kinship between the plurality of samples based on the first matrix of the plurality of predetermined sites comprises: initializing a third matrix (accumulation matrix) in a computer-readable storage medium, wherein the initial value of each element in the third matrix is 0;
[0033] For each first matrix of a predetermined site, the following steps are performed: writing the first matrix of the predetermined site into the computer-readable storage medium, accumulating the written first matrix into the third matrix, and releasing the storage space occupied by the written first matrix.
[0034] According to an embodiment of the present invention, the calculation is performed by assigning values to gene information, and the calculation is performed by calculating the difference values in the first matrix through the stored difference values.
[0035] Specifically, the stored difference values can be selectively stored in a difference value mapping table (or can be stored in other arrays, data frames or two-dimensional data that can achieve similar functions), and the difference value mapping table includes multiple difference values (the difference values include difference values preset in advance or calculated based on existing samples), and each difference value corresponds to an index (used to quickly locate the required data elements); obtain the genotype values of the two samples to be calculated at the predetermined sites; use the genotype values (such as 0, 1, 2, etc.) as indexes to obtain the difference values of the two samples to be calculated at the predetermined sites from the difference value mapping table.
[0036] For example, for a certain site (assuming there are 100 samples), if the genotype of the first 50 samples is AA and the second 50 samples are CC, then the genotype is (0, 0, 0...0, 1, 1, 1...1) with 50 0s in the first and 1 in the last. Usually, when calculating the difference between every two samples, abs(0-0) is used (taking every two samples in the first 50 samples as an example) to calculate the difference value, that is, abs(0-0) is run once for each comparison, which is recorded as Table[0][0]=0 in the two-dimensional data table; the genotype values of the two samples are used as the two subscript values of the table, so each time you only need to check: table[gene value of sample 1][gene value of sample 2] to know.
[0037] Specifically, the table of the simplest second matrix (difference matrix) is as follows;
[0038] table[0][0]=table[1][1]=table[2][2]=0;
[0039] table[0][1]=table[1][0]=table[1][2]=table[2][1]=1;
[0040] table[0][2]=table[2][0]=2.
[0041] It should be noted that the gene information can be stored in VCF format. VCF is a standard format for storing variant sites. The site information, quality control, filtering of low frequency, three bases and sites that do not conform to the Hardy-Weinberg law are read line by line from the VCF format file. Sites that fail quality control do not participate in the subsequent calculation process, avoiding problems such as additional format conversion during the filtering process.
[0042] In the second aspect of the present invention, the present invention proposes a method for analyzing gene information of multiple samples. According to an embodiment of the present invention, the method includes: obtaining gene information of multiple samples at multiple predetermined sites; determining the kinship between samples according to the method of the first aspect of the present invention; and performing at least one of principal component analysis and cluster analysis on the gene information based on the kinship between samples.
[0043] According to an embodiment of the present invention, the method of performing data dimension reduction and cluster analysis on multi-sample gene information using the above method can save the user's memory usage, and can also perform fast analysis without relying on high-configuration electronic devices or servers, saving analysis waiting time.
[0044] It should be noted that the method may also include a step of visualizing the analysis results. Visualizing the analysis results of sequencing data can convert complex data into easy-to-understand and interpretable graphs or charts to facilitate understanding and interpretation of the data; it can also make it easier to discover hidden associations and trends in the data; and the analysis results become more vivid and intuitive.
[0045] In the third aspect of the present invention, the present invention proposes a device for determining kinship between samples. According to an embodiment of the present invention, the device includes: a data collection unit, the data collection unit is used to obtain gene information of multiple samples at multiple predetermined sites; and a relationship matrix construction unit, the relationship matrix construction unit is connected to the data collection unit, and is used to obtain multiple initial kinship matrices based on the gene information differences between each of the samples for at least a part of the multiple predetermined sites; and a kinship determination unit, the kinship determination unit is connected to the relationship matrix construction unit, and is used to determine the kinship between the multiple samples based on the multiple initial kinship matrices.
[0046] According to an embodiment of the present invention, the device can accurately determine the kinship between the samples, and can automatically perform dimensionality reduction analysis on large amounts of genetic information, and is suitable for analysis of big data (such as population resequencing data, spatiotemporal expression data analysis, etc.).
[0047] It should be noted that, from a structural point of view, Figure 1As shown, the data collection unit S001 is connected to the relationship matrix construction unit S002, and the relationship matrix construction unit S002 is connected to the kinship determination unit S003.
[0048] In a fourth aspect of the present invention, the present invention proposes a system for analyzing gene information of multiple samples. According to an embodiment of the present invention, the device comprises: a data collection device, the data collection unit device is used to obtain gene information of multiple samples at multiple predetermined sites; and
[0049] The device for determining kinship between samples described in the second aspect of the present invention is connected to the data collection device and is used to determine the kinship between samples; and an analysis device, which is connected to the device for determining kinship between samples and is used to perform at least one of principal component analysis and cluster analysis on the genetic information based on the kinship between samples.
[0050] According to the embodiments of the present invention, the system is efficient and accurate, automated (no human intervention is required), diverse (the types of data and sample types that can be analyzed are diverse), and flexible (specific sites can be selected for kinship analysis). In the actual analysis process of bioinformatics, principal component analysis (PCA) can also be performed on gene information to obtain multiple principal components to represent its most important expression pattern.
[0051] It should be noted that if Figure 2 As shown, structurally, the data collection device S100 is connected to the device for determining kinship between samples S200 (including a data collection unit S001, a relationship matrix construction unit S002 and a kinship determination unit S003), and the device for determining kinship between samples S200 is connected to the analysis device S400.
[0052] It should be noted that the system may further include a visualization device, which is connected to the analysis device and is used to visualize the analysis results.
[0053] It should be noted that the visualization device S500 can be selectively connected to the analysis device S400 ( Figure 3 ).
[0054] In a fifth aspect of the present invention, the present invention provides a computing device. According to an embodiment of the present invention, the computing device includes: a memory and a processor;
[0055] The memory is used to store computer programs;
[0056] The processor is used to execute the computer program to implement the method described in the first aspect and the second aspect of the present invention.
[0057] According to an embodiment of the present invention, the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement a method for determining the kinship between samples and / or a method for analyzing sequencing results of multiple samples.
[0058] In a sixth aspect of the present invention, the present invention provides a computer-readable storage medium. According to an embodiment of the present invention, the storage medium includes computer instructions, and when the instructions are executed by a computer, the computer implements the method described in the first aspect and the second aspect of the present invention.
[0059] It should be noted that in the present application, the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or in combination with these instruction execution systems, devices or equipment. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples of computer-readable media (non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program may be printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or processing in other suitable ways as necessary, and then stored in a computer memory. The various computer-readable storage media described herein may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.
[0060] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0062] Figure 1 is a structural diagram of a device for determining affinity between samples according to an embodiment of the present invention;
[0063] Figure 2 is a system structure diagram for analyzing sequencing results of multiple samples according to an embodiment of the present invention;
[0064] Figure 3 is a system structure diagram (including a visualization unit) for analyzing sequencing results of multiple samples according to an embodiment of the present invention;
[0065] Figure 4 is an example diagram of the VCF file format according to an embodiment of the present invention;
[0066] Figure 5 is an example diagram of the Genotype format according to an embodiment of the present invention;
[0067] Figure 6 is the optimal K value estimated by SSE and DBI according to the embodiment of the present invention;
[0068] Figure 7 It is the PCA clustering diagram according to the embodiment of the present invention; wherein the K value is 4. DETAILED DESCRIPTION
[0069] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0070] Definition and Description
[0071] As used herein, unless otherwise indicated, the singular forms "a," "an," and the like include plural referents (more than one); "a set" or "a plurality" refers to two or more.
[0072] In this document, unless otherwise specified, the terms “comprise” or “include” are open expressions, that is, including the contents specified in the present invention but not excluding other contents.
[0073] In this document, unless otherwise specified, the terms "first", "second", "third", "fourth", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated; features specified as "first", "second", etc. may explicitly or implicitly include one or more of the said features.
[0074] In this article, the term "data dimensionality reduction" means reducing the number of variables with correlation in the data, replacing the original variables with fewer variables, so that they retain the information of the original variables as much as possible and are unrelated to each other. In practical applications, this method can retain the most important features and remove noise and unimportant features under the premise of losing a certain amount of information, thereby saving a lot of analysis time and cost, improving the speed of data processing, and facilitating understanding and analysis.
[0075] In this article, the term "Principal Components Analysis (PCA)", also known as principal component analysis or principal component regression analysis, is an unsupervised linear data dimensionality reduction method. The principal component analysis method was first introduced by Pearson for non-random variables, and later extended by Hotelling to the scenario of random vectors. When analyzing multivariate data, the PCA method first uses a linear transformation to transform the data into a new coordinate system, in which the first coordinate axis is the variable with the largest variance in the data. If the first coordinate axis cannot fully represent the original information, the second coordinate axis is selected. The second coordinate axis is selected with the largest variance in the plane orthogonal to the first coordinate axis, and the third axis is the largest variance in the plane orthogonal to the first and second axes. And so on, until the variance of the new coordinate axis is almost zero, n such coordinate axes can be obtained.
[0076] In this article, Figure 4 As shown, the sequencing result is presented in VCF format, encoded using UTF-8, and includes two parts: the first part is annotation information, including VCF version information, code information, chromosome length and other information; the second part is site information, including chromosome (CHROM), genome position (POS), variant site rsID number (ID), sites that are the same as the reference genome (REF), sites that are different from the reference genome (ALT), site quality number (QUAL), variant site filtering information (FILTER), additional information (INFO), etc.
[0077] Existing PCA analysis tools include: GCTA, GAPIT and plink, etc. This application takes GCTA and GAPIT analysis tools as examples for detailed description:
[0078] GCTA (Genome-wide Complex Trait Analysis) is a software package developed to estimate the proportion of phenotypic variation produced by all genome-wide SNPs of a complex trait, and has since been widely used in population genetics research. The PCA method used by the GCTA toolkit is EIGENSTRAT. The EIGENSTRAT method uses principal component analysis to explicitly model the differences between the treatment and control lines, and corrects for frequency changes in candidate markers between different lines, thereby maximizing the detection of true interspecies associations. The PCA analysis steps of the GCTA toolkit are as follows:
[0079] (1) Before analysis, the raw data in VCF format needs to be converted into binary files, and the association analysis software plink needs to be used to generate files in map, ped, bed, bim, fam and other formats;
[0080] (2) GCTA converts the binary file into a matrix file (sample*site) and performs PCA analysis;
[0081] (3) The obtained matrix can be visualized and analyzed using R language.
[0082] GAPIT (Genome Association and Prediction Integrated Tool) is a toolkit compiled based on the R language platform that integrates a variety of genome-wide association analysis and genome-wide selection algorithms. GAPIT uses the prcomp function for PCA analysis. Unlike the conventional PCA algorithm based on the eigenvalue decomposition covariance matrix, prcomp is based on the SVD decomposition covariance matrix. prcomp also requires the input file to be a [sample*site] matrix, so the original VCF format file needs to be converted into a matrix in advance.
[0083] Based on the analysis of the above existing PCA analysis tools, the inventors found that the above analysis tools have the following defects:
[0084] (1) The aforementioned analysis tools (e.g., GCTA, GAPIT, and plink) require quality control processing of input files and conversion into a specified format. For example, some software requires pre-filtering of VCF (Variant Call Format) three-base sites, thus requiring users to have a basic knowledge of bioinformatics analysis.
[0085] (2) The aforementioned analysis tools (such as GCTA, GAPIT, and plink, etc.) all calculate the [sample*sample] kinship matrix and PCA results based on the [sample*site] matrix, and need to read the data at one time. In population resequencing, millions of sites can usually be detected, and the VCF file size usually exceeds 100G. A one-time reading requires dozens or even hundreds of GB of memory. Therefore, these analyses can only be performed on high-performance clusters. While the memory requirements are very high, the existing methods often have a lot of useless repeated calculations during matrix operations, which makes the operation slow and takes a long time;
[0086] (3) Among the aforementioned analysis tools (e.g., GCTA, GAPIT, and plink), GCTA only gives the PCA results without visualization, and GAPIT only visualizes the PCA results without further analysis. Cluster analysis based on PCA results is one of the most important steps in population structure analysis.
[0087] To this end, the inventors improved the data dimensionality reduction method based on the defects of the prior art.
[0088] Specifically, if int data is used to store the genotype information of all sites of all samples in 1000 samples and 1M sites, the array size is intarry
[1000] [1000000]. By using two matrices [sample*sample] to store information, some information is recorded at each site read, and then the information of the site in that row is released in time to read the information of a new row. That is, intarryA[1000
[1000] and intarryB[1000
[1000] . These two arrays are only related to the sample size, and the memory is only affected by the sample size, that is, the memory has been stripped of the site association. Therefore, the memory occupied by these two arrays is 2*1000*1000, which is much smaller than the array of 1000*1000000, which significantly reduces the amount of memory required for operation.
[0089] Exemplarily, the kinship value is calculated by the following formula:
[0090]
[0091] Where i and j represent samples; represents the gene value of sample i; Represents the gene value of sample j.
[0092] The kinship value of samples i and j is the average of the differences in the genotypes of all sites at these two sites. For example, in the four samples at one site, the genotypes are AA, CC and heterozygous AC, and 0 represents AA, 2 represents CC and 1 represents AC. If the genotypes of the four samples (one site) are [0, 1, 2, 0], the difference is defined as the absolute value of the difference between the two genes.
[0093] Then, the genotypes of the first sample and the second sample are 0 (AA) and 1 (AC), and the degree of difference is abs (0-1), which is recorded as 1;
[0094] The genotypes of the first and third samples are 0 (AA) and 2 (CC), and the degree of difference is abs (0-2), which is recorded as 2;
[0095] The genotypes of the first and fourth samples are 0(AA) and 0(AA), and the degree of difference is abs(0-0), which is recorded as 0;
[0096] The genotypes of the second and third samples are 1 (AC) and 2 (CC), and the degree of difference is abs (1-2), which is recorded as 1;
[0097] The genotypes of the second and fourth samples are 1 (AC) and 0 (AA), and the degree of difference is abs (1-0), which is recorded as 1;
[0098] The genotypes of the third and fourth samples are 2 (CC) and 0 (AA), and the degree of difference is abs (2-0), which is recorded as 2;
[0099] If the genotypes are the same, that is, there is no difference, it is recorded as 0.
[0100] Finally, we get the matrix IBS kinship 4*4 (difference matrix p_dis)
[0101]
[0102] The inventors replaced the [sample*site] matrix with two [sample*sample] in the following way:
[0103] There are also 4 samples, but 5 loci, as shown below, so it is recorded as Genotype[4][5] matrix,
[0104]
[0105] Normally, this matrix is read into memory. According to the formula, when calculating the kinship between sample A and sample B, the absolute value of the difference is calculated for all sites of the two samples, and then the mean is calculated.
[0106] Then, when calculating the kinship of sample 1 and sample 3, the absolute value of the difference between the first and second columns above is accumulated, and then the average is taken.
[0107] That is, the absolute value of the difference between the first and third columns is used as follows
[0108]
[0109] Kinship[0][2]=[abs(0-2)+abs(0-2)+abs(0-2)+abs(0-2)+abs(0-2)] / 5=2;
[0110] Similarly, by comparing the other two samples in turn, we can finally get the following difference matrix
[0111]
[0112] The inventors used two [sample*sample] matrices, reading a line of the file each time. For example, there are 5 sites in total as shown above, and calculating while reading. When a site is read, the kinship value of the site is calculated.
[0113] For example, for the first site, we get the 4*4 kinship matrix of the site, and record it as Sitekinship.
[0114]
[0115] Then use another 4*4 kinship matrix, Sumkinship, to accumulate the Sitekinship matrices of these sites (Sumkinship + = Sitekinship). For example, after reading all 5 sites, Sumkinship has accumulated 5 Sitekinships that are the same as site 1 because the genotypes of each site are the same.
[0116]
[0117] By performing mean processing on Sumkinship, we can get the difference matrix
[0118]
[0119] The pseudo code is as follows:
[0120]
[0121] In order to shorten the waiting time of users, this application adopts two technical solutions to achieve acceleration at the same time. The specific solutions are as follows:
[0122] 1) Use a special data table to record repeated calculation results to reduce repeated calculations;
[0123] Specifically, when calculating the data of one hundred samples (i.e. A1, A2...A50, B1, B2...B50), it is necessary to calculate the difference between any two samples when using matrix calculation, and the two-dimensional data table can store the completed calculation results in the second matrix (difference matrix). For example, when calculating the difference between A1 and A3 (calculating the difference between two samples, that is, whether the genotypes are the same, where 0 represents AA, 2 represents CC and 1 represents AC), it is found that A3 and A2 are the same, then the calculation results of A1 and A2 can be used directly.
[0124] For example, for a certain locus, the genotype of the first 50 samples is AA, and the genotype of the last 50 samples is CC, so the genotype is (0, 0, 0...0, 1, 1, 1...1) with 50 0s in the first and 1 in the last. Usually, when calculating the difference between every two samples, abs(0-0) is used (taking every two samples in the first 50 samples as an example) to calculate the difference value, that is, abs(0-0) is run once for each comparison, and in the two-dimensional data table, it is recorded as Table[0][0]=0; the genotype values of the two samples are used as the two subscript values of the table, so each time you only need to check: table[gene value of sample 1][gene value of sample 2] to know.
[0125] Specifically, the table of the simplest difference matrix is as follows;
[0126] table[0][0]=table[1][1]=table[2][2]=0;
[0127] table[0][1]=table[1][0]=table[1][2]=table[2][1]=1;
[0128] table[0][2]=table[2][0]=2.
[0129] 2) Dynamic multi-threaded concurrent computing to shorten user waiting time
[0130] For example, by adopting the openMP (Chandra R, Dagum L, Menon R, et al. Parallel programming in OpenMP [M]. Morgan kaufmann, 2001.) multithreading library, the Kindship code that takes the longest time is transformed into a multithreaded one. Using the #pragma ompparallel for schedule (dynamic) command (https: / / blog.csdn.net / gengshenghong / article / details / 7000979), the program compiled with the "-fopenmp" instruction during compilation will make full use of the available threads, using dynamic (dynamic) to issue as many threads as possible based on the idle threads of the current system, that is, to make the user feel that the computing node is not stuck, and try to use the most threads, rather than the user specifying how many threads (CPU) to run this program.
[0131] In the present application, the inventors also used K-means algorithm to perform cluster analysis on the PCA results.
[0132] Specifically, for ease of understanding, the K-means algorithm used in this application to perform cluster analysis on PCA results is explained and illustrated in detail below. The specific steps of the technical solution are as follows:
[0133] The K-means algorithm, i.e., the k-means algorithm, is a common clustering algorithm. Its core goal is to divide the data set into k clusters, give the center point of each cluster, and represent it with the mean of all samples in the cluster, which is called the "centroid". Before performing K-means clustering, it is necessary to estimate the optimal k value. In cluster analysis, this method first calculates SSE (Within-cluster sum of squared errors) and DBI (Davies-Bouldin Index) to find the optimal k value, and then perform cluster analysis.
[0134] SSE estimates the optimal k value by calculating the sum of squares of the errors between the fitted data and the original data. The closer SSE is to 0, the better the model selection and fit. The formula is as follows.
[0135]
[0136] SSE can be used with K-means, using the elbow method and scree plot to select the optimal number of clusters k. The core idea of the elbow method (a method used to determine the optimal number of clusters in machine learning and data mining) is that the larger the k, the finer the sample division will be, and the degree of aggregation of each class will gradually increase, so the error sum of squares SSE will naturally decrease. As the number of clusters k increases, the sample division will be finer, and the degree of aggregation of each cluster will gradually increase, so the error sum of squares SSE will naturally decrease. In addition, when k is less than the optimal number of clusters, the increase in k will greatly increase the degree of aggregation of each cluster, so the decrease in SSE will be large. When k reaches the optimal number of clusters, the degree of aggregation obtained by increasing k will rapidly decrease, so the decrease in SSE will decrease sharply, and then tend to be flat as the k value continues to increase. That is to say, the relationship graph between SSE and k is in the shape of an elbow, and the k value corresponding to this elbow is the optimal number of clusters for the data. The scree plot can assist in judging the k value. When the broken line suddenly changes from steep to smooth, the k value corresponding to the transition from steep to smooth is the optimal k value.
[0137] DBI is calculated by calculating the sum of the average distances within any two groups divided by the maximum distance between the two cluster centers. The smaller the DBI, the smaller the distance within the group and the larger the distance between the groups. The formula is as follows.
[0138]
[0139] Where n is the number of categories, ci is the center of the i-th category, and is the average distance from all points in category i to the center;
[0140] It is generally believed that the value at which SSE first starts to smooth out and the minimum value of DBI are the optimal k values.
[0141] After obtaining the PCA and clustering results, the device or model can automatically complete the visualization work, output PCA dot plots colored by clustering, line graphs for determining the optimal k value, and scripts for adjusting the drawing parameters.
[0142] It should be noted that, in order to facilitate the description of the embodiment, this application names the above data analysis process as VCF2PCA.
[0143] Beneficial Effects
[0144] The technical solution of the present invention can directly use the VCF format ( Figure 4 ) or Genotype format ( Figure 5) to complete the whole process of calculation, analysis and visualization, without the need to perform additional SNP (Single Nucleotide Polymorphism) quality control, filtering and format conversion on VCF. The technical solution of the present invention has a fast calculation speed and requires a small amount of memory. The memory is only affected by the sample size. The memory required for the calculation of the data of 10,000 samples is also within 1G. The software performs calculations while reading the data. It uses the method of reading the site information line by line, calculating the difference information between the samples, and recording it in the kinship kinship matrix 5 in the [sample * sample] array format, and recording some relevant information at the same time. Therefore, the memory required during the calculation is only affected by the sample size, but not by the size of the VCF site file. The present invention can apply the K-means algorithm according to the calculated PCA results, perform cluster analysis by sample, and find the optimal subgroup K. At the same time, the present invention can also complete the visualization of the results in one step, and can also quickly optimize the drawing by adjusting the parameters.
[0145] Embodiments of the present invention will be described in more detail below, examples of which are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.
[0146] Example 1
[0147] According to an embodiment of the present invention, in order to verify the effectiveness and reliability of the VCF2PCA (data analysis process of the present invention) method, the present invention uses the 1000 Genomes Database (Clarke L, Zheng-Bradley X, Smith R, et al. The 1000 Genomes Project: data management and community access [J]. Nature methods, 2012, 9 (5): 459-462.) to download chr22 data (2504 samples 1M multi-sites) for evaluation and comparison. Among them, Tassel and gapit3 use the actual data of chr22 to run time, the memory is much greater than 150G, and the waiting time is more than 300min. It can be considered that these two software are not suitable for large sample kinship and PCA analysis. In addition, GCTA does not support direct reading of VCF, and the VCF file needs to be filtered and converted to the input of plink. Therefore, this embodiment converts the VCF format by plink.
[0148] The results are shown in Table 1, indicating that compared with the mainstream analysis process, VCF2PCA requires less waiting time for users to process the same data, and the analysis process described in this application can also cluster and visualize the sequencing data, with superior performance.
[0149] Table 1:
[0150]
[0151]
[0152] Example 2
[0153] Furthermore, this embodiment merged the vcf of chr1-22 (81271745 sites) together to obtain an uncompressed VCF file (size 728G), and re-ran plink and VCF2PCA. plink must re-bed the fam bim file, but this step Error: File write failure, and PCA cannot be performed.
[0154] The results are shown in Table 2. VCF2PCA has superior performance in cluster drawing and other aspects. It is more suitable for newcomers to use. It does not take up much time and memory. It can run PCA analysis of massive data VCF on ordinary personal computers.
[0155] Table 2:
[0156] program Memory Waiting time Plink 300G Error:File write failure. VCF2PCA 0.1G ~600min:8thread
[0157] Example 3
[0158] The data used in the embodiment of the present invention also comes from the 1000 Genomes Database (Clarke L, Zheng-Bradley X, Smith R, et al. The 1000 Genomes Project: data management and community access [J]. Nature methods, 2012, 9 (5): 459-462.).
[0159] 1) 1194 SNP loci were selected from chromosome 22, including VCF format raw data of 203 samples from four groups: CEU (49), CHB (46), JPT (56) and YRI (52) (https: / / www.internationalgenome.org / data-portal / sample);
[0160] 2) Input file format is VCF data ( Figure 4 ) or genotype file (including samples and loci, Figure 5 );
[0161] 3) The basic command to be run is "MingPCACluster -InVCF Khuman.vcf -OutPut OUT", as shown in Table 3;
[0162] Table 3: Optional parameter description
[0163]
[0164]
[0165] 4) The output files include kinship matrix file, PCA results, PCA visualization graph, cluster K graph and clustering information, as shown in Table 4;
[0166] Table 4: Output file description
[0167] Output File illustrate out.kinship The output kinship matrix shows the pairwise relationship between samples. out.eigenvec Output optimal clustering and PCA results out.eigenval Output the feature vector of PCA results out.PCA1_PCA2.pdf Output PCA graph colored by cluster out.K.pdf Output cluster K graph out.cluster Output of various K clustering results
[0168] 5) Use SSE and Davies–Bouldin index (DBI) methods to select the optimal number of clusters k, that is, the k value at which SSE starts to smooth out for the first time and the minimum value of DBI, such as Figure 6 As shown, the optimal k value is 4;
[0169] 6) The results and visualized PCA graph when k is 4 are as follows. Figure 7 As shown, the device described in the present application divides more than 200 samples of the 1000 Genomes data into four clusters. Among the four groups of data CEU, CHB, JPT and YRI, CEU and CHB can be well distinguished from the other two groups;
[0170] 7) The device described in this application also provides a custom plotting script "ploteig", which can be used to generate the PCA results output by the device described in this application. The parameter description is shown in Table 5;
[0171] Table 5: Description of optional parameters of ploteig
[0172] Optional parameters Parameter format Parameter Description InPCA String Input PCA result file OutPrefix String Output file prefix help / Display help information columns String The number of rows and columns of the output image ColShap / color border Integer Boundary demarcation method title String title keystyle String Legend position settings pointsize Integer Picture point size setting
[0173] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0174] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0175] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0176] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0177] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.
Claims
1. A method for determining the kinship between samples, characterized in that: include: Obtaining genetic information of multiple samples at multiple predetermined sites; For at least a portion of the plurality of predetermined sites, based on the gene information difference between the samples at each predetermined site, obtaining a first matrix for each predetermined site; and Based on the first matrix of the plurality of predetermined sites, a kinship relationship between the plurality of samples is determined.
2. The method according to claim 1, characterized in that The first matrix of each predetermined site is obtained by the following steps: Writing the gene information of the plurality of samples at predetermined sites into a computer-readable storage medium; and A first matrix based on the predetermined sites is calculated based on the gene information.
3. The method according to claim 1, characterized in that The relatedness is determined by superimposing a first matrix of the plurality of predetermined sites.
4. The method according to claim 2, characterized in that: The data of the first matrix of each predetermined site is stored in the second matrix in the form of two-dimensional data; Optionally, the second matrix comprises at least one of a two-dimensional data table, an array, or a data frame; Optionally, the second matrix is a two-dimensional data table; Optionally, the second matrix is determined based on genotype difference values between multiple samples.
5. The method according to claim 1, characterized in that The first matrix of the plurality of predetermined sites is obtained by multi-thread or multi-process parallel processing.
6. The method according to claim 1, characterized in that The obtaining of the first matrix of each predetermined site comprises: For a predetermined site, calculating the genotype difference between the multiple samples in parallel by multi-threading or multi-processing; and The initial kinship of the predetermined loci is determined based on the genotype differences between the multiple samples.
7. The method according to claim 1, characterized in that Determining the kinship between the plurality of samples based on the first matrix of the plurality of predetermined sites comprises: Initializing a third matrix in a computer-readable storage medium, wherein the initial value of each element in the third matrix is 0; For the first matrix of each predetermined site, perform the following steps: The first matrix of the predetermined position is written into the computer-readable storage medium, the written first matrix is accumulated into the third matrix, and the storage space occupied by the written first matrix is released.
8. The method according to claim 2, characterized in that: The calculation is to calculate the difference values in the first matrix by using the stored difference values.
9. A method for analyzing gene information of multiple samples, characterized in that: include: Obtaining genetic information of multiple samples at multiple predetermined sites; and Determining the kinship between samples according to the method described in any one of claims 1 to 8; and Based on the kinship between the samples, at least one of principal component analysis and cluster analysis is performed on the gene information.
10. A device for determining the affinity between samples, characterized in that: include: A data collection unit, the data collection unit is used to obtain gene information of multiple samples at multiple predetermined sites; and A relationship matrix construction unit, connected to the data collection unit, for obtaining a plurality of initial kinship matrices based on the gene information differences between the samples for at least a portion of the plurality of predetermined sites; and A kinship determination unit, the kinship determination unit is connected to the relationship matrix construction unit, and is used to determine the kinship between the multiple samples based on the multiple initial kinship matrices.
11. A system for analyzing gene information of multiple samples, characterized in that: include: A data collection device, wherein the data collection unit device is used to obtain genetic information of multiple samples at multiple predetermined sites; and The device for determining kinship between samples as claimed in claim 10, wherein the device for determining kinship between samples is connected to the data collection device and is used to determine kinship between samples; and An analysis device, connected to the device for determining the kinship between samples, is used to perform at least one of principal component analysis and cluster analysis on the gene information based on the kinship between samples.
12. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 9.
13. A computer-readable storage medium, characterized in that: The storage medium includes computer instructions. When the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Method for constructing high-density genotype map of outcrossing species
CN111508560A
Variety identification method and device, and construction method and device of variety identification prediction model
CN112102880A
Sample identification tag screening method and sample identification detection method based on SNP polymorphic sites
CN112466395A
