Tumor dna identification and tracing method based on large language model

Abstract

The application particularly relates to a tumor DNA recognition and tracing method based on a large language model, which comprises the following steps: firstly, aligning each sequencing read in a to-be-detected sample to a reference genome and recognizing the methylation state thereof; then, encoding nucleotide sequences, CpG methylation states and fragment end information into read-level input according to a fixed structure; and outputting the tumor sample probability of each read in the to-be-detected sample through a discrimination unit, so as to realize low-abundance tumor DNA fragment recognition, threshold enrichment, sample-level load estimation and cancer or tissue source tracing. The method of the application solves the problems that it is difficult to determine the tumor source of a single read before sample aggregation in a low-tumor fraction cfDNA sample, and it is difficult to form a traceable high-confidence molecule set.

Description

Technical Field

[0001] This invention relates to the field of molecular recognition technology, specifically to a method for tumor DNA identification and tracing based on a large language model. Background Technology

[0002] Liquid biopsy obtains tumor-related genetic, epigenetic, and tissue-origin information by detecting cell-free DNA (cfDNA) in peripheral blood. It can be used for early tumor screening, recurrence monitoring, efficacy evaluation, and dynamic tracking of tumor burden. Compared with tissue biopsy, cfDNA sampling is less invasive and can be repeated. However, in early-stage tumors, low-tumor-burden, post-treatment residual lesions, or early-stage recurrence, tumor-derived DNA fragments account for a very low proportion of circulating DNA. Furthermore, there is overlap between cfDNA and DNA derived from hematopoietic cells and normal tissues in terms of fragment length, local sequence, and methylation status, making the sample-level average signal easily masked by background molecules.

[0003] DNA methylation is characterized by its high cell type specificity, wide distribution across the genome, and stable changes during tumorigenesis, making it suitable for tissue origin inference, tumor detection, and tumor score estimation. However, many existing liquid biopsy methods rely on predefined differentially methylated regions, CpG islands, promoter regions, fixed short sequence fragment features, or reference methylation maps, and aggregate reads at the region or sample level before inference. These approaches fail to determine tumor origin before individual reads are aggregated at the sample level, making it difficult to retain low-abundance but high-confidence tumor-origin fragments.

[0004] Therefore, it is necessary to provide a method for identifying single-molecule tumor DNA from pretreated bisulfite sequencing reads. Before sample-level aggregation, the nucleotide sequence, CpG methylation status, and fragment end information should be encoded into read-level inputs with a fixed structure, and the tumor-like probability of each read should be output to achieve the identification of low-abundance tumor DNA fragments and trace their tissue origin.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] This invention provides a tumor DNA identification and tracing method based on a large language model, which can effectively overcome the defects existing in the prior art.

[0007] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0008] According to a first aspect of the present invention, a method for tumor DNA identification and tracing based on a large language model is provided, the method comprising: Each sequencing read in the sample set to be tested is aligned to the reference genome, and the methylation status of the sequencing read at CpG sites is identified. The methylated CpG sites in the sequencing read are replaced with MG markers, and the nucleotide sequences of the unmethylated CpG sites and the non-CpG sites are retained. The fragment end information of the sequencing read is extracted, and the sequencing read is converted into a joint sequence-methylation string. The combined sequence-methylated string is processed to obtain a unified read input structure; the read input structure is: [CLS]-Main molecular sequence-[SEP]-Fragment terminal motif-[SEP], where the [CLS] position is the summary vector position of the corresponding sequencing read; The read segment input structure is input into the discrimination unit for tissue origin prediction. The [CLS] position output is used as the read segment level vector, and the discrimination unit predicts and outputs the tumor-like probability of each read segment.

[0009] In some exemplary embodiments, the fragment end information selects the 3' end 6bp sequence motif and the MspI restriction site in the sequencing read as the fragment end structure, and records the read direction, fragment length and endpoint coordinates of the sequencing read.

[0010] In some exemplary embodiments, before inputting the read segment input structure into the discrimination unit for tissue origin prediction, the method further includes: A dataset including tumor origin annotation information is obtained. Based on the dataset containing tumor origin annotation information, the same sequence construction process as for each sequencing read in the test sample set is performed to obtain the read input structure including tumor origin annotation information. The input discriminant unit is used to predict the tissue origin. The output at the [CLS] position is used as a read-level vector to obtain the tumor-like tissue origin of each read in the test sample set.

[0011] In some exemplary embodiments, the training set data for the tumor origin annotation information includes tissue origin, tumor tissue, adjacent tissue, normal tissue, and leukocyte / hematopoietic background; The read segment vector at the [CLS] position is then connected to the source prediction head and the white blood cell / hematopoietic background prediction head. The source prediction head and the white blood cell / hematopoietic background prediction head are used to obtain the tissue source probability and white blood cell / hematopoietic background probability of the sample to be tested.

[0012] In some exemplary embodiments, after inputting the read input structure into the discrimination unit for tissue origin prediction, the method further includes: obtaining tumor-positive and negative reads based on tumor xenograft data, performing the same sequence construction process as for each sequencing read in the test sample set based on the tumor-positive and negative reads, and predicting and outputting the tumor-like probability of each read through the discrimination unit.

[0013] In some exemplary embodiments, when the tumor xenograft data is compared, the tumor xenograft reads are compared to the reference genome. If the tumor xenograft read is only compared to the reference genome, the read is retained as a tumor-positive read. Otherwise, the tumor xenograft read is removed, and leukocyte / hematopoietic cell reads are used as negative background molecules to obtain tumor-positive and negative reads.

[0014] In some exemplary embodiments, the method further includes predicting the tumor-like probability of each read segment by a discrimination unit. Based on the comparison process of the tumor xenograft data, dual reference labels are constructed. A supervisory loss function for the tumor DNA prediction head is constructed by combining the dual reference labels with the tumor-like probability of each read. The tumor DNA discrimination boundary is determined based on the loss function.

[0015] In some exemplary embodiments, the sample set to be tested is input into a pre-trained prediction model for prediction processing, and the tumor sample prediction probability of each read segment in the sample to be tested is output based on the tumor DNA prediction head. And calculate the average tumor-like burden of all reads in the sample set to be tested based on the tumor-like prediction probability; A preset probability threshold is used to determine the relationship between the probability threshold and the predicted probability of tumor samples. The proportion of high-confidence tumor sample reads and the enriched read set are obtained through an exponential function to complete tumor DNA identification.

[0016] In some exemplary embodiments, after tumor DNA identification is completed, the method further includes: The genomic location of each read in the enriched read set is recorded as input. Each read is mapped to obtain a read record table, which includes read number, genomic coordinates, probability value, fragment terminal motif, methylation status, and sample source identifier. The read record table is cross-mapped with a preset annotation table line by line to output a structured source table, which includes read number, coordinates, annotation category, probability value, and source tag, thus completing the tumor DNA tracing.

[0017] In some exemplary embodiments, the method further includes a training process for the prediction model: Phase One Training: Obtain a training set of sequencing reads without tumor tags, align each sequencing read in the training set to the reference genome, identify the methylation status of the sequencing reads at CpG sites, and convert the sequencing reads into joint sequence-methylation strings based on the methylation status; The combined sequence-methylated string is processed to construct a unified read input structure. The read input structure is: [CLS]-Main molecular sequence-[SEP]-Fragment terminal motif-[SEP], where [CLS] is the summation vector position of the corresponding sequencing read. After each sequencing read is constructed, the read input structure is masked to construct the mask reconstruction loss corresponding to each read input structure. The first-stage training initialization model parameters are obtained based on the mask reconstruction loss of each read. Second phase of training: A training set including tumor origin annotation information is obtained. Taking the initial parameters of the first stage training as the parameter starting point, the same training process as the first stage training is performed based on the training set with the tumor origin annotation information to obtain the read segment input structure including tumor origin annotation information. The read segment input is input to the discrimination unit to predict tissue origin and obtain the tissue origin probability and the white blood cell / hematopoietic background probability. The output at the [CLS] position is used as the read segment level vector. The loss between the tissue origin prediction probability and the tissue origin is obtained through the supervised parameter update method. The constraint origin prediction head is updated through the loss between the tissue origin prediction probability and the tissue origin. After the above update is completed, the parameters of the second stage training model are obtained. Phase Three Training: Based on tumor xenograft data, positive and negative tumor reads are obtained. Using the parameters of the second-stage training model as the starting point, the same training process as the second-stage training is performed based on the tumor positive and negative reads to complete the training of the prediction model.

[0018] The tumor DNA identification and tracing method based on a large language model proposed in this invention encodes nucleotide sequences, CpG methylation status, and fragment end information into read-level inputs according to a fixed structure before sample-level aggregation, and outputs the tumor-like probability of each read to achieve low-abundance tumor DNA fragment identification, threshold enrichment, sample-level load estimation, and cancer type or tissue origin tracing.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1This diagram illustrates the method for constructing the methylation marker read input structure and optimizing the mask reconstruction parameters in an exemplary embodiment of the present invention. Figure 2 This diagram illustrates a method for updating read segment prediction header parameters based on labeled source data in an exemplary embodiment of the present invention. Figure 3 The illustration shows a schematic diagram of a classification boundary construction and tumor DNA identification method based on dual reference samples in an exemplary embodiment of the present invention. Detailed Implementation

[0022] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0023] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0024] In related technologies, existing methylation liquid biopsy methods typically rely on predefined differentially methylated regions, CpG islands, promoter regions, fixed short sequence fragment features, or reference methylation maps, and aggregate reads at the region or sample level before inference. This approach fails to determine tumor origin before individual reads enter sample-level aggregation, making it difficult to retain low-abundance but high-confidence tumor-origin fragments.

[0025] The aforementioned existing technologies also have the following problems: First, region-first or map-first methods rely on pre-selected labeled regions. When the sample source is complex, the reference map is incomplete, or the proportion of tumor molecules is low, individual fragments with tumor-origin characteristics will be diluted by background fragments during the region averaging process. Second, many methods extract nucleotide sequences, CpG methylation status, fragment length, and fragment end sequences separately and then combine them, which cannot form a joint input structure constrained by the same read position. Third, traditional deconvolution or sample-level classification models usually only output sample-level conclusions and are difficult to output a set of high-confidence molecules that can be traced back to genomic coordinates. Fourth, scenarios such as low-abundance tumor DNA, FFPE tissue tumor content estimation, and dynamic monitoring of the treatment process require obtaining stable read-level probabilities under fixed processing rules.

[0026] In summary, the main problem this application aims to solve is that in low-tumor-score cfDNA samples, it is difficult to determine the tumor origin of a single read and to form a traceable, high-confidence molecule set before sample-level aggregation.

[0027] The following will describe in more detail each step of the tumor DNA identification and tracing method based on a large language model in this exemplary embodiment, with reference to the accompanying drawings and embodiments.

[0028] In this example implementation, refer to Figure 1 As shown, a method for identifying single-molecule tumor DNA based on joint encoding of methylation markers and fragment ends is provided, including the following steps: obtaining bisulfite sequencing reads after alignment, CpG methylation status retrieval, and quality filtering; performing joint encoding on each read, replacing methylated CpG dinucleotides with MG markers, retaining the identities of unmethylated CpG and non-CpG nucleotides, and extracting the 3' end 6bp motif and MspI restriction site or fragment endpoint information; generating a unified read input structure according to [CLS]-molecular master sequence-[SEP]-terminal motif-[SEP], where the [CLS] position is used as the summary vector position for the read; inputting the unified read input structure into a read-level discrimination unit, and generating a single-molecule tumor-like probability from the output corresponding to the [CLS] position. and according to Generate segment sorting results and threshold enrichment set and sample-level aggregated values; To limit the input, the reads are mapped to genomic coordinates and a structured source record is generated.

[0029] It should be noted that the above-mentioned joint coding structure is irreplaceable: the MG marker directly places methylated CpG as a discrete molecular event into the read marker level; the 3' end 6bp motif and MspI restriction site introduce cfDNA fragmentation endpoint structures, which cannot be replaced by the base sequence within the main molecular sequence; the read-level probability output at the [CLS] position completes single-molecule discrimination before sample-level aggregation. The input of the method in this application is limited to bisulfite sequencing reads that have undergone reference genome alignment, CpG state calling, quality filtering, length filtering, and sample source identification; the output is limited to the tumor-like probability of each read. Organizational origin probability White blood cell / hematopoietic background probability Sample-level average tumor-like burden High score ratio of threshold and high-confidence tumor-like reading sets .

[0030] In step S1, each sequencing read in the sample set to be tested is aligned to the reference genome, and the methylation status of the sequencing read at CpG sites is identified. The methylated CpG sites in the sequencing read are replaced with MG markers, the nucleotide sequences of the unmethylated CpG sites and the non-CpG sites are retained, and the fragment end information of the sequencing read is extracted. The sequencing read is then converted into a combined sequence-methylation string.

[0031] For example, the raw WGBS / RRBS data is aligned and methylated. Preferably, Bismark or an equivalent process is used to align the reads to the human reference genome hg19 or other versions of the reference genome to identify the methylation / unmethylation status of CpG sites. Reads with unclear alignment, too many unknown bases, insufficient sequence information, falling into the genome blacklist region, or not meeting quality control rules are removed. Reads with a length of 50bp to 200bp are retained. Reads longer than 200bp are truncated to 200bp. Shorter reads are padded after word segmentation, and the padded positions are excluded by attention masking.

[0032] For example, in the fragment end information, the 3' end 6bp sequence motif and the MspI restriction site in the sequencing read are selected as the fragment end structure, and the read direction, fragment length and endpoint coordinates of the sequencing read are recorded.

[0033] Specifically, for RRBS reads, the MspI restriction fragment end rules are consistently used to record the 5' end restriction site, and a variable 6bp 3' end sequence is extracted as the terminal sequence motif. Read direction, fragment length, and endpoint coordinates are also recorded. For other bisulfite sequencing data, 3', 5', or paired-end fragment sequence motifs can be recorded using the same field structure. This terminal information is encoded synchronously with the MG marker at the read input layer and is not used as a sample-level external statistic.

[0034] In step S2, the combined sequence-methylated string is subjected to sequence construction processing to obtain a unified read input structure; the read input structure is: [CLS]-Main molecular sequence-[SEP]-Fragment terminal motif-[SEP], where the [CLS] position is the summary vector position of the corresponding sequencing read.

[0035] For example, each qualified read is converted into a joint sequence-methylation string; wherein methylated CpG dinucleotides are replaced with dedicated MG markers, and unmethylated CpG and non-CpG sequences retain their nucleotide identities; this replacement rule makes the methylation state a discrete input marker at the same level as the nucleotide, and together with the restriction enzyme position, the 3' end 6bp motif, or other fragment endpoint information, it constitutes a read-level input.

[0036] A fixed input sequence is constructed, and the joint string is segmented using Byte-Pair Encoding (BPE) of SentencePiece or an equivalent sub-word segmentation method; the preferred vocabulary size is 1,028, including MG, [CLS], [SEP], [MASK], and padding tags; each read segment is constructed in the order of [CLS]-Main Molecular Sequence-[SEP]-Terminal Motif-[SEP]; the maximum input length is fixed at 64 tags; tag embedding, position embedding, methylation state embedding, and fragment end embedding are fused by position-wise addition to form the input of the multi-layer bidirectional attention neural network processing unit, i.e., the input of the discriminant unit.

[0037] For example, before inputting the read segment input structure into the discrimination unit for organization source prediction, the method further includes: A dataset including tumor origin annotation information is obtained. Based on the dataset containing tumor origin annotation information, the same sequence construction process as for each sequencing read in the test sample set is performed to obtain the read input structure including tumor origin annotation information. The input discriminant unit is used to predict the tissue origin. The output at the [CLS] position is used as a read-level vector to obtain the tumor-like tissue origin of each read in the test sample set.

[0038] For example, the training set data for the tumor origin annotation information includes tissue origin, tumor tissue, adjacent tissue, normal tissue, and leukocyte / hematopoietic background; The read segment vector at the [CLS] position is then connected to the source prediction head and the white blood cell / hematopoietic background prediction head. The source prediction head and the white blood cell / hematopoietic background prediction head are used to obtain the tissue source probability and white blood cell / hematopoietic background probability of the sample to be tested.

[0039] For example, before inputting the read input structure into the discrimination unit for tissue origin prediction, the method further includes: obtaining tumor-positive and negative reads based on tumor xenograft data; performing the same sequence construction process as for each sequencing read in the test sample set based on the tumor-positive and negative reads; and predicting and outputting the tumor-like probability of each read through the discrimination unit.

[0040] For example, when comparing the tumor xenograft data, the tumor xenograft reads are aligned to the reference genome. If the tumor xenograft read is aligned only to the reference genome, the read is retained as a tumor-positive read. Otherwise, the tumor xenograft read is removed, and leukocyte / hematopoietic cell reads are used as negative background molecules to obtain tumor-positive and negative reads.

[0041] For example, a dual reference label is constructed based on the comparison process of the tumor xenograft data. A supervisory loss function for the tumor DNA prediction head is constructed by the dual reference label and the tumor-like probability of each read. The tumor DNA discrimination boundary is determined based on the loss function.

[0042] Specifically, for PDX / CDX samples, reads are aligned to both the human and mouse reference genomes, such as hg19 and mm9. If a read uniquely aligns to the human reference genome and not to the mouse reference genome, it is retained as a positive read for human tumors. If a read aligns to the mouse reference genome or has both human and mouse ambiguous origins, it is discarded or marked as unusable for constructing positive borders. For background samples obtained from blood collection, WBC or hematopoietic cell reads are used as negative background molecules.

[0043] Furthermore, this example implementation also includes a parameter update stage. This step only limits the data structure, label source, and loss function type used for parameter updating. Specifically, the first parameter update stage can use methylated labeled reads without tumor labels to calculate the mask reconstruction loss of formula (1); the second parameter update stage can use tissue, tumor, adjacent tissue, and hematopoietic reference reads to update the read prediction header parameters, thereby generating... and The third parameter update stage uses the obtained human tumor positive reads and WBC / hematopoietic background reads to update the tumor DNA prediction head parameters, thereby generating... .

[0044] (1) (2) In the formula, For mask reconstruction loss, The set of marker locations that were selected and participated in the reconstruction. The number of masked positions. For the j-th tag to be reconstructed, Indicates the set The context sequence after the position mask, These represent the parameters of the embedding layer, the bidirectional attention processing unit, and the prediction head. To recover the conditional probability of the true label based on the context; in equation (2), For binary cross-entropy loss, For category tags, This represents the positive probability output by the prediction head; when used in a tumor DNA prediction head, A value of 1 indicates a human tumor read. A value of 0 indicates a WBC or hematopoietic background read.

[0045] In step S3, the read segment input structure is input to the discrimination unit for tissue origin prediction. The [CLS] position output is used as the read segment level vector, and the tumor-like probability of each read segment is predicted and output by the discrimination unit.

[0046] Specifically, in the discrimination unit, the output vector corresponding to the [CLS] position serves as the read-level vector, which is connected to the tumor DNA prediction head, tissue origin prediction head, or leukocyte / hematopoietic background prediction head to generate read-level probabilities. The discrimination unit is implemented by a bidirectional attention processing unit, comprising multiple sequentially connected bidirectional attention processing layers. Each bidirectional attention processing layer includes a multi-head self-attention sublayer, a first residual connection and layer normalization sublayer, a feedforward network sublayer, and a second residual connection and layer normalization sublayer. The multi-head self-attention sublayer calculates the association weights between different marker positions within the read and performs weighted aggregation of nucleotide sequence, methylation state, and fragment end information based on these association weights. The feedforward network sublayer performs a nonlinear transformation on the aggregated features. The residual connection and layer normalization sublayer preserves the input features and unifies the numerical scale of the outputs from each layer.

[0047] In one embodiment of this example implementation method, the bidirectional attention processing unit includes 12 sequentially connected bidirectional attention processing layers, with a hidden dimension of 768, 12 attention heads, a feedforward network intermediate dimension of 3072, and a maximum input length of 64 tags. After the read segment input structure is processed layer by layer by each bidirectional attention processing layer, the output vector corresponding to the [CLS] position of the last layer is extracted and input into the corresponding prediction head to output the tumor-like probability, tissue origin probability, or white blood cell / hematopoietic background probability.

[0048] For example, the sample set to be tested is input into a pre-trained prediction model for prediction processing, and the tumor sample prediction probability of each read segment in the sample to be tested is output based on the tumor DNA prediction head. And calculate the average tumor-like burden of all reads in the sample set to be tested based on the tumor-like prediction probability; A preset probability threshold is used to determine the relationship between the probability threshold and the predicted probability of tumor samples. The proportion of high-confidence tumor sample reads and the enriched read set are obtained through an exponential function to complete tumor DNA identification.

[0049] Specifically, after parameter updates are completed, the segmenter vocabulary, MG replacement rules, 3' end 6bp motif field, input sequence structure, padding mask rules, selected discriminant unit structure, and prediction head parameters are fixed. During inference, the same processing rules as S1 to S6 are applied to each qualified read segment of the test sample, and the same frozen parameter set is used to generate read-level probabilities. The MG replacement rules, motif field, [CLS]-molecular master sequence-[SEP]-motif-[SEP] input structure, and probability aggregation method must not be changed for different samples, different batches, or different downstream tasks.

[0050] Specifically, for the fixed structure input of the i-th read segment Tumor DNA prediction head outputs tumor-like probability This probability is expressed as: (3) In the formula, The input is the i-th read segment after MG replacement, terminal motif appending, word segmentation, and embedding fusion. A value of 1 indicates a tumor-like category. This represents the set of parameters after freezing. This represents the probability that the read belongs to a tumor-like molecule. For those containing... For a sample of qualified reading segments, the sample-level average probability loading is calculated according to formula (4). The proportion of high-scoring segments at the threshold is calculated according to formula (5). : (4) (5) In equation (4), The average value of the tumor-like probability of all qualified reads within the sample is used to characterize the sample-level tumor-like molecular burden; in equation (5), For the preset probability threshold, For indicator functions, when Greater than or equal to The value is 1 if the condition is met, and 0 otherwise. The proportion of high-confidence tumor-like reads. The corresponding enriched read set is defined as follows: (6) In equation (6), Indicates threshold The tumor-like read set obtained below, This represents the i-th original read or its genomic location record. This set is an intermediate data structure obtained by fixing the input structure, freezing the parameter set, and fixing the probability threshold, and serves as the sole read input for the source tracing step; when a fixed molecular budget is required, it can also be set to... Sort by high to low and retain the first K entries or the first preset percentage of read segments.

[0051] For example, after completing tumor DNA identification, the method further includes: The genomic location of each read in the enriched read set is recorded as input. Each read is mapped to obtain a read record table, which includes read number, genomic coordinates, probability value, fragment terminal motif, methylation status, and sample source identifier. The read record table is cross-mapped with a preset annotation table line by line to output a structured source table, which includes read number, coordinates, annotation category, probability value, and source tag, thus completing the tumor DNA tracing.

[0052] Specifically, the high-confidence tumor-like read set obtained As the sole read input, and with the genomic location of each read recorded. As coordinate input; for Each read segment is mapped according to a fixed mapping rule to generate a read segment record table, which includes at least the read segment number, genomic coordinates, probability value, fragment terminal motif, methylation status, and sample source identifier. The read segment record table is cross-mapped with a pre-determined annotation table line by line to output a structured source table, which includes at least the read segment number, coordinates, annotation category, probability value, and source label.

[0053] In summary, this application converts methylated CpG from sample or region statistics into discrete markers within reads using MG substitution rules, preserving methylation events and nucleotide sequences in the same single-molecule input structure; it introduces the 3' 6bp end sequence motif and MspI restriction site as fragment end structure fields into the same read input, fixing and encoding cfDNA fragmentation endpoint information before read-level discrimination; and it outputs the tumor-like probability of each read through the [CLS] position. Before sample-level aggregation, a set of high-confidence tumor sample reads that can be sorted, filtered, and traced back to genomic coordinates is generated. .

[0054] This example implementation also provides a method for constructing read-level representations and initializing parameters based on joint encoding of methylation markers and fragment ends. This method serves as the basic parameter optimization stage of the entire model, used to generate initial model parameters and provide a unified input encoding structure for subsequent stages, as detailed below: In the first phase of training: For example, a training set of sequencing reads without tumor tags is obtained, each sequencing read in the training set is aligned to a reference genome, and the methylation status of the sequencing read at CpG sites is identified. Based on the methylation status, the sequencing read is converted into a joint sequence-methylation string. The combined sequence-methylated string is processed to construct a unified read input structure. The read input structure is: [CLS]-Main molecular sequence-[SEP]-Fragment terminal motif-[SEP], where [CLS] is the summation vector position of the corresponding sequencing read. After each sequencing read is constructed, the read input structure is masked to construct the mask reconstruction loss corresponding to each read input structure. The first-stage training initialization model parameters are obtained based on the mask reconstruction loss of each read. Specifically, bisulfite sequencing reads are standardized and preprocessed by aligning RRBS or WGBS reads to a reference genome, calling CpG methylation status, removing reads with unclear alignment, too many unknown bases, insufficient sequence information, falling into blacklist regions, or not meeting quality control rules, and limiting qualified reads to the range of 50bp to 200bp.

[0055] For qualified reads, methylation marker replacement and fragment end information extraction were performed. Methylated CpG was replaced with MG markers, while the identities of unmethylated CpG and non-CpG nucleotides were preserved. Simultaneously, the MspI restriction site, 3' 6bp terminal motif, fragment length, read orientation, and endpoint coordinates were extracted. Let the main molecular sequence after MG replacement for the i-th read be... The fragment terminal motif field is Then input uniformly Represented as: (7) In the formula, [CLS] represents the segment-level summary position, and [SEP] represents the field separator. This is the main sequence of the methylated molecule. This is a fragment end structure field for the same read; this structure allows methylation status, nucleotide sequence, and fragment end information to be included in subsequent calculations under the same read coordinates.

[0056] A unified input structure for the reading segment is constructed. In this example implementation, SentencePiece / BPE is used to construct the vocabulary, which has a size of 1,028 and a maximum input length of 64 tags. Tag embedding, position embedding, methylation state embedding, and segment end embedding are fused in a position-by-position addition manner, and the fused vector at the t-th position is... for: (8) In the formula, This represents the methylation status field corresponding to the t-th position of the i-th read segment. This represents the end structure field of the read segment; the above four types of embeddings have the same dimension and are fused before being input to the read segment-level processing unit. In the implementation of this example, the read segment-level processing unit uses 12 bidirectional attention layers, with a hidden dimension of 768, 12 attention heads, and a feedforward layer intermediate dimension of 3,072.

[0057] Furthermore, this example implementation also includes performing mask reconstruction parameter optimization. 15% of the marked positions in the input segment are masked, with 80% replaced by [MASK], 10% replaced by random vocabulary tags, and 10% remaining unchanged; let the set of selected masked positions be... Mask reconstruction loss Defined as: (9) Initialization parameters are obtained based on mask reconstruction loss from multiple read segments. : (10) In this example implementation, 415 RRBS libraries are used as tumor-label-free masked reconstruction data in this stage; obtained through equations (7) to (10) As an initialization parameter for subsequent implementations, the constructed unified read segment input structure remains consistent throughout the entire methodology.

[0058] In the second phase of training: For example, a training set including tumor origin annotation information is obtained. Taking the initial parameters of the first stage training as the parameter starting point, the same training process as the first stage training is performed based on the training set with the tumor origin annotation information to obtain the read segment input structure including tumor origin annotation information. The read segment input is input to the discrimination unit to predict tissue origin and obtain the tissue origin probability and the white blood cell / hematopoietic background probability. The output at the [CLS] position is used as the read segment level vector. The loss between the tissue origin prediction probability and the tissue origin is obtained through the supervised parameter update method. The constraint origin prediction head is updated through the loss between the tissue origin prediction probability and the tissue origin. After the above update is completed, the parameters of the second stage training model are obtained. Specifically, refer to Figure 2 As shown; this example implementation continues the training based on the first stage, and uses the initialized model parameters obtained in the previous implementation. As a starting point for parameters, without changing the unified input structure of the reading segment, source annotation information is introduced to further constrain the training of the model. The specific implementation method in this example is as follows: A labeled dataset was constructed based on tissue origin, tumor tissue, adjacent tissue, normal tissue, and leukocyte / hematopoietic background, and encoded according to the preprocessing rules, MG replacement rules, terminal motif extraction rules, and unified input structure of the specific first-stage training. In the implementation of this example, the source labeled data includes 463 tissue libraries covering primary tumor, adjacent tissue, and normal tissue, and includes 177 RRBS hematopoietic or reference cell libraries as leukocyte / hematopoietic background data.

[0059] The results obtained in the first phase of training The read segment level coding model is initialized based on this, and the output at the [CLS] position is taken as the read segment level vector. : (11) exist The above connection is used to generate tissue origin probability prediction heads and white blood cell / hematopoietic background prediction heads. and white blood cell / hematopoietic background probability : (12) Furthermore, the loss between the predicted probability and the source label is calculated using a supervised parameter update method, and the source prediction header and its related parameters are updated accordingly. In one embodiment, the source constraint loss... Losses from the source of the organization and white blood cell / hematopoietic background loss composition: (13) (14) (15) In equations (13) to (15), Weights are assigned to white blood cell / hematopoietic background loss. Predict the probability of organization origin. The updated model parameters, obtained through the above steps, are denoted as... This parameter inherits the ability to jointly represent methylation markers and fragment ends obtained in Implementation Method 1, and adds source labeling constraints on this basis.

[0060] In the third phase of training: For example, based on tumor xenograft data, positive and negative tumor reads are obtained. Using the parameters of the second-stage training model as the starting point, the same training process as the second-stage training is performed based on the positive and negative tumor reads to complete the training of the prediction model.

[0061] Specifically, refer to Figure 3 As shown, this example implementation continues based on the second-stage training, and uses the model parameters obtained from the second-stage training implementation. As initialization parameters, these are used to construct the tumor DNA discrimination boundary and to achieve single-molecule-level probability output and sample-level aggregation calculation. The specific steps of this example implementation are as follows: PDX / CDX dual-reference classification boundary data were constructed, and PDX or CDX reads were aligned to human and mouse reference genomes, such as hg19 and mm9, respectively. Reads that uniquely aligned to the human reference genome and not to the mouse reference genome were retained as positive reads for human tumors. Reads aligned to the mouse reference genome or exhibiting ambiguous human / mouse origins were removed or marked as unsuitable for positive boundary construction, and WBC or hematopoietic reference reads were used as negative background molecules. Dual-reference labels were used. Defined as: (16) The positive and negative reads mentioned above are encoded according to a unified input structure and input into the read-level encoding model corresponding to the model parameters obtained in the second stage of training; the tumor DNA prediction head is then connected to the [CLS] output to output the tumor-like probability of each read. : (17) Based on dual reference labels and the probability of reading segments Calculate the supervision loss of the tumor DNA prediction head : (18) After training, the word segmenter, input length, embedding fusion method, segment-level processing unit parameters, and prediction head parameters are fixed to obtain the frozen parameters. During the inference phase, the same encoding process is performed on the test samples, and the single-molecule probability is output based on the frozen model. The sample-level average tumor-like burden was calculated using the following formula. High threshold reading ratio and high-confidence tumor-like reading sets : (19) Through the above progressive process, this example implementation inherits the model parameters and coding structure of the first-stage training and the second-stage training, making the whole method form a progressively enhanced unified model system from read segment-level representation initialization, source labeling constraints to the construction of tumor DNA discrimination boundaries.

[0062] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0063] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0064] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A tumor DNA identification and tracing method based on a large language model, characterized in that, The method includes: Each sequencing read in the sample set to be tested is aligned to the reference genome, and the methylation status of the sequencing read at CpG sites is identified. The methylated CpG sites in the sequencing read are replaced with MG markers, and the nucleotide sequences of the unmethylated CpG sites and the non-CpG sites are retained. The fragment end information of the sequencing read is extracted, and the sequencing read is converted into a joint sequence-methylation string. The combined sequence-methylated string is processed to obtain a unified read input structure; the read input structure is: [CLS]-Main molecular sequence-[SEP]-Fragment terminal motif-[SEP], where the [CLS] position is the summary vector position of the corresponding sequencing read; The read segment input structure is input into the discrimination unit for tissue origin prediction. The [CLS] position output is used as the read segment level vector, and the discrimination unit predicts and outputs the tumor-like probability of each read segment.

2. The tumor DNA identification and tracing method based on a large language model according to claim 1, characterized in that, In the fragment end information, the 3' end 6bp sequence motif and the MspI restriction site in the sequencing read are selected as the fragment end structure, and the read direction, fragment length and endpoint coordinates of the sequencing read are recorded.

3. The tumor DNA identification and tracing method based on a large language model according to claim 1, characterized in that, Before inputting the read segment input structure into the discrimination unit for tissue origin prediction, the method further includes: A dataset including tumor origin annotation information is obtained. Based on the dataset containing tumor origin annotation information, the same sequence construction process as for each sequencing read in the test sample set is performed to obtain the read input structure including tumor origin annotation information. The input discriminant unit is used to predict the tissue origin. The output at the [CLS] position is used as a read-level vector to obtain the tumor-like tissue origin of each read in the test sample set.

4. The tumor DNA identification and tracing method based on a large language model according to claim 3, characterized in that, The training set data for tumor origin annotation information includes tissue origin, tumor tissue, adjacent tissue, normal tissue, and leukocyte / hematopoietic background; The read segment vector at the [CLS] position is then connected to the source prediction head and the white blood cell / hematopoietic background prediction head. The source prediction head and the white blood cell / hematopoietic background prediction head are used to obtain the tissue source probability and white blood cell / hematopoietic background probability of the sample to be tested.

5. The tumor DNA identification and tracing method based on a large language model according to claim 1, characterized in that, Before inputting the read input structure into the discrimination unit for tissue origin prediction, the method further includes: obtaining tumor positive and negative reads based on tumor xenograft data; performing the same sequence construction process as for each sequencing read in the test sample set based on the tumor positive and negative reads; and predicting and outputting the tumor-like probability of each read through the discrimination unit.

6. The tumor DNA identification and tracing method based on a large language model according to claim 5, characterized in that, When comparing the tumor xenograft data, the tumor xenograft reads are aligned to the reference genome. If the tumor xenograft read is aligned only to the reference genome, the read is retained as a tumor-positive read. Otherwise, the tumor xenograft read is removed, and leukocyte / hematopoietic cell reads are used as negative background molecules to obtain tumor-positive and negative reads.

7. The tumor DNA identification and tracing method based on a large language model according to claim 5, characterized in that, The method further includes predicting the tumor-like probability of each read segment using a discrimination unit. Based on the comparison process of the tumor xenograft data, dual reference labels are constructed. A supervisory loss function for the tumor DNA prediction head is constructed by combining the dual reference labels with the tumor-like probability of each read. The tumor DNA discrimination boundary is determined based on the loss function.

8. The tumor DNA identification and tracing method based on a large language model according to claim 7, characterized in that, The test sample set is input into the pre-trained prediction model for prediction processing, and the tumor sample prediction probability of each read segment in the test sample is output based on the tumor DNA prediction head. And calculate the average tumor-like burden of all reads in the sample set to be tested based on the tumor-like prediction probability; A preset probability threshold is used to determine the relationship between the probability threshold and the predicted probability of tumor samples. The proportion of high-confidence tumor sample reads and the enriched read set are obtained through an exponential function to complete tumor DNA identification.

9. The tumor DNA identification and tracing method based on a large language model according to claim 8, characterized in that, After completing tumor DNA identification, the method further includes: The genomic location of each read in the enriched read set is recorded as input. Each read is mapped to obtain a read record table, which includes read number, genomic coordinates, probability value, fragment terminal motif, methylation status, and sample source identifier. The read record table is cross-mapped with a preset annotation table line by line to output a structured source table, which includes read number, coordinates, annotation category, probability value, and source tag, thus completing the tumor DNA tracing.

10. The tumor DNA identification and tracing method based on a large language model according to claim 1, characterized in that, The method also includes a training process for the prediction model: Phase One Training: Obtain a training set of sequencing reads without tumor tags, align each sequencing read in the training set to the reference genome, identify the methylation status of the sequencing reads at CpG sites, and convert the sequencing reads into joint sequence-methylation strings based on the methylation status; The combined sequence-methylated string is processed to construct a unified read input structure. The read input structure is: [CLS]-Main molecular sequence-[SEP]-Fragment terminal motif-[SEP], where [CLS] is the summation vector position of the corresponding sequencing read. After each sequencing read is constructed, the read input structure is masked to construct the mask reconstruction loss corresponding to each read input structure. The first-stage training initialization model parameters are obtained based on the mask reconstruction loss of each read. Second phase of training: A training set including tumor origin annotation information is obtained. Taking the initial parameters of the first stage training as the parameter starting point, the same training process as the first stage training is performed based on the training set with the tumor origin annotation information to obtain the read segment input structure including tumor origin annotation information. The read segment input is input to the discrimination unit to predict tissue origin and obtain the tissue origin probability and the white blood cell / hematopoietic background probability. The output at the [CLS] position is used as the read segment level vector. The loss between the tissue origin prediction probability and the tissue origin is obtained through the supervised parameter update method. The constraint origin prediction head is updated through the loss between the tissue origin prediction probability and the tissue origin. After the above update is completed, the parameters of the second stage training model are obtained. Phase Three Training: Based on tumor xenograft data, positive and negative tumor reads are obtained. Using the parameters of the second-stage training model as the starting point, the same training process as the second-stage training is performed based on the tumor positive and negative reads to complete the training of the prediction model.

Images