A Cancer Signal Identification Method Based on cfDNA Sequence Language Model and Its Application in Cancer Screening and MRD Monitoring

Abstract

This invention relates to a cancer signal identification method based on a cfDNA sequence language model and its application in cancer screening and MRD monitoring. The deep learning-based sequence language model analyzes whole-genome sequencing data of cfDNA to automatically extract and identify cancer-related signals, making it suitable for clinical scenarios such as early cancer screening and minimal residual disease (MRD) monitoring. This technology overcomes the limitations of targeting specific known mutations or manually labeled biomarkers, enabling end-to-end deep modeling of high-throughput whole-genome sequences of cfDNA. This significantly improves the sensitivity, specificity, and generalization ability of early cancer screening and MRD monitoring, providing an innovative solution for non-invasive screening of multiple cancer types and longitudinal disease monitoring.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of bioinformatics, molecular diagnostics, genomics and artificial intelligence, specifically to a technology for cancer signal identification using language modeling methods based on cell-free DNA (cfDNA) sequences in plasma. Background Technology

[0002] Cancer, as one of the leading causes of death worldwide, highlights the crucial role of early detection and intervention in improving patient survival and prognosis. However, because most cancers lack obvious symptoms in their early stages, a significant proportion of patients are diagnosed at middle or late stages, delaying the most effective treatment. Therefore, developing reliable, efficient, non-invasive early screening and minimal residual disease (MRD) monitoring technologies that cover multiple cancer types has become a critical issue urgently needing breakthroughs in the field of cancer prevention and treatment.

[0003] In recent years, liquid biopsy technology based on cell-free plasma DNA (cfDNA) has provided a new direction for non-invasive cancer screening and monitoring. cfDNA, as a molecular fragment reflecting the state and genetic characteristics of cells from various sources in the body, contains tumor-derived DNA (ctDNA), which carries multi-level tumor molecular information. Mainstream cfDNA analysis methods largely focus on the following strategies: First, detecting specific mutations or structural variations in a few known cancer-related genes (such as the panel method). This method lacks sensitivity in early-stage patients with low mutation frequencies or extremely low ctDNA levels and is difficult to apply to unknown or rare cancer types. Second, combining cfDNA fragment length, methylation patterns, copy number variations, and other multi-omics characteristics for multimodal analysis. While this improves diagnostic performance to some extent, the detection process is complex and costly, and it remains limited by the selection and integration capabilities of biomarkers.

[0004] Whole-genome sequencing (WGS) provides more comprehensive molecular feature coverage for cfDNA analysis, theoretically capable of capturing potential multidimensional aberrations in ctDNA without bias. However, traditional hypothesis-driven bioinformatics analysis methods typically focus only on partial variations or known signals, making it difficult to effectively mine and utilize high-dimensional, complex, rare, or even unknown cancer features. This limits the broad-spectrum performance and clinical applicability of non-invasive early cancer screening and MRD monitoring. Summary of the Invention

[0005] This invention utilizes deep learning-based sequence language models (such as the Transformer architecture) to analyze whole-genome sequencing data of cfDNA, enabling the automatic extraction and discrimination of cancer-related signals. It is applicable to clinical scenarios such as early cancer screening and minimal residual disease (MRD) monitoring. This technology overcomes the limitations of targeting specific known mutations or manually labeled biomarkers, enabling end-to-end deep modeling of high-throughput whole-genome sequences of cfDNA. This significantly improves the sensitivity, specificity, and generalization ability of early cancer screening and MRD monitoring, providing an innovative solution for non-invasive screening of multiple cancer types and longitudinal disease monitoring.

[0006] A cancer signal identification method based on cfDNA sequence language model, used for non-therapeutic and diagnostic purposes, includes the following steps: a. Data acquisition and processing steps: Obtain cfDNA from tumor samples and healthy human samples, and sequence them to obtain cfDNA fragment sequences. Align these fragments to a reference genome to obtain their positions on the reference genome. Then, extend the sequences upstream and downstream by several bases. Use the extended sequence information on the reference genome as the data information for a single sample. b. Pre-training of the basic model: The reference genome DNA of multiple species containing humans is divided into segments. While maintaining the original order, the model is trained using a masked language model mechanism to learn the potential linguistic structural features of the DNA sequence.

[0007] c. Model fine-tuning: Input the data information of tumor samples and healthy human samples obtained in step a into the pre-trained model obtained in step b, and convert them into their respective segment feature vectors; use a multi-instance learning model to perform weighted fusion of multiple segment feature vectors obtained in step b to generate a sample-level representation of the test sample; wherein, each sample contains a sample classification label; input the sample-level representation into the classifier to obtain the probability of having cancer; d. Classification and prediction steps: Input the data of the sample to be tested into the fine-tuned model to predict the probability of cancer.

[0008] In step b, dividing the DNA into fragments ensures that the DNA fragments are of consistent size.

[0009] In step b, the pre-training uses a pre-trained language model built on the Transformer architecture and understands the relationship between any two DNA sub-word fragments in a sequence based on the self-attention mechanism.

[0010] The multi-instance learning model in step c calculates the attention weights of the segment feature vectors through an attention mechanism, and performs a weighted summation of the segment feature vectors based on the attention weights to generate the sample-level representation.

[0011] The classifier in step d is a multi-layer fully connected neural network. Its training process uses the binary cross-entropy loss function to update parameters, and optimizes the parameters of the feature extraction module, the multi-instance learning model and the classifier through backpropagation to achieve end-to-end model training.

[0012] In the sequencing of cfDNA fragments in step a, when the sequencing data is aligned to the reference genome, cfDNA fragments with alignment quality scores not lower than a preset threshold need to be selected; and the length of the selected cfDNA fragments is limited to no more than a predetermined length threshold.

[0013] The preset threshold for the alignment quality score is 20-40; the preset length threshold for the cfDNA fragment length is 200-300 bp.

[0014] The pre-trained language model built on the Transformer architecture uses the BERT model and contains 10-20 Transformer encoder layers; the segment feature vector is a 600-900 dimensional vector.

[0015] During model training or inference, multiple cfDNA fragment sequences from the test sample are sampled in batches and input into the model; each batch contains 2000-4000 cfDNA fragment sequences.

[0016] It also includes: using the calculated attention weights to score the cancer signal contribution of individual cfDNA fragments and screening out a predetermined proportion of high-risk cfDNA fragments for minimal residual disease (MRD) monitoring or biomarker development.

[0017] A method for constructing a cfDNA sequence language model for non-therapeutic and diagnostic purposes includes the following steps: a. Data acquisition and processing steps: Obtain cfDNA from tumor samples and healthy human samples, and sequence them to obtain cfDNA fragment sequences. Align these fragments to a reference genome to obtain their positions on the reference genome. Then, extend the sequences upstream and downstream by several bases. Use the extended sequence information on the reference genome as the data information for a single sample. b. Pre-training of the basic model: Reference genomic DNA from multiple species, including humans, is divided into segments. While maintaining the original order, the model is trained using a masked language model mechanism to learn the latent linguistic structural features of the DNA sequence; c. Model fine-tuning: Input the data information of tumor samples and healthy human samples obtained in step a into the pre-trained model obtained in step b, and convert them into their respective segment feature vectors; use a multi-instance learning model to perform weighted fusion of multiple segment feature vectors obtained in step b to generate a sample-level representation of the test sample; wherein, each sample contains a sample classification label; input the sample-level representation into the classifier to obtain the probability of having cancer.

[0018] A computer-readable medium containing a computer program capable of performing the cancer signal recognition method described above.

[0019] A cancer signal recognition device based on a cfDNA sequence language model, comprising: Data acquisition and processing module: Used to acquire cfDNA from tumor samples and healthy human samples, sequence them to obtain cfDNA fragment sequences, align them to the reference genome to obtain their positions on the reference genome, and then extend them upstream and downstream by several bases. The sequence information on the reference genome after extension is used as the data information of a single sample. The pre-training module of the base model is used to divide reference genomic DNA from multiple species, including humans, into segments, and train the model using a masked language model mechanism while maintaining the original order, in order to learn the latent linguistic structural features of the DNA sequence. The model's fine-tuning module is used to input the data information of tumor samples and healthy human samples obtained from the data acquisition and processing module into the pre-trained model obtained from the pre-training module of the base model, converting them into their respective corresponding fragment feature vectors; a multi-instance learning model is used to weight and fuse multiple fragment feature vectors obtained from the pre-training module of the base model to generate a sample-level representation of the test sample; wherein, each sample contains a sample classification label; the sample-level representation is input into the classifier to obtain the probability of having cancer; Classification and prediction module: Used to input the data of the sample to be tested into the fine-tuned model to predict the probability of cancer. The beneficial effects of this invention are: Full-sequence automated modeling: Without the need for preset mutation sites or limited biomarkers, it directly conducts end-to-end learning of high-dimensional and complex "genomic language" based on the original cfDNA sequence, greatly expanding the analyzable molecular feature space.

[0020] Applicable to both pan-cancer and MRD scenarios: It is not only suitable for non-invasive early screening of multiple cancer types, but also supports highly sensitive dynamic monitoring of minimal residual lesions, making it suitable for a wide range of clinical applications.

[0021] Enhanced detection sensitivity and interpretability: The fusion of sequence language model and MIL framework is particularly suitable for early cases with extremely low ctDNA content, improving the detection rate of rare signals, while also providing fragment-level risk interpretation capabilities.

[0022] Standardized and automated processes: It integrates automatic data processing, batch analysis and machine judgment throughout the entire process, making it suitable for large-scale liquid biopsy cohort deployment.

[0023] Expanding biological mechanisms: It enables precise tracking and annotation of cfDNA fragments that make outstanding contributions to discrimination, providing tools for revealing the molecular mechanisms of cancer and discovering novel biomarkers.

[0024] This invention's model can be flexibly trained to support large-scale cohorts, heterogeneous sequencing batches, and cross-cancer generalization, and is automatically optimized through validation set metrics. The system supports both cross-sectional early cancer screening and longitudinal dynamic monitoring of individual MRD changes, enabling accurate prognostic assessment and recurrence early warning. The model also possesses the ability to independently discriminate and enrich specific high-risk ctDNA fragments, facilitating subsequent molecular biomarker development and mechanistic studies for precision medicine. Attached Figure Description

[0025] Figure 1 Technical roadmap of the present invention; Figure 2 The modeling flowchart of this invention; Figure 3 Training set AUC; Figure 4 Comparison of the distribution of the first 1% of read lengths predicted by the training set model; Figure 5 Comparison of GC content distribution in the top 1% of reads predicted by the training set model; Figure 6 Comparison of CNV before and after model prediction; Figure 7 Test set AUC; Figure 8 The test set showed the predictive sensitivity of various cancer types. Figure 9 Box plots of scores for each phase of the test set; Figure 10 Relapse-free survival curve; Figure 11 No-progression survival curve. Detailed Implementation

[0026] With the rapid advancements in artificial intelligence technologies, especially deep learning and language modeling methods, deep sequence models such as Transformer have demonstrated unprecedented information interpretation and feature extraction capabilities in the field of genomics. Genome language models can treat DNA sequences as a high-dimensional "language," automatically identifying key patterns in the sequence from an assumption-free perspective and capturing weak tumor signals from ctDNA, including but not limited to implicit features such as base combinations, fragment distribution, and epigenetic modifications. This provides a groundbreaking technological foundation for ultrasensitive cancer signal identification in blood using cfDNA and for early trace detection of MRD.

[0027] This invention proposes a systematic approach that directly uses deep genomic language models such as Transformer to accurately identify cancer signals from whole-genome cfDNA sequences and applies it to early screening of multiple cancer types and MRD monitoring. This approach integrates whole-genome cfDNA sequencing and advanced sequence language model-based cancer signal identification technologies, enabling early screening and MRD detection of single or multiple cancer types while improving sensitivity, specificity, and generalization ability. It has significant scientific innovation value and broad clinical application prospects.

[0028] This invention first requires the extraction, library construction, and sequencing of cfDNA from blood samples. The extraction and library construction methods are not particularly limited and can be adapted from existing extraction methods. The sequencing process can utilize existing sequencing technologies to obtain the base information of the cfDNA.

[0029] The classification information of the multiple cancer samples included in this invention is as follows:

[0030] The dataset used in the model construction process of this invention is as follows: each classification sample is obtained by random selection.

[0031]

[0032] In this invention, 66 samples with relapse-free survival (RFS) information and 133 samples with progression-free survival information were collected to evaluate the predictive performance of the model in MRD detection; the above samples were obtained from the applicant's own clinical testing database.

[0033] Methods for extracting and sequencing plasma cfDNA samples: 8ml of whole blood was collected from the patient using purple blood collection tubes (EDTA anticoagulant tubes). Plasma was immediately separated by centrifugation (within 2 hours) and transported to the laboratory. ctDNA was extracted from the plasma samples using the QIAGEN plasma DNA extraction kit according to the manufacturer's instructions. After library construction, 5x WGS sequencing was performed on the collected cfDNA samples. After obtaining the sequencing data, the data was aligned to the human reference genome to obtain the base information of the corresponding reads.

[0034] In this invention, the cfDNA data input into the model for both cancer samples and healthy human samples is obtained as follows: Sequencing data of the cfDNA is obtained separately. In the aligned BAM file, the quality, length, and alignment position information of each read are recorded. The human reference genome is the hg19 sequence provided by the University of California, Cruz. Using the Python package pysam, all paired sequencing reads with a mapping quality score ≥30 are extracted from the BAM file. The start and end coordinates of the cfDNA fragment on the genome are determined based on the alignment results of each pair of reads. To enhance the coverage of endpoint variations in the corresponding fragments, the start coordinate of each segment is extended upstream by 6 bases, and the end coordinate is extended downstream by 6 bases. Using the adjusted coordinates, the unmutated original cfDNA sequence is obtained from the hg19 reference genome. Only reference fragments with a length of 250 bp or less are retained as samples for subsequent analysis (DNA fragments longer than 250 bp may originate from non-apoptotic processes or be contaminated with genomic DNA released from leukocyte lysis during blood collection and processing; these long fragments do not carry typical apoptotic characteristics and are considered noise and interference for cancer signal identification based on fragment length patterns). For both cancer and healthy samples, the respective cfDNA obtained by the above method is aligned to the reference genome, and the corresponding sequence information is extended as input to the model. All obtained cfDNA fragment sequences and their chromosome coordinates are stored in HDF5 format for efficient downstream batch processing and analysis. Basic Language Model Training and Fine-tuning This step is used for basic model pre-training. A large number of DNA sequences are input into the model as language data. By incorporating DNA information from species other than humans as training text data, the model's ability to understand DNA text can be improved. In the cancer signal prediction model training method described in this invention, firstly, reference genome sequences from 97 species, including humans, are used. Each sample is divided into multiple DNA fragments of uniform length (e.g., each fragment is 512 bases long), maintaining the original base order. Subsequently, these DNA fragments are byte-pair encoded using a method that starts with characters and gradually merges high-frequency pairs to obtain sub-word units. This forms the input sequence for language modeling. This input is then fed into a BERT pre-trained model built on a transformer architecture. Because small fragments with specific functions appear as a whole, BPE can automatically discover these units. The model directly understands DNA text at the unit level, improving efficiency and depth, and understands the relationship between any two DNA sub-word fragments in a sequence based on a self-attention mechanism.

[0035] The model contains 15 transformer encoder layers to learn latent linguistic structural features from DNA fragments, with 12 attention heads per layer. During pre-training, a masked language model (MLM) mechanism is used to predict randomly masked segments in the input sequence, with the random masking ratio controlled at 15%. The loss function is defined as follows:

[0036] Where M represents the set of locations that are masked, x i Let θ represent the i-th masked base fragment, and θ be the model parameters. The goal is to maximize the model's prediction probability for the masked fragment.

[0037] In a cancer patient's blood sample, only a very small number of cfDNA fragments originate from the tumor, while the vast majority come from normal cells. Therefore, after completing the basic pre-training of the BERT model, it is necessary to enter a fine-tuning stage. In this stage, tagged DNA fragments are input into the BERT model. These tagged DNA fragment data come from labeled sample data of cancer patients and healthy individuals. Based on the pre-trained BERT model's ability to understand DNA sequences, the model is trained specifically to associate sequence features with diagnostic classification. The CLS vector representation of its last layer is extracted to represent the features used for binary classification prediction of cancer signals. The fine-tuning of the BERT model is accomplished through joint training of a multi-instance learning model. Specifically, to achieve joint modeling of multiple DNA fragments in the overall sample, a multi-instance learning attention mechanism is introduced. Suppose a sample contains K DNA fragments, each fragment Xk Features extracted by BERT are represented as vector H k At this point, all segment representations are input into the attention module, and the learnable attention weights a are used. k Each segment is weighted, and the weights are calculated as follows:

[0038] Where V, U, and W2 are trainable parameter matrices, ⊙ denotes element-wise multiplication, and σ denotes the sigmoid function; in the following specific implementation process, H k The dimension of V is 768, the dimension of the intermediate hidden layer is set to M=256, the dimensions of V and U are typically 768×M, and the dimension of W2 is M×1. This mechanism can automatically identify segments that contribute significantly to cancer prediction, thereby calculating the overall representation at the sample level through weighted summation.

[0039] Finally, the fused sample-level representation Y is input into a classifier consisting of a multi-layer fully connected neural network (DNN) (using two layers, with the first layer having 256 neurons). Cancer signal prediction is performed using the ReLU activation function, Dropout mechanism (20%), and a logits output layer. The final output probability is used to determine whether there is a cancer-related signal in the sample. The training process of this neural network uses a binary cross-entropy loss function with logits for parameter updates, and optimizes the parameters of all modules through backpropagation, ultimately realizing an end-to-end cancer signal prediction model training process.

[0040] The training objective is to minimize the cross-entropy loss function, which has the following form:

[0041] Where y∈{0,1} represents the true label of whether the sample contains cancer signals, and p∈(0,1) represents the model’s prediction probability of cancer association for the sample. This process enables the model to learn attention patterns associated with cancer signals in a single DNA fragment.

[0042] This invention employs a Multiple-Instance Learning (MIL) deep learning framework. Each cfDNA sequencing BAM file is treated as a "bag," and the entire file is labeled (cancer / non-cancer). However, specific cfDNA fragments (instances) within each bag are not assigned independent labels. For healthy samples, it is assumed that all cfDNA fragments originate from non-tumor tissue; while for cancer samples, fragments may originate partly from cancer tissue and partly from non-cancer tissue. Using this weakly supervised approach, cancer signals are identified through self-learning via the following deep model structure.

[0043] For each sample, a fixed number of cfDNA fragments (3,000 fragments per batch, patch sampling) are randomly selected. Byte Pair Encoding (BPE) is used to encode the DNA sequences, forming a vocabulary of 4096 tokens. This breaks down long nucleic acid sequences into tokenized short fragments to fit the input of a deep self-attention sequence processing network. All fragment sequence tokens are encoded via an embedding layer and input into a multi-layer self-attention encoder model to obtain fixed-length, high-dimensional fragment feature vectors (each fragment has 768 dimensions). The feature vector sets of the same batch of fragments are concatenated into a two-dimensional matrix and mapped through a fully connected layer to obtain a new feature space representation H. Using an attention weighting mechanism, weights are assigned to each fragment. These weights depend on model parameterization functions (such as activation functions and gating mechanisms) and can automatically adjust the importance of each fragment in the overall discrimination. The feature vectors of each fragment are weighted and averaged or aggregated with their attention weights to obtain a high-dimensional "bag-level" representation Y of the entire sample. The Y feature is further processed by a multilayer perceptron and a normalization layer, and then passed through the final classification output layer to obtain the cancer risk score or classification label of the sample (bag).

[0044] The model was trained using an NVIDIA H200 GPU, with 3000 patches. A binary cross-entropy loss function and an adaptive optimizer such as AdamW (learning rate 1×10⁻⁶) were employed. -4 The weight decays to 1×10 -3 The gradient accumulation steps are 8. The overall training cycle of the model can reach 5,000 generations, during which the parameters of the embedding layer, self-attention network, MIL, and perceptron are jointly optimized. During training, the validation set AUC and other indicators are continuously monitored, and the model weights with the best performance are finally saved.

[0045] During the inference phase, 128,000 cfDNA fragments are randomly selected from each sample, divided into 1000 mini-batches (128 fragments per mini-batch), and sequentially input into the deep model to complete feature encoding. The intermediate encoding results of all mini-batches (128,000×768 dimensional matrices) can be saved as intermediate files such as pickle to support efficient batch processing in memory-constrained environments. All fragment encoding results are then input into the attention aggregation and perceptron module, outputting the final risk score or classification conclusion for each sample.

[0046] To enrich tumor-derived cfDNA fragments, the attention aggregation and global classification layers in the trained model can be removed, retaining only the fragment coding and perceptron scoring components. An independent risk score is output for each cfDNA fragment, and the top 1% of high-scoring fragments are selected based on their scores; these are the ctDNA-enriched fragments. This set can be further used downstream for various applications such as functional validation and tumor marker development.

[0047] All of the above processes can be implemented on an automated pipeline platform, supporting automatic execution from raw data processing, fragment extraction, feature encoding, classification, ctDNA fragment screening to report generation. The output includes each sample's cancer risk score, classification label, detailed sequence coordinates of high-risk ctDNA fragments, and corresponding scores.

[0048] The training set AUC obtained by the above model is 0.993. Figure 3 The top 1% of reads predicted by each sample in the training set were collected, and 1% of reads were randomly selected from each sample for comparison in terms of length distribution and GC content distribution. The reads predicted by the model to be the highest score were generally shorter and had lower GC content, which is consistent with the characteristics of tumor-derived ctDNA length and GC content reported in current literature, proving that the model can effectively identify ctDNA signals. Figure 4 and Figure 5 The high-tumor-content sample was diluted to 0.5%, and the sequencing data of this sample was predicted using a model to extract the top 5% of reads. CNV analysis was performed on both the raw and extracted data using the same ichorCNA default workflow. The results showed that the extracted reads increased the tumor content in the sample data. Figure 6 ).

[0049] The above model yielded an AUC of 0.947 on the independent test set. The sensitivity of each cancer type in the validation set to predict cancer signals with 98% specificity is shown in the table below:

[0050] like Figure 10 and Figure 11As shown, the model in this invention performed survival analyses on 66 samples with recurrence-free survival (RFS) information and 133 samples with progression-free survival (PFS) information. Based on the risk scores predicted by the model, the model divided patients into a positive cancer signal group and a negative cancer signal group, and Kaplan-Meier survival curves were plotted. The curves show that there are certain differences between the two groups in recurrence-free survival / progression-free survival. The statistical significance of this difference was further assessed using a log-rank test to verify the model's ability to predict survival prognosis.

Claims

1. A cancer signal identification method based on a cfDNA sequence language model, used for non-therapeutic and diagnostic purposes, characterized in that, Includes the following steps: a. Data acquisition and processing steps: Obtain cfDNA from tumor samples and healthy human samples, and sequence them to obtain cfDNA fragment sequences. Align these fragments to a reference genome to obtain their positions on the reference genome. Then, extend the sequences upstream and downstream by several bases. Use the extended sequence information on the reference genome as the data information for a single sample. b. Pre-training of the basic model: The reference genome DNA of multiple species containing humans is divided into segments, and the model is trained through a masked language model mechanism while maintaining the original order, in order to learn the potential language structure features of the DNA sequence. c. Model fine-tuning: Input the data information of tumor samples and healthy human samples obtained in step a into the pre-trained model obtained in step b, and convert them into their respective segment feature vectors; use a multi-instance learning model to perform weighted fusion of multiple segment feature vectors obtained in step b to generate a sample-level representation of the test sample; wherein, each sample contains a sample classification label; input the sample-level representation into the classifier to obtain the probability of having cancer; d. Classification and prediction steps: Input the data of the sample to be tested into the fine-tuned model to predict the probability of cancer.

2. A method for constructing a cfDNA sequence language model for non-therapeutic and diagnostic purposes, comprising the following steps: a. Data acquisition and processing steps: Obtain cfDNA from tumor samples and healthy human samples, and sequence them to obtain cfDNA fragment sequences. Align these fragments to a reference genome to obtain their positions on the reference genome. Then, extend the sequences upstream and downstream by several bases. Use the extended sequence information on the reference genome as the data information for a single sample. b. Pre-training of the basic model: Reference genomic DNA from multiple species, including humans, is divided into segments. While maintaining the original order, the model is trained using a masked language model mechanism to learn the latent linguistic structural features of the DNA sequence; c. Model fine-tuning: Input the data information of tumor samples and healthy human samples obtained in step a into the pre-trained model obtained in step b, and convert them into their respective segment feature vectors; use a multi-instance learning model to perform weighted fusion of multiple segment feature vectors obtained in step b to generate a sample-level representation of the test sample; wherein, each sample contains a sample classification label; input the sample-level representation into the classifier to obtain the probability of having cancer.

3. The construction method according to claim 2, characterized in that, In step b, dividing the DNA into segments ensures that the DNA segments are of consistent size; in step b, the pre-training uses a pre-trained language model based on the Transformer architecture and uses a self-attention mechanism to understand the relationship between any two DNA sub-word segments in a sequence.

4. The construction method according to claim 2, characterized in that, The multi-instance learning model in step c calculates the attention weights of the segment feature vectors through an attention mechanism, and performs a weighted summation of the segment feature vectors based on the attention weights to generate the sample-level representation.

5. The construction method according to claim 2, characterized in that, The classifier in step d is a multi-layer fully connected neural network. Its training process uses the binary cross-entropy loss function to update parameters, and optimizes the parameters of the feature extraction module, the multi-instance learning model and the classifier through backpropagation to achieve end-to-end model training.

6. The construction method according to claim 2, characterized in that, In the sequencing of cfDNA fragments in step a, when the sequencing data is aligned to the reference genome, cfDNA fragments with alignment quality scores not lower than a preset threshold need to be selected; and the length of the selected cfDNA fragments is limited to no more than a predetermined length threshold.

7. The construction method according to claim 2, characterized in that, The preset threshold for the alignment quality score is 20-40; the preset length threshold for the cfDNA fragment is 200-300 bp; the pre-trained language model built based on the Transformer architecture adopts the BERT model, and the pre-trained language model contains 10-20 Transformer encoder layers; the fragment feature vector is a 600-900 dimensional vector.

8. The construction method according to claim 2, characterized in that, During model training or inference, multiple cfDNA fragment sequences from the test samples are input into the model in batches; each batch contains 2000-4000 cfDNA fragment sequences. It also includes: using the calculated attention weights to score the cancer signal contribution of individual cfDNA fragments and screening out a predetermined proportion of high-risk cfDNA fragments for minimal residual disease (MRD) monitoring or biomarker development.

9. A cancer signal recognition device based on a cfDNA sequence language model, comprising: Data acquisition and processing module: Used to acquire cfDNA from tumor samples and healthy human samples, sequence them to obtain cfDNA fragment sequences, align them to the reference genome to obtain their positions on the reference genome, and then extend them upstream and downstream by several bases. The sequence information on the reference genome after extension is used as the data information of a single sample. The pre-training module of the base model is used to divide reference genomic DNA from multiple species, including humans, into segments, and train the model using a masked language model mechanism while maintaining the original order, in order to learn the latent linguistic structural features of the DNA sequence. The model's fine-tuning module is used to input the data information of tumor samples and healthy human samples obtained from the data acquisition and processing module into the pre-trained model obtained from the pre-training module of the base model, converting them into their respective corresponding fragment feature vectors; a multi-instance learning model is used to weight and fuse multiple fragment feature vectors obtained from the pre-training module of the base model to generate a sample-level representation of the test sample; wherein, each sample contains a sample classification label; the sample-level representation is input into the classifier to obtain the probability of having cancer; Classification and prediction module: Used to input the data of the sample to be tested into the fine-tuned model to predict the probability of cancer.

10. A computer-readable medium having a computer program capable of running the cancer signal recognition method of claim 1.

Images