Fragment-level deep learning model (FLDL) for cancer classification

Abstract

The present disclosure provides methods and systems for training machine learning models. A method may comprise (a) receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from a subject; (b) generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality of nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment; and (c) using the fragment representation to train an attention-based sub-network, wherein the attention-based sub-network is trained to generate a sample representation of a test biological sample, based at least in part on analyzing fragment representations of the test biological sample.

Description

[0001] AttyDktNo.: 49407-793601

[0002] FRAGMENT-LEVEL DEEP LEARNING MODEL (FLDL) FOR CANCER CLASSIFICATION CROSS-REFERENCE

[0003] [1] This application claims the benefit of U.S. Provisional Application No.

[0004] 63 / 756,398, filed February 10, 2025, U.S. Provisional Application No. 63 / 964,820, filed January 21, 2026, and U.S. Provisional Application No. 63 / 970,223, filed January 28, 2026, each of which is incorporated by reference in its entirety.

[0005] BACKGROUND

[0006] [2] Cell-free deoxyribonucleic acid (cfDNA) sequencing data may be analyzed to determine a health status, such as a disease detection, disease screening, or diagnosis, of a subject. A sample may contain millions of cfDNA fragments, with relatively few being indicative of the health status or disease.

[0007] SUMMARY

[0008] [3] Next generation sequencing-based early disease (e.g., cancer) detection from cell-free DNA (cfDNA) may present an extreme-scale multiple instance learning challenge: identifying rare tumor signals amidst up to millions of instances per sample (witness rate can be as low as < 0.001%). Standard attention mechanisms may be computationally prohibitive for such tasks due to their quadratic complexity in number of instances. To address this challenge, the present disclosure provides a fragment-level deep learning (FLDL) model, an end-to-end machine learning framework utilizing an attention-based mechanism to perform dense associative retrieval over the massive instance space. The systems and methods disclosed herein may apply the FLDL model to high-impact tasks, such as detection of cancer from cfDNA. The FLDL model may be trained using a modest training dataset and may outperform other machine learning models. The FLDL model may also scale effectively with sample size or with number of instances per sample. The FLDL model may offer useful biological insights due to the interpretability of its attention weights or intermediate learned representations.

[0009] [4] In an aspect, the present disclosure provides methods for training a machine learning model, the method comprising: (a) receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from a subject, wherein the multimodal data comprises two or more data modalities selected from (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data; (b) generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality ofAttyDktNo.: 49407-793601

[0010] nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment; and (c) using the fragment representation to train an attention-based subnetwork, wherein the attention-based sub-network is trained to generate a sample representation of a test biological sample, based at least in part on analyzing fragment representations of the test biological sample. In some embodiments, the biological sample comprises a cell-free sample. In some embodiments, the cell-free sample comprises a plasma sample, a serum sample, a urine sample, a saliva sample, a cerebrospinal fluid (CSF) sample, a fecal sample, or any combination thereof. In some embodiments, the plurality of nucleic acid fragments comprises deoxyribonucleic acid (DNA) fragments or ribonucleic acid (RNA) fragments, or a combination thereof. In some embodiments, the method further comprises, prior to (a), assaying the biological sample to generate the multimodal data. In some embodiments, the assaying the biological sample comprises sequencing the plurality of nucleic acid fragments or derivatives thereof to obtain a plurality of sequencing reads. In some embodiments, the method further comprises generating the sequence data based on computer processing the plurality of sequencing reads to generate consensus fragments for the plurality of nucleic acid fragments. In some embodiments, the method further comprises generating the foundational model representations using a foundational model, based at least in part on the sequence data. In some embodiments, the foundational model is pre-trained on a human reference genome. In some embodiments, the method further comprises generating the genomic position data, based at least in part on performing an alignment of the plurality of sequencing reads to a human reference genome. In some embodiments, the method further comprises generating the strand data, based at least in part on determining whether a sequence read of the plurality of sequencing reads is derived from a forward strand or a reverse strand of a corresponding nucleic acid fragment. In some embodiments, the strand data comprises a boolean indicator indicative of a source strand of the plurality of nucleic acid fragments. In some embodiments, the assaying the biological sample comprises performing methyl conversion on the plurality of nucleic acid fragments to obtain the CpG methylation data or the methylation statistics data. In some embodiments, the methyl conversion comprises bisulfite conversion or enzymatic conversion. In some embodiments, the methyl conversion comprises the enzymatic conversion. In some embodiments, the methyl conversion comprises the bisulfite conversion. In some embodiments, the CpG methylation data comprises a per-CpG methylation status for the plurality of nucleic acid fragments. In some embodiments, the per-CpG methylation status is represented using a one-hot encoded matrix. In some embodiments, the methylation statistics data comprises sequence length, length-normalized methylated CpG count, length-normalized unmethylated CpG count, length-normalized total CpG count, or any combination thereof. In some embodiments, the method further comprisesAttyDktNo.: 49407-793601

[0011] using the multimodal data to train the encoding sub-network. In some embodiments, generating the fragment representation in (b) comprises generating a latent representation of a data modality of the two or more data modalities using a corresponding encoding model. In some embodiments, the corresponding encoding model comprises a convolutional neural network, a triangle encoder, a multi-layer perceptron, a long short-term memory (LSTM), a recurrent neural network (RNN), or any combination thereof. In some embodiments, generating the fragment representation in (b) comprises processing the latent representation of the data modality of the two or more data modalities using a multi-layer perceptron (MLP). In some embodiments, generating the fragment representation in (b) is performed at least in part using a pre-trained methylation-aware foundational model. In some embodiments, the attention-based sub-network comprises a Modem Hopfield Network (MHN). In some embodiments, the method further comprises using the attention-based sub-network to project the fragment representation to a highdimensional association space. In some embodiments, a plurality of fragment representations corresponding to the plurality of nucleic acid fragments are projected to a same high-dimensional association space. In some embodiments, the method further comprises using the attention-based sub-network to perform dense associative retrieval over the high-dimensional association space. In some embodiments, training the attention-based sub-network comprises determining a set of state patterns in the high-dimensional association space. In some embodiments, training the attention-based sub-network comprises initializing the set of state patterns. In some embodiments, training the attention-based sub-network comprises determining a value of a performance metric associated with the set of state patterns. In some embodiments, the performance metric comprises a binary cross entropy. In some embodiments, training the attention-based sub-network comprises updating the set of state patterns, based at least in part on the value of the performance metric. In some embodiments, the method further comprises using the attention-based sub-network to generate a sample representation of the biological sample, based at least in part on analyzing the fragment representation. In some embodiments, using the attention-based sub-network comprises projecting the fragment representation to the highdimensional association space, thereby obtaining a projected fragment representation. In some embodiments, using the attention-based sub-network comprises identifying an attention weight associated with the fragment representation, based at least in part on a set of learned state patterns. In some embodiments, identifying the attention weight comprises determining a dot product between the set of learned state patterns and the projected fragment representation. In some embodiments, the sample representation of the biological sample is generated based at least in part on the attention weight. In some embodiments, a complexity of the attention weight is proportional to a size of the plurality of fragments. In some embodiments, the attention weightAttyDktNo.: 49407-793601

[0012] is indicative of an importance of the nucleic acid fragment to a prediction task. In some embodiments, the method further comprises performing dimensionality reduction on the sample representation of the biological sample. In some embodiments, the dimensionality reduction comprises Uniform Manifold Approximation and Projection (UMAP). In some embodiments, the method further comprises identifying one or more clusters, based at least in part on the dimensionality reduction. In some embodiments, the method further comprises identifying, using the attention-based sub-network, one or more nucleic acid fragments associated with high attention weights. In some embodiments, the method further comprises partitioning the one or more nucleic acid fragments associated with high attention weights into one or more genomic bins. In some embodiments, the method further comprises using the sample representation of the biological sample to train a predictive sub-network, wherein the predictive sub-network is trained to predict a health status of the test subject, based at least in part on analyzing the sample representation of the test subject. In some embodiments, the health status comprises a presence or an absence of a disease, a clinical metadata, a disease stage, a disease subtype, a lesion size, or a biological characteristic of the subject. In some embodiments, the disease comprises a cancer. In some embodiments, the cancer comprises colorectal cancer, lung cancer, pancreatic cancer, gastric cancer, liver cancer, ovarian cancer, breast cancer, head or neck cancer, or any combination thereof. In some embodiments, the cancer comprises the colorectal cancer. In some embodiments, the predictive sub-network comprises an ensemble model. In some embodiments, the ensemble model comprises a plurality of classification models obtained from a plurality of cross validation folds. In some embodiments, the plurality of classification models comprises a MLP, a random forest model, a logistic regression model, a support vector machine (SVM), or any combination thereof. In some embodiments, training the predictive sub-network comprises initializing a set of model weights of the predictive sub-network. In some embodiments, training the predictive sub-network comprises determining a value of a performance metric associated with the set of model weights. In some embodiments, the performance metric comprises a sensitivity of the predictive sub-network at a pre-determined specificity value or an area under the receiver operating characteristic (AUROC). In some embodiments, the pre-determined specificity value is at least or at most about 70%, 80%, 90%, or 95%. In some embodiments, training the predictive sub-network comprises updating the set of model weights, based at least in part on the value of the performance metric. In some embodiments, training the predictive sub-network comprises updating one or more model hyperparameters, based at least in part on the value of the performance metric. In some embodiments, training the predictive sub-network comprises utilizing a dynamic minibatch balancing scheme to reduce an effect of one or more confounding variables on the prediction of the health status. In some embodiments, the encodingAttyDktNo.: 49407-793601

[0013] sub-network, the attention-based sub-network, and the predictive sub-network are trained end-to-end. In some embodiments, the method further comprises determining a panel of genomic regions of interest. In some embodiments, determining the panel of genomic regions of interest comprises determining one or more genomic regions of a target capture panel associated with the health status. In some embodiments, determining the panel of genomic regions of interest comprises filtering the one or more genomic regions to reduce a background noise. In some embodiments, the encoding sub-network or the attention-based sub-network are trained based at least in part on panel of genomic regions of interest. In some embodiments, the encoding subnetwork or the attention-based sub-network are trained using a subset of the plurality of nucleic acid fragments corresponding to the panel of genomic regions of interest.

[0014] [5] In another aspect, the present disclosure provides methods for training a machine learning model for predicting a presence or an absence of a disease, the method comprising: (a) receiving training multimodal data generated by assaying a plurality of cell-free deoxyribonucleic acid (cfDNA) fragments in a cell-free sample obtained or derived from a training subject, wherein the multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data; (b) using the training multimodal data to train a encoding sub-network of the machine learning model, wherein the encoding sub-network is trained to generate fragment representations of cfDNA fragments in a test cell-free sample, based on analyzing multimodal data of the test cell-free sample, wherein the encoding sub-network comprises a convolutional neural network (CNN), a multi-layer perceptron (MLP), a long short-term memory (LSTM), or a recurrent neural network (RNN); (c) using the encoding sub-network to generate a plurality of fragment representations, based on analyzing the training multimodal data; (d) using the plurality of fragment representations to train a Modern Hopfield Network (MHN), wherein the MHN is trained to generate a sample representation of the test cell-free sample, based on weighting and aggregating the fragment representations of the cfDNA fragments in the test cell-free sample; (e) using the MHN to generate a sample representation of the training cell-free sample, based on the plurality of fragment representations; and (f) using the sample representation of the training cell-free sample to train a predictive sub-network of the machine learning model to predict the presence or the absence of the disease in the test subject, based on analyzing the sample representation of the test subject, wherein the predictive sub-network comprises an ensemble model.

[0015] [6] In another aspect, the present disclosure provides methods for predicting a health status of a subject, the method comprising: (a) receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from the subject,AttyDktNo.: 49407-793601

[0016] wherein the multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data; (b) generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality of nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment; (c) generating, using an attention-based sub-network, a sample representation of the biological sample, based at least in part on analyzing the fragment representation; and (d) predicting, using a predictive sub-network, the health status of the subject, based at least in part on analyzing the sample representation. In some embodiments, the health status comprises a presence or an absence of a disease, a clinical metadata, or a biological characteristic of the subject. In some embodiments, the disease comprises a cancer. In some embodiments, the cancer comprises colorectal cancer, lung cancer, pancreatic cancer, gastric cancer, liver cancer, ovarian cancer, breast cancer, head or neck cancer, or any combination thereof. In some embodiments, the cancer comprises the colorectal cancer. In some embodiments, the method further comprises predicting the presence of the disease. In some embodiments, the method further comprises in response to predicting the presence of the disease, administering a treatment for treating the disease. In some embodiments, the treatment comprises a surgery, a colonoscopy, a radiation therapy, an immunotherapy, a chemotherapy, or a low-dose computed tomography (LDCT).

[0017] [7] In another aspect, the present disclosure provides computer systems, the system comprising: (a) one or more computer processors; and (b) a computer memory storing machine executable code that, upon execution by the one or more computer processors, implements the method of any one of the methods disclosed herein.

[0018] [8] In another aspect, the present disclosure provides a non-transitory computer-readable medium storing machine executable code that, upon execution by one or more computer processors, implements any one of the methods disclosed herein.

[0019] [9] In another aspect, the present disclosure provides systems for predicting a health status of a subject, the system comprising: an encoding sub-network configured to generate a fragment representation of a nucleic acid fragment in a biological sample obtained or derived from the subject, based at least in part on multimodal data of the nucleic acid fragment, wherein the multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data; an attention-based sub-network configured to generate a sample representation of the biological sample, based at least in part on analyzing the fragment representation of the nucleic acid fragment; and a predictive sub-network configured toAttyDktNo.: 49407-793601

[0020] predict the health status of the subject, based at least in part on analyzing the sample representation.

[0021]

[0010] Another aspect of the present disclosure provides a non-transitory computer readable medium comprising machine executable code that, upon execution by one or more computer processors, implements any of the methods described herein.

[0022]

[0011] Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods described herein.

[0023]

[0012] Additional aspects and advantages of the present disclosure will become readily apparent from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the present disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.

[0024] INCORPORATION BY REFERENCE

[0025]

[0013] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and / or take precedence over any such contradictory material.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027]

[0014] The novel features of the inventive concepts are set forth with particularity in the appended claims. A better understanding of the features and advantages of the inventive concepts will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the inventive concepts are utilized, and the accompanying drawings (also “Figure” and “FIG.” herein), of which:

[0028]

[0015] FIG. 1 shows an example of a method for extracting cell-free deoxyribonucleic acid (cfDNA) from a bloodstream of a subject.

[0029]

[0016] FIG. 2 shows an example of a fragment level deep learning (FLDL) model architecture. Top: The end-to-end pipeline embeds multi-modal data from millions of cfDNA fragments and aggregates them into a sample-level representation. Bottom: Hopfield PoolingAttyDktNo.: 49407-793601

[0030] aggregates fragment embeddings (Y) into a sample representation (Z) based on similarity to learned cancer-indicative prototypes (Q).

[0031]

[0017] FIG. 3 shows a bar graph depicting positive call rates among 148 replicates of the challenging contrived test set, at 90% specificity. Whiskers indicate Wilson’s method 95% Cis.

[0032]

[0018] FIG. 4 shows a graph depicting positive call rates among 148 replicates of the challenging contrived test set, at 90% specificity, for colorectal cancer panel FLDL (C-FLDL) models trained on subsets of the full training dataset (20% to 100% of the full dataset). Results are shown for the ensemble of 5 models (black) as well as for individual ensemble members.

[0033]

[0019] FIG. 5 shows an example of a UMAP projection of learned FLDL sample embeddings from the C-FLDL model, applied to real-world clinical, contrived blend, and artificial no-methylation samples.

[0034]

[0020] FIG. 6 shows a graph depicting classification relevance for genomic bins comprising the colorectal cancer (CRC) panel, in 13 correctly predicted late-stage CRC samples from real -world clinical test set; darker curve: density of high-attention fragments from C-FLDL; lighter curve: density of HMFs identified by ML Baseline; Arrows: regions prioritized by one model but not the other; Both curves are smoothed with a moving average.

[0035]

[0021] FIGs. 7A shows fragment count distributions for the Prior Denoised (PD) training set (“Train”) vs. the challenging contrived (“Contrived”) and real-world clinical (“Real-world”) test sets.

[0036]

[0022] FIG. 7B shows fragment count distributions for the CRC Panel training set (“Train”) vs. the challenging contrived (“Contrived”) and real-world clinical (“Real-world”) test sets.

[0037]

[0023] FIG. 7C shows fragment count distributions for the Expanded Region (ER) training set (“Train”) vs. the challenging contrived (“Contrived”) and real-world clinical (“Real-world”) test sets.

[0038]

[0024] FIG. 8 shows an example of a fragment embedding model: Each fragment modality is embedded by a dedicated encoder. Then, the per-modality encoding vectors are concatenated and processed using a multi-layer perceptron.

[0039]

[0025] FIG. 9 shows a model training and ensemble construction methodology using 5-fold cross validation with 5 random restarts.

[0040]

[0026] FIG. 10 shows an example of a computer system that is programmed or otherwise configured to implement methods provided herein.

[0041]

[0027] FIG. 11 shows an example of genomic and methylation data obtained by assaying DNA molecules of a subject.AttyDktNo.: 49407-793601

[0042]

[0028] FIG. 12 shows an example of genomic and methylation data obtained by assaying DNA molecules of a subject.

[0043]

[0029] FIG. 13 shows an example of a method for assessing methylation statistics of a DNA fragment.

[0044]

[0030] FIG. 14 shows an example of a method for minibatch balancing within a training dataset.

[0045]

[0031] FIG. 15 shows an example of data illustrating a training and tuning curve of a machine learning model.

[0046]

[0032] FIG. 16 shows an example of a method for pooling fragment representations into a sample representation.

[0047]

[0033] FIG. 17 shows an example of data illustrating a generalization of high-complexity FLDL models.

[0048]

[0034] FIG. 18 shows an example of data illustrating a generalization of high-complexity FLDL models.

[0049]

[0035] FIG. 19 shows an example of a method for training a FLDL model.

[0050]

[0036] FIG. 20 shows an example of a data format implementation.

[0051]

[0037] FIG. 21 shows an example of a convolutional neural network (CNN) kernel.

[0052] DETAILED DESCRIPTION

[0053]

[0038] While various embodiments of the inventive concepts have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the inventive concepts. It should be understood that various alternatives to the embodiments of the inventive concepts described herein may be employed.

[0054]

[0039] As used herein, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. Any reference to “or” herein is intended to encompass “and / or” unless otherwise stated.

[0055]

[0040] As used herein, the phrases “at least one,” “one or more,” and “and / or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, B and C,” “at least one of A, B, or C,” “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and / or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together. As used herein, the phrase “at most three” can mean less than one, one, two, or three.AttyDktNo.: 49407-793601

[0056]

[0041] Reference throughout this specification to “some embodiments,” “further embodiments,” or “a particular embodiment,” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in some embodiments,” or “in further embodiments,” or “in a particular embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0057]

[0042] Whenever the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[0058]

[0043] Whenever the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than,” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[0059]

[0044] Certain inventive embodiments herein contemplate numerical ranges. When ranges are present, the ranges include the range endpoints. Additionally, every sub range and value within the range is present as if explicitly written out. The term “about” or “approximately” may refer to within an acceptable error range for the particular value, which may depend in part on how the value is measured or determined, e.g., the limitations of the measurement system. For example, “about” may refer to within 1 or more than 1 standard deviation. Alternatively, “about” may refer to a range of up to 20%, up to 10%, up to 5%, or up to 1% of a given value. Where particular values are described in the application and claims, unless otherwise stated the term “about” generally refers to within an acceptable error range for the particular value.

[0060]

[0045] As used herein, the term “nucleic acid” generally refers to a polynucleotide comprising two or more nucleotides. It may be DNA or RNA. The nucleic acid may be a polymeric form of nucleotides of any length, either deoxyribonucleotides (dNTPs) or ribonucleotides (rNTPs), or analogs thereof. Nucleic acids may have any three-dimensional structure, and may perform any function, known or unknown. Non-limiting examples of nucleic acids include deoxyribonucleic (DNA), ribonucleic acid (RNA), coding or non-coding regions of a gene or gene fragment, loci (locus) defined from linkage analysis, exons, introns, messengerAttyDktNo.: 49407-793601

[0061] RNA (mRNA), transfer RNA, ribosomal RNA, short interfering RNA (siRNA), short-hairpin RNA (shRNA), micro-RNA (miRNA), ribozymes, cDNA, recombinant nucleic acids, branched nucleic acids, plasmids, vectors, isolated DNA of any sequence, isolated RNA of any sequence, nucleic acid probes, and primers. A nucleic acid may comprise one or more modified nucleotides, such as methylated nucleotides and nucleotide analogs. If present, modifications to the nucleotide structure may be made before or after assembly of the nucleic acid. The sequence of nucleotides of a nucleic acid may be interrupted by non-nucleotide components. A nucleic acid may be further modified after polymerization, such as by conjugation or binding with a reporter agent. A “variant” nucleic acid is a polynucleotide having a nucleotide sequence identical to that of its original nucleic acid except having at least one nucleotide modified, for example, deleted, inserted, or replaced, respectively. The variant may have a nucleotide sequence at least about 80%, 90%, 95%, or 99%, identity to the nucleotide sequence of the original nucleic acid.

[0062]

[0046] As used herein, the term “subject” generally refers to an individual, entity or a medium that has or is suspected of having testable or detectable genetic information or material. A subject can be a person, individual, or patient. The subject can be a vertebrate, such as, for example, a mammal. Non-limiting examples of mammals include humans, simians, farm animals, sport animals, rodents, and pets. The subject may be displaying a symptom(s) indicative of a health or physiological state or condition of the subject, such as a cancer or a stage of a cancer of the subject. As an alternative, the subject can be asymptomatic with respect to such health or physiological state or condition.

[0063]

[0047] As used herein, the terms “sample” and “biological sample” generally refer to a biological sample obtained from or derived from one or more subjects. Biological samples may be cell-free biological samples or substantially cell-free biological samples, or may be processed or fractionated to produce cell-free biological samples. For example, cell-free biological samples may include cell-free ribonucleic acid (cfRNA), cell-free deoxyribonucleic acid (cfDNA), cell-free protein and / or cell-free polypeptides. A biological sample may be tissue (e.g., tissue obtained by biopsy), blood (e.g., whole blood), plasma, serum, sweat, urine, saliva, or a derivative thereof. Cell-free biological samples may be obtained or derived from subjects using an ethylenediaminetetraacetic acid (EDTA) collection tube, a cell-free RNA collection tube (e.g., Streck), or a cell-free DNA collection tube (e.g., Streck). Cell-free biological samples may be derived from whole blood samples by fractionation. Biological samples or derivatives thereof may comprise cells. For example, a biological sample may be a blood sample or a derivative thereof (e.g., blood collected by a collection tube or blood drops), a tumor sample, a tissue sample, a urine sample, or a cell (e.g., tissue) sample.AttyDktNo.: 49407-793601

[0064] Fragment Level Deep Learning (FLDL) Model

[0065]

[0048] Next generation sequencing-based early disease (e.g., cancer) detection from cell-free DNA (cfDNA) may present an extreme-scale multiple instance learning challenge: identifying rare tumor signals amidst up to millions of instances per sample (witness rate can be as low as < 0.001%). Standard attention mechanisms may be computationally prohibitive for such tasks due to their quadratic complexity in number of instances. To address this challenge, the present disclosure provides a Fragment-Level Deep Learning (FLDL) model, an end-to-end framework utilizing Modern Hopfield Attention to perform dense associative retrieval over the massive instance space. Using held-out real-world clinical or challenging contrived test sets, the systems and methods disclosed herein may apply FLDL to a high-impact task: detecting colorectal cancer from cfDNA. The performance of the FLDL model may be compared to a deep learning model without attention (e.g., using max-pooling) or a state-of-the-art machine learning method (ML Baseline). In some embodiments, the attention-based FLDL model may outperform the ML Baseline, and it may achieve this in spite of a modest training set size (e.g., n = -4500). The FLDL model may also scale effectively with sample size or with number of instances per sample. The FLDL may offer useful biological insights due to the interpretability of its attention weights or intermediate learned representations. The present disclosure may establish a new frontier for scalable, attention-based deep learning in clinical cfDNA diagnostics.

[0066]

[0049] Attention-based deep learning methods, such as transformer architectures [Vaswani et al. (2017); which is incorporated by reference herein in its entirety], may have real-world applications across domains, such as natural language processing (NLP) and computer vision [Berroukham et al. 2023); He et al. (2023); Liang et al. (2024); Latif et al. (2023); Nerella et al. (2024); each of which is incorporated by reference herein in their entirety]. In some cases, such methods may face practical limitations when applied to samples with millions of input tokens [Bertsch et al. (2023); Alva Principe et al. (2025); each of which is incorporated by reference herein in their entirety] as standard self-attention mechanisms used in these methods may have quadratic complexity. Furthermore, training such models may require large datasets, which may not be available in specialized domains. Clinical diagnostics based on nextgeneration sequencing (NGS) of DNA are one example of a task that may face such challenges. NGS workflows may produce large numbers of sequences per sample [Eraslan et al. (2019); which is incorporated by reference herein in its entirety]. Unlike natural language, this data may not be easily organized into sentences or paragraphs for efficient computation. One example of a high-impact application of NGS to oncology is detection of colorectal cancer (CRC) from cell-free DNA (cfDNA), which may circulate in blood. CRC may be associated with up to billions ofAttyDktNo.: 49407-793601

[0067] nucleotide characters in up to millions of DNA sequences per sample, and there may be relatively low training case counts (in the thousands).

[0068]

[0050] Among the cancer indications in which the present inventions may be used to detect, colorectal cancer (CRC) is noted herein as it is a common cause of cancer-related deaths, despite potential declines in incidence or mortality in recent decades [Siegel et al. (2024); which is incorporated by reference herein in its entirety], CRC screening may improve cancer treatment or prevention by detecting or prompting removal of premalignant polyps. CRC screening may further improve cancer survival through early detection [Gupta (2022); which is incorporated by reference herein in its entirety]. Modalities for CRC screening include colonoscopy and non-invasive stool-based tests [Shaukat & Levin (2022); Shapiro et al. (2021); each of which is incorporated by reference herein in their entirety]. However, eligible individuals at risk of developing CRC may not be up to date with guideline-recommended screening [Bandi et al. (2025); which is incorporated by reference herein in its entirety]. Thus, non-invasive testing approaches, such as blood-based testing, may be more convenient, and may improve populationlevel adherence to CRC screening recommendations.

[0069]

[0051] Cell-free DNA (cfDNA) analysis may be useful for non-invasive blood-based tests. cfDNA circulating in blood of healthy individuals may be released by dying cells. cfDNA may be found in fragments of approximately 160-170 base pairs, and a large subset may originate from blood cells [Gao et al. (2022); Thierry (2023); Snyder et al. (2016); each of which is incorporated by reference herein in their entirety]. Unlike some other approaches of profiling cancer tissue, which may benefit from a high signal-to-noise ratio with tumor DNA concentrations sometimes exceeding 50%, the signal in cfDNA may be sparse. In cancer patients, tumors may release detectable amounts of circulating tumor DNA (ctDNA) at observed ranges from less than about 0.05% to more than about 90% of total cfDNA. In some cases, a tumor may release little to no detectable ctDNA (<0.001%) [Luo et al. (2021); Bettegowda et al. (2014); each of which is incorporated by reference herein in their entirety], cfDNA signal may be analyzed using various approaches, including fragmentomics, detection of somatic mutations, or DNA methylation. A DNA methylation approach may comprise detecting the conversion of cytosine to 5-methylcytosine (5mC) at CpG dinucleotides. Aberrant methylation patterns among cfDNA may be a non-invasive cancer biomarker [Yamaguchi et al., 2003; which is incorporated by reference herein in its entirety], NGS-based blood tests that detect cancer-associated methylation signatures associated with advanced colorectal neoplasia, including CRC, and advanced precancerous lesions (APLs) may be on track to receive regulatory approval [Shaukat et al. (2025); Chung et al. (2024); which are incorporated by reference herein in their entirety]. However, DNA methylation tests may be associated with computational challenges. It may beAttyDktNo.: 49407-793601

[0070] computationally difficult or challenging to identify a small number of ctDNA fragments out of a dataset with up to millions of healthy cfDNA fragments.

[0071]

[0052] Some methods for ctDNA detection may derive features based at least in part on biological prior knowledge. Methods for ctDNA detection may comprise summarizing methylation information by averaging or aggregating data at a per-CpG site ( >) or a per-fragment ( ) level [Li et al. (2018); which is incorporated by reference herein in its entirety]. Methods for ctDNA detection based on ct-values may utilize probabilistic deconvolution models. The probabilistic deconvolution model may be configured to estimate tumor fraction from joint methylation states of individual fragments [Li et al. (2018); which is incorporated by reference herein in its entirety]. In some cases, methods for ctDNA detection based on ct-values utilize a framework comprising methylation biomarker discovery or count-based profiling. Such methods may comprise applying stacked ensemble learners to enhance health status (e.g., cancer) or tissue of origin detection [Stackpole et al. (2022); which is incorporated by reference herein in its entirety]. In some cases, a method may comprise a fragmentomics approach. The fragmentomics approach may comprise considering fragment start / stop positions [Cristiano et al. (2019); which is incorporated by reference herein in its entirety]. The fragmentomics approach may utilize summarized fragment length profiles or locus specific coverage across the genome to classify samples. In some cases, methods may comprise a hybrid approach. The hybrid approach may comprise utilizing low-complexity deep learning to integrate fragment size information with additional fragmentomics, methylation, or other biological signal types [Bao et al. (2025); which is incorporated by reference herein in its entirety]. These methods may be effective but may depend on manual definitions and aggregations of biological features. These methods may not be configured to learn optimal representations directly from raw data.

[0072]

[0053] FIG. 1 shows an example of a method for extracting cell-free deoxyribonucleic acid (cfDNA) from a bloodstream of a subject for use in early cancer detection and / or screening. As shown in FIG. 1, the method may comprise obtaining a blood sample from the bloodstream of the subject. The method may comprise extracting the cfDNA fragments from the blood sample.

[0073]

[0054] Deep learning methods or domain-specific foundation models that learn from raw data may be used for methylation data analysis of cfDNA fragments obtained from a subject. Such deep learning methods may comprise the use of genomic foundation models, such as Evo 2 [Brixi et al. (2025); which is incorporated by reference herein in its entirety], DNA-BERT [Ji et al. (2021); which is incorporated by reference herein in its entirety], HyenaDNA [Nguyen et al. (2023); which is incorporated by reference herein in its entirety], or the Nucleotide Transformer [Dalia-Torre et al. (2025); which is incorporated by reference herein in its entirety]. TheseAttyDktNo.: 49407-793601

[0074] models may be trained using reference nucleotide sequences. In some cases, these models are not trained using cfDNA fragments and may be associated with limited applicability to methylation data. In some cases, pretrained models for methylation data, such as MethylNet [Levy et al. (2020); which is incorporated by reference herein in its entirety], CpGPT [Camillo et al. (2024); which is incorporated by reference herein in its entirety] or MethylGPT [Ying et al. (2024); which is incorporated by reference herein in its entirety], may be configured for microarraybased methylation data (e.g., from assays that measure [1 values). In some cases, the pretrained models for methylation data may be configured for downstream tasks such as age or mortality risk prediction. In some cases, the pretrained models for methylation data may be un-suitable for analyzing methylation sequencing data or for the task of early health status (e.g., cancer) detection.

[0075]

[0055] In some cases, methods for analyzing methylation data may comprise the use of multi-operation deep learning models configured to handle methylation sequencing data. For example, Methyl BERT [Jeong et al. (2025); which is incorporated by reference herein in its entirety], Pleiades [Niki et al. (2025); which is incorporated by reference herein in its entirety] or others [Deng et al. (2023); which is incorporated by reference herein in its entirety] may employ transformer architectures to generate fragment-level embeddings. In some cases, the multioperation deep learning models may not be trained end-to-end for final sample-level classification. In some cases, the multi-operation deep learning models may utilize heuristic aggregation of fragment-level predictions or hierarchical transformer models, which may employ multi-tier attention mechanisms to progressively pool fragments to determine patient status. It may be advantageous to leverage pre-trained representations for downstream classification in scenarios with limited training data. In some cases, a multi-operation aggregation process is associated with critical information loss in a very low signal-to-noise scenario. Robust detection of early-stage health status (e.g., cancer) may benefit from an architecture that is configured to preserve the ability to identify or adjust weights for informative fragments. Additionally, early-stage health status (e.g., cancer) detection may benefit from models trained using an end-to-end training approach, which may enable the models to learn representations optimized to capture subtle task-specific signatures.

[0076]

[0056] The present disclosure provides, inter alia, a Fragment-level Deep Learning (FLDL) model for use in determining the health status in a subject (e.g., cancer detection or screening). The FLDL model may bridge the gap between fragment-level representation learning and patient level classification. The FLDL model may comprise a deep learning method that formulates early health status change (e.g., cancer) detection as a Multiple Instance Learning (MIL) problem [Carbonneau et al. (2018); Use et al. (2018); each of which is incorporated byAttyDktNo.: 49407-793601

[0077] reference herein in their entirety]. The FLDL model may comprise a specialized attention module. The specialized attention module may be configured based on Modem Hopfield Networks [Ramsauer et al. (2020); which is incorporated by reference herein in its entirety], which may be effective for immune repertoire classification [Widrich et al. (2020); which is incorporated by reference herein in its entirety]. Modern Hopfield Networks may effectively aggregate signals from bags or groups comprising up to hundreds of thousands of immune receptor sequences to predict disease status. By adapting this mechanism to ctDNA detection, the methods provided herein may capture health status (e.g., cancer) signals directly from multimodal representation of cfDNA fragments. The multi-modal representations may comprise representations of fragment sequence, methylation state, genomic context, or any combination thereof. The methods disclosed herein may improve detection of ctDNA using large scale datasets, thereby solving a “needle-in-a-haystack” challenge from extreme scale.

[0078]

[0057] The methods provided herein may be used for blood-based detection of health statuses (e.g., cancer detection or screening). The methods provided herein may operate on up to billions of nucleotides from millions of multi-modal plasma-derived cfDNA fragments per subject. The methods provided herein may outperform a state-of-the-art machine learning method on contrived samples or on an independent test cohort of clinical samples. The methods provided herein may be capable of implicitly denoising a large input space. The methods provided herein may exhibit improved predictive performance as training data volume increases. The methods provided herein may support extraction of biological insights or model interpretability. The methods provided herein may allow assessment of model generalizability via learned intermediate embeddings. The methods provided herein may support the use of end-to-end deep learning models for blood-based early health status (e.g., cancer) detection.

[0079]

[0058] The present disclosure provides fragment-level deep learning models for health status (e.g., cancer) detection from, in some embodiments, a targeted Enzymatic Methylationsequencing (TEM-seq) assay using a multi-instance learning approach. The present disclosure provides models for colorectal cancer (CRC) detection. Similar models can be used for additional cancer types including, as non-limiting examples, lung cancer, pancreatic cancer, esophageal cancers, gastric cancers, liver cancer, ovarian cancer, breast cancer, head / neck cancers, prostate cancers, or any combination thereof.

[0080]

[0059] The present disclosure relates to a development of a fragment-level deep learning model (FLDL) model for health status classification using methylation data (e.g., TEM-Seq data). The present disclosure encompasses model input, model architecture, model training strategy, some aspects of model analysis like interpretability, and the like.AttyDktNo.: 49407-793601

[0081]

[0060] For developing the FLDL model, the model may be provided with a plurality of fragments per sample (without summarization based on counts) and the health status (e.g., cancer) detection task may be set up as a multiple instance learning problem. Each subject may be associated with millions of deoxyribonucleic acid (DNA) fragment sequences, with the fragment being “instance”, and each sample presenting a bag of instances (cell-free deoxyribonucleic acid (cfDNA) or disease cell DNA (e.g., ctDNA) fragments). The disease cell DNA fragments may be informative instances. During training, the FLDL model learns an intermediate representation of fragments, uses pooling mechanisms (max pooling or attention) to identify a subset of relevant fragments, learns a sample-level representation based on these relevant fragments, and predicts sample health status.

[0082]

[0061] The models for health status detection described herein, including models for colorectal cancer (CRC) detection, may use carefully designed features (e.g., counts of hypermethylated fragments (HMFs) and consecutive methylation counts (also a type of HMF)). The definition of an HMF and model architecture may be driven by an understanding of epigenetic regulation of gene expression and of how next generation sequencing (NGS) data behave.

[0083]

[0062] Deep learning (DL) models that learn from raw inputs (e.g., all sequenced fragments, pixels of an image) may be shown to outperform traditional machine learning (ML) models that rely on expert-guided data featurization. In domains like image processing, natural language processing, and protein folding, deep learners may outperform classical statistical modeling. This may be driven by the availability of large-scale training data to support deep learning model training, innovative model architectures e.g., long short-term memory (LSTMs), convolutional neural network (CNNs), transformers, training methodologies (e.g., selfsupervision, transfer learning) and availability of large-scale compute infrastructure.

[0084]

[0063] Given the current data size described herein, DL approaches may not consistently outperform certain other machine learning (ML) algorithms. However, DL methods may outperform alternatives for health status (e.g., cancer) detection. Keeping this in mind, a DL infrastructure is established and framework to evaluate benefits of state-of-art artificial intelligence (Al) techniques for health status (e.g., cancer) detection. Described herein is an endend deep learning model that uses all sequenced fragments from a sample (within a targeted panel) for early health status (e.g., cancer) prediction.

[0085]

[0064] The present disclosure evaluates whether deep learning models with minimal featurization outperform models with expert-guided features for health status (e.g., cancer) prediction. In some embodiments, there is a focus on 5mC TEM-seq data since it is high dimensional. In some embodiments, this can be extended to multimodal high-dimensional dataAttyDktNo.: 49407-793601

[0086] (e.g., electronic health records (EHR), larger protein features, 5hmC) and image data like that obtained from Low-dose computed tomography (LDCT).

[0087]

[0065] The present disclosure further evaluates other benefits of developing such a model. For example, it may be beneficial to be ready to apply and evaluate the performance of such models for when larger or multimodal datasets become available. It may be learned to “featurize” some aspects that the model is focusing on.

[0088]

[0066] Various deep learning and non-deep learning based models may be used for early health status (e.g., cancer) detection from methylation data. In some embodiments, health status (e.g., cancer) detection models disclosed herein may utilize a model architecture with improvements over the DeepRC model described in Modern Hopfield Networks and Attention for Immune Repertoire Classification (Widrich et al, 2020), which is incorporated by reference in its entirety herein. In some embodiments, the methods or systems disclosed herein may implement various modifications to the DeepRC model architecture. In some embodiments, the methods or systems disclosed herein may apply the modified model to the task of health status (e.g., cancer) detection.

[0089]

[0067] There are several problems with existing models. For example, one problem is lack of flexibility. Models based on statistical or non-DL Machine Learning (ML) methods may be designed with a known expert guided feature in mind. They may be unable to leverage additional information that is present in raw data (assuming additional information does exist and is important for disease detection). DL models make less assumptions about the features that may be useful, and “representations” may be learnt during training based on data, model architecture and training methodology. Another problem is model scalability and performance. DL models may improve in performance when model size gets bigger (no. of parameters increase), especially when matched with an increase in number of samples. This may not be true for statistical or non-DL ML models, whose complexity cannot always be increased.

[0090]

[0068] The present disclosure provides various advantages. One advantage is that the present disclosure is less expert guided pre-featurization, so the model has the potential to use fragment features that have not been pre-selected. Another advantage is that the present disclosure produces intermediate representations, of both fragments and samples, that can be used downstream for multiple tasks e.g., identifying differences between train & test distributions of datasets, clustering of intermediate representations of fragments and their relevance for health status (e.g., CRC) detection, build additivity models that go beyond the use of a “methyl score”, use of representations in multi-modal models (e.g., with electronic health records (EHR) data or image data). Another advantage is that the present disclosure establishes a deep learning training framework that is extendable to any health status.AttyDktNo.: 49407-793601

[0091]

[0069] The present disclosure provides a novel Attention based Pooling algorithm for at least or at most about 5 million, 10 million, 20 million, 50 million, 100 million, 200 million, or 500 million tokens. FLDL “tokens” are fragments, which are high-complexity multi-modal (many varieties of non-sequence and sequence) data. The methods and systems disclosed herein combine and pool domains in (e.g., via embeddings and concatenation). The present disclosure provides a novel approach of generating fragment (e.g., token) embeddings from per-fragment multi-modal data and generating a sample embedding over massive multiple instance learning data.

[0092]

[0070] The present disclosure provides a novel minibatch design for countering batch effects. The method may be tailored to mitigate unwanted exploitation of confounding factors in data. These unwanted confounding factors may stem from the imbalance in case-control numbers and assay differences in data collected and processed at different times. Without this minibatch design, deep learning models tend to learn unwanted confounder signals instead of the desired biologically meaningful signal. The methods and systems disclosed herein utilize a training procedure that includes creation of mini -batches to avoid undesired exploitation of confounding variables.

[0093]

[0071] The methods and systems disclosed herein may be configured to predict auxiliary tasks, such as disease (e.g., cancer) substage, age, lesion size, etc., with the weighting schema (e.g., weights associated with tasks) being decided based on parameter optimization. This may allow the model to generalize better to hold-out datasets. The auxiliary task design may be optimized for early health status (e.g., cancer) detection.

[0094]

[0072] The methods and systems disclosed herein may provide improved interpretability: The lower complexity of the pooling mechanism allows for usage of Integrated Gradients for contribution analysis. Provided herein are clustering of samples by sample embedding, identifying batch effects, identifying the hidden activation that caused the batch effect prediction, or using Integrated Gradients to identify which input features contributed to the batch effect prediction and in the end identify the batch effect.

[0095]

[0073] As disclosed herein, the FLDL model may comprise an attention-based multiple instance learning architecture. The FLDL model may employ Modern Hopfield Networks. The FLDL model may be evaluated against (i) a model without attention (e.g., which uses maxpooling) or (ii) a state-of-the-art machine learning method that uses expert-guided features and uses heuristics or statistical learning techniques.

[0096]

[0074] In some embodiments, the FLDL model with attention may outperform a model without attention and the state-of-the-art machine learning model for identifying rare circulating tumor DNA (ctDNA) fragments in next generation sequencing (NGS) data. The FLDL modelAttyDktNo.: 49407-793601

[0097] may scale effectively with sample size. The FLDL model may perform favorably, as input token count per sample increases. The FLDL model may provide high interpretability, thereby yielding biologically useful insights. In some embodiments, the FLDL model may illustrate the efficacy of specialized attention-based models in extremely low signal-to-noise environments and pave the way for an end-to-end deep learning model for blood-based early health status (e.g., cancer) detection.

[0098] Methods for Training a Machine Learning Model

[0099]

[0075] The FLDL models provided herein may be trained based on the attention mechanism of attention-based networks, such as modem Hopfield networks (MHNs) [Ramsauer et al. (2020); which is incorporated by reference herein in its entirety], MHNs may comprise a generalization of Hopfield networks (Hopfield (1982, 1984); each of which is incorporated by reference herein in their entirety). MHNs may represent a powerful trainable associative memory that may be directly integrated into deep learning architectures [Ramsauer et al. (2020); Hu et al. (2023); each of which is incorporated by reference herein in their entirety], MHNs may be successfully for reinforcement learning [Widrich et al. (2021); which is incorporated by reference herein in its entirety], contrastive learning [F' urst et al. (2022); which is incorporated by reference herein in its entirety], or tabular data [Schafl et al. (2022); which is incorporated by reference herein in its entirety], MHNs may be applied in diverse fields, such as immune repertoire classification [Widrich et al. (2020); which is incorporated by reference herein in its entirety] or chemical reaction prediction [Seidl et al. (2021); which is incorporated by reference herein in its entirety], MHNs may provide exponential storage capacity. As a result, MHNs may excel at “needle-in-a-haystack” problems, as demonstrated by the Deep Repertoire Classification (DeepRC) model [Widrich et al. (2020); which is incorporated by reference herein in its entirety]. The systems and methods disclosed herein may extend MHNs to the prediction of CRC from cfDNA.

[0100]

[0076] The health status (e.g., CRC) prediction may be formulated as a Multiple Instance Learning (MIL) problem, as in Widrich et al. (2020), which is incorporated by reference herein in its entirety. A sample may be formulated as comprising a bagX of N instances {s , . . . , SN}. For binary classification of CRC, one or more instances of the bag of instances (si) may be associated with a label J E {0, 1}. These instance level labels may be inaccessible. In some cases, a sample-level label y = max-tyt may be provided for one or more bags of instances. The method may comprise identifying the instances that are responsible for the label y to thereby correctly classify a positive sample [Foulds & Frank (2010); which is incorporated by reference herein in its entirety].AttyDktNo.: 49407-793601

[0101]

[0077] In some embodiments, the systems and methods may operate on a plurality of instances per bag (e.g., N 300, 000). In some cases, an instance of the plurality of instances comprises a sequence of up to 27 amino acids. The methods disclosed herein may represent one or more cfDNA fragments as instances. The methods disclosed herein may represent a blood sample as a bag of instances. This may result in a larger or more variable number of instances per bag (X). The number of instances in a bag may range from about 1 million to about 10 million. In some cases, the fraction of instances that indicate a positive bag label (e.g., the witness rate) may decrease from about 0.01% to less than about 0.001% of CRC-indicative cfDNA fragments. The methods disclosed herein may represent cfDNA fragments (5) using multi-modal data rather than a short sequence of amino acids. The multi-modal data may comprise fragment sequence, per-CpG methylation state, genomic context or any combination thereof. In some embodiments, the fragment sequence may be about 160-170 nucleotides. In some embodiments, the fragment sequence data of the multi-modal representation may comprise more than 2 billion nucleic acid characters for a sample.

[0102]

[0078] The methods disclosed herein may provide a model comprising an end-to-end trainable DL architecture (FIG. 2). The method may comprise using a dedicated sub-network (|)(-) (e.g., Fragment Embedding) to embed a fragment 5 independently into a fixed-sized vector hi = <|>(si) G Rm. The method may comprise aggregating a plurality of embedded fragments into a single fixed-sized vector for a sample. The aggregating may comprise using a MHN (e.g., Sample Embedding). The method may comprise determining a prediction of the sample’s CRC status using a fully-connected output network (e.g., Output Network).

[0103]

[0079] FIG. 2 shows an example of a FLDL model architecture. Top: The end-to-end pipeline embeds multi-modal data from millions of cfDNA fragments and aggregates them into a sample-level representation. Bottom: Hopfield Pooling aggregates fragment embeddings (Y) into a sample representation (Z) based on similarity to learned cancer-indicative prototypes (Q).

[0104]

[0080] The present disclosure provides Fragment-level Deep Learning (FLDL), an end-to-end MIL architecture designed to address the extreme needle-in-a-haystack challenge of early cancer detection from cfDNA in blood. By leveraging Modem Hopfield Networks to attend to rare ctDNA signals amidst millions of uninformative background fragments, FLDL may outperform a state-of-the-art ML approach or a max pooling DL alternative, for example in identifying low-signal APLs or challenging contrived samples. Additionally, the fact that the FLDL architecture can implicitly denoise large genomic search spaces suggests that FLDL may be effective for future multi-cancer detection tasks on a shared set of genomic targets.

[0105] Furthermore, FLDL may offer helpful biological interpretability: the model’s attention mechanism can be used to identify specific genomic regions driving positive classifications. ItsAttyDktNo.: 49407-793601

[0106] learned sample embedding may provide a mechanism for assessing generalizability. Finally, the analysis of scaling behavior may reveal that the model’s predictive performance may improve consistently with increasing training data volume, without reaching a plateau. This may be encouraging, as cfDNA based blood tests may generate data from hundreds of thousands or millions of patients per year. This may suggest that the FLDL model architecture may be well-positioned to leverage large future datasets for greater sensitivity.

[0107]

[0081] The systems and methods herein may successfully address the extreme-scale multiple instance learning challenge via the FLDL model. The FLDL model architecture may encourage evaluation of Modem Hopfield Networks or attention mechanisms in a broader class of biological problems (e.g., problems with low signal -to-noise ratios). The FLDL model architecture’s ability to implicitly denoise genomic search spaces suggests that it may be effectively repurposed for cfDNA-based detection of other cancers or diseases. The interpretability of the attention mechanism may allow for identification of high-attention genomic regions, potentially aiding in the discovery or better understanding of disease biomarkers.

[0108]

[0082] Colorectal cancer may be an important public health challenge and a common cause of cancer-related death, even though it may be preventable through screening (Siegel et al, 2025, which is incorporated by reference herein in its entirety). Adherence rates to screening methods such as colonoscopy or stool-based tests may be suboptimal (Bandi et al, 2025, which is incorporated by reference herein in its entirety). Blood-based tests may offer a non-invasive alternative that may increase screening adherence. A test with better sensitivity may be useful in this context, particularly for harder-to-detect cases with low tumor fraction, which may correspond to a stage when the disease is highly treatable. The FLDL model may improve upon state-of-the-art machine learning models for this application. The FLDL model may demonstrate the potential for further performance gains through training data scaling, pointing to even greater sensitivity.

[0109]

[0083] In some embodiments, confounding factors such as age, sex, race / ethnicity, or comorbidities may be used by the model for classification if they correlate with the target label in the training data. The training set disclosed herein comprises -4,500 samples. This may be substantial for the clinical diagnostics space but may be small compared to image or language model training datasets. Techniques for controlling confounding of this type (e.g., dynamic minibatch balancing) may be less effective in smaller training datasets. Further, there may be unknown confounders. The FLDL model's performance on a fully independent real-world clinical test set may be encouraging. In some embodiments, continued monitoring or validation in additional diverse, representative datasets may be useful for confirming generalizability. ItAttyDktNo.: 49407-793601

[0110] may also be important for these expanded datasets to increase sample counts for currently underrepresented subpopulations.

[0111]

[0084] In some embodiments, the systems and methods herein utilize clinical data, which may comprise sensitive patient information or high-resolution genomic sequences. In some embodiments, the clinical data may be subject to privacy regulations and may not be shared. In some embodiments, the methods and systems disclosed herein comprise obtaining written informed consent or redacting personally identifiable information (PII). In some embodiments, the PII may be redacted prior to use in model development.

[0112]

[0085] In some aspects, the present disclosure provides a method for training a machine learning model. In some embodiments, the method comprises receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from a subject. In some embodiments, the multimodal data comprises two or more data modalities selected from (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data. In some embodiments, the method comprises generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality of nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment. In some embodiments, the method comprises using the fragment representation to train the attention-based sub-network. In some embodiments, an attention-based sub-network is trained to generate a sample representation of a test biological sample, based at least in part on analyzing fragment representations of the test biological sample.

[0113]

[0086] In some aspects, the present disclosure provides a method for training a machine learning model for predicting a presence or an absence of a disease. In some embodiments, the method comprises receiving training multimodal data generated by assaying a plurality of cell-free deoxyribonucleic acid (cfDNA) fragments in a cell-free sample obtained or derived from a training subject. In some embodiments, the multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data. In some embodiments, the method comprises using the training multimodal data to train an encoding sub-network of the machine learning model. In some embodiments, the encoding subnetwork is trained to generate fragment representations of cfDNA fragments in a test cell-free sample, based on analyzing multimodal data of the test cell-free sample. In some embodiments, the encoding sub-network comprises a convolutional neural network (CNN), a multi-layer perceptron (MLP), a long short-term memory (LSTM), or a recurrent neural network (RNN). In some embodiments, the method comprises using the encoding sub-network to generate aAttyDktNo.: 49407-793601

[0114] plurality of fragment representations, based on analyzing the training multimodal data. In some embodiments, the method comprises using the plurality of fragment representations to train a Modern Hopfield Network (MHN), wherein the MHN is trained to generate a sample representation of the test cell-free sample, based on aggregating the fragment representations of the cfDNA fragments in the test cell-free sample. In some embodiments, the method comprises using the MHN to generate a sample representation of the training cell-free sample, based on the plurality of fragment representations. In some embodiments, the method comprises using the sample representation of the training cell-free sample to train a predictive sub-network of the machine learning model to predict the presence or the absence of the disease in the test subject, based on analyzing the sample representation of the test subject. In some embodiments, the predictive sub-network comprises an ensemble model.

[0115] Methods for Predicting a Health Status

[0116]

[0087] In some aspects, the present disclosure provides a method for predicting a health status of a subject. In some embodiments, the method comprises receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from the subject. In some embodiments, the multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data. In some embodiments, the method comprises generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality of nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment. In some embodiments, the method comprises generating, using an attention-based sub-network, a sample representation of the biological sample, based at least in part on analyzing the fragment representation. In some embodiments, the method comprises predicting, using a predictive, the health status of the subject, based at least in part on analyzing the sample representation.

[0117]

[0088] In some embodiments, the health status comprises a presence or absence of a disease, a clinical metadata, a disease stage, a disease subtype, a lesion size, a biological state or characteristic of the subject. For example, in some embodiments the disease may comprise inflammatory disorders, metabolic disorders, nervous system disorders, neurological disorders, cancer, or any combination thereof.

[0118]

[0089] In some embodiments, the disease comprises a cancer. In some embodiments, the cancer comprises colorectal cancer, lung cancer, pancreatic cancer, esophageal cancer, gastric cancer, liver cancer, ovarian cancer, breast cancer, head or neck cancer, prostate cancer, or any combination thereof. In some embodiments, the disease stage may comprise a stage of theAttyDktNo.: 49407-793601

[0119] cancer. For example, the disease may comprise colorectal cancer, and the disease stage may comprise stage I, stage II, stage III, stage IV, or any combination thereof. In some embodiments, the disease subtype comprises a subtype of the cancer. For example, the disease may comprise an advanced precancerous lesion (APL), and the disease subtype may comprise a subtype of the APL. In some embodiments, the cancer comprises the colorectal cancer. In other embodiments, the cancer comprises the lung cancer.

[0120]

[0090] In some embodiments, the method comprises predicting the presence of the disease. In some embodiments, the method comprises, in response to predicting the presence of the disease, administering a treatment for treating the disease. In some embodiments, the treatment comprises a colonoscopy, a surgery, a radiation therapy, an immunotherapy, a chemotherapy, a gene therapy, a cell therapy, a low-dose computed tomography (LDCT), a biopsy, a hormonal therapy, a targeted therapy, a stem cell transplant therapy, or any combination thereof.

[0121] Multimodal Data

[0122]

[0091] The multi-modal data that represent the fragment 5 may comprise a plurality of input modalities. The plurality of input modalities may comprise sequence and CpG methylation data. The nucleotide sequence and per-CpG methylation status of a fragment may be represented as a one-hot encoded matrix with channels for sequenced nucleotides, reference nucleotides, or CpG methylation status. The plurality of input modalities may comprise foundation model embeddings. The foundation model embeddings may comprise learned representations based on the HyenaDNA foundation model [Nguyen et al. (2023); which is incorporated by reference herein in its entirety]. These embeddings may capture long-range genomic dependencies or high-order sequence motifs, which may be derived from large-scale training on human reference genomes. The plurality of input modalities may comprise methylation statistics. Methylation statistics may comprise a vector of summarized methylation statistics for a fragment, such as sequence length and length-normalized counts of methylated, unmethylated, or total CpGs. The plurality of input modalities may comprise genomic position data. The genomic position data may comprise the numerical location of the fragment start position in a concatenated version of the human reference genome. The plurality of input modalities may comprise strand information. The strand information may comprise a boolean indicator for the DNA strand. In some cases, the sequencing workflow may process two strands of a cfDNA source fragment separately.

[0123]

[0092] In some embodiments, the biological sample comprises a cell-free sample (e.g., cfDNA). The biological sample may be obtained from or derived from one or more subjects, the method may comprise obtaining the biological sample from the one or more subjects. TheAttyDktNo.: 49407-793601

[0124] biological sample may be cell-free. The biological sample may comprise a blood sample. The blood sample may be a plasma sample. The blood sample may be a serum sample. The blood sample may be a buffy coat sample. The biological sample may be a fecal sample. The biological sample may comprise a cellular source. The cellular source may comprise a tissue sample. The cellular source may comprise a biopsy sample. The cellular source may comprise one or more cells isolated from a cell line.

[0125]

[0093] In some embodiments, the cell-free sample comprises a plasma sample, a serum sample, a urine sample, a saliva sample, a cerebrospinal fluid (CSF) sample, a fecal sample, or any combination thereof. In some cases, the method comprises processing a blood sample. The processing may include separating a blood sample into one or more portions. For example, the processing may include separating a blood sample into more than or equal to 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, or 15 portions. The processing may include separating a plasma sample from the blood sample. The processing may include separating a plasma sample from other blood components in a blood sample. Other blood components may include, but are not limited to, buffy coat, white blood cells, lymphocytes, blood platelets, red blood cells, granulocytes, or the like. In some cases, other blood components may include plasma. The processing may include use of centrifugation. Centrifugation may be referred to as an action or process of using a centrifuge to separate fluids of different densities or liquids from solids. Centrifugation may be used to separate a blood sample to a plasma sample and other blood components.

[0126]

[0094] In some embodiments, the plurality of nucleic acid fragments comprises deoxyribonucleic acid (DNA) fragments or ribonucleic acid (RNA) fragments, or a combination thereof. The DNA may be cell-free DNA (cfDNA). The RNA may be cell-free RNA (cfRNA).

[0127]

[0095] In some cases, the method comprises processing the biological sample.

[0128] Processing the biological sample may comprise providing one or more reagents to the biological sample. The one or more reagents may comprise a chemical reagent or an enzyme. Processing the biological sample may comprise an operation of an instrument. The processing may facilitate library preparation for next generation sequencing (NGS) assays. Various commercially available kits and instruments may be available for processing the biological sample.

[0129]

[0096] In some embodiments, the method comprises, prior to (a), assaying the biological sample to generate the multimodal data. In some cases, the methods may comprise the use of a detection or diagnostic assay. In some cases, the detection or diagnostic assay comprises a genomic assay. In some cases, the genomic assay comprises molecular assays. In some cases, the molecular assay includes, but is not limited to, single variant assays, single gene assays, gene panel assays, epigenetic analysis assays (methylation-based assays comprising either wholeAttyDktNo.: 49407-793601

[0130] methylome and / or targeted methylation assays), and whole exome sequencing or whole genome sequencing.

[0131]

[0097] In some cases, the method comprises extracting the plurality of nucleic acid fragments. The plurality of nucleic acid fragments may comprise deoxyribonucleic acid (DNA) fragments or ribonucleic acid (RNA) fragments, or a combination thereof. The DNA may be cell-free DNA (cfDNA). The RNA may be cell-free RNA (cfRNA). Extracting the plurality of nucleic acid fragments may comprise extracting DNA molecules or RNA molecules from the biological sample. Extracting the plurality of nucleic acid fragments may comprise extracting cfDNA molecules or cfRNA molecules from the biological sample. The extracting the nucleic acid molecules (e.g., RNA or DNA) may comprise various methods, such as a FastDNA® Kit protocol from MP Biomedicals®, a QIAamp® DNA cell-free biological mini kit from Qiagen®, or a cell-free biological DNA isolation kit protocol from Norgen Biotek®. The extraction method may extract all RNA or DNA molecules from a sample. Alternatively, the extract method may selectively extract a portion of RNA or DNA molecules from a sample. Extracted RNA molecules from a sample may be converted to DNA molecules by reverse transcription (RT). In some cases, the biological sample may be processed without any nucleic acid extraction.

[0132]

[0098] In some cases, the method comprises applying end repair to one or more nucleic acid molecules. Applying the end repair may comprise performing end repair and A-tailing (ERAT). Applying the end repair may comprise digesting 3’ overhangs of the one or more nucleic acid molecules. Applying the end repair may comprise filling in 5’ overhangs of the one or more nucleic acid molecules. Applying the end repair may comprise adding a single adenosine (A) overhang to the 3’ end of the one or more nucleic acid molecules.

[0133]

[0099] In some cases, the method comprises ligating adapters to the plurality of nucleic acid fragments. Ligating the adapters may comprise treating the plurality of nucleic acid fragments with a deoxyribonucleic acid (DNA) ligase. Ligating the adapters may comprise treating the plurality of nucleic acid fragments with a polynucleotide kinase. In some cases, the adapter ligation and the end repair are performed together. In some cases, the adapter ligation is performed without applying end repair. In some cases, the adapter ligation is performed before the methyl conversion.

[0134]

[0100] In some embodiments, the assaying the biological sample comprises sequencing the plurality of nucleic acid fragments or derivatives thereof to obtain a plurality of sequencing reads, some cases, the sequencing comprises next generation sequencing. This may permit a high depth of reads to be achieved for a given region. These may be high-throughput methods that include, for example, Illumina® (Solexa) sequencing, DNB-Sequencer T7 (DNBSEQ®) or G400 (MGI Tech Co., Ltd), GenapSys® sequencing (GenapSys, Inc.), Roche 454 sequencing (RocheAttyDktNo.: 49407-793601

[0135] Sequencing Solutions, Inc.), Ion Torrent sequencing (Thermo Fisher Scientific), and SOLiD sequencing (Thermo Fisher Scientific®). The number of sequencing reads may be adjusted depending on DNA input amount and depth of data required for analysis. In some embodiments, the sequence data is generated based on the plurality of sequencing reads. In some embodiments, the sequence data comprises a one-hot encoding of a sequencing read of the plurality of sequencing reads.

[0136]

[0101] In some cases, the method comprises hybridization. The hybridization method provided herein may be used in various formats of nucleic acid hybridizations, such as insolution hybridization and such as hybridization on a solid support (e.g., Northern, Southern and in situ hybridization on membranes, microarrays and cell / tissue slides). In particular, the method is suitable for in-solution hybrid capture for target enrichment of certain types of genomic DNA sequences (e.g., exons) employed in targeted next-generation sequencing. For hybrid capture approaches, a cell-free nucleic acid sample is subjected to library preparation. As used herein, “library preparation” comprises end-repair, A-tailing, adapter ligation, or any other preparation performed on the cell -free DNA to permit subsequent sequencing of DNA. In certain examples, a prepared cell-free nucleic acid library sequence comprises adapters, sequence tags, index barcodes that are ligated onto cell-free nucleic acid sample molecules. Various commercially available kits are available to facilitate library preparation for next-generation sequencing approaches. Next-generation sequencing library construction may comprise preparing nucleic acids targets using a coordinated series of enzymatic reactions to produce a random collection of DNA fragments, of specific size, for high throughput sequencing. Advances and the development of various library preparation technologies have expanded the application of nextgeneration sequencing to fields such as transcriptomics and epigenetics.

[0137]

[0102] Improvements in sequencing technologies have resulted in changes and improvements to library preparation. Next-generation sequencing library preparation kits, developed by companies such as Agilent®, Bioo Scientific®, Kapa Biosystems®, New England Biolabs®, Illumina®, Life Technologies®, Pacific Biosciences®, and Roche® provide consistency and reproducibility to various molecular biology reactions that ensure compatibility with the latest NGS instrument technology.

[0138]

[0103] In various examples for targeted capture gene panels, various library preparation kits may be selected from the group consisting of Nextera Flex (Illumina®), Illumina® DNA Prep (Illumina®), Ion AmpliSeq® (Thermo Fisher Scientific®), GeneXus® (Thermo Fisher Scientific®), Agilent ClearSeq (Illumina®), Agilent® SureSelect® Capture (Illumina®), Archer® FusionPlex® (Illumina®), Bioo Scientific® NEXTflex® (Illumina®), IDT® xGen (Illumina®),AttyDktNo.: 49407-793601

[0139] Illumina® TruSight® (Illumina®), NimbleGen® SeqCap® (Illumina®), and Qiagen® GeneRead® (Illumina®).

[0140]

[0104] In some embodiments, the method comprises generating the sequence data based on computer processing the plurality of sequencing reads to generate consensus fragments for the plurality of nucleic acid fragments. Generating the consensus fragments may improve a consistency of the sequence data obtained using the methods disclosed herein, and may thereby improve a classification consistency of the models disclosed herein.

[0141]

[0105] In some embodiments, the method comprises generating the foundational model representations using a foundational model, based at least in part on the sequence data. In some embodiments, the foundational model may be pre-trained on a plurality of nucleic acid sequences. In some embodiments, the foundational model may be pre-trained to learn patterns associated with nucleic acid sequences. In some embodiments, the foundational model representations are configured to capture informative patterns of the sequence data. In some embodiments, the foundational model representations have a smaller size, compared to the sequence data. In some embodiments, the foundational model representations provide a compressed representation of the sequence data while preserving informative patterns useful for downstream analyses. In some embodiments, the foundational model representations may be configured to capture long-range genomic dependencies or high-order sequence motifs.

[0142]

[0106] In some embodiments, the method comprises generating the genomic position data, based at least in part on performing an alignment of the plurality of sequencing reads to a human reference genome. As disclosed herein, the human reference genome may comprise GRCh37 or GRCh38. In some embodiments, the genomic position data comprises a start position, an end position, or an orientation, relative to the human reference genome. In some embodiments, the human reference genome comprises a concatenated human reference genome.

[0143]

[0107] In some embodiments, the method comprises generating the strand data, based at least in part on determining whether a sequence read of the plurality of sequencing reads is derived from a forward strand or a reverse strand of a corresponding nucleic acid fragment. In some embodiments, a nucleic acid fragment of the plurality of nucleic acid fragments comprises two strands. In some embodiments, sequence data of each strand may be processed separately. In some embodiments, the strand data is generated during the alignment of the plurality of sequencing reads with the human reference genome.

[0144]

[0108] In some embodiments, the strand data comprises a boolean indicator indicative of a source strand of the plurality of nucleic acid fragments. In some embodiments, the value of the boolean indicator identifies whether a sequencing read was derived from a forward strand or a reverse strand.AttyDktNo.: 49407-793601

[0145]

[0109] In some embodiments, the assaying the biological sample comprises performing methyl conversion on the plurality of nucleic acid fragments to obtain the CpG methylation data or the methylation statistics data. In some cases, the methylation conversion comprises combination conversion. The methyl conversion may convert unmethylated cytosine nucleotides to into uracil nucleotides. The methyl conversion may be performed before PCR amplification. The methyl conversion may facilitate the differentiation of methylated and unmethylated cytosine nucleotides in methylation sequencing methods.

[0146] [HO] In some embodiments, the methyl conversion comprises bisulfite conversion or enzymatic conversion. In some cases, the methyl conversion comprises reagent-based conversion. The reagent-based conversion may comprise bisulfite conversion.

[0147] [Hl] In some embodiments, the methyl conversion comprises the enzymatic conversion. The enzymatic conversion may comprise enzymatic methyl conversion. Data may suggest that the process of generating EM-seq libraries does not damage DNA in the same way as bisulfite sequencing. EM-seq libraries may give higher PCR yields despite using fewer PCR cycles for all DNA input amounts, indicating that less DNA is lost during enzymatic treatment and library preparation, as compared to whole genome bisulfite sequencing (WGBS). Reduced PCR cycles, in turn, may translate into more complex libraries and fewer PCR duplicates during sequencing. EM-seq libraries also may have larger average insert sizes than WGBS which further supports the fact that DNA remains intact. In the EM-seq workflow, TET2 oxidizes 5-mC and 5-hmC, providing protection from deamination by APOBEC in the next operation. In contrast, unmodified cytosines are deaminated to uracils. In some embodiments, the targeted method comprises enzymatic conversion of nucleic acid (TEM-seq). In some embodiments, the methylation sequencing methods are accomplished with the NEBNEXT® Enzymatic Methyl-seq (New England Biolabs, Ipswich, MA) which is useful for identification of 5mC and 5hmC.

[0148]

[0112] APOBEC-coupled epigenetic sequencing (ACE-seq) excludes bisulfite conversion altogether and relies on enzymatic conversion to detect 5hmC (e.g., as described by Schutsky, E.K., et al., Nat. Biotechnol., 2018 Oct 8, the contents of which are incorporated herein by reference). With this method, T4-BGT glucosylates 5hmC to 5ghmC and protects it from deamination by Apolipoprotein B mRNA editing enzyme subunit 3 A (APOBEC3 A). Cytosine and 5mC are deaminated by APOBEC3 A and sequenced as thymine.

[0149]

[0113] In some cases, the methylation sequencing comprises TET-assisted pyridine borane sequencing (TAPS). TAPS may provide a bi sulfite-free and base-level-resolution sequencing method for detection of 5mC and 5hmC. TAPS combines ten-eleven translocation (TET) oxidation of 5mC and 5hmC to 5-carboxylcytosine (5caC) with pyridine borane reduction of 5caC to dihydrouracil (DHU). Subsequent PCR converts DHU to thymine, enabling a C-to-TAttyDktNo.: 49407-793601

[0150] transition of 5mC and 5hmC. TAPS detects modifications directly with high sensitivity and specificity, without affecting unmodified cytosines, (e.g., as described by Liu, Y., et al. Nat Biotechnol. 2019 Apr;37(4):424-429, which is incorporated by reference herein in its entirety).

[0151]

[0114] TET-assisted 5-methylcytosine sequencing (TAmC-seq) enriches for 5mC loci and utilizes two sequential enzymatic reactions followed by an affinity pull-down (e.g., as described by Zhang, L. 2013, Nat Commun 4: 1517, the contents of which are incorporated herein by reference). Fragmented DNA is treated with T4-BGT which protects 5hmC by glucosylation. The enzyme mTETl is then used to oxidize 5mC to 5hmC, and T4-BGT labels the newly formed 5hmC using a modified glucose moiety (6-N3 -glucose). Click chemistry is used to introduce a biotin tag which enables enrichment of 5mC-containing DNA fragments for detection and genome wide profiling.

[0152]

[0115] In some cases, the method comprises targeted methylation sequencing. In targeted methylation sequencing approaches, targeted regions in a biological sample such as cfDNA are analyzed in order to determine the methylation state of the target gene sequences. In some embodiments, the target region comprises, or hybridizes under stringent conditions to, contiguous nucleotides of target regions of interest, such as at least about 16 contiguous nucleotides of a target region of interest. In different examples, targeted sequencing may be accomplished using hybridization capture and amplicon sequencing approaches.

[0153]

[0116] In some embodiments, the methyl conversion comprises the bisulfite conversion. Bisulfite conversion may refer to a tool used to assess CpG methylation status. A frequently used method for analyzing DNA for the presence of 5-methylcytosine (5-mC) is based upon the reaction of bisulfite with cytosine whereby, upon subsequent alkaline desulfonation, cytosine is converted to uracil which corresponds to thymine in its base pairing behavior. For example, genomic sequencing has been adapted for analysis of DNA methylation patterns and 5-methylcytosine distribution by using bisulfite treatment (e.g., as described by Frommer et al., Proc. Natl. Acad. Sci. USA 89:1827-1831, 1992, the contents of which are incorporated herein by reference). Significantly, however, 5-methylcytosine remains unmodified under these conditions. Consequently, the original DNA is converted in such a manner that methylcytosine (methyl-C), which originally could not be distinguished from cytosine by its hybridization behavior, can now be detected as the only remaining cytosine using various molecular biological techniques, for example, by amplification and hybridization, or by sequencing. In various examples, other reagents may affect the same result as bisulfite modification useful for methylation sequencing. Targeted Bisulfite Sequencing may refer to a commercially available NGS method used to evaluate site-specific DNA methylation changes. Probes are designed to be strand-specific as well as bisulfite-specific. Both methylated and unmethylated sequences areAttyDktNo.: 49407-793601

[0154] amplified. The process is similar to pyrosequencing but offers a much higher throughput overall. In some embodiments, next-generation sequencing platforms are used to deliver large amounts of useful DNA methylation information (e.g., EPIGENTEK, Farmingdale, NY and ZYMO RESEARCH, Irvine, CA). The methylation analysis at single-base resolution of individual cytosine in DNA may be facilitated by bisulfite treatment of DNA followed by PCR amplification of targeted region, library construction, and sequencing of the amplicon regions. Specific primers may be designed for the region of interest and cytosine methylation changes are evaluated within that region. Each DNA methylation site of interest may be assessed at high-sequencing depth of coverage for accurate, quantitative and single-base resolution data output.

[0155]

[0117] In another example, 5hmC may be also detected using TET-assisted bisulfite sequencing (TAB-seq) (e.g., as described by Yu, M., et al. (2012). Nat. Protoc. 7, 2159-2170, the contents of which are incorporated herein by reference) (WiseGene; Illumina®). Fragmented DNA may be enzymatically modified using sequential T4 Phage B-glucosyltransferase (T4-BGT), and then Ten-eleven translocation (TET) dioxygenase treatments before the addition of sodium bisulfite. T4-BGT glucosylates 5hmC to form beta-glucosyl-5-hydroxymethylcytosine (5ghmC) and TET is then used to oxidize 5mC to 5caC. Only 5ghmC is protected from subsequent deamination by sodium bisulfite and this enables 5hmC to be distinguished from 5mC by sequencing. Oxidative bisulfite sequencing (oxBS) provides another method to distinguish between 5mC and 5hmC (e.g., as described by Booth, M.J., et al., 2012 Science 336: 934-937, the contents of which are incorporated herein by reference). The oxidation reagent potassium perruthenate converts 5hmC to 5-formylcytosine (5fC) and subsequent sodium bisulfite treatment deaminates 5fC to uracil. 5mC remains unchanged and can therefore be identified using this method.

[0156]

[0118] In some cases, the sequencing comprises Illumina 5-base sequencing. Illumina 5-base sequencing may perform enzymatic conversion of 5mC to thymine, followed by ligationbased addition of adapters and indexes and mechanical shearing of genomic DNA. Illumina 5-base sequencing may provide dual detection of 5mC and DNA variants from a single sample, library prep, sequencing run, and analysis workflow. In some embodiments, the sequencing comprises targeted methylation sequencing. In targeted methylation sequencing approaches, targeted regions in a biological sample such as cfDNA are analyzed in order to determine the methylation state of the target gene sequences. In some embodiments, the target region comprises, or hybridizes under stringent conditions to, contiguous nucleotides of target regions of interest, such as at least about 16 contiguous nucleotides of a target region of interest. In different examples, targeted sequencing may be accomplished using hybridization capture and amplicon sequencing approaches.AttyDktNo.: 49407-793601

[0157]

[0119] In some embodiments, the CpG methylation data comprises a per-CpG methylation status for the plurality of nucleic acid fragments. In some embodiments, the CpG methylation data comprises one or more regions comprising at least a threshold number of CpG sites. In some embodiments, the CpG methylation data comprises a probability value associated with a CpG site. The probability value may comprise a likelihood that the CpG site is methylated. In some embodiments, the per-CpG methylation status comprises an indication of a type of modification (e.g., 5-methylcytosine or 5-hydroxymethylcytosine).

[0158]

[0120] In some embodiments, the per-CpG methylation status is represented using a one-hot encoded matrix. In some embodiments, methylated CpG sites are represented using a ‘ T while unmethylated CpG sites are represented using a ‘0’ in the one-hot encoded matrix. In some embodiments, the sequence data and the per-CpG methylation status may be represented together in a combined one-hot encoded matrix. In some embodiments, the combined one-hot encoded matrix comprises a plurality of channels. In some embodiments, a channel of the plurality of channels comprises the sequence data. In some embodiments, a second channel of the plurality of channels comprises the per-CpG methylation status. In some embodiments, a third channel of the plurality of channels comprises reference nucleic acid sequence data.

[0159]

[0121] In some embodiments, the methylation statistics data comprises sequence length, methylated CpG count, unmethylated CpG count, total CpG count, length-normalized methylated CpG count, length-normalized unmethylated CpG count, length-normalized total CpG count, or any combination thereof. In some embodiments, the methylation statistics data comprises a vector summary of the sequence length, methylated CpG count, unmethylated CpG count, total CpG count, length-normalized methylated CpG count, length-normalized unmethylated CpG count, length-normalized total CpG count, or any combination thereof.

[0160] Fragment Representations

[0161]

[0122] In some embodiments, the method comprises embedding an input modality of the plurality of input modalities as a vector of size m using a dedicated sub-network, thereby providing a balanced contribution from the heterogeneous data modalities. Sequence features may be processed via a ID Convolutional Neural Network (CNN), max-pooling, a multi-layer perceptron (MLP), or any combination thereof. The foundation model embeddings may be projected to m dimensions using an MLP. Scalar features like methylation statistics or position may be encoded using triangular encoding and processed by MLPs. The Boolean strand feature may be processed directly by an MLP. Individual modality embeddings, which may comprise size m, may be concatenated or processed by a MLP to form a unified fragment representation hi G Rm, as shown in FIG. 8.AttyDktNo.: 49407-793601

[0162]

[0123] FIG. 8 shows an example of a FLDL fragment embedding model: Each fragment modality is embedded by a dedicated encoder. Then, the per-modality encoding vectors are concatenated and processed using a multi-layer perceptron (MLP).

[0163]

[0124] In some embodiments, the method comprises using the multimodal data to train the encoding sub-network. In some embodiments, training the encoding sub-network comprises initializing a plurality of parameters of the encoding sub-network. In some embodiments, the plurality of parameters is initialized randomly. In some embodiments, the plurality of parameters is initialized based on pre-training. In some embodiments, training the encoding sub-network comprises determining a performance metric associated with a current value of the plurality of parameters. Determining the performance metric may comprise using the encoding sub-network to generate a fragment representation based on training multimodal data corresponding to a training nucleic acid fragment. The performance metric may be determined based on a downstream prediction performance (e.g., sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUC-ROC), etc.) associated with the fragment representation. In some embodiments, training the encoding sub-network comprises updating the plurality of parameters based in part on the performance metric. In some embodiments, the plurality of parameters is updated iteratively for a pre-determined number of iterations or until the performance metric exceeds a threshold value.

[0164]

[0125] In some embodiments, generating the fragment representation in (b) comprises generating a latent representation of a data modality of the two or more data modalities using a corresponding encoding model.

[0165]

[0126] In some embodiments, the corresponding encoding model comprises a convolutional neural network, a triangle encoder, a multi-layer perceptron, a long short-term memory (LSTM), a recurrent neural network (RNN), or any combination thereof. In some embodiments, the corresponding encoding model for the sequence data or the CpG methylation data comprises a convolutional neural network (CNN). The CNN may comprise a 1 -dimensional (ID) CNN. In some embodiments, the corresponding encoding model for the sequence data or the CpG methylation data comprises a max pooling sub-network. In some embodiments, the corresponding encoding model for the sequence data or the CpG methylation data comprises a multi-layer perceptron (MLP). In some embodiments, the corresponding encoding model for the foundational model representations comprises an MLP. In some embodiments, the corresponding encoding model for the methylation statistics data or the strand data comprises a triangle encoder.

[0166]

[0127] In some embodiments, generating the fragment representation in (b) comprises processing the latent representation of the data modality of the two or more data modalities usingAttyDktNo.: 49407-793601

[0167] a multi-layer perceptron (MLP). In some embodiments, latent representations are generated for each data modality of the two or more data modalities. In some embodiments, the latent representations of each data modality of the two or more data modalities comprise a same dimension size. In some embodiments, the latent representations of each data modality of the two or more data modalities are concatenated. In some embodiments, the latent representations of each data modality of the two or more data modalities are pooled (e.g., using max pooling).

[0168]

[0128] In some embodiments, generating the fragment representation is performed at least in part using a pre-trained methylation-aware foundational model.

[0169] Sample Representations

[0170]

[0129] In some embodiments, the method comprises aggregating fragment representations into a sample level embedding. In some embodiments, the aggregating comprises employing a Hopfield Pooling layer (Equation 1). In some embodiments, a set of K cancerindicative fragment prototypes (state patterns or queries') Q G RKxdmay be learned in a highdimensional association space. In some embodiments, a fully-connected self normalizing network NN : Rm-» Rd[ Klambauer et al. (2017); which is incorporated by reference herein in its entirety] may be applied row-wise to project the N embedded sample fragments (stored patterns) Y= [h , . . . , h . ] E RNxm(stored patterns) to the same high-dimensional association space RNxd(yielding keys). In some embodiments, a softmax-normalized dot-product similarity (attention) value between these state patterns and the projected stored patterns may be used to aggregate the embedded sample fragments Y into a sample embedding ZeRKxm(as shown in FIG. 2).

[0171] Z= softmax >NN(E)rE (Equation 1)

[0172]

[0130] In some embodiments, the scaling factor P controls the sharpness of the attention distribution, analogous to the inverse temperature in the MHN energy function. In some embodiments, fragment embeddings (X) serve two distinct roles. In some embodiments, they may be projected via NN to form keys for similarity matching. In some embodiments, the original embeddings may serve as values [Vaswani et al., 2017; which is incorporated by reference here in its entirety] for aggregation. In some embodiments, the methods disclosed herein may thereby learn to assign high attention weights to fragments resembling tumor-derived signal (ctDNA) based on optimizing the association space or learned cancer-indicative fragment prototypes. In some cases, Z is flattened into a vector z E RmKor passed to an MLP to predict the probability of CRC, or other biologically relevant auxiliary tasks.

[0173]

[0131] In some embodiments, the attention-based sub-network comprises a Modern Hopfield Network (MHN). In some embodiments, the attention-based sub-network comprises a recurrent neural network (RNN), a Transformer, an attention-based convolutional neural networkAttyDktNo.: 49407-793601

[0174] (ACNN), a self-attention network, a hierarchical attention network (HAN), a graph attention network (GAT), any combination thereof.

[0175]

[0132] In some embodiments, the method comprises using the attention-based subnetwork to project the fragment representation to a high-dimensional association space. In some embodiments, the attention-based sub-network comprises a neural network. In some embodiments, the neural network comprises a fully connected self-normalizing network. In some embodiments, the neural network may be applied row-wise to one or more fragment representations. In some embodiments, a dimensionality of the high-dimensional association space is greater than a dimensionality of the fragment representation. In some embodiments, the high-dimensional association space may enable identification or retrieval of associations within the fragment representation. In some embodiments, the high-dimensional association space may enable identification of associations between a plurality of fragment representations (e.g., a plurality of disease-associated fragment representations). In some embodiments, the projected fragment representation comprises a key of an attention mechanism of the attention-based subnetwork.

[0176]

[0133] In some embodiments, a plurality of fragment representations corresponding to the plurality of nucleic acid fragments are projected to a same high-dimensional association space. Alternatively, at least a subset of the plurality of fragment representations may be projected to different high-dimensional association spaces. In some embodiments, the plurality of fragment representations projected to the same high-dimensional association space correspond to nucleic acid fragments obtained or derived from a same biological sample.

[0177]

[0134] In some embodiments, the method comprises using the attention-based subnetwork to perform dense associative retrieval over the high-dimensional association space. In some embodiments, the dense associative retrieval comprises retrieving corresponding stored associations based on one or more queries. In some embodiments, the dense associative retrieval is part of an attention mechanism of the attention-based sub-network. In some embodiments, the corresponding stored associations may be dense if the number of stored associations is relatively large, compared to a number of data points (e.g., fragment representations).

[0178]

[0135] In some embodiments, training the attention-based sub-network comprises determining a set of state patterns in the high-dimensional association space. In some embodiments, the set of state patterns comprise a query of an attention mechanism of the attention-based sub-network. In some embodiments, the set of state patterns comprise fragment prototypes associated with a health status (e.g., cancer). In some embodiments, the fragment representation may be processed with the set of state patterns to determine an association between the fragment representation and the health status (e.g., cancer).AttyDktNo.: 49407-793601

[0179]

[0136] In some embodiments, training the attention-based sub-network comprises initializing the set of state patterns. In some embodiments, the set of state patterns is randomly initialized. In some embodiments, the random initialization comprises sampling from a random distribution or a random number generator (RNG). In some embodiments, the set of state patterns is determined based on pre-training the attention-based sub-network. The attentionbased sub-network may be pre-trained on nucleic acid fragment representations from a reference biological sample from a reference subject.

[0180]

[0137] In some embodiments, training the attention-based sub-network comprises determining a value of a performance metric associated with the set of state patterns. In some embodiments, determining the value of the performance metric comprises generating, using the attention-based sub-network comprising the set of state patterns, a sample representation of a training biological sample from a training subject, based at least in part on analyzing fragment representations of the training biological sample. In some embodiments, determining the value of the performance metric comprises comparing the sample representation of the training biological sample with a reference sample representation (e.g., determining a similarity between the sample representation of the training biological sample and the reference sample representation). In some embodiments, determining the value of the performance metric comprises inputting the sample representation of the training biological sample into a predictive machine learning model or predictive machine learning sub-network. In some embodiments, determining the value of the performance metric comprises determining a predicted label of the training biological sample, using the predictive machine learning model or predictive machine learning sub-network, based on analyzing the sample representation of the training biological sample. In some embodiments, determining the value of the performance metric comprises comparing the predicted label of the training biological sample with a ground truth label. The ground truth label may comprise a ground truth health status. The ground truth health status may be obtained from a medical record of the training subject or determined by a healthcare professional.

[0181]

[0138] In some embodiments, the performance metric comprises a binary cross entropy loss. In some embodiments, the binary cross entropy corresponds to a binary classification task (e.g., classifying a presence or an absence of a health status). In some embodiments, the performance metric comprises an accuracy, a specificity, a sensitivity, a precision, an Fl score, an area under the Receiver Operating Characteristic curve (AUROC), or any combination thereof. In some embodiments, the predictive machine learning model or predictive machine learning sub-network comprises a regression model configured to predict a continuous value (e.g., a lesion size or a predicted life expectancy). In some embodiments, the performance metric for the regression model comprises a mean squared error (MSE), a mean absolute error (MAE),AttyDktNo.: 49407-793601

[0182] an R-squared (R2), an adjusted R-squared, a root mean squared error (RMSE), or any combination thereof.

[0183]

[0139] In some embodiments, the predictive machine learning model or predictive machine learning sub-network comprises a multi-class classification model configured to classify a sample representation into a plurality of classes (e.g., a plurality of disease stages or a plurality of disease subtypes). In some embodiments, the performance metric for the multi-class classification model comprises a categorical cross-entropy loss, an Fl score, an accuracy, a precision, a recall, an area under the Precision-Recall curve (AUC-PR), or any combination thereof. In some embodiments, the predictive machine learning model or predictive machine learning sub-network is configured to classify a sample representation into the plurality of classes based on a binary prediction task. For example, the predictive machine learning model or predictive machine learning sub-network may be configured to classify a presence or an absence of each disease subtype of a plurality of disease subtypes or a presence or an absence of each disease stage of a plurality of disease stages.

[0184]

[0140] In some embodiments, training the attention-based sub-network comprises updating the set of state patterns, based at least in part on the value of the performance metric. In some embodiments, the value of the performance metric is compared to a threshold value. In some embodiments, the set of state patterns is updated if the value of the performance metric does not meet the threshold. In some embodiments, training the attention-based sub-network comprises determining a gradient associated with the performance metric. In some embodiments, training the attention-based sub-network comprises determining an updated value of the set of state patterns configured to improve the performance metric, based at least in part on the gradient. In some embodiments, the set of state patterns is updated using an optimizer. In some embodiments, the optimizer comprises Stochastic Gradient Descent (SGD), Mini-batch Gradient Descent, Adam Adaptive Moment Estimation (Adam), Root Mean Square Propagation (RMSProp), Adaptive Gradient Algorithm (AdaGrad), or any combination thereof. In some embodiments, training the attention-based sub-network comprises iteratively updating the set of state patterns. In some embodiments, the set of state patterns is iteratively updated for a predetermined number of iterations. In some embodiments, the set of state patterns is iteratively updated until the value of the performance metric meets the threshold value.

[0185] Attention Mechanism

[0186]

[0141] In some embodiments, the method comprises using the attention-based subnetwork to generate a sample representation of the biological sample, based at least in part on analyzing the fragment representation. In some embodiments, the sample representation isAttyDktNo.: 49407-793601

[0187] generated based on aggregating a plurality of fragment representations of the plurality of nucleic acid fragments in the biological sample. In some embodiments, the sample representation has a smaller size, compared to the plurality of fragment representations. In some embodiments, the sample representation preserves one or more patterns associated with a health status (e.g., cancer) within the fragment representation or the plurality of fragment representations.

[0188]

[0142] In some embodiments, using the attention-based sub-network comprises projecting the fragment representation to the high-dimensional association space, thereby obtaining a projected fragment representation. As disclosed herein, a dimension of the highdimensional association space is greater than a dimension of the fragment representation. As disclosed herein, the fragment representation may be projected to the high-dimensional association using a neural network (e.g., a fully connected self-normalizing network).

[0189]

[0143] In some embodiments, using the attention-based sub-network comprises identifying an attention weight associated with the fragment representation, based at least in part on a set of learned state patterns. As disclosed herein, the set of learned state patterns may be learned by training the attention-based sub-network. The set of learned state patterns may comprise fragment prototypes associated with a health status (e.g., cancer). The set of learned state patterns may comprise a query of an attention mechanism of the attention-based subnetwork. As disclosed herein, the fragment representation may comprise keys of the attention mechanism of the attention-based sub-network.

[0190]

[0144] In some embodiments, identifying the attention weight comprises determining a dot product between the set of learned state patterns and the projected fragment representation. In some embodiments, the dot product is multiplied by a scaling factor. In some embodiments, the scaling factor controls a sharpness of an attention distribution. In some embodiments, identifying the attention weight further comprises performing a softmax function on the dot product, after the multiplication by the scaling factor. The scaling factor may be analogous to the inverse temperature in the MHN energy function. The attention weight may be identified using Equation 1.

[0191]

[0145] In some embodiments, the sample representation of the biological sample is generated based at least in part on the attention weight. In some embodiments, the attentionbased sub-network is configured to assign higher attention weights to the fragment representation if the fragment representation is similar to a health status-derived signal. In some embodiments, the attention weight is multiplied by the fragment representation. As disclosed herein, the attention weight may be associated with the fragment representation. In some embodiments, a plurality of attention weights is identified for a plurality of fragment representations. A subset of the plurality of attention weights may correspond to each fragmentAttyDktNo.: 49407-793601

[0192] representation of the plurality of fragment representations. In some embodiments, the attention weights are used to aggregate the plurality of fragment representations. In some embodiments, the plurality of attention weights is multiplied by the plurality of fragment representations. In some embodiments, generating the sample representation further comprises flattening the product of the multiplication between the plurality of attention weights and the plurality of fragment representations.

[0193]

[0146] In some embodiments, a complexity of the attention weight is proportional to a size of the plurality of nucleic acid fragments. In some embodiments, the complexity of the attention weight is proportional to the size of the plurality of nucleic acid fragments and a size of the learned set of state patterns. In some embodiments, the size of the learned set of state patterns is smaller than a size of the plurality of nucleic acid fragments, so the overall complexity of the attention weight may comprise linear complexity. This may provide an advantage over other attention mechanisms, which may be associated with polynomial (e.g., quadratic) complexity. Using an attention mechanism with quadratic complexity, the complexity may scale with the square of the size of the plurality of nucleic acid fragments. As a result, complexity using the other attention mechanisms may increase much more quickly than the attention mechanisms disclosed herein, as the number of nucleic acid fragments increases. The systems and methods disclosed herein may therefore decrease a computational time or computational costs associated with processing large fragment datasets using attention-based sub-networks.

[0194]

[0147] In some embodiments, the attention-based sub-network comprises a linear attention model. In some embodiments, the linear attention model comprises a linear transformer, a Performer model (Wang et al., 2020; which is incorporated by reference herein in its entirety), a Linformer model (Choromanski et al., 2020; which is incorporated by reference herein in its entirety), or any combination thereof.

[0195]

[0148] In some embodiments, the attention weight is indicative of an importance of the nucleic acid fragment to a prediction task. As disclosed herein, the attention weight may be indicative of a similarity between the nucleic acid fragment or fragment representation and the set of learned state patterns. The set of learned state patterns may comprise health status archetypes and may capture patterns associated with a health status of interest (e.g., cancer). Therefore, the attention weight may enable the model to identify and more heavily weight nucleic acid fragments associated with the health status when aggregating a plurality of fragment representations into the sample representation.AttyDktNo.: 49407-793601

[0196] Model Interpretability

[0197]

[0149] In some embodiments, the method comprises performing dimensionality reduction on the sample representation of the biological sample. Dimensionality reduction techniques can be applied to one or more of the datasets disclosed herein to identify health status-associated clusters, batch effects, or select informative features, or any combination thereof. For example, high-dimensional fragment embeddings or pooled prototype sample embeddings may be projected into a smaller space using principal component analysis (PC A) to compress dominant variation, reduce noise, improve computational efficiency, or to detect batch effects.

[0198]

[0150] In some embodiments, the dimensionality reduction comprises Uniform Manifold Approximation and Projection (UMAP) or t-distributed Stochastic Neighbor Embedding (t-SNE). UMAP or t-SNE may be employed to produce a low dimensional visualization of the datasets.

[0199]

[0151] In some embodiments, the method comprises identifying one or more clusters. In some embodiments, the identification of the clusters comprises use of a clustering algorithm. In some embodiments, the clustering algorithm comprises k-means Clustering, hierarchical clustering, Gaussian Mixture Models (GMM), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), or any combination thereof. The embedding space may enable qualitative review, inspection of group structure including disease and health separation, identification of challenging low-ctDNA samples, or determination whether fragments learned by different prototypes form distinct clusters. Moreover, one or more of these methods can be employed to identify patterns in the various datasets or specific modalities in the datasets, including methylation status, fragment length, sequence variation, or other fragment attributes that correlate with disease status or health status. In some embodiments of the disclosure, PCA may be employed to reduce correlated variables into uncorrelated variables called “principal components,” the first principal component is selected such that it accounts for as much of the variability in the datasets or features in the dataset, alternatively first, second, and third principal components are selected when if together these account for a desirable percentage of the variability in the dataset. In such cases, PCA reduces dimensionality of the datasets and can identify new underlying disease associated features in the data.

[0200]

[0152] In some embodiments, the method further identifying, using the attention-based sub-network, one or more nucleic acid fragments associated with high attention weights. The attention weights may be determined based on processing fragment representations of the one or more nucleic acid fragments against the set of learned state patterns, as disclosed herein. In someAttyDktNo.: 49407-793601

[0201] embodiments, the attention weight is indicative of an importance of the one or more nucleic acid fragments to a prediction task.

[0202]

[0153] In some embodiments, the method comprises partitioning the one or more nucleic acid fragments associated with high attention weights into one or more genomic bins. In some embodiments, a number of nucleic acid fragments associated with high attention weights in each genomic bin of the one or more genomic bins may be determined. In some embodiments, the one or more genomic bins may be plotted. In some embodiments, the one or more genomic bins may be used to compare high-importance fragments identified by two or more different models or methods. In some embodiments, the genomic bins may be used to identify one or more genomic regions of interest to the prediction task (e.g., a differentially methylated region (DMR)).

[0203] Health Status Prediction

[0204]

[0154] In some embodiments, the methods described herein, further include using a sample representation of a biological sample to predict a health or disease status in a test subject. The sample representation may be generated by processing fragment level data from a biological sample with one or more machine learning operations that encode fragment level attributes into numerical embeddings and aggregate the fragment embeddings into a sample-level representation. In some embodiments, sample-level representation is processed using a predictive sub-network trained to predict disease status of the test subject, based at least in part on analysis of the sample representation. The predictive sub-network may comprise any suitable supervised, simi-supervised, or weakly supervised learning model and may output a classification, score, probability, or other quantitative or qualitative indicator of health or disease status in the test subject.

[0205]

[0155] The health status may comprise a presence or absence of a disease, a clinical metadata, a disease stage, a disease subtype, a lesion size, a biological state or characteristic of the subject. For example, in some embodiments the disease may comprise inflammatory disorders, metabolic disorders, nervous system disorders, neurological disorders, cancer, or any combination thereof. The cancer may include colorectal cancer, lung cancer, pancreatic cancer, gastric cancer, liver cancer, ovarian cancer, breast cancer, head or neck cancer, or any combination thereof. In some embodiments, the disease stage may comprise a stage of the cancer. For example, the disease may comprise colorectal cancer, and the disease stage may comprise stage I, stage II, stage III, stage IV, or any combination thereof. In some embodiments, the disease subtype comprises a subtype of the cancer. For example, the disease may comprise an advanced precancerous lesion (APL), and the disease subtype may comprise a subtype of the APL.AttyDktNo.: 49407-793601

[0206]

[0156] Additionally, in alternative embodiments the disease can comprise a neurological disorder or neurodegenerative diseases including amyotrophic lateral sclerosis, multiple sclerosis, Parkinson’s disease, Alzheimer’s disease, Huntington’s disease, and multiple system atrophy. Moreover, the methods described herein may be applied for the early detection and diagnosis of various inflammatory diseases including rheumatoid arthritis, multiple sclerosis, asthma, psoriasis, Crohn’s disease, and ulcerative colitis. The predictive sub-network may be trained using labeled samples from a plurality of subjects and may leverage patterns in the sample-level representation that correlate with disease status or other biological and / or biochemical states. Health or disease screening using the methods disclosed herein may be used for screening, diagnosis, prognosis, monitoring or stratification of subjects in clinical or research settings.

[0207] Predictive Sub-Network

[0208]

[0157] In some embodiments, the health status prediction is performed by a predictive sub-network that operates on a sample level representation of a biological sample. The predictive sub-network may comprise a single classification model or an ensemble model.

[0209]

[0158] In some embodiments, the ensemble model includes a plurality of classification models generated from different cross-validation folds of a training dataset. Each classification model in the ensemble may independently process the sample-level representation and produce a prediction, the prediction may then by combined using averaging, voting, weighting, or other ensemble aggregation techniques. Note that the prediction or output from each classification model in the ensemble model may be a score, a binary value, or a categorical value.

[0210]

[0159] In some embodiments, the ensemble model may comprise a plurality of classification models obtained from a plurality of cross validation folds. In some embodiments, the plurality of classification models may comprise a multilayer perceptron (MLP), a linear classifier, gradient-boosted decision trees including extreme grading hosting or XGBoost, a random forest model, a logistic regression model, a support vector machine (SVM), or any combination thereof.

[0211]

[0160] In some embodiments, training the predictive sub-network may include initializing a set of model weights. In some embodiments, the plurality of parameters is initialized randomly. In some embodiments, the plurality of parameters is initialized based on pre-training.

[0212]

[0161] In some embodiments, training the predictive sub-network comprises determining a value of a performance metric associated with the set of model weights. The performance metric may be determined by using the predictive sub-network to determine a predicted healthAttyDktNo.: 49407-793601

[0213] status of a test subject, based on analyzing the sample representation. In some embodiments, determining the value of the performance metric comprises comparing the predicted label of the training biological sample with a ground truth label. The ground truth label may comprise a ground truth health status. The ground truth health status may be obtained from a medical record of the training subject or determined by a healthcare professional.

[0214]

[0162] In some embodiments, the performance metric comprises sensitivity at a predetermined specificity value. In some embodiments, the pre-determined specificity value may range from about 55% to 100%. In some embodiments, the pre-determined specificity value is at least or at most about 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90% , 95%, or 100%.

[0215]

[0163] In some embodiments, the performance metric comprises an accuracy, a specificity, a sensitivity, a precision, an Fl score, an area under the Receiver Operating Characteristic curve (AUROC), a mean squared error (MSE), a mean absolute error (MAE), an R-squared (R2), an adjusted R-squared, a root mean squared error (RMSE), a categorical crossentropy loss, a precision, a recall, an area under the Precision-Recall curve (AUC-PR), or any combination thereof.

[0216]

[0164] In some embodiments, training the predictive sub-network comprises updating the model weights, based at least in part on the performance metric. In some embodiments, training the predictive sub-network comprises updating one or more model hyperparameters. In some embodiments, the value of the performance metric is compared to a threshold value. In some embodiments, the model weights or hyperparameters are updated if the value of the performance metric does not meet the threshold. In some embodiments, training the predictive sub-network comprises determining a gradient associated with the performance metric. In some embodiments, training the attention-based sub-network comprises determining an updated value of the model weights or hyperparameters configured to improve the performance metric, based at least in part on the gradient. In some embodiments, the model weights or hyperparameters is updated using an optimizer. In some embodiments, the optimizer comprises Stochastic Gradient Descent (SGD), Mini-batch Gradient Descent, Adam Adaptive Moment Estimation (Adam), Root Mean Square Propagation (RMSProp), Adaptive Gradient Algorithm (AdaGrad), or any combination thereof. In some embodiments, training the predictive sub-network comprises iteratively updating the model weights or hyperparameters. In some embodiments, the model weights or hyperparameters is iteratively updated for a pre-determined number of iterations. In some embodiments, the model weights or hyperparameters is iteratively updated until the value of the performance metric meets the threshold value.

[0217]

[0165] In some embodiments, the predictive sub-network is trained using minibatches. In some embodiments, training the predictive sub-network comprises utilizing a dynamic minibatchAttyDktNo.: 49407-793601

[0218] balancing scheme to reduce the effect of confounding variables on health status prediction. In some embodiments, minibatches may be designed such that correlation of health status labels with demographic variables (e.g., age, sex, cohort, etc.) is removed. Removing the correlations may comprise dynamic oversampling. The method may comprise selecting samples from a cohort to provide a minibatch. The method may comprise, for one or more samples in the minibatch, selecting a paired sample having a same demographic variable and a different health status label.

[0219]

[0166] In some embodiments, the predictive sub-network, is trained end-to-end, together with an encoding sub-network, including an attention-based sub-network, such that gradients repropagated thought both models during optimization. Alternatively, the encoding sub-network, the attention-based sub-network, or the predictive sub-network may be trained separately.

[0220]

[0167] In various embodiments, the predictive sub-network may comprise any suitable classification or regression models capable of operating on the sample representation. While certain embodiments describe the use of multilayer perceptrons or ensemble models, the disclosure is not limited to these examples. Additional models may be employed without departing from the scope of the disclosure, including linear models, generalized linear models, decision trees, kernel-based models, or any combination thereof. In various examples, the predictive sub-network includes a logistic regression, multiple linear regression, support vector machines (SVMs), decision tree, classification and regression trees (CART), tree-based methods, random forest, gradient boosted-trees, logistic regression, artificial neural networks, or any combination thereof.

[0221]

[0168] Additionally, the predictive sub-network may include one or more preprocessing steps that perform feature transformation and and / or representation learning on the sample representation prior to classification or regression. For example, employing one or more of, matrix factorization, multidimensional scaling (MDS), dimensionality reduction methods, t-distributed stochastic neighbor embedding (t-SNE), network clustering, dimension reduction, partial least squares (PLS) regression, principal component regression, autoencoders, variational autoencoders, singular value decomposition, Fourier bases, wavelets, discriminant analysis or any combination thereof, followed by a prediction model. Note that the ensemble model can comprise a homogenous such as a random forest or a heterogeneous ensemble model comprising different models. The models in the heterogenous ensemble model can include for example one or more linear regression models, logistic regression models, neural networks or any combination thereof. Each of these models generate a respective score or class label based at least on the sample representation. In certain implementations the sample representation along with sample meta data, clinical data available for the subject including imaging data, body massAttyDktNo.: 49407-793601

[0222] index, or lifestyle information (e.g., smoker, drinker, medication or supplement use, etc.) can be integrated to increase the sensitivity of the various machine learning models and sub-networks described herein.

[0223] Gene Panels

[0224]

[0169] In some embodiments, the method comprises determining a panel of genomic regions of interest. In some embodiments, determining the panel of genomic regions of interest comprises determining one or more genomic regions of a target capture panel associated with the health status.

[0225]

[0170] Enrichment panels configured to predict various outcomes in the subject can be selected to reduce sequencing cost, noise, and to enable deeper sequencing depth per locus. Regions of interest include specific gene pathways associated with disease. Non-limiting examples of gene panels are provided in WO2021202351A1 and WO2022204358A1, each of which is incorporated by reference herein in its entirety.

[0226]

[0171] In some embodiments, determining the panel of genomic regions of interest comprises filtering the one or more genomic regions to reduce a background noise. The filtering may enrich for highly predictive genomic regions, genes, or combinations thereof. Clustering, dimensionality reduction methods, or combinations thereof may be applied to identify informative genes, genomic regions, or combinations thereof. In some embodiments, the filtering comprises profiling methylation-associated signal in the one or more genomic regions. The profiling may comprise determining a level of background hypermethylation in the one or more genomic regions. In some embodiments, the profiling is based on a hold-out cohort of test subjects. In some embodiments, the filtering comprises filtering regions of the one or more genomic regions having elevated background hypermethylation, relative to a threshold methylation level. The threshold methylation level may be determined based on one or more reference subjects, one or more reference genomic regions, or obtained from a database or memory.

[0227]

[0172] In some embodiments, the encoding sub-network or the attention-based subnetwork may be trained based at least on one or more disease specific gene panels, genomic region panels, or combinations thereof. In certain examples, a plurality of training samples comprising targeted, exome, or whole genome sequencing genetic and / or epigenetic data are used to train the various machine learning models and sub-networks.

[0228]

[0173] In some embodiments, the encoding sub-network or the attention-based subnetwork are trained using a subset of the plurality of nucleic acid fragments corresponding to the panel of genomic regions of interest. This may decrease a size of the input data to the machineAtty Dkt No.: 49407-793601

[0229] learning model, while providing that important nucleic fragments to the health status are provided as input.

[0230] Examples of Machine Learning Methodologies

[0231]

[0174] As used herein, the terms “artificial intelligence,” “artificial intelligence techniques,” “artificial intelligence operation,” and “artificial intelligence algorithm” generally refer to any system or computational procedure that may take one or more actions that simulate human intelligence processes for enhancing or maximizing a chance of achieving a goal. The term “artificial intelligence” may include “generative modeling,” “machine learning” (ML), or “reinforcement learning” (RL). As used in this specification and the appended claims, the terms “machine learning,” “machine learning techniques,” “machine learning operation,” and “machine learning model” generally refer to any system or analytical or statistical procedure that may progressively improve computer performance of a task.

[0232]

[0175] In some cases, ML may generally involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data. ML may include a ML model (which may include, for example, a ML algorithm). Machine learning, whether analytical or statistical in nature, may provide deductive or abductive inference based on real or simulated data. The ML model may be a trained model. ML techniques may comprise one or more supervised, semi-supervised, self-supervised, or unsupervised ML techniques. For example, an ML model may be a trained model that is trained through supervised learning (e.g., various parameters are determined as weights or scaling factors). ML may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. ML may comprise: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principal component regression, least absolute shrinkage and selection operation (LASSO), least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked autoAttyDktNo.: 49407-793601

[0233] encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, residual neural networks, physics-informed neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, large language models, transformer models, vision transformers, or generative adversarial networks.

[0234]

[0176] Training the ML model may include, in some cases, selecting one or more untrained data models to train using a training data set. The selected untrained data models may include any type of untrained ML models for supervised, semi-supervised, self-supervised, or unsupervised machine learning. The selected untrained data models may be specified based upon input (e.g., user input) specifying relevant parameters to use as predicted variables or other variables to use as potential explanatory variables. For example, the selected untrained data models may be specified to generate an output (e.g., a prediction) based upon the input.

[0235] Conditions for training the ML model from the selected untrained data models may likewise be selected, such as limits on the ML model complexity or limits on the ML model refinement past a certain point. The ML model may be trained (e.g., via a computer system such as a server) using the training data set. In some cases, a first subset of the training data set may be selected to train the ML model. The selected untrained data models may then be trained on the first subset of training data set using appropriate ML techniques, based upon the type of ML model selected and any conditions specified for training the ML model. In some cases, due to the processing power requirements of training the ML model, the selected untrained data models may be trained using additional computing resources (e.g., cloud computing resources). Such training may continue, in some cases, until at least one aspect of the ML model is validated and meets selection criteria to be used as a predictive model.

[0236]

[0177] In some cases, one or more aspects of the ML model may be validated using a second subset of the training data set (e.g., distinct from the first subset of the training data set) to determine accuracy and robustness of the ML model. Such validation may include applying the ML model to the second subset of the training data set to make predictions derived from the second subset of the training data. The ML model may then be evaluated to determine whether performance is sufficient based upon the derived predictions. The sufficiency criteria applied to the ML model may vary depending upon the size of the training data set available for training, the performance of previous iterations of trained models, or user-specified performance requirements. If the ML model does not achieve sufficient performance, additional training may be performed. Additional training may include refinement of the ML model or retraining on a different first subset of the training dataset, after which the new ML model may again be validated and assessed. When the ML model has achieved sufficient performance, in some cases,AttyDktNo.: 49407-793601

[0237] the ML may be stored for present or future use. The ML model may be stored as sets of parameter values or weights for analysis of further input (e.g., further relevant parameters to use as further predicted variables, further explanatory variables, further user interaction data, etc.), which may also include analysis logic or indications of model validity in some instances. In some cases, a plurality of ML models may be stored for generating predictions under different sets of input data conditions. In some cases, the ML model may be stored in a database (e.g., associated with a server).

[0238] Neural Network

[0239]

[0178] In some embodiments, systems and methods of the present disclosure may comprise or comprise using a neural network. The neural network may comprise various architectures, loss functions, optimization algorithms, assumptions, and various other neural network design choices. In some embodiments, the neural network comprises an encoder. In some embodiments, the neural network comprises a decoder. In some embodiments, the neural network comprises a bottleneck architecture comprising the encoder and the decoder. In some embodiments, the bottleneck architecture comprises an autoencoder. In some embodiments, the neural network comprises a language model. In some embodiments, the neural network comprises a transformer model.

[0240]

[0179] Various types of layers may be used a neural network. In some embodiments, the neural network comprises a convolutional layer. In some embodiments, the neural network comprises a densely connected layer. In some embodiments, the neural network comprises a skip connection. In some embodiments, the neural network may comprise graph convolutional layers. In some embodiments, the neural network may comprise message passing layers. In some embodiments, the neural network may comprise attention layers. In some embodiments, the neural network may comprise recurrent layers. In some embodiments, the neural network may comprise a gated recurrent unit. In some embodiments, the neural network may comprise reversible layers. In some embodiments, the neural network may comprise a neural network with a bottleneck layer. In some embodiments, the neural network may comprise residual blocks. In some embodiments, the neural network may comprise one or more dropout layers. In some embodiments, the neural network may comprise one or more locally connected layers. In some embodiments, the neural network may comprise one or more batch normalization layers. In some embodiments, the neural network may comprise one or more pooling layers. In some embodiments, the neural network may comprise one or more upsampling layers. In some embodiments, the neural network may comprise one or more max -pooling layers.AttyDktNo.: 49407-793601

[0241]

[0180] In some embodiments, the neural network comprises a graph model. In some embodiments, a graph, graph model, and graphical model can refer to a method that models data in a graphical representation comprising nodes and edges. In some embodiments, the data may be stored in a various and alternative forms such as linked lists, dictionaries, spreadsheets, arrays, in permanent storage, in transient storage, and so on, and is not limited to specific embodiments disclosed herein.

[0242]

[0181] In some embodiments, the neural network may comprise an autoencoder. In some embodiments, the neural network may comprise a variational autoencoder. In some embodiments, the neural network may comprise a generative adversarial network. In some embodiments, the neural network may comprise a flow model. In some embodiments, the neural network may comprise an autoregressive model.

[0243]

[0182] The neural network may comprise various activation functions. In some embodiments, an activation function may be a non-linearity. In some embodiments, the neural network may comprise one or more activation functions. In some embodiments, the neural network may comprise a ReLU, softmax, tanh, sigmoid, softplus, softsign, selu, elu, exponential, LeakyReLU, or any combination thereof. Various activation functions may be used with a neural network.

[0244] Training

[0245]

[0183] Various loss functions can be used to train the neural network. In some embodiments, the neural network may comprise a regression loss function. In some embodiments, the neural network may comprise a logistic loss function. In some embodiments, the neural network may comprise a variational loss. In some embodiments, the neural network may comprise a prior. In some embodiments, the neural network may comprise a Gaussian prior. In some embodiments, the neural network may comprise a non-Gaussian prior. In some embodiments, the neural network may comprise a Laplacian prior. In some embodiments, the neural network may comprise a zero-inflated prior. In some case, the neural network may comprise a zero-inflated Poisson prior. In some embodiments, the neural network may comprise a zero-inflated negative binomial prior. In some embodiments, the neural network may comprise a Gaussian posterior. In some embodiments, the neural network may comprise a non-Gaussian posterior. In some embodiments, the neural network may comprise a Laplacian posterior. In some embodiments, the neural network may comprise an adversarial loss. In some embodiments, the neural network may comprise a reconstruction loss. In some embodiments, the loss functions may be formulated to optimize a regression loss, an evidence-based lower bound, a maximum likelihood, Kullback-Leibler divergence, applied with various distribution functions such as Gaussians, non-Gaussian, mixtures of Gaussians, mixtures of logistic functions, and so on.AttyDktNo.: 49407-793601

[0246]

[0184] Various optimizers can be used to train the neural network. In some embodiments, the neural network may be trained with the Adam optimizer. In some embodiments, the neural network may be trained with the stochastic gradient descent optimizer. In some embodiments, the neural network may be trained with an active learning algorithm. A neural network may be trained with various loss functions whose derivatives may be computed to update one or more parameters of the neural network. A neural network may be trained with hyperparameter searching algorithms. In some embodiments, the neural network hyperparameters are optimized with Gaussian Processes.

[0247]

[0185] Various training protocols can be used while training the neural network. In some embodiments, the neural network may be trained with train / validation / test data splits. In some embodiments, the neural network may be trained with k-fold data splits, with any positive integer for k.

[0248]

[0186] Training the neural network can involve providing inputs to the untrained neural network to generate predicted outputs, comparing the predicted outputs to the expected outputs, and updating the neural network’s parameters to account for the difference between the predicted outputs and the expected outputs. Based on the calculated difference, a gradient with respect to each parameter may be calculated by backpropagation to update the parameters of the neural network so that the output value(s) that the neural network computes are consistent with the examples included in the training set. This process may be iterated for a certain number of iterations or until some stopping criterion is met.

[0249]

[0187] The trained algorithm may be trained with a plurality of independent training samples. Each of the independent training samples may comprise a sensor data. In the case of a variational autoencoder (VAE), the training samples may comprise individual observed sensor data and corresponding reconstructed sensor data. The trained algorithm may be trained, at least in part, to optimize the accuracy of the reconstruction when compared to the original input data.

[0250]

[0188] After training the VAE, the encoder may be used to generate encodings (e.g., latent representations or latent descriptors) of sensor data. Compared to the original or reconstructed descriptors, the latent descriptors may comprise certain properties. In some embodiments, the latent descriptors may comprise a reduced noise compared to the original descriptor. Without wishing to be bound by a particular theory, because the latent representation generally comprises fewer dimensions than the input feature, the autoencoder may “learn” during training to only capture in the latent representation those patterns in the input data which are significant (e.g., important for accurate reconstruction) while ignoring those that are less important. The latent space may additionally learn a continuous representation of the input data.AttyDktNo.: 49407-793601

[0251] For example, sensor data which are similar to one another may be close to one another in the latent space while those which are dissimilar to one another may be far apart in the latent space.

[0252] Convolutional Neural Networks

[0253]

[0189] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement a convolutional neural network (CNN). Neural networks are a subset of machine learning and are often at the core of many deep learning algorithms. Neural networks may comprise node layers, each which may comprise one or more of an input layer, one or more hidden layers, and an output layer. Each node of a neural network may connect to another node of the neural network. Each node of a neural network may have an associated weight and threshold. In some cases, if an output from any individual node of a neural network is above a specified threshold value, that node is activated, thereby sending data to the next layer of the neural network; otherwise, no data is passed along to the next layer of the neural network.

[0254]

[0190] CNNs are a type of neural network. CNNs are often used for classification and computer vision tasks. Prior to CNNs, manual, time-consuming feature extraction methods were used to identify objects in images. However, CNNs provide a more scalable approach to image classification and object recognition tasks, leveraging principles from linear algebra, specifically matrix multiplication, to identify patterns within an image. That said, CNNs can be computationally demanding, using graphical processing units (GPUs) to train models.

[0255]

[0191] CNNs may be distinguished from other neural networks by their superior performance with image, speech, or audio signal inputs. CNNs may comprise three main types of layers: convolutional layers, pooling layers, and fully-connected (FC) layers. The convolutional layer may be the first layer of a CNN. While convolutional layers can be followed by additional convolutional layers or pooling layers, the fully-connected layer may be the final layer of the CNN.

[0256]

[0192] When applied to computer vision tasks within images, with each layer, the CNN increases in its complexity, identifying greater portions of the image. Earlier layers of a CNN may focus on simple features of an image, such as colors and edges. As the image data progresses through the layers of the CNN, the CNN starts to recognize larger elements or shapes of objects in the image until the CNN identifies the intended object.

[0257]

[0193] The convolutional layer is a core building block of a CNN and may be where much of the computation of the CNN occurs. Convolution layers may use components including input data, a filter, and a feature map. Provided, for example, the input data comprises a color image (which e.g., includes a matrix of pixels in 3D), the input may have three dimensions — a height, width, and depth — which correspond to RGB in an image. CNNs may further comprise aAttyDktNo.: 49407-793601

[0258] feature detector (also known as a kernel or a filter), which moves across receptive fields of the image, checking if a feature is present. This process may be known as a convolution.

[0259]

[0194] The feature detector may include a filter that is a two-dimensional array of weights, which represents part of an image. Filters of feature detectors may vary in size (e.g., 3x3 matrix), and the size may determine the size of the receptive field. The filter may be applied to an area of the image, and a dot product may be calculated between input pixels and the filter. The dot product may then be fed into an output array. Afterwards, the filter may shift by a stride, repeating the process until the filter has swept across the entire image. The final output from the series of dot products from the input and the filter may be known as a feature map, activation map, or a convolved feature. After each convolution operation, a CNN may apply a Rectified Linear Unit (ReLU) transformation to the feature map, introducing nonlinearity to the CNN.

[0260]

[0195] In some cases, another convolution layer can follow the initial convolution layer of the CNN. For example, the structure of the CNN can become hierarchical as the later layers can see the pixels within the receptive fields of prior layers. As an example, assume a CNN used to determine if an image comprises a bicycle. Each individual part of the bicycle (e.g., frame, handlebars, wheels, pedals, etc.) makes up a lower-level pattern in the CNN, and the combination of the parts represents a higher-level pattern, creating a feature hierarchy within the CNN.

[0261]

[0196] The pooling layers, also known as downsampling, are further layers of a CNN. Pooling layers may conduct dimensionality reduction, reducing the number of parameters in the input (e.g., image, video, audio, etc.). Similar to the convolutional layer, the pooling layer sweeps a filter across the entire input, but, unlike the convolution layers, the filters of the pooling layers do not have any weights. Instead, the filters of the pooling layers apply an aggregation function to values within the receptive field, populating the output array. There are two main types of pooling: max pooling and average pooling. Max pooling may comprise moving the filter across the input to select the pixel with the maximum value to send to the output array. Average pooling may comprise moving the filter across the input to calculate the average value within the receptive field to send to the output array. While a lot of information is lost in the pooling layer, the pooling layer also has a number of benefits to the CNN. For example, pooling layers may help to reduce complexity of the CNN, improve efficiency, and limit risk of overfitting of the CNN.

[0262]

[0197] Fully-connected layers are the final layer of a CNN. As previously disclosed, pixel values of an input image are not directly connected to output layers in partially connected layers. However, in the fully-connected layer, each node in the output layer connects directly to a node in the previous layer. The FC layer performs the task of classification based on the featuresAttyDktNo.: 49407-793601

[0263] extracted through the previous layers and their different filters. While convolutional layers and pooling layers tend to use ReLu functions, FC layers may leverage a softmax activation function to classify inputs appropriately, producing a probability from 0 to 1.

[0264] Examples of Long Short-Term Memory

[0265]

[0198] Long short-term memory (LSTM) may be an artificial neural network used in the fields of artificial intelligence and deep learning. Unlike standard feedforward neural networks, LSTM may use feedback connections. The LSTM architecture may provide a short-term memory for a recurrent neural network (RNN). Such RNN can process not only single data points (such as images), but also entire sequences of data (such as audio or video). This characteristic may enable LSTM networks to be well-suited for processing and predicting data. The name of LSTM may refer to the analogy that a standard RNN has both “long-term memory” and “short-term memory.” The connection weights and biases in the RNN may change once per episode of training, analogous to how physiological changes in synaptic strengths store longterm memories; the activation patterns in the network may change once per time-step, analogous to how the moment-to-moment change in electric firing patterns in the brain store short-term memories. The LSTM architecture may provide a short-term memory for an RNN that can last many (e.g., hundreds, thousands, tens of thousands, etc.) timesteps.

[0266]

[0199] In some cases, a LSTM unit may comprise a cell, an input gate, an output gate, and a forget gate. The cell may remember values over arbitrary time intervals and the input gate, the output gate, and the forget gate may regulate the flow of information into and out of the cell. Forget gates may be used to decide what information to discard from a previous state by assigning a previous state, compared to a current input, a value between 0 and. For example, a (e.g., rounded) value of 1 may mean to keep the information, and a (e.g., rounded) value of 0 means to discard it). The input gate may decide which pieces of new information to store in the current state, using the same system as the forget gates. The output gate may control which pieces of information in the current state to output (e.g., by assigning a value from 0 to 1 to the information, considering the previous and current states). Selectively outputting relevant information from the current state may allow the LSTM network to maintain useful, long-term dependencies to make predictions, both in current and future time-steps. In some cases, LSTM networks may be well-suited to classifying, processing and making predictions based on time series data, since there can be lags of unknown duration between important events in a time series. LSTMs may resolve the vanishing gradient problem that can be encountered when training certain RNNs. Relative insensitivity to gap length may be an advantage of LSTM over RNNs, hidden Markov models, and other sequence learning methods in numerous applications.AttyDktNo.: 49407-793601

[0267]

[0200] In some cases, LSTMs may be used with one or more various types of neural networks (e.g., convolutional neural networks (CNNs), deep neural network (DNNs), RNNs, etc.). In some cases, CNNs, DNNs, and LTSMs are complementary in their modeling capabilities and may be combined a unified architecture. For example, in such unified architecture, CNNs may be well-suited at reducing frequency variations, LSTMs may be well-suited at temporal modeling, and DNNs may be well-suited for mapping features to a more separable space. For example, input features to a ML model using LSTM techniques in the unified architecture may include segment features for each of a plurality of segments. To process the input features for each of the plurality of segments, the segment features for the segment may be processed using one or more CNN layers to generate first features for the segment; the first features may be processed using one or more LSTM layers to generate second features for the segment; and the second features may be processed using one or more fully connected neural network layers to generate third features for the segments, where the third features may be used for classification operations. In some cases, to process the first features using the one or more LSTM layers to generate the second features, the first features may be processed using a linear layer to generate reduced features having a reduced dimension from a dimension of the first features; and the reduced features may be processed using the one or more LSTM layers to generate the second features. Short-term features having a first number of contextual frames may be generated based on the input features, where features generated using the one or more CNN layers may include long-term features having a second number of contextual frames that are more than the first number of contextual frames of the short-term features. In some cases, the one or more CNN layers, the one or more LSTM layers, and the one or more fully connected neural network layers may have been jointly trained to determine trained values of parameters of the one or more CNN layers, the one or more LSTM layers, and the one or more fully connected neural network layers. In some cases, the input features may include log-mel features having multiple dimensions. The input features may include one or more contextual frames indicating a temporal context of a signal (e.g., input data). Advantageously, implementations for such unified architecture may leverage complementary advantages associated with each of a CNN, a DNN, and a LTSM. For example, convolutional layers may reduce spectral variation in input, which may help the modeling of LSTM layers. Having DNN layers after LSTM layers may help reduce variation in the hidden states of the LSTM layers. Training the unified architecture jointly may provide a better overall performance. Training in the unified architecture may also remove the need to have separate CNN, LSTM and DNN architectures, which may be expensive (e.g., in computational resource, in network traffic, in financial resources, in energy consumption, etc.).AttyDktNo.: 49407-793601

[0268] By adding multi-scale information into the unified architecture, information may be captured at different time scales.

[0269] Examples of Support Vector Machines

[0270]

[0201] As also described above, the machine learning model may implement support vector machine learning techniques. In machine learning, support vector machines (SVMs) may be supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. SVMs may be a robust prediction method, being based on statistical learning. SVMs may be well-suited for domains characterized by the existence of large amounts of data, noisy patterns, or the absence of general theories.

[0271]

[0202] In general terms, SVMs may map input vectors into high dimensional feature space through non-linear mapping function, chosen a priori. In this high dimensional feature space, an optimal separating hyperplane may be constructed. The optimal hyperplane may then be used to determine, for example, class separations, regression fit, accuracy in density estimation, etc. More formally, a SVM may construct a hyperplane or set of hyperplanes in a high or infinite-dimensional space, which can be used for classification, regression, or other tasks like outlier detection.

[0272]

[0203] Support vectors may be defined as the data points that lie closest to the decision surface (or hyperplane). Support vectors may therefore be the data points that are most difficult to classify and may have direct bearing on an optimum location of the decision surface. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm may build a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier (although methods such as Platt scaling exist to use SVM in a probabilistic classification setting). In some cases, SVMs may map training examples to points in space so as increase (e.g., maximize) the width of the gap between the two categories. New examples may then be mapped into that same space and predicted to belong to a category based on which side of the gap the new examples fall. In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what may be referred to as a kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.

[0273]

[0204] Within a support vector machine, the dimensionally of the feature space may be large. For example, a fourth-degree polynomial mapping function may cause a 200-dimensional input space to be mapped into a 1.6 billionth dimensional feature space. The kernel trick and the Vapnik-Chervonenkis dimension may allow the SVM to thwart the “curse of dimensionality” limiting other methods and effectively derive generalizable answers from this very highAttyDktNo.: 49407-793601

[0274] dimensional feature space. Accordingly, SVMs may assist in discovering knowledge from vast amounts of input data.

[0275] Examples of Decision Trees and Random Forest Classifiers

[0276]

[0205] As described above, the machine learning model may implement a decision tree. A decision tree may be a supervised ML algorithm that can be applied to both regression and classification problems. A decision tree may grow from a root (base condition), and when it meets a condition (internal node / feature), it may split into multiple branches. The end of the branch that does not split anymore may be an outcome (leaf). A decision tree can be generated using a training data set according to the following operations: (1) starting from a root node (the entire dataset), the algorithm may split the dataset in two branches using a decision rule or branching criterion; (2) each of these two branches may generate a new child node; (3) for each new child node, the branching process may be repeated until the dataset cannot be split any further; (4) each branching criterion may be chosen to maximize information gain (e.g., a quantification of how much a branching criterion reduces a quantification of how mixed the labels are in the children nodes). The labels may be the data or the classification that is predicted by the decision tree.

[0277]

[0206] A random forest regression is an extension of the decision tree model that tends to yield more robust predictions by stretching the use of the training data partition. Whereas a decision tree may make a single pass through the data, a random forest regression may bootstrap 50% of the data (e.g., with replacement) and build many trees. Rather than using all explanatory variables as candidates for splitting, a random subset of candidate variables may be used for splitting, which may enable trees that have different data and different variables (hence the term random). The predictions from the trees, which may be collectively referred to as the “forest,” may then be averaged to produce a final prediction. Many trees (e.g., ten trees, fifty trees, one hundred trees, one thousand trees, etc.) may be included in a random forest model, with a number (e.g., 3, 6, 10, etc.) of terms sampled per split, a minimum of number (e.g., 1, 2, 4, 10, etc.) of splits per tree, and a minimum split size (e.g., 16, 32, 64, 128, 256, etc.). Random forests may be trained in a similar way as decision trees. Specifically, training a random forest may include the following operations: (1) randomly select k features from the total number of features; (2) create a decision tree from these k features using the same operations as for generating a decision tree; and (3) repeat the previous two operations until a target number of trees is created.

[0278]

[0207] As disclosed, a random forest classifier, which may comprise a plurality of decision trees wherein the output prediction may be the mode of the predicted classifications ofAttyDktNo.: 49407-793601

[0279] the individual trees, can be helpful in reducing overfitting to training data. In some cases, an ensemble of decision trees can be constructed using a random subset of features at each split or decision node. The Gini criterion may be employed, in some cases, to choose the best partition, wherein decision nodes having the lowest calculated Gini impurity index are selected. The Gini impurity can be used, in some cases, as a criterion to find informative features based on which the splits in each decision tree may be constructed.

[0280]

[0208] In some cases, each decision tree of a random forest may comprise one or more decision nodes, wherein each decision node specifies a predicate condition. For example, decision node may predicate the condition that, for a given dataset, the outcome to an question is a specific outcome. At each decision node, a decision tree can be split based on whether the predicate condition attached to the decision node holds true, leading to various prediction nodes. Each prediction node can comprise output values that represent “votes” for one or more of the classifications or conditions being evaluated by the assessment model. At prediction time, a “vote” can be taken over all of the decision trees, and the majority vote (or mode of the predicted classifications) can be output as the predicted classification.

[0281]

[0209] In some cases, when the dataset being queried in the assessment model reaches a “leaf’, or a final prediction node with no further downstream splits, the output values of the leaf can be output as the votes for the particular decision tree. Since a random forest model comprises a plurality of decision trees, the final votes across all trees in the forest can be summed to yield the final votes and the corresponding classification of the subject. A large number of decision trees can help reduce overfitting of the assessment model to the training data, by reducing the variance of each individual decision tree. For example, an assessment model can comprise, for example, at least about 3 decision trees, at least about 5 decision trees, at least about 10 decision trees, at least about 20 decision trees, at least about 50 decision trees, at least about 100 decision trees, etc.

[0282] Computer systems

[0283]

[0210] Referring to FIG. 10, a block diagram is shown depicting an example machine that includes a computer system 1000 (e.g., a processing or computing system) within which a set of instructions can execute for causing a device to perform or execute any one or more of the aspects and / or methodologies for static code scheduling of the present disclosure. The components in FIG. 10 are examples only and do not limit the scope of use or functionality of any hardware, software, embedded logic component, or a combination of two or more such components implementing particular embodiments.AttyDktNo.: 49407-793601

[0284]

[0211] Computer system 1000 may include one or more processors 1001, a memory 1003, and a storage 1008 that communicate with each other, and with other components, via a bus 1040. The bus 1040 may also link a display 1032, one or more input devices 1033 (which may, for example, include a keypad, a keyboard, a mouse, a stylus, etc.), one or more output devices 1034, one or more storage devices 1035, and various tangible storage media 1036. All of these elements may interface directly or via one or more interfaces or adaptors to the bus 1040. For instance, the various tangible storage media 1036 can interface with the bus 1040 via storage medium interface 1026. Computer system 1000 may have any suitable physical form, including but not limited to one or more integrated circuits (ICs), printed circuit boards (PCBs), mobile handheld devices (such as mobile telephones or PDAs), laptop or notebook computers, distributed computer systems, computing grids, or servers.

[0285]

[0212] Computer system 1000 includes one or more processor(s) 1001 (e.g., central processing units (CPUs) or general purpose graphics processing units (GPGPUs)) that carry out functions. Processor(s) 1001 optionally comprises a cache memory unit 1002 for temporary local storage of instructions, data, or computer addresses. Processor(s) 1001 are configured to assist in execution of computer readable instructions. Computer system 1000 may provide functionality for the components depicted in FIG. 10 as a result of the processor(s) 1001 executing non-transitory, processor-executable instructions embodied in one or more tangible computer-readable storage media, such as memory 1003, storage 1008, storage devices 1035, and / or storage medium 1036. The computer-readable media may store software that implements particular embodiments, and processor(s) 1001 may execute the software. Memory 1003 may read the software from one or more other computer-readable media (such as mass storage device(s) 1035, 1036) or from one or more other sources through a suitable interface, such as network interface 1020. The software may cause processor(s) 1001 to carry out one or more processes or one or more steps of one or more processes described or illustrated herein. Carrying out such processes or steps may include defining data structures stored in memory 1003 and modifying the data structures as directed by the software.

[0286]

[0213] The memory 1003 may include various components (e.g., machine readable media) including, but not limited to, a random access memory component (e.g., RAM 1004) (e.g., static RAM (SRAM), dynamic RAM (DRAM), ferroelectric random access memory (FRAM), phase-change random access memory (PRAM), etc.), a read-only memory component (e.g., ROM 1005), and any combinations thereof. ROM 1005 may act to communicate data and instructions unidirectionally to processor(s) 1001, and RAM 1004 may act to communicate data and instructions bidirectionally with processor(s) 1001. ROM 1005 and RAM 1004 may include any suitable tangible computer-readable media described below. In one example, a basicAttyDktNo.: 49407-793601

[0287] input / output system 1006 (BIOS), including basic routines that help to transfer information between elements within computer system 1000, such as during start-up, may be stored in the memory 1003.

[0288]

[0214] Fixed storage 1008 is connected bidirectionally to processor(s) 1001, optionally through storage control unit 1007. Fixed storage 1008 provides additional data storage capacity and may also include any suitable tangible computer-readable media described herein. Storage 1008 may be used to store operating system 1009, executable(s) 1010, data 1011, applications 1012 (application programs), and the like. Storage 1008 can also include an optical disk drive, a solid-state memory device (e.g., flash-based systems), or a combination of any of the above. Information in storage 1008 may, in appropriate cases, be incorporated as virtual memory in memory 1003.

[0289]

[0215] In one example, storage device(s) 1035 may be removably interfaced with computer system 1000 (e.g., via an external port connector (not shown)) via a storage device interface 1025. Particularly, storage device(s) 1035 and an associated machine-readable medium may provide non-volatile and / or volatile storage of machine-readable instructions, data structures, program modules, and / or other data for the computer system 1000. In one example, software may reside, completely or partially, within a machine-readable medium on storage device(s) 1035. In another example, software may reside, completely or partially, within processor(s) 1001.

[0290]

[0216] Bus 1040 connects a wide variety of subsystems. Herein, reference to a bus may encompass one or more digital signal lines serving a common function, where appropriate. Bus 1040 may be any of several types of bus structures including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combinations thereof, using any of a variety of bus architectures. As an example and not by way of limitation, such architectures include an Industry Standard Architecture (ISA) bus, an Enhanced ISA (EISA) bus, a Micro Channel Architecture (MCA) bus, a Video Electronics Standards Association local bus (VLB), a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, an Accelerated Graphics Port (AGP) bus, HyperTransport (HTX) bus, serial advanced technology attachment (SATA) bus, and any combinations thereof.

[0291]

[0217] Computer system 1000 may also include an input device 1033. In one example, a user of computer system 1000 may enter commands and / or other information into computer system 1000 via input device(s) 1033. Examples of an input device(s) 1033 include, but are not limited to, an alpha-numeric input device (e.g., a keyboard), a pointing device (e.g., a mouse or touchpad), a touchpad, a touch screen, a multi-touch screen, a joystick, a stylus, a gamepad, an audio input device (e.g., a microphone, a voice response system, etc.), an optical scanner, a videoAttyDktNo.: 49407-793601

[0292] or still image capture device (e.g., a camera), and any combinations thereof. In some embodiments, the input device is a Kinect®, Leap Motion®, or the like. Input device(s) 1033 may be interfaced to bus 1040 via any of a variety of input interfaces 1023 (e.g., input interface 1023) including, but not limited to, serial, parallel, game port, USB, FIREWIRE, THUNDERBOLT, or any combination of the above.

[0293]

[0218] In particular embodiments, when computer system 1000 is connected to network 1030, computer system 1000 may communicate with other devices, specifically mobile devices and enterprise systems, distributed computing systems, cloud storage systems, cloud computing systems, and the like, connected to network 1030. Communications to and from computer system 1000 may be sent through network interface 1020. For example, network interface 1020 may receive incoming communications (such as requests or responses from other devices) in the form of one or more packets (such as Internet Protocol (IP) packets) from network 1030, and computer system 1000 may store the incoming communications in memory 1003 for processing. Computer system 1000 may similarly store outgoing communications (such as requests or responses to other devices) in the form of one or more packets in memory 1003 and communicated to network 1030 from network interface 1020. Processor(s) 1001 may access these communication packets stored in memory 1003 for processing.

[0294]

[0219] Examples of the network interface 1020 include, but are not limited to, a network interface card, a modem, and any combination thereof. Examples of a network 1030 or network segment 1030 include, but are not limited to, a distributed computing system, a cloud computing system, a wide area network (WAN) (e.g., the Internet, an enterprise network), a local area network (LAN) (e.g., a network associated with an office, a building, a campus or other relatively small geographic space), a telephone network, a direct connection between two computing devices, a peer-to-peer network, and any combinations thereof. A network, such as network 1030, may employ a wired and / or a wireless mode of communication. In general, any network topology may be used.

[0295]

[0220] Information and data can be displayed through a display 1032. Examples of a display 1032 include, but are not limited to, a cathode ray tube (CRT), a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic liquid crystal display (OLED) such as a passive-matrix OLED (PMOLED) or active-matrix OLED (AMOLED) display, a plasma display, and any combinations thereof. The display 1032 can interface to the processor(s) 1001, memory 1003, and fixed storage 1008, as well as other devices, such as input device(s) 1033, via the bus 1040. The display 1032 is linked to the bus 1040 via a video interface 1022, and transport of data between the display 1032 and the bus 1040 can be controlled via the graphics control 1021. In some embodiments, the display is a video projector. In someAttyDktNo.: 49407-793601

[0296] embodiments, the display is a head-mounted display (HMD) such as a VR headset. In further embodiments, suitable VR headsets include, by way of non-limiting examples, HTC Vive®, Oculus Rift®, Samsung Gear VR®, Microsoft HoloLens®, Razer OSVR®, FOVE VR®, Zeiss VR One®, Avegant Glyph®, Freefly VR® headset, and the like. In still further embodiments, the display is a combination of devices such as those disclosed herein.

[0297]

[0221] In addition to a display 1032, computer system 1000 may include one or more other peripheral output devices 1034 including, but not limited to, an audio speaker, a printer, a storage device, and any combinations thereof. Such peripheral output devices may be connected to the bus 1040 via an output interface 1024. Examples of an output interface 1024 include, but are not limited to, a serial port, a parallel connection, a USB port, a FIREWIRE port, a THUNDERBOLT port, and any combinations thereof.

[0298]

[0222] In addition or as an alternative, computer system 1000 may provide functionality as a result of logic hardwired or otherwise embodied in a circuit, which may operate in place of or together with software to execute one or more processes or one or more steps of one or more processes described or illustrated herein. Reference to software in this present disclosure may encompass logic, and reference to logic may encompass software. Moreover, reference to a computer-readable medium may encompass a circuit (such as an IC) storing software for execution, a circuit embodying logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware, software, or both.

[0299]

[0223] Various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described herein generally in terms of their functionality.

[0300]

[0224] The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.AttyDktNo.: 49407-793601

[0301]

[0225] The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by one or more processor(s), or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium. An example storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

[0302]

[0226] In accordance with the description herein, suitable computing devices include, by way of non-limiting examples, server computers, desktop computers, laptop computers, notebook computers, sub-notebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, Internet appliances, mobile smartphones, tablet computers, personal digital assistants, video game consoles, and vehicles. Select televisions, video players, and digital music players with optional computer network connectivity are suitable for use in the system described herein. Suitable tablet computers, in various embodiments, include those with booklet, slate, and convertible configurations.

[0303]

[0227] In some embodiments, the computing device includes an operating system configured to perform executable instructions. The operating system is, for example, software, including programs and data, which manages the device’s hardware and provides services for execution of applications. Suitable server operating systems include, by way of non-limiting examples, FreeBSD®, OpenBSD®, NetBSD®, Linux®, Apple® Mac OS X Server®, Oracle Solaris®, Windows Server®, and Novell NetWare®. Suitable personal computer operating systems include, by way of non-limiting examples, Microsoft Windows®, Apple Mac® OS X, UNIX®, and UNIX-like operating systems such as GNU / Linux®. In some embodiments, the operating system is provided by cloud computing. Suitable mobile smartphone operating systems include, by way of non-limiting examples, Nokia Symbian® OS, Apple® iOS, Research In Motion BlackBerry® OS, Google® Android®, Microsoft® Windows Phone® OS, Microsoft® Windows Mobile OS, Linux®, and Palm® WebOS. Suitable media streaming device operating systems include, by way of non-limiting examples, Apple TV®, Roku®, Boxee®, Google TV®, Google Chromecast®, Amazon Fire®, and Samsung® HomeSync®. Suitable video game console operating systems include, by way of non-limiting examples, Sony® PS3®, Sony® PS4®, Microsoft® Xbox 360®, Microsoft Xbox One®, Nintendo Wii®, Nintendo Wii U®, and Ouya®. Suitable virtual reality headset systems include, by way of non-limiting example, Meta Oculus®.AttyDktNo.: 49407-793601

[0304] Non-transitory computer readable storage mediums

[0305]

[0228] In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more non-transitory computer readable storage media encoded with a program including instructions executable by the operating system of an optionally networked computing device. In further embodiments, a computer readable storage medium is a tangible component of a computing device. In still further embodiments, a computer readable storage medium is optionally removable from a computing device. In some embodiments, a computer readable storage medium includes, by way of non-limiting examples, CD-ROMs, DVDs, flash memory devices, solid state memory, magnetic disk drives, magnetic tape drives, optical disk drives, distributed computing systems including cloud computing systems and services, and the like. In some cases, the program and instructions are permanently, substantially permanently, semi-permanently, or non-transitorily encoded on the media.

[0306] Computer programs

[0307]

[0229] In some embodiments, the platforms, systems, media, and methods disclosed herein include at least one computer program, or use of the same. A computer program includes a sequence of instructions, executable by one or more processor(s) of the computing device’s CPU, written to perform a specified task. Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), computing data structures, and the like, that perform particular tasks or implement particular abstract data types. In light of the present disclosure provided herein, a computer program may be written in various versions of various languages.

[0308]

[0230] The functionality of the computer readable instructions may be combined or distributed as desired in various environments. In some embodiments, a computer program comprises one sequence of instructions. In some embodiments, a computer program comprises a plurality of sequences of instructions. In some embodiments, a computer program is provided from one location. In other embodiments, a computer program is provided from a plurality of locations. In various embodiments, a computer program includes one or more software modules. In various embodiments, a computer program includes, in part or in whole, one or more web applications, one or more mobile applications, one or more standalone applications, one or more web browser plug-ins, extensions, add-ins, or add-ons, or combinations thereof.AttyDktNo.: 49407-793601

[0309] Web applications

[0310]

[0231] In some embodiments, a computer program includes a web application. In light of the present disclosure provided herein, a web application, in various embodiments, utilizes one or more software frameworks and one or more database systems. In some embodiments, a web application is created upon a software framework such as Microsoft® .NET or Ruby on Rails® (RoR). In some embodiments, a web application utilizes one or more database systems including, by way of non-limiting examples, relational, non-relational, object oriented, associative, and XML database systems. In further embodiments, suitable relational database systems include, by way of non-limiting examples, Microsoft® structured query language (SQL) Server, mySQL™, and Oracle®. A web application, in various embodiments, is written in one or more versions of one or more languages. A web application may be written in one or more markup languages, presentation definition languages, client-side scripting languages, server-side coding languages, database query languages, or combinations thereof. In some embodiments, a web application is written to some extent in a markup language such as Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), or extensible Markup Language (XML). In some embodiments, a web application is written to some extent in a presentation definition language such as Cascading Style Sheets (CSS). In some embodiments, a web application is written to some extent in a client-side scripting language such as Asynchronous Javascript and XML® (AJAX), Flash Actionscript, Javascript®, or Silverlight®. In some embodiments, a web application is written to some extent in a server-side coding language such as Active Server Pages® (ASP), ColdFusion®, Perl®, Java®, JavaServer Pages® (JSP), Hypertext Preprocessor® (PHP), Python®, Ruby®, Tel®, Smalltalk®, WebDNA®, or Groovy®. In some embodiments, a web application is written to some extent in a database query language such as Structured Query Language (SQL). In some embodiments, a web application integrates enterprise server products such as IBM Lotus Domino®. In some embodiments, a web application includes a media player element. In various further embodiments, a media player element utilizes one or more of many suitable multimedia technologies including, by way of non-limiting examples, Adobe® Flash®, HTML 5, Apple® QuickTime®, Microsoft Silverlight®, Java®, and Unity®.

[0311] Mobile applications

[0312]

[0232] In some embodiments, a computer program includes a mobile application provided to a mobile computing device. In some embodiments, the mobile application is provided to a mobile computing device at the time it is manufactured. In other embodiments, theAttyDktNo.: 49407-793601

[0313] mobile application is provided to a mobile computing device via the computer network described herein.

[0314]

[0233] In view of the present disclosure provided herein, a mobile application is created by techniques using hardware, languages, and development environments. Mobile applications are written in several languages. Suitable programming languages include, by way of nonlimiting examples, C, C++, C#, Objective-C, Java®, Javascript®, Pascal®, Object Pascal®, Python™, Ruby®, VB.NET®, WML®, and XHTML / HTML with or without CSS, or combinations thereof.

[0315]

[0234] Suitable mobile application development environments are available from several sources. Commercially available development environments include, by way of non-limiting examples, AirplaySDK®, alcheMo®, Appcelerator®, Celsius®, Bedrock®, Flash Lite®, .NET Compact Framework®, Rhomobile®, and WorkLight Mobile Platform®. Other development environments are available without cost including, by way of non-limiting examples, Lazarus®, MobiFlex®, MoSync®, and Phonegap®. Also, mobile device manufacturers distribute software developer kits including, by way of non-limiting examples, iPhone® and iPad® (iOS) SDK, Android® SDK, BlackBerry® SDK, BREW SDK, Palm® OS SDK, Symbian® SDK, webOS® SDK, and Windows® Mobile SDK.

[0316]

[0235] Several commercial sources are available for distribution of mobile applications including, by way of non-limiting examples, Apple® App Store, Google® Play, Chrome® WebStore, BlackBerry® App World, App Store® for Palm devices, App Catalog® for webOS, Windows® Marketplace for Mobile, Ovi Store for Nokia® devices, Samsung® Apps, and Nintendo® DSi Shop.

[0317] Standalone applications

[0318]

[0236] In some embodiments, a computer program includes a standalone application, which is a program that is run as an independent computer process, not an add-on to an existing process, e.g., not a plug-in. Standalone applications are often compiled. A compiler is a computer program(s) that transforms source code written in a programming language into binary object code such as assembly language or machine code. Suitable compiled programming languages include, by way of non-limiting examples, C, C++, Objective-C®, COBOL®, Delphi®, Eiffel®, Java®, Lisp®, Python®, Visual Basic®, and VB .NET®, or combinations thereof.

[0319] Compilation is often performed, at least in part, to create an executable program. In some embodiments, a computer program includes one or more executable compiled applications. Additionally, microservices related to Python® and JavaScript® may be used.AttyDktNo.: 49407-793601

[0320] Web browser plug-ins

[0321]

[0237] In some embodiments, the computer program includes a web browser plug-in (e.g., web extension, etc.). In computing, a plug-in is one or more software components that add specific functionality to a larger software application. Makers of software applications support plug-ins to enable third-party developers to create abilities which extend an application, to support easily adding new features, and to reduce the size of an application. When supported, plug-ins enable customizing the functionality of a software application. For example, plug-ins are commonly used in web browsers to play video, generate interactivity, scan for viruses, and display particular file types. Several web browser plug-ins may include Adobe Flash Player®, Microsoft Silverlight®, and Apple QuickTime®. In some embodiments, the toolbar comprises one or more web browser extensions, add-ins, or add-ons. In some embodiments, the toolbar comprises one or more explorer bars, tool bands, or desk bands.

[0322]

[0238] In view of the present disclosure provided herein, several plug-in frameworks are available that enable development of plug-ins in various programming languages, including, by way of non-limiting examples, C++, Delphi®, Java®, PHP®, Python®, and VB .NET®, or combinations thereof.

[0323]

[0239] Web browsers (also called Internet browsers) are software applications, designed for use with network-connected computing devices, for retrieving, presenting, and traversing information resources on the World Wide Web. Suitable web browsers include, by way of nonlimiting examples, Microsoft Internet Explorer®, Mozilla Firefox®, Google Chrome®, Apple Safari®, Opera Software Opera®, and KDE Konqueror®. In some embodiments, the web browser is a mobile web browser. Mobile web browsers (also called microbrowsers, mini-browsers, and wireless browsers) are designed for use on mobile computing devices including, by way of nonlimiting examples, handheld computers, tablet computers, netbook computers, subnotebook computers, smartphones, music players, personal digital assistants (PDAs), and handheld video game systems. Suitable mobile web browsers include, by way of non-limiting examples, Google Android® browser, RIM BlackBerry® Browser, Apple Safari®, Palm Blazer®, Palm WebOS® Browser, Mozilla Firefox® for mobile, Microsoft Internet Explorer Mobile®, Amazon Kindle Basic Web®, Nokia Browser®, Opera Software Opera Mobile®, and Sony PSP® browser.

[0324] Software modules

[0325]

[0240] In some embodiments, the platforms, systems, media, and methods disclosed herein include software, server, and / or database modules, or use of the same. In view of the present disclosure provided herein, software modules are created by techniques using machines, software, and languages. The software modules disclosed herein are implemented in a multitudeAttyDktNo.: 49407-793601

[0326] of ways. In various embodiments, a software module comprises a file, a section of code, a programming object, a programming structure, or combinations thereof. In further various embodiments, a software module comprises a plurality of files, a plurality of sections of code, a plurality of programming objects, a plurality of programming structures, or combinations thereof. In various embodiments, the one or more software modules comprise, by way of nonlimiting examples, a web application, a mobile application, and a standalone application. In some embodiments, software modules are in one computer program or application. In other embodiments, software modules are in more than one computer program or application. In some embodiments, software modules are hosted on one machine. In other embodiments, software modules are hosted on more than one machine. In further embodiments, software modules are hosted on a distributed computing platform such as a cloud computing platform. In some embodiments, software modules are hosted on one or more machines in one location. In other embodiments, software modules are hosted on one or more machines in more than one location.

[0327] Databases

[0328]

[0241] In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more databases (DB), or use of the same. In view of the present disclosure provided herein, many databases are suitable for storage and retrieval data. In various embodiments, suitable databases include, by way of non-limiting examples, relational databases, non-relational databases, object oriented databases, object databases, entity-relationship model databases, associative databases, XML databases, time-series databases, graph databases, and the like. Further non-limiting examples include SQL, PostgreSQL®, MySQL®, Oracle®, DB2®, and Sybase. In some embodiments, a database is internet-based. In further embodiments, a database is web-based. In still further embodiments, a database is cloud computing-based. In a particular embodiment, a database is a distributed database. In other embodiments, a database is based on one or more local computer storage devices.

[0329] EXAMPLES

[0330] Example 1: Predicting a Presence or Absence of CRC using an FLDL Model

[0331]

[0242] To demonstrate the utility of the FLDL model for the prediction of CRC, three aspects may be considered. Predictive performance: FLDL may be compared to a state-of-the-art ML method, or to DL models that use max pooling instead of MHN for fragment aggregation. FLDL and max pooling DL models that differ in the size of the genomic regions (input space) they operate on may also be evaluated. Thus, the extent of denoising needed for identifying true cancer signal from healthy cfDNA background may be evaluated. Scaling behavior withAttyDktNo.: 49407-793601

[0332] increasing number of training samples: To investigate the scaling laws of our architecture, performance of one of the FLDL variants may be evaluated as a function of training set size. Extraction of biological insights: interpretability techniques may be leveraged to extract insights at the sample or the fragment level.

[0333]

[0243] A training set may comprise 925 positive cases, comprising 377 CRCs and 577 of the more difficult APLs, as well as 3,469 negative controls. To assess classification accuracy on data that are representative of future use cases, two complementary independent hold-out test sets may be utilized. The test sets may comprise a real-world clinical sample set comprising 331 negative and 599 positive samples (388 APLs, 211 CRCs) collected independently of the training dataset. The test sets may comprise a challenging contrived set, to probe the detection of samples with low witness rates in a controlled setting or with much higher replication. The contrived set may comprise 148 replicates created by mixing material from a single advanced CRC donor into plasma from a healthy donor pool to yield a ctDNA level just above the state-of-the-art ML model’s detection limit.

[0334]

[0244] To inspect the learned representations of the model’s sample-level embeddings, additional hold-out test sets may be leveraged. The additional hold-out test sets may comprise a high-signal contrived set comprising 36 replicates created by mixing material from a single advanced CRC donor into plasma from a healthy donor pool at a level above the state-of-the-art ML model’s detection limit. The additional hold-out test sets may comprise an artificial nomethylation set comprising 92 synthetically unmethylated samples, where residual methylation signal may be attributed to technical noise.

[0335]

[0245] The variants of the FLDL model and two other architectures that its performance is compared to may be provided. The two other architectures may comprise a state-of-the-art ML baseline, and a max pooling based (instead of MHN) DL model.

[0336]

[0246] The wet-lab sample processing pipeline’s full capture panel may target at least about 500 kilobases (kb) of differentially methylated genomic regions. The differentially methylated genomic regions may be identified through an iterative process leveraging public or internal methylation data. The full panel may capture signals relevant to multiple cancer types. A subset may be tailored to maximize CRC signal detection, and this subset is referred to as the “CRC panel.” In order to detect sparse signals necessary for early cancer detection, both panels may retain certain loci that can exhibit sporadic background methylation in healthy individuals. During training, ML or DL models may learn to denoise, (e.g., ignore) such genomic loci, ensuring that non-ctDNA fragments from noisy regions do not lead to false positives. While the ML model may incorporate an explicit denoising operation during training (as disclosed herein), the FLDL architecture may not incorporate such an operation. The FLDL model may rely onAttyDktNo.: 49407-793601

[0337] attention for implicit denoising, and biologically informed pre-filtering may assist when training data may be limited.

[0338]

[0247] To investigate the effect of input space denoising on model performance at the current training dataset size, three distinct FLDL configurations may be evaluated.

[0339]

[0248] Prior Denoised (PD-FLDL): A variant incorporating explicit biological priors to reduce input noise. Methylation signatures in the CRC capture panel may be profiled using a hold-out cohort of 825 healthy individuals. Regions exhibiting elevated background hypermethylation in these controls may be aggressively filtered, yielding a reduced, higher-signal-to-noise input space. This data driven pre-filtration of the feature space may be referred to as denoising with “explicit biological priors.”

[0340]

[0249] CRC Panel (C-FLDL): The FLDL model operating on the complete CRC panel without prior filtering. This configuration may test the architecture’s capacity to implicitly denoise the input space or identify cancer-specific signals within the standard CRC target domain.

[0341]

[0250] Expanded Region (ER-FLDL): A scalability benchmark extending the input space to the full capture panel, (e.g., beyond the CRC-specific regions). This configuration may challenge the model to isolate CRC signals within a larger search space, thereby evaluating its robustness to lower signal-to-noise ratios.

[0342]

[0251] PD-FLDL regions may comprise a subset of C-FLDL regions, which may comprise a subset of ER-FLDL regions, with fragment distributions as shown in FIG. 7A, FIG.

[0343] 7B, and FIG. 7C, respectively.

[0344]

[0252] FIGs. 7A-7C show fragment count distributions for the Prior Denoised (PD), CRC Panel, and Expanded Region (ER) training sets (“Train”) vs. the challenging contrived (“Contrived”) and real-world clinical (“Real-world”) test sets.

[0345]

[0253] DL model based on max pooling. To isolate the contribution of the MHN attention mechanism, a baseline DL model that replaces the Hopfield Pooling layer with featurewise max pooling while retaining the FLDL fragment feature extractor may be evaluated.

[0346] Formally, given the bag of N embedded fragments Y = [hi, . . . , IIN] where each hi E Rm, the MaxPool baseline constructs the sample embedding z E Rmby first identifying the representative fragment for each feature dimension. For each feature index j E {1, . . . , m}, the fragment index i* (j) that maximizes the activation of that feature: i* (j) = arg maxi6{i,...,N} hk may be determined. A collection of vectors n, . . . , rmmay be defined, where each rj = hi* (j) E Rmcaptures the full embedding of the fragment achieving the maximum value in the j -th dimension. The final sample embedding z may be obtained by concatenating these vectors: z = ri || n II • • • II rmG Rm2. This pooling strategy may be motivated by two key considerations: a) it may act as a discreteAttyDktNo.: 49407-793601

[0347] analogue to attention-based pooling, where the max-feature activation serves as a proxy for importance, b) by preserving the full embedding hi* rather than a scalar maximum, the model may maintain the co-occurrence of features within a single fragment, thereby ensuring that the sample representation is constructed from intact, biologically-relevant fragment signatures. This architecture may be evaluated on the Prior Denoised (PD-MaxPool) and CRC (C -MaxPool) panels.

[0348]

[0254] State-of-the-art baseline ML model (ML Baseline). This model may operate by identifying hypermethylated fragments (HMFs) in the CRC panel, where HMFs may comprise cfDNA fragments showing significantly more CpG methylation than seen in similar fragments obtained from individuals without disease (Shaukat et al., 2025, which is incorporated by reference herein in its entirety). The model may subdivide the regions of interest in the CRC panel into bins b. For a bin, fragments derived from healthy samples may be compared to fragments derived from cases, and a per-bin methylated CpG threshold tb may be learned.

[0349] Fragments intersecting bin b and with a methylated CpG count at or above (tb) may be deemed to be hypermethylated. The model may aggregate HMF counts observed throughout the regions of interest in order to compute an overall score for the sample. This approach may effectively incorporate a denoising steps within the training process, as only the CRC Panel may be considered, or only fragments with methylated CpG counts exceeding that of healthy samples may be aggregated to contribute to the final classification score. For FLDL, an ensemble of 25 such models, which may be trained to different subsets of the data, may produce the final binary classification.

[0350]

[0255] Comparison of predictive performance

[0351]

[0256] Predictive performance, real-world clinical sample test set. One indicator of the model’s efficacy is its performance on the real-world clinical sample test set, which may provide an estimate of likely future performance in actual clinical settings. For blood-based CRC screening, a specificity at or near 90% may be considered clinically appropriate (Shaukat et al., 2025; Chung et al., 2024; each of which is incorporated by reference herein in their entirety). As a result, a relevant performance metric may not be the area under the receiver operating characteristic curve (AUROC), but rather, the sensitivity at a fixed specificity of 90%. As such, the achieved APL and CRC sensitivities at a specificity of 90% may be reported, along with Wilson’s method 95% confidence intervals (CI), in Table 1. For each method, a classification threshold may be selected to achieve the desired specificity among the negative samples in the real-world clinical sample set. As shown in Table 1, PD-FLDL may outperform all competing methods, achieving sensitivities of 30.2% for APLs and 89.6% for CRCs. This improvement over the ML Baseline may be more pronounced in the challenging APL group: a gain of 3.1AttyDktNo.: 49407-793601

[0352] points over the baseline’ s 27.1% vs. a gain of 1.4 points for CRCs. Further, PD-FLDL may outperform C-FLDL and ER-FLDL, which suggests that leveraging biological priors to effectively denoise background signals before FLDL training may be an effective mechanism for improving model detection sensitivity, at least at the current training set sample size.

[0353] Table 1: Real-world clinical sample test set sensitivity at 90% specificity, with Wilson’s method 95% Cis, for all models.

[0354]

[0355]

[0257] It may be noteworthy that the C-FLDL model, which operates on the CRC panel but does not use an additional negative sample set for biologically informed filtering, achieves a sensitivity of 28.9% for APL and 89.1% for CRC, also outperforming the ML Baseline.

[0356] Additionally, even when the input space is significantly expanded to include more task-irrelevant genomic regions (e.g., ER-FLDL), the architecture may still maintain competitive performance relative to the ML Baseline model. In contrast, models employing max pooling (PD-MaxPool, C-MaxPool) may show significantly degraded performance, with sensitivities dropping below 20% for APLs and below 80% for CRCs. While PD-MaxPool improves upon C-MaxPool, consistent with the hypothesized benefit of using additional negative samples for prior denoising, it may fail to match the performance of the ML Baseline or the attention-based FLDL variants. This evidence may support the hypothesis that a simple max pooling aggregation may be insufficient for identifying sparse ctDNA signals. The ability to dynamically attend to rare, informative fragments (here via MHN) may contribute to high-sensitivity cancer detection.

[0357]

[0258] The challenging contrived test set may provide control over the true ctDNA level, and the blending process may provide enough plasma volume for more replication than is possible with a clinical sample. Using the same model-specific 90% specificity classification thresholds, in FIG. 3 each model’s proportion of positive predictions among the 148 replicates may be reported.

[0358]

[0259] FIG. 3 shows positive call rates among 148 replicates of the challenging contrived test set, at 90% specificity. Whiskers indicate Wilson’s method 95% Cis.

[0359]

[0260] Performance on the challenging contrived test set (shown in FIG. 3) may recapitulate trends observed in the real-world clinical sample set. PD-FLDL may achieve a positive call rate of 84.2%, outperforming the ML Baseline (70.2%). PD-FLDL may outperformAttyDktNo.: 49407-793601

[0360] C-FLDL, which may outperform ER-FLDL. C-FLDL (74.3%) may exceed the ML Baseline, but ER-FLDL (66.9%) may underperform relative to the ML Baseline model. Finally, both PD-MaxPool and C-MaxPool may struggle on this task and may fail to achieve a competitive detection rate when presented with cases with a challenging ctDNA level. These results may validate the importance of the MHN attention mechanism for low-signal regimes or demonstrate that while biological prior denoising may yield improved FLDL performance (at the current training set sample size), the architecture may be capable of effective implicit denoising.

[0361]

[0261] Since neither C-FLDL nor ER-FLDL outperforms PD-FLDL on either test set, it may be inferred that the denoised panel (in spite of aggressive filtering) effectively retains predictive signal, or that current training data may be insufficient for the models to autonomously learn a superior noise filter (as disclosed herein). ER-FLDL and C-FLDL may perform competitively, showing that the FLDL architecture a) may scale to very large numbers of fragments per sample, b) may reduce the need for biological priors for pre-filtering of fragments or c) may operate on broad genomic panels, paving the way for multi-cancer detection beyond CRC.

[0362] Example 2: Scaling behavior with increasing number training samples

[0363]

[0262] In order to understand the scale of data useful for effective training for the CRC detection task or to identify the onset of potential performance plateaus, the scaling behavior of the FLDL architecture when trained on different dataset sizes may be explored. For this, the C-FLDL model may be trained on subsets of the full training dataset and resulting models’ positive call rate on the challenging contrived test samples may be reported. The positive cases (CRCs and APLs) and the negative controls may be subsampled separately to a given percentage (e.g., 20% to 100% in 20-point increments), which may maintain the relative proportion of positives and negatives in one or more subsets. To reduce computational effort in training, 5 models may be ensembled from a single 5-fold cross validation split (as shown in FIG. 9).

[0364]

[0263] As shown in FIG. 4, at all but the smallest subsampling fraction, the ensemble may outperform its individual members, suggesting that ensembling may be an effective strategy for smoothing noisy performance of individual members due to limited training data. A clear improvement in model performance may be observed as the training dataset size increases, and plateauing may not be observed, even when approaching the full training dataset size. The increasing performance trend indicates that the FLDL model, as currently parameterized, may continue to benefit from even more training samples. The data indicated that there may be a potential for further performance improvements by scaling the complexity the model along with increased training data volume in the future.AttyDktNo.: 49407-793601

[0365]

[0264] FIG. 4 shows positive call rates among 148 replicates of the challenging contrived test set, at 90% specificity, for C-FLDL models trained on subsets of the full training dataset (20% to 100% of the full dataset). Results are shown for the ensemble of 5 models (black) as well as for individual ensemble members.

[0366] Example 3: Latent space analysis and biological interpretability

[0367]

[0265] For FLDL interpretability at the fragment level, analyze the model’s attention values may be analyzed. At the sample level, embeddings after MHN aggregation may be used to visualize sample distributions.

[0368]

[0266] Robustness and good generalization may be important for deep learning models in clinical settings. To assess these properties, the C-FLDL model’s training and test set sample embeddings may be analyzed, which form a compressed representation of the input data. In FIG.

[0369] 5, the training and test set sample embeddings may be projected into a two-dimensional space using UMAP (Mclnnes et al., 2018; which is incorporated by reference herein in its entirety) to inspect the latent structure for potential distribution shifts or biological relevance.

[0370]

[0267] FIG. 5 shows UMAP projection of learned FLDL sample embeddings from the C-FLDL model, applied to real-world clinical, contrived blend, and artificial no-methylation samples.

[0371]

[0268] The visualization may reveal a coherent biological continuum: a smooth gradient transitioning from no-ctDNA negative clinical samples to challenging low-ctDNA clinical blends, to high-ctDNA clinical blends and real-world clinical APL and CRC samples (red) may be observed. Furthermore, two sets of artificial no-methylation samples, which lack CpG methylation and are hence biological implausible may form a distinct cluster that is well separated from real-world clinical samples and clinical blends, and in a direction that is distinct from that of the ctDNA gradient. Together, these results may demonstrate that the FLDL model learns a robust, biologically meaningful representation that may effectively distinguish among biologically plausible samples based on their ctDNA content, and may separate biologically implausible inputs.

[0372]

[0269] To identify the genomic loci driving the C-FLDL model’s predictions, the distribution of high-attention fragments within the CRC panel may be analyzed. The starting points for this analysis may be the fragments retained after the attention-based subsampling of the Hopfield pooling layer. To focus on fragments with meaningful contributions to the prediction, a further sample-specific filtering operation may be applied. Given the attention matrix A G RKx N, for each of the K state patterns, fragments with an attention value above threshold ti > r • maxNj=i(Aij) may be retained, where i G {1, . . . , K},j G {1, . . . , N},andr is aAttyDktNo.: 49407-793601

[0373] percentage threshold (e.g., set to 0.1). The genomic regions interrogated by the CRC panel may be partitioned into non-overlapping b base pair bins, maintaining consistency with the bins predefined for the ML Baseline model. The filtered fragments may be aligned to these bins, and the relevance count for a bin may be incremented for a fragment that at least partially overlaps the bin. This aggregation may identify the genomic regions prioritized by the C-FLDL model for its classification decisions. This may be compared to ML Baseline model’s per-bin HMF counts.

[0374]

[0270] FIG. 6 illustrates this comparison using data from 13 correctly predicted latestage CRC samples from the real-world clinical test set. While the models may exhibit high-frequency variation at the individual bin level, they may demonstrate substantial agreement after smoothing with a moving average, as shown by the overlaps between the darker and lighter curves. Note that a subset of each model’s prioritized genomic loci may ignored by the other model (e.g., arrows). This complementarity may suggest that neither model may fully saturate the available signal, or that the models may have made different but functionally similar choices when presented with sets of loci yielding correlated signal.

[0375]

[0271] FIG. 6 shows classification relevance for genomic bins comprising the CRC panel, in 13 correctly predicted late-stage CRC samples from real -world clinical test set; darker curve: density of high-attention fragments from C-FLDL; lighter curve: density of HMFs identified by ML Baseline; Arrows: regions prioritized by one model but not the other; Both curves are smoothed with a moving average.

[0376] Example 4: Fragment count distributions for different input space denoising strategies.

[0377]

[0272] FIGs. 7A-7C show fragment count distributions for the Prior Denoised (PD), CRC Panel, and Expanded Region (ER) training sets (“Train”) vs. the challenging contrived (“Contrived”) and real-world clinical (“Real-world”) test sets.

[0378]

[0273] FIGs. 7A-7C illustrate the distribution of fragment counts per sample, for training and held-out test datasets across the three FLDL configurations. As the model’s input space expands, a distinct rightward shift in the fragment count distributions may be observed. The PD-FLDL configuration, which applies aggressive biological filtering to remove noisy regions, may yield the lowest fragment counts, with the training distribution centered between 105and 106fragments. The C-FLDL configuration may target the complete CRC panel without prior filtering, resulting in an intermediate increase in data volume. Finally, the ER-FLDL configuration, which may target targets the full capture panel, may require the processing of the largest number of instances, with fragment counts sometimes exceeding 106per sample with a few reaching 107fragments. Of note, both hold-out test sets may be generated using an improved wet-lab sample processing pipeline and, as a result, may show an increase in fragments perAttyDktNo.: 49407-793601

[0379] sample compared to the training set. The pipeline improvements may be intended to increase accuracy of future blood test versions. As a side effect, the shift may also provide a rigorous testbed for evaluating model generalization in the face of an evolving input distribution.

[0380] Example 5: Implementation Details.

[0381]

[0274] As illustrated in FIG. 8 and described herein, the FLDL Fragment Embedding module may embed each of a plurality of fragment input domains using a dedicated modalityspecific encoder.

[0382]

[0275] In some cases, training on millions of fragments per sample may be associated with computational challenges. The methods disclosed herein may utilize a two-stage fragment subsampling strategy to mitigate runtime or GPU memory consumption. During training or inference for early stopping, the method may comprise random dropout of fragments. In some cases, the method comprises attention-based sub sampling that retains the fragments with the highest current attention scores. In some cases, during test-time inference, random dropout may be omitted. In some cases, during test-time inference, the attention-based subsampling is applied. In some cases, the method comprises training the model end-to-end in PyTorch [Paszke et al. (2019); which is incorporated by reference herein in its entirety] using the AdamW optimizer [Loshchilov & Hutter (2017); which is incorporated by reference herein in its entirety]. In some cases, the method comprises a dynamic minibatch balancing scheme that pairs samples with opposite class labels but similar confounding characteristics, thereby preventing exploitation of confounding effects such as collection batch or patient age. In some cases, the method comprises training the model with auxiliary tasks. In some embodiments, the auxiliary tasks predicting relevant clinical metadata and biological characteristics alongside the primary binary disease status.

[0383]

[0276] In some cases, due to a relatively low number of training samples or a low signal-to-noise ratio (e.g., as a result of low witness rate), the representation learned may vary among trained models based on the random weight initialization or the order in which samples are selected for use during training. In some embodiments, the methods disclosed herein comprise ensembling 25 models, which may be obtained from 5 random restarts of 5-fold cross validation (CV). As illustrated in FIG. 9, a 5-fold cross validation may be applied with 5 random restarts to obtain 25 members for the final ensemble model. In some cases, different hyperparameter settings may be considered. In some cases, the optimal choice for that iteration is the one that maximizes a balanced training and tuning set accuracy to enforce regularization. In some cases, the final model score of a sample comprises a mean over the scores of the 25 models in the ensemble.AttyDktNo.: 49407-793601

[0384] Example 6: Training a Fragment Level Deep Learning Model.

[0385]

[0277] The method may comprise measuring cytosine methylation in a DNA molecule. Methylation of cytosines within CpG (p=phosphodiester bond) dinucleotides provides a unique mark on DNA. This may differ from tissue to tissue, and even cell type to cell type. Methylation is involved in transcriptional gene silencing. In the earliest stages of cancer development DNA methylation signatures become aberrant. Like tumor mutations detected in cfDNA, aberrant cfDNA methylation signatures can be attributed specifically to cancer cells.

[0386]

[0278] FIG. 11 shows an example of genomic and methylation data obtained by assaying DNA molecules of a subject. The assaying may convert cytosine (C) to thymine (T), and guanine (G) to adenine (A), while methylated CpG sites may remain as CpG sites. The assaying may provide a 5 base genome corresponding to A, C, T, G, and M (methylated cytosine). The data shows CpG dense regions, which may be captured using targeted capture panels. The assaying may provide high mean target coverage of about 500x to lOOOx. Error correction of reads may be performed based on unique molecular identifiers.

[0387]

[0279] FIG. 12 shows an example of genomic and methylation data obtained by assaying DNA molecules of a subject. As shown, asymptomatic screening populations may have less clean differential signal, as the subjects may have earlier cancer stages, smaller lesions, or precancerous samples.

[0388]

[0280] FIG. 13 shows an example of a method for assessing methylation statistics of a DNA fragment. Tumor derived signal may be exceedingly rare in early stage cancer (10‘4- 1 O’5). ctDNA fragments may be aberrantly methylated. Per-site quantification may dull rare events and be more susceptible to noise due to library preparation or sequencing noise. Fragment-level analysis enables identification of Hyper-methylated fragments (HMFs). The HMFs may amplifies signal when present and may be less likely to be technical artifacts. Current models based on HMF counts may include heuristics & statistical ML models.

[0389]

[0281] FIG. 14 shows an example of a method for minibatch balancing within a training dataset. Minibatches were designed such that correlation of class label with age / cohort / etc. is removed via dynamic oversampling. The method may involve drawing samples for a minibatch (without replacement, each sample is drawn once per epoch). For every sample in the minibatch, a paired sample that has opposite class label and is within same age group, same cohort, etc. is added.

[0390]

[0282] FIG. 15 shows an example of data illustrating a training and tuning curve of a machine learning model. FIG. 15 shows large variance in the performance of the training curves. Random downsampling to 400k random fragments per sample was performed when tuning. AAAttyDktNo.: 49407-793601

[0391] training performance on training set may increase due to overfitting, but the tuning set CRC performance increases indicate improved model performance.

[0392]

[0283] FIG. 16 shows an example of a method for pooling fragment representations into a sample representation. As shown in FIG. 16, the method may comprise random dropout and subsampling.

[0393]

[0284] FIG. 17 shows an example of data illustrating a generalization of high-complexity FLDL models. As shown in FIG. 17, high complexity models may not generalize well.

[0394]

[0285] FIG. 18 shows an example of data illustrating a generalization of high-complexity FLDL models. High complexity model showed promising CV performance but had trouble generalizing to hold-out cohort. Sample embedding analysis showed that the test cohort cluster in distinct groups in high complexity models. Low complexity models can generalize to holdout test but had lower sensitivity.

[0395]

[0286] FIG. 19 shows an example of a method for training a FLDL model. The method comprises identifying a potential source for lack of generalizability and eliminating features during training. The method comprises adding auxiliary tasks during training. The method comprises training FLDL ensemble models for evaluation with tests on multiple hold-out datasets. The method comprises infrastructure related changes.

[0396]

[0287] FIG. 20 shows an example of a data format implementation. Custom file format based on multiple numpy memmap files is used. Each attribute is an uncompressed memmap file (streamed from Google Cloud Storage (GCS) via random access memory (RAM) or disk).

[0397] Fragment sequences are padded / stacked into uint82D array (n fragments, max len). Decoding of feature values from uint8 is performed bit operations. Evaluated alternatives include concatenation to ID array, loading via split / pad / stack (numba / torch), and using hdf5, tiledb*, or zarr files.

[0398]

[0288] FIG. 21 shows an example of a convolutional neural network (CNN) kernel. A fully-connected network may comprise various hyperparameters, including number of neurons, number of layers, or activation function. Triangle encoding may include parameters such as number of neurons. Sequence networks may comprise a CNN and max pooling. The CNN may comprise a number of kernels, a kernel size, a stride or maxpooling size, a number of layers or any activation function. Additional hyperparameters include learning rate, regularization, minibatch size, number of total update steps, number of repeated update steps per minibatch, premodel and model-based subsampling, and feature dropout rates.AttyDktNo.: 49407-793601

[0399] Example 7: Training a Fragment Level Deep Learning Model.

[0400]

[0289] To demonstrate the utility of the FLDL model for the prediction of lung cancer, three aspects may be considered. Predictive performance: FLDL may be compared to a ML method, or to DL models that use max pooling instead of MHN (Modern Hopfield Network) for fragment aggregation. FLDL and max pooling DL models that differ in the size of the genomic regions (input space) that they operate on may also be evaluated. Thus, the extent of denoising needed for identifying true lung cancer signals from healthy cfDNA background (e.g., background from inflammatory pulmonary conditions) may be evaluated. Scaling behavior with increasing number of training samples: To investigate the scaling laws of the lung-specific architecture, performance of one of the FLDL variants may be evaluated as a function of training set size. Extraction of biological insights: Interpretability techniques may be leveraged to extract insights at the sample or the fragment level, specifically identifying methylation patterns associated with non-small cell lung cancer (NSCLC) subtypes.

[0401]

[0290] A training set may comprise a number of confirmed positive cases, comprising some confirmed NSCLC cases and some early-stage (Stage I / II) lung cancer cases, and some negative controls (for example, including age-matched smokers and individuals with benign pulmonary nodules). In this example, to assess classification accuracy on data representative of future clinical use cases, two complementary independent hold-out test sets may be utilized. The test sets may comprise a real-world clinical sample set comprising a number of negative and positive samples (for example, comprising early-stage and advanced NSCLC samples) collected independently of the training dataset. The test sets may comprise a challenging contrived set, to probe the detection of samples with low witness rates (low tumor fraction) in a controlled setting. In this example, the contrived set may comprise a number of replicates created by mixing material from a single advanced lung cancer donor into plasma from a healthy donor pool to yield a ctDNA level that may be just above existing ML model’s detection limit.

[0402]

[0291] In this example, to inspect the learned representations of the model’s sample-level embeddings for lung cancer, additional hold-out test sets may be leveraged. These may include, but are not limited to, a high-signal contrived set comprising a number of replicates at a level that is significantly above the detection limit, and an artificial no-methylation set comprising a number of synthetically unmethylated samples to characterize technical noise floor.

[0403]

[0292] In this example, the variants of the FLDL model and two other architectures for comparison may be provided: a ML baseline (utilizing hypermethylated fragment counts) and a max pooling-based DL model.

[0404]

[0293] In this example, the wet-lab sample processing pipeline’s full capture panel may target a number of differentially methylated genomic regions (DMRs) associated with pulmonaryAttyDktNo.: 49407-793601

[0405] malignancies. These regions may be identified via iterative analysis of public (e.g., TCGA) or internal lung tissue methylation data. In this example, while the full panel may capture signals relevant to multiple cancer types, a subset may be tailored to maximize lung cancer signal detection, referred to as the “Lung Panel.” During training, in this example, the FLDL architecture may rely on attention for implicit denoising of "noisy” loci (e.g., those methylated in cases of Chronic Obstructive Pulmonary Disease (COPD) but not cancer), while biologically informed pre-filtering may be used to assist when training data is limited.

[0406]

[0294] To investigate the effect of input space denoising on lung cancer detection in this example, three distinct FLDL configurations may be evaluated:

[0407]

[0295] Prior Denoised (PD-FLDL): Incorporates explicit biological priors. Regions exhibiting elevated background hypermethylation in a cohort of a number of high-risk smokers (without cancer) are filtered, yielding a reduced, higher-signal-to-noise input space:

[0408]

[0296] Lung Panel (L-FLDL): The FLDL model operating on the complete Lung Panel without prior filtering, testing the architecture’s capacity to implicitly denoise the lung-specific target domain.

[0409]

[0297] Expanded Region (ER-FLDL): A scalability benchmark extending the input space to a full multi-cancer capture panel.

[0410]

[0298] Comparison of predictive performance: Real-world clinical sample test set.

[0411] For blood-based lung cancer screening, where early detection is critical for surgical intervention, a specificity at or near 90% or 95% may be considered clinically appropriate. The achieved sensitivity for early-stage (Stage VII) and total lung cancer at a fixed specificity of 90% may be measured.

[0412]

[0299] PD-FLDL is expected to outperform competing methods, particularly in the difficult early-stage group, and show significant gains over the ML Baseline.

[0413] Example 8: Training a Fragment Level Deep Learning Model.

[0414]

[0300] Esophageal Cancer (EC)

[0415]

[0301] To demonstrate the utility of the FLDL model for the prediction of Esophageal Cancer (EC), predictive performance may be compared to a ML method and DL models using max pooling instead of MHN for fragment aggregation.

[0416]

[0302] Training and Testing: A training set may comprise a number of positive cases, including a number of early-stage Esophageal Adenocarcinomas and a number of cases of Barrett’s Esophagus with high-grade dysplasia.

[0417]

[0303] Denoising and Panel Design: The wet-lab capture panel may target genomic regions tailored for EC signal detection (the “EC Panel”). A variant incorporating explicitAttyDktNo.: 49407-793601

[0418] biological priors (PD-FLDL) may aggressively filter regions exhibiting background hypermethylation in a cohort of a number of individuals with chronic gastroesophageal reflux disease (GERD).

[0419]

[0304] Expected Results: PD-FLDL is expected to outperform the ML Baseline in detecting early-stage EC at 90% specificity, due to the model's capacity to attend to rare, informative fragments within the esophageal target domain.

[0420]

[0305] Liver Cancer (HCC)

[0421]

[0306] The FLDL model may be leveraged for the early detection of Hepatocellular Carcinoma (HCC).

[0422]

[0307] Training and Testing: The training set may comprise a number of positive cases (HCC) and a number of negative controls, including individuals with cirrhosis or chronic Hepatitis B / C to ensure the model learns to distinguish cancer signals from inflammatory background noise.

[0423]

[0308] Architectural Benchmarking: To isolate the contribution of the MHN attention mechanism, a MaxPool baseline may be evaluated, which is expected to show degraded performance compared to FLDL variants in identifying sparse ctDNA signals in a high-noise cirrhotic background.

[0424]

[0309] Performance Metrics: In a real-world clinical test set, the FLDL architecture is expected to achieve higher sensitivity for Stage I HCC compared to hypermethylated fragment count (HMF) baselines, due to its efficacy in low-tumor-fraction regimes.

[0425]

[0310] Gastric Cancer (GC)

[0426]

[0311] To investigate the scalability of the FLDL architecture for Gastric Cancer (GC), performance may be evaluated as a function of training set size and panel complexity.

[0427]

[0312] Training and Testing: A training set may comprise a number of positive cases (GC) and a number of negative controls. To assess accuracy, a hold-out clinical set comprising a number of negative and a number of positive samples (including early-stage gastric lesions) may be utilized.

[0428]

[0313] Implicit vs. Explicit Denoising: While an ML baseline may require explicit perbin methylated CpG thresholds (HMF counts), the FLDL model may rely on attention for implicit denoising, dynamically attending to informative fragments.

[0429]

[0314] Expected Results: Similar to the CRC model, PD-FLDL for gastric cancer is expected to achieve a sensitivity at 90% specificity that outperforms max pooling architectures.AttyDktNo.: 49407-793601

[0430] Example 9: Multi-Cancer Detection and Tissue-of-Origin (TOO) Prediction using an Integrated FLDL Platform.

[0431]

[0315] To demonstrate the scalability and pan-cancer utility of the FLDL architecture, an integrated model may be evaluated for the simultaneous detection of Colorectal Cancer (CRC), Lung Cancer, Esophageal Cancer (EC), Liver Cancer (HCC), Ovarian Cancer (OC), and Gastric Cancer (GC). Predictive performance may be compared across a multi -omic input space to evaluate the model's capacity for implicit denoising when presented with a vast array of task-irrelevant genomic regions.

[0432]

[0316] A pan-cancer training set may comprise a heterogeneous cohort of a number of positive cases across the six target indications, including both early-stage and advanced malignancies, as well as a robust negative control set of a number of samples. To assess classification accuracy on data representative of a multi-cancer screening environment, a real-world clinical hold-out test set may be utilized. This set may comprise a number of negative samples and a number of positive samples distributed across all target cancer types.

[0433]

[0317] The wet-lab processing pipeline may utilize an Expanded Region (ER) capture panel targeting a number of differentially methylated genomic regions. In this example, this panel is designed to capture a broad spectrum of signals relevant to multiple cancer types, including those with sparse ctDNA witness rates. While the ML baseline model may require a subdivided approach — identifying hypermethylated fragments (HMFs) within specific indication-optimized bins — the ER-FLDL model operates on the full capture panel without such explicit pre-filtering.

[0434]

[0318] To evaluate the architecture's robustness, three distinct multi-cancer configurations may be compared:

[0435]

[0319] Indication-Specific Prior Denoised (PD-FLDL): Models trained using explicit biological priors for each specific cancer type to reduce input noise.

[0436]

[0320] Pan-Cancer FLDL (PC-FLDL): A single FLDL model trained to identify any of the six cancer types from the expanded genomic regions.

[0437]

[0321] Pan-Cancer MaxPool: A baseline DL model utilizing feature-wise max pooling for fragment aggregation, which identifies the representative fragment for each feature dimension to construct the sample embedding.

[0438]

[0322] Comparison of Pan-Cancer Predictive Performance. The performance of the ER-FLDL architecture may be evaluated by its sensitivity at a fixed high specificity (e.g., 90%, 95%, or 98%) to reflect the requirements of a multi-cancer screening tool. The FLDL architecture is expected to maintain competitive sensitivities across all indications, even when operating on the significantly expanded input space of the ER panel.AttyDktNo.: 49407-793601

[0439]

[0323] The FLDL architecture, via the Modern Hopfield Network (MHN) attention mechanism, may effectively performs implicit denoising. It is capable of isolating rare, informative fragments from a high-noise, large-scale genomic input space (ER panel) that would otherwise overwhelm traditional max pooling aggregation methods. This capability facilitates the expansion of the diagnostic platform to a multi-cancer detection regime without a corresponding loss in per-indication sensitivity.

[0440] References

[0441]

[0324] Alva Principe, R., Chiarini, N., and Viviani, M. Long document classification in the transformer era: A survey on challenges, advances, and open issues. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 15 (2):e70019, 2025; which is incorporated by reference herein in its entirety.

[0442]

[0325] Bao, H., Yang, S., Chen, X., Dong, G., Mao, Y., Wu, S., Cheng, X., Wu, X., Tang, W., Wu, M., et al. Early detection of multiple cancer types using multidimensional cell-free dna fragmentomics. Nature Medicine, 2025. doi: 10.1038 / s41591-025-03735-2; which is incorporated by reference herein in its entirety.

[0443]

[0326] Berroukham, A., Housni, K., and Lahraichi, M. Vision transformers: a review of architecture, applications, and future directions. In 2023 7th IEEE congress on information science and Technology (CiSt), pp. 205-210. IEEE, 2023; which is incorporated by reference herein in its entirety.

[0444]

[0327] Bertsch, A., Alon, U., Neubig, G., and Gormley, M. Unlimiformer: Long-range transformers with unlimited length input. Advances in Neural Information Processing Systems, 36:35522-35543, 2023; which is incorporated by reference herein in its entirety.

[0445]

[0328] Bettegowda, C. et al. Detection of circulating tumor DNA in early- and late-stage human malignancies. Science Translational Medicine, 6(224) :224ra24, 2014; which is incorporated by reference herein in its entirety.

[0446]

[0329] Brixi, G., Durrant, M. G., Ku, J., Poli, M., Brockman, G., Chang, D., Gonzalez, G. A., King, S. H., Li, D. B., Merchant, A. T., et al. Genome modeling and design across all domains of life with evo 2. bioRxiv, pp. 2025-02, 2025; which is incorporated by reference herein in its entirety.

[0447]

[0330] Camillo, L. P. d. L., Sehgal, R., Armstrong, J., Higgins-Chen, A. T., Horvath, S., and Wang, B. Cpgpt: a foundation model for dna methylation. bioRxiv, 2024; which is incorporated by reference herein in its entirety.AttyDktNo.: 49407-793601

[0448]

[0331] Carbonneau, M.-A., Cheplygina, V., Granger, E., and Gagnon, G. Multiple instance learning: A survey of problem characteristics and applications. Pattern recognition, 77:329-353, 2018; which is incorporated by reference herein in its entirety.

[0449]

[0332] Chung, D. C. et al. A cell-free DNA blood-based test for colorectal cancer screening. New England Journal of Medicine, 390(11):973-983, 2024. doi:

[0450] 10.1056 / NEJMoa2304714; which is incorporated by reference herein in its entirety.

[0451]

[0333] Cristiano, S., Leal, A., Phallen, J., et al. Genome-wide cell-free dna fragmentation in patients with cancer. Nature, 570(7761 ):385— 389, 2019; which is incorporated by reference herein in its entirety.

[0452]

[0334] Dalia-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Carranza, N. L., Grzywaczewski, A. H., Oteri, F., Dallago, C., Trop, E., de Almeida, B. P., Sirelkhatim, H., et al. Nucleotide transformer: building and evaluating robust foundation models for human genomics. Nature Methods, 22:287-297, 2025; which is incorporated by reference herein in its entirety.

[0453]

[0335] Deng, Z., Ji, Y., Han, B., Tan, Z., Ren, Y., Gao, J., Chen, N., Ma, C., Zhang, Y., Yao, Y., et al. Early detection of hepatocellular carcinoma via no end-repair enzymatic methylation sequencing of cell-free dna and pre-trained neural network. Genome Medicine, 15(1):93, 2023; which is incorporated by reference herein in its entirety.

[0454]

[0336] Eraslan, G., Avsec, Z., Gagneur, J., and Theis, F. J. Deep learning: new computational modelling techniques for genomics. Nature reviews genetics, 20 (7):389-403, 2019; which is incorporated by reference herein in its entirety.

[0455]

[0337] Gao, Q. et al. Circulating cell-free DNA for cancer early detection. The Innovation, 3(4): 100259, 2022; which is incorporated by reference herein in its entirety.

[0456]

[0338] Gupta, S. Screening for colorectal cancer. Hematology / Oncology Clinics of North America, 36(3):393-414, 2022; which is incorporated by reference herein in its entirety.

[0457]

[0339] He, K., Gan, C., Li, Z., Rekik, I., Yin, Z., Ji, W ., Gao, Y., Wang, Q., Zhang, J., and Shen, D. Transformers in medical image analysis. Intelligent Medicine, 3 (l):59-78, 2023; which is incorporated by reference herein in its entirety.

[0458]

[0340] Use, M., Tomczak, J., and Welling, M. Attention-based deep multiple instance learning. In International conference on machine learning, pp. 2127-2136. PMLR, 2018; which is incorporated by reference herein in its entirety.

[0459]

[0341] Jeong, Y., Gerh' auser, C., Sauter, G., Schlomm, T., Rohr, K., and Lutsik, P. Methylbert enables read-level dna methylation pattern identification and tu mour deconvolution using a transformer-based model. Nature Communications, 16(1):788, 2025; which is incorporated by reference herein in its entirety.AttyDktNo.: 49407-793601

[0460]

[0342] Ji, Y., Zhou, Z., Liu, H., and Davuluri, R. V. Dnabert: pre-trained bidirec tional encoder representations from transformers for dna-language model. Bioinformatics,

[0461] 37( 15):2112-2120, 2021; which is incorporated by reference herein in its entirety.

[0462]

[0343] Latif, S., Zaidi, A., Cuayahuitl, H., Shamshad, F., Shoukat, M., and Qadir, J. Transformers in speech processing: A survey. arXiv preprint ar Xiv: 2303.11607, 2023; which is incorporated by reference herein in its entirety.

[0463]

[0344] Levy, J. et al. Methylnet: an automated and modular deep learning approach for dna methylation analysis. BMC Bioinformatics, 2020; which is incorporated by reference herein in its entirety.

[0464]

[0345] Li, W., Li, Q., Kang, S., Same, M., Zhou, Y., Sun, C., Liu, C.-C., Matsuoka, L., Sher, L., Wong, W. H., et al. Cancerdetector: ultrasensitive and non invasive cancer detection at the resolution of individual reads using cell-free dna methylation sequencing data. Nucleic Acids Research, 46(15):e89, 2018; which is incorporated by reference herein in its entirety.

[0465]

[0346] Liang, J. T., Yang, C., and Myers, B. A. A large-scale survey on the usability of ai programming assistants: Successes and challenges. In Proceedings of the 46th IEEE / ACM international conference on software engineering, pp. 1-13, 2024; which is incorporated by reference herein in its entirety.

[0466]

[0347] Luo, H. et al. Liquid biopsy of methylation biomarkers in cell-free DNA. Trends in Molecular Medicine, 27(5):482-500, 2021; which is incorporated by reference herein in its entirety.

[0467]

[0348] Nerella, S., Bandyopadhyay, S., Zhang, J., Contreras, M., Siegel, S., Bumin, A., Silva, B., Sena, J., Shickel, B., Bihorac, A., et al. Transformers and large language models in healthcare: A review. Artificial intelligence in medicine, 154:102900, 2024; which is incorporated by reference herein in its entirety.

[0468]

[0349] Nguyen, E., Poli, M., Faizi, M., Thomas, A., Wornow, M., Birch-Sykes, C., Massaroli, S., Patel, A., Rabideau, C., Bengio, Y., et al. Hyenadna: Long range genomic sequence modeling at single nucleotide resolution. Advances in neural information processing systems, 36:43177-43201, 2023; which is incorporated by reference herein in its entirety.

[0469]

[0350] Niki, P., Nalmpantis, C., Ganbat, J.-O., Byrne, D., Babikir, H., Jhutty, A., Rowe, W., Liu, T., Loyfer, N., Toniolo, S., et al. Human whole epigenome modelling for clinical applications with pleiades. bioRxiv, pp. 2025-07, 2025; which is incorporated by reference herein in its entirety.

[0470]

[0351] Pawlowski, N., Bhooshan, S., Ballas, N., Ciompi, F., Glocker, B., and Drozdzal, M. Needles in haystacks: On classifying tiny objects in large images. arXiv preprint arXiv: 1908.06037, 2019; which is incorporated by reference herein in its entirety.AttyDktNo.: 49407-793601

[0471]

[0352] Ramsauer, H., Sch'afl, B., Lehner, J., Seidl, P., Widrich, M., Gruber, L., Hol zleitner, M., Pavlovi'c, M., Sandve, G. K., Greiff, V., Kreil, D., Kopp, M., Klambauer, G., Brandstetter, J., and Hochreiter, S. Hopfield networks is all you need. ArXiv, 2008.02217, 2020; which is incorporated by reference herein in its entirety.

[0472]

[0353] Shapiro, J. A. et al. Screening for colorectal cancer in the United States:

[0473] Correlates and time trends by type of test. Cancer Epidemiology, Biomarkers & Prevention, 30(8): 1554-1565, 2021; which is incorporated by reference herein in its entirety.

[0474]

[0354] Shaukat, A. and Levin, T. R. Current and future colorectal cancer screening strategies. Nature Reviews Gastroenterology & Hepatology, 19(8): 521—531, 2022; which is incorporated by reference herein in its entirety.

[0475]

[0355] Shaukat, A. et al. Clinical validation of a circulating tumor DNA-based blood test to screen for colorectal cancer. JAMA, 334, 2025; which is incorporated by reference herein in its entirety.

[0476]

[0356] Siegel, R. L., Wagle, N. S., Cercek, A., Smith, R. A., and Jemal, A. Colorectal cancer statistics, 2023. CA: A Cancer Journal for Clinicians, 73(3):233-254, 2023; which is incorporated by reference herein in its entirety.

[0477]

[0357] Siegel, R. L., Giaquinto, A. N., and Jemal, A. Cancer statistics, 2024. CA: A Cancer Journal for Clinicians, 74(1): 12-49, 2024; which is incorporated by reference herein in its entirety.

[0478]

[0358] Snyder, M. W. et al. Cell-free DNA comprises an in vivo nucleosome footprint that informs its tissues-of-origin. Cell, 164(l):57-68, 2016; which is incorporated by reference herein in its entirety.

[0479]

[0359] Stackpole, M. L., Zeng, W., Li, S., Liu, C.-C., Zhou, Y., He, S., Yeh, A., Wang, Z., Sun, F., Li, Q., et al. Cost-effective methylome sequencing of cell-free dna for accurately detecting and locating cancer. Nature Communications, 13(1): 5566, 2022. doi: 10.1038 / s41467- 022-32995-6; which is incorporated by reference herein in its entirety.

[0480]

[0360] Thierry, A. R. Circulating DNA fragmentomics and cancer screening. Cell Genomics, 3(1): 100242, 2023; which is incorporated by reference herein in its entirety.

[0481]

[0361] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017; which is incorporated by reference herein in its entirety.

[0482]

[0362] Widrich, M., Sch' afl, B., Pavlovi'c, M., Ramsauer, H., Gruber, L., Holzleitner, M., Brandstetter, J., Sandve, G. K., Greiff, V., Hochreiter, S., and Klambauer, G. Modern Hopfield networks and attention for immune repertoire classification. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information ProcessingAttyDktNo.: 49407-793601

[0483] Systems, volume 33, pp. 18832-18845. Curran Associates, Inc., 2020; which is incorporated by reference herein in its entirety.

[0484]

[0363] Yamaguchi, S. et al. High frequency of DAP-kinase gene promoter methylation in colorectal cancer specimens and its identification in serum. Cancer Letters, 194(1):99- 105, 2003; which is incorporated by reference herein in its entirety.

[0485]

[0364] Ying, K., Song, J., Cui, H., Zhang, Y., Li, S., Chen, X., Liu, H., Eames, A., McCartney, D. L., Marioni, R. E., et al. Methylgpt: a foundation model for the dna methylome. bioRxiv, 2024; which is incorporated by reference herein in its entirety.

[0486]

[0365] Cai, G. et al. A multilocus blood-based assay targeting circulating tumor DNA methylation enables early detection and early relapse prediction of colorectal cancer.

[0487] Gastroenterology, 161(6):2053-2056, 2021; which is incorporated by reference herein in its entirety.

[0488]

[0366] Didion, J. P., Martin, M., and Collins, F. S. Atropos: specific, sensitive, and speedy trimming of sequencing reads. PeerJ, 5:e3720, 2017; which is incorporated by reference herein in its entirety.

[0489]

[0367] Foulds, J. and Frank, E. A review of multi-instance learning assumptions. The Knowledge Engineering Review , 25(1): 1-25, 2010; which is incorporated by reference herein in its entirety.

[0490]

[0368] Furst, A., Rumetshofer, E., Lehner, J., Tran, V. T., Tang, F., Ramsauer, H., Kreil, D., Kopp, M., Klambauer, G., Bitto, A., et al. Cloob: Modem hopfield networks with infoloob outperform clip. Advances in neural information processing systems, 35:20450-20468, 2022; which is incorporated by reference herein in its entirety.

[0491]

[0369] Gao, Q. et al. Unintrusive multi-cancer detection by circulating cell-free DNA methylation sequencing (THUNDER): development and independent validation studies. Annals of Oncology, 34(5):486-495, 2023; which is incorporated by reference herein in its entirety.

[0492]

[0370] Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79 (8):2554-2558, 1982; which is incorporated by reference herein in its entirety.

[0493]

[0371] Hopfield, J. J. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences, 81(10):3088-3092, 1984. doi: 10.1073 / pnas.81.10.3088; which is incorporated by reference herein in its entirety.

[0494]

[0372] Hu, J. Y.-C., Yang, D., Wu, D., Xu, C., Chen, B.-Y., and Liu, H. On sparse modern hopfield model. Advances in neural information processing systems, 36:27594-27608, 2023; which is incorporated by reference herein in its entirety.AttyDktNo.: 49407-793601

[0495]

[0373] Kerns, E. V. and Weber, J. N. Variable performance of widely used bisulfite sequencing methods and read mapping software for dna methylation. bioRxiv, pp.

[0496] 2025.03.14.643302, 2025. doi: 10.1101 / 2025.03.14.643302; which is incorporated by reference herein in its entirety.

[0497]

[0374] Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S. Self-normalizing neural networks. In Advances in Neural Information Processing Systems, pp. 971-980, 2017; which is incorporated by reference herein in its entirety.

[0498]

[0375] Liang, N. et al. Ultrasensitive detection of circulating tumour DNA via deep methylation sequencing aided by machine learning. Nature Biomedical Engi neering, 5(6):586- 599, 2021; which is incorporated by reference herein in its entirety.

[0499]

[0376] Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017; which is incorporated by reference herein in its entirety.

[0500]

[0377] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: an imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, pp. 8024-8035, 2019; which is incorporated by reference herein in its entirety.

[0501]

[0378] Pedersen, B. S., Eyring, K., De, S., Yang, I. V., and Schwartz, D. A. Fast and accurate alignment of long bisulfite-seq reads. arXiv preprint arXiv: 1401.1129, 2014. doi: 10.48550 / arxiv.1401.1129; which is incorporated by reference herein in its entirety.

[0502]

[0379] Plongthongkum, N., Diep, D. H., and Zhang, K. Advances in the profiling of DNA modifications: cytosine methylation and beyond. Nature Reviews Genetics, 15(10):647- 661, 2014; which is incorporated by reference herein in its entirety.

[0503]

[0380] Ramsauer, H., Sch'afl, B., Lehner, J., Seidl, P., Widrich, M., Gruber, L., Hol zleitner, M., Pavlovi'c, M., Sandve, G. K., Greiff, V., Kreil, D., Kopp, M., Klambauer, G., Brandstetter, J., and Hochreiter, S. Hopfield networks is all you need. ArXiv, 2008.02217, 2020; which is incorporated by reference herein in its entirety.

[0504]

[0381] Sch' afl, B., Gruber, L., Bitto-Nemling, A., and Hochreiter, S. Hopular: Modem hopfield networks for tabular data. arXiv preprint arXiv:2206.00664, 2022; which is incorporated by reference herein in its entirety.

[0505]

[0382] Seidl, P., Renz, P., Dyubankova, N., Neves, P., Verhoeven, J., Wegner, J. K., Hochreiter, S., and Klambauer, G. Modern hopfield networks for few- and zero-shot reaction prediction. ArXiv, 2104.03279, 2021; which is incorporated by reference herein in its entirety.

[0506]

[0383] Shen, S. Y. et al. Sensitive tumour detection and classification using plasma cell-free DNA methylomes. Nature, 563(7732):579-583, 2018; which is incorporated by reference herein in its entirety.AttyDktNo.: 49407-793601

[0507]

[0384] Shen, S. Y. et al. Preparation of cfMeDIP-seq libraries for methylome profiling of plasma cell-free DNA. Nature Protocols, 14(10):2749-2780, 2019; which is incorporated by reference herein in its entirety.

[0508]

[0385] Widrich, M., Hofmarcher, M., Patil, V. P., Bitto-Nemling, A., and Hochreiter, S. Modern hopfield networks for return decomposition for delayed rewards. In Deep RL Workshop NeurlPS 2021, 2021; which is incorporated by reference herein in its entirety.

[0509]

[0386] Cui, H., Wang, C., Maan, H. et al. scGPT: toward building a foundation model for single-cell multi -omics using generative Al. Nat Methods 21, 1470-1480 (2024). https: / / doi.org / 10.1038 / s41592-024-02201-0; which is incorporated by reference herein in its entirety.

[0510]

[0387] Wen Tai, H. T. Kung, Xin Dong, Marcus Comiter, and Chang-Fu Kuo. 2020. exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1433-1439, Online. Association for Computational Linguistics; which is incorporated by reference herein in its entirety.

[0511]

[0388] Mclnnes, L., Healy, J., and Melville, J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv: 1802.03426, 2018; which is incorporated by reference herein in its entirety.

[0512]

[0389] Widman, A. J., Shah, M., Frydendahl, A. et al. Ultrasensitive plasma-based monitoring of tumor burden using machine-leaming-guided signal enrichment. Nat Med 30, 1655-1666 (2024). https: / / doi.org / 10.1038 / s41591-024-03040-4; which is incorporated by reference herein in its entirety.

[0513]

[0390] Hu, E.J., Shen, Y., Wallis P., et al. LoRA: Lor-Rank Adaptation of Large Language Models. International Conference on Learing Representations; which is incorporated by reference herein in its entirety.

[0514]

[0391] Mayr, A., Klambauer, G., Unterthiner, T., & Hochreiter, S. (2016). DeepTox: Toxicity prediction using Deep Learning. Frontiers in Environmental Science, 3. https: / / doi.org / 10.3389 / fenvs.2015.00080; which is incorporated by reference herein in its entirety.

[0515]

[0392] Courant, R. Variational methods for the solution of problems of equilibrium and vibrations. Bulletin of the American Mathematical Society, pp. 1-23, 1941; which is incorporated by reference herein in its entirety.

[0516]

[0393] Bandi, P., Star, J., Mazzitelli, N., Nargis, N., Islami, F., Siegel, R. L., Yabroff, K. R., and Jemal, A. Prevalence and review of major modifiable cancer risk factors, hpv vaccination, and cancer screenings in the united states: 2025 update. Cancer Epidemiology,AttyDktNo.: 49407-793601

[0517] Biomarkers & Prevention, 34(6):836-849, 2025; which is incorporated by reference herein in its entirety.

[0518]

[0394] Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020). Linformer: Selfattention with linear complexity. arXiv preprint arXiv:2006.04768; which is incorporated by reference herein in its entirety.

[0519]

[0395] Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., ... & Weller, A. (2020). Rethinking attention with performers. arXiv preprint arXiv:2009.14794; which is incorporated by reference herein in its entirety.

[0520]

[0396] While preferred embodiments of the present inventive concepts have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the inventive concepts be limited by the specific examples provided within the specification. While the inventive concepts have been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the inventive concepts. Furthermore, it shall be understood that all aspects of the inventive concepts are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the inventive concepts described herein may be employed in practicing the inventive concepts. It is therefore contemplated that the inventive concepts shall also cover any such alternatives, modifications, variations, or equivalents. It is intended that the following claims define the scope of the inventive concepts and that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims

AttyDktNo.: 49407-793601CLAIMS WHAT IS CLAIMED IS:

1. A method for training a machine learning model, the method comprising:(a) receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from a subject, wherein the multimodal data comprises two or more data modalities selected from (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data;(b) generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality of nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment; and(c) using the fragment representation to train an attention-based sub-network, wherein the attention-based sub-network is trained to generate a sample representation of a test biological sample, based at least in part on analyzing fragment representations of the test biological sample.

2. The method of claim 1, wherein the biological sample comprises a cell-free sample.

3. The method of claim 2, wherein the cell-free sample comprises a plasma sample, a serum sample, a urine sample, a saliva sample, a cerebrospinal fluid (CSF) sample, a fecal sample, or any combination thereof.

4. The method of any one of claims 1 to 3, wherein the plurality of nucleic acid fragments comprises deoxyribonucleic acid (DNA) fragments or ribonucleic acid (RNA) fragments, or a combination thereof.

5. The method of any one of claims 1 to 4, further comprising, prior to (a), assaying the biological sample to generate the multimodal data.

6. The method of claim 5, wherein the assaying the biological sample comprises sequencing the plurality of nucleic acid fragments or derivatives thereof to obtain a plurality of sequencing reads.

7. The method of claim 6, further comprising generating the sequence data based on computer processing the plurality of sequencing reads to generate consensus fragments for the plurality of nucleic acid fragments.

8. The method of claim 7, further comprising generating the foundational model representations using a foundational model, based at least in part on the sequence data.

9. The method of claim 8, wherein the foundational model is pre-trained on a human reference genome.AttyDktNo.: 49407-79360110. The method of any one of claims 6 to 8, further comprising generating the genomic position data, based at least in part on performing an alignment of the plurality of sequencing reads to a human reference genome.

11. The method of any one of claims 6 to 10, further comprising generating the strand data, based at least in part on determining whether a sequence read of the plurality of sequencing reads is derived from a forward strand or a reverse strand of a corresponding nucleic acid fragment.

12. The method of claim 11, wherein the strand data comprises a boolean indicator indicative of a source strand of the plurality of nucleic acid fragments.

13. The method of any one of claims 5 to 12, wherein the assaying the biological sample comprises performing methyl conversion on the plurality of nucleic acid fragments to obtain the CpG methylation data or the methylation statistics data.

14. The method of claim 13, wherein the methyl conversion comprises bisulfite conversion or enzymatic conversion.

15. The method of claim 14, wherein the methyl conversion comprises the enzymatic conversion.

16. The method of claim 14, wherein the methyl conversion comprises the bisulfite conversion.

17. The method of any one of claims 13 to 16, wherein the CpG methylation data comprises a per-CpG methylation status for the plurality of nucleic acid fragments.

18. The method of any one of claims 13 to 17, wherein the per-CpG methylation status is represented using a one-hot encoded matrix.

19. The method of any one of claims 13 to 18, wherein the methylation statistics data comprises sequence length, length-normalized methylated CpG count, length-normalized unmethylated CpG count, length-normalized total CpG count, or any combination thereof.

20. The method of any one of claims 1 to 19, further comprising using the multimodal data to train the encoding sub-network.

21. The method of any one of claims 1 to 20, wherein generating the fragment representation in (b) comprises generating a latent representation of a data modality of the two or more data modalities using a corresponding encoding model.

22. The method of claim 21, wherein the corresponding encoding model comprises a convolutional neural network, a triangle encoder, a multi-layer perceptron, a long shortterm memory (LSTM), a recurrent neural network (RNN), or any combination thereof.AttyDktNo.: 49407-79360123. The method of claim 21 or 22, wherein generating the fragment representation in (b) comprises processing the latent representation of the data modality of the two or more data modalities using a multi-layer perceptron (MLP).

24. The method of any one of claims 1 to 23, wherein generating the fragment representation in (b) is performed at least in part using a pre-trained methylation-aware foundational model.

25. The method of any one of claims 1 to 24, wherein the attention-based sub-network comprises a Modem Hopfield Network (MHN).

26. The method of any one of claims 1 to 25, further comprising using the attention-based sub-network to project the fragment representation to a high-dimensional association space.

27. The method of claim 26, wherein a plurality of fragment representations corresponding to the plurality of nucleic acid fragments are projected to a same high-dimensional association space.

28. The method of claim 26 or 27, further comprising using the attention-based sub-network to perform dense associative retrieval over the high-dimensional association space.

29. The method of any one of claims 26 to 28, wherein training the attention-based subnetwork comprises determining a set of state patterns in the high-dimensional association space.

30. The method of claim 29, wherein training the attention-based sub-network comprises initializing the set of state patterns.

31. The method of claim 29 or 30, wherein training the attention-based sub-network comprises determining a value of a performance metric associated with the set of state patterns.

32. The method of claim 31, wherein the performance metric comprises a binary cross entropy.

33. The method of claim 31 or 32, wherein training the attention-based sub-network comprises updating the set of state patterns, based at least in part on the value of the performance metric.

34. The method of any one of claims 1 to 33, further comprising using the attention-based sub-network to generate a sample representation of the biological sample, based at least in part on analyzing the fragment representation.

35. The method of claim 34, wherein using the attention-based sub-network comprises projecting the fragment representation to the high-dimensional association space, thereby obtaining a projected fragment representation.AttyDktNo.: 49407-79360136. The method of claim 35, wherein using the attention-based sub-network comprises identifying an attention weight associated with the fragment representation, based at least in part on a set of learned state patterns.

37. The method of claim 36, wherein identifying the attention weight comprises determining a dot product between the set of learned state patterns and the projected fragment representation.

38. The method of claim 36 or 37, wherein the sample representation of the biological sample is generated based at least in part on the attention weight.

39. The method of any one of claims 36 to 38, wherein a complexity of the attention weight is proportional to a size of the plurality of fragments.

40. The method of any one of claims 36 to 39, wherein the attention weight is indicative of an importance of the nucleic acid fragment to a prediction task.

41. The method of any one of claims 34 to 39, further comprising performing dimensionality reduction on the sample representation of the biological sample.

42. The method of claim 41, wherein the dimensionality reduction comprises Uniform Manifold Approximation and Projection (UMAP).

43. The method of claim 41 or 42, further comprising identifying one or more clusters, based at least in part on the dimensionality reduction.

44. The method of any one of claims 34 to 43, further comprising identifying, using the attention-based sub-network, one or more nucleic acid fragments associated with high attention weights.

45. The method of claim 44, further comprising partitioning the one or more nucleic acid fragments associated with high attention weights into one or more genomic bins.

46. The method of any one of claims 34 to 45, further comprising using the sample representation of the biological sample to train a predictive sub-network, wherein the predictive sub-network is trained to predict a health status of the test subject, based at least in part on analyzing the sample representation of the test subject.

47. The method of claim 46, wherein the health status comprises a presence or absence of a disease, a clinical metadata, a disease stage, a disease subtype, a lesion size, or a biological characteristic of the subject.

48. The method of claim 47, wherein the disease comprises a cancer.

49. The method of claim 48, wherein the cancer comprises colorectal cancer, lung cancer, pancreatic cancer, gastric cancer, liver cancer, ovarian cancer, breast cancer, head or neck cancer, or any combination thereof.

50. The method of claim 49, wherein the cancer comprises the colorectal cancer.AttyDktNo.: 49407-79360151. The method of any one of claims 46 to 50, wherein the predictive sub-network comprises an ensemble model.

52. The method of claim 51, wherein the ensemble model comprises a plurality of classification models obtained from a plurality of cross validation folds.

53. The method of claim 52, wherein the plurality of classification models comprises a MLP, a random forest model, a logistic regression model, a support vector machine (SVM), or any combination thereof.

54. The method of any one of claims 46 to 53, wherein training the predictive sub-network comprises initializing a set of model weights of the predictive sub-network.

55. The method of claim 54, wherein training the predictive sub-network comprises determining a value of a performance metric associated with the set of model weights.

56. The method of claim 55, wherein the performance metric comprises a sensitivity of the second machine learning sub-network at a pre-determined specificity value or an area under the receiver operating characteristic (AUROC).

57. The method of claim 56, wherein the pre-determined specificity value is at least or at most about 70%, 80%, 90%, or 95%.

58. The method of any one of claims 55 to 57, wherein training the predictive sub-network comprises updating the set of model weights, based at least in part on the value of the performance metric.

59. The method of any one of claims 55 to 58, wherein training the predictive sub-network comprises updating one or more model hyperparameters, based at least in part on the value of the performance metric.

60. The method of any one of claims 46 to 59, wherein training the predictive sub-network comprises utilizing a dynamic minibatch balancing scheme to reduce an effect of one or more confounding variables on the prediction of the health status.

61. The method of any one of claims 46 to 60, wherein the encoding sub-network, the attention-based sub-network, and the predictive sub-network are trained end-to-end.

62. The method of any one of claims 1 to 61, further comprising determining a panel of genomic regions of interest.

63. The method of claim 62, wherein determining the panel of genomic regions of interest comprises determining one or more genomic regions of a target capture panel associated with the health status.

64. The method of claim 63, wherein determining the panel of genomic regions of interest comprises filtering the one or more genomic regions to reduce a background noise.AttyDktNo.: 49407-79360165. The method of any one of claims 62 to 64, wherein the encoding sub-network or the attention-based sub-network are trained based at least in part on panel of genomic regions of interest.

66. The method of claim 65, wherein the encoding sub-network or the attention-based subnetwork are trained using a subset of the plurality of nucleic acid fragments corresponding to the panel of genomic regions of interest.

67. A method for training a machine learning model for predicting a presence or an absence of a disease, the method comprising:(a) receiving training multimodal data generated by assaying a plurality of cell-free deoxyribonucleic acid (cfDNA) fragments in a cell-free sample obtained or derived from a training subject, wherein the multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data;(b) using the training multimodal data to train a encoding sub-network of the machine learning model, wherein the encoding sub-network is trained to generate fragment representations of cfDNA fragments in a test cell-free sample, based on analyzing multimodal data of the test cell-free sample, wherein the encoding sub-network comprises a convolutional neural network (CNN), a multi-layer perceptron (MLP), a long short-term memory (LSTM), or a recurrent neural network (RNN);(c) using the encoding sub-network to generate a plurality of fragment representations, based on analyzing the training multimodal data;(d) using the plurality of fragment representations to train a Modem Hopfield Network (MHN), wherein the MHN is trained to generate a sample representation of the test cell- free sample, based on weighting and aggregating the fragment representations of the cfDNA fragments in the test cell-free sample;(e) using the MHN to generate a sample representation of the training cell-free sample, based on the plurality of fragment representations; and(f) using the sample representation of the training cell-free sample to train a predictive subnetwork of the machine learning model to predict the presence or the absence of the disease in the test subject, based on analyzing the sample representation of the test subject, wherein the predictive sub-network comprises an ensemble model.

68. A method for predicting a health status of a subject, the method comprising:(a) receiving multimodal data generated by assaying a plurality of nucleic acid fragments in a biological sample obtained or derived from the subject, wherein the multimodal dataAttyDktNo.: 49407-793601comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data;(b) generating, using an encoding sub-network, a fragment representation of a nucleic acid fragment of the plurality of nucleic acid fragments, based at least in part on analyzing multimodal data corresponding to the nucleic acid fragment;(c) generating, using an attention-based sub-network, a sample representation of the biological sample, based at least in part on analyzing the fragment representation; and (d) predicting, using a predictive sub-network, the health status of the subject, based at least in part on analyzing the sample representation.

69. The method of claim 68, wherein the health status comprises a presence or an absence of a disease, a clinical metadata, or a biological characteristic of the subject.

70. The method of claim 69, wherein the disease comprises a cancer.

71. The method of claim 70, wherein the cancer comprises colorectal cancer, lung cancer, pancreatic cancer, gastric cancer, liver cancer, ovarian cancer, breast cancer, head or neck cancer, or any combination thereof.

72. The method of claim 71, wherein the cancer comprises the colorectal cancer.

73. The method of any one of claims 69 to 72, further comprising predicting the presence of the disease.

74. The method of claim 73, further comprising, in response to predicting the presence of the disease, administering a treatment for treating the disease.

75. The method of claim 74, wherein the treatment comprises a surgery, a colonoscopy, a radiation therapy, an immunotherapy, a chemotherapy, or a low-dose computed tomography (LDCT).

76. A computer system, the system comprising:(a) one or more computer processors; and(b) a computer memory storing machine executable code that, upon execution by the one or more computer processors, implements the method of any one of claims 1 to 75.

77. A non-transitory computer-readable medium storing machine executable code that, upon execution by one or more computer processors, implements the method of any one of claims 1 to 75.

78. A system for predicting a health status of a subject, the system comprising:(i) an encoding sub-network configured to generate a fragment representation of a nucleic acid fragment in a biological sample obtained or derived from the subject, based at least in part on multimodal data of the nucleic acid fragment, wherein theAttyDktNo.: 49407-793601multimodal data comprises two or more data modalities selected from: (i) sequence data, (ii) CpG methylation data, (iii) methylation statistics data, (iv) foundational model representations, (v) genomic position data, and (vi) strand data;(ii) an attention-based sub-network configured to generate a sample representation of the biological sample, based at least in part on analyzing the fragment representation of the nucleic acid fragment; and(iii) a predictive sub-network configured to predict the health status of the subject, based at least in part on analyzing the sample representation.

Images