Using noise characteristics to improve nucleotide sequence identification

Abstract

In some embodiments, a computer-implemented method of determining a pattern represented by sensor signal data is provided. A computing system receives sensor signal data. The computing system filters the sensor signal data using a first filter to generate a first filtered signal. The computing system also filters the sensor signal data using a second filter to generate a second filtered signal. The computing system provides the first filtered signal as a first input to a first machine learning model to generate a first pattern result as output. The computing system provides the second filtered signal as a second input to a second machine learning model to generate a second pattern result as output. The computing system combines the first pattern result and the second pattern result to generate an output pattern result.

Description

Docket No. 3915-P1363WO.UWUSING NOISE CHARACTERISTICS TO IMPROVE NUCLEOTIDE SEQUENCE IDENTIFICATIONCROSS REFERENCE(S) TO RELATED APPLICATION(S)

[0001] This application claims the benefit of Provisional Application No. 63 / 709313, filed October 18, 2024, the entire disclosure of which is hereby incorporated by reference herein for all purposes.STATEMENT OF GOVERNMENT LICENSE RIGHTS

[0002] This invention was made with Government support under Grant Nos. 1R21CA280520-01 and 3R21CA280520-01S1, awarded by the National Institutes of Health. The Government has certain rights in the invention.BACKGROUND

[0003] Increasing numbers of important technologies that involve the detection of discrete patterns from signal data are being developed. One particular area in which these techniques are increasingly needed is sequencing, where various technologies are used that generate signals from which the identity of amino acids or nucleotides in a sequence can be determined. As one example, the advancement of nanopore sequencing technology has revolutionized genomic research. However, the need for high-accuracy, high-throughput, real-time detection of patterns such as nucleotide bases from electrical signals on low- resource devices remains a significant challenge.

[0004] Techniques that can provide high-quality detection of patterns from signal data such as nanopore sequencing data, even on relatively weak computing hardware such as smartphones and other mobile computing devices, would help improve access to this technology. Certain machine learning model architectures are capable of processing this information on weak computing hardware, but given the signal noise and context-related variations, these model architectures tend to provide poor accuracy. What is desired are techniques that improve the accuracy of pattern detection in noisy data that can be executed using relatively weak computing hardware such as mobile computing devices.SUMMARY

[0005] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intendedDocket No. 3915-P1363WO.UW to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0006] In some embodiments, a computer-implemented method of determining a pattern represented by sensor signal data is provided. A computing system receives sensor signal data. The computing system filters the sensor signal data using a first filter to generate a first filtered signal. The computing system also filters the sensor signal data using a second filter to generate a second filtered signal. The computing system provides the first filtered signal as a first input to a first machine learning model to generate a first pattern result as output. The computing system provides the second filtered signal as a second input to a second machine learning model to generate a second pattern result as output. The computing system combines the first pattern result and the second pattern result to generate an output pattern result.

[0007] In some embodiments, a computer-implemented method of determining a sequence represented by nanopore signal data is provided. A computing system receives a raw nanopore signal, and generates an encoding of the raw nanopore signal using a transformer model. The computing system compares the encoding to one or more cached signal patterns associated with known sequences. In response to determining a high-confidence match between the encoding and a cached signal pattern, the computing system returns the known sequence associated with the cached signal pattern as an output sequence. In response to determining no high-confidence match between the encoding and a cached signal pattern, the computing system determines the output sequence using a method as described above for determining a pattern represented by sensor signal data.

[0008] In some embodiments, a non-transitory computer-readable medium having computer-executable instructions stored thereon is provided. The instructions, in response to execution by one or more processors of a computing system, cause the computing system to perform a method as described above.

[0009] In some embodiments, a computing system is provided that comprises at least one processor and a non-transitory computer-readable medium. The non-transitory computer- readable medium has computer-executable instructions stored thereon that, in response to execution by the at least one processor, cause the computing system to perform a method as described above.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to theDocket No. 3915-P1363WO.UW following detailed description, when taken in conjunction with the accompanying drawings, wherein:

[0011] FIG. l is a block diagram that illustrates a generic system for identifying patterns in sensor signal data, according to various aspects of the present disclosure.

[0012] FIG. 2 is a schematic illustration of a system for nanopore-based analysis, as a nonlimiting example of a specific type of system suitable for use as the system illustrated in FIG. 1.

[0013] FIG. 3 is a schematic illustration of a non-limiting example embodiment of a flow cell according to various aspects of the present disclosure.

[0014] FIG. 4 is a block diagram that illustrates aspects of a non-limiting example embodiment of a pattern identification computing system according to various aspects of the present disclosure.

[0015] FIG. 5 is a schematic illustration of a non-limiting example embodiment of a workflow for processing sensor signal data according to various aspects of the present disclosure.

[0016] FIG. 6A and FIG. 6B illustrate a non-limiting example of a model architecture suitable for use as the first machine learning model and / or the second machine learning model of FIG. 5, according to various aspects of the present disclosure.

[0017] FIG. 7A and FIG. 7B illustrate another non-limiting example of a model architecture suitable for use as the first machine learning model and / or the second machine learning model of FIG. 5, according to various aspects of the present disclosure.

[0018] FIG. 8 is a flowchart that illustrates a non-limiting example embodiment of a method of determining a pattern represented by sensor signal data, according to various aspects of the present disclosure.

[0019] FIG. 9 is a flowchart that illustrates a non-limiting example embodiment of a method of determining a pattern represented by sensor signal data using caching, according to various aspects of the present disclosure.DETAILED DESCRIPTION

[0020] In the present disclosure, the detection of patterns in sensor signal data is improved by applying multiple different filters to the received sensor signal data, detecting patterns in the differentially filtered sensor signal data, and then comparing the detected patterns to determine a consensus result. By applying different filters to multiple copies of the sensor signal data (e.g., different levels of noise reduction), long-term dependencies and short-termDocket No. 3915-P1363WO.UW dependencies within the sensor signal data can be captured more accurately, and so even model architectures capable of being executed by relatively weak computing systems and capable of providing results in real-time while sensor signal data is being received can nevertheless produce accurate results.

[0021] FIG. l is a block diagram that illustrates a generic system for identifying patterns in sensor signal data, according to various aspects of the present disclosure. As shown, the system 100 includes a signal generation system 102 and a pattern identification computing system 104. The signal generation system 102 may be any system that generates sensor signal data 106 from which patterns can be detected. In some embodiments, sensor signal data 106 is a time series of data generated by one or more sensors that indicates a series of states over time. One non-limiting example embodiment of a signal generation system 102 is a protein sequencing device, wherein the sensor signal data 106, when processed, indicates a series of amino acids that constitute a protein. An example of a system 100 that utilizes a protein sequencing device (specifically, a nanopore sequencing device) is discussed in further detail below. However, this example should not be seen as limiting, and in other embodiments, other types of signal generation systems 102 may be used and other types of patterns may be detected. For example, the signal generation system 102 may be an electrocardiogram (ECG) system or photoplethysmograph (PPG) system, and the patterns detected may include heartbeat characteristics (e.g., heart rate, inter-beat interval, arrhythmia vs normal rhythms, etc.). As another example, the signal generation system 102 may be an electroencephalograph (EEG) system, and the patterns detected may include wave types or abnormal wave patterns.

[0022] FIG. 2 is a schematic illustration of a system for nanopore-based analysis, as a nonlimiting example of a specific type of system suitable for use as the system illustrated in FIG. 1. The system 200 is a non-limiting example of specific type of system suitable for use as the system 100 illustrated in FIG. 1. As shown, in the system 200, a sample 206 is obtained from a subject 202 using known techniques. The sample 206 may be a tissue biopsy, a swab, a blood sample, or any other suitable type of sample 206. The sample 206 is prepared (e.g., combined with one or more buffers, enzymes, etc.), and the prepared sample 206 is provided to a flow cell 204 of a sequencing device. One non-limiting example of a sequencing device is a MinlON sequencing device provided by Oxford Nanopore Technologies pic. Some non-limiting examples of devices for implementing a flow cell 204 are a Flongle Flow Cell, a MinlON Flow Cell, and the PromethlON Flow Cell, each also provided by Oxford Nanopore Technologies pic. The flow cell 204 generates signals based on interactions between the sample 206 and the nanopores of the flow cellDocket No. 3915-P1363WO.UW204, and provides the signals to the pattern identification computing system 104 for analysis. In the system 200, the pattern identification performed by the pattern identification computing system 104 includes conducting basecalling on raw nanopore data (e.g., “squiggles”) provided by the flow cell 204.

[0023] FIG. 3 is a schematic illustration of a non-limiting example embodiment of a flow cell according to various aspects of the present disclosure. As shown, the flow cell 204 includes a sample well 304, a plurality of nanopores 302, a processor 306, and a communication interface 308. The sample well 304 is configured to accept the sample 206 (e.g., to receive drops of sample 206 from a pipette) and to provide the sample 206 to the plurality of nanopores 302. The processor 306 is configured to control a voltage applied to the plurality of nanopores 302 and to read signals generated by the nanopores 302. In some embodiments, the processor 306 may also be configured to segment the signals generated by the nanopores 302 into a plurality of segmented events, each segmented event representing an interaction of a molecule with a nanopore 302 of the plurality of nanopores 302. In some embodiments, the communication interface 308 is configured to transmit the signals detected by the processor 306 and / or the segmented events to another device, such as the pattern identification computing system 104, using a wired or wireless network, a USB connection, or any other suitable communication technique. In some embodiments, the processor 306, communication interface 308, and potentially other components (such as a computer-readable medium) may be implemented on an ASIC or FPGA that is part of the flow cell 204.

[0024] FIG. 4 is a block diagram that illustrates aspects of a non-limiting example embodiment of a pattern identification computing system according to various aspects of the present disclosure. The illustrated pattern identification computing system 104 may be implemented by any computing device or collection of computing devices, including but not limited to a desktop computing device, a laptop computing device, a mobile computing device, a server computing device, a computing device of a cloud computing system, and / or combinations thereof. In some embodiments, the pattern detection techniques implemented by the pattern identification computing system 104 may be adapted to execute particularly efficiently using limited hardware resources, such that the pattern identification computing system 104 may be implemented using a smartphone, a tablet computing device, or another low-powered computing device.

[0025] As shown, the pattern identification computing system 104 includes one or more processors 402, one or more communication interfaces 404, a model data store 408, and a computer-readable medium 406.Docket No. 3915-P1363WO.UW

[0026] In some embodiments, the processors 402 may include any suitable type of general- purpose computer processor. In some embodiments, the processors 402 may include one or more special-purpose computer processors or Al accelerators optimized for specific computing tasks, including but not limited to graphical processing units (GPUs), vision processing units (VPUs), and tensor processing units (TPUs).

[0027] In some embodiments, the communication interfaces 404 include one or more hardware and or software interfaces suitable for providing communication links between components. The communication interfaces 404 may support one or more wired communication technologies (including but not limited to Ethernet, FireWire, and USB), one or more wireless communication technologies (including but not limited to Wi-Fi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and / or combinations thereof.

[0028] As shown, the computer-readable medium 406 has stored thereon logic that, in response to execution by the one or more processors 402, cause the pattern identification computing system 104 to provide a data collection engine 412, a noise processing engine 414, and a pattern identification engine 410.

[0029] As used herein, "computer-readable medium" refers to a removable or nonremovable device that implements any technology capable of storing information in a volatile or non-volatile manner to be read by a processor of a computing device, including but not limited to: a hard drive; a flash memory; a solid state drive; random-access memory (RAM); read-only memory (ROM); a CD-ROM, a DVD, or other disk storage; a magnetic cassette; a magnetic tape; and a magnetic disk storage.

[0030] In some embodiments, the data collection engine 412 is configured to receive the sensor signal data 106 from the signal generation system 102. In some embodiments, the data collection engine 412 may also preprocess the sensor signal data 106 to prepare it for further analysis. In some embodiments, the noise processing engine 414 is configured to apply multiple filters to the received sensor signal data 106, in order to generate versions of the received sensor signal data 106 with different levels of noise. In some embodiments, the pattern identification engine 410 is configured to load one or more machine learning models from the model data store 408, and to use the machine learning models to detect patterns in the filtered sensor signal data by comparing results after different filters are applied.

[0031] Further description of the configuration of each of these components is provided below.

[0032] As used herein, "engine" refers to logic embodied in hardware or software instructions, which can be written in one or more programming languages, including but notDocket No. 3915-P1363WO.UW limited to C, C++, C#, COBOL, JAVA™, PHP, Perl, HTML, CSS, JavaScript, VBScript, ASPX, Go, and Python. An engine may be compiled into executable programs or written in interpreted programming languages. Software engines may be callable from other engines or from themselves. Generally, the engines described herein refer to logical modules that can be merged with other engines, or can be divided into sub-engines. The engines can be implemented by logic stored in any type of computer-readable medium or computer storage device and be stored on and executed by one or more general purpose computers, thus creating a special-purpose computer configured to provide the engine or the functionality thereof. The engines can be implemented by logic programmed into an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or another hardware device.

[0033] As used herein, "data store" refers to any suitable device configured to store data for access by a computing device. One example of a data store is a highly reliable, highspeed relational database management system (DBMS) executing on one or more computing devices and accessible over a high-speed network. Another example of a data store is a keyvalue store. However, any other suitable storage technique and / or device capable of quickly and reliably providing the stored data in response to queries may be used, and the computing device may be accessible locally instead of over a network, or may be provided as a cloud-based service. A data store may also include data stored in an organized manner on a computer-readable storage medium, such as a hard disk drive, a flash memory, RAM, ROM, or any other type of computer-readable storage medium. One of ordinary skill in the art will recognize that separate data stores described herein may be combined into a single data store, and / or a single data store described herein may be separated into multiple data stores, without departing from the scope of the present disclosure.

[0034] FIG. 5 is a schematic illustration of a non-limiting example embodiment of a workflow for processing sensor signal data according to various aspects of the present disclosure. The workflow 500 begins when sensor signal data 106 is received and undergoes optional preprocessing 502. The preprocessing 502 may include normalizing, correction for baseline shifts, and / or other preprocessing actions suitable to the received sensor signal data 106. For nanopore squiggle data, the preprocessing 502 may include one or more of performing signal alignment to align raw electrical signals with a reference genome (e.g., using the “re-squiggle” technique from the Tombo framework), normalization using per-read mean and variance scaling, or chunking long signals into fixed-length windows (e.g., windows of a configurable length between 400-4000 timesteps).Docket No. 3915-P1363WO.UW

[0035] The output of the preprocessing 502 actions is then provided to a first filter 504 and a second filter 506 to generate versions of the preprocessed sensor signal data 106 that have different characteristics. In some embodiments, the first filter 504 and the second filter 506 are noise filters that are configured to remove different amounts of noise. As a non-limiting example, the first filter 504 and second filter 506 may be Butterworth filters whose denoising capabilities may be controlled by providing an input parameter, such that the first filter 504 is configured to remove more or less noise than the second filter 506. By allowing more noise to pass through the first filter 504 than the second filter 506 (or vice versa), long-term dependencies and short-term dependencies can be captured more accurately, thus resulting in a more accurate pattern detection. Noise-removal filters are a non-limiting example of the processing that may be performed by the first filter 504 and second filter 506. In other embodiments, other types of processing might be performed by the first filter 504 and the second filter 506, including but not limited to injecting additional noise.

[0036] After the filtering, the output of the first filter 504 is provided to a first machine learning model 508, and the output of the second filter 506 is provided to a second machine learning model 510, each of which are configured to detect patterns (e.g., nucleotide sequences, etc.) in their input and to provide the detected patterns as output. The output of the first machine learning model 508 and the second machine learning model 510 may also include confidence scores or other data related to the predictions. In some embodiments, the first machine learning model 508 and the second machine learning model 510 may have matching architectures, and / or may be trained on the same training data set. In some embodiments, the first machine learning model 508 may be trained using a training data set having been filtered by the first filter 504, while the second machine learning model 510 may be trained using a training data set having been filtered by the second filter 506.

[0037] Any suitable architectures may be used for the first machine learning model 508 and the second machine learning model 510. In some embodiments, the same architecture may be used for the first machine learning model 508 and the second machine learning model 510. In some embodiments, a first architecture may be used for the first machine learning model 508 and a second, different architecture may be used for the second machine learning model 510. In some embodiments, model architectures that are capable of being executed on relatively weak computing hardware, such as mobile computing devices, may be used for the first machine learning model 508 and the second machine learning model 510. Two non-limiting examples of machine learning model architectures appropriate for use as the first machine learning model 508 and / or the second machine learning model 510Docket No. 3915-P1363WO.UW are illustrated as machine learning model 602 and machine learning model 726, and are described in further detail below.

[0038] Once the first machine learning model 508 and the second machine learning model 510 have generated detected patterns as output, the detected patterns are provided to a consensus mechanism 512, which combines the output of the first machine learning model 508 and the second machine learning model 510 to create an output pattern result 514. Any suitable consensus mechanism 512 may be used to combine the output of the first machine learning model 508 and the second machine learning model 510, including but not limited to applying weights to the outputs before adding the outputs together; using an ensemble learning technique, and / or providing the output of the first machine learning model 508 and the output of the second machine learning model 510 to a third machine learning model (not illustrated) trained to predict the output sequence.

[0039] While the architecture shown in the workflow 500 can provide adequate performance, in some embodiments, additional techniques may be used to reduce storage space and processing time consumed by at least one of the first machine learning model 508 or the second machine learning model 510. For example, in some embodiments, at least one of the first machine learning model 508 or the second machine learning model 510 may be compressed via dynamic quantization or static quantization in order to reduce the size of the data stored and processed by the model. Quantization reduces the precision of the model's parameters from floating-point (32-bit) to lower-bit representations (16-bit or 8-bit), which significantly decreases the model size and speeds up inference. Dynamic quantization applies quantization at runtime, allowing for adaptive precision that can improve performance on specific hardware. Static quantization converts the weights and activations of the model to lower precision during training or post-training, enabling faster inference and reduced memory usage.

[0040] As another example, in some embodiments, at least one of the first machine learning model 508 or the second machine learning model 510 may be compressed via unstructured pruning or structured pruning to reduce the size of the model by removing portions that do not produce a significant effect on the result. Pruning removes redundant or less significant weights from the model, effectively reducing its complexity without substantially impacting accuracy. For unstructured pruning, individual weights are removed based on their importance, typically determined through a threshold value. For structured pruning, entire neurons, filters, or layers may be eliminated, which can lead to more significant reductions in model size and faster computation times.Docket No. 3915-P1363WO.UW

[0041] As yet another example, in some embodiments, at least one of the first machine learning model 508 or the second machine learning model 510 is a student model that is trained to mimic outputs of a teacher model, wherein the student model includes fewer parameters than the teacher model in order to reduce storage space consumed. This technique, known as knowledge distillation, involves training the smaller, student model to replicate the behavior of the larger, pre-trained teacher model, allowing it to achieve similar performance with fewer parameters. In some embodiments, the student model may be trained using the softened probability outputs of the teacher model, providing more informative gradients than hard labels. In some embodiments, the student model can also be trained to match the intermediate layer outputs of the teacher model, capturing more of the learned features of the teacher model.

[0042] As still another example, model architecture optimizations for efficiency may be applied to lead to a more compact and faster model without sacrificing accuracy. For example, implementing architectural changes such as depthwise separable convolutions, bottleneck layers, and / or skip connections may reduce computational overhead. As another example, automated methods such as neural architecture search (NAS) may be used to search for the most efficient architecture tailored to the specific task and hardware constraints.

[0043] As mentioned above, any suitable architecture may be used to implement the first machine learning model 508 and the second machine learning model 510. FIG. 6A and FIG. 6B illustrate a non-limiting example of a model architecture suitable for use as the first machine learning model 508 and / or the second machine learning model 510, according to various aspects of the present disclosure. The machine learning model 602 illustrated in FIG. 6A is a Gated Recurrent Unit (GRU) based transformer model that utilizes a Luong- style attention. This architecture has been found to effectively balance between model size and effectiveness in detecting patterns such as nucleotide sequences.

[0044] As shown, the sensor signal data 106 is provided to the machine learning model 602, which includes an encoder that is implemented as a cascade of one or more gated recurrent unit-based encoders 604. The output of the gated recurrent unit-based encoders 604 is provided to a decoder 608, which provides the detected pattern as output.

[0045] FIG. 6B illustrates a non-limiting example embodiment of a gated recurrent unitbased encoder 604 as illustrated in FIG. 6A. The incoming data is provided to a 1-D convolutional neural network 612, which may be a dilated convolutional neural network. The output of the 1-D convolutional neural network 612 is provided in parallel to one orDocket No. 3915-P1363WO.UW more gated recurrent units 614. The output of the one or more gated recurrent units 614 is provided to one or more self-attention 616 blocks, and the output of self-attention 616 is provided to one or more feed-forward neural networks 618 to produce the output encoded version of the input signal.

[0046] FIG. 7A and FIG. 7B illustrate another non-limiting example of a model architecture suitable for use as the first machine learning model 508 and / or the second machine learning model 510, according to various aspects of the present disclosure. Benchmarking studies using nanopore sequencing data have shown that while transformerbased basecallers, such as Bonito and Dorado, can outperform traditional CNN and RNN- based models (particularly in homopolymer resolution), there is a computational tradeoff. While transformers can provide enhanced basecalling accuracy, they also demand more computational resources. The machine learning model 726 illustrated in FIG. 7A and FIG. 7B integrates conformer blocks for hybrid convolution— attention-based signal processing, rotary positional encoding (RoPE) to improve long-sequence attention, and dilated convolutions in a CNN front end for more efficient feature extraction.

[0047] As shown, the sensor signal data 106 is first provided to a gated convolution block 702, which employs one or more gated convolutional layers using Gated Linear Unit (GLU) activation to extract low-level signal features from the sensor signal data 106. Unlike standard convolutions, gated convolutions use a gating mechanism that selectively filters information, improving feature selectivity and reducing unnecessary noise. This can help improve processing of signals such as nanopore signals, where the raw signals contain substantial noise and fluctuations due to the stochastic nature of molecular translocation through the nanopore 302.

[0048] The output of the gated convolution block 702 is provided to adaptive instance normalization 704 to normalize the extracted features. Adaptive instance normalization 704, or AdalN, dynamically adjusts normalization statistics based on the input, helping to stabilize training across different signal distributions. For signal data such as nanopore data that has varying signal intensity across different flow cells 204 and sequencing runs, AdalN can improve model robustness and generalization across data sets.

[0049] The output of adaptive instance normalization 704 is provided to a multi-scale feature extractor 706. For signal data such as nanopore data, the signals may encode multi - scale temporal dependencies, where short-term variations capture individual base characteristics, while long-term patterns influence sequence context. To address these issues, the multi-scale feature extractor 706 uses three CNNs 710, where each CNN 710Docket No. 3915-P1363WO.UW progressively refines the input representation. Each CNN 710 applies sigmoid-weighted linear unit (SiLU) activation and batch normalization, to help ensure stable gradients and efficient feature propagation. This multi-scale feature extractor 706 helps capture finegrained base-level information for nanopore signals while preserving long-range context.

[0050] Features extracted by the multi-scale feature extractor 706 are provided to gated depthwise separable convolutions 728. One computational challenge in transformer-based pattern identification for applications such as basecalling is the high processing cost of selfattention over long input sequences. To mitigate this, gated depthwise separable convolutions 728 (e.g., DWSConv) may be applied before the transformer encoder. Depthwise convolutions reduce parameter overhead by separating spatial and channel-wise processing, while GLU-based gating mechanisms enhance feature selection. Thus, the use of gated depthwise separable convolutions 728 can reduce redundant computation while maintaining strong feature extraction capabilities, allowing transformers to focus on capturing global dependencies without being overloaded by redundant local details.

[0051] The projection provided by the gated depthwise separable convolutions 728 is then provided for long-term context capture 730, which is provided by a stack of one or more conformers 712 and one or more bidirectional long short-term memory layers (bi-LSTMs 714). Any suitable number of conformers 712 and layers of bi-LSTM 714 may be used. In one example, a stack of four conformers 712 and a five-layer stack of bi-LSTMs 714 may be used, though in other examples, other numbers of conformers 712 and layers of bi-LSTMs 714 may be used. By using stacked conformers 712, the long-term context capture 730 efficiently balances local pattern extraction and long-range sequence modeling, making it particularly effective for finding patterns in continuous signals such as nanopore data.While transformers (as provided by the conformers 712) are excellent at capturing attentionbased dependencies, RNN-based architectures such as bi-LSTM 714 explicitly encode sequential order, which helps ensure accuracy in detecting patterns such as nucleotide sequences. The bi-LSTM 714 layers refine global context while maintaining sequence continuity, effectively handling ambiguities in signal-to-pattern mapping.

[0052] The final projection from the bi-LSTM 714 maps the output of the bi-LSTM 714 to a latent representation that is compatible with a decoder 716. The decoder 716 may include a fully connected linear layer which maps the extracted feature representation to an output with a number of classes which represent the possible elements of the pattern. For example, for basecalling, the output may include five classes: A, C, G, T, and a blank token used in CTC decoding. In some embodiments, the decoder 716 may also output a confidence estimation branch, which applies a 1 -dimensional convolutional layer to predict confidenceDocket No. 3915-P1363WO.UW scores for each element of the detected pattern. This provides an uncertainty estimation mechanism, which is useful for downstream applications including consensus mechanisms or error correction.

[0053] FIG. 7B is a block diagram that illustrates a non-limiting example embodiment of a conformer used by the machine learning model 726 of FIG. 7A. As shown, the conformer 712 includes one or more feed-forward layers 720, a multi-head self attention 722 mechanism, and one or more depth-wise convolutions 724. The feed-forward layers 720 may use SiLU activations and dropout to ensure stable gradient flow.

[0054] The multi-head self attention 722 mechanism may use rotary positional encoding (RoPE). Unlike natural language processing tasks where positional encoding is absolute, some pattern detection tasks such as basecalling require relative position awareness, since sequencing signals are continuous and lack explicit tokenized structure. Using RoPE within layers implementing multi-head self attention 722 enhances attention mechanisms by encoding relative positional relationships directly into the query and key vectors. This allows the model to capture long-range dependencies without sacrificing the ability to generalize across varying sequence lengths. In the context of basecalling, RoPE improves homopolymer resolution by reinforcing relative signal dependencies, making it easier to distinguish long runs of identical bases (e.g., AAAA or GGGG). The depth-wise convolutions 724 are useful for refining local feature representations.

[0055] Any suitable technique may be used to train the machine learning model 726. As one non-limiting example, an embodiment of the machine learning model 726 was trained using training data that included nanopore reads split into chunks of 2,000 non-overlapping data points to capture local signal patterns and improve model efficiency on continuous signals. A batch size of 128 was used, with a learning rate having a warm-up from 0 to le-3 over 500 steps, and cosine decay to le-4 afterward. An RAdam optimizer was used with a weight decay of zero and gradient clipping of -2 to 2 for 10 total epochs. This technique performed adequately on a single GPU with 16GB of memory, with priority given to memory efficiency with clipping and careful batching.

[0056] FIG. 8 is a flowchart that illustrates a non-limiting example embodiment of a method of determining a pattern represented by sensor signal data, according to various aspects of the present disclosure.

[0057] From a start block, the method 800 proceeds to block 802 where a data collection engine 412 of a pattern identification computing system 104 receives sensor signal data 106. The sensor signal data 106 may be any type of sensor signal data 106 that includes a patternDocket No. 3915-P1363WO.UW to be detected, including but not limited to signal data from a nanopore 302 of a flow cell 204, signals from an EKG, PPG, or EEG device, or other types of signals that represent patterns to be detected.

[0058] At optional block 804, the data collection engine 412 performs pre-processing on the sensor signal data 106 to prepare the sensor signal data 106 for analysis. As discussed above, pre-processing may include normalizing, correction for baseline shifts, signal alignment, chunking, or any other type of pre-processing suitable for the type of sensor signal data 106 being used. The optional block 804 is described as optional because in some embodiments, the pre-processing actions may not be performed if they are not necessary.

[0059] At block 806, a noise processing engine 414 of the pattern identification computing system 104 applies a first filter 504 to the sensor signal data 106 to generate a first filtered signal, and at block 808, the noise processing engine 414 applies a second filter 506 to the sensor signal data 106 to generate a second filtered signal. As described above, the first filter 504 and the second filter 506 may be similar types of noise filters, but configured to allow different amounts of noise to be filtered out (or added to) the sensor signal data 106 to create the first filtered signal and the second filtered signal. In some embodiments, the actions of block 806 and block 808 may be performed at least partially in parallel in order to decrease the overall processing time of the method 800. In some embodiments, the noise processing engine 414 may create a first copy and a second copy of the (optionally preprocessed) original sensor signal data 106, and may apply the first filter 504 and the second filter 506 to the separate copies. In some embodiments, the noise processing engine 414 may apply the first filter 504 to the sensor signal data 106 to generate the first filtered signal, and may apply the second filter 506 to the first filtered signal in order to generate the second filtered signal.

[0060] At block 810, a pattern identification engine 410 of the pattern identification computing system 104 provides the first filtered signal as input to a first machine learning model 508 to generate a first pattern result as output, and at block 812, the pattern identification engine 410 provides the second filtered signal as input to a second machine learning model 510 to generate a second pattern result as output. Because the first filtered signal and the second filtered signal include different amounts of noise, the first machine learning model 508 and the second machine learning model 510 may produce slightly different pattern results, and the differences between the pattern results can help improve the accuracy of the final result. In some embodiments, the actions of block 810 and block 812 may be performed at least partially in parallel in order to decrease the overall processing time of the method 800.Docket No. 3915-P1363WO.UW

[0061] At block 814, the pattern identification engine 410 applies a consensus mechanism 512 to the first pattern result and the second pattern result to generate an output pattern result. As discussed above, any suitable consensus mechanism 512 may be used, including but not limited to applying weights to the outputs before adding the outputs together; using an ensemble learning technique, and / or providing the output of the first machine learning model 508 and the output of the second machine learning model 510 to a third machine learning model trained to predict the output sequence. In some embodiments, the consensus mechanism 512 may be applied to the overall first pattern result and the second pattern result to generate the output pattern result. In some embodiments, the consensus mechanism 512 may be applied to sub-portions of the first pattern result and the second pattern result (e.g., to individual bases or chunks of bases of a nucleotide sequence).

[0062] The method 800 then proceeds to an end block and terminates.

[0063] Though illustrated as terminating, in some embodiments, additional processing appropriate to the type of information processed by the method 800 may occur. For example, when the sensor signal data 106 is signal data from a nanopore 302 of a flow cell 204, additional processing for correction of homopolymer errors and / or modification- mediated errors may occur.

[0064] For homopolymer errors, a tool such as homopolish may be used. Homopolish is a tool designed for the correction of systematic errors in nanopore sequencing data that uses homologous sequences and a support vector machine (SVM) to distinguish between sequencing errors and strain variations. In some embodiments, homopolish may be applied to the output pattern result. Features may be extracted from the homologous alignment profile at each locus of the draft genome, and the SVM within homopolish may be used to classify each locus as an error or a variation. The systematic errors (such as homopolymer errors) may be corrected while retaining strain variations.

[0065] For modification-mediated errors, a tool such as modpolish may be used. Modpolish targets modification-mediated errors caused by chemical modifications (including, but not limited to, methylated cytosines) present in the DNA or RNA bases. It aims to retain the epigenetic information while enhancing sequence quality. Modpolish is offered as a submodule within the homopolish framework, and may be applied after the error correction of homopolish described above.

[0066] In some embodiments, caching techniques may be used to further enhance the efficiency of the pattern determinations. FIG. 9 is a flowchart that illustrates a non-limitingDocket No. 3915-P1363WO.UW example embodiment of a method of determining a pattern represented by sensor signal data using caching, according to various aspects of the present disclosure.

[0067] From a start block, the method 900 proceeds to block 902, where a data collection engine 412 of a pattern identification computing system 104 receives sensor signal data 106. The receipt of sensor signal data 106 and the types of data received at block 902 are similar to those described in block 802, and so is not described in further detail here for the sake of brevity.

[0068] At block 904, a pattern identification engine 410 of the pattern identification computing system 104 generates an encoding of the sensor signal data 106 using a transformer model. The transformer model encodes the sensor signal data 106, or at least a chunk thereof, into a latent representation, or encoding, that represents the sensor signal data 106 within a latent space.

[0069] At block 906, the pattern identification engine 410 compares the encoding to one or more cached signal patterns associated with known patterns. The method 900 then proceeds to a decision block 908, where a determination is made based on whether a high-confidence match between the encoding and a cached signal pattern was found. If a high-confidence match was found, then the result of decision block 908 is YES, and the method 900 proceeds to block 910, where the pattern identification engine 410 returns the known pattern associated with the cached signal pattern as an output pattern result 514. The method 900 then proceeds to an end block and terminates.

[0070] Returning to decision block 908, if a high-confidence match between the encoding and a cached signal pattern was not found, then the result of decision block 908 is NO, and the method 900 proceeds to subroutine block 912. At subroutine block 912, actions are performed wherein the pattern identification computing system 104 performs actions for determining the pattern based on the sensor signal data 106. In some embodiments, the actions performed at subroutine block 912 are the actions of the method 800 described in detail above. The method 900 then proceeds to an end block and terminates.EXAMPLES

[0071] The following paragraphs provide a numbered set of non-limiting example embodiments of the present disclosure.

[0072] Example 1 : A computer-implemented method of determining a pattern represented by sensor signal data, the method comprising: receiving, by a computing system, sensor signal data; filtering, by the computing system using a first filter, the sensor signal data to generate a first filtered signal; filtering, by the computing system using a second filter, theDocket No. 3915-P1363WO.UW sensor signal data to generate a second filtered signal; providing, by the computing system, the first filtered signal as a first input to a first machine learning model to generate a first pattern result as output; providing, by the computing system, the second filtered signal as a second input to a second machine learning model to generate a second pattern result as output; and combining, by the computing system, the first pattern result and the second pattern result to generate an output pattern result.

[0073] Example 2: The computer-implemented method of example 1, wherein the sensor signal data includes a signal generated by a nanopore that represents a molecule transiting the nanopore.

[0074] Example 3: The computer-implemented method of any one of examples 1-2, wherein the output pattern result is a nucleotide sequence.

[0075] Example 4: The computer-implemented method of example 3, further comprising correcting, by the computing system, homopolymer errors in the nucleotide sequence.

[0076] Example 5: The computer-implemented method of any one of examples 3-4, further comprising correcting, by the computing system, modification-mediated errors in the nucleotide sequence.

[0077] Example 6: The computer-implemented method of any one of examples 1-5, wherein the first machine learning model and the second machine learning model include one or more transformer models.

[0078] Example 7: The computer-implemented method of example 6, wherein the transformer models are Gated Recurrent Unit (GRU)-based transformer models.

[0079] Example 8: The computer-implemented method of example 7, wherein the GRU- based transformer models utilize Luong-style attention

[0080] Example 9: The computer-implemented method of any one of examples 7-8, wherein the GRU-based transformer models include a cascaded network of GRU encoders, wherein each GRU encoder includes a 1 -dimensional dilated convolutional neural network (CNN) and a plurality of GRUs.

[0081] Example 10: The computer-implemented method of any one of examples 1-9, wherein the first filter includes a denoising filter configured to apply a first type and first amount of denoising; wherein the second filter includes a denoising filter configured to apply a second type and second amount of denoising; and wherein at least one of the first type of denoising is different from the second type of denoising, or the first amount of denoising is different from the second amount of denoising.Docket No. 3915-P1363WO.UW

[0082] Example 11 : The computer-implemented method of any one of examples 1-10, wherein combining the first pattern result and the second pattern result to generate an output pattern result includes one or more of: applying weights to the first pattern result and the second pattern result; using an ensemble learning technique to combine the first pattern result and the second pattern result; or providing the first pattern result and the second pattern result to a machine learning model trained to predict the output pattern result.

[0083] Example 12: The computer-implemented method of any one of examples 1-11, wherein at least one of the first machine learning model and the second machine learning model are compressed via dynamic quantization or static quantization.

[0084] Example 13: The computer-implemented method of any one of examples 1-12, wherein at least one of the first machine learning model and the second machine learning model are compressed via unstructured pruning or structured pruning.

[0085] Example 14: The computer-implemented method of any one of examples 1-13, wherein at least one of the first machine learning model and the second machine learning model is a student model trained to mimic outputs of a teacher model, wherein the student model includes fewer parameters than the teacher model.

[0086] Example 15: The computer-implemented method of any one of examples 1-14, wherein at least one of the first machine learning model and the second machine learning model includes at least one depthwise separable convolution, at least one bottleneck layer, or at least one skip connection.

[0087] Example 16: A computer-implemented method of determining a sequence represented by nanopore signal data, the method comprising: receiving, by a computing system, a raw nanopore signal; generate, by the computing system, an encoding of the raw nanopore signal using a transformer model; comparing, by the computing system, the encoding to one or more cached signal patterns associated with known sequences; in response to determining a high-confidence match between the encoding and a cached signal pattern, returning the known sequence associated with the cached signal pattern as an output sequence; and in response to determining no high-confidence match between the encoding and a cached signal pattern, determining the output sequence using a method as recited in any one of examples 1-15.

[0088] Example 17: A non-transitory computer-readable medium having computerexecutable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform a method as recited in any one of example 1 to example 16.Docket No. 3915-P1363WO.UW

[0089] Example 18: A computing system comprising: at least one processor; and a non- transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by the at least one processor, cause the computing system to perform a method as recited in any one of example 1 to example 16.

[0090] Example 19: The computing system of example 18, wherein the computing system is a smartphone or a tablet computing device.

[0091] Example 20: The computing system of any one of examples 18-19, wherein the at least one processor includes a tensor processing unit (TPU).

[0092] The complete disclosure of all patents, patent applications, and publications, and electronically available material cited herein are incorporated by reference in their entirety. Supplementary materials referenced in publications (such as supplementary tables, supplementary figures, supplementary materials and methods, and / or supplementary experimental data) are likewise incorporated by reference in their entirety. In the event that any inconsistency exists between the disclosure of the present application and the disclosure(s) of any document incorporated herein by reference, the disclosure of the present application shall govern.

[0093] The foregoing detailed description and examples have been given for clarity of understanding only. No unnecessary limitations are to be understood therefrom. The disclosure is not limited to the exact details shown and described, for variations obvious to one skilled in the art will be included within the disclosure defined by the claims.

[0094] The description of embodiments of the disclosure is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. While the specific embodiments of, and examples for, the disclosure are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the disclosure.

[0095] Specific elements of any foregoing embodiments can be combined or substituted for elements in other embodiments. Moreover, the inclusion of specific elements in at least some of these embodiments may be optional, wherein further embodiments may include one or more embodiments that specifically exclude one or more of these specific elements. Furthermore, while advantages associated with certain embodiments of the disclosure have been described in the context of these embodiments, other embodiments may also exhibit such advantages, and not all embodiments need necessarily exhibit such advantages to fall within the scope of the disclosure.Docket No. 3915-P1363WO.UW

[0096] As used herein and unless otherwise indicated, the terms “a” and “an” are taken to mean “one”, “at least one” or “one or more”. Unless otherwise required by context, singular terms used herein shall include pluralities and plural terms shall include the singular.

[0097] Unless the context clearly requires otherwise, throughout the description and the claims, the words ‘comprise’, ‘comprising’, and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to”. Words using the singular or plural number also include the plural and singular number, respectively. Additionally, the words “herein,” “above,” and “below” and words of similar import, 10 when used in this application, shall refer to this application as a whole and not to any particular portions of the application.

[0098] Unless otherwise indicated, all numbers expressing quantities of components, molecular weights, and so forth used in the specification and claims are to be understood as being modified in all instances by the term "about." Accordingly, unless otherwise indicated to the contrary, the numerical parameters set forth in the specification and claims are approximations that may vary depending upon the desired properties sought to be obtained by the present disclosure. At the very least, and not as an attempt to limit the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be construed in light of the number of reported significant digits and by applying ordinary rounding techniques.

[0099] Notwithstanding that the numerical ranges and parameters setting forth the broad scope of the disclosure are approximations, the numerical values set forth in the specific examples are reported as precisely as possible. All numerical values, however, inherently contain a range necessarily resulting from the standard deviation found in their respective testing measurements.

[0100] All headings are for the convenience of the reader and should not be used to limit the meaning of the text that follows the heading, unless so specified.

[0101] All of the references cited herein are incorporated by reference. Aspects of the disclosure can be modified, if necessary, to employ the systems, functions, and concepts of the above references and application to provide yet further embodiments of the disclosure. These and other changes can be made to the disclosure in light of the detailed description.

[0102] It will be appreciated that, although specific embodiments of the disclosure have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the disclosure. Accordingly, the disclosure is not limited except as by the claims.

Claims

Docket No. 3915-P1363WO.UWCLAIMSThe embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. A computer-implemented method of determining a pattern represented by sensor signal data, the method comprising: receiving, by a computing system, sensor signal data; filtering, by the computing system using a first filter, the sensor signal data to generate a first filtered signal; filtering, by the computing system using a second filter, the sensor signal data to generate a second filtered signal; providing, by the computing system, the first filtered signal as a first input to a first machine learning model to generate a first pattern result as output; providing, by the computing system, the second filtered signal as a second input to a second machine learning model to generate a second pattern result as output; and combining, by the computing system, the first pattern result and the second pattern result to generate an output pattern result.

2. The computer-implemented method of claim 1, wherein the sensor signal data includes a signal generated by a nanopore that represents a molecule transiting the nanopore.

3. The computer-implemented method of claim 1, wherein the output pattern result is a nucleotide sequence.

4. The computer-implemented method of claim 3, further comprising correcting, by the computing system, homopolymer errors in the nucleotide sequence.

5. The computer-implemented method of claim 3, further comprising correcting, by the computing system, modification-mediated errors in the nucleotide sequence.

6. The computer-implemented method of claim 1, wherein the first machine learning model and the second machine learning model include one or more transformer models.

7. The computer-implemented method of claim 6, wherein the transformer models are Gated Recurrent Unit (GRU)-based transformer models.

8. The computer-implemented method of claim 7, wherein the GRU-based transformer models utilize Luong-style attentionDocket No. 3915-P1363WO.UW9. The computer-implemented method of claim 7, wherein the GRU-based transformer models include a cascaded network of GRU encoders, wherein each GRU encoder includes a 1 -dimensional dilated convolutional neural network (CNN) and a plurality of GRUs.

10. The computer-implemented method of claim 1, wherein the first filter includes a denoising filter configured to apply a first type and first amount of denoising; wherein the second filter includes a denoising filter configured to apply a second type and second amount of denoising; and wherein at least one of the first type of denoising is different from the second type of denoising, or the first amount of denoising is different from the second amount of denoising.

11. The computer-implemented method of claim 1, wherein combining the first pattern result and the second pattern result to generate an output pattern result includes one or more of applying weights to the first pattern result and the second pattern result; using an ensemble learning technique to combine the first pattern result and the second pattern result; or providing the first pattern result and the second pattern result to a machine learning model trained to predict the output pattern result.

12. The computer-implemented method of claim 1, wherein at least one of the first machine learning model and the second machine learning model are compressed via dynamic quantization or static quantization.

13. The computer-implemented method of claim 1, wherein at least one of the first machine learning model and the second machine learning model are compressed via unstructured pruning or structured pruning.

14. The computer-implemented method of claim 1, wherein at least one of the first machine learning model and the second machine learning model is a student model trained to mimic outputs of a teacher model, wherein the student model includes fewer parameters than the teacher model.

15. The computer-implemented method of claim 1, wherein at least one of the first machine learning model and the second machine learning model includes at least one depthwise separable convolution, at least one bottleneck layer, or at least one skip connection.Docket No. 3915-P1363WO.UW16. A computer-implemented method of determining a sequence represented by nanopore signal data, the method comprising: receiving, by a computing system, a raw nanopore signal; generate, by the computing system, an encoding of the raw nanopore signal using a transformer model; comparing, by the computing system, the encoding to one or more cached signal patterns associated with known sequences; in response to determining a high-confidence match between the encoding and a cached signal pattern, returning the known sequence associated with the cached signal pattern as an output sequence; and in response to determining no high-confidence match between the encoding and a cached signal pattern, determining the output sequence using a method as recited in claim 1.

17. A non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform a method as recited in any one of claim 1 to claim 16.

18. A computing system comprising: at least one processor; and a non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by the at least one processor, cause the computing system to perform a method as recited in any one of claim 1 to claim 16.

19. The computing system of claim 18, wherein the computing system is a smartphone or a tablet computing device.

20. The computing system of claim 18, wherein the at least one processor includes a tensor processing unit (TPU).

Images