Split architecture for artificial intelligence based base caller

Abstract

The technology disclosed relates to a system comprising a spatial convolution network and a bus network. The spatial convolution network is configured to process windows of a set of per-cycle sequencing images one cycle at a time by a respective spatial processing pipeline processing a respective set of per-cycle spatial feature maps for a respective sequencing cycle by processing a respective set of per-cycle sequencing images through a respective spatial processing pipeline, respectively. The bus network is configured to form a bus between spatial convolution layers within the respective spatial processing pipeline. The bus is configured to cause a respective set of per-cycle spatial feature maps generated by two or more of the spatial convolution layers in a particular sequence of spatial convolution layers for a particular sequencing cycle to be combined into a combined set of per-cycle spatial feature maps and to provide the combined set of per-cycle spatial feature maps as input to another of the spatial convolution layers in the particular sequence of spatial convolution layers.

Description

Technical Field

[0001] The technologies disclosed in this invention relate to artificial intelligence-type computers and digital data processing systems, as well as corresponding data processing methods and products for simulating intelligence (i.e., knowledge-based systems, inference systems, and knowledge acquisition systems); and include systems for uncertainty inference (e.g., fuzzy logic systems), adaptive systems, machine learning systems, and artificial neural networks. Specifically, the disclosed technologies relate to using deep neural networks, such as deep convolutional neural networks, for data analysis.

[0002] Priority application

[0003] This PCT application claims priority and interest in U.S. Provisional Patent Application No. 62 / 979,411 (Attorney's File No. ILLM 1029-1 / IP-1964-PRV), filed February 20, 2020, entitled "DATA COMPRESSION FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLING," and U.S. Patent Application No. 17 / 179,395 (Attorney's File No. ILLM 1029-2 / IP-1964-US), filed February 18, 2021, entitled "DATA COMPRESSION FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLING." These priority applications are incorporated herein by reference as if fully illustrated herein, for all purposes.

[0004] This PCT application claims priority and interest in U.S. Provisional Patent Application No. 62 / 979,399 (Attorney's File No. ILLM 1030-1 / IP-1982-PRV), filed February 20, 2020, entitled “SQUEEZING LAYER FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLER”, and U.S. Patent Application No. 17 / 180,480 (Attorney's File No. ILLM 1030-2 / IP-1982-US), filed February 19, 2021, entitled “SPLIT ARCHITECTURE FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLER”. These priority applications are incorporated herein by reference as if fully illustrated herein, for all purposes.

[0005] This PCT application claims priority and benefit to U.S. Patent Application No. 17 / 180,513 (Attorney’s File No. ILLM1031-2 / IP-1965-US), filed February 19, 2021, entitled “NETWORK FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLER”. These priority applications are hereby incorporated by reference as if fully illustrated herein, for all purposes.

[0006] Literature merged

[0007] The following references are incorporated herein by reference as if they were shown in their entirety in this article:

[0008] U.S. Provisional Patent Application No. 62 / 979,384, entitled “ARTIFICIAL INTELLIGENCE-BASED BASE CALLINGOF INDEX SEQUENCES”, filed on February 20, 2020 (Attorney’s File No. ILLM 1015-1 / IP-1857-PRV);

[0009] U.S. Provisional Patent Application No. 62 / 979,414, entitled “ARTIFICIAL INTELLIGENCE-BASED MANY-TO-MANYBASE CALLING”, filed on February 20, 2020 (Attorney’s File No. ILLM 1016-1 / IP-1858-PRV);

[0010] U.S. Provisional Patent Application No. 62 / 979,385 (Attorney’s File No. ILLM 1017-1 / IP-1859-PRV), filed on February 20, 2020, entitled “KNOWLEDGE DISTILLATION-BASED COMPRESSION OFARTIFICIAL INTELLIGENCE-BASED BASE CALLER”;

[0011] U.S. Provisional Patent Application No. 63 / 072,032, entitled “DETECTING AND FILTERING CLUSTERS BASED ON ARTIFICIAL INTELLIGENCE-PREDICTED BASE CALLS”, filed on August 28, 2020 (Attorney’s File No. ILLM 1018-1 / IP-1860-PRV);

[0012] U.S. Provisional Patent Application No. 62 / 979,412, entitled “MULTI-CYCLE CLUSTER BASED REAL TIMEANALYSIS SYSTEM”, filed on February 20, 2020 (Attorney’s File No. ILLM 1020-1 / IP-1866-PRV);

[0013] U.S. non-provisional patent application No. 16 / 825,987, entitled “TRAINING DATA GENERATION FOR ARTIFICIALINTELLIGENCE-BASED SEQUENCING”, filed on March 20, 2020 (Attorney’s File No. ILLM 1008-16 / IP-1693-US);

[0014] U.S. non-provisional patent application No. 16 / 825,991, entitled “ARTIFICIAL INTELLIGENCE-BASED GENERATION OF SEQUENCING METADATA”, filed on March 20, 2020 (Attorney’s File No. ILLM 1008-17 / IP-1741-US);

[0015] U.S. non-provisional patent application number 16 / 826,126, entitled “ARTIFICIAL INTELLIGENCE-BASED BASE CALLING”, filed on March 20, 2020 (Attorney’s file number ILLM 1008-18 / IP-1744-US);

[0016] U.S. non-provisional patent application No. 16 / 826,134 (Attorney's File No. ILLM 1008-19 / IP-1747-US), filed on March 20, 2020, entitled "ARTIFICIAL INTELLIGENCE-BASED QUALITYSCORING"; and

[0017] U.S. non-provisional patent application number 16 / 826,168, entitled “ARTIFICIAL INTELLIGENCE-BASED SEQUENCING”, filed on March 21, 2020 (Attorney’s file number ILLM 1008-20 / IP-1752-PRV-US). Background Technology

[0018] The topics discussed in this section should not be considered prior art simply because they are mentioned here. Similarly, problems mentioned in this section or related to the topics provided as background art should not be assumed to have been previously recognized in the prior art. The topics in this section merely represent different methods, which themselves may correspond to specific implementations of the technology protected by the claims.

[0019] In recent years, the rapid increase in computing power has enabled deep convolutional neural networks (CNNs) to achieve great success in many computer vision tasks with significantly improved accuracy. During the inference phase, many applications require low latency processing of an image with stringent power consumption requirements, which reduces the efficiency of graphics processing units (GPUs) and other general-purpose platforms. This has created opportunities for specific acceleration hardware (e.g., field-programmable gate arrays (FPGAs)) by customizing digital circuitry specifically for deep learning algorithm inference. However, deploying CNNs on portable and embedded systems remains challenging due to large data volumes, computationally intensive operations, varying algorithmic architectures, and frequent memory accesses.

[0020] Since convolution contributes a large portion of the computation in CNNs, convolution acceleration schemes significantly impact the efficiency and performance of hardware CNN accelerators. Convolution involves a multiplication and accumulation (MAC) operation with four recurrent stages sliding along the kernel and feature maps. The first recurrent stage computes the MAC of the pixels within the kernel window. The second recurrent stage accumulates the sum of the products of the MACs across different input feature maps. After completing the first and second recurrent stages, a bias is added to obtain the final output pixel. The third recurrent stage slides the kernel window within the input feature maps. The fourth recurrent stage generates different output feature maps.

[0021] FPGAs have gained increasing attention and popularity, particularly in accelerating inference tasks, due to their (1) high reconfigurability, (2) faster development time compared to application-specific integrated circuits (ASICs) to keep pace with the rapid development of CNNs, (3) good performance, and (4) superior energy efficiency compared to GPUs. The high performance and efficiency of FPGAs can be achieved by synthesizing circuits tailored to specific computations to directly process billions of operations using customized memory systems. For example, hundreds to thousands of digital signal processing (DSP) blocks on modern FPGAs support core convolution operations such as multiplication and addition with high parallelism. Dedicated data buffers between external on-chip memory and on-chip processing engines (PEs) can be designed to achieve optimized data flow by configuring tens of megabytes of on-chip block random access memory (BRAM) on the FPGA chip.

[0022] High-performance CNNs require efficient data flow and accelerated hardware architectures to minimize data communication while maximizing resource utilization. This presents an opportunity to design methods and frameworks for accelerating the inference process of various CNN algorithms on high-performance, efficient, and highly flexible accelerated hardware. Attached Figure Description

[0023] In the accompanying drawings, similar reference numerals generally refer to similar parts in all different views. Furthermore, the drawings are not necessarily drawn to scale, but rather emphasize the principles of the disclosed technology. In the following description, various specific embodiments of the disclosed technology are described with reference to the following drawings, wherein:

[0024] Figure 1A A specific implementation of the disclosed compression logic is shown, which generates a compressed spatial atlas for the first iteration of base detection.

[0025] Figure 1B A specific implementation is shown that processes compressed spatial atlases to generate temporal atlases using the disclosed temporal logic.

[0026] Figure 1C A specific implementation is shown that processes the time atlas using the disclosed output logic to generate base detection classification data.

[0027] Figure 1D An example is shown of a sequence of feature map volumes generated sequentially by the cascaded response of spatial convolutional layers to each cycle of image patch processing for a target sequencing loop.

[0028] Figure 1E An example illustrating how a 1×1 convolution compresses a feature map is depicted.

[0029] Figure 1F The compression ratio achieved by the disclosed compression logic is shown to be a function of the number of compression filters applied to the disclosed compression logic.

[0030] Figure 1G An example softmax function is shown.

[0031] Figure 1H An exemplary per-cluster, per-cycle probability quadruple is depicted using the disclosed technique.

[0032] Figure 2A This shows that during the second iteration of base detection, only the spatial map and the corresponding compressed spatial map are generated for non-overlapping sequencing cycle 6.

[0033] Figure 2BThe compressed spatial atlas generated during the first iteration of base detection is shown to be used in combination with the compressed spatial atlas generated during the second iteration of base detection to generate base detection for central sequencing cycle 4.

[0034] Figure 2C The output layer processing generates the final time map during the second iteration of base detection and produces base detection for central sequencing cycle 4.

[0035] Figure 3A This shows that during the third iteration of base detection, only the space map and the corresponding compressed space map are generated for non-overlapping sequencing cycle 7.

[0036] Figure 3B The compressed spatial atlas generated during the first and second iterations of base detection is shown to be used in combination with the compressed spatial atlas generated during the third iteration of base detection to generate base detection for central sequencing cycle 5.

[0037] Figure 3C The final time map generated during the third iteration of base detection in the output layer processing is shown, and base detection is generated for central sequencing cycle 5.

[0038] Figure 4A The fourteenth iteration of base detection for base detection in central sequencing cycle 16 is shown.

[0039] Figure 4B The compressed spatial maps previously generated for sequencing cycles 1 through 29 are shown to be used to generate the final time map set for base detection in the central sequencing cycle 16.

[0040] Figure 4C The final time map generated during the fourteenth iteration of base detection in the output layer processing is shown, and base detection is generated for central sequencing cycle 16.

[0041] Figure 5A This illustrates a specific implementation of using filtering logic to filter the compressed spatial atlas for the corresponding sequencing cycles 1, 2, 3, 4, and 5 during the first iteration of base detection to generate the corresponding compressed, filtered spatial maps.

[0042] Figure 5B The output layer processing shows the final filtered time map generated during the first iteration of base detection and the base detection for central sequencing cycle 3.

[0043] Figure 6AThis illustrates a specific implementation of using filtering logic to filter the compressed spatial atlas for the corresponding sequencing cycles 2, 3, 4, 5, and 6 during the second iteration of base detection to generate the corresponding compressed, filtered spatial maps.

[0044] Figure 6B The output layer processing generates the final filtered time map during the second iteration of base detection and produces base detection for central sequencing cycle 4.

[0045] Figure 7A This illustrates a specific implementation of using filtering logic to filter the compressed spatial atlas for the corresponding sequencing cycles 3, 4, 5, 6, and 7 during the third iteration of base detection to generate the corresponding compressed, filtered spatial maps.

[0046] Figure 7B The output layer processing shows the final filtered timescale generated during the third iteration of base detection and the base detection for central sequencing cycle 5.

[0047] Figure 8A A specific implementation is shown to generate a corresponding compressed time feature set by processing the time feature set generated during the first iteration of base detection through compression logic.

[0048] Figure 8B The output layer processing generates the final compressed timescale during the first iteration of base detection and produces base detection for central sequencing cycle 3.

[0049] Figure 9A The compressed time map generated during the first base detection iteration is shown to be used in combination with the compressed time map generated during the second base detection iteration to generate base detection for central sequencing cycle 4.

[0050] Figure 9B The output layer processing generates the final compressed timescale during the second iteration of base detection and produces base detection for central sequencing cycle 4.

[0051] Figure 10A The compressed timescale generated during the first and second iterations of base detection is shown to be used in combination with the compressed timescale generated during the third iteration of base detection to generate base detection for central sequencing cycle 5.

[0052] Figure 10B The output layer processing generates the final compressed timescale during the third iteration of base detection and produces base detection for central sequencing cycle 5.

[0053] Figure 11AA specific implementation is shown to generate a corresponding compressed, filtered time feature set by processing the filtered time feature set during the first iteration of base detection through compression logic.

[0054] Figure 11B The output layer processing generates the final compressed, filtered time map during the first iteration of base detection and produces base detection for central sequencing cycle 3.

[0055] Figure 12A The compressed, filtered timescale generated during the first base detection iteration is shown to be used in combination with the compressed, filtered timescale generated during the second base detection iteration to generate base detection for central sequencing cycle 4.

[0056] Figure 12B The output layer processing generates the final compressed, filtered time map during the second iteration of base detection and produces base detection for central sequencing cycle 4.

[0057] Figure 13A The compressed, filtered timescales generated during the first and second base detection iterations are shown to be used in combination with the compressed, filtered timescales generated during the third base detection iteration to generate base detections for central sequencing cycle 5.

[0058] Figure 13B The output layer processing generates the final compressed, filtered time map during the third iteration of base detection and produces base detection for central sequencing cycle 5.

[0059] Figure 14 A first exemplary architecture of the neural network-based base detector disclosed herein is shown.

[0060] Figure 15 A second exemplary architecture of the neural network-based base detector disclosed herein is shown.

[0061] Figure 16 A third exemplary architecture of the neural network-based base detector disclosed herein is shown.

[0062] Figure 17 A fourth exemplary architecture of the neural network-based base detector disclosed herein is shown.

[0063] Figure 18 A specific implementation of filter configuration logic is shown, which configures the count (or number) of convolutional filters in the compressed layer based on the number of channels in the input data.

[0064] Figure 19Aand Figure 19B A specific implementation of a sequencing system is described. This sequencing system includes a configurable processor.

[0065] Figure 19C This is a simplified block diagram of a system for analyzing sensor data (such as base detection sensor output) from a sequencing system.

[0066] Figure 20A This is a simplified diagram illustrating various aspects of a base detection operation, including the functionality of a runtime program executed by the host processor.

[0067] Figure 20B This is a simplified diagram of the configuration of a configurable processor.

[0068] Figure 21 Another specific implementation of the disclosed data flow logic is shown, which makes the compressed space map generated during the first base detection iteration available from off-chip memory (e.g., off-chip DRAM, host RAM, host high bandwidth memory (HBM)) during the second base detection iteration.

[0069] Figure 22 A specific implementation of the disclosed data flow logic is shown, which makes a compressed spatial map generated during the first base detection iteration available from on-chip memory (e.g., on-chip DRAM, on-chip SRAM, on-chip BRAM, DRAM attached to the processor via interconnects) during the second base detection iteration.

[0070] Figure 23 A specific implementation of the so-called splitting architecture of the disclosed neural network-based base detector is shown.

[0071] Figure 24A The residual connections are described by re-injecting previous information downstream via feature map addition.

[0072] Figure 24B A specific implementation of residual blocks and jump connections is described.

[0073] Figure 24C The residual architecture of a neural network-based base detector is shown, in which spatial convolutional layers are grouped into residual blocks with skip connections.

[0074] Figure 25A Details of the disclosed bus network for the neural network-based base detector described herein are shown.

[0075] Figure 25B An exemplary operation of the disclosed bus network is shown.

[0076] Figure 25CA specific implementation of the dimensional compatibility logic of the disclosed bus network is shown.

[0077] Figure 26 Another example of the disclosed bus network is shown.

[0078] Figure 27 Another example of the disclosed bus network is shown.

[0079] Figure 28 A specific implementation of the scaling logic for the disclosed bus network is shown.

[0080] Figure 29 This illustrates a specific implementation of skip connections between temporal convolutional layers in a temporal network.

[0081] Figure 30 The base detection performance of a network-based base detector (sqz2 base detector) with compression logic was compared with that of a network-based base detector without compression logic and Illumina's non-neural network-based base detector real-time analysis (RTA) software.

[0082] Figure 31 The savings in RAM and DRAM usage resulting from the use of the disclosed compression logic are shown.

[0083] Figure 32 The base detection performance of a network-based base detector configured with a split and skip architecture (split_res) was compared with that of an RTA base detector and another version of a network-based base detector without a split and skip architecture (distilled).

[0084] Figure 33 It is a computer system that can be used to implement the disclosed technology. Detailed Implementation

[0085] The following discussion is presented to enable any person skilled in the art to implement and use the disclosed technology, and is provided in the context of a particular application and its requirements. Various modifications to the specific embodiments disclosed will be apparent to those skilled in the art, and the general principles defined herein can be applied to other specific embodiments and applications without departing from the spirit and scope of the disclosed technology. Therefore, the disclosed technology is not intended to be limited to the specific embodiments shown, but is to be accorded the broadest scope consistent with the principles and features disclosed herein.

[0086] Sequencing images

[0087] Base detection is the process of determining the nucleotide composition of a sequenced product. Base detection involves analyzing image data, which is the sequencing image generated during a sequencing run (or sequencing reaction) performed by sequencing instruments such as Illumina's iSeq, HiSeqX, HiSeq 3000, HiSeq 4000, HiSeq2500, NovaSeq 6000, NextSeq 550, NextSeq 1000, NextSeq 2000, NextSeqDx, MiSeq, and MiSeqDx.

[0088] Based on a specific implementation, the following discussion outlines how sequencing images are generated and what they depict.

[0089] Base detection decodes the intensity data encoded in the sequencing image into nucleotide sequences. In one specific implementation, the Illumina sequencing platform employs cycle reversible termination (CRT) chemistry for base detection. This process relies on growing a nascent strand complementary to the template strand of the fluorescently labeled nucleotide while tracking the emission signal of each newly added nucleotide. The fluorescently labeled nucleotide has a 3' removable block that anchors the fluorophore signal of the nucleotide type.

[0090] Sequencing is performed in repeated cycles, each cycle consisting of three steps: (a) elongating the nascent strand by adding fluorescently labeled nucleotides; (b) exciting the fluorophore using one or more lasers of the sequencing instrument's optical system and imaging it through different filters of the optical system to produce a sequencing image; and (c) lysing the fluorophore and removing the 3' block to prepare for the next sequencing cycle. The binding and imaging cycles are repeated until a specified number of sequencing cycles are reached, thereby defining the read length. Using this method, each cycle queries a new location along the template strand.

[0091] The immense power of Illumina sequencers stems from their ability to simultaneously execute and sense millions, or even billions, of clusters (also known as “analytes”) undergoing a CRT reaction. A cluster comprises approximately a thousand identical copies of the template strand, but differs in size and shape. Clusters grow from the template strand before sequencing runs by bridging or exclusion amplification of the input library. The purpose of amplification and cluster growth is to increase the intensity of the emission signal, as imaging devices cannot reliably sense the signal of a single-stranded fluorophore. However, the strands within a cluster are physically close together, so the imaging device perceives the cluster of strands as a single point.

[0092] Sequencing is performed in a flow cell (or biosensor)—a small slide that holds the input chain. The flow cell is connected to an optical system that includes microscopic imaging, an excitation laser, and a fluorescence filter. The flow cell comprises multiple chambers called channels. These channels are physically separated from each other and may contain different labeled sequencing libraries that can be distinguished without sample cross-contamination. In some implementations, the flow cell includes a patterned surface. A “patterned surface” refers to the arrangement of different regions in or on an exposed layer of a solid support.

[0093] Imaging devices in sequencing instruments (e.g., solid-state imagers, such as charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) sensors) take snapshots at multiple locations along the channel, in a series of non-overlapping regions (called blocks). For example, each channel may have 64 or 96 blocks. The blocks hold hundreds of thousands to millions of clusters.

[0094] The output of a sequencing run is a sequencing image. A sequencing image uses a grid (or array) of pixelated units (e.g., pixels, superpixels, subpixels) to depict the intensity emission of clusters and their surrounding background. Intensity emission is stored as intensity values ​​for the pixelated units. A sequencing image has a grid of pixelated units with dimensions w × h, where w (width) and h (height) are any numbers ranging from 1 to 100,000 (e.g., 115 × 115, 200 × 200, 1800 × 2000, 2200 × 25000, 2800 × 3600, 4000 × 400). In some implementations, w and h are the same. In other implementations, w and h are different. A sequencing image depicts the intensity emission generated due to nucleotide incorporation into the nucleotide sequence during the sequencing run. Intensity emission originates from the associated clusters and their surrounding background.

[0095] Neural Network-Based Base Detection

[0096] The following discussion focuses on the neural network-based base detector 100 described herein. First, the input to the neural network-based base detector 100 is described according to a specific embodiment. Then, examples of the structure and form of the neural network-based base detector 100 are provided. Finally, the output of the neural network-based base detector 100 is described according to a specific embodiment.

[0097] The data flow logic provides the sequencing image to a neural network-based base detector 100 for base detection. The neural network-based base detector 100 accesses the sequencing image patch by patch (or block by block). Each patch is a sub-grid (or sub-array) of pixelated units within a grid of pixelated units that form the sequencing image. The patch has a sub-grid dimension q×r, where q (width) and r (height) are any numbers ranging from 1 to 10000 (e.g., 3×3, 5×5, 7×7, 10×10, 15×15, 25×25, 64×64, 78×78, 115×115). In some embodiments, q and r are the same. In other embodiments, q and r are different. In some embodiments, the patches extracted from the sequencing image have the same size. In other embodiments, the patches have different sizes. In some implementations, patches may have overlapping pixelated units (e.g., at the edges).

[0098] For each of the corresponding m image channels, sequencing produces m sequencing images in each sequencing cycle. That is, each sequencing image has one or more image (or intensity) channels (similar to the red, green, and blue (RGB) channels in a color image). In one embodiment, each image channel corresponds to one of a plurality of filter wavelength bands. In another embodiment, each image channel corresponds to one of a plurality of imaging events in a sequencing cycle. In yet another embodiment, each image channel corresponds to a combination of illumination using a specific laser and imaging through a specific optical filter. For a given sequencing cycle, an image patch is tiled (or accessed) from each of the m image channels. In different embodiments such as four-channel chemistry, two-channel chemistry, and single-channel chemistry, m is 4 or 2. In other embodiments, m is 1, 3, or greater than 4.

[0099] For example, consider using two different image channels to implement a sequencing run: a blue channel and a green channel. Then, in each sequencing cycle, the sequencing run produces blue and green images. Thus, for a series of k sequencing cycles of the sequencing run, k pairs of blue and green image sequences are produced as output and stored as sequencing images. Therefore, sequences of k pairs of blue and green image patches are generated for patch-level processing by a neural network-based base detector 100.

[0100] For a single iteration of base detection (or a single instance of forward pass or a single forward traversal), the input image data for the neural network-based base detector 100 includes data from a sliding window used for multiple sequencing cycles. The sliding window may include, for example, the current sequencing cycle, one or more previous sequencing cycles, and one or more subsequent sequencing cycles.

[0101] In one specific implementation, the input image data includes data for three sequencing cycles, such that the data to be detected for the current (time t) sequencing cycle is accompanied by (i) data for the left wing / context / previous / before / in front (time t-1) sequencing cycle and (ii) data for the right wing / context / next / follower / subsequent (time t+1) sequencing cycle.

[0102] In another specific implementation, the input image data includes data from 5 sequencing cycles, such that the data of the current (time t) sequencing cycle for base detection is accompanied by (i) the data of the first left wing / context / previous / before / before (time t-1) sequencing cycle, (ii) the data of the second left wing / context / previous / before / before (time t-2) sequencing cycle, (iii) the data of the first right wing / context / next / follower / subsequent (time t+1) sequencing cycle, and (iv) the data of the second right wing / context / next / follower / subsequent (time t+2) sequencing cycle.

[0103] In another embodiment, the input image data includes data from seven sequencing cycles, such that the data from the current (time t) sequencing cycle for base detection is accompanied by data from (i) the first left-wing / context / previous / before / before (time t-1) sequencing cycle, (ii) the second left-wing / context / previous / before / before (time t-2) sequencing cycle, (iii) the third left-wing / context / previous / before / before (time t-3) sequencing cycle, (iv) the first right-wing / context / next / follower / subsequent (time t+1) sequencing cycle, (v) the second right-wing / context / next / follower / subsequent (time t+2) sequencing cycle, and (vi) the third right-wing / context / next / follower / subsequent (time t+3) sequencing cycle. In other embodiments, the input image data includes data for a single sequencing cycle. In other specific implementations, the input image data includes data for 10, 15, 20, 30, 58, 75, 92, 130, 168, 175, 209, 225, 230, 275, 318, 325, 330, 525, or 625 sequencing cycles.

[0104] According to one embodiment, a neural network-based base detector 100 processes image patches through its convolutional layers and generates alternative representations. The output layer (e.g., a softmax layer) then uses these alternative representations to generate base detections for the current (time t) sequencing cycle or each sequencing cycle within a sequencing cycle, i.e., the current (time t) sequencing cycle, the first preceding sequencing cycle, and the second preceding (time t-1, time t-2) sequencing cycles, as well as the first subsequent sequencing cycle and the second subsequent (time t+1, time t+2) sequencing cycles. The resulting base detections form sequencing reads.

[0105] In one embodiment, the neural network-based base detector 100 outputs the base detection of a single target cluster for a specific sequencing cycle. In another embodiment, the neural network-based base detector 100 outputs the base detection of each of multiple target clusters for a specific sequencing cycle. In yet another embodiment, the neural network-based base detector 100 outputs the base detection of each of multiple target clusters for each sequencing cycle in multiple sequencing cycles, thereby generating a base detection sequence for each target cluster.

[0106] In one embodiment, the neural network-based base detector 100 is a multilayer perceptron. In another embodiment, the neural network-based base detector 100 is a feedforward neural network. In yet another embodiment, the neural network-based base detector 100 is a fully connected neural network. In yet another embodiment, the neural network-based base detector 100 is a fully convolutional neural network. In yet another embodiment, the neural network-based base detector 100 is a semantic segmentation neural network. In yet another embodiment, the neural network-based base detector 100 is a generative adversarial network (GAN).

[0107] In one embodiment, the neural network-based base detector 100 is a convolutional neural network (CNN) with multiple convolutional layers. In another embodiment, the neural network-based base detector 100 is a recurrent neural network (RNN), such as a long short-term memory network (LSTM), a bidirectional LSTM (Bi-LSTM), or a gated recurrent unit (GRU). In yet another embodiment, the neural network-based base detector 100 includes both CNNs and RNNs.

[0108] In other specific implementations, the neural network-based base detector 100 may use 1D convolution, 2D convolution, 3D convolution, 4D convolution, 5D convolution, dilated or dilated convolution, transposed convolution, depthwise separable convolution, pointwise convolution, 1×1 convolution, grouped convolution, flattened convolution, spatial and cross-channel convolution, shuffled grouped convolution, spatially separable convolution, and deconvolution. The neural network-based base detector 100 may use one or more loss functions, such as logistic regression / logarithmic loss, multi-class cross-entropy / softmax loss, binary cross-entropy loss, mean squared error loss, L1 loss, L2 loss, smoothed L1 loss, and Huber loss. The neural network-based base detector 100 may use any parallelism, efficiency, and compression scheme, such as TFRecords, compressed encoding (e.g., PNG), sharpening, parallel detection of map transformations, batch processing, prefetching, model parallelism, data parallelism, and synchronous / asynchronous stochastic gradient descent (SGD). The neural network-based base detector 100 may include upsampling layers, downsampling layers, recursive connections, gates and gate memory cells (such as LSTM or GRU), residual blocks, residual connections, high-speed connections, skip connections, peephole connections, activation functions (e.g., nonlinear transformation functions such as Modified Linear Unit (ReLU), Leaking ReLU, Exponential Liner Unit (ELU), sigmoid, and hyperbolic tangent (tanh)), batch normalization layers, regularization layers, dropout layers, pooling layers (e.g., max or average pooling), global average pooling layers, and attention mechanisms.

[0109] The neural network-based base detector 100 is trained using a gradient update technique based on backpropagation. Exemplary gradient descent techniques that can be used to train the neural network-based base detector 100 include stochastic gradient descent, batch gradient descent, and mini-batch gradient descent. Some examples of gradient descent optimization algorithms that can be used to train the neural network-based base detector 100 are Momentum, Nesterov accelerated gradient, Adagrad, Adadelta, RMSprop, Adam, AdaMax, Nadam, and AMSGrad.

[0110] In one specific implementation, the neural network-based base detector 100 uses a specialized architecture to isolate data processing across different sequencing cycles. The motivation for using this specialized architecture is first described. As mentioned above, the neural network-based base detector 100 processes image patches for the current sequencing cycle, one or more preceding sequencing cycles, and one or more subsequent sequencing cycles. Data from the additional sequencing cycles provides sequence-specific context. The neural network-based base detector 100 learns the sequence-specific context during training and performs base detection based on that context. Furthermore, data from the preceding and subsequent sequencing cycles provide second-order contributions of the pre-phase and phasing signals to the current sequencing cycle.

[0111] However, images captured at different sequencing cycles and in different image channels are misaligned relative to each other and have residual registration errors. To address this misalignment, specialized architectures include spatial convolutional layers that do not mix information between sequencing cycles and only mix information within a sequencing cycle.

[0112] Spatial convolutional layers (or spatial logic) use so-called "isolated convolutions," which achieve isolation by processing data from each of multiple sequencing cycles independently via "dedicated, non-shared" convolutional sequences. Isolated convolutions convolve only the data and resulting feature maps from a given sequencing cycle (i.e., within that cycle), without convolving the data and resulting feature maps from any other sequencing cycles.

[0113] For example, consider input image data comprising (i) the current image patch of the current (time t) sequencing cycle for which base detection is to be performed, (ii) the previous image patch of the previous (time t-1) sequencing cycle, and (iii) the next image patch of the next (time t+1) sequencing cycle. The specialized architecture then initiates three separate convolutional pipelines: the current convolutional pipeline, the previous convolutional pipeline, and the next convolutional pipeline. The current data processing pipeline receives the current image patch of the current (time t) sequencing cycle as input and processes it independently through multiple spatial convolutional layers to produce a so-called "current spatial convolutional representation" as the output of the final spatial convolutional layer. The previous convolutional pipeline receives the previous image patch of the previous (time t-1) sequencing cycle as input and processes it independently through multiple spatial convolutional layers to produce a so-called "previous spatial convolutional representation" as the output of the final spatial convolutional layer. The next convolutional pipeline receives the next image patch from the next (time t+1) sequencing cycle as input and processes the next data independently through multiple spatial convolutional layers to produce the so-called "next spatial convolutional representation" as the output of the final spatial convolutional layer.

[0114] In some implementations, the current convolutional pipeline, the previous convolutional pipeline, and the next convolutional pipeline are executed in parallel. In some implementations, spatial convolutional layers are part of a spatial convolutional network (or subnetwork) within a specialized architecture.

[0115] The neural network-based base detector 100 also includes a temporal convolutional layer (or temporal logic) that mixes information between sequencing cycles (i.e., between cycles). The temporal convolutional layer receives its input from the spatial convolutional network and operates on the spatial convolutional representation produced by the final spatial convolutional layer of the corresponding data processing pipeline.

[0116] The inter-cycle operability freedom of temporal convolutional layers stems from the fact that misalignment attributes are cleared from the spatial convolutional representation by stacking or cascading isolated convolutions performed by a sequence of spatial convolutional layers, which exist in the image data fed into the spatial convolutional network as input.

[0117] Temporal convolutional layers use so-called "combined convolutions," which convolve input channels in subsequent inputs group by group on a sliding window basis. In one specific implementation, these subsequent inputs are the subsequent outputs produced by previous spatial or temporal convolutional layers.

[0118] In some implementations, temporal convolutional layers are part of a temporal convolutional network (or subnetwork) within a specialized architecture. The temporal convolutional network receives its input from a spatial convolutional network. In one implementation, the first temporal convolutional layer of the temporal convolutional network combines spatial convolutional representations between sequencing cycles group by group. In another implementation, subsequent temporal convolutional layers of the temporal convolutional network combine subsequent outputs of previous temporal convolutional layers. The output of the final temporal convolutional layer is fed into an output layer that produces the final output. The output is used for base detection of one or more clusters at one or more sequencing cycles.

[0119] Further details regarding the neural network-based base detector 100 can be found in U.S. Provisional Patent Application No. 62 / 821,766, filed March 21, 2019, entitled “ARTIFICIAL INTELLIGENCE-BASED SEQUENCING” (Attorney’s File No. ILLM 1008-9 / IP-1752-PRV), which is incorporated herein by reference.

[0120] Compressed Network

[0121] As described above, the specialized architecture of the neural network-based base detector 100 is a sliding window that processes image patches in the corresponding sequencing cycle. There is overlap between sequencing cycles in subsequent sliding windows. This causes the neural network-based base detector 100 to redundantly process image patches for overlapping sequencing cycles. This, in turn, leads to a waste of computational resources. For example, in one specific implementation, each spatial convolutional layer of the neural network-based base detector 100 performs nearly 100 million multiplication operations. Then, for a window of 5 sequencing cycles and a cascade (or sequence) of 7 spatial convolutional layers, the spatial convolutional neural network performs approximately 620 million multiplication operations. Furthermore, the temporal convolutional neural network performs approximately 10 million multiplication operations.

[0122] Since the image data of cycle N-1 in the current sliding window (or the current iteration of base detection) was processed as cycle N in the previous sliding window (or the previous iteration of base detection), there is an opportunity to store intermediate results of the processing completed in the current sliding window and their intermediate results in subsequent sliding windows, thereby bypassing (or avoiding) redundant processing (or reprocessing) of the input image data for use in overlapping sequencing cycles between subsequent sliding windows.

[0123] However, intermediate results require terabytes of data, necessitating impractical storage. To overcome this technical problem, the disclosed technique proposes compressing intermediate results when the neural network-based base detector 100 first generates them, and reusing the compressed intermediate results in subsequent sliding windows to avoid redundant computation, thus avoiding (or only generating) regeneration of intermediate results. In some specific implementations, the disclosed technique saves approximately 80% of convolutions in the spatial network of the neural network-based base detector 100. In one specific implementation, an 80% saving in spatial convolutions can be observed when the compression logic and the reuse of compressed feature maps in subsequent sequencing cycles are used for input windows of 5 sequencing cycles (e.g., cycle N, cycle N+1, cycle N-1, cycle N+2, cycle N-2). In another specific implementation, a 90% saving was observed in spatial convolution when the compression logic and the reuse of compressed feature maps in subsequent sequencing cycles were used for an input window of 10 sequencing cycles (e.g., cycle N, cycle N+1, cycle N-1, cycle N+2, cycle N-2, cycle N+3, and cycle N-3). That is, the larger the window size, the greater the saving from using the compression logic and reusing the compressed feature maps, and the larger the window size, the better the base detection performance due to the incorporation of greater context from the additional flanking cycles. Therefore, for a given computational power, the greater saving from a larger window improves overall performance.

[0124] The computational efficiency and compact footprint resulting from compression logic facilitate the hardware implementation of the neural network-based base detector 100 on resource-constrained processors such as central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), coarse-grained reconfigurable architectures (CGRAs), application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), and digital signal processors (DSPs).

[0125] The computation preserved by the compression logic allows for the incorporation of more convolutional operators in the neural network-based base detector 100. Examples include adding more convolutional filters in spatial and temporal convolutional layers, increasing the size of the convolutional filters, and increasing the number of spatial and temporal convolutional layers. The additional convolutional operations improve the intensity pattern detection and overall base detection accuracy of the neural network-based base detector 100.

[0126] The computation preserved by the compression logic also allows for the expansion of the input image data to the target sliding window to include an increased number of sequencing cycles. The expanded sliding window extends the base detection context by introducing additional image patches from the additional flanking sequencing cycles.

[0127] Furthermore, in contrast to the original intermediate results, any decrease in accuracy that may occur due to the use of compressed intermediate results is compensated for by combining additional convolution operators and the expansion of the sliding window.

[0128] Figure 1A A specific implementation of the disclosed compression logic is shown, which generates a compressed spatial atlas for the first iteration of base detection. In the example shown, the first window of the sequencing cycles includes sequencing cycles 1, 2, 3, 4, and 5. The corresponding image patches 102, 112, 122, 132, and 142 (or per-cycle analyte channel sets) for the respective sequencing cycles 1, 2, 3, 4, and 5 are processed by spatial logic 104 (or a spatial network, spatial sub-network, or spatial convolutional neural network) to generate corresponding spatial maps 106, 116, 126, 136, and 146 (or intermediate results, spatial output sets, or spatial feature atlases) for the respective sequencing cycles 1, 2, 3, 4, and 5. The spatial convolutional network 104 can use 1D, 2D, or 3D convolutions.

[0129] Spatial logic 104 comprises a sequence (or cascade) of spatial convolutional layers. Each spatial convolutional layer has a filter bank with multiple spatial convolutional filters that perform isolated convolutions. Therefore, each spatial convolutional layer produces multiple spatial feature maps as output. The number of spatial feature maps produced by the target spatial convolutional layer is a function of the number of spatial convolutional filters configured in the target spatial convolutional layer. For example, if the target spatial convolutional layer has 14 spatial convolutional filters, then the target spatial convolutional layer produces 14 spatial feature maps. From an overall perspective, 14 spatial feature maps can be considered as a spatial feature map volume (or tensor) with 14 channels (or depth dimension = 14).

[0130] Furthermore, the next spatial convolutional layer after the target spatial convolutional layer can also be configured with 14 spatial convolutional filters. In this case, the next spatial convolutional layer will process the 14 spatial feature maps generated by the target spatial convolutional layer as input, and it will generate 14 new spatial feature maps as output. Figure 1A Five spatial feature maps 106, 116, 126, 136, and 146 are shown, generated by the final spatial convolutional layer of spatial network 104 for the corresponding sequencing cycles 1, 2, 3, 4, and 5. In the example shown, each of the five spatial feature maps 106, 116, 126, 136, and 146 has 14 feature maps.

[0131] Figure 1D The diagram illustrates the sequences of seven spatial feature map sets 196a, 196b, 196c, 196d, 196e, 196f, and 196g generated by a cascade of seven spatial convolutional layers of spatial network 104. The input patch data 194 for each cycle of the target sequencing loop i has a spatial dimension of 115 × 115 and a depth dimension of 2 (due to the two image channels in the original sequencing image). In one specific implementation, each of the seven spatial convolutional layers uses a 3 × 3 convolution, reducing the spatial dimension of the subsequent spatial feature map volume by 2, for example, from 10 × 10 to 8 × 8.

[0132] The first spatial feature map volume 196a has a spatial dimension of 113×113 (i.e., a 3×3 convolution from 115×115 reduced by the first spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels resulting from the 14 spatial convolutional filters in the first spatial convolutional layer). The second spatial feature map volume 196b has a spatial dimension of 111×111 (i.e., a 3×3 convolution from 113×113 reduced by the second spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels resulting from the 14 spatial convolutional filters in the second spatial convolutional layer). The third spatial feature map volume 196c has a spatial dimension of 109×109 (i.e., a 3×3 convolution from 111×111 reduced by the third spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels resulting from the 14 spatial convolutional filters in the third spatial convolutional layer). The fourth spatial feature map volume 196d has a spatial dimension of 107×107 (i.e., a 3×3 convolution from 109×109 reduced to the fourth spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels resulting from the 14 spatial convolutional filters in the fourth spatial convolutional layer). The fifth spatial feature map volume 196e has a spatial dimension of 105×105 (i.e., a 3×3 convolution from 107×107 reduced to the fifth spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels resulting from the 14 spatial convolutional filters in the fifth spatial convolutional layer). The sixth spatial feature map volume 196f has a spatial dimension of 103×103 (i.e., a 3×3 convolution from 105×105 reduced to the sixth spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels resulting from the 14 spatial convolutional filters in the sixth spatial convolutional layer). The seventh spatial feature map volume of 196g has a spatial dimension of 101×101 (i.e., a 3×3 convolution of the seventh spatial convolution layer reduced from 103×103) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the seventh spatial convolution layer).

[0133] Similar to Figure 1A The multi-cycle example shown, for five sequencing cycles 1, 2, 3, 4, and 5, and five per-cycle image patches 102, 112, 122, 132, and 142, spatial logic 104 generates five corresponding sequences for seven spatial feature map volumes 196a, 196b, 196c, 196d, 196e, 196f, and 196g, respectively. Figure 1A Spatial diagrams 106, 116, 126, 136, and 146 are equivalent to Figure 1D The final spatial feature map volume is 196g for 5 individual instances.

[0134] Compression logic 108 (or compression network, compression subnetwork, compression layer, or squeeze layer) processes the output of spatial logic 104 and generates a compressed representation of the output. In one specific implementation, compression network 108 includes compressed convolutional layers that reduce the depth dimension of the feature maps generated by spatial network 104.

[0135] For example, in Figure 1A In the original text, spatial maps 106, 116, 126, 136, and 146 have a depth dimension of 14 (i.e., each space outputs fourteen feature maps or fourteen channels). The compression network 108 decays spatial maps 106, 116, 126, 136, and 146 into corresponding compressed spatial maps 110, 120, 130, 140, and 150 for the corresponding sequencing cycles 1, 2, 3, 4, and 5. Each of the compressed spatial maps 110, 120, 130, 140, and 150 has a depth dimension of 2 (i.e., each compressed space outputs two feature maps or two channels). In other embodiments, the compressed spatial maps 110, 120, 130, 140, and 150 may have a depth dimension of 3 or 4 (i.e., each compressed space outputs three or four feature maps or three or four channels). In other implementations, the compressed spatial sets 110, 120, 130, 140, and 150 may have a depth dimension of 1 (i.e., each compressed spatial output has one feature map or one channel). In one implementation, the compression layer 108 does not include an activation function similar to ReLU. In other implementations, it may include an activation function. In other implementations, the compression logic 108 may configure the corresponding compressed spatial sets to each have more than four feature maps.

[0136] Now let's discuss how compression logic 108 generates compressed output.

[0137] In one specific implementation, the compression logic 108 uses 1×1 convolutions to reduce the number of feature maps (i.e., the number of depth dimensions or channels) while introducing non-linearity. The kernel size of a 1×1 convolution is 1. A 1×1 convolution can transform the volumetric depth into another squeezed or expanded representation without changing the spatial dimensions. The 1×1 convolution operation is analogous to a fully connected linear layer across the input channels. This is useful for mapping from feature maps with many channels to fewer feature maps. Figure 1E In this approach, a single 1×1 convolution is applied to an input tensor with two feature maps. The 1×1 convolution compresses the two-channel input into a single-channel output.

[0138] The number of compressed outputs (or compressed feature maps, or compressed spatial maps, or compressed temporal maps) generated by compression layer 108 is a function of the number of 1×1 convolutional filters (or compressed convolutional filters, or compressed filters) configured in compression layer 108. Figure 1FIn the compression layer 108, there are two 1×1 convolutional filters 198a and 198b. The first 1×1 convolutional filter 198a processes the spatial feature volume 196g with 14 feature maps and generates a first feature map 199a while maintaining a spatial dimension of 101×101. The second 1×1 convolutional filter 198b also processes the spatial feature volume 196g with 14 feature maps and generates a second feature map 199b while maintaining a spatial dimension of 101×101. Therefore, the compression layer 108 reduces the spatial feature volume 196g with 14 feature maps to a compressed output with two spatial feature maps 199a and 199b (i.e., compression ratio = 7).

[0139] From a time series perspective, sequencing cycle 5 is the central sequencing cycle (N), sequencing cycles 1 and 2 are left-wing sequencing cycles (N-1, N-2), and sequencing cycles 4 and 5 are left-wing sequencing cycles (N+1, N+2). Therefore, a central compressed output of 130 is generated for the central sequencing cycle (N), a left-wing compressed output of 120 is generated for the left-wing sequencing cycle (N-1), another left-wing compressed output of 110 is generated for the other left-wing sequencing cycle (N-2), a right-wing compressed output of 140 is generated for the right-wing sequencing cycle (N+1), and another right-wing compressed output of 150 is generated for the other right-wing sequencing cycle (N+2).

[0140] From a pipeline perspective, the neural network-based base detector 100 executes five parallel and independent pipelines, which process image patches 102, 112, 122, 132, and 142 respectively via spatial logic 104 and compression logic 108 (e.g., as multi-threaded execution or multi-cluster execution based on data parallelism). Therefore, five compressed outputs 110, 120, 130, 140, and 150 are generated by the neural network-based base detector 100 separately, simultaneously, and independently.

[0141] In some implementations, the compression layer 108 can be considered as the final spatial convolutional layer of the spatial network 104. In other implementations, the compression network 108 can be considered as a separate network inside or outside the specialized architecture of the neural network-based base detector 100.

[0142] Figure 1B A specific implementation is shown of processing compressed spatial atlases 110, 120, 130, 140, and 150 via temporal logic 160 (or temporal network, temporal subnetwork, or temporal convolutional neural network) using a neural network-based base detector 100. Temporal logic 160 processes multiple sets of subsequent compressed spatial atlases on a sliding window basis. For example, in Figure 1BIn the sequence, temporal logic 160 processes the compressed spatial atlases 110, 120, and 130 of the first group / window for the corresponding sequencing cycles 1, 2, and 3, and generates a temporal plot 172 (or a temporal atlas or a temporal feature map or a temporal feature atlas) as output. Temporal logic 160 processes the compressed spatial atlases 120, 130, and 140 of the second group / window for the corresponding sequencing cycles 2, 3, and 4, and generates a temporal plot 174 as output. Temporal logic 160 processes the compressed spatial atlases 130, 140, and 150 of the third group / window for the corresponding sequencing cycles 3, 4, and 5, and generates a temporal plot 176 as output. The temporal convolutional network 160 can use 1D, 2D, or 3D convolutions.

[0143] Figure 1B The three instances of temporal logic 160 shown represent three filter banks of the first temporal convolutional layer of the temporal network 160. The first filter bank applies the first set of temporal convolutional filters to the first set of compressed spatial graphs 110, 120, and 130, generating the first set of temporal graphs 172. The second filter bank applies the second set of temporal convolutional filters to the second set of compressed spatial graphs 120, 130, and 140, generating the second set of temporal graphs 174. The third filter bank applies the third set of temporal convolutional filters to the third set of compressed spatial graphs 130, 140, and 150, generating the third set of temporal graphs 176.

[0144] The first set of time maps 172, the second set of time maps 174, and the third set of time maps 176 are processed by time logic 160 into a single set to generate time map 182. Figure 1B The fourth instance of the temporal logic 160 shown represents the second temporal convolutional layer of the temporal network 160, which produces outputs for all sequencing cycles 1, 2, 3, 4, and 5, with each cycle's image patch pair being fed as input for all sequencing cycles. Figure 1A The neural network-based base detector 100 in the example.

[0145] The temporal network 160 has a cascade of temporal convolutional layers (e.g., 2, 3, 4, 5 or more temporal convolutional layers arranged sequentially). The cascade of temporal convolutional layers processes data in a hierarchical manner with different grouping levels. That is, at a given level, a sliding window method processes the input group by group at that level to generate the output that is subsequently processed group by group at the next level in a sliding window manner.

[0146] The temporal convolutional layer is configured with a temporal convolutional filter that performs ensemble convolution. Ensemble convolution mixes information between feature maps spanning multiple sequencing cycles. Ensemble convolution combines data between subsequent sequencing cycles in the target group / window at the current level in the temporal network 160. For example, the first temporal convolutional layer combines first-group compressed spatial maps 110, 120, and 130 for first-group sequencing cycles 1, 2, and 3 to generate a first-group temporal map 172; combines second-group compressed spatial maps 120, 130, and 140 for second-group sequencing cycles 2, 3, and 4 to generate a second-group temporal map 174; and combines third-group compressed spatial maps 130, 140, and 150 for third-group sequencing cycles 3, 4, and 5 to generate a third-group temporal map 176.

[0147] Combinatorial convolutions also combine data from subsequent sequencing cycles within the target group / window at the current level in the temporal network 160. For example, the second temporal convolutional layer combines the first, second, and third timemaps (172, 174, and 176) into a final timemap (182). In the second level, sequencing cycles from the first, second, and third groups / windows of the first level are grouped within the first group / window of sequencing cycles 1, 2, 3, 4, and 5.

[0148] Combining convolutions are configured with as many kernels as there are inputs to be combined (i.e., the depth columns or fibers of the temporal convolutional filters match the number of inputs in the target group / window of the current level). For example, when a temporal convolutional layer combines three compressed spatial maps, it uses temporal convolutional filters, each with three kernels, which perform element-wise multiplication and summation across the entire depth of the three compressed spatial maps.

[0149] The last set of time maps 182 is generated by the final (or last) temporal convolutional layer of the temporal network 160. Figure 1C A specific implementation is shown that processes the final time map 182 through the disclosed output logic 190 (or output layer, output network, or output subnetwork) to generate base detection classification data. In one implementation, base detection is performed simultaneously on multiple clusters for one or more sequencing cycles. Figure 1CIn the example shown, base detection 192 is generated for many clusters only for the central sequencing cycle 3. In other embodiments, according to one embodiment, the disclosed technique causes the output logic 190 to generate base detection for a given input window, not only for the central sequencing cycle but also for the flanking sequencing cycles (as indicated by the optional dashed lines). For example, in one embodiment, the disclosed technique generates base detection simultaneously for cycle N, cycle N+1, cycle N-1, cycle N+2, cycle N-2, etc., for a given input window. That is, a single forward propagation / traversal / base detection iteration of the neural network-based base detector 102 generates base detection for multiple sequencing cycles in the input window of the sequencing cycles, which is referred to herein as "many-to-many base detection".

[0150] Examples of output layer 190 include the softmax function, log-softmax function, ensemble output averaging function, multilayer perceptron uncertainty function, Bayesian Gaussian distribution function, and cluster strength function. In one specific implementation, output layer 190 generates per-cluster, per-cycle probability quadruples for each cluster and for each sequencing cycle.

[0151] The following discussion focuses on per-cluster, per-cycle probability quadruples using the softmax function as an example. The softmax function will be explained first, followed by the per-cluster, per-cycle probability quadruples.

[0152] The softmax function is a preferred function for multi-class classification. It calculates the probability of each target class relative to all possible target classes. The output of the softmax function ranges between 0 and 1, and the sum of all probabilities equals 1. The softmax function calculates the exponent of a given input value and the sum of the exponents of all input values. The ratio of the exponent of the input values ​​to the sum of the exponents is the output of the softmax function, referred to in this paper as "exponential normalization".

[0153] Formally, training a so-called softmax classifier is about regressing to class probabilities, not to the true classifier, because it doesn't return the classes themselves, but rather the confidence predictions of the probabilities for each class. The softmax function takes a class of values ​​and transforms them into probabilities that sum to 1. The softmax function compresses an arbitrary n-dimensional vector of real values ​​into an n-dimensional vector of real values ​​in the range of 0 to 1. Therefore, using the softmax function ensures that the output is a valid, exponentially normalized probability mass function (non-negative and summing to 1).

[0154] Intuitively, the softmax function is a "soft" version of the maximization function. The term "soft" comes from the fact that the softmax function is continuous and differentiable. Instead of choosing a maximum element, it decomposes a vector into parts of a whole, where the largest input element receives a proportionally larger value, and the other input element receives a smaller proportional value. This property of the output probability distribution makes the softmax function suitable for probabilistic interpretation in classification tasks.

[0155] Consider z as the input vector to the softmax layer. The number of units in the softmax layer is the number of nodes in the softmax layer. Therefore, the length of the z vector is the number of units in the softmax layer (if there are 10 output units, there are 10 z elements).

[0156] For an n-dimensional vector Z = [z1, z2, ... z2] n The softmax function uses exponential normalization (exp) to produce another n-dimensional vector p(Z), whose normalized values ​​are in the range [0,1] and are summed to unit:

[0157] as well as,

[0158]

[0159] Figure 1G An example softmax function is shown. The softmax function is applied to three classes as follows: It should be noted that the three outputs always sum to 1. Therefore, they define a discrete probability mass function.

[0160] The probability of a base being incorporated into a specific cluster in a specific sequencing cycle is identified by a specific cluster probability quadruple as A, C, T, and G. When the output layer of the neural network-based base detector 100 uses the softmax function, the probabilities in the cluster-per-cluster, per-cycle probability quadruple are exponentially normalized classification scores, and their sum is in units. Figure 1H Exemplary per-cluster, per-cycle probability quadruples 123 are depicted, generated by the softmax functions of cluster 1 (121, shown in brown) and sequencing cycles 1 through S (122), respectively. In other words, the first subset of sequencing cycles comprises S sequencing cycles.

[0161] The unreliable cluster identifier 125 identifies unreliable clusters based on filtered values ​​generated from per cluster, per cycle probability quadruples. In this application, per cluster, per cycle probability quadruples are also referred to as base detection classification scores or normalized base detection classification scores or initial base detection classification scores or normalized initial base detection classification scores or initial base detections.

[0162] The filter calculator 127 determines the filter value for each per-cluster, per-cycle probability quadruple based on its recognition probability, thereby generating a sequence of filter values ​​for each cluster. The sequence of filter values ​​is stored as filter value 124.

[0163] The filter value for each cluster, each cycle probability quadruple is determined based on calculations involving one or more probabilities. In one specific implementation, the calculation used by the filter calculator 127 is subtraction. For example, in Figure 1H In the specific implementation shown, the filter value for each cluster and each cycle probability quadruple is determined by subtracting the second highest probability (shown in blue) from the highest probability (shown in magenta).

[0164] In another embodiment, the calculation used by filter calculator 116 is division. For example, the filter value for each cluster, each cycle probability quadruple is determined as the ratio of the highest probability (shown in magenta) to the second highest probability (shown in blue). In yet another embodiment, the calculation used by filter calculator 127 is addition. In yet another embodiment, the calculation used by filter calculator 127 is multiplication.

[0165] In one implementation, the filter calculator 127 uses a filtering function to generate the filtered value 124. In one example, the filtering function is a chirped filter in which chirp is defined as the ratio of the brightest detected intensity to the sum of the brightest and second brightest detected intensities. In another example, the filtering function is at least one of the following: the maximum logarithmic probability function, the least squares error function, the average signal-to-noise ratio (SNR), and the minimum absolute error function.

[0166] Unreliable cluster identifier 125 uses filter value 124 to identify some of the multiple clusters as unreliable cluster 128. The data identifying unreliable cluster 128 can be in a computer-readable format or on a medium. Unreliable clusters can be identified by instrument ID, instrument run number, flow cell ID, channel number, block number, cluster X coordinate, cluster Y coordinate, and unique molecular identifier (UMI). Unreliable cluster identifier 125 identifies those clusters among the multiple clusters that contain a "G" filter value below a threshold "H" as unreliable clusters. In one embodiment, "G" ranges from 1 to 5. In another embodiment, "H" ranges from 0.5 to 0.99. In one embodiment, unreliable cluster 128 identifies those pixels corresponding to unreliable clusters (i.e., those depicting their intensity emission). As described later in this application, such pixels are filtered out by filter logic 502.

[0167] An unreliable cluster is a low-quality cluster that emits a certain amount of the desired signal that is insignificant compared to the background signal. The signal-to-noise ratio (SNR) of an unreliable cluster is quite low, for example, less than 1. In some embodiments, an unreliable cluster may not produce any amount of the desired signal. In other embodiments, an unreliable cluster may produce a signal amount that is very low relative to the background. In one embodiment, the signal is an optical signal and is intended to include, for example, fluorescence, emission, scattering, or absorption signals. The signal level refers to the amount or quantity of detected energy or encoded information with desired or predefined characteristics. For example, an optical signal can be quantized by one or more of intensity, wavelength, energy, frequency, power, brightness, etc. Other signals can be quantized based on characteristics such as voltage, current, electric field strength, magnetic field strength, frequency, power, temperature, etc. A lack of signal in an unreliable cluster is understood as a signal level of zero or a signal level that is not significantly different from noise.

[0168] There are many possible causes for low-quality signals in unreliable clusters. If polymerase chain reaction (PCR) errors already exist in colony amplification, causing a significant proportion of molecules in the ~1000 molecules of an unreliable cluster to contain different bases at a certain position, a two-base signal can be observed—this is interpreted as a sign of low quality and is called phase error. Phase error occurs when individual molecules in an unreliable cluster do not incorporate nucleotides in a cycle (e.g., due to incomplete removal of the 3' terminator, called phase stabilization) and then lag behind other molecules, or when individual molecules incorporate more than one nucleotide in a single cycle (e.g., due to nucleotide incorporation without effective 3' blocking, called phase stabilization). This leads to a loss of synchronization in the reads of the sequence copies. The proportion of unreliable clusters affected by phase stabilization and pre-phase stabilization increases with cycle number, which is a major reason why read quality tends to decline at high cycle numbers.

[0169] Fading also leads to unreliable clusters. Fading is the exponential decay of the signal strength of unreliable clusters as a function of cycle number. As sequencing progresses, strands in unreliable clusters are overwashed, exposed to laser radiation that produces reactive substances, and subjected to harsh environmental conditions. All of these result in the gradual loss of fragments from unreliable clusters, thereby reducing their signal strength.

[0170] Unreliable clusters also arise from underdeveloped colonies, i.e., small cluster sizes that produce empty or partially filled wells in the patterned flow cell. Specifically, in some embodiments, unreliable clusters indicate empty, polyclonal, and dark wells in the patterned flow cell. Unreliable clusters also arise from overlapping colonies caused by non-exclusive amplification. Unreliable clusters are also caused by insufficient or uneven illumination, for example, due to their location at the edge of the flow cell. Unreliable clusters are also caused by impurities in the flow cell that confuse the emitted signal. Unreliable clusters also include polyclonal clusters when multiple clusters are deposited in the same well.

[0171] The first window of the sequencing cycles includes sequencing cycles 1, 2, 3, 4, and 5, and the first iteration of base detection produces base detection 192 for the central sequencing cycle 3. The second window of the sequencing cycles includes sequencing cycles 2, 3, 4, 5, and 6, and the second iteration of base detection produces base detection 292 for the central sequencing cycle 4. Therefore, sequencing cycles 2, 3, 4, and 5 are overlapping sequencing cycles between the first and second windows of base detection or between the second and third iterations.

[0172] The disclosed base detection system and technique store compressed spatial atlases 120, 130, 140, and 150 generated during the first iteration of base detection for the corresponding sequencing cycles 2, 3, 4, and 5 in memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM). During the second iteration of base detection, the disclosed base detection system and technique do not reprocess the corresponding input image patches 112, 122, 132, and 142 of overlapping cycles 2, 3, 4, and 5 via spatial network 104. Instead, during the second iteration of base detection, the disclosed base detection system and technique reuse the previously generated compressed spatial atlases 120, 130, 140, and 150 instead of the corresponding input image patches 112, 122, 132, and 142.

[0173] The compression logic is further configured to require that the compressed spatial atlases 120, 130, 140, and 150, as well as the corresponding input image patches 112, 122, 132, and 142, have the same number of per-cycle feature maps / channels. This ensures that the compressed spatial atlases 120, 130, 140, and 150 are lossless representations of the corresponding input image patches 112, 122, 132, and 142. That is, if the corresponding input image patches 112, 122, 132, and 142 each have two feature maps / channels, then the compression logic 108 configures the compressed spatial atlases 120, 130, 140, and 150 to also have two feature maps / channels. Similarly, if the corresponding input image patches 112, 122, 132, and 142 each have three feature maps / channels, then the compression logic 108 configures the compressed spatial atlases 120, 130, 140, and 150 to also have three feature maps / channels. Similarly, if the corresponding input image patches 112, 122, 132 and 142 each have four feature maps / channels, then the compression logic 108 configures the compressed spatial atlases 120, 130, 140 and 150 to also have four feature maps / channels.

[0174] Figure 2A As shown, during the second iteration of base detection, the input image data 222 is processed by spatial logic 104 and compression logic 108, generating a spatial map 226 and a corresponding compressed spatial map 230 only for non-overlapping sequencing loop 6. Therefore, the input image patches 112, 122, 132, and 142 used for overlapping loops 2, 3, 4, and 5 (highlighted in gray in the legend) are not reprocessed to avoid redundant convolutions.

[0175] Figure 2B The compressed spatial atlases 120, 130, 140, and 150 generated during the first iteration of base detection are shown to be used in conjunction with a compressed spatial atlas 230 generated during the second iteration of base detection to generate base detection 292 for central sequencing cycle 4. Figure 2B In the process, the filtered temporal atlases 174, 176, and 278 are generated by the first temporal convolutional layer of the temporal network 160 in a manner similar to that of the time-series network 160. Figure 1B The method discussed above is used to generate the network. It consists of a second temporal convolutional layer and a final temporal convolutional layer of the temporal network 160, similar to the method described above. Figure 1B The method discussed generates time atlases 282. Figure 2C The output layer 190 processes the final time map 282 generated during the second iteration of base detection and produces base detection 292 for central sequencing cycle 4.

[0176] The third window of the sequencing cycle includes sequencing cycles 3, 4, 5, 6, and 7, and the third iteration of base detection produces base detection 392 for the central sequencing cycle 5. Therefore, sequencing cycles 3, 4, 5, and 6 are overlapping sequencing cycles between the second and third windows of base detection or between the second and third iterations.

[0177] The disclosed base detection system and technique store compressed spatial atlases 130, 140, and 150 generated during the first iteration of base detection for the corresponding sequencing cycles 3, 4, and 5, and a compressed spatial atlas 230 generated during the second iteration of base detection for sequencing cycle 6, in a memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM). During the third iteration of base detection, the disclosed base detection system and technique do not reprocess the corresponding input image patches 122, 132, 142, and 222 of overlapping cycles 3, 4, 5, and 6 via spatial network 104. Instead, during the third iteration of base detection, the disclosed base detection system and technique reuse the previously generated compressed spatial atlases 130, 140, 150, and 230 instead of the corresponding input image patches 122, 132, 142, and 222.

[0178] Figure 3A As shown, during the third iteration of base detection, the input image data 322 is processed by spatial logic 104 and compression logic 108, generating a spatial map 326 and a corresponding compressed spatial map 330 only for non-overlapping sequencing loop 7. Therefore, the input image patches 122, 132, 142, and 222 used for overlapping loops 3, 4, 5, and 6 (highlighted in gray in the legend) are not reprocessed to avoid redundant convolutions.

[0179] Figure 3B The compressed spatial atlases 130, 140, 150, and 230 generated during the first and second iterations of base detection are shown to be used in combination with the compressed spatial atlas 330 generated during the third iteration of base detection to generate base detection 392 for central sequencing cycle 5. Figure 3B In the process, the filtered temporal atlases 176, 278, and 378 are processed by the first temporal convolutional layer of the temporal network 160 in a manner similar to that of the time-series network. Figure 1B The method discussed above is used to generate the network. It consists of a second temporal convolutional layer and a final temporal convolutional layer of the temporal network 160, similar to the method described above. Figure 1B The method discussed generates time atlas 382. Figure 3C The output layer 190 processes the final time map 382 generated during the third iteration of base detection and produces base detection 392 for the central sequencing cycle 5.

[0180] Once a compressed spatial atlas has been generated for a given sequencing cycle, it can be reused for base detection in any subsequent sequencing cycle. Figure 4A The fourteenth iteration of base detection for base detection in central sequencing cycle 16 is shown. Figure 4B The compressed spatial maps previously generated for sequencing cycles 1 to 29 are shown to be used to generate the final time map set 482 for base detection in the central sequencing cycle 16. Figure 4C The output layer 190 processes the final time map 482 generated during the fourteenth iteration of base detection and produces base detection 492 for the central sequencing cycle 16.

[0181] Figure 5A A specific implementation is shown that the compressed spatial maps 110, 120, 130, 140, and 150 are filtered using filtering logic 502 to generate corresponding compressed, filtered spatial maps 510, 520, 530, 540, and 550 (depicting only reliable clusters) for the corresponding sequencing cycles 1, 2, 3, 4, and 5 during the first iteration of base detection. As described above, the unreliable cluster data 128 identifies those portions (e.g., pixels) in the spatial maps and compressed spatial maps corresponding to unreliable clusters. For example, such pixels can be identified based on the location coordinates of the unreliable clusters.

[0182] The filtering logic 502 uses the data 128 that identifies unreliable clusters to filter out (or discard or remove) those pixels from the compressed spatial atlases 110, 120, 130, 140, and 150 that correspond to unreliable clusters (i.e., those depicting their intensity emissions). In some implementations, this results in 75% of the pixels being discarded from the compressed spatial atlas, thus preventing many unproductive convolutions.

[0183] exist Figure 5A In this process, filtered temporal maps 572, 574, and 576 (depicting only reliable clusters) are generated from compressed, filtered spatial maps 510, 520, 530, 540, and 550 for base detection in central sequencing cycle 3. The filtered temporal maps 572, 574, and 576 (depicting only reliable clusters) are processed by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above. Figure 1B The method discussed above is used to generate the network. It consists of a second temporal convolutional layer and a final temporal convolutional layer of the temporal network 160, similar to the method described above. Figure 1B The method discussed generates filtered time atlases 582 (depicting only reliable clusters). Figure 5B The output layer 190 processes the final filtered time map 582 generated during the first iteration of base detection and produces base detection 592 for the central sequencing cycle 3.

[0184] Figure 6A A specific implementation is shown that filtering logic 502 is used to filter compressed spatial atlases 120, 130, 140, 150, and 230 to generate corresponding compressed, filtered spatial maps 520, 530, 540, 550, and 650 (depicting only reliable clusters) for sequencing cycles 2, 3, 4, 5, and 6 during the second iteration of base detection. Filtering logic 502 uses data 128 identifying unreliable clusters to filter out (or discard or remove) those pixels from compressed spatial atlases 120, 130, 140, 150, and 230 corresponding to unreliable clusters (i.e., depicting their intensity emission).

[0185] exist Figure 6A In this process, filtered temporal maps 574, 576, and 676 (depicting only reliable clusters) are generated from compressed, filtered spatial maps 520, 530, 540, 550, and 650 for base detection in central sequencing cycle 4. The filtered temporal maps 574, 576, and 676 (depicting only reliable clusters) are processed by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above. Figure 1B The method discussed above is used to generate the network. It consists of a second temporal convolutional layer and a final temporal convolutional layer of the temporal network 160, similar to the method described above. Figure 1B The method discussed generates filtered time atlas 682 (depicting only reliable clusters). Figure 6B The output layer 190 processes the final filtered time map 682 generated during the second iteration of base detection and produces base detection 692 for the central sequencing cycle 4.

[0186] Figure 7A A specific implementation is shown that filtering logic 502 is used to filter compressed spatial atlases 130, 140, 150, 230, and 330 to generate corresponding compressed, filtered spatial maps 530, 540, 550, 650, and 750 (depicting only reliable clusters) for sequencing cycles 3, 4, 5, 6, and 7 during the third iteration of base detection. Filtering logic 502 uses data 128 identifying unreliable clusters to filter out (or discard or remove) those pixels from compressed spatial atlases 130, 140, 150, 230, and 330 that correspond to unreliable clusters (i.e., depicting their intensity emission).

[0187] exist Figure 7A In this process, filtered temporal maps 576, 676, and 776 (depicting only reliable clusters) are generated from compressed, filtered spatial maps 530, 540, 550, 650, and 750 for base detection in central sequencing cycle 5. The filtered temporal maps 576, 676, and 776 (depicting only reliable clusters) are processed by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above. Figure 1B The method discussed above is used to generate the network. It consists of a second temporal convolutional layer and a final temporal convolutional layer of the temporal network 160, similar to the method described above. Figure 1B The method discussed generates filtered time atlas 782 (depicting only reliable clusters). Figure 7B The output layer 190 processes the final filtered time map 782 generated during the third iteration of base detection and produces base detection 792 for the central sequencing cycle 5.

[0188] In other specific implementations, compression logic 108 can configure the corresponding compression time map sets to each have more than four feature maps.

[0189] The compression logic 108 discussed above regarding spatial feature maps is equivalently applied to the compression of temporal feature maps generated by temporal logic 160. The reuse of previously generated compressed spatial feature maps in subsequent sequencing cycles is also equivalently applied to the reuse of previously generated compressed temporal feature maps in subsequent sequencing cycles.

[0190] In some implementations, reusing compressed temporal feature maps results in two orders of magnitude of efficiency and computational savings compared to reusing compressed spatial feature maps, since the compressed temporal feature maps are generated from compressed spatial feature maps at a later stage of the processing pipeline. Reusing intermediate results from another processing engine (i.e., temporal network 160) increases the number of earlier processing steps that can be skipped. That is, reusing compressed spatial feature maps eliminates redundant processing of the original image data via spatial network 104, but may include redundant processing of the compressed spatial feature maps via temporal network 160. Conversely, reusing compressed temporal feature maps eliminates redundant processing of the original image data via spatial network 104 and redundant processing of the compressed spatial feature maps via temporal network 160.

[0191] Figure 8A A specific implementation is shown that the time feature sets 172, 174, and 176 generated during the first iteration of base detection are processed by compression logic 108 to generate corresponding compressed time feature sets 802, 804, and 806. Compression logic 108 is similar to the above regarding... Figure 1E and Figure 1F The compressed temporal feature sets 802, 804, and 806 are generated in the manner discussed. That is, for example, if the first temporal feature set 172 has, for example, 21 feature maps (or channels or depth = 21), then the compression logic 108 can configure the corresponding compressed temporal feature set 802 to have 1, 2, 3, 4, or more feature maps. The compressed sets of temporal feature maps 802, 804, and 806 are generated by the second temporal convolutional layer of the temporal network 160 in a manner similar to that described above. Figure 1BThe discussed method is used to generate the final compressed temporal feature atlas 814. Figure 8B The output layer 190 processes the final compressed temporal feature set 814 generated during the first iteration of base detection and produces base detection 892 for the central sequencing cycle 3.

[0192] Figure 9A This illustrates a specific implementation where compressed timemaps generated in the first base detection iteration are reused in the second base detection iteration. Specifically, the first set of compressed timemaps 804 and the second set of compressed timemaps 806 are used in... Figure 8A The process involves iterative generation targeting the first base, and is now being implemented. Figure 9A and Figure 9B The second base detection iteration shown is reused.

[0193] It should be noted that, Figure 8A In the process, the first set of compressed time maps 804 and the second set of compressed time maps 806 are generated from the first set of filtered time maps 172 and the second set of filtered time maps 174. Further note that the first set of time maps 172 and the second set of time maps 174 are... Figure 1B The compressed spatial maps 110, 120, 130, and 140 are generated from the compressed spatial maps, which are then... Figure 1A The corresponding spatial diagrams 106, 116, 126, and 136 are generated from these spatial diagrams, which are then... Figure 1A The image is generated from the corresponding image patches 102, 112, 122 and 132.

[0194] and Figure 9A Redundant generation of overlapping time maps 172, 174, and 176 Figure 1B , Figure 2B and Figure 3B Different, overlapping time plots 174 and 176 (in Figure 9A (The text is depicted using dotted lines and shadows) is not from... Figure 8A (First base detection iteration) to Figure 9A The second base detection iteration is redundantly generated. This is because compression logic 108 is incorporated into timing network 160 to generate a first set of compressed timing maps 804 and a second set of compressed timing maps 806 in the first base detection iteration. The first and second sets of compressed timing maps replace overlapping timing maps 174 and 176 in the second base detection iteration. The compressed timing maps can be stored in memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM).

[0195] Figure 9AIt is also shown that non-overlapping timemaps 278 (i.e., non-overlap between the first and second base detection iterations) are processed by compression logic 108 to generate compressed timemaps 906. Compressed timemap sets 804, 806, and 906 are generated by the second temporal convolutional layer of the temporal network 160 in a manner similar to that described above. Figure 1B The discussed method is used to process the data to generate the final compressed temporal feature set 914. Figure 9B The output layer 190 processes the final compressed temporal feature set 914 generated during the second iteration of base detection and produces base detection 992 for the central sequencing cycle 4.

[0196] and Figure 10A Redundant generation of overlapping time maps 174, 176, and 278 Figure 1B , Figure 2B and Figure 3B Different, overlapping time plots 176 and 278 (in Figure 10A (The text is depicted using dotted lines and shadows) is not from... Figure 9A (Second base detection iteration) to Figure 10A The (third base detection iteration) is redundantly generated. This is because compression logic 108 is incorporated into timing network 160 to generate a first set of compressed timing maps 806 and a second set of compressed timing maps 906 in the first and second base detection iterations, replacing overlapping timing maps 176 and 278 in the third base detection iteration. The compressed timing maps can be stored in memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM).

[0197] Figure 10A It is also shown that non-overlapping timemaps 378 (i.e., the non-overlap between the second and third base detection iterations) are processed by compression logic 108 to generate compressed timemap 1006. Compressed timemap sets 806, 906, and 1006 are generated by the second temporal convolutional layer of temporal network 160 in a manner similar to that described above. Figure 1B The discussed method is used to process the data to generate the final compressed temporal feature set 1014. Figure 10B The output layer 190 processes the final compressed temporal feature set 1014 generated during the third iteration of base detection and produces base detection 1092 for the central sequencing cycle 5.

[0198] Figure 11A A specific implementation is shown that processes the filtered time feature sets 572, 574, and 576 generated by compression logic 108 during the first iteration of base detection to generate corresponding compressed, filtered time feature sets 1102, 1104, and 1306. Compression logic 108 is similar to the above regarding... Figure 1E and Figure 1F The discussed method generates compressed, filtered temporal feature sets 1102, 1104, and 1106 (depicting only reliable clusters). That is, for example, if the first filtered temporal feature set 572 has, for example, 21 feature maps (or channels or depth = 21), then compression logic 108 can configure the corresponding compressed, filtered temporal feature set 1102 to have 1, 2, 3, or 4 feature maps. The sets of compressed, filtered temporal feature maps 1102, 1104, and 1106 are generated by a second filtered temporal convolutional layer of the filtered temporal network 160 in a manner similar to that described above. Figure 1B The discussed method is used to generate the final compressed, filtered time feature atlas 1114. Figure 8B The output layer 190 processes the final compressed, filtered temporal feature set 1114 generated during the first iteration of base detection and produces base detection 1192 for the central sequencing cycle 3.

[0199] In other specific implementations, compression logic 108 can configure the corresponding compressed feature map sets to each have more than four feature maps.

[0200] Figure 12A This illustrates a specific implementation where compressed, filtered timemaps generated in the first base detection iteration are reused in the second base detection iteration. Specifically, the first set of compressed, filtered timemaps 1104 and the second set of compressed, filtered timemaps 1106 are reused in the second base detection iteration. Figure 11A The process involves iterative generation targeting the first base, and is now being implemented. Figure 12A and Figure 2B The second base detection iteration shown is reused.

[0201] It should be noted that, Figure 11A In the process, the first set of compressed and filtered time maps 1104 and the second set of compressed and filtered time maps 1106 are generated from the first set of filtered time maps 572 and 574. Further note that the first set of filtered time maps 572 and 574 are... Figure 5A The compressed and filtered spatial maps 510, 520, 530, and 540 are generated from these compressed and filtered spatial maps, which are then... Figure 5A The corresponding compressed spatial maps 110, 120, 130, and 140 are generated, and these corresponding compressed spatial maps are then... Figure 1A The corresponding spatial diagrams 106, 116, 126, and 136 are generated from these spatial diagrams, which are then... Figure 1A The image is generated from the corresponding image patches 102, 112, 122 and 132.

[0202] and Figure 12A Overlapping, filtered time maps 572, 574, and 576 are redundantly generated. Figure 5A , Figure 6A and Figure 7A Different, overlapping, filtered time plots 574 and 576 (in Figure 12A (The text is depicted using dotted lines and shadows) is not from... Figure 11A (First base detection iteration) to Figure 12A (The second base detection iteration) is redundantly generated. This is because compression logic 108 is incorporated into the filtered timing network 160 to generate a first set of compressed, filtered timing maps 1104 and a second set of compressed, filtered timing maps 1106 in the first base detection iteration. The first and second sets of compressed, filtered timing maps replace the overlapping filtered timing maps 574 and 576 in the second base detection iteration. The compressed, filtered timing maps can be stored in memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM).

[0203] Figure 12A It is also shown that non-overlapping, filtered timemaps 676 (i.e., non-overlapping between the first and second base detection iterations) are processed by compression logic 108 to generate compressed, filtered timemaps 1206 (only reliable clusters are shown). Compressed, filtered timemap sets 1104, 1106, and 1206 are generated by a second filtered temporal convolutional layer of the filtered temporal network 160 in a manner similar to that described above. Figure 1B The discussed method is used to process the data to generate the final compressed, filtered time feature atlas 1214. Figure 12B The output layer 190 processes the final compressed, filtered temporal feature set 1214 generated during the second iteration of base detection and produces base detection 1292 for the central sequencing cycle 4.

[0204] and Figure 13A Overlapping, filtered time plots 574, 576, and 676 are redundantly generated. Figure 5A , Figure 6A and Figure 7A Different, overlapping, filtered time plots 576 and 676 (in Figure 13A (The text is depicted using dotted lines and shadows) is not from... Figure 12A (Second base detection iteration) to Figure 13A(The third base detection iteration) is redundantly generated. This is because compression logic 108 is incorporated into the filtered timing network 160 to generate a first set of compressed, filtered timing maps 1106 and a second set of compressed, filtered timing maps 1206 in the first and second base detection iterations. The first and second sets of compressed, filtered timing maps replace the overlapping filtered timing maps 576 and 676 in the third base detection iteration. The compressed, filtered timing maps can be stored in memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM).

[0205] Figure 13A It is also shown that non-overlapping, filtered timemaps 776 (i.e., non-overlapping between the second and third base detection iterations) are processed by compression logic 108 to generate compressed, filtered timemaps 1306 (only reliable clusters are shown). The compressed, filtered timemap sets 1106, 1206, and 1306 are generated by a second filtered temporal convolutional layer of the filtered temporal network 160 in a manner similar to that described above. Figure 1B The discussed method is used to process the data to generate the final compressed, filtered time feature atlas 1314. Figure 13B The output layer 190 processes the final compressed, filtered temporal feature set 1314 generated during the third iteration of base detection and produces base detection 1392 for the central sequencing cycle 5.

[0206] Figure 14 A first exemplary architecture of a neural network-based base detector 100 is shown. In the illustrated embodiment, the neural network-based base detector 100 includes a spatial network 104, a compressed network 108, and a temporal network 160. The spatial network 104 includes seven spatial convolutional layers. The compressed network 108 includes a compressed layer. The temporal network 160 includes two temporal convolutional layers.

[0207] Each of the seven spatial convolutional layers may have the same number of convolutional filters, or may have a different number of convolutional filters. The first spatial convolutional layer may have S1 filters, where S1 can be, for example, 7, 14, 21, 64, 128, or 254. The second spatial convolutional layer may have S2 filters, where S2 can be, for example, 7, 14, 21, 64, 128, or 254. The third spatial convolutional layer may have S3 filters, where S3 can be, for example, 7, 14, 21, 64, 128, or 254. The fourth spatial convolutional layer may have S4 filters, where S4 can be, for example, 7, 14, 21, 64, 128, or 254. The fifth spatial convolutional layer may have S5 filters, where S5 can be, for example, 7, 14, 21, 64, 128, or 254. The sixth spatial convolutional layer may have S6 filters, where S6 can be, for example, 7, 14, 21, 64, 128, or 254. The seventh spatial convolutional layer may have S7 filters, where S7 can be, for example, 7, 14, 21, 64, 128, or 254.

[0208] The compression layer may have C1 filters, where C1 can be, for example, 1, 2, 3, 4 or more.

[0209] Each of the two temporal convolutional layers may have the same number of convolutional filters, or may have a different number of convolutional filters. The first temporal convolutional layer may have T1 filters, where T1 can be, for example, 7, 14, 21, 64, 128, or 254. The second temporal convolutional layer may have T2 filters, where T2 can be, for example, 7, 14, 21, 64, 128, or 254. Figure 14 Feature maps 1412 generated by the layers of the neural network-based base detector 100 are also shown.

[0210] Figure 15 A second exemplary architecture of a neural network-based base detector 100 is shown. Figure 15 Filtering logic 502 is shown as part of a neural network-based base detector 100. In other embodiments, filtering logic 502 is not part of the neural network-based base detector 100. The compressed feature map C1 has a spatial dimension of P1×P2. Filtering logic 502 filters out pixels in the compressed feature map C1 that correspond to unreliable clusters and generates a compressed, filtered feature map F1 with a spatial dimension of P3×P4. The compressed, filtered feature map F1 depicts only reliable clusters. In one embodiment, filtering logic 502 discards 75% of the pixels in the compressed feature map C1, so P3 is 25% of P1 and P4 is 25% of P2. Figure 15 Feature maps 1512 generated by the layers of the neural network-based base detector 100 are also shown.

[0211] Figure 16 A third exemplary architecture of a neural network-based base detector 100 is shown. Figure 16 The output of compression network 108 is shown for compressing spatial network 104 and temporal network 160. Figure 16 Feature maps 1612 generated by the layers of the neural network-based base detector 100 are also shown.

[0212] Figure 17 A fourth exemplary architecture of a neural network-based base detector 100 is shown. Figure 17 The filter logic 502 is shown to be applied to the compressed output of the spatial network 104 to generate a compressed and filtered time output from the temporal network 160. Figure 17 Feature map 1712 generated by the layers of the neural network-based base detector 100 is also shown.

[0213] Figure 18 A specific implementation of filter configuration logic 1804 is shown, which configures the count (or number) of convolutional filters in compression layer 108 based on the number of channels in the input data. This allows the compressed feature map to be a lossless representation of the input data. In some implementations, the input data can be rewritten in memory with the corresponding compressed representation for reuse in subsequent sequencing cycles.

[0214] In one embodiment, for input data containing only one channel 1812 (e.g., only one image channel) per per-cycle input, filter configuration logic 1804 configures compression layer 108 with only one convolutional filter 1816, which generates only one compressed feature map 1818 per sequencing cycle. In another embodiment, for input data containing two channels 1822 per per-cycle input (e.g., two image channels, such as blue and green image channels corresponding to blue and green lasers in a sequencing image), filter configuration logic 1804 configures compression layer 108 with two convolutional filters 1826, which generate two compressed feature maps 1828 per sequencing cycle. In yet another embodiment, for input data containing three channels 1832 (e.g., three image channels) per per-cycle input, filter configuration logic 1804 configures compression layer 108 with three convolutional filters 1836, which generate three compressed feature maps 1838 per sequencing cycle. In another embodiment, for input data containing four channels 1842 in each per-cycle input (e.g., four image channels in a sequencing image corresponding to nucleotides A, C, T, and G, such as A, C, T, and G channels), filter configuration logic 1804 configures compression layer 108 with four convolutional filters 1846, which generate four compressed feature maps 1848 in each sequencing cycle. In other embodiments, compression logic 108 may configure the corresponding compressed feature map sets to each have more than four feature maps, and thus select more than four filters for compression layer 108.

[0215] Figure 19A and Figure 19B A specific implementation of the sequencing system 1900A is described. The sequencing system 1900A includes a configurable processor 1946. The configurable processor 1946 implements the base detection technology disclosed herein. The sequencing system is also referred to as a "sequencing instrument".

[0216] The sequencing system 1900A is operable to obtain any information or data relating to at least one of biological or chemical substances. In some specific embodiments, the sequencing system 1900A is a workstation that may resemble a desktop device or desktop computer. For example, most (or all) of the systems and components used to perform the desired reaction may be located within a common housing 1902.

[0217] In certain specific implementations, the Sequencing System 1900A is a nucleic acid sequencing system configured for a variety of applications, including but not limited to de novo sequencing, resequencing of whole or target genome regions, and metagenomics. The sequencer can also be used for DNA or RNA analysis. In some implementations, the Sequencing System 1900A can also be configured to generate reaction sites in biosensors. For example, the Sequencing System 1900A can be configured to receive a sample and generate surface-attached clusters of clonal amplified nucleic acids derived from the sample. Each cluster can constitute a reaction site in the biosensor or be part of it.

[0218] An exemplary sequencing system 1900A may include a system socket or interface 1910 configured to interact with a biosensor 1912 to perform a desired reaction within the biosensor 1912. (The following is about...) Figure 19A In the description, the biosensor 1912 is loaded into the system socket 1910. However, it should be understood that a cartridge including the biosensor 1912 can be inserted into the system socket 1910, and in some states, the cartridge can be temporarily or permanently removed. As described above, among other things, the cartridge may also include fluid control components and fluid storage components.

[0219] In a particular implementation, the sequencing system 1900A is configured to perform a large number of parallel reactions within the biosensor 1912. The biosensor 1912 includes one or more reaction sites where the desired reaction can occur. These reaction sites may be, for example, fixed to a solid surface of the biosensor or to a bead (or other movable substrate) located within a corresponding reaction chamber of the biosensor. The reaction sites may include, for example, clusters of cloned amplified nucleic acids. The biosensor 1912 may include a solid-state imaging device (e.g., a CCD or CMOS imager) and a flow cell mounted thereon. The flow cell may include one or more flow channels that receive a solution from the sequencing system 1900A and direct the solution to the reaction sites. Optionally, the biosensor 1912 may be configured to incorporate a thermal element for transferring heat energy into or out of the flow channels.

[0220] The sequencing system 1900A may include various components, parts, and systems (or subsystems) that interact with each other to perform predetermined methods or assay protocols for biological or chemical analysis. For example, the sequencing system 1900A includes a system controller 1906 that can communicate with the various components, parts, and subsystems of the sequencing system 1900A and the biosensor 1912. For example, in addition to the system socket 1910, the sequencing system 1900A may also include a fluid control system 1908 to control the flow of fluids throughout the fluid network of the sequencing system 1900A and the biosensor 1912; a fluid storage system 1914 configured to maintain all fluids (e.g., gases or liquids) usable by the bioassay system; a temperature control system 1904 that regulates the temperature of the fluids in the fluid network, the fluid storage system 1914, and / or the biosensor 1912; and an illumination system 1916 configured to illuminate the biosensor 1912. As described above, if a cartridge with a biosensor 1912 is loaded into a system socket 1910, the cartridge may also include fluid control components and fluid storage components.

[0221] As also shown in the figure, the sequencing system 1900A may include a user interface 1918 for user interaction. For example, the user interface 1918 may include a display 1920 for displaying or requesting information from the user and a user input device 1922 for receiving user input. In some specific embodiments, the display 1920 and the user input device 1922 are the same device. For example, the user interface 1918 may include a touch-sensitive display configured to detect the presence of an individual touch and also identify the location of the touch on the display. However, other user input devices 1922 may be used, such as a mouse, touchpad, keyboard, keypad, handheld scanner, voice recognition system, motion recognition system, etc. As will be discussed in more detail below, the sequencing system 1900A may communicate with various components, including a biosensor 1912 (e.g., in the form of a cartridge), to perform desired responses. The sequencing system 1900A may also be configured to analyze data obtained from the biosensors to provide the user with the required information.

[0222] System controller 1906 may include any processor-based or microprocessor-based system, including those using microcontrollers, reduced instruction set computers (RISC), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), coarse-grained reconfigurable architectures (CGRAs), logic circuits, and any other circuitry or processor capable of performing the functions described herein. The examples above are merely illustrative and are therefore not intended to limit the definition and / or meaning of the term "system controller" in any way. In an exemplary specific implementation, system controller 1906 executes a set of instructions stored in one or more storage elements, memories, or modules to perform at least one of acquiring detection data and analyzing the detection data. Detection data may include multiple pixel signal sequences, such that pixel signal sequences from each of millions of sensors (or pixels) can be detected within numerous base detection cycles. Storage elements may be in the form of information sources or physical memory elements within the sequencing system 1900A.

[0223] The instruction set may include various commands instructing the sequencing system 1900A or biosensor 1912 to perform specific operations (such as the methods and procedures described herein in various specific embodiments). The instruction set may be in the form of a software program that forms part of one or more tangible, non-transitory computer-readable media. As used herein, the terms “software” and “firmware” are interchangeable and include any computer program stored in memory for execution by a computer, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are merely exemplary and therefore do not limit the types of memory that can be used to store computer programs.

[0224] The software can take various forms, such as system software or application software. Furthermore, the software can be a collection of independent programs, or a program module or part of a program module within a larger program. The software may also include modular programming in the form of object-oriented programming. After the detection data is obtained, the detection data can be processed automatically by the sequencing system 1900A in response to user input or in response to a request from another processor (e.g., a remote request via a communication link). In the illustrated embodiment, the system controller 1906 includes an analysis module 1944. In other embodiments, the system controller 1906 does not include the analysis module 1944, but the analysis module 1944 is accessible (e.g., the analysis module 1944 may be hosted separately in the cloud).

[0225] The system controller 1906 can be connected via a communication link to the biosensor 1912 and other components of the sequencing system 1900A. The system controller 1906 can also be communicatively connected to off-site systems or servers. The communication link can be hardwired, wired, or wireless. The system controller 1906 can receive user input or commands from the user interface 1918 and the user input device 1922.

[0226] The fluid control system 1908 includes a fluid network and is configured to guide and regulate the flow of one or more fluids through the fluid network. The fluid network may be in fluid communication with a biosensor 1912 and a fluid storage system 1914. For example, selected fluid may be drawn from the fluid storage system 1914 and guided in a controlled manner to the biosensor 1912, or fluid may be drawn from the biosensor 1912 and directed toward, for example, a waste reservoir in the fluid storage system 1914. Although not shown, the fluid control system 1908 may include a flow sensor that detects the velocity or pressure of the fluid within the fluid network. The sensor may communicate with a system controller 1906.

[0227] Temperature control system 1904 is configured to regulate the temperature of fluids at different regions of the fluid network, fluid storage system 1914, and / or biosensor 1912. For example, temperature control system 1904 may include a thermal cycler that interfaces with biosensor 1912 and controls the temperature of fluid flowing along reaction sites in biosensor 1912. Temperature control system 1904 may also regulate the temperature of solid-state elements or components of sequencing system 1900A or biosensor 1912. Although not shown, temperature control system 1904 may include sensors for detecting the temperature of fluids or other components. These sensors may communicate with system controller 1906.

[0228] The fluid storage system 1914 is in fluid communication with the biosensor 1912 and can store various reaction components or reactants for carrying out the desired reaction therein. The fluid storage system 1914 can also store fluids for washing or cleaning the fluid network and the biosensor 1912, as well as for diluting the reactants. For example, the fluid storage system 1914 may include various reservoirs for storing samples, reagents, enzymes, other biomolecules, buffer solutions, aqueous solutions, and nonpolar solutions, etc. Furthermore, the fluid storage system 1914 may include a waste reservoir for receiving waste from the biosensor 1912. In embodiments including a cartridge, the cartridge may include one or more of a fluid storage system, a fluid control system, or a temperature control system. Therefore, one or more components relating to those systems described herein may be housed within a cartridge housing. For example, the cartridge may have various reservoirs for storing samples, reagents, enzymes, other biomolecules, buffer solutions, aqueous solutions, and nonpolar solutions, waste, etc. Therefore, one or more of the fluid storage system, fluid control system, or temperature control system may be removably coupled to the bioassay system via the cartridge or other biosensor.

[0229] The illumination system 1916 may include a light source (e.g., one or more LEDs) and multiple optical components for illuminating the biosensor. Examples of light sources may include lasers, arc lamps, LEDs, or laser diodes. Optical components may be, for example, reflectors, dichroic mirrors, beam splitters, collimators, lenses, filters, wedge mirrors, prisms, mirrors, detectors, etc. In a specific embodiment using the illumination system, the illumination system 1916 may be configured to direct excitation light to the reaction site. As an example, the fluorophore may be excited by light of a green wavelength, thus the wavelength of the excitation light may be approximately 1932 nm. In one embodiment, the illumination system 1916 is configured to produce illumination parallel to the surface normal of the surface of the biosensor 1912. In another embodiment, the illumination system 1916 is configured to produce illumination at an angle relative to the surface normal of the surface of the biosensor 1912. In yet another embodiment, the illumination system 1916 is configured to produce illumination with multiple angles, including some parallel illumination and some angled illumination.

[0230] System socket or interface 1910 is configured to engage biosensor 1912 in at least one of mechanical, electrical, and fluidic methods. System socket 1910 can hold biosensor 1912 in a desired orientation to facilitate fluid flow through biosensor 1912. System socket 1910 may also include electrical contacts configured to engage biosensor 1912, enabling sequencing system 1900A to communicate with and / or power biosensor 1912. Furthermore, system socket 1910 may include a fluid port (e.g., a nozzle) configured to engage biosensor 1912. In some embodiments, biosensor 1912 is removably coupled to system socket 1910 in mechanical, electrical, and fluidic methods.

[0231] Furthermore, the sequencing system 1900A can communicate remotely with other systems or networks, or with other bioassay systems 1900A. Detection data obtained by the bioassay system 1900A can be stored in a remote database.

[0232] Figure 19B It is possible Figure 19A The block diagram of the system controller 1906 used in the system is shown below. In one specific implementation, the system controller 1906 includes one or more processors or modules that can communicate with each other. Each of the processors or modules may include an algorithm (e.g., instructions stored on a tangible and / or non-transitory computer-readable storage medium) or a sub-algorithm for performing a particular process. The system controller 1906 is conceptually shown as a collection of modules, but can be implemented using any combination of dedicated hardware boards, DSPs, processors, etc. Alternatively, the system controller 1906 may be implemented using an off-the-shelf PC with a single processor or multiple processors, where functional operations are distributed among the processors. As a further alternative, the modules described below may be implemented using a hybrid configuration, where some modular functions are executed using dedicated hardware, while other modular functions are executed using an off-the-shelf PC, etc. Modules may also be implemented as software modules within a processing unit.

[0233] During operation, communication port 1950 can communicate with biosensor 1912 ( Figure 19A ) and / or subsystems 1908, 1914, 1904 ( Figure 19A It transmits information (e.g., commands) or receives information (e.g., data) from it. In a specific implementation, communication port 1950 may output a sequence of multiple pixel signals. Communication link 1934 may be connected to user interface 1918 ( Figure 19AThe system receives user input and transmits data or information to the user interface 1918. Data from the biosensor 1912 or subsystems 1908, 1914, 1904 can be processed in real time by the system controller 1906 during a bioassay session. Alternatively, data can be temporarily stored in the system memory during a bioassay session and processed at a slower rate than in real-time or offline operation.

[0234] like Figure 19B As shown, the system controller 1906 may include multiple modules 1926 to 1948 that communicate with the main control module 1924 and the central processing unit (CPU) 1952. The main control module 1924 may communicate with the user interface 1918. Figure 19A Communication. Although modules 1926 to 1948 are shown communicating directly with the main control module 1924, modules 1926 to 1948 can also communicate directly with each other, and directly with the user interface 1918 and the biosensor 1912. Additionally, modules 1926 to 1948 can communicate with the main control module 1924 through other modules.

[0235] Multiple modules 1926 to 1948 include system modules 1928 to 1932, 1926 that communicate with subsystems 1908, 1914, 1904, and 1916, respectively. Fluid control module 1928 can communicate with fluid control system 1908 to control valves and flow sensors in the fluid network, thereby controlling the flow of one or more fluids through the fluid network. Fluid storage module 1930 can notify the user when the fluid volume is low or when the waste storage tank is at or near its capacity. Fluid storage module 1930 can also communicate with temperature control module 1932 to allow fluid to be stored at a desired temperature. Illumination module 1926 can communicate with illumination system 1916 to illuminate reaction sites at specified times during the program, such as after a desired reaction (e.g., a binding event) has occurred. In some embodiments, illumination module 1926 can communicate with illumination system 1916 to illuminate reaction sites at a specified angle.

[0236] Multiple modules 1926 to 1948 may also include a device module 1936 that communicates with the biosensor 1912 and an identification module 1938 that determines identification information associated with the biosensor 1912. The device module 1936 may, for example, communicate with a system socket 1910 to confirm that the biosensor has established electrical and fluid connections with the sequencing system 1900A. The identification module 1938 may receive signals that identify the biosensor 1912. The identification module 1938 may use the identity of the biosensor 1912 to provide additional information to the user. For example, the identification module 1938 may determine and subsequently display a batch number, manufacturing date, or suggest a protocol for operation with the biosensor 1912.

[0237] Multiple modules 1926 to 1948 also include an analysis module 1944 (also referred to as a signal processing module or signal processor) for receiving and analyzing signal data (e.g., image data) from the biosensor 1912. The analysis module 1944 includes memory (e.g., RAM or flash memory) for storing the detection / image data. The detection data may include multiple pixel signal sequences, such that pixel signal sequences from each of millions of sensors (or pixels) can be detected within numerous base detection cycles. The signal data may be stored for subsequent analysis or may be transmitted to a user interface 1918 to display desired information to the user. In some specific implementations, the signal data may be processed by a solid-state imager (e.g., a CMOS image sensor) before being received by the analysis module 1944.

[0238] Analysis module 1944 is configured to acquire image data from a photodetector at each of multiple sequencing cycles. The image data originates from the emission signal detected by the photodetector, and the image data of each sequencing cycle is processed by a neural network-based base detector 100, generating base detection for at least some analytes at each of the multiple sequencing cycles. The photodetector may be part of one or more top-view cameras (e.g., an Illumina GAIIx CCD camera that captures images of clusters on biosensor 1912 from above), or it may be part of biosensor 1912 itself (e.g., an Illumina iSeq CMOS image sensor located below the clusters on biosensor 1912 and capturing images of the clusters from the bottom).

[0239] The output of the photodetector is a sequencing image, each depicting the intensity emission of the cluster and its surrounding background. The sequencing image depicts the intensity emission resulting from nucleotide incorporation into the sequence during sequencing. The intensity emission originates from the associated analyte and its surrounding background. The sequencing images are stored in memory 1948.

[0240] Protocol modules 1940 and 1942 communicate with main control module 1924 to control the operation of subsystems 1908, 1914, and 1904 during a predetermined assay protocol. Protocol modules 1940 and 1942 may include a set of instructions for instructing sequencing system 1900A to perform specific operations according to a predetermined protocol. As shown, the protocol module may be sequencing-by-synthesis (SBS) module 1940, which is configured to issue various commands for performing the sequencing-by-synthesis process. In SBS, the extension of nucleic acid primers along a nucleic acid template is monitored to determine the sequence of nucleotides in the template. The underlying chemical process may be polymerization (e.g., catalyzed by a polymerase) or ligation (e.g., catalyzed by a ligase). In a specific polymerase-based SBS implementation, fluorescently labeled nucleotides are added to primers in a template-dependent manner (causing primer extension), such that detection of the sequence and type of nucleotides added to the primers can be used to determine the sequence of the template. For example, to initiate a first SBS cycle, a command can be issued to deliver one or more labeled nucleotides, DNA polymerase, etc., to / through a flow cell containing an array of nucleic acid templates. The nucleic acid templates may be located at corresponding reaction sites. Those reaction sites where primer extension results in the incorporation of labeled nucleotides can be detected by an imaging event. During the imaging event, an illumination system 1916 can provide excitation light to the reaction sites. Optionally, the nucleotides may also include a reversible termination property that terminates further primer extension once the nucleotide is added to the primer. For example, a nucleotide analog with a reversible termination moiety can be added to the primer such that subsequent extension does not occur until a deblocking agent is delivered to remove that moiety. Thus, in a specific implementation using reversible termination, a command can be issued to deliver a deblocking agent to the flow cell (before or after detection occurs). One or more commands can be issued to perform washing between the various delivery steps. This cycle can then be repeated n times to extend the primer by n nucleotides, thereby detecting a sequence of length n. Exemplary sequencing techniques are described in, for example, Bentley et al., Nature 4196:193-199(20019); WO 04 / 0119497; US 7,0197,026; WO91 / 066719; WO 07 / 123744; US 7,329,492; US7,211,414; US 7,3119,019; US 7,4019,2191 and US 20019 / 01470190192, each of which is incorporated herein by reference.

[0241] For the nucleotide delivery step in the SBS cycle, a single type of nucleotide can be delivered at once, or multiple different nucleotide types (e.g., A, C, T, and G together) can be delivered. For nucleotide delivery configurations where only a single type of nucleotide is present at a time, the different nucleotides do not need to have different labels, as they can be distinguished based on the inherent time intervals in individualized delivery. Therefore, sequencing methods or apparatus can use monochromatic detection. For example, the excitation source only needs to provide excitation at a single wavelength or within a single wavelength range. For nucleotide delivery configurations where delivery results in the simultaneous presence of multiple different nucleotides in the flow cell, the sites incorporating different nucleotide types can be distinguished based on different fluorescent labels attached to the corresponding nucleotide types in the mixture. For example, four different nucleotides can be used, each with one of four different fluorophores. In one specific implementation, excitation in four different regions of the spectrum can be used to distinguish the four different fluorophores. For example, four different excitation radiation sources can be used. Alternatively, fewer than four different excitation sources can be used, but optical filtering of excitation radiation from a single source can be used to generate different ranges of excitation radiation at the flow cell.

[0242] In some specific implementations, fewer than four different colors can be detected in a mixture containing four different nucleotides. For example, nucleotide pairs can be detected at the same wavelength, but distinguished based on the intensity difference of one member relative to the other, or based on a change in the presence or absence of a signal in one member that results in a signal that is significantly present or absent compared to the detected signal of the other member of that pair (e.g., by chemical modification, photochemical modification, or physical modification). Exemplary apparatuses and methods for distinguishing four different nucleotides using detection of fewer than four colors are described, for example, in U.S. Patent Application Serial Nos. 61 / 19319,294 and 61 / 619,19719, the entire contents of which are incorporated herein by reference. U.S. Application 13 / 624,200, filed September 21, 2012, is also incorporated herein by reference in its entirety.

[0243] Multiple protocol modules may also include a sample preparation (or generation) module 1942, configured to command a fluid control system 1908 and a temperature control system 1904 to amplify the product within the biosensor 1912. For example, the biosensor 1912 may be coupled to a sequencing system 1900A. The amplification module 1942 may instruct the fluid control system 1908 to deliver necessary amplification components to the reaction chamber within the biosensor 1912. In other embodiments, the reaction site may already contain components for amplification, such as template DNA and / or primers. After delivering the amplification components to the reaction chamber, the amplification module 1942 may instruct the temperature control system 1904 to cycle through different temperature phases according to a known amplification protocol. In some embodiments, amplification and / or nucleotide incorporation are performed isothermally.

[0244] The SBS module 1940 can issue commands to perform bridged PCR, in which clusters of cloned amplicones are formed on localized regions within the channels of the flow cell. After amplicon generation via bridged PCR, the amplicones can be "linearized" to prepare single-stranded template DNA or sstDNA, and sequencing primers can be hybridized to universal sequences side-linked to regions of interest. For example, sequencing-by-synthesis methods based on reversible terminators can be used as described above or as follows.

[0245] Each base detection or sequencing cycle can be performed by extending sstDNA with a single base, which can be accomplished, for example, by using a modified DNA polymerase and a mixture of four types of nucleotides. Different types of nucleotides can have unique fluorescent labels, and each nucleotide can also have a reversible terminator that allows only single-base incorporation in each cycle. After adding a single base to the sstDNA, excitation light can be incident on the reaction site and fluorescence emission can be detected. After detection, the fluorescent label and terminator can be chemically cleaved from the sstDNA. This can then be followed by another similar base detection or sequencing cycle. In this sequencing protocol, the SBS module 1940 directs the fluid control system 1908 to guide the flow of reagents and enzyme solutions through the biosensor 1912. Exemplary SBS methods based on reversible terminators that can be used with the devices and methods described herein are described in U.S. Patent Application Publication No. 2007 / 01667019 A1, U.S. Patent Application Publication No. 2006 / 01196*3901 A1, U.S. Patent No. 7,0197,026, U.S. Patent Application Publication No. 2006 / 0240439 A1, U.S. Patent Application Publication No. 2006 / 021914714709 A1, PCT Publication No. WO 019 / 0619514, U.S. Patent Application Publication No. 20019 / 014700900 A1, PCT Publication No. WO 06 / 019B199, and PCT Publication No. WO 07 / 014702191, the entire contents of each of which are incorporated herein by reference. Exemplary reagents for SBS based on reversible terminators are described in US 7,1941,444; US 7,0197,026; US 7,414,14716; US 7,427,673; US 7,1966,1937; US 7,1992,4319 and WO 07 / 14193193619, the entire contents of each of which are incorporated herein by reference.

[0246] In some implementations, the amplification module and the SBS module can operate in a single assay protocol, where, for example, template nucleic acids are amplified and then sequenced within the same kit.

[0247] The sequencing system 1900A also allows users to reconfigure assay protocols. For example, the sequencing system 1900A can provide users with options for modifying the determined protocol via the user interface 1918. For instance, if it is determined that the biosensor 1912 will be used for amplification, the sequencing system 1900A can request the temperature for the annealing cycle. Furthermore, if the user has provided user input that is generally unacceptable for the selected assay protocol, the sequencing system 1900A can issue a warning to the user.

[0248] In a specific implementation, the biosensor 1912 comprises millions of sensors (or pixels), each sensor (or pixel) generating multiple pixel signal sequences in a subsequent base detection cycle. The analysis module 1944 detects the multiple pixel signal sequences based on the row-by-row and / or column-by-column positions of the sensors on the sensor array and assigns them to the corresponding sensors (or pixels).

[0249] Figure 19C This is a simplified block diagram of a system used to analyze sensor data (such as base detection sensor output) from a sequencing system 1900A. Figure 19C In the example, the system includes a configurable processor 1946. The configurable processor 1946 can coordinate with a runtime program / logic 1980 executed by a central processing unit (CPU) 1952 (i.e., the host processor) to execute a base detector (e.g., a neural network-based base detector 100). The sequencing system 1900A includes a biosensor 1912 and a flow cell. The flow cell may include one or more blocks in which clusters of genetic material are exposed to a sequence of analytical streams, the sequence of which is used to elicit a reaction in the clusters to identify bases in the genetic material. The sensor senses the reaction of each cycle of that sequence in each block of the flow cell to provide block data. Genetic sequencing is a data-intensive operation that converts base detection sensor data into base detection sequences for each cluster of genetic material sensed during the base detection operation.

[0250] The system in this example includes a CPU 1952 that executes runtime program / logic 1980 to coordinate base detection operations, a memory 1948B for storing sequences of block data arrays, base detection reads generated by the base detection operations, and other information used in the base detection operations. Additionally, in this illustration, the system includes a memory 1948A for storing configuration files (or multiple files) such as FPGA bit files and model parameters for configuring and reconfiguring the neural network of the configurable processor 1946, and for executing the neural network. The sequencing system 1900A may include programs for configuring the configurable processor and, in some specific implementations, a reconfigurable processor to execute the neural network.

[0251] The sequencing system 1900A is coupled to a configurable processor 1946 via a bus 1989. The bus 1989 can be implemented using high-throughput technologies, such as, in one example, a bus technology compatible with the PCIe standard (Rapid Peripheral Component Interconnect) currently maintained and developed by the PCI-SIG (PCI Special Interest Group). Also in this example, a memory 1948A is coupled to the configurable processor 1946 via a bus 1993. The memory 1948A can be on-board memory disposed on a circuit board having the configurable processor 1946. The memory 1948A is used by the configurable processor 1946 for high-speed access to working data used in base detection operations. The bus 1993 can also be implemented using high-throughput technologies such as bus technologies compatible with the PCIe standard.

[0252] Configurable processors, including Field Programmable Gate Arrays (FPGAs), Coarse-grained Reconfigurable Arrays (CGRAs), and other configurable and reconfigurable devices, can be configured to perform a variety of functions more efficiently or faster than could be achieved using a general-purpose processor executing a computer program. Configuring a configurable processor involves compiling a functional description to produce a configuration file, sometimes called a bitstream or bitfile, and distributing the configuration file to configurable elements on the processor. This configuration file defines the logical functions to be performed by the configurable processor by configuring circuitry to set data flow patterns, the use of distributed memory and other on-chip memory resources, lookup table contents, the operation of configurable logic blocks and configurable execution units (such as multiply-accumulate units, configurable interconnects, and other elements of the configurable array). A configurable processor is reconfigurable if the configuration file can be changed in the field by changing the loaded configuration file. For example, the configuration file can be stored in volatile SRAM elements, non-volatile read-write memory elements, and combinations thereof, distributed across an array of configurable elements on the configurable or reconfigurable processor. A variety of commercially available configurable processors are suitable for base detection operations as described herein. Examples include Google's Tensor Processing Unit (TPU). TM Rackmount solutions (such as GX4 Rackmount Series) TM GX9 Rackmount Series TM NVIDIA DGX-1 TM Microsoft's Stratix V FPGA TM , Graphcore’s Intelligent Processor Unit (IPU) TM Qualcomm's Snapdragon processors TM Zeroth Platform TMNVIDIA's Volta TM NVIDIA's DRIVE PX TM , NVIDIA's JETSON TX1 / TX2MODULE TM Intel's Nirvana TM Movidius VPU TM ,Fujitsu DPI TM ARM's DynamicIQ TM IBM TrueNorth TM It has Testa V100s TM Lambda GPU servers, Xilinx Alveo TM U200, Xilinx Alveo TM U2190, Xilinx Alveo TM U280, Intel / Altera Stratix TM GX2800, Intel / AlteraStratix TM GX2800 and Intel Stratix TM GX10M. In some examples, the host CPU can be implemented on the same integrated circuit as the configurable processor.

[0253] The specific implementation described herein uses a configurable processor 1946 to implement a neural network-based base detector 100. The configuration file for the configurable processor 1946 can be implemented by specifying the logic functions to be performed using a high-level description language (HDL) or register-transfer level RTL language specification. The specification can be compiled to generate the configuration file using resources designed for the selected configurable processor. The same or similar specifications can be compiled to generate designs for application-specific integrated circuits (ASICs) that may not be configurable processors.

[0254] Therefore, in all the specific embodiments described herein, alternative embodiments of the configurable processor 1946 include a configured processor comprising an application-specific ASIC or application-specific integrated circuit or integrated circuit group, or a system-on-a-chip (SoC) device, or a graphics processing unit (GPU) processor or a coarse-grained reconfigurable architecture (CGRA) processor, the configured processor being configured to perform the neural network-based base detection operation as described herein.

[0255] Generally speaking, the configurable processor and the configured processor described herein, which are configured to execute the operation of a neural network, are referred to herein as neural network processors.

[0256] In this example, the configurable processor 1946 is configured by a configuration file loaded by a program executed using CPU 1952, or by other sources configuring an array of configurable elements 1991 (e.g., configuration logic blocks (CLBs), such as lookup tables (LUTs), flip-flops, computational processing units (PMUs) and computational memory units (CMUs), configurable I / O blocks, programmable interconnects) on the configurable processor to perform the base detection function. In this example, the configuration includes data flow logic 1997, which is coupled to buses 1989 and 1993 and performs functions for distributing data and control parameters among the elements used in the base detection operation.

[0257] Furthermore, the configurable processor 1946 is configured with data flow logic 1997 to execute a neural network-based base detector 100. Logic 1997 includes multiple loop execution clusters (e.g., 1979), which in this example include execution clusters 1 through X. The number of multiple loop execution clusters can be selected based on a trade-off between the required throughput of the operation and the available resources on the configurable processor 1946.

[0258] The multi-loop execution cluster is coupled to the data flow logic 1997 via a data flow path 1999 implemented using configurable interconnects and memory resources on the configurable processor 1946. Additionally, the multi-loop execution cluster is coupled to the data flow logic 1997 via a control path 1995 implemented, for example, using configurable interconnects and memory resources on the configurable processor 1946. This control path provides control signals indicating the available execution clusters, providing input units for the execution of the neural network-based base detector 100, providing trained parameters for the neural network-based base detector 100, providing output patches for base detection classification data, and other control data for executing the neural network-based base detector 100.

[0259] Configurable processor 1946 is configured to execute the operation of neural network-based base detector 100 using trained parameters to generate classification data for sensing cycles of base detection operations. The execution of neural network-based base detector 100 generates classification data for subject sensing cycles of base detection operations. The operation of neural network-based base detector 100 operates on a sequence comprising a digital array of N blocks of data from corresponding sensing cycles of N sensing cycles, wherein the N sensing cycles, in the examples described herein, provide sensor data for a different base detection operation at a base position for each operation in the time series. Optionally, some of the N sensing cycles may be out of order, depending on the specific neural network model being executed, if desired. The number N can be any number greater than 1. In some examples described herein, the sensing cycles in the N sensing cycles represent a set of sensing cycles that precede at least one sensing cycle and follow at least one sensing cycle in the time series. Examples described herein are in which the number N is an integer equal to or greater than five.

[0260] Data flow logic 1997 is configured to move at least some trained parameters of block data and model parameters from memory 1948A to a configurable processor 1946 for the operation of a neural network-based base detector 100 using an input unit for a given run. This input unit includes block data of patches for spatial alignment of N arrays. The input unit can be moved via a direct memory access operation in a DMA operation, or in smaller units that move in coordination with the execution of the deployed neural network during available time slots.

[0261] Block data for sensing cycles, as described herein, may include an array of sensor data having one or more features. For example, the sensor data may include two images, which are analyzed to identify one of four bases at a base position in the genetic sequence of DNA, RNA, or other genetic material. The block data may also include metadata about the images and the sensors. For example, in a specific implementation of a base detection operation, the block data may include information about the alignment of the image with the clusters, such as information about the distance from the center, which indicates the distance of each pixel in the sensor data array from the center of the cluster of genetic material on the block.

[0262] During the execution of the neural network-based base detector 100 as described below, the block data may also include data generated during the execution of the neural network-based base detector 100, referred to as intermediate data, which may be reused rather than recalculated during the operation of the neural network-based base detector 100. For example, during the execution of the neural network-based base detector 100, data flow logic 1997 may write intermediate data into memory 1948A instead of sensor data for a given patch of the block data array. Specific implementations similar to this are described in more detail below.

[0263] As shown in the figure, a system for analyzing the output of a base detection sensor is described. This system includes a memory (e.g., 1948A) accessible by a runtime program / logic 1980, which stores block data including sensor data from blocks of sensing loops of a base detection operation. Additionally, the system includes a neural network processor, such as a configurable processor 1946 with access to the memory. The neural network processor is configured to execute the operation of a neural network using trained parameters to generate classification data for the sensing loops. As described herein, the operation of the neural network operates on a sequence of N arrays of block data from the corresponding sensing loops (including the subject loop) of N sensing loops to generate classification data for the subject loop. Data flow logic 1997 is provided to move the block data and trained parameters from the memory to the neural network processor for the operation of the neural network using input units (including data from spatially aligned patches of the N arrays of the corresponding sensing loops of the N sensing loops).

[0264] Additionally, a system is described in which a neural network processor has access to memory and includes multiple execution clusters configured to execute a neural network. Dataflow logic 1997 accesses memory and the execution clusters to provide an input unit of block data to an available execution cluster among the multiple execution clusters. This input unit includes N digital spatial alignment patches from an array of block data from a corresponding sensing loop (including a subject sensing loop), and causes the execution cluster to apply the N spatial alignment patches to the neural network to produce an output patch of classification data for the spatial alignment patches of the subject sensing loop, where N is greater than 1.

[0265] Figure 20AThis is a simplified diagram illustrating various aspects of a base detection operation, including the functionality of a runtime program (e.g., runtime logic 1980) executed by the host processor. In this diagram, the output from the image sensor in the flow cell is provided on line 2000 to image processing thread 2001, which performs image processing such as alignment and arrangement of individual blocks in the sensor data array and image resampling. This can be used by a process that calculates a block cluster mask for each block in the flow cell, identifying pixels in the sensor data array corresponding to clusters of genetic material on the corresponding block in the flow cell. Depending on the state of the base detection operation, the output of image processing thread 2001 is provided on line 2002 to scheduling logic 2003 in the CPU, which routes the block data array to a data cache 2005 (e.g., an SSD storage device) on high-speed bus 2004, or to neural network processor hardware 2007, such as... Figure 19C The configurable processor 1946. Processed and transformed images can be stored on data cache 2005 for use in previously used sensing loops. Hardware 2007 returns the classification data output by the neural network to scheduling logic 2003, which passes the information to data cache 2005, or to thread 2009 on line 2008 to perform base detection and quality score calculation using the classification data, and can arrange the data for base detection reads in a standard format. On line 2010, the output of thread 2009 performing base detection and quality score calculation is provided to thread 2011, which aggregates base detection reads, performs other operations such as data compression, and writes the resulting base detection output to a specified destination for client use.

[0266] In some implementations, the host may include final processing of the output of hardware 2007 to support threads of the neural network (not shown). For example, hardware 2007 may provide output of classification data from the final layer of a multi-cluster neural network. The host processor may perform output activation functions such as a softmax function on the classification data to configure the data for use by the base detection and quality scoring threads 2002. Additionally, the host processor may perform input operations (not shown), such as batch normalization of block data before it is input to hardware 2007.

[0267] Figure 20B It is a configurable processor 1946, such as Figure 19C A simplified diagram of the configuration of the configurable processor. In Figure 20B In this configuration, the configurable processor 1946 includes an FPGA with multiple high-speed PCIe interfaces. The FPGA is configured with a package 2090, which includes a reference... Figure 19CThe data flow logic 1997 is described. The package 2090 manages the interface and coordination with the runtime program in the CPU via CPU communication link 2077, and manages communication with the onboard DRAM 2099 (e.g., memory 1448A) via DRAM communication link 2097. The data flow logic 1997 in the package 2090 provides patch data retrieved by traversing a block data array of N cycles on the onboard DRAM 2099 to cluster 2085, and retrieves process data 2087 from cluster 2085 for delivery back to the onboard DRAM 2099. The package 2090 also manages data transfer between the onboard DRAM 2099 and host memory for both the input array of block data and the output patches of classified data. The package transfers patch data on line 2083 to the assigned cluster 2085. The package provides training parameters such as weights and biases to the cluster 2085 retrieved from the onboard DRAM 2099 on line 2081. The wrapper on line 2079 provides configuration and control data to cluster 2085, which is provided from or generated in response to a runtime program on the host via CPU communication link 2077. The cluster can also provide status signals on line 2089 to wrapper 2090, which cooperate with control signals from the host to manage the traversal of the block data array, thereby providing spatially aligned patch data and using the resources of cluster 2085 to perform multi-recurrent neural networks on the patch data.

[0268] As described above, multiple clusters may exist on a single configurable processor managed by the wrapper 2090, and these clusters are configured to execute on corresponding patches of multiple patches of block data. Each cluster may be configured to provide classification data for base detection in a subject sensing cycle using block data from multiple sensing cycles as described herein.

[0269] In system examples, model data (including kernel data such as filter weights and biases) can be sent from the host CPU to a configurable processor, allowing the model to be updated based on the number of loops. As a representative example, a base detection operation may include approximately several hundred sensing loops. In some implementations, the base detection operation may include paired-end reads. For example, model training parameters may be updated every 20 loops (or another number of loops), or according to an update pattern implemented for a specific system and neural network model. In some implementations that include paired-end reads, where the sequence of a given string in a genetic cluster on a block comprises a first portion extending down (or up) along the string from a first end and a second portion extending up (or down) along the string from a second end, trained parameters may be updated during the transition from the first portion to the second portion.

[0270] In some examples, image data from multiple cycles of sensing data for a block can be sent from the CPU to the packager 2090. The packager 2090 may optionally perform some preprocessing and transformation on the sensing data and write the information to the onboard DRAM 2099. The input block data for each sensing cycle may include a sensor data array comprising approximately 4000 × 3000 pixels or more per block per sensing cycle, where two features represent the colors of two images of the block, and each feature is one or two bytes per pixel. For a specific implementation where the number N is three sensing cycles to be used in each run of the multi-recurrent neural network, the block data array for each run of the multi-recurrent neural network may consume approximately several hundred megabytes per block. In some specific implementations of the system, the block data may also include an array of distance-to-cluster-center (DFC) data stored once per block, or other types of metadata about the sensor data and the block.

[0271] In operation, when a multi-cycle cluster becomes available, the wrapper allocates a patch to the cluster. The wrapper retrieves the next patch of block data during block traversal and sends it, along with appropriate control and configuration information, to the allocated cluster. The cluster can be configured to have sufficient memory on a configurable processor to hold data patches, including those from multiple cycles in several systems and currently being processed in-situ, as well as data patches to be processed when the current patch is processed using either ping-pong buffering or raster scanning techniques in various implementations.

[0272] When an assigned cluster completes its neural network operation on the current patch and produces an output patch, it signals the packager. The packager reads the output patch from the assigned cluster, or alternatively, the assigned cluster pushes data to the packager. The packager then assembles the output patch for the processed block in DRAM 2099. When processing of the entire block is complete and the output patch of data has been transferred to DRAM, the packager sends the processed output array of the block back to the host / CPU in a specified format. In some implementations, the onboard DRAM 2099 is managed by memory management logic in packager 2090. The runtime program can control the sequencing operation to perform analysis of all arrays of block data in all loops during operation in a continuous streaming manner, thereby providing real-time analysis.

[0273] Figure 21 Another specific implementation of the disclosed data flow logic is shown, which makes the compressed space map generated during the first base detection iteration available from off-chip memory 2116 (e.g., off-chip DRAM, host RAM, host high bandwidth memory (HBM)) during the second base detection iteration.

[0274] In one implementation, host memory (e.g., memory 1948B) attached to a host processor (e.g., CPU 1952) is configured to receive the progress of sequencing image 2102 during a sequencing run. A configurable processor (e.g., configurable processor 1946) has an array of processing units. The processing units in the array are configured to execute a neural network-based base detector 100 to produce base detection predictions. Data flow logic 1997 has access to the host memory, the host processor, and the configurable processor. For a first base detection iteration, data flow logic 1997 loads a first window of the sequencing cycle (e.g., ...) from the host memory onto the configurable processor. Figure 1A Sequencing images of sequencing cycles 1 to 5 in the sequencing cycle.

[0275] Runtime logic 1980 is configured to cause the processing units of the configurable processor to execute the spatial network 104 of the neural network-based base detector 100 on the sequencing image 2102 in a loop, and to generate a spatial feature map set 2106 for each sequencing loop in the first window of the sequencing loop. In one embodiment, runtime logic 1980 executes multiple processing clusters of the neural network-based base detector 100 in parallel on patches 2104 of the sequencing image 2102. The multiple processing clusters apply the spatial network 104 on patches 2104 patch by patch 2105.

[0276] Runtime logic 1980 is configured to cause the processing unit of the configurable processor to execute the compressed network 108 of the neural network-based base detector 100 on the spatial feature map set 2106 in a loop, generating a compressed spatial feature map set 2107, and processing the compressed spatial feature map set 2107 through a temporal network 160 and an output network 190 to generate base detection predictions 2111 for one or more sequencing cycles in the first window of the sequencing cycle. The temporal network 160 generates the temporal feature map 2108. The output network 190 generates a base detection classification score 2110 (e.g., a non-normalized base-by-base score). In one embodiment, the compressed spatial feature map set 2107 is stored on off-chip memory 2116.

[0277] In one implementation, data flow logic 1997 is configured to move compressed spatial feature atlas 2107 to host memory 2116 and rewrite the corresponding sequencing image 2102 with compressed spatial feature atlas 2107. In other implementations, the corresponding patch in patch 2104 is replaced by compressed spatial feature atlas 2107.

[0278] For the second base detection iteration and for the second window of a sequencing cycle that shares one or more overlapping sequencing cycles (e.g., sequencing cycles 2 to 5) with the first window of the sequencing cycle and has at least one non-overlapping sequencing cycle (e.g., sequencing cycle 6), Figure 2A In sequencing cycles 2 to 6), data flow logic 1997 is configured to load compressed spatial feature maps 2126 of overlapping sequencing cycles and sequencing images 2122 (or patches 2124) of non-overlapping sequencing cycles from host memory on a configurable processor.

[0279] Runtime logic 1980 is configured to enable the processing units of the configurable processor to perform spatial network 104 on sequencing image 2122 for non-overlapping sequencing cycles and generate spatial feature maps 2126 for non-overlapping sequencing cycles. In one specific implementation, multiple processing clusters apply spatial network 104 patch by patch 2125 on patch 2124.

[0280] Runtime logic 1980 is configured to cause the processing unit of the configurable processor to execute compression network 108 on spatial feature map set 2126 and generate compressed spatial feature map set 2127 for non-overlapping sequencing cycles. It then processes the compressed spatial feature map set 2126 of overlapping sequencing cycles and the compressed spatial feature map set 2127 of non-overlapping sequencing cycles through temporal network 160 and output network 190 to generate base detection prediction 2131 for one or more sequencing cycles in a second window of the sequencing cycles. Temporal network 160 generates temporal feature map 2128. Output network 190 generates base detection classification score 2129 (e.g., unnormalized per-base score). In one embodiment, compressed spatial feature map set 2127 is stored on off-chip memory 2116.

[0281] Figure 22 A specific implementation of the disclosed data flow logic is shown, which makes a compressed spatial map generated during the first base detection iteration available from on-chip memory 2216 (e.g., processor memory, such as on-chip DRAM, on-chip SRAM, on-chip BRAM, DRAM attached to the processor via interconnects) during the second base detection iteration. Figure 22 In the process, compressed spatial feature atlas 2107 and compressed spatial feature atlas 2127 are stored on on-chip memory 2216. Similarly, in... Figure 22 In this configuration, the data flow logic 1997 is configured to load a compressed spatial feature atlas 2126 of overlapping sequencing cycles from on-chip memory 2216 onto a configurable processor.

[0282] Split Architecture

[0283] Figure 23A specific implementation of the so-called split architecture of the neural network-based base detector 100 is shown. As described above, the spatial convolutional network 104 is configured to convolve the corresponding per-cycle sequencing image set in the window of the per-cycle sequencing image set through the corresponding sequences 2301, 2302, 2303, 2304, and 2405 of the spatial convolutional layers, respectively, processing the window of the per-cycle sequencing image set of a series of sequencing cycles (cycles N+2, N+1, N, N-1, and N-2) for the sequencing run cyclically, to generate a corresponding per-cycle spatial feature map set for the corresponding sequencing cycle in the series of sequencing cycles. For example, each of the five sequences 2301, 2302, 2303, 2304, and 2405 of the spatial convolutional layers has seven spatial convolutional layers (i.e., Figure 23 (Layer L1 to Layer L7 in the middle).

[0284] The corresponding sequences 2301, 2302, 2303, 2304, and 2405 of the spatial convolutional layers have corresponding sequences of spatial convolutional filter banks (e.g., sequences of spatial convolutional filter banks 2310, 2311, 2312, 2313, 2314, 2315, and 2316 including the spatial convolutional filter bank 2301). In one specific implementation, the trained coefficients (or weights) of the spatial convolutional filters in the corresponding sequences of the spatial convolutional filter banks vary between the sequences of the spatial convolutional layers in the corresponding sequences of the spatial convolutional layers.

[0285] For example, spatial convolutional layer sequences 2301, 2302, 2303, 2304, and 2405 are configured with convolutional filters having different trained coefficients. In another example, the convolutional filters in corresponding levels of spatial convolutional layers have different trained coefficients (e.g., convolutional filter sets 2382, 2383, 2384, 2385, and 2312 in the corresponding third spatial convolutional layers of the five spatial convolutional layer sequences 2301, 2302, 2303, 2304, and 2405).

[0286] The temporal convolutional network 160 is configured to process each cyclic spatial feature map set group by group by convolving corresponding overlapping groups (e.g., groups 2360, 2361, and 2362) of each cyclic spatial feature map set with corresponding temporal convolutional filter groups 2321, 2322, and 2323 of the first temporal convolutional layer 2320, to generate corresponding temporal feature map sets for each corresponding overlapping group of the cyclic spatial feature map set. In one embodiment, the trained coefficients (or weights) of the temporal convolutional filters in the corresponding temporal convolutional filter groups vary among the temporal convolutional filter groups 2321, 2322, and 2323 in the corresponding temporal convolutional filter groups.

[0287] Jump architecture

[0288] Figure 24A This describes residual (or skip) connections that re-inject previous information downstream via feature map addition. Residual connections involve re-injecting previous representations into the downstream data stream by adding past output tensors to later output tensors, which helps prevent information loss along the data processing flow. Residual connections address two common problems that plague any large-scale deep learning model: vanishing gradients and representation bottlenecks.

[0289] Residual connections involve using the output of an earlier layer as input to a later layer, effectively creating shortcuts in sequential networks. Instead of connecting the earlier output to the later activation, the earlier output is added to it, assuming the two activations are of the same size. If they have different sizes, a linear transformation can be used to reshape the earlier activation into the desired shape.

[0290] Figure 24B A specific implementation of residual blocks and skip connections is described. The residual network stacks multiple residual units to mitigate the degradation of training accuracy. The residual blocks utilize special additive skip connections to alleviate vanishing gradients in deep neural networks. At the beginning of a residual block, the data stream is split into two streams: the first stream carries the block's invariant input, while the second stream applies weights and nonlinearities. At the end of the block, element-wise summation is used to merge the two streams. The main advantage of this construction is that it allows gradients to flow more easily through the network.

[0291] Equipped with a residual network, the neural network-based base detector 100 is easy to train in some implementations and can achieve improved accuracy for image classification and object detection. The neural network-based base detector 100 will... th The layer's output is connected as input to (l+1). th This causes the following layer transitions: x l =H l (x l-1 The residual block adds a skip connection that bypasses the nonlinear transformation using an identity function: x l =H l (x l-1 )+x l-1 The advantage of residual blocks is that gradients can be directly transferred from later layers to earlier layers (e.g., spatial and temporal convolutional layers) via the identity function. The identity function and Hi l The output is combined by summation (addition).

[0292] Figure 24CThe residual architecture of a neural network-based base detector 100 is shown, wherein spatial convolutional layers are grouped into residual blocks with skip connections. In other specific embodiments, the temporal convolutional layers of the neural network-based base detector 100 are grouped into residual blocks with skip connections.

[0293] exist Figure 24C In the specific implementation shown, the second spatial convolutional layer and the third spatial convolutional layer are grouped into a first residual block 2412; the fourth spatial convolutional layer and the fifth spatial convolutional layer are grouped into a second residual block 2422; and the sixth spatial convolutional layer and the seventh spatial convolutional layer are grouped into a third residual block 2432.

[0294] Figure 25A Details of the bus network of the neural network-based base detector 100 are shown. In one embodiment, a given residual block 2585 of the bus network includes a set of spatial convolutional layers 2590 and 2592. A first spatial convolutional layer 2590 in this set receives as input a preceding output 2586 generated by a preceding spatial convolutional layer that is not part of the given residual block 2585 (e.g., a null convolutional layer preceding the first spatial convolutional layer 2590 in spatial network 104). The first spatial convolutional layer 2590 processes the preceding output 2586 and generates a first output 2591. A second spatial convolutional layer 2592 in this set of spatial convolutional layers, following the first spatial convolutional layer 2590, receives the first output 2591, processes the first output 2591, and generates a second output 2593. In one embodiment, the first spatial convolutional layer 2590 has a non-linear activation function, such as ReLU, that generates the first output 2591. In another specific implementation, the second spatial convolutional layer 2592 lacks a non-linear activation function.

[0295] Skip connection 2589 provides the first output 2586 to adder 2594. Adder 2594 also receives a second output 2593 from the second spatial convolutional layer 2592. Adder 2594 combines the first output 2586 and the second output 2593 to generate a summed output 2595. The summed output 2595 is further processed by a nonlinear activation such as ReLU to generate a final summed output 2587. In some implementations, the final summed output 2587 is then fed as input to a subsequent residual block. In some implementations, the first output 2586 is modified to be dimension-compatible with the second output 2593. For example, the edges of the feature map in the first output 2586 are trimmed to produce a feature map with the same spatial dimension as the feature map in the second output 2593.

[0296] Figure 25BExemplary operation of the disclosed bus network is illustrated. In one specific implementation, the bus network is configured to form a bus (e.g., 2516, 2526, 2536, 2602, 2604, 2702, and 2712) between spatial convolutional layers within a corresponding sequence of spatial convolutional layers. This bus is configured to combine corresponding per-cycle spatial feature maps generated by two or more spatial convolutional layers in a specific sequence for a specific sequencing cycle into a combined per-cycle spatial feature map, and to provide this combined per-cycle spatial feature map as input to another spatial convolutional layer in the specific sequence of spatial convolutional layers.

[0297] For example, consider a first residual block 2412. Here, two or more spatial convolutional layers include a first spatial convolutional layer and a third spatial convolutional layer. The first spatial convolutional layer generates a first per-cycle spatial feature set 2520. The first spatial convolutional layer provides the first per-cycle spatial feature set 2520 as input to a second spatial convolutional layer. The second spatial convolutional layer processes the first per-cycle spatial feature set 2520 and generates a second per-cycle spatial feature set 2522. The second spatial convolutional layer provides the second per-cycle spatial feature set 2522 as input to a third spatial convolutional layer. The third spatial convolutional layer processes the second per-cycle spatial feature set 2522 and generates a third per-cycle spatial feature set 2524. A bus (e.g., a jump bus 2519) is further configured to combine the first spatial feature set 2520 and the third per-cycle spatial feature set 2524 (e.g., by summing or cascading by combiner 2502) into a combined per-cycle spatial feature set 2518. Then, another spatial convolutional layer is a fourth spatial convolutional layer, immediately following the third spatial convolutional layer in a specific sequence of spatial convolutional layers. The fourth spatial convolutional layer processes the combined per-cycle spatial feature map set 2518 as input. The same concept is similarly applied to the second residual block 2422 and the third residual block 2432, where 2526 and 2536 are jump buses such as jump bus 2516, and causes the corresponding combiners 2512 and 2532 to generate the corresponding combined per-cycle spatial feature map sets 2528 and 2538.

[0298] Figure 25C A specific implementation of dimension compatibility logic 2532 is shown, which ensures that the incoming feature map provided by the jump bus is modified (e.g., trimmed) to have the same spatial dimension as the received feature map before combination, and the incoming and received feature maps are combined by a combiner of the bus network.

[0299] Figure 26 Another example of the disclosed bus network is shown, in which the jump bus 2602 combines the output of the first spatial convolutional layer with the output of the first residual block. Figure 26It is also shown that the jump bus 2604 can also enable feature maps to be combined across residual blocks and across discontinuous layers (e.g., from layer 1 and layer 5) to generate a combined representation that can be processed by another layer (e.g., to layer 6).

[0300] Figure 27 Another example of the disclosed bus network is shown, wherein the inputs and combined representations from multiple consecutive and / or non-consecutive layers (e.g., from layer 1, combiner 2502 and combiner 2512) can be combined by exemplary jump buses 2702, 2604 and 2712 to generate a combined representation that can be processed by another layer (e.g., to layer 6).

[0301] Figure 28 A specific implementation of scaling logic 2832 is shown, which scales an incoming feature map before combining it with a received feature map provided by a jump bus, the combination of which is performed by a combiner of the bus network. The values ​​used by scaling logic 2832 can be, for example, any value between 0 and 1, including 0 and 1. For example, scaling logic can be used to attenuate or amplify the intensity / magnitude / value (e.g., feature value (e.g., floating-point value)) of the incoming feature map.

[0302] Figure 29 A specific implementation of skip connections between temporal convolutional layers 2902, 2912, 2922, 2932, 2942, 2952, 2962, and 2972 ​​of the temporal network 160 is shown. For example, skip connection 2922 provides a temporal feature map from the first temporal convolutional layer 2902 to the third temporal convolutional layer 2932.

[0303] Figure 30 The base detection performance of a network-based base detector 100 (sqz2 base detector) configured with compression logic 108 was compared with that of a network-based base detector 100 without compression logic 108 (used as a baseline neural network model) and Illumina's non-neural network-based base detector real-time analysis (RTA) software (used as a baseline conventional image processing model). Figure 30 As shown in the figure, the sqz2 base detector (purple fitted line) has a lower base detection error percentage ("error %" on the Y-axis) than the RTA base detector (black fitted line) and two instances of the network-based base detector 100 without compression logic 108 (red and cyan fitted lines).

[0304] Figure 31 The savings in RAM and DRAM usage resulting from the use of the disclosed compression logic 108 are shown.

[0305] Figure 32The base detection performance of a network-based base detector 100 configured with a split and skip architecture (split_res) was compared with that of an RTA base detector and another version of the network-based base detector 100 without a split and skip architecture (distillation). Figure 32 As shown in the figure, the split_res base detector (orange fitted line) has a lower percentage of base detection errors ("error count" on the Y-axis) than the RTA base detector (blue fitted line).

[0306] As used herein, “logic” (e.g., dataflow logic) can be implemented in the form of a computer product including a non-transitory computer-readable storage medium having computer-usable program code for performing the steps of the methods described herein. “Logic” can be implemented in the form of an apparatus including memory and at least one processor coupled to the memory and operable to perform the exemplary method steps. “Logic” can be implemented in the form of an apparatus for performing one or more method steps described herein; the apparatus may include (i) a hardware module, (ii) a software module executing on one or more hardware processors, or (iii) a combination of hardware and software modules; any of (i) through (iii) implements the specific techniques set forth herein, and the software module is stored in a computer-readable storage medium (or a plurality of such media). In one particular implementation, the logic implements data processing functions. The logic can be a general-purpose, single-core, or multi-core processor with a computer program having specified functions, a digital signal processor having a computer program, configurable logic such as an FPGA with a configuration file, special-purpose circuitry such as a state machine, or any combination thereof. Furthermore, a computer program product can embody both the computer program and configuration file portions of the logic.

[0307] Figure 33 The computer system 3300 is a sequencing system 1900A that can be used to implement the base detection technology disclosed herein. The computer system 3300 includes at least one central processing unit (CPU) 3372 that communicates with a plurality of peripheral devices via a bus subsystem 3355. These peripheral devices may include a storage subsystem 3358, which includes, for example, memory devices and a file storage subsystem 3336, a user interface input device 3338, a user interface output device 3376, and a network interface subsystem 3374. The input and output devices allow users to interact with the computer system 3300. The network interface subsystem 3374 provides an interface to an external network, including interfaces to corresponding interface devices in other computer systems.

[0308] In one implementation, the system controller 1906 may be communicatively linked to the storage subsystem 3310 and the user interface input device 3338.

[0309] User interface input device 3338 may include: a keyboard; pointing devices such as a mouse, trackball, touchpad, or graphics tablet; a scanner; a touchscreen integrated into a display; audio input devices such as a voice recognition system and a microphone; and other types of input devices. Generally, the term "input device" is intended to encompass all possible types of devices and methods for inputting information into computer system 3300.

[0310] User interface output device 3376 may include a display subsystem, printer, fax machine, or non-visual display such as an audio output device. The display subsystem may include an LED display, cathode ray tube (CRT), flat panel device such as a liquid crystal display (LCD), projection device, or other mechanisms for producing visible images. The display subsystem may also provide non-visual displays such as audio output devices. Generally, the term "output device" is intended to encompass all possible types of devices and methods for outputting information from computer system 3300 to a user or to another machine or computer system.

[0311] The storage subsystem 3358 provides programming and data construction for some or all of the functions and methods described in this document. These software modules are typically executed by the deep learning processor 3378.

[0312] The deep learning processor 3378 can be a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or a coarse-grained reconfigurable architecture (CGRA). The deep learning processor 3378 can be powered by deep learning cloud platforms such as Google Cloud Platform. TM Xilinx TM and Cirrascale TM Hosted. Examples of deep learning processors include Google's Tensor Processing Unit (TPU). TM Rackmount solutions (such as GX4 Rackmount Series) TM GX33 Rackmount Series TM NVIDIA DGX-1 TM Microsoft's Stratix V FPGA TM , Graphcore’s Intelligent Processor Unit (IPU) TM Qualcomm's Snapdragon processors TM Zeroth Platform TMNVIDIA's Volta TM NVIDIA's DRIVE PX TM , NVIDIA's JETSON TX1 / TX2 MODULE TM Intel's Nirvana TM Movidius VPU TM ,Fujitsu DPI TM ARM's DynamicIQ TM IBM TrueNorth TM It has Testa V100s TM Lambda GPU servers and SambaNova's reconfigurable data flow units (RDUs) TM wait.

[0313] The memory subsystem 3322 used in the storage subsystem 3358 may include multiple memories, including a main random access memory (RAM) 3332 for storing instructions and data during program execution and a read-only memory (ROM) 3334 for storing fixed instructions. The file storage subsystem 3336 may provide persistent storage for program files and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical disk drives, or removable media magnetic tapes. Modules implementing certain specific functionalities may be stored by the file storage subsystem 3336 in the storage subsystem 3358 or in other machines accessible to the processor.

[0314] The bus subsystem 3355 provides mechanisms for enabling various components and subsystems of the computer system 3300 to communicate with each other as intended. Although the bus subsystem 3355 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0315] The computer system 3300 itself can be of different types, including personal computers, portable computers, workstations, computer terminals, network computers, televisions, mainframes, server clusters, a loosely networked group of widely distributed computers, or any other data processing system or user equipment. Due to the constantly evolving nature of computers and networks, [the following applies]. Figure 33 The description of the computer system 3300 depicted herein is intended only as a specific example to illustrate a preferred embodiment of the invention. Many other configurations of the computer system 3300 are possible, which have... Figure 33 The computer system depicted in the text has more or fewer components.

[0316] Terms and Conditions

[0317] This invention discloses the following provisions:

[0318] Compression (extrusion)

[0319] 1. A base detection method based on artificial intelligence, the method comprising:

[0320] Access a set of analyte channels generated per cycle for the sequencing run;

[0321] The first window of the analyte channel set in each cycle of the sequencing cycle in the series is processed by a spatial network of a base detector based on a neural network for the sequencing cycle of the sequencing run, and the corresponding sequence of the spatial output set is generated for the corresponding sequencing cycle in the first window of the sequencing cycle.

[0322] The corresponding final spatial output set in the corresponding sequence of the spatial output set is processed by the compression network of the neural network-based base detector, and a corresponding compressed spatial output set is generated for the corresponding sequencing cycle in the first window of the sequencing cycle; and

[0323] Based on the corresponding compressed spatial output set, base detection predictions are generated for one or more sequencing cycles in the first window of the sequencing cycle.

[0324] 2. The artificial intelligence-based method according to Clause 1, wherein the corresponding final spatial output set has M channels (feature maps), wherein the corresponding compressed spatial output set has N channels (feature maps), and wherein M>N.

[0325] 3. The artificial intelligence-based method described in Clause 1 further includes:

[0326] For the second window of the sequencing cycle in the sequencing run, the second window shares with the first window of the sequencing cycle one or more overlapping sequencing cycles for which the spatial network previously generated spatial output sets and at least one non-overlapping sequencing cycle for which the spatial network has not yet generated spatial output sets.

[0327] The spatial network processes the analyte channel set for each non-overlapping sequencing cycle only and generates spatial output set sequences for the non-overlapping sequencing cycles, thereby bypassing the reprocessing of the corresponding analyte channel set for the overlapping sequencing cycles via the spatial network.

[0328] The final spatial output set in the spatial output set sequence is processed by the compression network, and a compressed spatial output set is generated for the non-overlapping sequencing cycle, wherein the final spatial output has M channels (feature maps), and the compressed spatial output has N channels (feature maps), and where M>N;

[0329] Based on the corresponding compressed spatial output set of the overlapping sequencing cycles previously generated for the first window of the sequencing cycle, and based on the compressed spatial output set, base detection predictions for one or more sequencing cycles in the second window of the sequencing cycle are generated, thereby replacing the corresponding compressed spatial output set of the overlapping sequencing cycles with the corresponding per-cycle analyte channel set of the overlapping sequencing cycles.

[0330] 4. The artificial intelligence-based method described in Clause 3 further includes:

[0331] For the third window of the sequencing cycle in the sequencing run, the third window shares with the first and second windows of the sequencing cycle one or more overlapping sequencing cycles for which the spatial network previously generated spatial output sets and at least one non-overlapping sequencing cycle for which the spatial network has not yet generated spatial output sets.

[0332] The spatial network processes the analyte channel set for each non-overlapping sequencing cycle only and generates spatial output set sequences for the non-overlapping sequencing cycles, thereby bypassing the reprocessing of the corresponding analyte channel set for the overlapping sequencing cycles via the spatial network.

[0333] The final spatial output set in the spatial output set sequence is processed by the compression network, and a compressed spatial output set is generated for the non-overlapping sequencing cycle, wherein the final spatial output has M channels (feature maps), and the compressed spatial output has N channels (feature maps), and where M>N;

[0334] Based on the corresponding compressed spatial output sets of the overlapping sequencing cycles generated previously for the first and second windows of the sequencing cycles, and based on the compressed spatial output sets, base detection predictions for one or more sequencing cycles in the third window of the sequencing cycles are generated, thereby replacing the corresponding compressed spatial output sets of the overlapping sequencing cycles with the corresponding per-cycle analyte channel sets of the overlapping sequencing cycles.

[0335] 5. The artificial intelligence-based method according to Clause 1, wherein each per-cycle analyte channel set in the series depicts the intensity recorded in response to nucleotide incorporation in the analyte during the corresponding sequencing cycle in the sequencing run.

[0336] 6. The artificial intelligence-based method according to Clause 5, wherein the spatial network has a sequence of spatial convolutional layers that process each cycle analyte channel set in a specific window of the sequencing cycle for a specific window of the sequencing run, and generate a spatial output set sequence for each sequencing cycle in the specific window of the sequencing cycle, including starting with a first spatial convolutional layer that combines intensity only within the analyte channel set of the target sequencing cycle and not between the analyte channel sets of different sequencing cycles in the specific window of the sequencing cycle, and continuing with subsequent spatial convolutional layers that combine the spatial output of the previous spatial convolutional layer only within the target sequencing cycle and not between the different sequencing cycles in the specific window of the sequencing cycle.

[0337] 7. The artificial intelligence-based method according to Clause 6, wherein the corresponding spatial convolutional layers in the spatial convolutional layer sequence have convolutional filters with different counts, wherein the final spatial convolutional layer in the spatial convolutional layer sequence has M convolutional filters, and wherein M is an integer greater than four.

[0338] 8. The artificial intelligence-based method according to Clause 7, wherein the corresponding spatial convolutional layers in the spatial convolutional layer sequence have convolutional filters with the same count, wherein the same count is M, and wherein M is an integer greater than four.

[0339] 9. The artificial intelligence-based method according to Clause 8, wherein the convolutional filter in the spatial network uses two-dimensional (2D) convolution.

[0340] 10. The artificial intelligence-based method according to Clause 8, wherein the convolutional filter in the spatial network uses three-dimensional (3D) convolution.

[0341] 11. The artificial intelligence-based method according to Clause 6, wherein the neural network-based base detector has a temporal network, wherein the temporal network has a sequence of temporal convolutional layers, the temporal convolutional layer sequence processing in groups the corresponding compressed spatial output sets of windows of subsequent sequencing cycles within the specific window of the sequencing cycle, and generating a sequence of temporal output sets for the specific window of the sequencing cycle, including starting with a first temporal convolutional layer, the first temporal convolutional layer combining compressed spatial output sets between the different sequencing cycles within the specific window of the sequencing cycle, and continuing with subsequent temporal convolutional layers, the subsequent temporal convolutional layers combining subsequent temporal outputs of the previous temporal convolutional layer.

[0342] 12. The artificial intelligence-based method described in Clause 11 further includes:

[0343] For the first window of the sequencing cycle,

[0344] The first temporal convolutional layer in the temporal convolutional layer sequence of the temporal network processes the corresponding compressed spatial output set of the window of the subsequent sequencing cycle in the first window of the sequencing cycle, and generates multiple temporal output sets of the first window of the sequencing cycle.

[0345] The compression network processes the plurality of time output sets and generates corresponding compressed time output sets for the corresponding time output sets in the plurality of time output sets, wherein the corresponding time output sets have M channels (feature maps), wherein the corresponding compressed time output sets have N channels (feature maps), and wherein M>N.

[0346] The corresponding compressed temporal output set is processed through the final temporal convolutional layer in the temporal convolutional layer sequence of the temporal network, and the final temporal output set is generated for the first window of the sequencing cycle; and

[0347] Based on the final time output set, the base detection prediction is generated for one or more sequencing cycles within the first window of the sequencing cycle.

[0348] The output layer processes the final time output set and generates a final output for the first window of the sequencing cycle, wherein the base detection prediction is generated based on the final output.

[0349] 13. The artificial intelligence-based method described in Clause 12 further includes:

[0350] For the second window of a sequencing cycle, the second window shares with the first window of the sequencing cycle one or more overlapping windows of subsequent sequencing cycles for which the first temporal convolutional layer previously generated temporal output sets, and at least one non-overlapping window of subsequent sequencing cycles for which the first temporal convolutional layer has not yet generated temporal output sets.

[0351] The first temporal convolutional layer processes the corresponding compressed spatial output set for the corresponding sequencing cycle in the non-overlapping window of the subsequent sequencing cycle only, and generates a temporal output set for the non-overlapping window of the subsequent sequencing cycle, thereby bypassing the reprocessing of the corresponding compressed spatial output set for the corresponding sequencing cycle in the overlapping window of the subsequent sequencing cycle by the first temporal convolutional layer.

[0352] The time output set is processed by the compression network, and a compressed time output set is generated for the non-overlapping window of the subsequent sequencing cycle, wherein the time output set has M channels (feature maps), wherein the compressed time output has N channels (feature maps), and wherein M>N.

[0353] The final temporal convolutional layer processes the previously generated compressed temporal output set for the first window of the sequencing cycle and the corresponding compressed temporal output set for the overlapping window of the subsequent sequencing cycle, and generates the final temporal output set for the second window of the sequencing cycle, thereby replacing the corresponding compressed temporal output set for the overlapping window of the subsequent sequencing cycle with the corresponding per-cycle analyte channel set for the overlapping window of the subsequent sequencing cycle; and

[0354] Based on the final time output set, the base detection prediction is generated for one or more sequencing cycles within the second window of the sequencing cycle.

[0355] The output layer processes the final time output set and generates a final output for the second window of the sequencing cycle, wherein the base detection prediction is generated based on the final output.

[0356] 14. The artificial intelligence-based method described in Clause 13 further includes:

[0357] For the third window of the sequencing cycle, the third window shares with the first and second windows of the sequencing cycle one or more overlapping windows of subsequent sequencing cycles for which the first temporal convolutional layer previously generated a temporal output set, and at least one non-overlapping window of subsequent sequencing cycles for which the first temporal convolutional layer has not yet generated a temporal output set.

[0358] The first temporal convolutional layer processes the corresponding compressed spatial output set for the corresponding sequencing cycle in the non-overlapping window of the subsequent sequencing cycle only, and generates a temporal output set for the non-overlapping window of the subsequent sequencing cycle, thereby bypassing the reprocessing of the corresponding compressed spatial output set for the corresponding sequencing cycle in the overlapping window of the subsequent sequencing cycle by the first temporal convolutional layer.

[0359] The time output set is processed by the compression network, and a compressed time output set is generated for the non-overlapping window of subsequent sequencing cycles, wherein the time output set has M channels (feature maps), and the compressed time output has N channels (feature maps), where M > N; and

[0360] The final temporal convolutional layer processes the first and second windows of the sequencing cycle and generates the corresponding compressed temporal output sets of the overlapping windows of subsequent sequencing cycles on the compressed temporal output sets, and generates the final temporal output set for the third window of the sequencing cycle, thereby replacing the corresponding compressed temporal output sets of the overlapping windows of subsequent sequencing cycles with the corresponding per-cycle analyte channel sets of the overlapping windows of subsequent sequencing cycles; and

[0361] Based on the final time output set, the base detection prediction is generated for one or more sequencing cycles within the third window of the sequencing cycle.

[0362] The output layer processes the final time output set and generates a final output for the third window of the sequencing cycle, wherein the base detection prediction is generated based on the final output.

[0363] 15. The artificial intelligence-based method according to Clause 11, wherein the corresponding temporal convolutional layers in the temporal convolutional layer sequence of the temporal network have convolutional filters with different counts, wherein the first temporal convolutional layer has M convolutional filters, and wherein M is an integer greater than four.

[0364] 16. The artificial intelligence-based method according to Clause 11, wherein the corresponding temporal convolutional layers in the temporal convolutional layer sequence of the temporal network have convolutional filters with the same count, wherein the same count is M, and wherein M is an integer greater than four.

[0365] 17. The artificial intelligence-based method according to Clause 16, wherein the convolutional filter in the temporal network uses one-dimensional (1D) convolution.

[0366] 18. The artificial intelligence-based method according to Clause 1, wherein the compression network uses 1×1 convolution to control multiple compressed spatial outputs in a compressed spatial output set, wherein the compression network has N convolutional filters, and wherein N is an integer equal to or less than four.

[0367] 19. The AI-based method according to Clause 1 further includes using data identifying unreliable analytes to remove portions of compressed spatial outputs corresponding to the unreliable analytes from a compressed spatial output set, generating a compressed, filtered spatial output set to replace the compressed spatial output set, and generating base detection predictions only for analytes that are not the unreliable analytes.

[0368] 20. The artificial intelligence-based method according to Clause 19 further includes processing the compressed, filtered spatial output set, rather than the corresponding compressed spatial output set, through the time network.

[0369] 21. The artificial intelligence-based method according to Clause 20 further includes generating a compressed temporal output set from the compressed, filtered spatial output set.

[0370] 22. The artificial intelligence-based method according to Clause 19, wherein the data identifying the unreliable analyte identifies pixels that depict the intensity of the unreliable cluster.

[0371] 23. The artificial intelligence-based method according to Clause 19, wherein the data identifying the unreliable analysis object identifies pixels that do not depict any intensity.

[0372] 24. The artificial intelligence-based method according to Clause 20, wherein the total number of pixels in the compressed spatial output set is four to nine times that of the corresponding, compressed, filtered spatial output set.

[0373] 25. The artificial intelligence-based method according to Clause 24, wherein the compressed, filtered spatial output set enables the temporal network to operate on 75% fewer pixels, thereby reducing the computational operations, memory accesses, and memory usage of the temporal network by 75%.

[0374] 26. The artificial intelligence-based method according to Clause 5, wherein the computational operations, memory accesses and memory usage of the temporal network are reduced by 80% by bypassing reprocessing through the spatial network.

[0375] 27. The artificial intelligence-based method according to Clause 14, wherein bypassing reprocessing through the temporary network reduces computational operations, memory accesses, and memory usage of the temporary network.

[0376] 28. The AI-based method according to Clause 27 further includes reallocating computational resources provided by the compressed network to add supplementary convolutional filters in the spatial network and the temporal network.

[0377] 29. The AI-based method according to Clause 27 further includes reallocating computational resources provided by the compression network to add a supplementary per-cycle analyte channel set in each window of the per-cycle analyte channel set used to generate base detection predictions for a particular sequence.

[0378] 30. The AI-based method according to Clause 27 further includes reallocating computing resources provided by the compressed network to add supplemental spatial convolutional layers to the spatial network.

[0379] 31. The AI-based method according to Clause 27 further includes reallocating computing resources provided by the compressed network to add supplemental temporal convolutional layers to the temporal network.

[0380] 32. The artificial intelligence-based method according to Clause 1 further includes combining one or more compressed spatial output sets generated in a previous window for one or more sequencing cycles with one or more compressed spatial output sets generated in a current window for the sequencing cycle to generate base detection predictions for one or more sequencing cycles in the current window of the sequencing cycle.

[0381] 33. The artificial intelligence-based method according to Clause 1 further includes combining one or more compressed spatial output sets generated for one or more subsequent windows of a sequence cycle with one or more compressed spatial output sets generated for the current window of the sequence cycle to generate base detection predictions for one or more sequencing cycles in the current window of the sequencing cycle.

[0382] 34. The artificial intelligence-based method according to Clause 1 further includes combining one or more compressed time output sets generated in a previous window for one or more sequencing cycles with one or more compressed time output sets generated in a current window for the sequencing cycle to generate base detection predictions for one or more sequencing cycles in the current window for the sequencing cycle.

[0383] 35. The artificial intelligence-based method according to Clause 1 further includes combining one or more compressed time output sets generated for one or more subsequent windows of a sequence cycle with one or more compressed time output sets generated for the current window of the sequence cycle to generate base detection predictions for one or more sequencing cycles in the current window of the sequencing cycle.

[0384] 36. The artificial intelligence-based method according to Clause 1, wherein each cycle of analyte channel set encodes analyte data of the analytes sequenced during the sequencing run.

[0385] 37. The artificial intelligence-based method according to Clause 36, wherein the analyte data is image data identifying intensity emissions collected from the analyte.

[0386] 38. The artificial intelligence-based method according to Clause 37, wherein the image data has multiple image channels (images).

[0387] 39. The artificial intelligence-based method according to Clause 38, wherein an image channel (image) is generated by a combination of (i) illumination with a specific laser and (ii) imaging through a specific filter.

[0388] 40. The artificial intelligence-based method according to Clause 36, wherein the analyte data is based on current and / or voltage data for analyte activity detection.

[0389] 41. The artificial intelligence-based method according to Clause 36, wherein the analyte data is pH-scaled data based on analyte activity detection.

[0390] 42. The artificial intelligence-based method according to Clause 1, wherein the number of channels in each per-cycle analysis channel set in the series determines the number of convolutional filters in the compression network, and thus determines the number of channels in the compressed spatial output set and the compressed temporal output set.

[0391] 43. The artificial intelligence-based method according to Clause 1, wherein the compressed spatial output set, the compressed, filtered spatial output set, and the compressed temporal output set are stored in a quantized form.

[0392] 44. A system comprising:

[0393] A host memory, which is attached to a host processor and configured to receive the progress of sequencing images during a sequencing run;

[0394] A configurable processor having an array of processing units, wherein the processing units in the array are configured to perform a neural network-based base detector to generate base detection predictions;

[0395] Data stream logic, which is capable of accessing the host memory, the host processor, and the configurable processor, and is configured to load a sequencing image of a sequencing cycle in a first window of the sequencing cycle from the host memory on the configurable processor;

[0396] The runtime logic is configured to cause the processing unit to execute the spatial network of the neural network-based base detector on the sequencing image of the sequencing cycle in the first window of the sequencing cycle one by one, and to generate a spatial feature map set for each sequencing cycle in the first window of the sequencing cycle.

[0397] The runtime logic is configured to cause the processing unit to execute the neural network-based base detector compression network on the spatial feature map in a loop and generate a compressed spatial feature map, and to process the compressed spatial feature map through a temporal network and an output network to generate base detection predictions for one or more sequencing cycles in the first window of the sequencing cycle.

[0398] The data stream logic is configured to move the compressed spatial feature map to the host memory and rewrite the sequencing image with the compressed spatial feature map.

[0399] For a second window of a sequencing cycle that shares one or more overlapping sequencing cycles with the first window of the sequencing cycle and has at least one non-overlapping sequencing cycle, the data flow logic is configured to load a compressed spatial feature map of the overlapping sequencing cycles and a sequencing image of the non-overlapping sequencing cycles from the host memory on the configurable processor.

[0400] The runtime logic is configured to cause the processing unit to perform the spatial network on the sequencing image for the non-overlapping sequencing cycle, and to generate a spatial feature map set for the non-overlapping sequencing cycle; and

[0401] The runtime logic is configured to cause the processing unit to execute the compression network on the spatial feature map set and generate a compressed spatial feature map set for the non-overlapping sequencing cycle, and to process the compressed spatial feature map set of the overlapping sequencing cycle and the compressed spatial feature map set of the non-overlapping sequencing cycle through the temporal network and the output network to generate base detection predictions for one or more sequencing cycles in the second window of the sequencing cycle.

[0402] 45. A system comprising:

[0403] A host memory, which is attached to a host processor and configured to receive the progress of sequencing images during a sequencing run;

[0404] A configurable processor having an array of processing units attached to a processor memory, wherein the processing units in the array of processing units are configured to execute a neural network-based base detector to generate base detection predictions.

[0405] A data stream logic, which is capable of accessing the host memory, the host processor, the configurable processor, and the processor memory, and is configured to load a sequencing image of a sequencing cycle in a first window of the sequencing cycle from the host memory on the configurable processor;

[0406] The runtime logic is configured to cause the processing unit to execute the spatial network of the neural network-based base detector on the sequencing image of the sequencing cycle in the first window of the sequencing cycle one by one, and to generate a spatial feature map set for each sequencing cycle in the first window of the sequencing cycle.

[0407] The runtime logic is configured to cause the processing unit to execute the neural network-based base detector compression network on the spatial feature map in a loop and generate a compressed spatial feature map, and to process the compressed spatial feature map through a temporal network and an output network to generate base detection predictions for one or more sequencing cycles in the first window of the sequencing cycle.

[0408] The data flow logic is configured to move the compressed spatial feature map to the processor memory;

[0409] For a second window of a sequencing cycle that shares one or more overlapping sequencing cycles with the first window of the sequencing cycle and has at least one non-overlapping sequencing cycle, the data flow logic is configured to load a compressed spatial feature map of the overlapping sequencing cycles from the host memory on the configurable processor, and load a sequencing image of the non-overlapping sequencing cycles from the host memory.

[0410] The runtime logic is configured to cause the processing unit to perform the spatial network on the sequencing image for the non-overlapping sequencing cycle, and to generate a spatial feature map set for the non-overlapping sequencing cycle; and

[0411] The runtime logic is configured to cause the processing unit to execute the compression network on the spatial feature map set and generate a compressed spatial feature map set for the non-overlapping sequencing cycle, and to process the compressed spatial feature map set of the overlapping sequencing cycle and the compressed spatial feature map set of the non-overlapping sequencing cycle through the temporal network and the output network to generate base detection predictions for one or more sequencing cycles in the second window of the sequencing cycle.

[0412] 46. ​​A system comprising:

[0413] The neural network logic is configured to perform a first traversal of the neural network graph to independently process a corresponding input in a first set of inputs through a first processing logic, and to generate a corresponding alternative representation of the corresponding input in the first set of inputs without mixing information among the corresponding inputs in the first set of inputs, and to generate the output of the first traversal based on the corresponding alternative representation of the corresponding input in the first set of inputs;

[0414] The neural network logic is configured to perform a second traversal of the neural network graph to independently process the corresponding inputs in the second set of inputs through the first processing logic, and to generate a corresponding alternative representation of the corresponding inputs in the second set of inputs without mixing information between the corresponding inputs in the second set of inputs, and to generate the output of the second traversal based on the corresponding alternative representation of the corresponding inputs in the second set of inputs, wherein the first set of inputs and the second set of inputs have one or more overlapping inputs and at least one non-overlapping input;

[0415] Runtime logic, configured with the neural network logic to perform the first traversal to generate the corresponding alternative representation of the corresponding input in the first set of inputs, store the corresponding alternative representation of the corresponding input in the first set of inputs in compressed form in memory, and generate the output of the first traversal based on the compressed form of the corresponding alternative representation of the corresponding input in the first set of inputs; and

[0416] The runtime logic is configured to perform a second traversal to process only the non-overlapping inputs through the first processing logic and generate alternative representations of the non-overlapping inputs, to store the alternative representations of the non-overlapping inputs in the compressed form in memory, to retrieve the compressed form of the corresponding alternative representation of the overlapping inputs generated in the first traversal, to compensate for redundant generation of the corresponding alternative representations of the overlapping inputs in the second traversal, and to generate the output of the second traversal based on the compressed form of the corresponding alternative representation of the overlapping inputs and the compressed form of the alternative representation of the non-overlapping inputs.

[0417] 47. The system according to Clause 46, wherein the memory is an on-chip memory.

[0418] 48. The system according to Clause 46, wherein the memory is off-chip memory.

[0419] 49. The system according to Clause 46, wherein the number of channels in the compressed form corresponds to the number of channels in the first set of inputs and the second set of inputs.

[0420] 50. A base detection method based on artificial intelligence, the method comprising:

[0421] Access a series of per-cycle analyte channel sets generated for a sequencing cycle of a sequencing run, wherein the target per-cycle analyte channel set encodes analyte data detected for an analyte in the target sequencing cycle of the sequencing run;

[0422] The target per-cycle analyte channel set is processed by the first processing module of the neural network, and an intermediate representation of the target per-cycle analyte channel set with M feature maps is generated.

[0423] The intermediate representation is processed by the second processing module of the neural network to generate a simplified intermediate representation of the target per-cycle analyte channel set with N feature maps, where M>N; and

[0424] Using the simplified intermediate representation of the target analyte channel set per cycle, base detection predictions are generated for the analyte in the target sequencing cycle and / or other sequencing cycles of the sequencing run.

[0425] 51. The artificial intelligence-based method according to Clause 50, wherein the first processing module is a convolutional layer having M convolutional filters.

[0426] 52. The artificial intelligence-based method according to Clause 50, wherein the second processing module is a convolutional layer having N convolutional filters.

[0427] 53. A base detection method based on artificial intelligence, the method comprising:

[0428] Based on the sliding window, a neural network-based base detector processes the progress of the analyte channel set generated for each sequencing cycle in the sequencing run, enabling subsequent sliding windows to have overlapping sequencing cycles, including:

[0429] For the current window of a sequencing cycle that includes one or more pre-sequencing cycles, a central sequencing cycle, and one or more subsequent sequencing cycles:

[0430] Based on applying the neural network-based base detector to the current window of the analyte channel set for each cycle, a spatial intermediate representation and a compressed intermediate representation are generated for each of the preceding sequencing cycle, the central sequencing cycle, and the subsequent sequencing cycles, wherein the spatial intermediate representation has M channels, the compressed intermediate representation has N channels, and M > N; and

[0431] Based on the compressed intermediate representation generated for the preceding sequencing cycle, the central sequencing cycle, and the subsequent sequencing cycles, base detection is performed at least for the central sequencing cycle; and

[0432] Base detection is performed at least in the central sequencing cycle within the next window of the sequencing cycle using the compressed intermediate representation generated for the preceding sequencing cycle, the central sequencing cycle, and the subsequent sequencing cycles.

[0433] 54. An artificial intelligence-based system for base detection, the system comprising:

[0434] Host processor;

[0435] The host processor can access a memory that stores analyte data for sequencing cycles used in sequencing runs; and

[0436] A configurable processor, capable of accessing the memory, comprising:

[0437] Multiple execution clusters, wherein the execution clusters are configured as execution neural networks; and

[0438] A data flow logic, capable of accessing the memory and the execution clusters among the plurality of execution clusters, is configured to provide the analyte data to an available execution cluster among the plurality of execution clusters, causing the execution cluster to apply the analyte data to the neural network to generate an intermediate representation and a compressed intermediate representation of the analyte data for use in the current base detection step, and to feed the compressed intermediate representation back to the memory for use as a replacement for the analyte data in future base detection steps, wherein the intermediate representation has M channels, the compressed intermediate representation has N channels, and M>N.

[0439] 55. A system comprising:

[0440] Runtime logic, configured to perform a first iteration of the base detector to process the input and generate an intermediate representation of the input;

[0441] Compression logic, configured to process the intermediate representation and generate a compressed intermediate representation of the input; and

[0442] The runtime logic is configured to use the compressed intermediate representation instead of the input in subsequent iterations of the base detector.

[0443] 56. A system comprising:

[0444] Runtime logic, configured to perform a first iteration of the base detector to process the input and generate an intermediate representation of the input;

[0445] Compression logic, configured to process the intermediate representation and generate a compressed intermediate representation, wherein the compressed intermediate representation is configured to have the same number of channels as the input; and

[0446] The runtime logic is configured to use the compressed intermediate representation instead of the input in subsequent iterations of the base detector.

[0447] 57. The system according to Clause 56, wherein the channel corresponds to a feature map.

[0448] 58. The system according to Clause 56, wherein the channel corresponds to the depth dimension.

[0449] 59. The system according to Clause 56, wherein the channel corresponds to a spatial dimension.

[0450] Split

[0451] 1. A system comprising:

[0452] A spatial convolutional network is configured to convolve the corresponding per-cycle sequencing image set in the window of each cycle sequencing image set through the corresponding sequences of the spatial convolutional layer, and process the window of the per-cycle sequencing image set of a series of sequencing cycles for sequencing runs in turn to generate a corresponding per-cycle spatial feature map set for the corresponding sequencing cycle in the series of sequencing cycles.

[0453] The corresponding sequence of the spatial convolutional layer has a corresponding sequence of spatial convolutional filter banks, wherein the trained coefficients of the spatial convolutional filters in the spatial convolutional filter banks of the corresponding sequence of the spatial convolutional layer vary between the spatial convolutional layer sequences in the corresponding sequence of the spatial convolutional layer.

[0454] A temporal convolutional network is configured to process each cyclic spatial feature map set in groups by convolving corresponding overlapping groups of each cyclic spatial feature map set with corresponding temporal convolutional filter banks of a first temporal convolutional layer, so as to generate corresponding temporal feature map sets for each group of overlapping groups of each cyclic spatial feature map set; and

[0455] The trained coefficients of the temporal convolutional filters in the respective temporal convolutional filter banks vary among the temporal convolutional filter banks in the respective temporal convolutional filter banks.

[0456] 2. The system according to Clause 1, wherein the spatial convolution filter uses cyclic in-isolation convolution.

[0457] 3. The system according to Clause 1, wherein the temporal convolution filter uses intercyclic combined convolution.

[0458] 4. The system according to Clause 1 is further configured to include a compression network that convolves the respective per-cycle spatial feature map set with corresponding compression convolutional layers to generate a corresponding per-cycle compressed spatial feature map set for the respective sequencing cycle.

[0459] 5. The system according to Clause 4, wherein the trained coefficients of the compressed convolutional filters in the respective compressed convolutional layers vary between the compressed convolutional layers in the respective compressed convolutional layers.

[0460] 6. The system according to Clause 5, wherein the temporal convolutional network is further configured to process each set of temporal features in groups by convolving the corresponding overlapping groups of each set of temporal features in each set of temporal features with the corresponding temporal convolutional filter bank of the second temporal convolutional layer, so as to generate a corresponding additional set of temporal features for the corresponding overlapping groups of each set of temporal features.

[0461] 7. The system according to Clause 6 is further configured to include an output network that processes the final temporal feature map generated by the final temporal convolutional layer to generate a final output.

[0462] 8. The system according to Clause 7 is further configured to generate base detection predictions for one or more sequencing cycles in the series of sequencing cycles based on the final output.

[0463] 9. A system comprising:

[0464] A spatial convolutional network is configured to convolve the corresponding per-cycle sequencing image set in the window of each cycle sequencing image set through the corresponding sequences of the spatial convolutional layer, and process the window of the per-cycle sequencing image set of a series of sequencing cycles for sequencing runs in turn to generate a corresponding per-cycle spatial feature map set for the corresponding sequencing cycle in the series of sequencing cycles.

[0465] A temporal convolutional network is configured to process each cyclic spatial feature map set group by grouping it by convolving corresponding overlapping groups of each cyclic spatial feature map set with corresponding temporal convolutional filter banks, so as to generate corresponding temporal feature map sets for each group of overlapping groups of each cyclic spatial feature map set; and

[0466] The trained coefficients of the temporal convolutional filters in the respective temporal convolutional filter banks vary among the temporal convolutional filter banks in the respective temporal convolutional filter banks.

[0467] 10. The system according to Clause 9, wherein the corresponding sequence of spatial convolutional layers has a corresponding sequence of spatial convolutional filter banks, wherein the trained coefficients of the spatial convolutional filters in the spatial convolutional filter banks of the corresponding sequence of spatial convolutional filter banks are shared among the spatial convolutional layer sequences in the corresponding sequence of spatial convolutional layers.

[0468] 11. The system according to Clause 9 is further configured to include a compression network that convolves the respective per-cycle spatial feature map set through respective compression convolutional layers to generate a respective per-cycle compressed spatial feature map set for the respective sequencing cycle, wherein the trained coefficients of the compression convolutional filters in the respective compression convolutional layers vary between the compression convolutional layers in the respective compression convolutional layers.

[0469] 12. A base detection method based on artificial intelligence, the method comprising:

[0470] The corresponding sequence of each sequencing image set in the window of each sequencing image set is convolved by the corresponding sequence of the spatial convolutional layer. The window of each sequencing image set of a series of sequencing cycles for sequencing operation is processed cyclically by the spatial convolutional network, and the corresponding spatial feature map set of each sequencing cycle is generated for the corresponding sequencing cycle in the series of sequencing cycles.

[0471] The corresponding sequence of the spatial convolutional layer has a corresponding sequence of spatial convolutional filter banks, wherein the trained coefficients of the spatial convolutional filters in the spatial convolutional filter banks of the corresponding sequence of the spatial convolutional layer vary between the spatial convolutional layer sequences in the corresponding sequence of the spatial convolutional layer.

[0472] By using the corresponding temporal convolutional filter banks of the first temporal convolutional layer to convolve the corresponding overlapping groups of each cycle spatial feature map set in each cycle spatial feature map set, the temporal convolutional network processes each cycle spatial feature map set group by group, and generates corresponding temporal feature map sets for each group of overlapping groups in each cycle spatial feature map set; and

[0473] The trained coefficients of the temporal convolutional filters in the respective temporal convolutional filter banks vary among the temporal convolutional filter banks in the respective temporal convolutional filter banks.

[0474] 13. The artificial intelligence-based method according to Clause 12 further includes convolving the corresponding per-cycle spatial feature map set by the corresponding compressed convolutional layers of the compressed network, and generating a corresponding per-cycle compressed spatial feature map set for the corresponding sequencing cycle.

[0475] 14. The artificial intelligence-based method according to Clause 13, wherein the trained coefficients of the compressed convolutional filters in the respective compressed convolutional layers vary between the compressed convolutional layers in the respective compressed convolutional layers.

[0476] 15. The artificial intelligence-based method according to Clause 14 further includes convolving the corresponding overlapping groups of each set of time feature maps in each set of time feature maps by using the corresponding temporal convolutional filter bank of the second temporal convolutional layer, processing each set of time feature maps by group through the temporal convolutional network, and generating a corresponding additional set of time feature maps for each set of time feature maps for the corresponding overlapping groups of each set of time feature maps.

[0477] 16. The artificial intelligence-based method according to Clause 15 further includes processing the final temporal feature map generated by the final temporal convolutional layer through the output network, and generating the final output.

[0478] 17. The artificial intelligence-based method according to Clause 16 further includes generating base detection predictions for one or more sequencing cycles in the series of sequencing cycles based on the final output.

[0479] 18. A base detection method based on artificial intelligence, the method comprising:

[0480] The corresponding sequence of each sequencing image set in the window of each sequencing image set is convolved by the corresponding sequence of the spatial convolutional layer. The window of each sequencing image set of a series of sequencing cycles for sequencing operation is processed cyclically by the spatial convolutional network, and the corresponding spatial feature map set of each sequencing cycle is generated for the corresponding sequencing cycle in the series of sequencing cycles.

[0481] By using the corresponding temporal convolutional filter banks of the first temporal convolutional layer to convolve the corresponding overlapping groups of each cycle spatial feature map set in each cycle spatial feature map set, the temporal convolutional network processes each cycle spatial feature map set group by group, and generates corresponding temporal feature map sets for each group of overlapping groups in each cycle spatial feature map set; and

[0482] The trained coefficients of the temporal convolutional filters in the respective temporal convolutional filter banks vary among the temporal convolutional filter banks in the respective temporal convolutional filter banks.

[0483] 19. The artificial intelligence-based method according to Clause 18, wherein the corresponding sequence of spatial convolutional layers has a corresponding sequence of spatial convolutional filter banks, wherein the trained coefficients of the spatial convolutional filters in the spatial convolutional filter banks of the corresponding sequence of spatial convolutional filter banks are shared among the spatial convolutional layer sequences in the corresponding sequence of spatial convolutional layers.

[0484] 20. The artificial intelligence-based method according to Clause 18 further includes convolving the respective per-cycle spatial feature map set by the respective compressed convolutional layers of the compressed network, and generating a respective per-cycle compressed spatial feature map set for the respective sequencing cycle, wherein the trained coefficients of the compressed convolutional filters in the respective compressed convolutional layers vary between the compressed convolutional layers in the respective compressed convolutional layers.

[0485] 21. A system comprising:

[0486] A spatial convolutional network, configured to apply corresponding sequences from spatial convolutional layers to corresponding per-cycle sequencing images within a window of each cycle sequencing image; and

[0487] The corresponding sequence of the spatial convolutional layer has a corresponding sequence of the spatial convolutional filter bank, and the corresponding sequence of the spatial convolutional filter bank is different from one sequence of the spatial convolutional layer to another sequence of the spatial convolutional layer.

[0488] 22. A system comprising:

[0489] A temporal convolutional network, wherein the temporal convolutional network is configured with a first temporal convolutional layer, the first temporal convolutional layer being configured to apply corresponding sets of temporal convolutional filters to corresponding sliding windows of a spatial feature map; and

[0490] The corresponding multiple sets of temporal convolutional filters in the first temporal convolutional layer have different temporal convolutional filters from one set of temporal convolutional filters to another set of temporal convolutional filters.

[0491] 23. The system according to Clause 22, wherein the temporal convolutional network is configured with a second temporal convolutional layer following the first temporal convolutional layer, wherein the second convolutional layer is configured to apply corresponding multiple sets of temporal convolutional filters to corresponding sliding windows of a temporal feature map, and wherein the corresponding multiple sets of temporal convolutional filters in the second temporal convolutional layer have different temporal convolutional filters from one set of temporal convolutional filters to another set of temporal convolutional filters.

[0492] jump

[0493] 1. A system comprising:

[0494] A spatial convolutional network is configured to process windows of each sequencing image set in a series of sequencing cycles sequentially by processing corresponding sequencing image sets in windows of each sequencing cycle image set through corresponding spatial processing pipelines. The corresponding spatial processing pipelines are configured to convolve the corresponding sequencing image sets of each sequencing cycle through corresponding sequences of spatial convolutional layers to generate corresponding spatial feature maps for each sequencing cycle in the series of sequencing cycles.

[0495] A bus network connected to the spatial convolutional network and configured to form a bus between spatial convolutional layers within the respective sequences of spatial convolutional layers, the bus being configured to combine corresponding per-cycle spatial feature maps generated by two or more spatial convolutional layers in a particular spatial convolutional layer sequence for a particular sequencing cycle into a combined per-cycle spatial feature map, and to provide the combined per-cycle spatial feature map as input to another spatial convolutional layer in the particular sequence of spatial convolutional layers.

[0496] 2. The system according to Clause 1, wherein the two or more spatial convolutional layers comprise a first spatial convolutional layer and a third spatial convolutional layer, wherein the first spatial convolutional layer generates a first per-cycle spatial feature map set, wherein the first spatial convolutional layer provides the first per-cycle spatial feature map set as input to a second spatial convolutional layer, wherein the second spatial convolutional layer processes the first per-cycle spatial feature map set and generates a second per-cycle spatial feature map set, wherein the second spatial convolutional layer provides the second per-cycle spatial feature map set as input to the third spatial convolutional layer, and wherein the third spatial convolutional layer processes the second per-cycle spatial feature map set and generates a third per-cycle spatial feature map set.

[0497] 3. The system according to Clause 2, wherein the bus is further configured to combine the first spatial feature set and the third per-cycle spatial feature set into the combined per-cycle spatial feature set.

[0498] 4. The system according to Clause 3, wherein the other spatial convolutional layer is a fourth spatial convolutional layer, which is immediately following the third spatial convolutional layer in the particular sequence of spatial convolutional layers, wherein the fourth spatial convolutional layer processes the combined per-cycle spatial feature map set as input.

[0499] 5. The system according to Clause 2, wherein the two or more spatial convolutional layers include a first spatial convolutional layer and a seventh spatial convolutional layer, wherein the third spatial convolutional layer provides the third per-cycle spatial feature map set as input to a fourth spatial convolutional layer, wherein the fourth spatial convolutional layer processes the third per-cycle spatial feature map set and generates a fourth per-cycle spatial feature map set, wherein the fourth spatial convolutional layer provides the fourth per-cycle spatial feature map set as input to a fifth spatial convolutional layer, and wherein the fifth spatial convolutional layer processes the fourth per-cycle spatial feature map set and generates a fifth per-cycle spatial feature map set.

[0500] 6. The system according to Clause 5, wherein the bus is further configured to combine the first spatial feature set and the fifth spatial feature set into a per-cycle spatial feature set of the combination.

[0501] 7. The system according to Clause 6, wherein the other spatial convolutional layer is a sixth spatial convolutional layer, which is immediately following the fifth spatial convolutional layer in the particular sequence of spatial convolutional layers, wherein the sixth spatial convolutional layer processes the combined per-cycle spatial feature map set as input.

[0502] 8. The system according to Clause 5, wherein the two or more spatial convolutional layers include the first spatial convolutional layer, the third spatial convolutional layer, and the fifth spatial convolutional layer, and wherein the bus is further configured to combine the first per-cycle spatial feature set, the third per-cycle spatial feature set, and the fifth per-cycle spatial feature set into the combined per-cycle spatial feature set.

[0503] 9. The system according to Clause 8, wherein the other spatial convolutional layer is a sixth spatial convolutional layer that processes the per-cycle spatial feature map set of the combination as input.

[0504] 10. The system according to Clause 1, wherein the bus is further configured to combine the per-cycle sequencing image set of the particular sequencing cycle provided as input to the first spatial convolutional layer and the third per-cycle spatial feature map set into the combined per-cycle spatial feature map set.

[0505] 11. The system according to Clause 10, wherein the other spatial convolutional layer is a fourth spatial convolutional layer that processes the per-cycle spatial feature map set of the combination as input.

[0506] 12. The system according to Clause 1, wherein the bus network is further configured to include dimension compatibility logic, the dimension compatibility logic being configured to modify the spatial and depth dimensions of an incoming per-cycle spatial feature set combined with a received per-cycle spatial feature set to generate the combined per-cycle spatial feature set.

[0507] 13. The system according to Clause 12, wherein the dimensionality compatibility logic is a dimensionality reduction operation, including convolution, pooling, or averaging.

[0508] 14. The system according to Clause 12, wherein the bus network is further configured to include scaling logic, the scaling logic being configured to scale the feature values ​​of the incoming per-cycle spatial feature set combined with the received per-cycle spatial feature set to generate the combined per-cycle spatial feature set.

[0509] 15. The system according to Clause 1 is further configured to include a temporal convolutional network configured to process each cyclic spatial feature map in groups by convolving a corresponding overlapping group of each cyclic spatial feature map in the cyclic spatial feature map by using a corresponding temporal convolutional filter bank of a first temporal convolutional layer to generate a corresponding group of temporal feature maps for the corresponding overlapping group of each cyclic spatial feature map.

[0510] 16. The system according to Clause 15 is further configured to include the bus network connected to the temporal convolutional network and configured to form a bus between temporal convolutional layers within the respective sequences of the temporal convolutional layers, the bus being configured to combine corresponding per-cycle temporal feature maps generated by two or more temporal convolutional layers in a particular sequence of the temporal convolutional layers for a particular sequencing cycle into a combined per-cycle temporal feature map, and to provide the combined per-cycle spatial feature map as input to another temporal convolutional layer in the particular sequence of the temporal convolutional layers.

[0511] 17. An artificial intelligence-based method, comprising:

[0512] The corresponding sequencing image set in each cycle of sequencing image set is processed in the window of the corresponding sequencing image set in each cycle through the corresponding spatial processing pipeline, including convolving the corresponding sequencing image set in each cycle of sequencing through the corresponding sequence of the spatial convolutional layer, and processing the window of the sequencing image set in each cycle of a series of sequencing cycles through the spatial convolutional network in turn, so as to generate a corresponding spatial feature map set for each cycle of sequencing for the corresponding sequencing cycle in the series of sequencing cycles; and

[0513] The corresponding per-cycle spatial feature maps generated by two or more spatial convolutional layers in a specific sequence for a specific sequencing cycle are combined into a combined per-cycle spatial feature map, and the combined per-cycle spatial feature map is provided as input to another spatial convolutional layer in the specific sequence.

[0514] 18. The artificial intelligence-based method according to Clause 17, wherein the two or more spatial convolutional layers include a first spatial convolutional layer and a third spatial convolutional layer, wherein the first spatial convolutional layer generates a first per-cycle spatial feature map set, wherein the first spatial convolutional layer provides the first per-cycle spatial feature map set as input to a second spatial convolutional layer, wherein the second spatial convolutional layer processes the first per-cycle spatial feature map set and generates a second per-cycle spatial feature map set, wherein the second spatial convolutional layer provides the second per-cycle spatial feature map set as input to the third spatial convolutional layer, and wherein the third spatial convolutional layer processes the second per-cycle spatial feature map set and generates a third per-cycle spatial feature map set.

[0515] 19. The artificial intelligence-based method according to Clause 18, wherein the bus is further configured to combine the first spatial feature set and the third per-cycle spatial feature set into the combined per-cycle spatial feature set.

[0516] 20. The artificial intelligence-based method according to Clause 19, wherein the other spatial convolutional layer is a fourth spatial convolutional layer, which is immediately following the third spatial convolutional layer in the particular sequence of spatial convolutional layers, wherein the fourth spatial convolutional layer processes the combined per-cycle spatial feature map set as input.

[0517] Other embodiments of the above methods may include a non-transitory computer-readable storage medium storing instructions that can be executed by a processor to perform any of the methods described above. Yet another embodiment of the methods described in this section may include a system comprising a memory and one or more processors operable to execute instructions stored in the memory to perform any of the methods described above.

Claims

1. A system comprising: At least one processor; and A non-transitory computer-readable medium containing instructions that, when executed by the at least one processor, cause the system to: A set of sequencing images per cycle, containing signals from nucleic acid reaction sites, is captured at imaging events of sequencing cycles during a sequencing run using one or more photodetectors of the sequencing instrument. The spatial convolutional network component of the convolutional neural network independently convolves the window of each cycle sequencing image set with each associated intra-cycle data in the corresponding cycle sequencing image set of the corresponding depiction cluster through independent sequences of the spatial convolutional layer. The window of the cycle sequencing image set of the depiction cluster of a series of sequencing cycles is processed cyclically for each cycle to generate a corresponding cycle spatial feature map for each corresponding sequencing cycle in the series of sequencing cycles. The cycle spatial feature map encodes an alternative representation of the captured signal within the cycle sequencing image set. The per-cycle sequencing image set depicting the clusters illustrates the intensity emission generated due to the incorporation of nucleotides into the clusters of associated analytes on the substrate during the sequencing cycle of the sequencing run; The corresponding sequence of the spatial convolutional layer has the corresponding sequence of the spatial convolutional filter bank, wherein the trained coefficients of the spatial convolutional filter in the spatial convolutional filter bank vary between the spatial convolutional layer sequences in the corresponding sequence of the spatial convolutional layer. The temporal convolutional network component of the convolutional neural network processes each cyclic spatial feature map set in groups by convolving the corresponding overlapping groups of each cyclic spatial feature map set with the corresponding temporal convolutional filter bank of the first temporal convolutional layer, so as to generate a corresponding temporal feature map set for each cyclic spatial feature map set with respect to the corresponding overlapping groups of each cyclic spatial feature map set. The trained coefficients of the temporal convolutional filters in the respective temporal convolutional filter banks vary among the temporal convolutional filter banks in the respective temporal convolutional filter banks; and By processing each set of temporal feature maps via the output network component of a convolutional neural network, a base detection prediction for the corresponding signal captured in each cycle of sequencing image set is generated.

2. The system of claim 1, wherein the spatial convolution filter uses cyclic in-isolation convolution.

3. The system according to claim 1 or 2, wherein the temporal convolution filter uses intercyclic combined convolution.

4. The system according to claim 1 or 2, further comprising instructions that, when executed by the at least one processor, cause the system to convolve the corresponding per-cycle spatial feature map set using a compressed network through corresponding compressed convolutional layers, to generate a corresponding per-cycle compressed spatial feature map set for the corresponding sequencing cycle.

5. The system of claim 4, wherein the trained coefficients of the compressed convolutional filters in the respective compressed convolutional layers vary between the compressed convolutional layers in the respective compressed convolutional layers.

6. The system of claim 1 or 2, further comprising instructions, when executed by the at least one processor, causing the system to process each set of time features in groups by convolving the corresponding overlapping groups of each set of time features in each set of time features via the temporal convolutional network component using the corresponding temporal convolutional filter bank of the second temporal convolutional layer, to generate a corresponding additional set of time features for the corresponding overlapping groups of each set of time features.

7. A system comprising: At least one processor; and A non-transitory computer-readable medium containing instructions that, when executed by the at least one processor, cause the system to: A set of sequencing images per cycle, containing signals from nucleic acid reaction sites, is captured at imaging events of sequencing cycles during a sequencing run using one or more photodetectors of the sequencing instrument. By means of a spatial convolutional network, the windows of each cycle sequencing image set are independently convolved with each associated intra-cycle data in the corresponding cycle sequencing image set of the corresponding depiction cluster through independent sequences of the spatial convolutional layer. The windows of the cycle sequencing image set of the depiction cluster of a series of sequencing cycles in the sequencing cycle are processed cyclically to generate a corresponding cycle spatial feature map set for the corresponding sequencing cycle in the series of sequencing cycles. The cycle spatial feature map set encodes an alternative representation of the captured signal in the cycle sequencing image set. Each cyclic spatial feature map is processed in groups by using a temporal convolutional network to convolve the corresponding overlapping groups of each cyclic spatial feature map in each cyclic spatial feature map set using a corresponding temporal convolutional filter bank, so as to generate a corresponding group of temporal feature maps for the corresponding overlapping groups of each cyclic spatial feature map set. The trained coefficients of the temporal convolutional filters in the corresponding temporal convolutional filter bank vary among the temporal convolutional filter banks in the corresponding temporal convolutional filter bank. as well as For each sequencing cycle and based on the corresponding time feature map set, a base detection prediction of nucleotide type is generated for the corresponding signal captured in the sequencing image set of each cycle.

8. The system of claim 7, wherein the corresponding sequence of spatial convolutional layers has a corresponding sequence of spatial convolutional filter banks, wherein the trained coefficients of the spatial convolutional filters in the spatial convolutional filter banks of the corresponding sequence of spatial convolutional filter banks are shared among the spatial convolutional layer sequences in the corresponding sequence of spatial convolutional layers.

9. The system of claim 7 or 8, further comprising instructions that, when executed by the at least one processor, cause the system to convolve the corresponding per-cycle spatial feature map set through corresponding compressed convolutional layers using a compressed network to generate a corresponding per-cycle compressed spatial feature map set for the corresponding sequencing cycle, wherein the trained coefficients of the compressed convolutional filters in the corresponding compressed convolutional layers vary between the compressed convolutional layers in the corresponding compressed convolutional layers.

10. A base detection method based on artificial intelligence, the method comprising: A set of sequencing images per cycle, containing signals from nucleic acid reaction sites, is captured at imaging events of sequencing cycles during a sequencing run using one or more photodetectors of the sequencing instrument. The spatial convolutional network component of the convolutional neural network processes the window of the per-cycle sequencing image set of the depicting cluster for a series of sequencing cycles in a cyclic manner as follows: the window of the per-cycle sequencing image set is independently convolved with each associated intra-cycle data in the per-cycle sequencing image set of the corresponding depicting cluster through independent sequences of the spatial convolutional layer, and a corresponding per-cycle spatial feature map set is generated for each corresponding sequencing cycle in the series of sequencing cycles, the per-cycle spatial feature map set encoding an alternative representation of the captured signal within the per-cycle sequencing image set; The per-cycle sequencing image set depicting the clusters illustrates the intensity emission generated due to the incorporation of nucleotides into the clusters of associated analytes on the substrate during the sequencing cycle of the sequencing run; The corresponding sequence of the spatial convolutional layer has the corresponding sequence of the spatial convolutional filter bank, wherein the trained coefficients of the spatial convolutional filter in the spatial convolutional filter bank vary between the spatial convolutional layer sequences in the corresponding sequence of the spatial convolutional layer. The temporal convolutional network component of the convolutional neural network processes each cyclic spatial feature map set in groups in the following manner: by convolving the corresponding overlapping groups of each cyclic spatial feature map set in the cyclic spatial feature map set using the corresponding temporal convolutional filter group of the first temporal convolutional layer, and generating a corresponding temporal feature map set for the corresponding overlapping groups of each cyclic spatial feature map set. The trained coefficients of the temporal convolutional filters in the respective temporal convolutional filter banks vary among the temporal convolutional filter banks in the respective temporal convolutional filter banks; and For each sequencing cycle and based on the corresponding time feature map set, a base detection prediction of nucleotide type is generated for the corresponding signal captured in the sequencing image set of each cycle.

11. The artificial intelligence-based base detection method according to claim 10, wherein the artificial intelligence-based method further comprises convolving the corresponding per-cycle spatial feature map set by the corresponding compressed convolutional layer of the compressed network and generating a corresponding per-cycle compressed spatial feature map set for the corresponding sequencing cycle.

12. The artificial intelligence-based base detection method according to claim 11, wherein the trained coefficients of the compressed convolutional filter in the corresponding compressed convolutional layer vary between the compressed convolutional layers in the corresponding compressed convolutional layer.

13. The artificial intelligence-based base detection method according to any one of claims 10-12, wherein the artificial intelligence-based method further comprises convolving the corresponding overlapping groups of each group of time feature maps in each group of time feature maps by using the corresponding time convolution filter bank of the second time convolution layer, processing each group of time feature maps by group through the time convolution network component, and generating a corresponding additional group of time feature maps for each group of time feature maps for the corresponding overlapping groups of each group of time feature maps.

Images