Bus network for artificial-intelligence-based base caller

Abstract

To provide a portable and embedded type system and an artificial intelligence based method which arrange a deep Convolution Neural Network (CNN).SOLUTION: The spatial convolution network is configured to: process a window of per-cycle sequencing image sets; and generate respective per-cycle spatial feature map sets. Trained coefficients of spatial convolution filters in spatial convolution filter banks of respective sequences of spatial convolution filter banks vary between sequences of spatial convolution layers in respective sequences of spatial convolution layers. The per-cycle spatial feature map sets are processed on a groupwise basis, and respective per-group temporal feature map sets are generated. Trained coefficients of temporal convolution filters in respective temporal convolution filter banks vary between temporal convolution filter banks in respective temporal convolution filter banks.SELECTED DRAWING: Figure 25A

Description

[Technical Field]

[0001] The disclosed technology relates to artificial intelligence type computers and digital data processing systems, and corresponding data processing methods and products for emulating intelligence (i.e., knowledge-based systems, inference systems, and knowledge acquisition systems), including systems for reasoning under uncertainty (e.g., fuzzy logic systems), adaptive systems, machine learning systems, and artificial neural networks. Specifically, the disclosed technology relates to using deep neural networks, such as deep convolutional neural networks, to analyze data.

[0002] (Priority application) This PCT application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 979,411, entitled "DATA COMPRESSION FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLING," filed February 20, 2020 (Attorney Docket No. ILLM 1029-1 / IP-1964-PRV), and U.S. Patent Application No. 17 / 179,395, entitled "DATA COMPRESSION FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLING," filed February 18, 2021 (Attorney Docket No. ILLM1029-2 / IP-1964-US), which priority application is incorporated herein by reference for all purposes as if fully set forth herein.

[0003] This PCT application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 979,399, entitled "SQUEEZING LAYER FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLING," filed February 20, 2020 (Attorney Docket No. ILLM1030-1 / IP-1982-PRV), and U.S. Patent Application No. 17 / 180,480, entitled "SPLIT ARCHITECTURE FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLER," filed February 19, 2021 (Attorney Docket No. ILLM1030-2 / IP-1982-US), which priority applications are incorporated herein by reference for all purposes as if fully set forth herein.

[0004] This PCT application claims priority to and the benefit of U.S. patent application Ser. No. 17 / 180,513 (Attorney Docket No. ILLM1031-2 / IP-1965-US), entitled "BUS NETWORK FOR ARTIFICIAL INTELLIGENCE-BASED BASE CALLER," filed February 19, 2021. The priority application is incorporated herein by reference for all purposes as if fully set forth herein.

[0005] (built-in) The following documents are incorporated by reference as if fully set forth herein: U.S. Provisional Patent Application No. 62 / 979,384, entitled "ARTIFICIAL INTELLIGENCE-BASED BASE CALLING OF INDEX SEQUENCES," filed February 20, 2020 (Attorney Docket No. ILLM 1015-1 / IP-1857-PRV); U.S. Provisional Patent Application No. 62 / 979,414, entitled "ARTIFICIAL INTELLIGENCE-BASED MANY-TO-MANY BASE CALLING," filed February 20, 2020 (Attorney Docket No. ILLM 1016-1 / IP-1858-PRV); U.S. Provisional Patent Application No. 62 / 979,385, entitled "KNOWLEDGE DISTILLATION-BASED COMPRESSION OF ARTIFICIAL INTELLIGENCE-BASED BASE CALLER," filed February 20, 2020 (Attorney Docket No. ILLM 1017-1 / IP-1859-PRV); U.S. Provisional Patent Application No. 63 / 072,032, entitled "DETECTING AND FILTERING CLUSTERS BASED ON ARTIFICIAL INTELLIGENCE-PREDICTED BASE CALLS," filed August 28, 2020 (Attorney Docket No. ILLM 1018-1 / IP-1860-PRV); U.S. Provisional Patent Application No. 62 / 979,412, entitled "MULTI-CYCLE CLUSTER BASED REAL TIME ANALYSIS SYSTEM," filed February 20, 2020 (Attorney Docket No. ILLM 1020-1 / IP-1866-PRV); U.S. Nonprovisional Patent Application No. 16 / 825,987, entitled "TRAINING DATA GENERATION FOR ARTIFICIAL INTELLIGENCE-BASED SEQUENCING," filed March 20, 2020 (Attorney Docket No. ILLM 1008-16 / IP-1693-US); U.S. Nonprovisional Patent Application No. 16 / 825,991, entitled "ARTIFICIAL INTELLIGENCE-BASED GENERATION OF SEQUENCING METADATA," filed March 20, 2020 (Attorney Docket No. ILLM 1008-17 / IP-1741-US); U.S. Nonprovisional Patent Application No. 16 / 826,126, entitled "ARTIFICIAL INTELLIGENCE-BASED BASE CALLING," filed March 20, 2020 (Attorney Docket No. ILLM 1008-18 / IP-1744-US); U.S. Nonprovisional Patent Application No. 16 / 826,134, entitled "ARTIFICIAL INTELLIGENCE-BASED QUALITY SCORING," filed March 20, 2020 (Attorney Docket No. ILLM 1008-19 / IP-1747-US); and U.S. Non-Provisional Patent Application No. 16 / 826,168, entitled "ARTIFICIAL INTELLIGENCE-BASED SEQUENCING," filed March 21, 2020 (Attorney Docket No. ILLM 1008-20 / IP-1752-PRV-US). [Background technology]

[0006] The subject matter discussed in this section should not be assumed to be prior art merely as a result of its mention in this section. Similarly, it should not be assumed that the problems mentioned in this section, or problems associated with the subject matter provided as background, have been previously recognized in the prior art. The subject matter in this section merely represents different approaches, which themselves may also correspond to embodiments of the claimed technology.

[0007] Rapid improvements in computing power have enabled deep convolutional neural networks (CNNs) to achieve great success in many computer vision tasks in recent years, with significantly improved accuracy. During the inference phase, many applications require low-latency processing of a single image with strict power consumption requirements, which reduces the efficiency of graphics processing units (GPUs) and other general-purpose platforms. This creates opportunities for specific acceleration hardware, such as field programmable gate arrays (FPGAs), by customizing digital circuits to be particularly effective for inferencing deep learning algorithms. However, deploying CNNs in portable and embedded systems remains challenging due to large data volumes, intensive computations, diverse algorithm structures, and frequent memory accesses.

[0008] Because convolution provides most of the operations in CNNs, the convolution acceleration scheme significantly impacts the efficiency and performance of hardware CNN accelerators. Convolution involves multiply-and-accumulate (MAC) operations with four levels of loops that slide along kernels and feature maps. The first loop level calculates the MAC for pixels within a kernel window. The second loop level accumulates the sum of MAC products across various different input feature maps. After completing the first and second loop levels, the final output pixel is obtained by adding a bias. The third loop level slides the kernel window within the input feature map. The fourth loop level generates various different output feature maps.

[0009] FPGAs, particularly for accelerating inference tasks, have attracted increasing interest and become more widely used. This is due to their (1) high reconfigurability, (2) superiority over application-specific integrated circuits (ASICs) in terms of the development time required to keep up with the rapid evolution of CNNs, (3) good performance, and (4) superior energy efficiency compared to GPUs. The high performance and efficiency of FPGAs can be achieved by synthesizing circuits customized for specific computations and directly processing billions of operations with a customized memory system. For example, hundreds to thousands of digital signal processing (DSP) blocks in modern FPGAs support core convolution operations, such as multiply-and-accumulate operations with high parallelism. Dedicated data buffers between external on-chip memory and on-chip processing engines (PEs) can be designed to achieve prioritized data flow by configuring tens of megabytes of on-chip block random access memory (BRAM) on field-programmable gate array (FPGA) chips.

[0010] Efficient data flow and hardware architecture for CNN acceleration is desired to minimize data communication while maximizing resource utilization to achieve high performance. This creates an opportunity to design methodologies and frameworks to accelerate the inference process of various CNN algorithms on acceleration hardware and achieve high performance, high efficiency, and high flexibility.

[0011] In the drawings, like reference characters generally refer to like parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosed technology. In the following description, various embodiments of the disclosed technology are described with reference to the following drawings: [Brief explanation of the drawings]

[0012] [Figure 1A] 1 shows one embodiment of the disclosed compression logic generating a set of compressed spatial maps for a first iteration of base calling. [Figure 1B] 1 illustrates one embodiment of processing a compressed spatial map set by the disclosed temporal logic to generate a temporal map set. [Figure 1C] 1 illustrates one embodiment of processing a set of temporal maps with the disclosed output logic to generate base call classification data. [Figure 1D] 10 shows an example of a sequence of feature map volumes successively generated by a cascade of spatial convolutional layers in response to processing an image patch per cycle for a target sequencing cycle. [Figure 1E] Here is an example showing how a 1x1 convolution compresses a feature map. [Figure 1F] 1 shows that the compression ratio achieved by the disclosed compression logic is a function of the number of compression filters applied by the disclosed compression logic. [Figure 1G] 1 illustrates an exemplary softmax function. [Figure 1H] 10 shows an example of per-cluster, per-cycle probability quartiles generated by the disclosed technique. [Figure 2A] We show that during the second iteration of base calling, a spatial map and corresponding compressed spatial map are generated only for non-overlapping sequencing cycle 6. [Figure 2B]1 shows that the condensed spatial map set generated during the first iteration of base calling is used in conjunction with the condensed spatial map set generated during the second iteration of base calling to generate base calls for central sequencing cycle 4. [Figure 2C] The output layer shows processing the final temporal map set produced during the second iteration of base calling to generate base calls for central sequencing cycle 4. [Figure 3A] We show that during the third iteration of base calling, a spatial map and corresponding compressed spatial map are generated only for non-overlapping sequencing cycle 7. [Figure 3B] FIG. 10 shows that the condensed spatial map sets generated during the first and second iterations of base calling are used in conjunction with the condensed spatial map set generated during the third iteration of base calling to generate base calls for central sequencing cycle 5. [Figure 3C] The output layer shows processing the final temporal map set produced during the third iteration of base calling, generating base calls for central sequencing cycle 5. [Figure 4A] 14 shows the 14th iteration of base calling for base calling central sequencing cycle 16. [Figure 4B] Shown is the use of previously generated compressed spatial maps for sequencing cycles 1-29 to generate the final temporal map set for base calling central sequencing cycle 16. [Figure 4C] The output layer is shown processing the final temporal map set generated during the 14th iteration of base calling to generate base calls for central sequencing cycle 16. [Figure 5A] FIG. 10 illustrates one embodiment in which filtering logic is used to filter the condensed spatial map sets for each of sequencing cycles 1, 2, 3, 4, and 5 to generate respective filtered condensed spatial maps during the first iteration of base calling. [Figure 5B]The output layer shows processing the filtered final time map set produced during the first iteration of base calling to generate base calls for central sequencing cycle 3. [Figure 6A] FIG. 10 illustrates one embodiment in which filtering logic is used to filter the condensed spatial map sets for each of sequencing cycles 2, 3, 4, 5, and 6 to generate respective filtered condensed spatial maps during the second iteration of base calling. [Figure 6B] The output layer shows processing the filtered final time map set produced during the second iteration of base calling to generate base calls for central sequencing cycle 4. [Figure 7A] FIG. 10 shows one embodiment in which filtering logic is used to filter the condensed spatial map sets for each of sequencing cycles 3, 4, 5, 6, and 7 to generate respective filtered condensed spatial maps during the third iteration of base calling. [Figure 7B] The output layer shows processing the filtered final time map set produced during the third iteration of base calling to generate base calls for central sequencing cycle 5. [Figure 8A] FIG. 10 illustrates one embodiment in which compression logic processes the set of temporal feature maps generated during the first iteration of base calling to generate a respective set of compressed temporal feature maps. [Figure 8B] The output layer shows processing the final compressed time map set, produced during the first iteration of base calling, to generate base calls for central sequencing cycle 3. [Figure 9A] 1 shows that the condensed time map set generated during the first base calling iteration is used in conjunction with the condensed time map set generated during the second iteration of base calling to generate base calls for central sequencing cycle 4. [Figure 9B]The output layer shows processing the final compressed time map set, produced during the second iteration of base calling, to generate base calls for central sequencing cycle 4. [Figure 10A] 1 shows that the condensed time map sets generated during the first and second base calling iterations are used in conjunction with the condensed time map set generated during the third iteration of base calling to generate base calls for central sequencing cycle 5. [Figure 10B] The output layer shows processing the final compressed time map set, produced during the third iteration of base calling, to generate base calls for central sequencing cycle 5. [Figure 11A] FIG. 10 illustrates one embodiment in which compression logic processes a set of filtered temporal feature maps generated during a first iteration of base calling to generate a respective set of filtered compressed temporal feature maps. [Figure 11B] The output layer is shown processing the filtered final compressed time map set produced during the first iteration of base calling to generate base calls for central sequencing cycle 3. [Figure 12A] 1 shows that the filtered compressed time map set generated during the first base calling iteration is used in conjunction with the filtered compressed time map set generated during the second base calling iteration to generate base calls for central sequencing cycle 4. [Figure 12B] The output layer is shown processing the filtered final compressed time map set produced during the second iteration of base calling to generate base calls for central sequencing cycle 4. [Figure 13A] 1 shows that the filtered condensed time map sets generated during the first and second base calling iterations are used in conjunction with the filtered condensed time map set generated during the third base calling iteration to generate base calls for central sequencing cycle 5. [Figure 13B]The output layer shows processing the filtered final compressed time map set produced during the third iteration of base calling to generate base calls for central sequencing cycle 5. [Figure 14] 1 illustrates a first exemplary architecture of a neural network-based base caller as disclosed herein. [Figure 15] 1 illustrates a second exemplary architecture of a neural network-based base caller as disclosed herein. [Figure 16] 1 illustrates a third exemplary architecture of a neural network-based base caller as disclosed herein. [Figure 17] 10 illustrates a fourth exemplary architecture of a neural network-based base caller as disclosed herein. [Figure 18] 1 illustrates one embodiment of filter configuration logic that configures the number (or large number) of convolution filters in a compression layer depending on the number of channels in the input data. [Figure 19A] 1 shows one embodiment of a sequencing system. [Figure 19B] 1 illustrates one embodiment of a sequencing system, the sequencing system including a configurable processor. [Figure 19C] FIG. 1 is a simplified block diagram of a system for analysis of sensor data from a sequencing system, such as base call sensor output. [Figure 20A] FIG. 1 is a simplified diagram illustrating aspects of a base call operation, including the functionality of a runtime program executed by a host processor. [Figure 20B] FIG. 1 is a simplified diagram of a configurable processor configuration. [Figure 21] 1 illustrates another embodiment of the disclosed data flow logic making the compressed spatial map generated during a first base calling iteration available during a second base calling iteration from off-chip memory (e.g., off-chip DRAM, host RAM, host high bandwidth memory (HBM)). [Figure 22] 1 illustrates one embodiment of the disclosed data flow logic making a compressed spatial map generated during a first base calling iteration available during a second base calling iteration from on-chip memory (e.g., on-chip DRAM, on-chip SRAM, on-chip BRAM, DRAM attached to the processor via an interconnect). [Figure 23] 1 illustrates one implementation of the disclosed neural network-based base caller, the so-called split architecture. [Figure 24A] We show residual connections that reinject previous information downstream via feature map addition. [Figure 24B] 1 illustrates one embodiment of residual blocks and skip connections. [Figure 24C] We present a residual architecture for neural network-based base callers, where spatial convolutional layers are grouped into residual blocks with skip connections. [Figure 25A] 1 shows details of the disclosed bus network of the neural network-based bass caller described herein. [Figure 25B] 1 illustrates an exemplary operation of the disclosed bus network. [Figure 25C] 1 illustrates one embodiment of the dimensional compatibility logic of the disclosed bus network. [Figure 26] 1 illustrates another example of the disclosed bus network. [Figure 27] 1 illustrates yet another example of the disclosed bus network. [Figure 28] 1 illustrates one embodiment of scaling logic for the disclosed bus network. [Figure 29] 1 illustrates one implementation of skip connections between temporal convolutional layers of a temporal network. [Figure 30]FIG. 10 is a graph comparing the base calling performance of a neural network-based base caller configured with compression logic (the sqz2 base caller) to a neural network-based base caller without compression logic and to Illumina's non-neural network-based base caller Real-Time Analysis (RTA) software. [Figure 31] 1 illustrates the savings in RAM and DRAM usage resulting from the use of the disclosed compression logic. [Figure 32] FIG. 10 is a graph comparing base calling performance by a network-based base caller configured with a split-and-skip architecture (split_res) against the RTA base caller and against another version of the network-based base caller without the split-and-skip architecture (distilled). [Figure 33] A computer system that can be used to implement the disclosed techniques. DETAILED DESCRIPTION OF THE INVENTION

[0013] The following discussion is presented to enable any person skilled in the art to make and use the disclosed technology and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosed technology. Thus, the disclosed technology is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0014] Sequence images Base calling is the process of determining the nucleotide composition of a sequence. Base calling involves the analysis of image data, or sequence images, generated during a sequencing run (or sequencing reaction) performed by sequencing instruments such as Illumina's iSeq, HiSeqX, HiSeq3000, HiSeq4000, HiSeq2500, NovaSeq6000, NextSeq550, NextSeq1000, NextSeq2000, NextSeqDx, MiSeq, and MiSeqDx.

[0015] The following description outlines how the sequence of images is generated and what they depict, according to one embodiment.

[0016] Base calling decodes the intensity data encoded in the sequence image into a nucleotide sequence. In one embodiment, Illumina's sequencing platform employs cyclic reversible termination (CRT) chemistry for base calling. This process relies on a growing nascent strand complementary to the template strand bearing fluorescently labeled nucleotides, tracking the emission signal of each newly added nucleotide. Fluorescently labeled nucleotides have a 3' removable block that anchors the fluorophore signal of the nucleotide.

[0017] Sequencing is performed in iterative cycles, each of which involves three steps: (a) extension of the nascent strand by adding fluorescently labeled nucleotides, (b) excitation of the fluorophore using one or more lasers in the sequencing instrument's optical system and imaging through different filters in the optical system to generate a sequence image, and (c) cleavage of the fluorophore and removal of the 3' block in preparation for the next sequencing cycle. The incorporation and imaging cycle is repeated for up to a specified number of sequencing cycles, defining the read length. Using this approach, each cycle interrogates a new position along the template strand.

[0018] The tremendous power of Illumina sequencers comes from their ability to simultaneously run and sense millions or even billions of clusters (analytes) undergoing CRT reactions. Clusters vary in size and shape, but each cluster contains approximately 1,000 identical copies of a template strand. Clusters are grown from template strands by bridge amplification or exclusion amplification of the input library prior to the sequencing run. The purpose of amplification and cluster growth is to increase the intensity of the emitted signal, since imaging devices cannot reliably sense the fluorophore signal of a single strand. However, because the physical distance between strands within a cluster is small, imaging devices perceive the cluster of strands as a single spot.

[0019] Sequencing is performed in a flow cell (or biosensor), a small glass slide that holds the input strands. The flow cell is connected to an optical system that includes a microscope imager, an excitation laser, and a fluorescence filter. The flow cell contains multiple chambers called lanes. The lanes are physically separated from each other and may contain different tagged sequencing libraries, allowing them to be distinguished without cross-contamination of samples. In some embodiments, the flow cell comprises a patterned surface. "Patterned surface" refers to the arrangement of different regions within or on an exposed layer of a solid support.

[0020] An imager (e.g., a solid-state imager such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) sensor) takes snapshots at multiple locations along the lane, in a series of non-overlapping regions called tiles. For example, there may be 64 or 96 tiles per lane. A tile holds hundreds of thousands to millions of clusters.

[0021] The output of a sequencing run is a sequence image. A sequence image shows the intensity radiation of clusters and their surrounding background using a grid (or array) of pixelated units (e.g., pixels, superpixels, subpixels). The intensity radiation is stored as intensity values ​​of the pixelated units. A sequence image has dimensions w by h of the grid of pixelated units, where w (width) and h (height) are any number ranging from 1 to 100,000 (e.g., 115 x 115, 200 x 200, 1800 x 2000, 2200 x 25000, 2800 x 3600, 4000 x 400). In some embodiments, w and h are the same. In other embodiments, w and h are different. A sequence image shows the intensity radiation generated as a result of incorporating nucleotides into a nucleotide sequence during a sequencing run. The intensity radiation is from the associated clusters and their surrounding background.

[0022] Neural network-based base calling The following discussion focuses on the neural network-based base caller 100 described herein. First, the input to the neural network-based base caller 100, according to one embodiment, is described. Then, an example of the structure and configuration of the neural network-based base caller 100 is provided. Finally, the output of the neural network-based base caller 100, according to one embodiment, is described.

[0023] The data flow logic provides the sequence image to the neural network-based base caller 100 for base calling. The neural network-based base caller 100 accesses the sequence image patch by patch (or tile by tile). Each patch is a subgrid (or subarray) of pixelated units within the grid of pixelated units that forms the sequence image. A patch has dimensions of q by r of the subgrid of pixelated units, where q (width) and r (height) are any number ranging from 1 to 10,000 (e.g., 3x3, 5x5, 7x7, 10x10, 15x15, 25x25, 64x64, 78x78, 115x115). In some embodiments, q and r are the same. In other embodiments, q and r are different from each other. In some embodiments, patches extracted from a single sequence image are the same size. In other embodiments, patches are of different sizes. In some implementations, patches can have overlapping pixelated units (eg, on edges).

[0024] Sequencing generates m sequence images per sequencing cycle for the corresponding m image channels. That is, each sequence image has one or more image (or intensity) channels (similar to the red, green, and blue (RGB) channels of a color image). In one embodiment, each image channel corresponds to one of multiple filter wavelength bands. In another embodiment, each image channel corresponds to one of multiple imaging events in a sequencing cycle. In yet another embodiment, each image channel corresponds to a combination of illumination with a specific laser and imaging through a specific optical filter. Image patches are tiled (or accessed) from each of the m image channels for a particular sequencing cycle. In different embodiments, such as 4-, 2-, and 1-channel chemistries, m is 4 or 2. In other embodiments, m is greater than 1, 3, or 4.

[0025] For example, consider a sequencing run performed using two different imaging channels, i.e., a blue channel and a green channel. Then, in each sequencing cycle, the sequencing run generates a blue image and a green image. In this way, for a series of k sequencing cycles of the sequencing run, a sequence of k pairs of blue and green images is generated as output and stored as a sequence image. Therefore, a sequence of k pairs of blue and green image patches is generated for patch-level processing by the neural network-based base caller 100.

[0026] The input image data to the neural network-based base caller 100 for one base calling iteration (or one instance of a forward pass or single forward traversal) includes data for one sliding window that includes multiple sequencing cycles. The sliding window can include, for example, the current sequencing cycle, one or more preceding sequencing cycles, and one or more subsequent sequencing cycles.

[0027] In one embodiment, the input image data includes data from three sequencing cycles, where the data of the current (time t) sequencing cycle being base called is accompanied by (i) data from the adjacent / context / previous / preceding / earlier (time t-1) sequencing cycle to the left, and (ii) data from the adjacent / context / next / following / later (time t+1) sequencing cycle to the right.

[0028] In another embodiment, the input image data includes data from five sequencing cycles, and the data for the current (time t) sequencing cycle to be base called involves (i) data for the first left adjacent / context / previous / preceding / previous (time t-1) sequencing cycle, (ii) data for the second left adjacent / context / previous / preceding / previous (time t-2) sequencing cycle, (iii) data for the first right adjacent / context / next / subsequent / subsequent (time t+1) sequencing cycle, and (iv) data for the second right adjacent / context / next / subsequent / subsequent (time t+2) sequencing cycle.

[0029] In yet another embodiment, the input image data includes data for seven sequencing cycles, and the data for the current (time t) sequencing cycle to be base called involves: (i) data for the first left adjacent / context / previous / preceding / previous (time t-1) sequencing cycle; (ii) data for the second left adjacent / context / previous / preceding / previous (time t-2) sequencing cycle; (iii) data for the third left adjacent / context / previous / preceding / previous (time t-3) sequencing cycle; (iv) data for the first right adjacent / context / next / subsequent / subsequent (time t+1) sequencing cycle; (v) data for the second right adjacent / context / next / subsequent / subsequent (time t+2) sequencing cycle; and (vi) data for the third right adjacent / context / next / subsequent / subsequent (time t+3) sequencing cycle. In other embodiments, the input image data includes data from one sequencing cycle, hi still other embodiments, the input image data includes data from 10, 15, 20, 30, 58, 75, 92, 130, 168, 175, 209, 225, 230, 275, 318, 325, 330, 525, or 625 sequencing cycles.

[0030] According to one embodiment, the neural network-based base caller 100 processes image patches through its convolutional layers to generate alternative representations. The alternative representations are then used by an output layer (e.g., a softmax layer) to generate base calls for either the current sequencing cycle (time t) or each of the sequencing cycles (i.e., the current sequencing cycle (time t), the first and second preceding sequencing cycles (time t-1, time t-2), and the first and second subsequent sequencing cycles (time t+1, time t+2)). The resulting base calls form sequencing reads.

[0031] In one embodiment, the neural network-based base caller 100 outputs a base call for a single target cluster for a particular sequencing cycle. In another embodiment, the neural network-based base caller 100 outputs a base call for each target cluster within a plurality of target clusters at a particular sequencing cycle. In yet another embodiment, the neural network-based base caller 100 outputs a base call for each target cluster within a plurality of target clusters at each sequencing cycle within the plurality of sequencing cycles, thereby generating a base call sequence for each target cluster.

[0032] In one embodiment, the neural network-based base caller 100 is a multilayer perceptron (MLP). In another embodiment, the neural network-based base caller 100 is a feed-forward neural network. In yet another embodiment, the neural network-based base caller 100 is a fully connected neural network. In a further embodiment, the neural network-based base caller 100 is a fully convolutional neural network. In yet a further embodiment, the neural network-based base caller 100 is a semantic segmentation neural network. In yet another further embodiment, the neural network-based base caller 100 is a generative adversarial network (GAN).

[0033] In one embodiment, neural network-based base caller 100 is a convolutional neural network (CNN) having multiple convolutional layers. In another embodiment, neural network-based base caller 100 is a recurrent neural network (RNN), such as a long short-term memory network (LSTM), a bi-directional LSTM (Bi-LSTM), or a gated recurrent unit (GRU). In yet another embodiment, neural network-based base caller 100 includes both a CNN and an RNN.

[0034] In yet other embodiments, the neural network-based base caller 100 may use 1D convolution, 2D convolution, 3D convolution, 4D convolution, 5D convolution, dilated or expanded convolution, transposed convolution, depth-separable convolution, pointwise convolution, 1x1 convolution, group convolution, flattened convolution, spatial and cross-channel convolution, shuffled grouped convolution, spatially separable convolution, and deconvolution. The neural network-based base caller 100 may use one or more loss functions, such as logistic regression / logarithmic loss, multi-class cross-entropy / softmax loss, binary cross-entropy loss, mean squared error loss, L1 loss, L2 loss, smoothed L1 loss, and Huber loss. The neural network-based base caller 100 can use any parallel, efficient, and compression scheme, such as TFRecord, compression encoding (e.g., PNG), sharpening, parallel calls to map transforms, batching, prefetching, model parallelism, data parallelism, and synchronous / asynchronous stochastic gradient descent (SDG). The neural network-based base caller 100 can include nonlinear transformation functions, such as upsampling layers, downsampling layers, recurrent connections, gates and gated memory units (e.g., LSTM or GRU), residual blocks, residual connections, highway connections, skip connections, peephole connections, activation functions (e.g., nonlinear transformation functions (e.g., rectifying linear unit (ReLU), leaky ReLU, exponential liner unit (ELU), sigmoid, and hyperbolic tangent (tanh)), etc.), batch normalization layers, regularization layers, dropout, pooling layers (e.g., max or mean pooling), global mean pooling layers, and attention mechanisms.

[0035] The neural network-based base caller 100 trains using a backpropagation-based gradient update technique. Exemplary gradient descent techniques that the neural network-based base caller 100 can use to train include stochastic gradient descent, batch gradient descent, and mini-batch gradient descent. Some examples of gradient descent optimization algorithms that the neural network-based base caller 100 can use to train include Momentum, Nestorv accelerated gradient, Adagrad, Adadelta, RMSprop, Adam, AdaMax, Nadam, and AMSGrad.

[0036] In one embodiment, the neural network-based base caller 100 uses a dedicated architecture to separate the processing of data for different sequencing cycles. The motivation for using such a dedicated architecture is first explained. As described above, the neural network-based base caller 100 processes image patches for the current sequencing cycle, one or more previous sequencing cycles, and one or more subsequent sequencing cycles. Data for the additional sequencing cycles provides a unique context for each sequence. During training, the neural network-based base caller 100 learns a unique context for each sequence and base calls them. Furthermore, data for the pre- and post-sequencing cycles provide secondary contributions of pre-phasing and phasing signals to the current sequencing cycle.

[0037] However, images captured in different sequencing cycles and in different image channels are misaligned and have residual registration errors with each other. To account for this misalignment, the specialized architecture includes a spatial convolution layer that does not mix information between sequencing cycles, but only mixes information within the same sequencing cycle.

[0038] The spatial convolutional layer (or spatial logic) uses so-called "decoupled convolutions" that operate on separation by processing the data for each of multiple sequencing cycles independently through a "dedicated, unshared" sequence of convolutions. Decoupled convolutions convolve on the data and resulting feature maps only within a given sequencing cycle, i.e., the cycle, without convolving on the data and resulting feature maps of any other sequencing cycles.

[0039] For example, consider input image data including (i) a current image patch for the current (time t) sequencing cycle to be base-called, (ii) a previous image patch for the previous (time t-1) sequencing cycle, and (iii) a next image patch for the next (time t+1) sequencing cycle. The dedicated architecture then initiates three separate convolution pipelines: a current convolution pipeline, a previous convolution pipeline, and a next convolution pipeline. The current data processing pipeline receives the current image patch for the current (time t) sequencing cycle as input and processes it independently through multiple spatial convolution layers to generate a so-called "current spatial convolution representation" as the output of the final spatial convolution layer. The previous convolution pipeline receives the previous image patch for the previous (time t-1) sequencing cycle as input and processes it independently through multiple spatial convolution layers to generate a so-called "previous spatial convolution representation" as the output of the final spatial convolution layer. The next convolution pipeline receives the next data for the next (time t+1) sequencing cycle as input and processes it independently through multiple spatial convolution layers to produce the so-called “next spatial convolution representation” as the output of the final spatial convolution layer.

[0040] In some implementations, the current, previous, and next convolution pipelines run in parallel. In some implementations, the spatial convolution layer is part of a spatial convolution network (or sub-network) within a dedicated architecture.

[0041] The neural network-based base caller 100 further includes temporal convolutional layers (or temporal logic) that blend information between sequencing cycles, i.e., between cycles. The temporal convolutional layers receive their inputs from the spatial convolutional networks and operate on the spatially convolved representations produced by the final spatial convolutional layer for each data processing pipeline.

[0042] The inter-cycle operational freedom of the temporal convolutional layers arises from the fact that misalignment features present in the image data fed as input to the spatial convolutional network are purged from the spatial convolutional representation by the stack or cascade of separated convolutions performed by the sequence of spatial convolutional layers.

[0043] The temporal convolutional layer uses so-called "combinatorial convolution," which convolves group-wise on input channels with subsequent inputs on a sliding window basis. In one implementation, the subsequent inputs are subsequent outputs generated by previous spatial or temporal convolutional layers.

[0044] In some embodiments, the temporal convolutional layer is part of a temporal convolutional network (or sub-network) in a dedicated architecture. The temporal convolutional network receives its input from a spatial convolutional network. In one embodiment, the first temporal convolutional layer of the temporal convolutional network combines the spatial convolutional representations between sequencing cycles by group. In another embodiment, subsequent temporal convolutional layers of the temporal convolutional network combine subsequent outputs of previous temporal convolutional layers. The output of the final temporal convolutional layer is fed to an output layer, which generates an output. The output is used to base call one or more clusters in one or more sequencing cycles.

[0045] Additional details regarding the neural network-based base caller 100 can be found in U.S. Provisional Patent Application No. 62 / 821,766, entitled "ARTIFICIAL INTELLIGENCE-BASED SEQUENCING," filed March 21, 2019 (Attorney Docket No. ILLM1008-9 / IP-1752-PRV), which is incorporated herein by reference.

[0046] Compression Network As described above, the specialized architecture of the neural network-based base caller 100 processes a sliding window of image patches of a corresponding sequencing cycle. There is overlap between the sequencing cycles of subsequent sliding windows. This causes the neural network-based base caller 100 to redundantly process image patches of overlapping sequencing cycles. This results in wasted computational resources. For example, in one embodiment, each spatial convolutional layer of the neural network-based base caller 100 has approximately 100 million multiplication operations. Then, for a window of five sequencing cycles and a cascade (or sequence) of seven spatial convolutional layers, the spatial convolutional neural network performs approximately 620 million multiplication operations. Furthermore, the temporal convolutional neural network performs approximately 10 million multiplication operations.

[0047] Because image data for cycle N-1 in the current sliding window (or current iteration of base calling) is processed as cycle N in the previous sliding window (or previous iteration of base calling), an opportunity is provided to store intermediate results of processing performed in the current sliding window and intermediate results of processing performed in subsequent windows, thereby avoiding (or eliminating) redundant processing (or reprocessing) of input image data for sequencing cycles that overlap between subsequent sliding windows.

[0048] However, the intermediate results are several terabytes of data, requiring impractical amounts of storage. To overcome this technical problem, the disclosed technique proposes compressing the intermediate results when they are first generated by the neural network-based base caller 100, and then repurposing and utilizing the compressed intermediate results in subsequent sliding windows to avoid redundant computations, thereby not regenerating the intermediate results (or generating them only once). In some embodiments, the disclosed technique saves approximately 80% of convolutions in the spatial network of the neural network-based base caller 100. In one embodiment, the 80% savings is observed in spatial convolutions when compression logic and repurposing and utilizing compressed feature maps are used for input windows of five sequencing cycles (e.g., cycle N, cycle N+1, cycle N-1, cycle N+2, and cycle N-2) in subsequent sequencing cycles. In another embodiment, when compression logic and repurposing of compressed feature maps in subsequent sequencing cycles are used for an input window of 10 sequencing cycles (e.g., cycle N, cycle N+1, cycle N-1, cycle N+2, cycle N-2, cycle N+3, and cycle N-3), a 90% savings in spatial convolution is observed. That is, the larger the window size, the greater the savings from using compression logic and repurposing of compressed feature maps, and the larger the window size, the better the base calling performance due to the incorporation of greater context from additional adjacent cycles. The greater savings for larger windows improves overall performance for a given computational power.

[0049] The computational efficiency and compact computational footprint provided by the compression logic facilitates hardware implementation of the neural network-based basis caller 100 on resource-constrained processors such as central processing units (CPUs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), coarse-grained reconfigurable architectures (CGRAs), application-specific integrated circuits (ASICs), application-specific instruction-set processors (ASIPs), and digital signal processors (DSPs).

[0050] The computations saved by the compression logic allow for the incorporation of more convolutional operators into the neural network-based base caller 100. Examples include adding more convolutional filters to the spatial and temporal convolutional layers, increasing the size of the convolutional filters, and increasing the number of spatial and temporal convolutional layers. The additional convolutional operations improve the intensity pattern detection and overall base calling accuracy of the neural network-based base caller 100.

[0051] The computations saved by the compression logic also allow the input image data of the sliding window of interest to be expanded to include more sequencing cycles, which broadens the base call context by providing redundant image patches from additional adjacent sequencing cycles.

[0052] Furthermore, any loss of accuracy that may occur due to the use of compressed intermediate results as opposed to the original intermediate results is compensated for by the incorporation of additional convolution operators and the extension of the sliding window.

[0053] 1A shows one embodiment of the disclosed compression logic generating a set of compressed spatial maps for a first iteration of base calling. In the illustrated example, the first window of sequencing cycles includes sequencing cycles 1, 2, 3, 4, and 5. Respective image patches 102, 112, 122, 132, and 142 (or sets of analyte channels per cycle) for each of sequencing cycles 1, 2, 3, 4, and 5 are separately processed by spatial logic 104 (or a spatial network or spatial subnetwork or spatial convolutional neural network) to generate respective spatial maps 106, 116, 126, 136, and 146 (or intermediate results or sets of spatial output or sets of spatial feature maps) for each of sequencing cycles 1, 2, 3, 4, and 5. The spatial convolutional network 104 can use 1D, 2D, or 3D convolution.

[0054] The spatial logic 104 includes a sequence (or cascade) of spatial convolutional layers. Each spatial convolutional layer has a filter bank with multiple spatial convolutional filters that implement decoupled convolutions. Therefore, each spatial convolutional layer generates multiple spatial feature maps as output. The number of spatial feature maps generated by a given subject spatial convolutional layer is a function of the number of spatial convolutional filters configured in that subject spatial convolutional layer. For example, if a subject spatial convolutional layer has 14 spatial convolutional filters, the subject spatial convolutional layer generates 14 spatial feature maps. Collectively, the 14 spatial feature maps can be viewed as a volume (or tensor) of spatial feature maps with 14 channels (or depth dimension = 14).

[0055] Furthermore, the next spatial convolutional layer following the target spatial convolutional layer may also be composed of 14 spatial convolutional filters. In such a case, the next spatial convolutional layer processes the 14 spatial feature maps as input to generate the target spatial convolutional layer, which itself generates 14 new spatial feature maps as output. FIG. 1A shows five spatial feature map sets 106, 116, 126, 136, and 146 generated by the final spatial convolutional layer of spatial network 104 for sequencing cycles 1, 2, 3, 4, and 5, respectively. In the illustrated example, each of the five spatial feature map sets 106, 116, 126, 136, and 146 has 14 feature maps.

[0056] 1D shows a sequence of seven spatial feature map sets 196a, 196b, 196c, 196d, 196e, 196f, and 196g generated by a cascade of seven spatial convolutional layers of spatial network 104. The input patch data 194 per cycle for a given sequencing cycle i has a spatial dimensionality of 115×115 and a depth dimensionality of 2 (due to the two image channels in the original sequence images). In one implementation, each of the seven spatial convolutional layers uses a 3×3 convolution, which reduces the spatial dimensionality of the subsequent spatial feature map volume by two, e.g., from 10×10 to 8×8.

[0057] The first spatial feature map volume 196a has spatial dimensions of 113x113 (i.e., reduced from 115x115 by the 3x3 convolutions of the first spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the first spatial convolutional layer). The second spatial feature map volume 196b has spatial dimensions of 111x111 (i.e., reduced from 113x113 by the 3x3 convolutions of the second spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the second spatial convolutional layer). The third spatial feature map volume 196c has a spatial dimension of 109 x 109 (i.e., reduced from 111 x 111 by the 3 x 3 convolutions of the third spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the third spatial convolutional layer). The fourth spatial feature map volume 196d has a spatial dimension of 107 x 107 (i.e., reduced from 109 x 109 by the 3 x 3 convolutions of the fourth spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the fourth spatial convolutional layer). The fifth spatial feature map volume 196e has a spatial dimension of 10x10 (i.e., reduced from 10x10 by the 3x3 convolutions of the fifth spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the fifth spatial convolutional layer). The sixth spatial feature map volume 196f has a spatial dimension of 10x10 (i.e., reduced from 10x10 by the 3x3 convolutions of the sixth spatial convolutional layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the sixth spatial convolutional layer).The seventh spatial feature map volume 196g has a spatial dimension of 101×101 (i.e., reduced from 103×103 by the 3×3 convolutions of the seventh spatial convolution layer) and a depth dimension of 14 (i.e., 14 feature maps or 14 channels due to the 14 spatial convolution filters in the seventh spatial convolution layer).

[0058] Similar to the multi-cycle example shown in Figure 1A, for five sequencing cycles 1, 2, 3, 4, and 5 and five image patches 102, 112, 122, 132, and 142 per cycle, spatial logic 104 generates five separate sequences of seven spatial feature map volumes 196a, 196b, 196c, 196d, 196e, 196f, and 196g, respectively, with spatial maps 106, 116, 126, 136, and 146 of Figure 1A being equivalent to five separate instances of final spatial feature map volume 196g of Figure 1D.

[0059] The compression logic 108 (or compression network or compression sub-network or compression layer or squeeze layer) processes the output of the spatial logic 104 and generates a compressed representation of the output. In one embodiment, the compression network 108 comprises a compressed convolutional layer that reduces the depth dimensionality of the feature maps generated by the spatial network 104.

[0060] 1A, spatial maps 106, 116, 126, 136, and 146 have a depth dimensionality of 14 (i.e., 14 feature maps or 14 channels per spatial output). Compression network 108 reduces spatial maps 106, 116, 126, 136, and 146 to respective compressed spatial map sets 110, 120, 130, 140, and 150 for sequencing cycles 1, 2, 3, 4, and 5, respectively. Each of compressed spatial map sets 110, 120, 130, 140, and 150 has a depth dimensionality of 2 (i.e., two feature maps or two channels per compressed spatial output). In other embodiments, the compressed spatial map sets 110, 120, 130, 140, and 150 may have a depth dimensionality of three or four (i.e., three or four feature maps, or three or four channels, per compressed spatial output). In yet other embodiments, the compressed spatial map sets 110, 120, 130, 140, and 150 may have a depth dimensionality of one (i.e., one feature map, or one channel, per compressed spatial output). In one embodiment, the compression layer 108 does not include an activation function, such as ReLU. In other embodiments, the compression layer 108 may include an activation function. In other embodiments, the compression logic 108 may configure the corresponding compressed spatial map sets to each have more than four feature maps.

[0061] We now discuss how the compression logic 108 generates the compressed output.

[0062] In one implementation, the compression logic 108 uses 1×1 convolutions to reduce the number of feature maps (i.e., the depth dimension, or number of channels) while introducing nonlinearity. A 1×1 convolution has a kernel size of 1. A 1×1 convolution can convert volumetric depth to another squeezed or dilated representation without changing the spatial dimension. A 1×1 convolution operates like a fully connected linear layer across the input channels. This is useful for mapping from feature maps with many channels to a smaller number of feature maps. In FIG. 1E, a single 1×1 convolution is applied to an input tensor with two feature maps. The 1×1 convolution compresses a two-channel input to a single-channel output.

[0063] The number of compressed outputs (or compressed feature maps, or compressed spatial maps, or compressed temporal maps) generated by the compression layer 108 is a function of the number of 1×1 convolution filters (or compressed convolution filters, or compressed filters) configured within the compression layer 108. In FIG. 1F , the compression layer 108 includes two 1×1 convolution filters 198a and 198b. The first 1×1 convolution filter 198a processes the spatial feature volume 196g having 14 feature maps and generates the first feature map 199a while preserving the 101×101 spatial dimensionality. The second 1×1 convolution filter 198b also processes the spatial feature volume 196g having 14 feature maps and generates the second feature map 199b while preserving the 101×101 spatial dimensionality. Thus, compression layer 108 reduces spatial feature volume 196g with 14 feature maps to a compressed output with two spatial feature maps 199a and 199b (ie, compression ratio=7).

[0064] From a chronological perspective, sequencing cycle 5 is the central sequencing cycle (N), sequencing cycles 1 and 2 are left adjacent sequencing cycles (N-1, N-2), and sequencing cycles 4 and 5 are left adjacent sequencing cycles (N+1, N+2). Thus, a central compressed output 130 is generated for the central sequencing cycle (N), a left adjacent compressed output 120 is generated for the left adjacent sequencing cycle (N-1), a further left adjacent compressed output 110 is generated for the further left adjacent sequencing cycle (N-2), a right adjacent compressed output 140 is generated for the right adjacent sequencing cycle (N+1), and a further right adjacent compressed output 150 is generated for the further right adjacent sequencing cycle (N+2).

[0065] From a pipeline perspective, neural network-based base caller 100 executes five parallel and independent pipelines that process image patches 102, 112, 122, 132, and 142, respectively, via spatial logic 104 and compression logic 108 (e.g., as a multi-threaded or multi-clustered execution based on data parallelism). Thus, five compressed outputs 110, 120, 130, 140, and 150 are generated separately, simultaneously, and independently by neural network-based base caller 100.

[0066] In some implementations, the compression layer 108 can be considered the final spatial convolutional layer of the spatial network 104. In other implementations, the compression network 108 can be considered a separate network, either within or outside of the dedicated architecture of the neural network-based base caller 100.

[0067] FIG. 1B illustrates one implementation of processing compressed spatial map sets 110, 120, 130, 140, and 150 through temporal logic 160 (or a temporal network, or a temporal subnetwork, or a temporal convolutional neural network) of neural network-based base caller 100. Temporal logic 160 processes successive groups of compressed spatial map sets on a sliding window basis. For example, in FIG. 1B, temporal logic 160 processes a first group / window of compressed spatial map sets 110, 120, and 130 for sequencing cycles 1, 2, and 3, respectively, and generates as output temporal map 172 (or a set of temporal maps or a set of temporal feature maps or a set of temporal feature maps). Temporal logic 160 processes a second group / window of compressed spatial map sets 120, 130, and 140 for sequencing cycles 2, 3, and 4, respectively, and generates as output temporal map 174. Temporal logic 160 processes the third group / window of compressed spatial map sets 130, 140, and 150 for sequencing cycles 3, 4, and 5, respectively, and produces as output a temporal map 176. Temporal convolution network 160 can use 1D, 2D, or 3D convolution.

[0068] 1B represent three filter banks of the first temporal convolutional layer of the temporal network 160. The first filter bank applies a first set of temporal convolution filters to a first group of compressed spatial maps 110, 120, and 130 to generate a first set of temporal maps 172. The second filter bank applies a second set of temporal convolution filters to a second group of compressed spatial maps 120, 130, and 140 to generate a second set of temporal maps 174. The third filter bank applies a third set of temporal convolution filters to a third group of compressed spatial maps 130, 140, and 150 to generate a third set of temporal maps 176.

[0069] First set 172, second set 174, and third set 176 of temporal maps are processed as a group by temporal logic 160 to generate temporal map 182. The fourth instance of temporal logic 160 shown in FIG. 1B represents the second temporal convolutional layer of temporal network 160, which generates outputs for all sequencing cycles 1, 2, 3, 4, and 5, for which the image patch pairs per cycle were provided as input to neural network-based base caller 100 of FIG. 1A.

[0070] The temporal network 160 has a cascade of temporal convolutional layers (e.g., two, three, four, five, or more temporal convolutional layers arranged in sequence). The cascade of temporal convolutional layers processes data in a hierarchical manner with different levels of grouping. That is, at a given level, a sliding window approach processes the input at that level in a group-wise manner to generate an output, which is then processed in a group-wise manner at the next level in a sliding window fashion.

[0071] The temporal convolutional layer is constructed using temporal convolution filters that implement combinatorial convolutions, which blend information between feature maps across multiple sequencing cycles and combine data between subsequent sequencing cycles in a group / window of interest at the current level in the temporal network 160. For example, the first temporal convolutional layer combines the first group of compressed spatial maps 110, 120, and 130 for the first group of sequencing cycles 1, 2, and 3 to generate a first set of temporal maps 172; the first temporal convolutional layer further combines the second group of compressed spatial maps 120, 130, and 140 for the second group of sequencing cycles 2, 3, and 4 to generate a second set of temporal maps 174; and the first temporal convolutional layer further combines the third group of compressed spatial maps 130, 140, and 150 for the third group of sequencing cycles 3, 4, and 5 to generate a third set of temporal maps 176.

[0072] Combinatorial convolutions also combine data between successive groups of sequencing cycles in a group / window of interest at the current level in the temporal network 160. For example, the second temporal convolutional layer combines the first set 172, second set 174, and third set 176 of temporal maps into a final set 182 of temporal maps. At level 2, the first, second, and third groups / windows of sequencing cycles from level 1 are grouped into the first group / window of sequencing cycles 1, 2, 3, 4, and 5.

[0073] A combinatorial convolution is configured with as many kernels as the number of inputs being combined (i.e., the depth columns or fibers of a temporal convolution filter match the number of inputs in the group / window of interest at the current level). For example, if a temporal convolution layer combines three compressed spatial maps, the temporal convolution layer uses multiple temporal convolution filters, each with three kernels that perform element-wise multiplication and summation throughout the depth of the three compressed spatial maps.

[0074] The final set of temporal maps 182 is generated by the final (or last) temporal convolutional layer of the temporal network 160. FIG. 1C illustrates one embodiment of processing the final set of temporal maps 182 via the disclosed output logic 190 (or output layer, output network, or output sub-network) to generate base call classification data. In one embodiment, multiple clusters are simultaneously base called for one or more sequencing cycles. In the example shown in FIG. 1C, base calls 192 are generated for many clusters only for central sequencing cycle 3. In other embodiments, the disclosed technology allows the output logic 190 to generate base calls for not only the central sequencing cycle but also adjacent sequencing cycles (shown with optional dotted lines) for a given input window. For example, in one embodiment, the disclosed technology simultaneously generates base calls for cycle N, cycle N+1, cycle N-1, cycle N+2, cycle N-2, etc. for a given input window. That is, a single forward propagation / traverse / base calling iteration of the neural network-based base caller 102 generates base calls for multiple sequencing cycles in an input window of sequencing cycles, referred to herein as "many-to-many base calling."

[0075] Examples of output layer 190 include a softmax function, a log-softmax function, an ensemble output mean function, a multilayer perceptron uncertainty function, a Bayesian Gaussian distribution function, and a cluster strength function. In one implementation, output layer 190 generates per-cluster, per-cycle probability quartiles for each cluster and for each sequencing cycle.

[0076] The following discussion focuses on cluster-wise, cycle-wise probability quartiles, using the softmax function as an example. First, we describe the softmax function, then cluster-wise, cycle-wise probability quartiles.

[0077] The softmax function is a preferred function for multi-class classification. The softmax function calculates the probability of each target class across all possible target classes. The output of the softmax function ranges between zero and one, and the sum of all probabilities equals one. The softmax function calculates the exponent of a given input value and the sum of the exponent values ​​of all input values. The ratio of the exponent of the input value to the sum of the exponent values ​​is the output of the softmax function, referred to herein as "exponential normalization."

[0078] Formally, training a so-called softmax classifier is a regression onto class probabilities rather than a true classifier, since it returns not the classes but rather confidence predictions of the probabilities of each class. The softmax function takes some kind of value and converts them into probabilities that sum to 1. The softmax function compresses any real-valued n-dimensional vector into an n-dimensional vector of real values ​​in the range 0 to 1. Therefore, using a softmax function guarantees that the output is a valid, exponentially normalized probability mass function (non-negative and sums to 1).

[0079] Intuitively, the softmax function is a "soft" version of the max function. The term "soft" comes from the fact that the softmax function is continuous and differentiable. Instead of selecting a single maximum element, it decomposes the vector into parts of the whole, such that the maximum input element gets a proportionally larger value, while the others get a smaller proportion of the value. This property of outputting a probability distribution makes the softmax function suitable for probabilistic interpretation in classification tasks.

[0080] Consider z as a vector of inputs to a softmax layer. The softmax layer units are the number of nodes in the softmax layer, so the length of the z vector is the number of units in the softmax layer (if you have 10 output units, there will be 10 z elements).

[0081] For an n-dimensional vector Z = [z1, z2, ... zn], the softmax function uses exponential normalization (exp) to generate another n-dimensional vector p(Z) with normalized values ​​in the range [0, 1] whose sum equals 1.

[0082]

number

[0083] Figure 1G shows an example softmax function. The softmax function is

[0084]

number

[0085] A particular cluster-by-cluster, cycle-by-cycle probability quartile identifies the probability of A, C, T, and G being incorporated into a particular cluster in a particular sequencing cycle. When the output layer of the neural network-based base caller 100 uses a softmax function, the cluster-by-cluster and cycle-by-cycle probability quartiles are exponentially normalized classification scores that sum to 1. Figure 1H shows an example of cluster-by-cycle probability quartiles 123 generated by the softmax function for cluster 1 (121, shown in brown) and for sequencing cycles 1 through S (122). In other words, the first subset of sequencing cycles includes S sequencing cycles.

[0086] The unreliable cluster identifier 125 identifies unreliable clusters based on generating filter values ​​from cluster, cycle-wise probability quartiles. In this application, the cluster, cycle-wise probability quartiles are also referred to as base call classification scores or normalized base call classification scores or initial base call classification scores or normalized initial base call classification scores or initial base calls.

[0087] Filter calculator 127 determines filter values ​​for the probability quartiles for each cluster and each cycle based on the probabilities it identifies, thereby generating a sequence of filter values ​​for each cluster, which are stored as filter values ​​124.

[0088] The filter values ​​for the probability quartiles per cluster, per cycle are determined based on a calculation that includes one or more of the probabilities. In one embodiment, the calculation used by filter calculator 127 is subtraction. For example, in the embodiment shown in FIG. 1H, the filter values ​​for the probability quartiles per cluster, per cycle are determined by subtracting the second-highest probability (shown in blue) from the highest probability (shown in magenta).

[0089] In another embodiment, the calculation used by filter calculator 116 is division. For example, the filter value for each cluster, each cycle probability quartile is determined as the ratio of the highest one of the probabilities (shown in magenta) to the second highest one of the probabilities (shown in blue). In yet another embodiment, the calculation used by filter calculator 127 is addition. In yet a further embodiment, the calculation used by filter calculator 127 is multiplication.

[0090] In one embodiment, filter calculator 127 uses a filtering function to generate filter values ​​124. In one example, the filtering function is a Chastity filter that defines Chastity as the ratio of the brightest base intensity divided by the sum of the brightest and second brightest base intensities. In another example, the filtering function is at least one of a maximum log-likelihood function, a least squares error function, a mean signal-to-noise ratio (SNR), and a least absolute error function.

[0091] The unreliable cluster identifier 125 uses the filter value 124 to identify some clusters within the plurality of clusters as unreliable clusters 128. Data identifying the unreliable clusters 128 may be in a computer-readable format or medium. The unreliable clusters may be identified by an instrument ID, a run number on the instrument, a flow cell ID, a lane number, a tile number, an X coordinate of the cluster, a Y coordinate of the cluster, and a unique molecular identifier (UMI). The unreliable cluster identifier 125 identifies as unreliable clusters those clusters within the plurality of clusters whose sequence of filter values ​​includes "G" number of filter values ​​below a threshold "H". In one embodiment, "G" ranges from 1 to 5. In another embodiment, "H" ranges from 0.5 to 0.99. In one embodiment, the unreliable cluster 128 identifies pixels corresponding to (i.e., depicting) the intensity radiation of) the unreliable cluster. Such pixels are filtered out by filtering logic 502, as described later in this application.

[0092] An unreliable cluster is a low-quality cluster that emits an insignificant amount of the desired signal compared to the background signal. The signal-to-noise ratio of an unreliable cluster is substantially low, e.g., less than 1. In some embodiments, an unreliable cluster may not generate any desired signal at all. In other embodiments, an unreliable cluster may generate only a very small amount of signal compared to the background. In one embodiment, the signal is an optical signal, which is intended to include, for example, fluorescence, luminescence, scattering, or absorption signals. Signal level refers to the amount of detected energy or encoded information having a desired or predetermined characteristic. For example, optical signals can be quantified by one or more of intensity, wavelength, energy, frequency, power, brightness, etc. Other signals can be quantified according to characteristics such as voltage, current, electric field strength, magnetic field strength, frequency, power, temperature, etc. The absence of signal in an unreliable cluster is understood to be a signal level of zero or a signal level that is not significantly distinguishable from noise.

[0093] There are many potential reasons for the insufficient quality of signals in unreliable clusters. If there are polymerase chain reaction (PCR) errors in colony amplification, such that a significant proportion of the approximately 1,000 molecules in an unreliable cluster contain a different base at a specific position, signals for two bases may be observed, which is interpreted as a sign of insufficient quality and is referred to as a phase error. Phase errors occur when individual molecules in an unreliable cluster do not incorporate nucleotides in some cycles, lagging behind other molecules (e.g., due to incomplete removal of the 3' terminator, known as phasing), or when individual molecules incorporate two or more nucleotides in a single cycle (e.g., due to incorporation of nucleotides without an effective 3' block, known as prephasing). This results in a loss of synchronization in the readout of sequence copies. The proportion of sequences in an unreliable cluster affected by phasing and prephasing increases with increasing cycle number, which is the main reason why read quality tends to decrease at higher cycle numbers.

[0094] Unreliable clusters also result from fading, which is the exponential decay in the signal strength of unreliable clusters as a function of cycle number. As the sequencing run progresses, strands of unreliable clusters are excessively washed, exposed to laser radiation that creates reactive species, and subjected to harsh environmental conditions. All of this results in the gradual loss of fragments in unreliable clusters, reducing their signal strength.

[0095] Unreliable clusters can also result from poorly developed colonies, i.e., small cluster sizes of unreliable clusters, which can result in empty or partially filled wells on a patterned flow cell. That is, in some embodiments, unreliable clusters represent empty wells, polyclonal wells, and ambiguous wells on a patterned flow cell. Unreliable clusters can also result from overlapping colonies caused by non-exclusive amplification. Unreliable clusters can also result from insufficient or uneven illumination, for example, due to being located at the edge of the flow cell. Unreliable clusters can also result from impurities on the flow cell that obscure the emitted signal. Unreliable clusters also include polyclonal clusters, which occur when multiple clusters are deposited in the same well.

[0096] The first window of sequencing cycles includes sequencing cycles 1, 2, 3, 4, and 5, with the first iteration of base calling generating base call 192 for central sequencing cycle 3. The second window of sequencing cycles includes sequencing cycles 2, 3, 4, 5, and 6, with the second iteration of base calling generating base call 292 for central sequencing cycle 4. Thus, sequencing cycles 2, 3, 4, and 5 are overlapping sequencing cycles between the first and second windows, or between the second and third iterations of base calling.

[0097] The disclosed base calling systems and techniques store in memory (e.g., on-chip DRM, on-chip SRAM or BRAM, off-chip DRAM) the compressed spatial map sets 120, 130, 140, and 150 generated during the first iteration of base calling for sequencing cycles 2, 3, 4, and 5. During the second iteration of base calling, the disclosed base calling systems and techniques do not reprocess the overlapping input image patches 112, 122, 132, and 142, respectively, for cycles 2, 3, 4, and 5 through the spatial network 104. Instead, the disclosed base calling systems and techniques reuse the previously generated compressed spatial map sets 120, 130, 140, and 150 in place of the input image patches 112, 122, 132, and 142, respectively, during the second iteration of base calling.

[0098] The compression logic 108 is further configured to require the compressed spatial map sets 120, 130, 140, and 150 and the respective input image patches 112, 122, 132, and 142 to have the same number of cyclic feature maps / channels, thereby ensuring that the compressed spatial map sets 120, 130, 140, and 150 are lossless representations of the respective input image patches 112, 122, 132, and 142. That is, if the respective input image patches 112, 122, 132, and 142 each have two feature maps / channels, then the compression logic 108 configures the compressed spatial map sets 120, 130, 140, and 150 to also have two feature maps / channels. Similarly, if each of input image patches 112, 122, 132, and 142 has three feature maps / channels, then compression logic 108 configures compressed spatial map sets 120, 130, 140, and 150 to also have three feature maps / channels. Similarly, if each of input image patches 112, 122, 132, and 142 has four feature maps / channels, then compression logic 108 configures compressed spatial map sets 120, 130, 140, and 150 to also have four feature maps / channels.

[0099] 2A shows that during the second iteration of base calling, processing of input image data 222 by spatial logic 104 and compression logic 108 produces spatial map 226 and corresponding compressed spatial map 230 only for non-overlapping sequencing cycle 6. Thus, input image patches 112, 122, 132, and 142 (highlighted with gray shading in the legend) for overlapping cycles 2, 3, 4, and 5 are not processed again to avoid overlapping convolutions.

[0100] 2B shows that compressed spatial map sets 120, 130, 140, and 150 generated during the first iteration of base calling are used in conjunction with compressed spatial map set 230 generated during the second iteration of base calling to generate base call 292 for central sequencing cycle 4. In FIG. 2B, temporal map sets 174, 176, and 278 are generated by the first temporal convolutional layer of temporal network 160 in a manner similar to that discussed above with respect to FIG. 1B. Temporal map set 282 is generated by the second and final temporal convolutional layer of temporal network 160 in a manner similar to that discussed above with respect to FIG. 1B. FIG. 2C shows that output layer 190 processes final temporal map set 282 generated during the second iteration of base calling to generate base call 292 for central sequencing cycle 4.

[0101] The third window of sequencing cycles includes sequencing cycles 3, 4, 5, 6, and 7, and the third iteration of base calling generates base call 392 for central sequencing cycle 5. Thus, sequencing cycles 3, 4, 5, and 6 are overlapping sequencing cycles between the second and third windows, or between the second and third iterations of base calling.

[0102] The disclosed base calling systems and techniques store in memory (e.g., on-chip DRAM, on-chip SRAM or BRAM, off-chip DRAM) compressed spatial map sets 130, 140, and 150 generated during the first iteration of base calling for sequencing cycles 3, 4, and 5, respectively, and compressed spatial map set 230 generated during the second iteration of base calling for sequencing cycle 6. During the third iteration of base calling, the disclosed base calling systems and techniques do not reprocess the overlapping input image patches 122, 132, 142, and 222, respectively, for cycles 3, 4, 5, and 6 through spatial network 104. Instead, the disclosed base calling systems and techniques reuse the previously generated compressed spatial map sets 130, 140, 150, and 230 in place of the input image patches 122, 132, 142, and 222, respectively, during the third iteration of base calling.

[0103] 3A shows that during the third iteration of base calling, processing of input image data 322 by spatial logic 104 and compression logic 108 produces spatial map 326 and corresponding compressed spatial map 330 only for non-overlapping sequencing cycle 7. Thus, input image patches 122, 132, 142, and 222 (highlighted with gray shading in the legend) for overlapping cycles 3, 4, 5, and 6 are not processed again to avoid overlapping convolutions.

[0104] 3B shows that compressed spatial map sets 130, 140, 150, and 230 generated during the first and second iterations of base calling are used in conjunction with compressed spatial map set 330 generated during the third iteration of base calling to generate base call 392 for central sequencing cycle 5. In FIG. 3B, temporal map sets 176, 278, and 378 are generated by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to FIG. 1B. Temporal map set 382 is generated by the second and final temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to FIG. 1B. FIG. 3C shows that output layer 190 processes final temporal map set 382 generated during the third iteration of base calling to generate base call 392 for central sequencing cycle 5.

[0105] Once a compressed spatial map set is generated for a given sequencing cycle, that compressed spatial map set can be reused to base call any subsequent sequencing cycles. Figure 4A shows the 14th iteration of base calling for base calling central sequencing cycle 16. Figure 4B shows the use of the previously generated compressed spatial maps for sequencing cycles 1-29 to generate a final temporal map set 482 for base calling central sequencing cycle 16. Figure 4C shows the output layer 190 processing the final temporal map set 482 generated during the 14th iteration of base calling to generate base calls 492 for central sequencing cycle 16.

[0106] 5A illustrates one implementation in which filtering logic 502 is used to filter compressed spatial map sets 110, 120, 130, 140, and 150 for respective sequencing cycles 1, 2, 3, 4, and 5 to generate respective filtered compressed spatial maps 510, 520, 530, 540, and 550 (showing only reliable clusters) during the first iteration of base calling. As discussed above, unreliable cluster data 128 identifies those portions (e.g., pixels) of the spatial map and compressed spatial map that correspond to unreliable clusters. Such pixels can be identified, for example, based on the location coordinates of the unreliable clusters.

[0107] The filtering logic 502 uses the data 128 identifying unreliable clusters to filter out (or discard or remove) pixels that correspond to (i.e., depict the intensity radiation of) the unreliable clusters from the compressed spatial map sets 110, 120, 130, 140, and 150. In some implementations, this results in 75% of the pixels being discarded from the compressed spatial map sets, thereby preventing many unproductive convolutions.

[0108] In Figure 5A, for base calling central sequencing cycle 3, filtered temporal map sets 572, 574, and 576 (showing only high-confidence clusters) are generated from filtered compressed spatial maps 510, 520, 530, 540, and 550. Filtered temporal map sets 572, 574, and 576 (showing only high-confidence clusters) are generated by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to Figure 1B. Filtered temporal map set 582 (showing only high-confidence clusters) is generated by the second and final temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to Figure 1B. Figure 5B shows that output layer 190 processes final filtered temporal map set 582 generated during the first iteration of base calling to generate base call 592 for central sequencing cycle 3.

[0109] 6A illustrates one embodiment in which filtering logic 502 is used to filter compressed spatial map sets 120, 130, 140, 150, and 230 for respective sequencing cycles 2, 3, 4, 5, and 6 to generate respective filtered compressed spatial maps 520, 530, 540, 550, and 650 (showing only high-confidence clusters) during a second iteration of base calling. Filtering logic 502 uses data 128 identifying unconfidence clusters to filter out (or discard or remove) pixels corresponding to (i.e., depicting the intensity emissions of) the unconfidence clusters from compressed spatial map sets 120, 130, 140, 150, and 230.

[0110] In Figure 6A, filtered temporal map sets 574, 576, and 676 (showing only high-confidence clusters) are generated from filtered compressed spatial maps 520, 530, 540, 550, and 650 to base call central sequencing cycle 4. Filtered temporal map sets 574, 576, and 676 (showing only high-confidence clusters) are generated by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to Figure 1B. Filtered temporal map set 682 (showing only high-confidence clusters) is generated by the second and final temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to Figure 1B. Figure 6B shows output layer 190 processing final filtered temporal map set 682 generated during the second iteration of base calling to generate base call 692 for central sequencing cycle 4.

[0111] 7A illustrates one embodiment in which filtering logic 502 is used to filter compressed spatial map sets 130, 140, 150, 230, and 330 for respective sequencing cycles 3, 4, 5, 6, and 7 to generate respective filtered compressed spatial maps 530, 540, 550, 650, and 750 (showing only high-confidence clusters) during the third iteration of base calling. Filtering logic 502 uses data 128 identifying unconfidence clusters to filter out (or discard or remove) pixels corresponding to (i.e., depicting the intensity emissions of) the unconfidence clusters from compressed spatial map sets 130, 140, 150, 230, and 330.

[0112] In Figure 7A, filtered temporal map sets 576, 676, and 776 (showing only high-confidence clusters) are generated from filtered compressed spatial maps 530, 540, 550, 650, and 750 to base call central sequencing cycle 5. Filtered temporal map sets 576, 676, and 776 (showing only high-confidence clusters) are generated by the first temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to Figure 1B. Filtered temporal map set 782 (showing only high-confidence clusters) is generated by the second and final temporal convolutional layer of temporal network 160 in a manner similar to that described above with respect to Figure 1B. Figure 7B shows output layer 190 processing final filtered temporal map set 782 generated during the third iteration of base calling to generate base call 792 for central sequencing cycle 5.

[0113] In other implementations, the compression logic 108 may configure corresponding sets of compressed temporal maps each having more than four feature maps.

[0114] The compression logic 108 described above with respect to spatial feature maps applies equally to compressing the temporal feature maps generated by the temporal logic 160. Reusing a compressed spatial feature map once generated in a subsequent sequencing cycle also applies equally to reusing a compressed temporal feature map once generated in a subsequent sequencing cycle.

[0115] In some implementations, reusing compressed temporal feature maps results in over 100 times greater efficiency and computational savings than reusing compressed spatial feature maps, because the compressed temporal feature maps are generated from the compressed spatial feature maps at a later stage in the processing pipeline. Repurposing intermediate results from a further processing engine (i.e., the temporal network 160) increases the number of preceding processing steps that can be skipped. That is, reusing compressed spatial feature maps eliminates redundant processing of the original image data via the spatial network 104, but may involve redundant processing of the compressed spatial feature maps via the temporal network 160. In contrast, reusing compressed temporal feature maps eliminates both redundant processing of the original image data via the spatial network 104 and redundant processing of the compressed spatial feature maps via the temporal network 160.

[0116] FIG. 8A illustrates one implementation in which the compression logic 108 processes the sets of temporal feature maps 172, 174, and 176 generated during the first iteration of base calling to generate respective sets of compressed temporal feature maps 802, 804, and 806. The compressed temporal feature map sets 802, 804, and 806 are generated by the compression logic 108 in a manner similar to that described above with respect to FIGS. 1E and 1F. That is, for example, if the first temporal feature map set 172 has, for example, 21 feature maps (or a channel or depth of 21), the compression logic 108 can configure the corresponding compressed temporal feature map set 802 to have one, two, three, or four feature maps. The compressed temporal feature map sets 802, 804, and 806 are processed by the second temporal convolutional layer of the temporal network 160 in a manner similar to that described above with respect to FIG. 1B to generate the final compressed temporal feature map set 814. FIG. 8B shows that output layer 190 processes the final compressed time feature map set 814 produced during the first iteration of base calling to generate base calls 892 for central sequencing cycle 3.

[0117] Figure 9A shows one embodiment of reusing the compressed time maps generated in the first base calling iteration in the second base calling iteration, i.e., the first and second sets of compressed time maps 804 and 806 generated in Figure 8A for the first base calling iteration are now repurposed in the second base calling iteration shown in Figures 9A and 9B.

[0118] Note that the first and second sets of compressed temporal maps 804 and 806 were generated in Figure 8A from the first and second sets of temporal maps 172 and 174. Furthermore, the first and second sets of temporal maps 172 and 174 were generated in Figure 1B from compressed spatial maps 110, 120, 130, and 140, which were generated from corresponding spatial maps 106, 116, 126, and 136, respectively, in Figure 1A, which were generated from corresponding image patches 102, 112, 122, and 132, respectively, in Figure 1A.

[0119] 1B, 2B, and 3B, which redundantly generate overlapping time maps 172, 174, and 176, in FIG. 9A, overlapping time maps 174 and 176 (shown in FIG. 9A with dotted lines and lighter text) are not redundantly generated from FIG. 8A (first base call iteration) to FIG. 9A (second base call iteration). This occurs because compression logic 108 is incorporated into temporal network 160 to generate first and second sets 804 and 806 of compressed time maps in the first base call iteration, which replace overlapping time maps 174 and 176 in the second base call iteration. The compressed time maps can be stored in memory (e.g., on-chip DRAM, on-chip SRAM, or BRAM, off-chip DRAM).

[0120] Figure 9A also shows that the compression logic 108 processes the non-overlapping temporal maps 278 (i.e., no overlap between the first and second base call iterations) to generate a compression logic map 906. The compressed temporal map sets 804, 806, and 906 are processed by a second temporal convolutional layer of the temporal network 160 in a manner similar to that described above with respect to Figure 1B to generate a final compressed temporal feature map set 914. Figure 9B shows that the output layer 190 processes the final compressed temporal feature map set 914 generated during the second iteration of base calling to generate a base call 992 for central sequencing cycle 4.

[0121] 1B, 2B, and 3B, which redundantly generate overlapping time maps 174, 176, and 278, in FIG. 10A, overlapping time maps 176 and 278 (shown in FIG. 10A with dotted lines and lighter text) are not redundantly generated from FIG. 9A (second base call iteration) to FIG. 10A (third base call iteration). This occurs because compression logic 108 is incorporated into temporal network 160 to generate first and second sets of compressed time maps 806 and 906 in the first and second base call iterations, which replace overlapping time maps 176 and 278 in the third base call iteration. The compressed time maps can be stored in memory (e.g., on-chip DRAM, on-chip SRAM, or BRAM, off-chip DRAM).

[0122] Figure 10A also shows that the compression logic 108 processes the non-overlapping temporal maps 378 (i.e., no overlap between the second and third base call iterations) to generate the compression logic map 1006. The compressed temporal map sets 806, 906, and 1006 are processed by the second temporal convolutional layer of the temporal network 160 in a manner similar to that described above with respect to Figure 1B to generate the final compressed temporal feature map set 1014. Figure 10B shows that the output layer 190 processes the final compressed temporal feature map set 1014 generated during the third iteration of base calling to generate the base call 1092 for central sequencing cycle 5.

[0123] 11A illustrates one implementation in which the compression logic 108 processes the sets of filtered temporal feature maps 572, 574, and 576 generated during the first iteration of base calling to generate respective sets of filtered, compressed temporal feature maps 1102, 1104, and 1306. The filtered, compressed temporal feature map sets 1102, 1104, and 1106 (showing only the high-confidence clusters) are generated by the compression logic 108 in a manner similar to that described above with respect to FIGS. 1E and 1F. That is, for example, if the first filtered temporal feature map set 572 has, for example, 21 feature maps (or a channel or depth of 21), the compression logic 108 can configure the corresponding filtered, compressed temporal feature map set 1102 to have one, two, three, or four feature maps. The sets of filtered compressed temporal feature maps 1102, 1104, and 1106 are processed by a second filtered temporal convolutional layer of the filtered temporal network 160 in a manner similar to that described above with respect to Figure 1B to generate a final filtered compressed temporal feature map set 1114. Figure 8B shows that the output layer 190 processes the final filtered compressed temporal feature map set 1114 produced during the first iteration of base calling to generate a base call 1192 for central sequencing cycle 3.

[0124] In other implementations, the compression logic 108 may configure corresponding compressed feature map sets each having more than four feature maps.

[0125] Figure 12A shows one embodiment of reusing the filtered compressed time maps generated in the first base calling iteration in the second base calling iteration, i.e., the first and second sets of filtered compressed time maps 1104 and 1106 generated in Figure 11A for the first base calling iteration are now repurposed in the second base calling iteration shown in Figure 12A and Figure 2B.

[0126] Note that the first and second sets of filtered compressed time maps 1104 and 1106 were generated in FIG. 11A from the first and second sets of filtered time maps 572 and 574. Furthermore, first and second sets of filtered temporal maps 572 and 574 are generated from filtered compressed spatial maps 510, 520, 530 and 540 in FIG. 5A, which are generated from corresponding compressed spatial maps 110, 120, 130 and 140, respectively, in FIG. 5A, which are generated from corresponding spatial maps 106, 116, 126 and 136, respectively, in FIG. 1A, which are generated from corresponding image patches 102, 112, 122 and 132, respectively, in FIG. 1A.

[0127] 5A, 6A, and 7A, which redundantly generate filtered overlapping time maps 572, 574, and 576, in FIG. 12A, filtered overlapping time maps 574 and 576 (shown in FIG. 12A with dotted lines and lighter text) are not redundantly generated from FIG. 11A (first base call iteration) to FIG. 12A (second base call iteration). This occurs because compression logic 108 is incorporated into filtered time network 160 to generate first and second sets of filtered condensed time maps 1104 and 1106 in the first base call iteration, which replace filtered overlapping time maps 574 and 576 in the second base call iteration. The filtered condensed time maps can be stored in memory (e.g., on-chip DRAM, on-chip SRAM, or BRAM, off-chip DRAM).

[0128] Figure 12A also shows that the compression logic 108 processes the filtered non-overlapping temporal map 676 (i.e., no overlap between the first and second base call iterations) to generate a filtered compression logic map 1206 (showing only high-confidence clusters). The set of filtered compressed temporal maps 1104, 1106, and 1206 is processed by a second filtered temporal convolutional layer of the temporal network 160 in a manner similar to that described above with respect to Figure 1B to generate a filtered final compressed temporal feature map set 1214. Figure 12B shows that the output layer 190 processes the filtered final compressed temporal feature map set 1214 generated during the second iteration of base calling to generate a base call 1292 for central sequencing cycle 4.

[0129] Unlike Figures 5A, 6A, and 7A, which redundantly generate filtered overlapping time maps 574, 576, and 676, in Figure 13A, filtered overlapping time maps 576 and 676 (shown in Figure 13A with dotted lines and lighter text) are not redundantly generated from Figure 12A (second base call iteration) to Figure 132A (third base call iteration). This occurs because compression logic 108 is incorporated into filtered time network 160 to generate first and second sets of filtered condensed time maps 1106 and 1206 at the first and second base call iterations, which replace filtered overlapping time maps 576 and 676 at the third base call iteration. The filtered condensed time maps can be stored in memory (e.g., on-chip DRAM, on-chip SRAM, or BRAM, off-chip DRAM).

[0130] Figure 13A also shows that the compression logic 108 processes the filtered non-overlapping temporal map 776 (i.e., no overlap between the second and third base call iterations) to generate a filtered compression logic map 1306 (showing only high-confidence clusters). The set of filtered compressed temporal maps 1106, 1206, and 1306 is processed by a second filtered temporal convolutional layer of the temporal network 160 in a manner similar to that described above with respect to Figure 1B to generate a filtered final compressed temporal feature map set 1314. Figure 13B shows that the output layer 190 processes the filtered final compressed temporal feature map set 1314 generated during the third iteration of base calling to generate a base call 1392 for central sequencing cycle 5.

[0131] 14 shows a first exemplary architecture of a neural network-based base caller 100. In the illustrated implementation, the neural network-based base caller 100 includes a spatial network 104, a compression network 108, and a temporal network 160. The spatial network 104 includes seven spatial convolutional layers. The compression network 108 includes a compression layer. The temporal network 160 includes two temporal convolutional layers.

[0132] Each of the seven spatial convolutional layers may have the same number of convolutional filters or may have a different number of convolutional filters. The first spatial convolutional layer may have a number of filters S1, where S1 may be, for example, 7, 14, 21, 64, 128, or 254. The second spatial convolutional layer may have a number of filters S2, where S2 may be, for example, 7, 14, 21, 64, 128, or 254. The third spatial convolutional layer may have a number of filters S3, where S3 may be, for example, 7, 14, 21, 64, 128, or 254. The fourth spatial convolutional layer may have a number of filters S4, where S4 may be, for example, 7, 14, 21, 64, 128, or 254. The fifth spatial convolutional layer may have a number of filters S5, where S5 may be, for example, 7, 14, 21, 64, 128, or 254. The sixth spatial convolutional layer may have a number of filters S6, where S6 may be, for example, 7, 14, 21, 64, 128, or 254. The seventh spatial convolutional layer may have a number of filters S7, where S7 may be, for example, 7, 14, 21, 64, 128, or 254.

[0133] The compression layer may have a number C1 of filters, where C1 may be, for example, 1, 2, 3, 4, or more.

[0134] Each of the two temporal convolutional layers can have the same number of convolutional filters or different numbers of convolutional filters. The first temporal convolutional layer can have number T1 of filters, where T1 can be, for example, 7, 14, 21, 64, 128, or 254. The second temporal convolutional layer can have number T2 of filters, where T2 can be, for example, 7, 14, 21, 64, 128, or 254. Figure 14 also shows feature maps 1412 generated by each of the layers of the neural network-based base caller 100.

[0135] FIG. 15 illustrates a second exemplary architecture of the neural network-based base caller 100. FIG. 15 illustrates filtering logic 502 as part of the neural network-based base caller 100. In other implementations, filtering logic 502 is not part of the neural network-based base caller 100. The compressed feature map C1 has a spatial dimensionality of P1 × P2. The filtering logic 502 filters out pixels in the compressed feature map C1 that correspond to unreliable clusters to generate a filtered compressed feature map F1 having a spatial dimensionality of P3 × P4. The filtered compressed feature map F1 represents only reliable clusters. In one implementation, the filtering logic 502 discards 75% of the pixels in the compressed feature map C1, so that P3 is 25% of P1 and P4 is 25% of P2. FIG. 15 also illustrates feature maps 1512 generated by each layer of the neural network-based base caller 100.

[0136] Figure 16 shows a third exemplary architecture of the neural network-based base caller 100. Figure 16 shows that a compression network 108 is used to compress the output of the spatial network 104 and the output of the temporal network 160. Figure 16 also shows the feature maps 1612 produced by each of the layers of the neural network-based base caller 100.

[0137] Figure 17 illustrates a fourth exemplary architecture of the neural network-based base caller 100. Figure 17 shows that filtering logic 502 is applied to the compressed output of the spatial network 104 to produce a compressed and filtered temporal output from the temporal network 160. Figure 17 also illustrates the feature maps 1712 produced by each of the layers of the neural network-based base caller 100.

[0138] 18 illustrates one embodiment of filter configuration logic 1804, which configures the number (or large number) of convolution filters in compression layer 108 according to the number of channels in the input data. This allows the compressed feature map to be a lossless representation of the input data. In some embodiments, the input data can be overwritten in memory with the corresponding compressed representation for reuse in subsequent sequencing cycles.

[0139] In one embodiment, for input data that includes only one channel 1812 for each input per cycle (e.g., only one image channel), the filter configuration logic 1804 configures the compression layer 108 to have only one convolution filter 1816 that generates only one compressed feature map 1818 per sequencing cycle. In another embodiment, for input data that includes two channels 1822 for each input per cycle (e.g., two image channels, such as blue and green image channels in a sequence image corresponding to blue and green lasers), the filter configuration logic 1804 configures the compression layer 108 with two convolution filters 1826 that generate two compressed feature maps 1828 per sequencing cycle. In yet another embodiment, for input data including only three channels 1832 for each input per cycle (e.g., only three image channels), the filter configuration logic 1804 configures the compression layer 108 to have three convolution filters 1836 that generate three compressed feature maps 1838 for each sequencing cycle. In yet a further embodiment, for input data including four channels 1842 for each input per cycle (e.g., four image channels, such as the A, C, T, and G channels in a sequence image corresponding to the nucleotides A, C, T, and G), the filter configuration logic 1804 configures the compression layer 108 with four convolution filters 1846 that generate four compressed feature maps 1848 for each sequencing cycle. In other embodiments, the compression logic 108 can be configured to have corresponding compressed feature map sets each having more than four feature maps, and therefore select more than four filters for the compression layer 108.

[0140] 19A and 19B show one embodiment of a sequencing system 1900A. The sequencing system 1900A includes a configurable processor 1946. The configurable processor 1946 implements the base calling techniques disclosed herein. The sequencing system is also referred to as a "sequencer."

[0141] The sequencing system 1900A can obtain any information or data related to at least one of biological or chemical substances. In some embodiments, the sequencing system 1900A is a workstation, which can be similar to a benchtop device or desktop computer. For example, most (or all) of the systems and components for performing the desired reactions can be within a common housing 1902.

[0142] In certain embodiments, the sequencing system 1900A is a nucleic acid sequencing system configured for various applications, including, but not limited to, de novo sequencing, whole genome or targeted genomic region resequencing, and metagenomics. The sequencer may also be used for DNA or RNA analysis. In some embodiments, the sequencing system 1900A may also be configured to generate reaction sites within a biosensor. For example, the sequencing system 1900A may be configured to receive a sample and generate surface-bound clusters of clonovirus-amplified nucleic acids from the sample. Each cluster may constitute or be part of a reaction site within a biosensor.

[0143] The exemplary sequencing system 1900A may include a system receptacle or interface 1910 configured to interact with a biosensor 1912 to effect a desired reaction within the biosensor 1912. In the following description with respect to FIG. 19A , the biosensor 1912 is loaded into the system receptacle 1910. However, it is understood that a cartridge containing the biosensor 1912 may be inserted into the system receptacle 1910, and that in some conditions the cartridge may be temporarily or permanently removed. As discussed above, the cartridge may include, among other things, fluid control and fluid storage components.

[0144] In certain embodiments, sequencing system 1900A is configured to perform multiple parallel reactions within biosensor 1912. Biosensor 1912 includes one or more reaction sites where desired reactions can occur. The reaction sites may be immobilized, for example, on a solid surface of the biosensor or on beads (or other movable substrates) located within corresponding reaction chambers of the biosensor. The reaction sites may include, for example, clusters of clonally amplified nucleic acids. Biosensor 1912 may include a solid-state imaging device (e.g., a CCD or CMOS imager) and a flow cell attached thereto. The flow cell may include one or more flow paths that receive solutions from sequencing system 1900A and direct the solutions toward the reaction sites. Optionally, biosensor 1912 may be configured to engage a thermal element for transferring thermal energy into and out of the flow paths.

[0145] Sequencing system 1900A may include various components, assemblies, and systems (or subsystems) that interact with each other to perform a predetermined method or assay protocol for biological or chemical analysis. For example, sequencing system 1900A includes a system controller 1906, which may be in communication with the various components, assemblies, and subsystems of sequencing system 1900A, as well as with biosensor 1912. For example, in addition to system receptacle 1910, sequencing system 1900A may also include a fluid control system 1908 for controlling fluid flow throughout the fluidic network of sequencing system 1900A and biosensor 1912, a fluid reservoir system 1914 configured to hold any fluids (e.g., gases or liquids) that may be used by the bioassay system, a temperature control system 1904 that may regulate the temperature of fluids in the fluidic network, fluid reservoir system 1914, and / or biosensor 1912, and an illumination system 1916 configured to illuminate biosensor 1912. As mentioned above, when a cartridge having a biosensor 1912 is loaded into the system container 1910, the cartridge may also include fluid control and fluid storage components.

[0146] The sequencing system 1900A may also include a user interface 1918 for interacting with a user. For example, the user interface 1918 may include a display 1920 for displaying or requesting information from the user and a user input device 1922 for receiving user input. In some embodiments, the display 1920 and the user input device 1922 are the same device. For example, the user interface 1918 may include a touch-sensitive display configured to detect the presence of an individual touch and identify the location of the touch on the display. However, other user input devices 1922, such as a mouse, touchpad, keyboard, keypad, handheld scanner, voice recognition system, motion recognition system, etc., may also be used. As described in more detail below, the sequencing system 1900A may communicate with various components, including a biosensor 1912 (e.g., in the form of a cartridge), to perform desired reactions. The sequencing system 1900A may also be configured to analyze data obtained from the biosensor to provide desired information to the user.

[0147] The system controller 1906 comprises a microcontroller, a reduced instruction set computer (RISC), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a coarse-grained reconfigurable architecture (CGRA), a logic circuit, and any other circuit or processor capable of performing the functions described herein. The above examples are merely illustrative and thus are not intended to limit the definition and / or meaning of the term system controller. In an exemplary implementation, the system controller 1906 executes a set of instructions stored in one or more storage elements, memories, or modules to at least one of acquire and analyze detection data. The detection data can include multiple sequences of pixel signals, thereby allowing sequences of pixel signals from each of millions of sensors (or pixels) to be detected over many base call cycles. The storage elements may be in the form of information sources or physical memory elements within the sequencing system 1900A.

[0148] The instruction set may include various commands that instruct the sequencing system 1900A or biosensor 1912 to perform specific operations, such as the methods and processes of various embodiments described herein. The instruction set may be in the form of a software program, which may form part of a tangible, non-transitory computer-readable medium or media. As used herein, the terms "software" and "firmware" are used interchangeably and include any computer program stored in memory executed by a computer, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are exemplary only and thus not limiting of the types of memory that may be used to store a computer program.

[0149] The software may be in various forms, such as system software or application software. Furthermore, the software may be in the form of a collection of separate programs, or a program module or portion of a program module within a larger program. The software may also include modular programming in the form of object-oriented programming. After acquiring the detection data, the detection data may be processed automatically by sequencing system 1900A in response to user input, or in response to a request made by another processing machine (e.g., a remote request via a communications link). In the illustrated embodiment, system controller 1906 includes analysis module 1944. In other embodiments, system controller 1906 does not include analysis module 1944, but instead has access to analysis module 1944 (e.g., analysis module 1944 may be separately hosted on the cloud).

[0150] The system controller 1906 may be connected to the biosensor 1912 and other components of the sequencing system 1900A via a communication link. The system controller 1906 may also be communicatively connected to an off-site system or server. The communication link may be a wire, a cord, or wireless. The system controller 1906 may receive user input or commands from a user interface 1918 and a user input device 1922.

[0151] The fluid control system 1908 includes a fluid network and is configured to direct the flow of one or more fluids through the fluid network. The fluid network may be in fluid communication with a biosensor 1912 and a fluid reservoir system 1914. For example, fluid may be selected from the fluid reservoir system 1914 and directed to the biosensor 1912 in a controlled manner, or fluid may be drawn from the biosensor 1912 and directed, for example, to a waste reservoir within the fluid reservoir system 1914. Although not shown, the fluid control system 1908 may include a flow sensor that detects the flow rate or pressure of the fluid within the fluid network. The sensor may be in communication with the system controller 1906.

[0152] The temperature control system 1904 is configured to regulate the temperature of fluid in different regions of the fluid network, the fluid reservoir system 1914, and / or the biosensor 1912. For example, the temperature control system 1904 may include a thermal circulator that interacts with the biosensor 1912 and controls the temperature of the fluid flowing along a reaction site within the biosensor 1912. The temperature control system 1904 may also regulate the temperature of solid elements or components of the sequencing system 1900A or the biosensor 1912. Although not shown, the temperature control system 1904 may include sensors for detecting the temperature of the fluid or other components. The sensors may be in communication with the system controller 1906.

[0153] The fluid storage system 1914 is in fluid communication with the biosensor 1912 and may store various reaction components or reactants used to carry out a desired reaction. The fluid storage system 1914 may also store fluids for washing or cleaning the fluidic network and the biosensor 1912 and for diluting reactants. For example, the fluid storage system 1914 may include various reservoirs for storing samples, reagents, enzymes, other biomolecules, buffers, aqueous and non-polar solutions, etc. Additionally, the fluid storage system 1914 may also include a waste reservoir for receiving waste from the biosensor 1912. In embodiments including a cartridge, the cartridge may include one or more of a fluid storage system, a fluid control system, or a temperature control system. Accordingly, one or more of the components described herein for these systems may be contained within the cartridge housing. For example, the cartridge may have various reservoirs for storing samples, reagents, enzymes, other biomolecules, buffers, aqueous and non-polar solutions, waste, etc. Thus, one or more of the fluid reservoir system, fluid control system, or temperature control system may be removably engaged with the bioassay system via a cartridge or other biosensor.

[0154] The illumination system 1916 may include a light source (e.g., one or more light-emitting diodes (LEDs)) and multiple optical components for illuminating the biosensor. Examples of light sources include lasers, arc lamps, LEDs, or laser diodes. The optical components may be, for example, reflectors, polarizers, beam splitters, collimators, lenses, filters, wedges, prisms, mirrors, detectors, etc. In embodiments using an illumination system, the illumination system 1916 may be configured to direct excitation light toward the reaction sites. As an example, a fluorophore may be excited by a green wavelength of light, so the wavelength of the excitation light may be approximately 1932 nm. In one embodiment, the illumination system 1916 is configured to generate illumination parallel to a surface normal to the surface of the biosensor 1912. In another embodiment, the illumination system 1916 is configured to generate illumination that is off-angled relative to the surface normal to the surface of the biosensor 1912. In yet another embodiment, the illumination system 1916 is configured to generate illumination having multiple angles, including some parallel illumination and some off-angle illumination.

[0155] The system receptacle or interface 1910 is configured to engage the biosensor 1912 in at least one of mechanical, electrical, and fluidic manners. The system receptacle 1910 can hold the biosensor 1912 in a desired orientation to facilitate fluid flow through the biosensor 1912. The system receptacle 1910 can also include electrical contacts configured to engage the biosensor 1912 so that the sequencing system 1900A can communicate with and / or provide power to the biosensor 1912. Additionally, the system receptacle 1910 can include a fluid port (e.g., a nozzle) configured to engage the biosensor 1912. In some embodiments, the biosensor 1912 is removably coupled to the system receptacle 1910 mechanically, electrically, and fluidically.

[0156] Additionally, the sequencing system 1900A may communicate remotely with other systems or networks, or with other bioassay systems 1900A. Detection data obtained by the bioassay system 1900A may be stored in a remote database.

[0157] FIG. 19B is a block diagram of a system controller 1906 that can be used in the system of FIG. 19A. In one embodiment, the system controller 1906 includes one or more processors or modules that can communicate with each other. Each of the processors or modules may include algorithms (e.g., instructions stored on a tangible and / or non-transitory computer-readable storage medium) or sub-algorithms for performing a particular process. The system controller 1906 is conceptually illustrated as a collection of modules, but may be implemented using any combination of dedicated hardware boards, DSPs, processors, etc. Alternatively, the system controller 1906 may be implemented using an off-the-shelf PC with a single processor or multiple processors, with functional operations distributed among the processors. As a further option, the modules described below may be implemented using a hybrid configuration in which certain modular functions are implemented using dedicated hardware, while remaining modular functions are implemented using an off-the-shelf PC, etc. The modules may also be implemented as software modules within a processing unit.

[0158] During operation, the communication port 1950 may send information (e.g., commands) to or receive information (e.g., data) from the biosensor 1912 ( FIG. 19A ) and / or the subsystems 1908, 1914, 1904 ( FIG. 19A ). In certain embodiments, the communication port 1950 may output multiple sequences of pixel signals. The communication link 1934 may receive user input from the user interface 1918 ( FIG. 19A ) or send data or information to the user interface 1918. Data from the biosensor 1912 or the subsystems 1908, 1914, 1904 may be processed in real time by the system controller 1906 during a bioassay session. Additionally or alternatively, the data may be temporarily stored in system memory during a bioassay session and processed in slower than real time or in offline operation.

[0159] As shown in FIG. 19B, the system controller 1906 may include a central processing unit (CPU) 1952 as well as multiple modules 1926-1948 in communication with a main control module 1924. The main control module 1924 may be in communication with a user interface 1918 (FIG. 19A). While the modules 1926-1948 are shown in direct communication with the main control module 1924, the modules 1926-1948 may also be in direct communication with each other, the user interface 1918, and the biosensor 1912. The modules 1926-1948 may also be in communication with the main control module 1924 through other modules.

[0160] The plurality of modules 1926-1948 includes system modules 1928-1932, 1926, which communicate with subsystems 1908, 1914, 1904, and 1916, respectively. Fluid control module 1928 may communicate with fluid control system 1908 to control valves and flow sensors in the fluid network to control the flow of one or more fluids through the fluid network. Fluid storage module 1930 may notify a user when fluid storage is low or when a waste reservoir is at or near capacity. Fluid storage module 1930 may also communicate with temperature control module 1932 so that fluids can be stored at a desired temperature. Illumination module 1926 may communicate with illumination system 1916 to illuminate reaction sites at specified times during a protocol, such as after a desired reaction (e.g., a binding event) has occurred. In some embodiments, illumination module 1926 may communicate with illumination system 1916 to illuminate the reaction sites at a specified angle.

[0161] The plurality of modules 1926-1948 may also include an apparatus module 1936 that communicates with the biosensor 1912 and an identification module 1938 that determines identification information associated with the biosensor 1912. The apparatus module 1936 may, for example, communicate with the system receptacle 1910 to confirm that the biosensor has established electrical and fluidic connectivity with the sequencing system 1900A. The identification module 1938 may receive a signal that identifies the biosensor 1912. The identification module 1938 may use the identification information of the biosensor 1912 to provide other information to a user. For example, the identification module 1938 may determine and subsequently display the lot number, manufacturing date, or a recommended protocol to be run on the biosensor 1912.

[0162] The plurality of modules 1926-1948 also includes an analysis module 1944 (also referred to as a signal processing module or signal processor) that receives and analyzes signal data (e.g., image data) from the biosensor 1912. The analysis module 1944 includes memory (e.g., RAM or flash) for storing the detection / image data. The detection data can include multiple sequences of pixel signals, whereby sequences of pixel signals from each of millions of sensors (or pixels) can be detected over many base call cycles. The signal data can be stored for subsequent analysis or transmitted to the user interface 1918 to display desired information to a user. In some embodiments, the signal data can be processed by a solid-state image sensor (e.g., a CMOS image sensor) before the analysis module 1944 receives the signal data.

[0163] Analysis module 1944 is configured to acquire image data from the photodetector in each of a plurality of sequencing cycles. The image data is derived from the luminescence signals detected by the photodetector and processes the image data for each of the plurality of sequencing cycles via neural network-based base caller 100 to generate base calls for at least some of the analytes in each of the plurality of sequencing cycles. The photodetector may be part of one or more overhead cameras (e.g., a CCD camera in an Illumina GAIIx that takes images of the clusters on biosensor 1912 from above) or may be part of biosensor 1912 itself (e.g., a CMOS image sensor in an Illumina iSeq that is below the clusters on biosensor 1912 and takes images of the clusters from the bottom).

[0164] The output of the photodetector is a sequence image showing the intensity emissions of each cluster and their surrounding background. The sequence image shows the intensity emissions generated as a result of incorporating nucleotides into a sequence during sequencing. The intensity emissions are from the associated analytes and their surrounding background. The sequence image is stored in memory 1948.

[0165] Protocol modules 1940 and 1942 communicate with main control module 1924 to control the operation of subsystems 1908, 1914, and 1904 in implementing a predetermined assay protocol. Protocol modules 1940 and 1942 may include instruction sets for instructing sequencing system 1900A to perform specific operations according to a predetermined protocol. As shown, the protocol modules may be a sequencing-by-synthesis (SBS) module 1940 configured to issue various commands to execute a sequencing-by-synthesis process. In SBS, the extension of a nucleic acid primer along a nucleic acid template is monitored to determine the nucleotide sequence in the template. The underlying chemical process may be polymerization (e.g., catalyzed by a polymerase enzyme) or ligation (e.g., catalyzed by a ligase enzyme). In certain polymer-based SBS embodiments, fluorescently labeled nucleotides are added to primers (thereby extending the primers) in a template-dependent manner, such that detection of the order and type of nucleotides added to the primers can be used to determine the sequence of the template. For example, to initiate the first SBS cycle, one or more labeled nucleotides, DNA polymerase, etc. can be delivered into / through a flow cell containing an array of nucleic acid templates. The nucleic acid templates may be located at corresponding reaction sites. Primer extension can detect incorporated labeled nucleotides through an imaging event, and these reaction sites can be detected. During the imaging event, an illumination system 1916 can provide excitation light to the reaction sites. Optionally, the nucleotides can further include a reversible termination feature that terminates further primer extension once the nucleotide is added to the primer. For example, a nucleotide analog with a reversible terminator moiety can be added to the primer to prevent further extension until a deblocking agent is delivered to remove the moiety.Thus, in another embodiment using reversible termination, a command can be given to deliver a deblocking reagent to the flow cell (either before or after detection). One or more commands can be given to provide wash(s) between the various delivery steps. The cycle is then repeated n times to extend the primer by n nucleotides, thereby allowing a sequence of length n to be detected. Exemplary sequencing techniques are described, for example, in Bentley et al., Nature 4196:193-199 (20019), International Patent Application Publication No. 04 / 0119497, U.S. Patent No. 7,0197,026, International Patent Application Publication No. 91 / 066719, International Patent Application Publication No. 07 / 123744, U.S. Patent No. 7,329,492, U.S. Patent No. 7,211,414, U.S. Patent No. 7,3119,019, U.S. Patent No. 7,4019,2191, and U.S. Patent Application Publication No. 20019 / 01470190192, each of which is incorporated herein by reference.

[0166] In the nucleotide delivery step of the SBS cycle, any single type of nucleotide can be delivered at a time, or multiple different nucleotide types (e.g., A, C, T, and G) can be delivered. In nucleotide delivery configurations where only a single type of nucleotide is present at a time, different nucleotides do not need to have distinct labels because they can be distinguished based on the temporal separation inherent in individualized delivery. Thus, a sequencing method or device can use single-color detection. For example, the excitation source only needs to provide excitation at a single wavelength or a single wavelength range. In nucleotide delivery configurations where delivery results in multiple different nucleotides being present in the flow cell at a given time, the sites at which different nucleotide types incorporate can be distinguished based on the different fluorescent labels attached to each nucleotide type in the mixture. For example, four different nucleotides, each bearing one of four different fluorophores, can be used. In one embodiment, four different fluorophores can be distinguished using excitation in four different regions of the spectrum. For example, four different excitation radiation sources can be used. Alternatively, fewer than four different excitation sources can be used, but optical filtering of the excitation radiation from a single source can be used to generate different excitation radiation ranges in the flow cell.

[0167] In some embodiments, fewer than four different colors can be detected in a mixture having four different nucleotides. For example, pairs of nucleotides can be detected at the same wavelength but can be distinguished based on differences in intensity for one member of the pair, or based on a change to one member of the pair (e.g., through chemical modification, photochemical modification, or physical modification) that causes a distinct signal to appear or disappear compared to the signal detected for the other member of the pair. Exemplary devices and methods for distinguishing four different nucleotides using detection of fewer than four colors are described, for example, in U.S. Patent Application Nos. 61 / 19319,294 and 61 / 619,19719, which are incorporated herein by reference in their entireties. U.S. Patent Application No. 13 / 624,200, filed September 21, 2012, is also incorporated by reference in its entirety.

[0168] The multiple protocol modules may also include a sample preparation (or generation) module 1942 configured to issue instructions to the fluidic control system 1908 and the temperature control system 1904 to amplify the product in the biosensor 1912. For example, the biosensor 1912 may be coupled to a sequencing system 1900A. The amplification module 1942 can issue instructions to the fluidic control system 1908 to deliver the necessary amplification components to a reaction chamber in the biosensor 1912. In other embodiments, the reaction site may already contain some components for amplification, such as template DNA and / or primers. After delivering the amplification components to the reaction chamber, the amplification module 1942 can instruct the temperature control system 1904 to cycle through different temperature steps according to a known amplification protocol. In some embodiments, amplification and / or nucleotide incorporation is performed isothermally.

[0169] The SBS module 1940 can issue commands to perform bridge PCR, in which clusters of clonal amplicons are formed over localized regions within the flow cell channel. After generating amplicons via bridge PCR, the amplicons may be "linearized" to create single-stranded template DNA, and sstDNA and sequencing primers may be hybridized to universal sequences flanking the region of interest. For example, reversible terminator-based sequencing by synthesis methods can be used, as described above or as follows.

[0170] Each base calling or sequencing cycle can extend the sstDNA by a single base, which can be achieved, for example, by using a modified DNA polymerase and a mixture of four types of nucleotides. Different types of nucleotides can have unique fluorescent labels, and each nucleotide can further have a reversible terminator that allows only a single base to be incorporated in each cycle. After a single base is added to the sstDNA, excitation light can be incident on the reaction site and fluorescence emission can be detected. After detection, the fluorescent label and terminator can be chemically cleaved from the sstDNA. Another similar base calling or sequencing cycle can follow. In such a sequencing protocol, the SBS module 1940 can instruct the fluid control system 1908 to direct the flow of reagent and enzyme solutions through the biosensor 1912. Exemplary reversible terminator-based SBS methods that can be utilized with the devices and methods described herein are described in U.S. Patent Application Publication No. 2007 / 01667019(A1), U.S. Patent Application Publication No. 2006 / 01196*3901(A1), U.S. Patent No. 7,0197,026, U.S. Patent Application Publication No. 2006 / 0240439(A1), U.S. Patent Application Publication No. 2006 / 021914714709(A1), WO 019 / 0619514, U.S. Patent Application Publication No. 20019 / 014700900(A1), WO 06 / 019B199, and WO 07 / 014702191, each of which is incorporated by reference in its entirety. Exemplary reagents for reversible terminator-based SBS are described in U.S. Pat. No. 7,1941,444, U.S. Pat. No. 7,0197,026, U.S. Pat. No. 7,414,14716, U.S. Pat. No. 7,427,673, U.S. Pat. No. 7,1966,1937, U.S. Pat. No. 7,1992,4319, and International Patent Application Publication No. WO 07 / 14193193619, each of which documents is incorporated herein by reference in its entirety.

[0171] In some embodiments, the amplification and SBS modules may operate in a single assay protocol, for example, template nucleic acids are amplified and subsequently sequenced within the same cartridge.

[0172] The sequencing system 1900A may also allow the user to reconfigure the assay protocol. For example, the sequencing system 1900A may provide the user with options through the user interface 1918 to modify the determined protocol. For example, if it is determined that the biosensor 1912 is to be used for amplification, the sequencing system 1900A may request the temperature of the annealing cycle. Furthermore, the sequencing system 1900A may issue a warning to the user if the user provides user input that is not generally accepted for the selected assay protocol.

[0173] In an embodiment, biosensor 1912 includes millions of sensors (or pixels), each of which generates multiple sequences of pixel signals over subsequent base call cycles. Analysis module 1944 detects the multiple sequences of pixel signals and attributes them to corresponding sensors (or pixels) according to the row and / or column positions of the sensors on the array of sensors.

[0174] FIG. 19C is a simplified block diagram of a system for analyzing sensor data, such as base call sensor output, from a sequencing system 1900A. In the example of FIG. 19C, the system includes a configurable processor 1946. The configurable processor 1946 can execute a base caller (e.g., neural network-based base caller 100) in coordination with a runtime program / logic 1980 executed by a central processing unit (CPU) 1952 (i.e., a host processor). The sequencing system 1900A includes a biosensor 1912 and a flow cell. The flow cell may include one or more tiles in which clusters of genetic material are exposed to a series of analyte flows that are used to trigger reactions within the clusters to identify bases in the genetic material. A sensor detects reactions for each cycle of sequencing in each tile of the flow cell to provide tile data. Genetic sequencing is a data-intensive operation that converts base call sensor data into a sequence of base calls for each group of genetic material sensed during the base calling operation.

[0175] The system of this example includes a CPU 1952 that executes runtime program / logic 1980 for coordinating base calling operations, and memory 1948B that stores sequences of arrays of tile data, base call reads generated by the base calling operations, and other information used in the base calling operations. In this figure, the system also includes a configuration file (or multiple configuration files), e.g., field programmable gate array (FPGA) bit files, and memory 1948A for storing model parameters used to configure and reconfigure configurable processor 1946 and run the neural network. Sequencing system 1900A can include a program for configuring the configurable processor, and in some embodiments, can include a reconfigurable processor to run the neural network.

[0176] Sequencing system 1900A is coupled to configurable processor 1946 by bus 1989. Bus 1989, in one example, can be implemented using high-throughput technology, such as bus technology compatible with the PCIe (Peripheral Component Interconnect Express) standard currently maintained and developed by the PCI-SIG (Peripheral Components Interconnect Express) standard. Also in this example, memory 1948A is coupled to configurable processor 1946 by bus 1993. Memory 1948A may be on-board memory located on a circuit board with configurable processor 1946. Memory 1948A is used for fast access by configurable processor 1946 of working data used in base call operations. Bus 1993 can also be implemented using high-throughput technology, such as bus technology compatible with the PCIe standard.

[0177] Configurable processors, including field programmable gate arrays (FPGAs), coarse-grained reconfigurable arrays (CGRAs), and other configurable and reconfigurable devices, can be configured to implement various functions more efficiently or faster than can be achieved using general-purpose processors running computer programs. Configuring a configurable processor involves compiling a functional description to generate a configuration file, sometimes referred to as a bitstream or bitfile, and distributing the configuration file to configurable elements on the processor. The configuration file configures the circuit to set dataflow patterns, including the use of distributed memory and other on-chip memory resources, lookup table contents, the operation of configurable logic blocks, and configurable execution units such as configurable interconnects and other elements of the configurable array. A configuration file is reconfigurable if it can be changed in the field by modifying a loaded configuration file. For example, the configuration file may be stored in a volatile SRAM element, a non-volatile read-write memory element, or distributed among an array of configurable elements on a configurable or reconfigurable processor. Various commercially available configurable processors are suitable for use in basecall operations as described herein.Examples include Google's Tensor Processing Unit (TPU)™, GX4 Rackmount Series™, GX9 Rackmount Series™, NVIDIA DGX-1™, Microsoft's Stratix V FPGA™, Graphcore's Intelligent Processor Unit (IPU)™, Qualcomm's Zeroth Platform™ Snapdragon processors™, NVIDIA Volta™, NVIDIA's Drive PX™, NVIDIA's JETSON TX1 / TX2 MODULE™, Intel's Nirvana™, Movidius VPU™, Fujitsu DPI™, Arm DynamicIQ™, IBM TrueNorth™, Lambda GPU Server with Testa V100s™, Xilinx Alveo™ U200, Xilinx Alveo™ U2190, Xilinx Alveo™ U280, Intel / Altera Stratix™ GX2800, Intel / Altera Stratix™ GX2800, and Intel Stratix™ GX10M. In some embodiments, the host CPU may be implemented on the same integrated circuit as the configurable processor.

[0178] The embodiments described herein implement the neural network-based basis caller 100 using a configurable processor 1946. The configuration file for the configurable processor 1946 can be implemented by specifying the logic functions to be performed using a high-level description language HDL or a register-transfer level RTL language specification. This specification can be compiled using resources designed for a selected configurable processor to generate the configuration file. The same or similar specifications can be compiled to generate a design for an application-specific integrated circuit, which may not be a configurable processor.

[0179] Thus, alternatives to the configurable processor 1946 in all embodiments described herein include a configured processor comprising an application specific ASIC or dedicated integrated circuit or set of integrated circuits, or is a system-on-chip SOC device, or a graphics processing unit (GPU) processor or a Coarse-Grained Reconfigurable Architecture (CGRA) processor configured to perform neural network-based base call operations as described herein.

[0180] In general, the configurable and configured processors described herein that are configured to perform neural network operations are referred to herein as neural network processors.

[0181] Configurable processor 1946, in this example, is configured using a program executed by CPU 1952 or by a configuration file loaded by another source to configure an array of configurable elements 1991 (e.g., Configuration Logic Blocks (CLBs), such as Look Up Tables (LUTs), flip-flops, Processing Units (PMUs), and Compute Memory Units (CMUs), configurable I / O blocks, programmable interconnects) to perform base calling functions. In this example, the configuration includes data flow logic 1997 coupled to buses 1989 and 1993, which performs the function of distributing data and control parameters among elements used in base calling operations.

[0182] Configurable processor 1946 is also configured with data flow logic 1997 to execute neural network-based base caller 100. Logic 1997 includes multi-cycle execution clusters (e.g., 1979), which in this example include execution cluster 1 through execution cluster X. The number of multi-cycle execution clusters can be selected according to tradeoffs involving the desired throughput of operation and available resources on configurable processor 1946.

[0183] The multi-cycle execution clusters are coupled to data flow logic 1997 by data flow paths 1999 implemented using configurable interconnect and memory resources on configurable processor 1946. The multi-cycle execution clusters are also coupled to data flow logic 1997 by control paths 1995 implemented using configurable interconnect and memory resources, for example, on configurable processor 1946, that are ready to provide control signals indicative of available execution clusters, provide input units to available execution clusters for execution of operations of neural network based base caller 100, provide learned parameters of neural network based base caller 100, and provide output patches of base call classification data, as well as other control data used in the execution of neural network based base caller 100.

[0184] The configurable processor 1946 is configured to execute a run of the neural network-based base caller 100 using the learned parameters to generate classification data for detection cycles of the base calling operation. The neural network-based base caller 100 executes a run to generate classification data for subject detection cycles of the base calling operation. The run of the neural network-based base caller 100 operates in a sequence including a number N of arrays of tile data from each detection cycle of the N detection cycles, where the N detection cycles provide sensor data for different base calling operations for one base position per operation in the time sequence described herein. Optionally, some of the N detection cycles can be removed from the sequence as needed according to the particular neural network model being implemented. The number N can be any number greater than 1. In some examples described herein, the detection cycles of the N detection cycles represent a set of detection cycles for at least one detection cycle preceding the subject detection cycle and at least one detection cycle following the subject cycle. Examples described herein include an integer number N of 5 or greater.

[0185] Data flow logic 1997 is configured to use an input unit for a given operation that includes tile data for an array of N spatially aligned patches to move the tile data and at least some of the learned parameters of the model parameters from memory 1948A to configurable processor 1946 for a run of neural network-based bass caller 100. The input unit can be moved by direct memory access operations in a single DMA operation, or in smaller units that move during available time slots in coordination with the execution of the deployed neural network.

[0186] The tile data of the sensing cycles described herein can include an array of sensor data having one or more features. For example, the sensor data can include two images analyzed to identify one of four bases at a base position in a genetic sequence of DNA, RNA, or other genetic material. The tile data can also include metadata about the images and sensors. For example, in a base calling implementation, the tile data can include information about the alignment of the images with the clusters, such as distance from center information indicating the distance of each pixel in the array of sensor data from the center of the group of genetic material on the tile.

[0187] As described below, during execution of the neural network-based base caller 100, the tile data may also include data generated during execution of the neural network-based base caller 100, which is referred to as intermediate data that can be reused rather than recomputed during a run of the neural network-based base caller 100. For example, during execution of the neural network-based base caller 100, the data flow logic 1997 may write the intermediate data to memory 1948A in place of the sensor data for a given patch of the array of tile data. Such implementations are described in more detail below.

[0188] As shown, a system for analyzing base calling sensor output is described that includes a memory (e.g., 1948A) accessible by runtime program / logic 1980 that stores tile data including sensor data for tiles from a detection cycle of a base calling operation. The system also includes a neural network processor, such as configurable processor 1946, with access to the memory. The neural network processor is configured to execute a neural network run using learned parameters to generate classification data for the detection cycle. As described herein, the neural network operates on a sequence of N arrays of tile data from each of the N detection cycles comprising a subject cycle to generate classification data for the subject cycle. Data flow logic 1997 is provided to move the tile data and learned parameters from the memory to the neural network processor for execution of the neural network, using input units including data for the N arrays of spatially aligned patches from each of the N detection cycles.

[0189] Also described is a system in which a neural network processor has access to a memory and includes a plurality of execution clusters, the execution clusters being configured to execute a neural network. Data flow logic 1997 has access to the memory and executes a cluster in the plurality of execution clusters to provide an input unit of tile data to an available execution cluster in the plurality of execution clusters, the input unit including a number N of spatially aligned patches of the array of tile data from each sensing cycle, and causing the execution cluster to apply the N spatially aligned patches to the neural network to generate an output patch of classification data for the spatially aligned patches of the subject sensing cycle, where N is greater than 1.

[0190] Figure 20A is a simplified diagram illustrating aspects of a base calling operation, including the functionality of a runtime program (e.g., runtime logic 1980) executed by a host processor. In this diagram, image sensor output from a flow cell is provided on line 2000 to image processing thread 2001, which can perform processes on the image, such as aligning and positioning individual tiles within an array of sensor data and resampling the image, which can be used by a process to calculate a tile cluster mask for each tile within the flow cell, which can be used by a process to identify pixels within the array of sensor data that correspond to clusters of genetic material on the corresponding tile of the flow cell. The output of image processing thread 2001 is provided via line 2002 to dispatch logic 2003 within the CPU, which transfers the array of tile data to neural network processor hardware 2007, such as configurable processor 1946 of Figure 19C, on high-speed bus 2004 or on high-speed bus 2006 to data cache 2005 (e.g., SSD storage) according to the state of the base calling operation. The processed and transformed images may be stored on a data cache 2005 to detect previously used cycles. Hardware 2007 returns the classification data output by the neural network to dispatch logic 2003, which passes the information to data cache 2005 or on line 2008 to thread 2009, which uses the classification data to perform base calling and quality score calculations and can arrange the data in a standard format for base called reads. The output of thread 2009, which performs base calling and quality score calculations, is provided on line 2010 to thread 2011, which aggregates the base called reads, performs other operations such as data compression, and writes the resulting base calling output to a specified destination for consumption by the customer.

[0191] In some implementations, the host may include a thread (not shown) that performs final processing of the output of the hardware 2007 supporting the neural network. For example, the hardware 2007 may provide classification data output from the final layer of a multi-cluster neural network. The host processor may perform output activation functions, such as a softmax function, over the classification data to populate the data used by the base calling and quality score thread 2002. The host processor may also perform input operations (not shown), such as batch normalization of the tile data before input to the hardware 2007.

[0192] FIG. 20B is a simplified diagram of a configurable processor 1946 configuration such as that of FIG. 19C. In FIG. 20B, the configurable processor 1946 includes an FPGA with multiple high-speed PCIe interfaces. The FPGA is configured with a wrapper 2090 including data flow logic 1997, as described with reference to FIG. 19C. The wrapper 2090 manages interfacing and coordination with the runtime program in the CPU via CPU communication link 2077 and communication with on-board DRAM 2099 (e.g., memory 1448A) via DRAM communication link 2097. The data flow logic 1997 in the wrapper 2090 provides patch data obtained by traversing an array of tile data on on-board DRAM 2099 to clusters 2085, and obtains process data 2087 from clusters 2085 and delivers it to on-board DRAM 2099, for a number N of cycles. The wrapper 2090 also manages the transfer of data between onboard DRAM 2099 and host memory, both for input arrays of tile data and for output patches of classification data. The wrapper transfers patch data on line 2083 to assigned clusters 2085. The wrapper provides learned parameters, such as weights and biases on line 2081, to clusters 2085 retrieved from onboard DRAM 2099. The wrapper provides configuration and control data on line 2079 to clusters 2085 provided by, or generated in response to, a runtime program on the host via CPU communication link 2077. The clusters can also provide status signals on line 2089 to wrapper 2090, which are used in conjunction with control signals from the host to provide spatially aligned patch data and to run a multi-cycle neural network on the patch data using the resources of clusters 2085.

[0193] As described above, multiple clusters may reside on a single configurable processor managed by a wrapper 2090 configured to run on corresponding ones of the multiple patches of tile data. Each cluster may be configured to provide classification data for base calls in a subject detection cycle using the tile data of multiple sensing cycles described herein.

[0194] In an example system, model data, including kernel data such as filter weights and biases, can be sent from the host CPU to the configurable processor, so that the model can be updated as a function of cycle number. Base calling operations can typically involve hundreds of sensing cycles. In some implementations, base calling operations can include paired end reads. For example, model-learned parameters can be updated every 20 cycles (or other number of cycles) or according to an update pattern implemented in a particular system and neural network model. In some implementations, where a sequence for a given string in a genetic cluster on a tile includes paired end reads that include a first portion extending downward (or upward) from a first end of the string and a second portion extending upward (or downward) from a second end of the string, learned parameters can be updated at the transition from the first portion to the second portion.

[0195] In some embodiments, image data for multiple cycles of sensor data for a tile can be sent from the CPU to the wrapper 2090. The wrapper 2090 can optionally perform some preprocessing and transformation of the sensor data and write the information to onboard DRAM 2099. The input tile data for each sensing cycle can include an array of sensor data containing 4000 x 3000 pixels / tile or more per tile, with two features representing the colors of two images of the tile and including one or two bytes per pixel. For embodiments where the number N is three sensing cycles used in each operation of the multi-cycle neural network, the array of tile data for each operation of the multi-cycle neural network can consume several hundred megabytes per number. In some implementations of the system, the tile data also includes an array of distance-from-cluster center (DFC) data stored once per tile, or other types of metadata about the sensor data and tile.

[0196] In operation, if a multi-cycle cluster is available, the wrapper assigns the patch to the cluster. The wrapper fetches the next patch of tile data for the cross section of the tile and sends it to the assigned cluster along with the appropriate control and configuration information. The cluster can be configured with enough memory on the configurable processor to have enough memory to hold the patch of data, including the patch, from multiple cycles in some systems being processed in place, and in various implementations are processed using ping-pong buffer or raster scan techniques.

[0197] When an assigned cluster completes its operation of the neural network for the current patch and generates an output patch, it signals the wrapper. The wrapper either reads the output patch from the assigned cluster, or the assigned cluster pushes data to the wrapper. The wrapper then assembles the output patch for the processed tile in DRAM 2099. Once processing of the entire tile is complete and the output patch of data is transferred to DRAM, the wrapper sends the processed output array back to the host / CPU in a specific format. In some embodiments, the on-board DRAM 2099 is managed by memory management logic within the wrapper 2090. The runtime program can control the sequencing operations to complete analysis of all tile data arrays for every cycle, operating in a continuous flow to provide real-time analysis.

[0198] FIG. 21 illustrates another embodiment of the disclosed data flow logic that makes the compressed spatial map generated during the first base calling iteration available during the second base calling iteration from off-chip memory (e.g., off-chip DRAM, host RAM, host high bandwidth memory (HBM)) 2116.

[0199] In one embodiment, a host memory (e.g., memory 1948B) attached to a host processor (e.g., CPU 1952) is configured to receive the progression of sequence images 2102 as the sequencing run progresses. The configurable processor (e.g., configurable processor 1946) has an array of processing units. The processing units in the array of processing units are configured to execute the neural network-based base caller 100 to generate base call predictions. Data flow logic 1997 has access to the host memory, the host processor, and the configurable processor. For the first base calling iteration, the data flow logic 1997 loads the sequence images for the sequencing cycles of the first window of sequencing cycles (e.g., sequencing cycles 1-5 of FIG. 1A) from the host memory onto the configurable processor.

[0200] The runtime logic 1980 is configured to cause the processing units of the configurable processor to execute the spatial network 104 of the neural network-based base caller 100 on the sequence image 2102 for each cycle to generate a spatial feature map set 2106 for each sequencing cycle within a first window of sequencing cycles. In one embodiment, the runtime logic 1980 executes, in parallel, multiple processing clusters of the neural network-based base caller 100 on patches 2104 tiled from the sequence image 2102. The multiple processing clusters apply the spatial network 104 on the patches 2104 patch by patch 2105.

[0201] The runtime logic 1980 is configured to cause the processing unit of the configurable processor to execute the compression network 108 of the neural network-based base caller 100 on the spatial feature map set 2106 for each cycle to generate a compressed spatial feature map set 2107, and further process the compressed spatial feature map set 2107 through the temporal network 160 and the output network 190 to generate base call predictions 2111 for one or more sequencing cycles within a first window of sequencing cycles. The temporal network 160 generates the temporal feature maps 2108. The output network 190 generates base call classification scores 2110 (e.g., unnormalized basewise scores). In one embodiment, the compressed spatial feature map set 2107 is stored on an off-chip memory 2116.

[0202] In one embodiment, the data flow logic 1997 is configured to move the compressed spatial feature map set 2107 to the host memory 2116 and overwrite the compressed spatial feature map set 2107 with the corresponding one in the sequence image 2102. In another embodiment, the compressed spatial feature map set 2107 replaces the corresponding one in the patch 2104.

[0203] The second window of sequencing cycles (sequencing cycles 2-6 in FIG. 2A ) shares one or more overlapping sequencing cycles (e.g., sequencing cycles 2-5) with the first window of sequencing cycles, and has at least one non-overlapping sequencing cycle (e.g., sequencing cycle 6), but for the second base call iteration and for the second window of sequencing cycles, the data flow logic 1997 is configured to load, from host memory, into the scalable processor, the compressed spatial feature map set 216 for the overlapping sequencing cycles and the sequence image 2122 (or patch 2124) for the non-overlapping sequencing cycles.

[0204] The runtime logic 1980 is configured to cause a processing unit of the configurable processor to execute the spatial network 104 on the sequence images 2122 of the non-overlapping sequencing cycles to generate a set of spatial feature maps 2126 for the non-overlapping sequencing cycles. In one embodiment, multiple processing clusters apply the spatial network 104 to the patches 2124 patch by patch 2125.

[0205] The runtime logic 1980 is configured to cause a processing unit of the configurable processor to execute the compression network 108 on the spatial feature map set 2126 to generate a compressed spatial feature map set 2127 for non-overlapping sequencing cycles, process the compressed spatial feature map set 2126 for overlapping sequencing cycles, and process the compressed spatial feature map set 2127 for non-overlapping sequencing cycles via the time network 160 and the output network 190, and generate base call predictions 2131 for one or more sequencing cycles in the second window of sequencing cycles. The time network 160 generates a temporal feature map 2128. The output network 190 generates a base call classification score 2129 (e.g., a non-normalized basewise score). In one embodiment, the compressed spatial feature map set 2127 is stored on the off-chip memory 2116.

[0206] Figure 22 illustrates one embodiment of the disclosed data flow logic for making the compressed spatial map generated during the first base calling iteration available during the second base calling iteration from on-chip memory (e.g., processor memory such as on-chip DRAM, on-chip SRAM, on-chip BRAM, or DRAM attached to the processor via an interconnect) 2216. In Figure 22, compressed spatial feature map set 2107 and compressed spatial feature map set 2127 are stored in on-chip memory 2216. Also in Figure 22, data flow logic 1997 is configured to load compressed spatial feature map set 2126 of overlapping sequencing cycles from on-chip memory 2216 onto a configurable processor.

[0207] Split Architecture 23 illustrates one implementation of the so-called partitioned architecture of the neural network-based base caller 100. As described above, the spatial convolutional network 104 is configured to process a series of sequencing cycles (cycles N+2, N+1, N, N-1, N-2, etc.) of a sequencing run by convolving a window of sequence image sets for each cycle through respective sequences 2301, 2302, 2303, 2304, and 2405 of spatial convolutional layers, generating a set of spatial feature maps for each sequencing cycle. For example, each of the five sequences 2301, 2302, 2303, 2304, and 2405 of spatial convolutional layers has seven spatial convolutional layers (i.e., layers L1 through L7 in FIG. 23).

[0208] Each sequence of spatial convolutional layers 2301, 2302, 2303, 2304, and 2305 has a corresponding sequence of spatial convolutional filter banks (e.g., sequence of spatial convolutional layers 2301 includes spatial convolutional filter banks 2310, 2311, 2312, 2313, 2314, 2315, and 2316). In one implementation, the learned coefficients (or weights) of the spatial convolutional filters in each sequence of spatial convolutional filter banks vary between sequences of spatial convolutional layers within each sequence of spatial convolutional layers.

[0209] For example, the spatial convolutional layer sequences 2301, 2302, 2303, 2304, and 2405 are configured using convolutional filters with different trained coefficients. In another example, the convolutional filters in the spatial convolutional layers at corresponding levels have different trained coefficients (e.g., the convolutional filter banks 2382, 2383, 2384, 2385, and 2312 in the third spatial convolutional layer of each of the five spatial convolutional layer sequences 2301, 2302, 2303, 2304, and 2405).

[0210] The temporal convolutional network 160 is configured to process the per-cycle spatial feature map sets group by group by convolving each overlapping group (e.g., groups 2360, 2361, 2362) of the per-cycle spatial feature map sets in the per-cycle spatial feature map sets, and to generate a respective group-wise temporal feature map set for each overlapping group of the per-cycle spatial feature map sets using respective temporal convolutional filter banks 2321, 2322, and 2323 of the first temporal convolutional layer 2320. In one embodiment, the learned coefficients (or weights) of the temporal convolutional filters in each temporal convolutional filter bank vary between the temporal convolutional filter banks 2321, 2322, and 2323 within each temporal convolutional filter bank.

[0211] Skip Architecture Figure 24A shows residual (or skip) connections that reinject previous information downstream via feature map addition. Residual connections involve reinjecting previous representations into the downstream flow of data by appending past output tensors to later output tensors, which helps prevent information loss along the data processing flow. Residual connections are intended to address two common problems that plague any large-scale deep learning model: vanishing gradients and representational bottlenecks.

[0212] Residual connections involve making the output of a previous layer available as input to a subsequent layer, effectively creating a shortcut in a continuous network. The output of the previous layer is summed with the subsequent activation rather than concatenated to it. Note that in this case, both activations are assumed to be the same size. If they are different sizes, a linear transformation can be used to reshape the previous activation to the target shape.

[0213] Figure 24B shows one implementation of a residual block and skip connections. The residual network stacks several residual units to mitigate degradation of training accuracy. The residual block uses special additive skip connections to combat gradient vanishing in deep neural networks. At the start of the residual block, the data flow is separated into two streams: the first stream carries the block's unchanged input, while the second applies weights and nonlinearities. At the end of the block, the two streams are merged using element-wise summation. The main advantage of such a configuration is that it allows gradients to flow more easily through the network.

[0214] When configured with a residual network, in some embodiments, the neural network-based base caller 100 can be easily trained and achieve improved accuracy for image classification and object detection. The neural network-based base caller 100 connects the output of the lth layer as the input of the (l+1)th layer, which enables the transition to the next layer: x = Hl(xl-1). The residual block adds a skip connection that bypasses the nonlinear transformation with the discriminant function x = Hl(xl-1) + xl-1. The advantage of the residual block is that gradients can flow directly from later layers to earlier layers (e.g., spatial and temporal convolutional layers) through the discriminant function. The outputs of the discriminant function and Hl are combined by summation (addition).

[0215] 24C shows the residual architecture of a neural network-based base caller 100 in which the spatial convolutional layers are grouped into residual blocks with skip connections. In another embodiment, the temporal convolutional layers of the neural network-based base caller 100 are grouped into residual blocks with skip connections.

[0216] In the embodiment shown in FIG. 24C, the second and third spatial convolutional layers are grouped into a first residual block 2412, the fourth and fifth spatial convolutional layers are grouped into a second residual block 2422, and the sixth and seventh spatial convolutional layers are grouped into a third residual block 2432.

[0217] FIG. 25A shows details of the bass network of the neural network-based bass caller 100. In one implementation, a given residual block 2585 of the bass network includes a set of spatial convolutional layers 2590 and 2592. A first spatial convolutional layer 2590 in the set of spatial convolutional layers receives as input a previous output 2586 generated by a previous spatial convolutional layer that is not part of the given residual block 2585 (e.g., a null spatial convolutional layer preceding the first spatial convolutional layer 2590 in the spatial network 104). The first spatial convolutional layer 2590 processes the previous output 2586 and generates a first output 2591. A second spatial convolutional layer 2592 in the set of spatial convolutional layers, following the first spatial convolutional layer 2590, receives the first output 2591, processes it, and generates a second output 2593. In one implementation, the first spatial convolutional layer 2590 has a non-linear activation function, such as ReLU, that produces the first output 2591. In another implementation, the second spatial convolutional layer 2592 lacks a non-linear activation function.

[0218] The skip connection 2589 provides the previous output 2586 to an adder 2594. The adder 2594 also receives the second output 2593 from the second spatial convolutional layer 2592. The adder 2594 combines the previous output 2586 and the second output 2593 to generate a summed output 2595. The summed output 2595 is further processed through a nonlinear invocation, such as a ReLU, to generate a final summed output 2587. In some implementations, the final summed output 2587 is then provided as an input to a subsequent residual block. In some implementations, the previous output 2586 is modified to be dimensionally compatible with the second output 2593. For example, the edges of the feature map in the previous output 2586 are trimmed to generate a feature map with the same spatial dimensionality as the feature map in the second output 2593.

[0219] 25B illustrates an exemplary operation of the disclosed bus network. In one embodiment, the bus network is configured to form buses (e.g., 2516, 2526, 2536, 2602, 2604, 2702, and 2712) between spatial convolutional layers in each sequence of spatial convolutional layers. The buses are configured to combine the spatial feature map sets for each cycle generated by two or more spatial convolutional layers in a particular sequence of spatial convolutional layers for a particular sequencing cycle to form a combined per-cycle spatial feature map set, and to provide the combined per-cycle spatial feature map set as an input to another spatial convolutional layer in the particular sequence of spatial convolutional layers.

[0220] For example, consider the first residual block 2412. Here, the two or more spatial convolutional layers include a first spatial convolutional layer and a third spatial convolutional layer. The first spatial convolutional layer generates a first set of spatial feature maps per cycle 2520. The first spatial convolutional layer provides the first set of spatial feature maps per cycle 2520 as input to the second spatial convolutional layer. The second spatial convolutional layer processes the first set of spatial feature maps per cycle 2520 to generate a second set of spatial feature maps per cycle 2522. The second spatial convolutional layer provides the second set of spatial feature maps per cycle 2522 as input to the third spatial convolutional layer. The third spatial convolutional layer processes the second set of spatial feature maps per cycle 2522 to generate a third set of spatial feature maps per cycle 2524. A bus (e.g., skip bus 2519) is further configured to combine (e.g., sum or concatenate by combiner 2502) first spatial feature map set 2520 and third per-cycle spatial feature map set 2524 into per-cycle spatial feature map set 2518. Another spatial convolutional layer is then a fourth spatial convolutional layer that immediately follows the third spatial convolutional layer in the particular sequence of spatial convolutional layers. The fourth spatial convolutional layer processes the combined per-cycle spatial feature map set 2518 as input. The same concept applies to second and third residual blocks 2422 and 2432, except that 2526 and 2536 are skip buses like skip bus 2516, causing respective combiners 2512 and 2532 to generate per-cycle spatial feature map sets 2528 and 2538.

[0221] FIG. 25C shows one implementation of dimensionality compatibility logic 2532 that ensures that the input feature maps provided by the skip buses are modified (e.g., cropped) before combining to ensure that the incoming feature maps have the same spatial dimensions as the receiving feature maps that will be combined by the combiner in the bus network.

[0222] Figure 26 shows another example of the disclosed bus network in which a skip bus 2602 combines the output of the first spatial convolutional layer with the output of the first residual block. Figure 26 also shows that a skip bus 2604 can combine feature maps across multiple residual blocks and across non-subsequent layers (e.g., layers 1 through 5) to generate a combined representation that can be processed by another layer (e.g., layer 6).

[0223] FIG. 27 illustrates yet another example of the disclosed bus network, in which inputs from multiple subsequent and / or non-subsequent layers and representations of the combined representation (e.g., from layer 1, combiner 2502, and combiner 2512) can be combined by exemplary skip buses 2702, 2604, and combiner 2712 to generate a combined representation that can be processed by another layer (e.g., layer 6).

[0224] 28 shows one implementation of scaling logic 2832 that scales (adjusts) the incoming feature maps provided by the skip bus before they are combined with the receiving feature maps to be combined by the combiner of the bus network. The values ​​used by scaling logic 2832 can be, for example, anywhere between 0 and 1, inclusive. The scaling logic can be used, for example, to attenuate or amplify the strength / magnitude / value of the incoming feature map (e.g., its feature quantity (e.g., floating point value)).

[0225] 29 shows one implementation of skip connections between temporal convolutional layers 2902, 2912, 2922, 2932, 2942, 2952, 2962, and 2972 ​​of the temporal network 160. For example, skip connection 2922 feeds temporal feature maps from the first temporal convolutional layer 2902 to the third temporal convolutional layer 2932.

[0226] Figure 30 is a graph comparing base calling performance by a network-based base caller 100 configured with compression logic 108 (the sqz2 base caller) against a network-based base caller 100 without compression logic 108 (used as a baseline for the neural model) and Illumina's non-neural network-based base caller Real-Time Analysis (RTA) software (a baseline for the traditional image processing model). As can be seen in the chart in Figure 30, the sqz2 base caller (purple fit line) has a lower base calling error rate ("% Error" on the Y-axis) than the RTA base caller (black fit line), and also has a lower rate than the two cases of the network-based base caller 100 without compression logic 108 (red and cyan fit lines).

[0227] FIG. 31 illustrates the savings in RAM and DRAM usage that result from use of the disclosed compression logic 108.

[0228] Figure 32 is a graph comparing base calling performance by a network-based base caller 100 configured with a split-and-skip architecture (split_res) against the RTA base caller and against another version of the network-based base caller 100 without the split-and-skip architecture (distilled). As can be seen in the chart in Figure 32, the split_res base caller (orange fit line) has a lower base calling error rate ("error count on the Y-axis") than the RTA base caller (blue fit line).

[0229] As used herein, "logic" (e.g., data flow logic) may be implemented in the form of a computer product including a non-transitory computer-readable storage medium with computer-usable program code for performing the method steps described herein. "Logic" may be implemented in the form of an apparatus including a memory and at least one processor coupled to the memory and operative to perform the exemplary method steps. "Logic" may be embodied in the form of a means for performing one or more of the method steps described herein. The means may include (i) hardware module(s), (ii) software modules executing on one or more hardware processors, or (iii) a combination of hardware and software modules, any of which implements specific techniques described herein, and the software modules are stored on a computer-readable storage medium (or multiple such media). In one embodiment, the logic implements a data processing function. The logic may be a general-purpose, single-core, or multi-core processor with a computer program specifying the functions; a digital signal processor with a computer program; logic such as an FPGA configurable by a configuration file; special-purpose circuitry such as a state machine; or any combination thereof. Also, the computer program product may embody computer program and configuration file portions of logic.

[0230] 33 illustrates a computer system 3300 that may be used by sequencing system 1900A to implement the techniques disclosed herein. Computer system 3300 includes at least one central processing unit (CPU) 3372 that communicates with a number of peripheral devices via a bus subsystem 3355. These peripheral devices may include a storage subsystem 3358, which includes, for example, memory devices and file storage subsystem 3336, user interface input devices 3338, user interface output devices 3376, and a network interface subsystem 3374. The input and output devices enable user interaction with computer system 3300. Network interface subsystem 3374 provides an interface to external networks, including interfaces to corresponding interface devices in other computer systems.

[0231] In one embodiment, the system controller 1906 is communicatively linked to a storage subsystem 3310 and a user interface input device 3338 .

[0232] The user interface input devices 3338 may include keyboards, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, scanners, touch screens integrated into displays, audio input devices such as voice recognition systems and microphones, and other types of input devices. In general, use of the term "input device" is intended to encompass all possible types of devices and ways of inputting information into the computer system 3300.

[0233] The user interface output devices 3376 can include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem can include a flat panel device such as an LED display, a cathode ray tube (CRT), a liquid crystal display (LCD), a projection device, or some other mechanism for producing a visible image. The display subsystem can also provide non-visual displays such as an audio output device. In general, use of the term "output device" is intended to include all possible types of devices and methods for outputting information from the computer system 3300 to a user or to another machine or computer system.

[0234] The storage subsystem 3358 stores programming and data constructs that provide the functionality of some or all of the modules and methods described herein. These software modules are generally executed by the deep learning processor 3378.

[0235] The deep learning processor 3378 may be a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or a coarse-grained reconfigurable architecture (CGRA). The deep learning processor 3378 may be hosted by a deep learning cloud platform such as Google Cloud Platform™, Xilinx™, and Cirrascale™. Examples of deep learning processors 3378 include Google's Tensor Processing Unit (TPU)™, rackmount solutions such as the GX4 Rackmount Series™, GX33 Rackmount Series™, NVIDIA DGX-1™, Microsoft's Stratix V FPGA™, Graphcore's Intelligent Processor Unit (IPU)™, Qualcomm's Zeroth Platform™ with Snapdragon processors™, NVIDIA's Volta™, NVIDIA's DRIVE PX™, NVIDIA's JETSON TX1 / TX2 MODULE™, Intel's Nirvana™, Movidius VPU™, Fujitsu DPI™, ARM's DynamicIQ™, IBM TrueNorth™, Lambda GPU Server with Testa V100s™, SambaNova's Reconfigurable Dataflow Unit (RDU)™, and others.

[0236] The memory subsystem 3322 used in the storage subsystem 3358 may include multiple memories, including a main random access memory (RAM) 3332 for storing instructions and data during program execution, and a read-only memory (ROM) 3334 in which fixed instructions are stored. The file storage subsystem 3336 may provide persistent storage for program and data files and may include a hard disk drive, associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of particular embodiments may be stored by the file storage subsystem 3336, within the storage subsystem 3358, or within other machines accessible by the processor.

[0237] Bus subsystem 3355 provides a mechanism for allowing the various components and subsystems of computer system 3300 to communicate with each other as intended. Although bus subsystem 3355 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0238] The computer system 3300 itself can be of various types, including a personal computer, a portable computer, a workstation, a computer terminal, a network computer, a television, a mainframe, a server farm, a loosely distributed set of loosely networked computers, or any other data processing system or user device. Due to the ever-changing nature of computers and networks, the description of computer system 3300 shown in Figure 33 is intended only as a specific example for purposes of illustrating a preferred embodiment of the present invention. Many other configurations of computer system 3300 can have more or fewer components than the computer system shown in Figure 33.

[0239] item The present inventors disclose the following items. Compression (Squeeze) 1. An artificial intelligence-based method of base calling, comprising: accessing a series of per-cycle analyte channel sets generated for the sequencing cycles of the sequencing run; processing a first window of analyte channel sets for each cycle in the series for a first window of sequencing cycles of the sequencing run through a spatial network of a neural network-based base caller, and generating respective sequences of spatial output sets for each sequencing cycle in the first window of sequencing cycles; processing each final spatial output set in each sequence of spatial output sets through a neural network-based base caller compression network and generating a respective compressed spatial output set for each sequencing cycle within a first window of the sequencing cycle; generating base call predictions for one or more sequencing cycles within the first window of sequencing cycles based on the respective compressed space output sets. 2. The artificial intelligence-based method of item 1, wherein each final spatial output set has M channels (feature maps), each compressed spatial output set has N channels (feature maps), and M is greater than N. 3. for a second window of sequencing cycles of a sequencing run that shares the first window of sequencing cycles with one or more overlapping sequencing cycles for which the spatial network has previously generated a spatial output set and at least one non-overlapping sequencing cycle for which the spatial network has not yet generated a spatial output set; processing the per-cycle analyte channel sets for only the non-overlapping sequencing cycles through a spatial network to generate a sequence of spatial output sets for the non-overlapping sequencing cycles, thereby avoiding re-processing the per-cycle analyte channel sets for each overlapping sequencing cycle through the spatial network; processing a final spatial output set in a sequence of spatial output sets through a compression network to generate compressed spatial output sets for non-overlapping sequencing cycles, wherein the final spatial output has M channels (feature maps) and the compressed spatial output has N channels (feature maps), where M is greater than N; 2. The artificial intelligence-based method of claim 1, further comprising: generating base call predictions for one or more sequencing cycles within a second window of sequencing cycles based on the respective compressed spatial output sets for the overlapping sequencing cycles previously generated for the first window of sequencing cycles and the compressed spatial output sets, thereby replacing the respective compressed spatial output sets for the overlapping sequencing cycles with the respective cycle-by-cycle analyte channel sets for the overlapping sequencing cycles. 4. for a third window of sequencing cycles of a sequencing run that shares the first and second windows of sequencing cycles with one or more overlapping sequencing cycles for which the spatial network previously generated a spatial output set and at least one non-overlapping sequencing cycle for which the spatial network has not yet generated a spatial output set; processing the per-cycle analyte channel sets for only the non-overlapping sequencing cycles through a spatial network to generate a sequence of spatial output sets for the non-overlapping sequencing cycles, thereby avoiding re-processing the per-cycle analyte channel sets for each overlapping sequencing cycle through the spatial network; processing a final spatial output set in a sequence of spatial output sets through a compression network to generate compressed spatial output sets for non-overlapping sequencing cycles, wherein the final spatial output has M channels (feature maps) and the compressed spatial output has N channels (feature maps), where M is greater than N; 4. The artificial intelligence-based method of claim 3, further comprising: generating base call predictions for one or more sequencing cycles within a third window of sequencing cycles based on the respective compressed spatial output sets for the overlapping sequencing cycles previously generated for the first and second windows of sequencing cycles and the compressed spatial output sets, thereby replacing the respective compressed spatial output sets for the overlapping sequencing cycles with the respective cycle-by-cycle analyte channel sets for the overlapping sequencing cycles. 5. The artificial intelligence-based method of item 1, wherein the analyte channel set for each cycle in the series indicates the intensity registered in response to nucleotide incorporation into the analyte at the corresponding sequencing cycle in the sequencing run. 6. The artificial intelligence-based method of item 5, wherein the spatial network has a sequence of spatial convolutional layers that separately process each analyte channel set per cycle in a particular window of the series of analyte channel sets per cycle in a sequencing cycle of a sequencing run, generating a sequence of spatial output sets for each sequencing cycle within the particular window of sequencing cycles, starting with a first spatial convolutional layer that combines intensities only within a single analyte channel set per cycle of a sequencing cycle of interest, and not between analyte channel sets per cycle of different sequencing cycles within the particular window of sequencing cycles, and continuing with subsequent spatial convolutional layers that combine spatial outputs of preceding spatial convolutional layers only within a sequencing cycle of interest, and not between different sequencing cycles within the particular window of sequencing cycles. 7. The artificial intelligence-based method of item 6, wherein each spatial convolutional layer in the sequence of spatial convolutional layers has a different number of convolutional filters, and wherein a final spatial convolutional layer in the sequence of spatial convolutional layers has M convolutional filters, where M is an integer greater than 4. 8. The artificial intelligence-based method of item 7, wherein each spatial convolutional layer in the sequence of spatial convolutional layers has the same number of convolutional filters, that same number being M, where M is an integer greater than 4. 9. The artificial intelligence-based method described in item 8, wherein the convolutional filters in the spatial network use two-dimensional (2D) convolutions. 10. The artificial intelligence-based method described in item 8, wherein the convolutional filters in the spatial network use three-dimensional (3D) convolutions. 11. The artificial intelligence-based method of item 6, wherein the neural network-based base caller has a temporal network, the temporal network having a sequence of temporal convolutional layers that group and process respective compressed spatial output sets for windows of subsequent sequencing cycles within a particular window of a sequencing cycle, generating a sequence of temporal output sets for the particular window of sequencing cycles, starting with a first temporal convolutional layer that combines compressed spatial output sets between different sequencing cycles within the particular window of sequencing cycles, and continuing with subsequent temporal convolutional layers that combine subsequent temporal outputs of the preceding temporal convolutional layers. 12. For the first window of the sequencing cycle, processing each compressed spatial output set for a subsequent window of the sequencing cycle within the first window of the sequencing cycle through a first temporal convolutional layer in a sequence of temporal convolutional layers of the temporal network to generate a plurality of temporal output sets for the first window of the sequencing cycle; processing a plurality of time output sets through a compression network to generate a respective compressed time output set for each time output set in the plurality of time output sets, wherein each time output set has M channels (feature maps), and each compressed time output set has N channels (feature maps), where M is greater than N; processing each compressed temporal output set through a final temporal convolutional layer in a sequence of temporal convolutional layers of the temporal network to generate a final temporal output set for a first window of the sequencing cycle; generating base call predictions for one or more sequencing cycles within the first window of sequencing cycles based on the final time output set; Item 12. The artificial intelligence-based method of item 11, wherein an output layer processes the final time output set and generates a final output for the first window of the sequencing cycle, and base call predictions are generated based on the final output. 13. for a first window of a sequencing cycle and one or more overlapping windows of a subsequent sequencing cycle in which the first temporal convolutional layer previously generated a temporal output set, and a second window of a sequencing cycle that shares at least one non-overlapping window of a subsequent sequencing cycle in which the first temporal convolutional layer has not yet generated a temporal output set; processing each compressed-spatial output set through a first temporal convolutional layer only for each sequencing cycle in a non-overlapping window of the subsequent sequencing cycle to generate a temporal output set for the non-overlapping window of the subsequent sequencing cycle, thereby avoiding re-processing each compressed-spatial output set through the first temporal convolutional layer for each sequencing cycle within an overlapping window of the subsequent sequencing cycle; processing the time output sets through a compression network to generate compressed time output sets for non-overlapping windows of subsequent sequencing cycles, wherein the time output sets have M channels (feature maps) and the compressed time outputs have N channels (feature maps), where M is greater than N; processing, via a final temporal convolutional layer, the respective compressed time output sets for the overlapping windows of the subsequent sequencing cycle previously generated for the first window of the sequencing cycle and the compressed time output sets, to generate a final temporal output set for the second window of the sequencing cycle, thereby replacing the respective compressed time output sets for the overlapping windows of the subsequent sequencing cycle with the respective per-cycle analyte channel sets for the overlapping windows of the subsequent sequencing cycle; generating base call predictions for one or more sequencing cycles within a second window of sequencing cycles based on the final time output set; Item 13. The artificial intelligence-based method of item 12, wherein an output layer processes the final time output set and generates a final output for a second window of the sequencing cycle, and base call predictions are generated based on the final output. 14. for a third window of the sequencing cycle that shares the first and second windows of the sequencing cycle with one or more overlapping windows of the subsequent sequencing cycle in which the first temporal convolutional layer previously generated a temporal output set, and at least one non-overlapping window of the subsequent sequencing cycle in which the first temporal convolutional layer has not yet generated a temporal output set; processing each compressed-spatial output set through a first temporal convolutional layer only for each sequencing cycle in a non-overlapping window of the subsequent sequencing cycle to generate a temporal output set for the non-overlapping window of the subsequent sequencing cycle, thereby avoiding re-processing each compressed-spatial output set through the first temporal convolutional layer for each sequencing cycle within an overlapping window of the subsequent sequencing cycle; processing the time output sets through a compression network to generate compressed time output sets for non-overlapping windows of subsequent sequencing cycles, wherein the time output sets have M channels (feature maps) and the compressed time outputs have N channels (feature maps), where M is greater than N; processing, via a final temporal convolutional layer, the respective compressed time output sets for the overlapping windows of the subsequent sequencing cycle previously generated for the first and second windows and compressed time output sets of the sequencing cycle to generate a final temporal output set for the third window of the sequencing cycle, thereby replacing the respective compressed time output sets for the overlapping windows of the subsequent sequencing cycle with the respective per-cycle analyte channel sets for the overlapping windows of the subsequent sequencing cycle; generating base call predictions for one or more sequencing cycles within a third window of sequencing cycles based on the final time output set; Item 14. The artificial intelligence-based method of item 13, wherein an output layer processes the final time output set and generates a final output for the third window of the sequencing cycle, and base call predictions are generated based on the final output. 15. The artificial intelligence-based method of item 11, wherein each temporal convolutional layer in the sequence of temporal convolutional layers of the temporal network has a different number of convolutional filters, and the first temporal convolutional layer has M convolutional filters, where M is an integer greater than 4. 16. The artificial intelligence-based method of item 11, wherein each temporal convolutional layer in the sequence of temporal convolutional layers of the temporal network has the same number of convolutional filters, that same number being M, where M is an integer greater than 4. 17. The artificial intelligence-based method of item 16, wherein the convolutional filters in the temporal network use one-dimensional (1D) convolution. 18. The artificial intelligence-based method of item 1, wherein the compression network uses 1x1 convolutions to control the number of compressed spatial outputs in the compressed spatial output set, and the compression network has N convolution filters, where N is an integer less than or equal to 4. 19. The artificial intelligence-based method of item 1, further comprising using the data identifying the unreliable analytes to remove portions of the compressed space outputs in the compressed space output set that correspond to the unreliable analytes, generating a filtered compressed space output set to replace the compressed space output set, and generating base call predictions only for those analytes that are not unreliable analytes. 20. The artificial intelligence-based method of item 19, further comprising processing the filtered compressed spatial output set through a temporal network instead of the corresponding compressed spatial output set. 21. The artificial intelligence-based method of item 20, further comprising generating a compressed temporal output set from the filtered compressed spatial output set. 22. The artificial intelligence-based method of item 19, wherein the data identifying unreliable analytes identifies pixels that exhibit unreliable cluster intensities. 23. The artificial intelligence-based method of item 19, wherein the data identifying unreliable analytes identifies pixels that do not exhibit any intensity. 24. The artificial intelligence-based method of item 20, wherein the compressed space output set has a total number of pixels that is 4 to 9 times the total number of pixels of the corresponding filtered compressed space output set. 25. The artificial intelligence-based method of item 24, wherein the filtered compressed spatial output set allows the temporal network to operate on 75% fewer pixels, thereby reducing the computational operations, memory accesses, and memory occupation of the temporal network by 75%. 26. The artificial intelligence-based method described in item 5, wherein avoiding reprocessing through the spatial network reduces computational operations, memory accesses, and memory occupation for the temporal network by 80%. 27. The artificial intelligence-based method of item 14, wherein avoiding reprocessing through the time network reduces the computational operations, memory accesses, and memory occupation of the time network. 28. The artificial intelligence-based method of item 27, further comprising reallocating computational resources made available by the compression network to adding supplemental convolutional filters in the spatial and temporal networks. 29. The artificial intelligence-based method of item 27, further comprising reallocating computational resources made available by the compression network to adding supplemental per-cycle analyte channel sets within each window of per-cycle analyte channel sets used to generate base call predictions for a particular sequencing cycle. 30. The artificial intelligence-based method of item 27, further comprising reallocating computational resources made available by the compression network to adding supplemental spatial convolutional layers in the spatial network. 31. The artificial intelligence-based method of item 27, further comprising reallocating computational resources made available by the compression network to adding supplemental temporal convolutional layers in the temporal network. 32. The artificial intelligence-based method of item 1, further comprising generating base call predictions for one or more sequencing cycles in the current window of sequencing cycles using one or more compressed space output sets generated for one or more preceding windows of sequencing cycles in conjunction with one or more compressed space output sets generated for the current window of sequencing cycles. 33. The artificial intelligence-based method of item 1, further comprising generating base call predictions for one or more sequencing cycles in the current window of sequencing cycles using one or more compressed space output sets generated for one or more subsequent windows of sequencing cycles in conjunction with one or more compressed space output sets generated for the current window of sequencing cycles. 34. The artificial intelligence-based method of item 1, further comprising generating base call predictions for one or more sequencing cycles in the current window of sequencing cycles using one or more compressed time output sets generated for one or more preceding windows of sequencing cycles in conjunction with one or more compressed time output sets generated for the current window of sequencing cycles. 35. The artificial intelligence-based method of item 1, further comprising generating base call predictions for one or more sequencing cycles in the current window of sequencing cycles using one or more compressed time output sets generated for one or more subsequent windows of sequencing cycles in conjunction with one or more compressed time output sets generated for the current window of sequencing cycles. 36. The artificial intelligence-based method described in item 1, wherein the analyte channel set per cycle encodes analyte data for analytes sequenced during a sequencing run. 37. The artificial intelligence-based method of item 36, wherein the analyte data is image data identifying intensity radiation collected from the analyte. 38. The artificial intelligence-based method according to item 37, wherein the image data has multiple image channels (images). 39. The artificial intelligence-based method according to item 38, wherein the image channel (image) is generated by a combination of (i) illumination with a specific laser and (ii) imaging through a specific optical filter. 40. The artificial intelligence-based method of item 36, wherein the analyte data is current and / or voltage data detected based on analyte activity. 41. The artificial intelligence-based method according to item 36, wherein the analyte data is pH scale data detected based on analyte activity. 42. The artificial intelligence-based method described in item 1, wherein the number of channels in the analyte channels per cycle set in series determines the number of convolution filters in the compression network and therefore the number of channels in the compressed spatial output set and the compressed temporal output set. 43. The artificial intelligence-based method of item 1, wherein the compressed spatial output set, the compressed filter spatial output set, and the compressed temporal output set are stored in quantized form. 44. A system comprising: a host memory attached to the host processor and configured to receive a progression of sequence images as the sequencing run progresses; a configurable processor having an array of processing units, wherein processing units in the array of processing units are configured to execute a neural network-based base caller to generate base call predictions; data flow logic having access to a host memory, a host processor, and a configurable processor, the data flow logic being configured to load a sequence image for a sequencing cycle within a first window of sequencing cycles from the host memory onto the configurable processor; run-time logic configured to cause a processing unit to run a neural network-based base caller spatial network on sequence images of the sequencing cycles in a first window of the sequencing cycles, for each cycle, to generate a spatial feature map set for each of the sequencing cycles in the first window of the sequencing cycles; runtime logic configured to cause a processing unit to run a condensed network of a neural network-based base caller on the spatial feature map set for each cycle to generate a condensed spatial feature map set, and process the condensed spatial feature map set through a temporal network and an output network to generate base call predictions for one or more sequencing cycles within a first window of sequencing cycles; data flow logic configured to move the compressed spatial feature map set to a host memory and overwrite the sequence image with the compressed spatial feature map set; For a second window of sequencing cycles that shares one or more overlapping sequencing cycles with the first window of sequencing cycles and has at least one non-overlapping sequencing cycle, the data flow logic is configured to load, from the host memory onto the configurable processor, compressed spatial feature map sets for the overlapping sequencing cycles and sequence images for the non-overlapping sequencing cycles; the runtime logic is configured to cause the processing unit to execute the spatial network on the sequence images of the non-overlapping sequencing cycles and generate a set of spatial feature maps for the non-overlapping sequencing cycles; The system, wherein the runtime logic is configured to cause the processing unit to run a compression network on the spatial feature map sets to generate compressed spatial feature map sets for non-overlapping sequencing cycles, process the compressed spatial feature map sets for overlapping sequencing cycles, and process the compressed spatial feature map sets via a temporal network and an output network for non-overlapping sequencing cycles, and generate base call predictions for one or more sequencing cycles in a second window of sequencing cycles. 45. A system comprising: a host memory attached to the host processor and configured to receive a progression of sequence images as the sequencing run progresses; a configurable processor having an array of processing units attached to a processor memory, wherein a processing unit in the array of processing units is configured to execute a neural network based base caller to generate base call predictions; data flow logic having access to a host memory, a host processor, a configurable processor, and the processor memory, configured to load a sequence image from the host memory for a sequencing cycle within a first window of sequencing cycles on the configurable processor; Runtime logic configured to cause a processing unit to run a neural network-based base caller spatial network on sequence images of the sequencing cycles in a first window of the sequencing cycles for each cycle, and generate a spatial feature map set for each of the sequencing cycles in the first window of the sequencing cycles; runtime logic configured to cause a processing unit to run a condensed network of a neural network-based base caller on the spatial feature map set for each cycle to generate a condensed spatial feature map set, and process the condensed spatial feature map set through a temporal network and an output network to generate base call predictions for one or more sequencing cycles within a first window of sequencing cycles; data flow logic configured to move the set of compressed spatial feature maps into a processor memory; For a second window of sequencing cycles that shares one or more overlapping sequencing cycles with the first window of sequencing cycles and has at least one non-overlapping sequencing cycle, the data flow logic is configured to load, on the configurable processor, from the processor memory, compressed spatial feature map sets for the overlapping sequencing cycles and load sequence images from the host memory for the non-overlapping sequencing cycles; the runtime logic is configured to cause the processing unit to execute the spatial network on the sequence images of the non-overlapping sequencing cycles and generate a set of spatial feature maps for the non-overlapping sequencing cycles; The system, wherein the runtime logic is configured to cause the processing unit to run a compression network on the spatial feature map sets to generate compressed spatial feature map sets for non-overlapping sequencing cycles, process the compressed spatial feature map sets for overlapping sequencing cycles, and process the compressed spatial feature map sets via a temporal network and an output network for non-overlapping sequencing cycles, and generate base call predictions for one or more sequencing cycles in a second window of sequencing cycles. 46. ​​A system comprising: neural network logic configured to perform a first traversal of a neural network graph to process each input in a first input set independently via first processing logic, generate alternative representations of each input in the first input set without mixing information between each input in the first input set, and generate an output of the first traversal based on the alternative representations of each input in the first input set; neural network logic configured to perform a second traversal of the neural network graph to independently process each input in a second input set via the first processing logic, generate alternative representations for each input in the second input set without mixing information between each input in the second input set, and generate an output of the second traversal based on the alternative representations for each input in the second input set, the first and second input sets having one or more overlapping inputs and at least one non-overlapping input; run-time logic configured with the neural network logic to perform a first traversal to generate respective alternative representations for each input in the first input set, store the respective alternative representations for each input in the first input set in a compressed format in a memory, and generate an output for the first traversal based on the respective alternative representations in the compressed format for each input in the first input set; and runtime logic configured to perform a second traversal to process only the non-overlapping inputs via the first processing logic, generate alternative representations of the non-overlapping inputs, store the alternative representations of the non-overlapping inputs in compressed form in memory, read the compressed alternative representations of each of the overlapping inputs generated during the first traversal as compensation to avoid redundant generation of alternative representations of each of the overlapping inputs in the second traversal, and generate output for the second traversal based on the compressed alternative representations of each of the overlapping inputs and the compressed alternative representations of the non-overlapping inputs. 47. The system of item 46, wherein the memory is on-chip memory. 48. The system of item 46, wherein the memory is off-chip memory. 49. The system of item 46, wherein the number of channels in the compressed form corresponds to the number of channels of the inputs in the first and second input sets. 50. An artificial intelligence-based method for base calling, comprising: accessing a series of per-cycle analyte channel sets generated for a sequencing cycle of the sequencing run, wherein a given per-cycle analyte channel set encodes analyte data sensed for an analyte in a given sequencing cycle of the sequencing run; processing the subject's per-cycle analyte channel set through a first processing module of a neural network to generate an intermediate representation of the subject's per-cycle analyte channel set having M feature maps; processing the intermediate representations through a second processing module of the neural network to generate a reduced number of intermediate representations of the analyte channel set for each cycle of interest having N feature maps, where N is less than M; and generating base call predictions for analytes in the target sequencing cycle and / or other sequencing cycles of the sequencing run using a reduced number of intermediate representations of the analyte channel set for each target cycle. 51. The artificial intelligence-based method of item 50, wherein the first processing module is a convolution layer having M convolution filters. 52. The artificial intelligence-based method of item 50, wherein the second processing module is a convolution layer with N convolution filters. 53. An artificial intelligence-based method for base calling, comprising: processing the progression of the generated cycle-by-cycle analyte channel sets through the sequencing cycles of the sequencing run through a sliding window-based neural network-based base caller such that successive sliding windows have overlapping sequencing cycles; For a current window of sequencing cycles that includes one or more preceding sequencing cycles, a central sequencing cycle, and one or more following sequencing cycles, generating a spatial intermediate representation and a compressed intermediate representation for each of the preceding, middle, and subsequent sequencing cycles based on applying a neural network-based base caller to a current window of the analyte channel set for each cycle, wherein the spatial intermediate representation has M channels and the compressed intermediate representation has N channels, where M is greater than N; base calling at least the central sequencing cycle based on the compressed intermediate representations generated for the preceding sequencing cycle, the central sequencing cycle, and the subsequent sequencing cycle; and calling bases in at least the central sequencing cycle within a next window of sequencing cycles using the compressed intermediate representations generated for the preceding sequencing cycle, the central sequencing cycle, and the subsequent sequencing cycle. 54. An artificial intelligence-based system for base calling, comprising: A host processor; a memory accessible by the host processor that stores analyte data for a sequencing cycle of a sequencing run; A configurable processor having access to a memory, comprising: a plurality of execution clusters, an execution cluster within the plurality of execution clusters configured to execute the neural network; 1. An artificial intelligence-based system comprising: a processor including: data flow logic having access to a memory and to an execution cluster within a plurality of execution clusters, the data flow logic being configured to provide analyte data to an available execution cluster within the plurality of execution clusters, the execution cluster applying the analyte data to a neural network to generate an intermediate representation of the analytic data and a compressed intermediate representation for use in a current base calling step, and feeding the compressed intermediate representation back to memory to be used in place of the analytic data in a future base calling step, wherein the intermediate representation has M channels and the compressed intermediate representation has N channels, and M is greater than N. 55. A system comprising: runtime logic configured to execute a first iteration of the base caller to process the input and generate an intermediate representation of the input; compression logic configured to process the intermediate representation and generate a compressed intermediate representation of the input; The system, wherein the runtime logic is configured to use the compressed intermediate representation in place of the input in subsequent iterations of the base caller. 56. A system comprising: runtime logic configured to execute a first iteration of the base caller to process the input and generate an intermediate representation of the input; compression logic configured to process the intermediate representation and generate a compressed intermediate representation, the compressed intermediate representation configured to have the same number of channels as the input; The system, wherein the runtime logic is configured to use the compressed intermediate representation in place of the input in subsequent iterations of the base caller. 57. The system of item 56, wherein the channels correspond to feature maps. 58. The system of item 56, wherein the channel corresponds to a depth dimension. 59. The system according to item 56, wherein the channels correspond to spatial dimensions.

[0240] Split 1. A system comprising: a spatial convolutional network configured to process, for each cycle, a window of per-cycle sequence image sets for a series of sequencing cycles of a sequencing run by separately convolving each per-cycle sequence image set within the window of per-cycle sequence image sets through a respective sequence of spatial convolutional layers to generate, for each sequencing cycle in the series of sequencing cycles, a respective per-cycle spatial feature map set; a spatial convolutional network, wherein each sequence of spatial convolutional layers has a respective sequence of spatial convolutional filter banks, and spatial convolutional filters in the spatial convolutional filter banks of each sequence of spatial convolutional filter banks are changed between sequences of spatial convolutional layers in each sequence of learned coefficients of the spatial convolutional layers; a temporal convolutional network configured to process the per-cycle spatial feature map sets group-wise by convolving with a respective temporal convolutional filter bank of a first temporal convolutional layer on each overlapping group of the per-cycle spatial feature map sets in the per-cycle spatial feature map sets to generate a respective group-wise temporal feature map set for each overlapping group of the per-cycle spatial feature map sets; a temporal convolution network that varies learned coefficients of the temporal convolution filters in each temporal convolution filter bank between the temporal convolution filter banks in each temporal convolution filter bank. 2. The system of item 1, wherein the spatial convolution filter uses split convolution in cycles. 3. The system of item 1, wherein the temporal convolution filter uses cycle-to-cycle combinatorial convolution. 4. The system of item 1, further configured to include a compression network that separately convolves each per-cycle spatial feature map set through a respective compressed convolution layer to generate a respective per-cycle compressed spatial feature map set for each sequencing cycle. 5. The system of item 4, wherein the learned coefficients of the compressed convolution filters in each compressed convolution layer vary between compressed convolution layers in each compressed convolution layer. 6. The system of item 5, wherein the temporal convolutional network is further configured to process the group-wise temporal feature map sets by convolving each overlapping group of the group-wise temporal feature map sets in the group-wise temporal feature map sets with a respective temporal convolutional filter bank of a second temporal convolutional layer, and to generate a respective further group-wise temporal feature map set for each overlapping group of the group-wise temporal feature map sets. 7. The system of item 6, further configured to include an output network that processes the final set of temporal feature maps produced by the final temporal convolutional layer to generate a final output. 8. The system of item 7, further configured to generate base call predictions for one or more sequencing cycles in the series of sequencing cycles based on the final output. 9. A system comprising: a spatial convolutional network configured to process, for each cycle, a window of per-cycle sequence image sets for a series of sequencing cycles of a sequencing run by separately convolving each per-cycle sequence image set within the window of per-cycle sequence image sets through a respective sequence of spatial convolutional layers to generate, for each sequencing cycle in the series of sequencing cycles, a respective per-cycle spatial feature map set; a temporal convolutional network configured to process the per-cycle spatial feature map sets group-wise by convolving with a respective temporal convolutional filter bank on each overlapping group of the per-cycle spatial feature map sets to generate a respective group-wise temporal feature map set for each overlapping group in the per-cycle spatial feature map sets; a temporal convolution network, wherein learned coefficients of the temporal convolution filters within each temporal convolution filter bank vary between temporal convolution filter banks within each temporal convolution filter bank. 10. The system described in item 9, wherein each sequence of spatial convolutional layers has a respective sequence of spatial convolutional filter banks, and the spatial convolutional filters in the spatial convolutional filter banks of each sequence of spatial convolutional filter banks are shared between the sequences of spatial convolutional layers in each sequence of learned coefficients of the spatial convolutional layers. 11. The system of item 9, further configured to include a compression network that separately convolves each set of spatial feature maps for each cycle through a respective compressed convolutional layer to generate a respective compressed spatial feature map set for each corresponding sequencing cycle, wherein the compressed convolutional filters within each compressed convolutional layer vary between compressed convolutional layers for each learned coefficient. 12. An artificial intelligence-based method of base calling, comprising: processing the window of per-cycle sequence image sets for the series of sequencing cycles of the sequencing run, for each cycle, through a spatial convolutional network by separately convolving each per-cycle sequence image set within the window of per-cycle sequence image sets through each sequence of spatial convolutional layers, to generate a respective per-cycle spatial feature map set for each sequencing cycle in the series of sequencing cycles; Each sequence of spatial convolutional layers has a respective sequence of spatial convolutional filter banks, and spatial convolutional filters in the spatial convolutional filter banks of each sequence of spatial convolutional filter banks are changed between the sequences of spatial convolutional layers in each sequence of learned coefficients of the spatial convolutional layers; processing the per-cycle spatial feature map sets group-wise through a temporal convolutional network by convolving each overlapping group of the per-cycle spatial feature map sets in the per-cycle spatial feature map sets with a respective temporal convolutional filter bank of the first temporal convolutional layer to generate a respective group-wise temporal feature map set for each overlapping group of the per-cycle spatial feature map sets; the learned coefficients of the temporal convolution filters within each temporal convolution filter bank vary between temporal convolution filter banks within each temporal convolution filter bank. 13. The artificial intelligence-based method of item 12, further comprising separately convolving each per-cycle spatial feature map set through a respective compressed convolutional layer of a compression network to generate a per-cycle compressed spatial feature map set for each sequencing cycle. 14. The artificial intelligence-based method of item 13, wherein the learned coefficients of the compressed convolution filters within each compressed convolution layer vary between compressed convolution layers within each compressed convolution layer. 15. The artificial intelligence-based method of item 14, further comprising: processing the group-wise temporal feature map sets through a temporal convolutional network by convolving each overlapping group of the group-wise temporal feature map sets in the group-wise temporal feature map sets with a respective temporal convolutional filter bank of a second temporal convolutional layer to generate a respective further group-wise temporal feature map set for each overlapping group of the group-wise temporal feature map sets. 16. The artificial intelligence-based method of item 15, further comprising processing the final set of temporal feature maps produced by the final temporal convolutional layer through an output network to produce a final output. 17. The artificial intelligence-based method of item 16, further comprising generating base call predictions for one or more sequencing cycles in the series of sequencing cycles based on the final output. 18. An artificial intelligence-based method for base calling, comprising: processing the window of per-cycle sequence image sets for the series of sequencing cycles of the sequencing run, for each cycle, through a spatial convolutional network by separately convolving each per-cycle sequence image set within the window of per-cycle sequence image sets through each sequence of spatial convolutional layers, to generate a respective per-cycle spatial feature map set for each sequencing cycle in the series of sequencing cycles; processing the per-cycle spatial feature map sets group-wise through a temporal convolutional network by convolving each overlapping group of the per-cycle spatial feature map sets in the per-cycle spatial feature map sets with a respective temporal convolutional filter bank of the first temporal convolutional layer to generate a respective group-wise temporal feature map set for each overlapping group of the per-cycle spatial feature map sets; The artificial intelligence based method, wherein the learned coefficients of the temporal convolution filters within each temporal convolution filter bank vary between temporal convolution filter banks within each temporal convolution filter bank. 19. The artificial intelligence-based method described in item 18, wherein each sequence of spatial convolutional layers has a respective sequence of spatial convolutional filter banks, and the spatial convolutional filters in the spatial convolutional filter banks of each sequence of spatial convolutional filter banks are shared between the sequences of spatial convolutional layers in each sequence of spatial convolutional layers. 20. The artificial intelligence-based method of item 18, further comprising separately convolving each set of spatial feature maps for each cycle through a respective compressed convolutional layer of the compression network to generate a set of compressed spatial feature maps for each sequencing cycle, wherein the learned coefficients of the compressed convolutional filters within each compressed convolutional layer vary between compressed convolutional layers within each compressed convolutional layer. 21. A system comprising: a spatial convolutional network configured to apply a respective sequence of spatial convolutional layers to each per-cycle sequence image within a window of per-cycle sequence images; A system in which each sequence of spatial convolutional layers has a respective sequence of spatial convolutional filter banks that is different for each sequence of spatial convolutional layers. 22. A system comprising: a temporal convolutional network comprising a first temporal convolutional layer configured to apply a respective set of temporal convolutional filters to a respective sliding window of the spatial feature map; 1. A system, wherein each set of temporal convolution filters in a first temporal convolution layer has temporal convolution filters that are different for each set of temporal convolution filters. 23. The system of item 22, wherein the temporal convolutional network is configured with a first temporal convolutional layer followed by a second temporal convolutional layer, the second convolutional layer being configured to apply respective sets of temporal convolutional filters to respective sliding windows of the temporal feature map, and each set of temporal convolutional filters in the second temporal convolutional layer having a different temporal convolutional filter for each set of temporal convolutional filters.

[0241] skip 1. A system comprising: For a series of sequencing cycles of the sequencing run, a window of sequence image sets per cycle is processed by separately processing each per-cycle sequence image set within the window of sequence image sets per cycle through a respective spatial processing pipeline, each spatial processing pipeline being configured to convolve each per-cycle sequence image set through a respective sequence of spatial convolutional layers, and a spatial convolutional network being configured to generate a respective per-cycle spatial feature map set for each sequencing cycle in the series of sequencing cycles. a bus network connected to the spatial convolutional network and configured to form buses between spatial convolutional layers in each sequence of spatial feature maps of the spatial convolutional layers, the bus being configured to combine each per-cycle set generated by two or more spatial convolutional layers in a particular sequence of spatial convolutional layers for a particular sequencing cycle to form a combined per-cycle spatial feature map set, and to provide the combined per-cycle spatial feature map set as an input to another spatial convolutional layer in the particular sequence of spatial convolutional layers. 2. The two or more spatial convolutional layers include a first spatial convolutional layer and a third spatial convolutional layer, wherein the first spatial convolutional layer generates a set of spatial feature maps per first cycle, the first spatial convolutional layer provides the set of spatial feature maps per first cycle as an input to a second spatial convolutional layer, and the second spatial convolutional layer processes the set of spatial feature maps per first cycle and generates a set of spatial feature maps per second cycle; Item 1. The system of item 1, wherein the second spatial convolutional layer provides the second set of spatial feature maps per cycle as input to the third spatial convolutional layer, and the third spatial convolutional layer processes the second set of spatial feature maps per cycle to generate a third set of spatial feature maps per cycle. 3. The system of item 2, wherein the bus is further configured to combine the first spatial feature map set and the third per-cycle spatial feature map set into a combined per-cycle spatial feature map set. 4. The system of item 3, wherein another spatial convolutional layer is a fourth spatial convolutional layer that immediately follows the third spatial convolutional layer in a particular sequence of spatial convolutional layers, and the fourth spatial convolutional layer processes the combined set of spatial feature maps for each cycle as input. 5. The system of item 2, wherein the two or more spatial convolutional layers include a first spatial convolutional layer and a seventh spatial convolutional layer, wherein the third spatial convolutional layer provides a set of spatial feature maps per third cycle as input to a fourth spatial convolutional layer, which processes the set of spatial feature maps per third cycle to generate a set of spatial feature maps per fourth cycle, and the fourth spatial convolutional layer provides the set of spatial feature maps per fourth cycle as input to a fifth spatial convolutional layer, which processes the set of spatial feature maps per fourth cycle to generate a set of spatial feature maps per fifth cycle. 6. The system of item 5, wherein the bus is further configured to combine the first spatial feature map set and the fifth spatial feature map set into a combined per-cycle spatial feature map set. 7. The system of item 6, wherein another spatial convolutional layer is a sixth spatial convolutional layer that immediately follows the fifth spatial convolutional layer in a particular sequence of spatial convolutional layers, and the sixth spatial convolutional layer processes the combined per-cycle spatial feature map set as input. 8. The system of item 5, wherein the two or more spatial convolutional layers include a first spatial convolutional layer, a third spatial convolutional layer, and a fifth spatial convolutional layer, and wherein the bus is further configured to combine the first per-cycle spatial feature map set, the third per-cycle spatial feature map set, and the fifth per-cycle spatial feature map set into a combined per-cycle spatial feature map set. 9. The system of item 8, wherein another spatial convolutional layer is a sixth spatial convolutional layer that processes the combined set of spatial feature maps per cycle as input. 10. The system of item 1, wherein the bus is further configured to provide the per-cycle sequence image set for a particular sequencing cycle as input to a first spatial convolutional layer and combine the third per-cycle spatial feature map set into a combined per-cycle spatial feature map set. 11. The system of item 10, wherein another spatial convolutional layer is a fourth spatial convolutional layer that processes the combined cycle-by-cycle spatial feature map set as input. 12. The system of item 1, wherein the bus network includes dimensionality compatibility logic configured to modify the spatial and depth dimensionality of an incoming per-cycle spatial feature map set to be combined with a receiving per-cycle spatial feature map set, and further configured to generate a combined per-cycle spatial feature map set. 13. The system of item 12, wherein the dimensionality compatibility logic is a dimensionality reduction operation including convolution, pooling, or averaging. 14. The system of item 12, wherein the bus network is further configured to include scaling logic configured to scale feature values ​​of an incoming per-cycle spatial feature map set that is combined with the receiving per-cycle spatial feature map set to generate a per-cycle spatial feature map set. 15. The system of item 1, further configured to include a temporal convolutional network configured to process the per-cycle spatial feature map sets group-wise by convolving each overlapping group of per-cycle spatial feature map sets in the per-cycle spatial feature map sets with a respective temporal convolutional filter bank of the first temporal convolutional layer to generate a respective group-wise temporal feature map set for each overlapping group of per-cycle spatial feature map sets. 16. The system of item 15, further configured to include a bus network connected to the temporal convolutional network and configured to form buses between temporal convolutional layers in each sequence of temporal convolutional layers, the bus being configured to combine each per-cycle temporal feature map set generated by two or more temporal convolutional layers in a particular sequence of temporal convolutional layers for a particular sequencing cycle to form a combined per-cycle temporal feature map set, and to provide the combined per-cycle temporal feature map set as an input to another temporal convolutional layer in the particular sequence of temporal convolutional layers. 17. An artificial intelligence-based method comprising: processing a window of per-cycle sequence image sets for a series of sequencing cycles of a sequencing run through a spatial convolutional network by separately processing each per-cycle sequence image set within the per-cycle sequence image set window through a respective spatial processing pipeline, for each cycle, including convolving each per-cycle sequence image set through a respective sequence of spatial convolutional layers to generate a respective per-cycle spatial feature map set for each sequencing cycle in the series of sequencing cycles; combining respective per-cycle spatial feature map sets generated by two or more spatial convolutional layers in a particular sequence of spatial convolutional layers for a particular sequencing cycle to form a combined per-cycle spatial feature map set, and providing the combined per-cycle spatial feature map set as an input to another spatial convolutional layer in the particular sequence of spatial convolutional layers. 18. The two or more spatial convolutional layers include a first spatial convolutional layer and a third spatial convolutional layer, wherein the first spatial convolutional layer generates a set of spatial feature maps per first cycle, the first spatial convolutional layer provides the set of spatial feature maps per first cycle as input to a second spatial convolutional layer, and the second spatial convolutional layer processes the set of spatial feature maps per first cycle and generates a set of spatial feature maps per second cycle; Item 18. The artificial intelligence-based method of item 17, wherein the second spatial convolutional layer provides the second set of spatial feature maps per cycle as input to the third spatial convolutional layer, and the third spatial convolutional layer processes the second set of spatial feature maps per cycle to generate a third set of spatial feature maps per cycle. 19. The artificial intelligence-based method of item 18, wherein the bus is further configured to combine the first spatial feature map set and the third per-cycle spatial feature map set into a combined per-cycle spatial feature map set. 20. The artificial intelligence-based method of item 19, wherein another spatial convolutional layer is a fourth spatial convolutional layer that immediately follows the third spatial convolutional layer in a particular sequence of spatial convolutional layers, and the fourth spatial convolutional layer processes the combined per-cycle spatial feature map set as input.

[0242] Other implementations of the methods described above may include a non-transitory computer-readable storage medium storing instructions executable by a processor to perform any of the above-described methods. Yet another implementation of the methods described in this section may include a system including a memory and one or more processors operable to execute instructions stored in the memory, and may perform any of the above-described methods. [Explanation of symbols]

[0243] 100 Neural Network Based Base Callers 102 Cycle 1 Input 104 Spatial Logic 106 Cycle 1 Spatial Map 108 Compression Logic 110 Cycle 1 Compressed Space Map 112 Cycle 2 Input 116 Cycle 2 Spatial Map 120 Cycle 2 Compressed Space Map 122 Cycle 3 Input 125 Unreliable cluster identifier 126 Cycle 3 Spatial Map 127 Filter Calculator 128 Untrusted Cluster 130 Cycle 3 Compressed Space Map 132 Cycle 4 Input 136 Cycle 4 Spatial Map 140 Cycle 4 Compressed Space Map 142 Cycle 5 Input 146 Cycle 5 Spatial Map 150 Cycle 5 Compressed Space Map 160 Time Logic 172 Time Map of Cycles 1, 2, and 3 174 Time Map of Cycles 2, 3, and 4 176 Time Map of Cycles 3, 4, and 5 182 Time Map of Cycles 1, 2, 3, 4, and 5 190 Output Logic 192 Cycle 3 base calls 194 Input of cycle i 196a Spatial map of spatial layer 1 198 Compression Filter 199 Compressed Space Map 230 Cycle 6 Compressed Space Map 278 Time Map of Cycles 4, 5, and 6 282 Time Map of Cycles 2, 3, 4, 5, and 6 292 Cycle 4 base calls 322 Cycle 7 Input 326 Cycle 7 Spatial Map 330 Cycle 7 Compressed Space Map 378 Time Map of Cycles 5, 6, and 7 382 Time Map for Cycles 3, 4, 5, 6, and 7 392 Cycle 5 base calls 422 cycle 30 input 426 Cycle 30 Spatial Map 430 cycle 30 compressed space map 482 Time map of cycles 1, 2, 3, 4, 5, 6, 7, ···, 30 493 cycle 16 base calls 502 Filtering Logic 510 Cycle 1 filtered compressed space map 520 Cycle 2 filtered compressed space map 530 Cycle 3 filtered compressed space map 540 Cycle 4 filtered compressed space map 550 Cycle 5 filtered compressed space map 572 Filtered time maps for cycles 1, 2, and 3 574 Filtered time maps for cycles 2, 3, and 4 576 Filtered time maps for cycles 3, 4, and 5 582 Filtered time maps for cycles 1, 2, 3, 4, and 5 592 Cycle 3 base calls 650 cycle 6 filtered compressed spatial map 676 Filtered time maps for cycles 4, 5, and 6 682 Filtered time maps for cycles 2, 3, 4, 5, and 6 692 Cycle 4 base calls 750 Cycle 7 Filtered Compressed Space Map 776 Filtered time maps for cycles 5, 6, and 7 782 Filtered time maps for cycles 3, 4, 5, 6, and 7 792 Cycle 5 base calls 802 Compressed time map of cycles 1, 2, and 3 804 Compressed time map of cycles 2, 3, and 4 806 Compressed time map of cycles 3, 4, and 5 814 Compressed time map of cycles 1, 2, 3, 4, and 5 892 Cycle 3 base calls 906 Compressed time map of cycles 4, 5, and 6 914 Compressed time map of cycles 2, 3, 4, 5, and 6 992 Cycle 4 base calls 1006 Compressed time map of cycles 5, 6, and 7 1014 Compressed time map of cycles 3, 4, 5, 6, 7 1092 cycle 5 base calls 1102 Filtered compressed time maps for cycles 1, 2, and 3 1104 Filtered compressed time maps for cycles 2, 3, and 4 1106 Filtered compressed time maps for cycles 3, 4, and 5 1114 Filtered compressed time maps for cycles 1, 2, 3, 4, and 5 1192 Cycle 3 base calls 1206 Filtered compressed time maps for cycles 4, 5, and 6 1214 Filtered compressed time maps for cycles 2, 3, 4, 5, and 6 1292 Cycle 4 base calls 1306 Filtered compressed time maps for cycles 5, 6, and 7 1314 Filtered compressed time maps for cycles 3, 4, 5, 6, and 7 1392 cycle 5 base calls 1412 Feature Map 1512 feature maps 1612 feature maps 1712 Feature Map 1804 Filter Selection Logic 1808 Compressed Output 1812 1 input channel 1816 1 filter 1818 1 feature map 1822 2 input channels 1826 Two Filters 1828 Two feature maps 1832 3 input channels 1836 3 filters 1838 Three Feature Maps 1842 4 input channels 1846 4 filters 1848 4 feature maps 1900A Sequencing System 1902 Housing 1904 Temperature Control System 1906 System Controller 1908 Fluid Control System 1910 System Receptacle or Interface 1912 Biosensor 1914 Fluid Storage System 1916 Lighting System 1918 User Interface 1920 display 1922 User Input Devices 1924 Main Control Module 1926 Lighting Module 1928 Fluid Control Module 1930 Fluid Storage Module 1932 Temperature Control Module 1936 Equipment Module 1938 Identification Module 1940 SBS Module 1942 Amplification Module 1944 Analysis Module 1946 configurable processor 1948 Memory 1950 communication port 1952 host processor (CPU) 1979 Multi-cycle execution cluster 1980 Runtime Logic 1989 Bus 1991 Components 1993 Bus 1995 Control Pathway 1997 Data Flow Logic 1999 Data Flow 2000 lines 2001 Image Processing 2002 Line 2003 Dispatch 2004 Expressway Bus 2005 Data Cache 2006 Expressway Bus 2007 Hardware 2008 Line 2009 Base Calls and Q Scores 2010 Line 2011 Aggregation, writing 2077 CPU communication links 2079 line 2081 line 2083 lines 2085 cluster 2087 Process Data 2089 lines 2090 Rapper 2097 DRAM communication links 2099 onboard DRAM 2102 Image data 2104 Patch Data 2106 Spatial Feature Map Set 2107 Compressed Spatial Feature Map Set 2108 Temporal Feature Map 2110 base call classification scores 2111 Base call for cycle 3 2116 Compressed Space Map for Cycles 1, 2, 3, 4, 5 2126 Compressed space map for overlapping cycles 2, 3, 4, and 5 2122 Image data 2124 patch data 2126 Spatial Feature Map Set 2127 Compressed Spatial Feature Map Set 2128 Temporal Feature Map 2129 base call classification scores 2131 Base Calls for Cycle 4 2216 Compressed Space Map for Cycle 6 2131 Base Calls for Cycle 4 2301 Spatial Convolutional Layer Sequence 2302 Spatial Convolutional Layer Sequence 2303 Spatial Convolutional Layer Sequence 2304 Spatial Convolutional Layer Sequence 2310 Spatial Convolution Filter Bank 2311 Spatial Convolution Filter Bank 2312 Spatial Convolution Filter Bank 2313 Spatial Convolution Filter Bank 2314 Spatial Convolution Filter Bank 2315 Spatial Convolution Filter Bank 2316 Spatial Convolution Filter Bank 2320 First Temporal Convolutional Layer 2321 Temporal Convolution Filter Bank 2322 Temporal Convolution Filter Bank 2323 Temporal Convolution Filter Bank 2360 Spatial feature map sets for each overlapping group 2361 each overlapping group of spatial feature map sets 2362 each overlapping group of spatial feature map sets 2382 Convolutional filter bank in the third spatial convolutional layer 2383 Convolutional filter bank in the third spatial convolutional layer 2384 Convolutional filter bank in the third spatial convolutional layer 2385 Convolutional filter bank in the third spatial convolutional layer 2412 Remaining Block 1 2422 Remaining Block 2 2432 Remaining Block 3 2502 Combiner 2512 Combiner 2516 Bus 2518 spatial feature map set 2520 spatial feature map sets for each first cycle 2522 Spatial feature map set for each second cycle 2524 Spatial feature map sets for each third cycle 2526 Bus 2528 Spatial feature map sets for each cycle 2532 Dimensional Conformity Logic 2536 Bus 2538 Spatial feature map sets for each cycle 2585 ​​Residual Blocks 2586 preceding outputs 2587 Final Total Output 2589 Skip Connection 2590 First spatial convolution layer 2591 First Output 2592 Second spatial convolution layer 2593 Second Output 2594 adder 2595 summed output 2602 Bus 2604 Bus 2702 Bus 2712 Bus 2902 Temporal Convolutional Layer of Temporal Network 160 2912 Temporal Network 160 Temporal Convolutional Layer 2922 Temporal Network 160 Temporal Convolutional Layer 2932 Temporal Network 160 Temporal Convolutional Layer 2942 Temporal Network 160 Temporal Convolutional Layer 2952 Temporal Network 160 Temporal Convolutional Layer 2962 Temporal Network 160 Temporal Convolutional Layer 2972 Temporal Network 160 Temporal Convolutional Layer 3300 Computer Systems 3310 Storage Subsystem 3322 Memory Subsystem 3332 RAM 3334 ROM 3336 File Storage Subsystem 3338 User Interface Input Devices 3355 Bus Subsystem 3372 CPU 3374 Network Interface Subsystem 3376 User Interface Output Devices 3378 Deep Learning Processor

Claims

1. 1. A system comprising: a spatial convolutional network configured to process the windows of per-cycle sequence image sets, cycle by cycle, for a series of sequencing cycles of the sequencing run by separately processing each per-cycle sequence image set within the windows of per-cycle sequence image sets through a respective spatial processing pipeline; a bus network connected to the spatial convolutional network and configured to form buses between spatial convolutional layers in each sequence of spatial convolutional layers of the respective spatial processing pipelines, the bus being configured to combine each per-cycle spatial feature map set generated by two or more spatial convolutional layers in a particular sequence of spatial convolutional layers for a particular sequencing cycle into a combined per-cycle spatial feature map set, and to provide the combined per-cycle spatial feature map set as an input to another spatial convolutional layer in the particular sequence of spatial convolutional layers.

2. 2. The system of claim 1, wherein the two or more spatial convolutional layers include a first spatial convolutional layer and a third spatial convolutional layer, wherein the first spatial convolutional layer generates a first set of spatial feature maps per cycle, the first spatial convolutional layer provides the first set of spatial feature maps per cycle as input to a second spatial convolutional layer, the second spatial convolutional layer processes the first set of spatial feature maps per cycle and generates a second set of spatial feature maps per cycle, and the second spatial convolutional layer provides the second set of spatial feature maps per cycle as input to the third spatial convolutional layer, and the third spatial convolutional layer processes the second set of spatial feature maps per cycle and generates a third set of spatial feature maps per cycle.

3. 3. The system of claim 2, wherein the bus is further configured to combine the first spatial feature map set and the third per-cycle spatial feature map set into the combined per-cycle spatial feature map set.

4. 4. The system of claim 3, wherein the another spatial convolutional layer is a fourth spatial convolutional layer that immediately follows the third spatial convolutional layer in the particular sequence of spatial convolutional layers, and the fourth spatial convolutional layer processes the combined set of per-cycle spatial feature maps as an input.

5. 3. The system of claim 2, wherein the two or more spatial convolutional layers include the first spatial convolutional layer and a seventh spatial convolutional layer, wherein the third spatial convolutional layer provides the third set of spatial feature maps per cycle as input to a fourth spatial convolutional layer, which processes the third set of spatial feature maps per cycle to generate a fourth set of spatial feature maps per cycle, and the fourth spatial convolutional layer provides the fourth set of spatial feature maps per cycle as input to a fifth spatial convolutional layer, which processes the fourth set of spatial feature maps per cycle to generate a fifth set of spatial feature maps per cycle.

6. 6. The system of claim 5, wherein the bus is further configured to combine the first spatial feature map set and the fifth spatial feature map set into the combined per-cycle spatial feature map set.

7. 7. The system of claim 6, wherein the another spatial convolutional layer is a sixth spatial convolutional layer that immediately follows the fifth spatial convolutional layer in the particular sequence of spatial convolutional layers, and the sixth spatial convolutional layer processes the combined set of per-cycle spatial feature maps as an input.

8. 6. The system of claim 5, wherein the two or more spatial convolutional layers include the first spatial convolutional layer, the third spatial convolutional layer, and the fifth spatial convolutional layer, and the bus is further configured to combine the first per cycle spatial feature map set, the third per cycle spatial feature map set, and the fifth per cycle spatial feature map set into the combined per cycle spatial feature map set.

9. 9. The system of claim 8, wherein the another spatial convolutional layer is the sixth spatial convolutional layer that processes the combined set of per-cycle spatial feature maps as input.

10. 2. The system of claim 1, wherein the bus is further configured to combine the per-cycle sequence image set for the particular sequencing cycle provided as input to the first spatial convolutional layer with the third per-cycle spatial feature map set into the combined per-cycle spatial feature map set.

11. 11. The system of claim 10, wherein the another spatial convolutional layer is the fourth spatial convolutional layer that processes the combined set of per-cycle spatial feature maps as input.

12. 2. The system of claim 1, wherein the bus network is further configured to include dimensional compatibility logic configured to modify spatial and depth dimensions of an incoming per-cycle spatial feature map set to be combined with a receiving per-cycle spatial feature map set to generate the combined per-cycle spatial feature map set.

13. The system of claim 12 , wherein the dimensionality compatibility logic is a dimensionality reduction operation including convolution, pooling, or averaging.

14. 13. The system of claim 12, wherein the bus network is further configured to include scaling logic configured to scale feature values ​​of the incoming per-cycle spatial feature map set to be combined with the receiving per-cycle spatial feature map set to generate the combined per-cycle spatial feature map set.

15. 2. The system of claim 1, further configured with a temporal convolutional network configured to process the per-cycle spatial feature map sets group-wise by convolving each overlapping group of per-cycle spatial feature map sets in the per-cycle spatial feature map sets with a respective temporal convolutional filter bank of a first temporal convolutional layer to generate a respective group-wise temporal feature map set for each overlapping group of per-cycle spatial feature map sets.

16. 16. The system of claim 15, further comprising: a bus network connected to the temporal convolutional network and configured to form a bus between temporal convolutional layers in each sequence of temporal convolutional layers, the bus configured to combine each per-cycle temporal feature map set generated by two or more temporal convolutional layers in a particular sequence of temporal convolutional layers for a particular sequencing cycle into a combined per-cycle temporal feature map set, and to provide the combined per-cycle temporal feature map set as an input to another temporal convolutional layer in the particular sequence of temporal convolutional layers.

17. 1. An artificial intelligence based method, comprising: processing the per-cycle sequence image set windows for a series of sequencing cycles of the sequencing run through a spatial convolutional network by separately processing each per-cycle sequence image set within the per-cycle sequence image set window through each sequence of spatial convolutional layers; combining respective per-cycle spatial feature map sets generated by two or more spatial convolutional layers in a particular sequence of spatial convolutional layers for a particular sequencing cycle into a combined per-cycle spatial feature map set, and providing the combined per-cycle spatial feature map set as an input to another spatial convolutional layer in the particular sequence of spatial convolutional layers.

18. 18. The artificial intelligence-based method of claim 17, wherein the two or more spatial convolutional layers comprise a first spatial convolutional layer and a third spatial convolutional layer, wherein the first spatial convolutional layer generates a first set of spatial feature maps per cycle, the first spatial convolutional layer provides the first set of spatial feature maps per cycle as input to a second spatial convolutional layer, the second spatial convolutional layer processes the first set of spatial feature maps per cycle and generates a second set of spatial feature maps per cycle, the second spatial convolutional layer provides the second set of spatial feature maps per cycle as input to the third spatial convolutional layer, and the third spatial convolutional layer processes the second set of spatial feature maps per cycle and generates a third set of spatial feature maps per cycle.

19. 20. The artificial intelligence based method of claim 18, wherein the bus is further configured to combine the first spatial feature map set and the third per-cycle spatial feature map set into the combined per-cycle spatial feature map set.

20. 20. The artificial intelligence based method of claim 19, wherein the another spatial convolutional layer is a fourth spatial convolutional layer that immediately follows the third spatial convolutional layer in the particular sequence of spatial convolutional layers, and the fourth spatial convolutional layer processes the combined set of per-cycle spatial feature maps as input.

Images