Efficient execution of machine learning models on dedicated hardware

Abstract

Systems and methods of executing a machine learning model on a dedicated computing device can include obtaining, by a first computing device, raw input data; obtaining the machine learning model comprising a function that applies a set of M model parameters to at least one channel of the raw input data; determining a configuration parameter K for the dedicated computing device; configuring the original input data based on configuration parameters to obtain configured input data; configuring the machine learning model based on the configuration parameters to obtain a configured machine learning model, the configured machine learning model having a configured model dimension corresponding to a data size of an acceleration path; executing the configured machine learning model with configured model parameters using the configured input data to obtain output data; and providing the output data.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 453,827, filed March 22, 2023, the contents of which are incorporated herein by reference in their entirety for all purposes. Background Technology

[0003] Machine learning, particularly deep learning and artificial neural networks (ANNs), has become increasingly useful for big data analytics and data-driven decision-making in modern scientific research and industrial applications. These ANNs have been of great help in classification and prediction across many disciplines, such as computer science, electrical engineering, and biology. ANN models, including convolutional neural network (CNN) models, are often trained in ways that restrict the adaptability of the trained model to different user needs. For example, it is often difficult to utilize the acceleration paths of dedicated hardware, such as the tensor cores of a graphics processing unit (GPU), to execute trained models.

[0004] Some existing solutions involve adding extra layers to the CNN model, creating a memory copy with spacing, or retraining the CNN model with specific requirements. However, these solutions increase computational and memory demands, reduce computational speed and efficiency, and are impractical in many cases. Summary of the Invention

[0005] This disclosure generally relates to executing machine learning models on a dedicated computing device, and more specifically, to embodiments that can be configured with data of various sizes and dimensions, as well as various types of models, to suit execution on various acceleration paths of a dedicated computing device. For example, some embodiments may reshape data and replicate or readjust functions (e.g., filters of a CNN model) based on requirements regarding the use of the acceleration path of the dedicated computing device. Various techniques can be used to configure data and models so that a configured model with configured data can be executed on the acceleration path of a dedicated computing device, and computational efficiency can be achieved.

[0006] These and other embodiments of this disclosure are described in detail below. For example, other embodiments relate to systems, apparatuses, and computer-readable media associated with the methods described herein.

[0007] The nature and advantages of embodiments of this disclosure can be better understood by referring to the following detailed description and accompanying drawings. Attached Figure Description

[0008] Figure 1 shows a block diagram of an example system 100 according to various embodiments of the present invention, which is used to acquire data and configure data and machine learning models for computation, prediction and classification in a dedicated computing device.

[0009] Figure 2 shows flowchart 200, which illustrates example methods for configuring raw input data and machine learning models according to various embodiments of the present invention.

[0010] Figure 3A shows exemplary raw sequencing data obtained at box 210 in Figure 2 according to certain embodiments.

[0011] Figure 3B shows exemplary preprocessed sequencing data according to certain embodiments.

[0012] Figure 4A shows exemplary raw sequencing data with a filter featuring a CNN model according to certain embodiments.

[0013] Figure 4B shows exemplary three-dimensional raw input data for a filter with a CNN model according to certain embodiments.

[0014] Figure 5 illustrates an example of convolving the original input data using channels of filters in a convolutional layer of a CNN model according to certain embodiments.

[0015] Figure 6 illustrates an example of convolving the original input data using multiple filters in the convolutional layer of a CNN model according to certain embodiments.

[0016] Figure 7A illustrates an exemplary visualization of raw 3D input data configured to be executed on an accelerated path of a GPU, according to certain embodiments.

[0017] Figure 7B illustrates another exemplary visualization of configuring raw 3D input data to be suitable for execution on an accelerated path of a GPU, according to certain embodiments.

[0018] Figure 8 shows three examples of configuring raw input data according to various implementation methods.

[0019] Figures 9A and 9B illustrate two exemplary ways, according to certain embodiments, to configure filters in a CNN model using configured input data to execute on an accelerated path of an 8-channel GPU.

[0020] Figures 10 and 11 illustrate, according to certain embodiments, the exemplary execution of a configured model on configured data.

[0021] Figure 12 shows an example of a physical computing environment according to certain embodiments.

[0022] Figure 13 shows a block diagram of an example computer system that can be used with systems and methods according to an embodiment of the present invention. Detailed Implementation

[0023] The techniques disclosed herein relate to the automatic transformation and analysis of raw input data (including sequencing data generated from sequencing devices) to suit various machine learning models and specialized hardware for efficient computation, prediction, and classification. Different sequencing devices can generate raw sequencing data, which can be preprocessed to provide raw input data for further analysis in machine learning models. Both the raw input data and the machine learning model can be configured in specialized hardware with acceleration paths. To utilize the acceleration paths of specialized hardware (e.g., tensor cores of a graphics processing unit (GPU)), both the raw input data and the machine learning model require specific configurations. However, machine learning models are typically trained on conventional computing systems without any specialized hardware or any configuration that does not consider specialized hardware.

[0024] To address this problem, the embodiments described herein provide methods and techniques for configuring raw input data and machine learning models to execute machine learning models on an accelerated path of a dedicated computing device. In some cases, the data is configured to have a desired number of dimensions and functions (e.g., filters in a CNN model). For example, functions can be copied and refitted to fit the dimensionality based on requirements regarding the use of an accelerated path on a dedicated computing device. Various techniques can be used to configure data and models so that a configured model with configured data can be executed on an accelerated path of a dedicated computing device, and computational efficiency can be achieved.

[0025] I. Deep learning networks and dedicated computing devices

[0026] Machine learning is a key concept in the field of artificial intelligence and has been used and developed in various industries such as biotechnology and pharmaceuticals. Deep learning, a subfield of machine learning (which classifies models through multiple layers or levels), is becoming increasingly popular in providing useful and accurate classification information in biotechnology and pharmaceuticals. Deep learning models are typically trained using neural network architectures such as artificial neural networks (ANNs) or convolutional neural networks (CNNs). Different information is extracted through different layers in such neural networks and combined for prediction or classification. For example, image data can be used to train deep learning models to predict the location of objects in images. Sequencing pulse (signal) data can also be used to train these models to improve the accuracy of base calls during sequencing.

[0027] However, running machine learning (ML) models can be computationally expensive. For this reason, dedicated hardware has been developed to execute such models. For example, a graphics processing unit (GPU) can be used to efficiently execute machine learning models. However, even with dedicated hardware, large datasets can require ML models to run for extended periods. This can be especially true when dedicated hardware cannot perform optimally. The embodiments described herein can rearrange data and ML models to operate more efficiently, for example, by utilizing acceleration paths in a more consistent manner. Several example ML models and examples of dedicated hardware are described below.

[0028] A. Convolutional Neural Networks and Other Deep Learning Networks

[0029] CNNs are commonly used deep learning models when the input data is image or signal data and the output is a classification or prediction of the image or signal data. CNNs are useful and popular in the fields of biology and biotechnology, in part because they are inspired and designed to resemble neurons interacting within biological systems. A typical CNN consists of an input layer, multiple hidden layers that perform convolutions, and an output layer.

[0030] Filters (or kernels) are a key concept in CNNs. In a CNN model, input data, including image or signal data, is typically transformed into matrices. Similarly, filters in a CNN model are matrices of a certain size. Sometimes, filters are 3x3, 5x5, 7x7, 1x3, 1x5, or 1x7 matrices. In these cases, the filter is a two-dimensional filter (2D filter). In some cases, the filter can have more than two dimensions. For example, filters for RGB image input are often three-dimensional. Filters in a CNN model help extract specific features from the input data, such as signal peaks or vertical edges in an image. The basic mechanism for feature extraction from filters is to overlap the filter matrix with the input matrix, multiply the overlapping entries, and sum all the multiplications to obtain a new value.

[0031] The processes of overlapping, multiplying, and adding are repeated through the input matrix by moving the filter matrix based on a predetermined stride to produce a feature matrix. The output (feature matrix) of one layer in a CNN model forms the basis of the input to the next layer in the CNN model. The training process of a CNN model learns the value of each entry in the filter matrix, and the filters in each layer become the parameters of the CNN model. With the help of filters, CNN models are able to perform complex classification and prediction tasks. Below are some examples of CNN models (or CNN architectures) that researchers or industry practitioners can use in data training and model classification. Many ML models, including the CNN models below, are applicable to the methods and systems disclosed in this paper.

[0032] Residual Neural Networks (ResNet) are one of the most commonly used CNN architectures. Research has found that traditional, deeper CNN models lead to higher training error rates and overfitting compared to shallower CNN models. ResNet addresses this problem by employing residual blocks and skip connections to skip some layers and avoid overestimation. Typical ResNet models are implemented using two or three skip connections.

[0033] GoogLeNet is a 22-layer (27 layers total, including pooling layers) CNN architecture designed for classification tasks. The GoogLeNet model achieves a significantly lower error rate and utilizes a variety of techniques, including 1x1 convolutions and global average pooling, to create a deeper architecture. Therefore, the GoogLeNet architecture is relatively computationally expensive. To reduce the number of necessary parameters, the GoogLeNet model uses a large number of non-pooling layers on top of a regular CNN to eliminate spatial redundancy during training.

[0034] LeNet is a representative of early CNN architectures. The LeNet architecture typically consists of multiple convolutional and pooling layers, followed by one or more fully connected layers. For example, a typical LeNet-5 model has seven layers: two convolutional layers, two pooling layers, and a dense block consisting of three fully connected layers.

[0035] Deep learning networks have wide applications in many fields, such as automatic speech recognition, image recognition, natural language processing, drug discovery and toxicology, medical image analysis, and bioinformatics. CNN models are not the only techniques that will be applied to these fields; other ANN models, such as deep neural network (DNN) models and recurrent neural network (RNN) models, can also be deployed to solve problems in these areas. They are also applicable to the techniques described in this paper.

[0036] B. Dedicated computing device

[0037] Traditionally, deep learning network models, including CNN models, are executed on general-purpose computing devices such as central processing units (CPUs). However, executing deep learning network models on CPUs can be computationally intensive, time-consuming, and expensive. The current trend is to use dedicated computing devices, such as graphics processing units (GPUs) or specialized neural processing units (NPUs), to execute trained deep learning network models.

[0038] Graphics Processing Units (GPUs) are dedicated processors used to accelerate graphics rendering and other graphics computations. They are commonly used in computer systems to improve the performance of applications that require complex graphics processing, such as video games, 3D modeling software, and scientific simulations. Due to their highly parallel architecture and dedicated hardware for graphics rendering, GPUs are considered particularly well-suited for executing deep learning network models.

[0039] Many modern GPUs include tensor cores, which are dedicated units designed to efficiently perform tensor computations, such as matrix multiplication. Tensor cores can significantly improve the performance of executing deep learning network models. However, many tensor cores have specific prerequisites regarding the size and dimensions of the input data. When executing a trained deep learning network model in a GPU, it may not be able to fully utilize the acceleration path or the GPU's tensor cores, and therefore may not achieve its optimal performance. For example, ResNet models are typically trained with three-channel input data, while tensor cores in some GPUs require 8 or 16 input channels. When the input data size does not meet the prerequisites, the tensor core is not used, and execution will fall back to different cores that do not perform matrix multiplication faster. The embodiments described herein provide methods and techniques for configuring input data and deep learning network models to suit execution on dedicated computing devices.

[0040] II. Measurement and analysis using dedicated computing devices.

[0041] Data and ML models can be configured and executed in many different ways within a dedicated computing device. For example, in various embodiments, data is generated by a data generation device such as a sequencer, collected by a data collection unit, and preprocessed by a data preprocessing unit. The preprocessed data can be configured according to configuration parameters dependent on the dedicated computing device, such as the required number of input channels, which is a multiple of the configuration parameters, e.g., 8. The model can be collected by a model collection unit and configured according to the same configuration parameters associated with the dedicated computing device. The configured data and the configured model can be executed by the dedicated computing device through its accelerated path, and the output is provided by an output unit. Many different ways can also exist to configure data and models and execute these ways by the dedicated computing device through its accelerated path.

[0042] Figure 1 illustrates a block diagram of an example system 100 according to various embodiments of the present invention, which is used to acquire data and configure data and machine learning models for computation, prediction, and classification on a dedicated computing device. Any unit of system 100 may be a personal computer or part of a personal computer, such as a CPU or GPU, or a unit as part of a network-based server. In some cases, the data collection unit 110, data preprocessing unit 120, data configuration unit 130, model collection unit 140, model configuration unit 150, and output unit 180 may be integrated on the same computer or different computers. In some cases, the units of system 100 may be integrated on the same dedicated computing device 160.

[0043] Optional block 105 can be a data generation device (such as a sequencing device) to generate raw data. When the data generation device 105 is a sequencing device, it can be a Sanger sequencer, a 454 DNA sequencer, a next-generation sequencer, a fluorescence microscopy sequencing device, a hydrogen ion measurement-based sequencing device, a nanopore-based sequencing device, etc. In some cases, the data generation device 105 includes sensors. For example, a nanopore-based sequencing device can be a collection of analog circuits constituting different surface locations, pores, or cells. In some cases, the data generation device 105 includes one or more photon sensing devices, such as a camera. LiDAR, sonar, and radar measurement devices can also be used as the data generation device 105.

[0044] The generated raw data can be acquired by the data collection unit 110. In some cases, the data collection unit 110 collects raw input data from a database. The collection function can be performed by user input or according to a program stored in local memory 115. The raw input data collected by the data collection unit 110 can be sequencing data or image data. In some embodiments, the raw input data can be stored in local memory 115. Local memory 115 can be the same memory as in blocks 125, 135, 155, or 165.

[0045] The generated raw data acquired by data collection unit 110 can be preprocessed by data preprocessing unit 120 before any configuration. In some cases, data preprocessing unit 120 may be the same as data collection unit 110. The preprocessing process can be implemented according to a program in local memory 125; alternatively, the preprocessing process can be performed on a network-based server. In some embodiments, local memory 125 may be the same memory as in blocks 115, 135, 155, or 165.

[0046] In some cases, the generated raw data can be normalized. Normalization can be performed channel-based or across channels. For example, in the case of fluorescence microscopy sequencing, a specific wavelength of dye that excites the fluorophore attached to DNA nucleotides can be used to preprocess the data. In some cases, normalization is time-based (e.g., flattening). For example, when using electronic circuitry to collect signals, capacitive components may be employed that can eventually saturate and skew the input signal over time. Time-based normalization may be ideal to compensate for this "gain drift" and re-flatten the signal. In some cases, preprocessing includes data aggregation. Aggregating data points over time may be ideal to reduce the overall input data rate. Various aggregation methods (such as minimum, maximum, average, weighted average, Kalman filter) can be used to eliminate noise or spikes in the input signal.

[0047] Typically, a dedicated computing device 160 is selected before configuring the data. In many cases, data and models need to be configured based on the dedicated computing device 160. The dedicated computing device 160 has at least one acceleration path that can perform computations, predictions, or classifications using the configured model and data. Information about the dedicated computing device 160, including configuration parameters for the acceleration path, can be obtained by the dedicated computing device 160 and can be stored in memory 165. This information can be used to configure data and models in blocks 130 and 150. The information about the dedicated computing device 160 includes configuration parameters corresponding to the size of the data that enables the acceleration path of the dedicated computing device to run. For example, the dedicated computing device 160 may be a graphics processing unit (GPU) with at least one tensor core for faster matrix multiplication, and the configuration parameter may be determined to be 8, which is the number of input channels required by the tensor core. Memory 165 may also be used to store and process data and models. Memory 165 may be GPU memory. In some embodiments, memory 165 may be the same memory as in blocks 115, 125, 135, or 155.

[0048] Data acquired by data collection unit 110 or preprocessed by data preprocessing unit 120 is configured by data configuration unit 130 based on information about dedicated computing device 160, including configuration parameters. For example, data acquired by data collection unit 110 may be image data with three channels, and the configuration parameter may be equal to 8. Therefore, the acquired data needs to be configured by data configuration unit 130 to have 8 channels, or a number of channels that is a multiple of 8. In some cases, data configuration unit 130 may be the same as data collection unit 110 or data preprocessing unit 120. Information about dedicated computing device 160 may be sent to data configuration unit 130 via bus 170; alternatively, this information may be pre-acquired or determined by general-purpose computing device and stored in local memory 135. Data is configured in data configuration unit 130 according to a program in local memory 135. In some embodiments, local memory 135 may be the same memory as in blocks 115, 125, 155, or 165.

[0049] The model collection unit 140 can acquire one or more machine learning models. These one or more machine learning models are trained models that can be used in a computing device. The machine learning models can be deep learning models, artificial neural network (ANN) models including convolutional neural network (CNN) models, or any other suitable model that can be used in data analysis, computation, prediction, or classification. In some cases, the model collection unit 140 can also perform model generation functions. Machine learning models can be generated using the techniques shown below.

[0050] For example, sequencing data can be used to train a CNN model to predict base calls in nucleotide sequences. Input sequencing data can be single-channel sequencing data obtained using nanopore sequencing technology (e.g., sequencing by monitoring changes in current as nucleic acids pass through protein nanopores) or by reading nucleotide sequences using pH measurements (e.g., ion-fluidic sequencing), dual-channel sequencing data obtained using two-channel sequencing-by-synthesis (SBS) technology (e.g., Illumina's 2-channel SBS technology), or quad-channel sequencing data using Illumina's 4-channel SBS technology. In some examples, a channel can correspond to voltage or current. 2-channel and 4-channel sequencing can use different filters to detect different colors of dye for different nucleotides. Template nucleic acid molecules can be used, making the sequence known.

[0051] Sequencing data of template nucleic acid molecules can be generated using a sequencing device and used as input in training a CNN model, with the sequence of the template nucleic acid molecule serving as a label during training. For training purposes, the dataset of sequencing data and its corresponding sequences can be divided into (i) a training set, (ii) a test set, and / or (iii) a validation set. For some training, more than one set in the training / test / validation sets is required. For example, for three loops in the training process, at least three training sets and three test sets can be used, each set being different from the others. The disclosed methods and techniques are also applicable to data with varying numbers of channels. For example, the disclosed methods and techniques can be used to configure or train image data with three input channels.

[0052] Some features or hyperparameters of a CNN model can be predetermined. Examples of such hyperparameters include the number of convolutional layers, the number of pooling layers, the number of fully connected layers, the number of neurons per layer, the size of the filters per layer, the stride, and / or the learning rate. The training set is then fed into the CNN model, and the performance of the trained CNN model is tested using a test set. If the target performance is not achieved, a second training loop can be performed, in which it is possible to readjust the hyperparameters of the trained CNN model.

[0053] As another example, other (e.g., more time-consuming) techniques can be used to determine a human genome, such as those used to determine a reference sequence (e.g., the genome of a specific chromosomal region). This sequence can then be determined by aligning the sequence of a specific nucleic acid molecule to the reference sequence. The resulting sequence can be used as a label for supervised learning. A CNN model can then be optimized using preset criteria, where the trained CNN model can predict base calls, for example, using single-channel, dual-channel, or quad-channel sequencing data, or other sequencing data with some other number of channels. In some cases, similar functionality can be achieved using non-neural network models or ensembles of models including trained neural network models, and the disclosed methods and techniques are applicable to non-neural network models or the entire ensemble of models.

[0054] In some cases, the model collection unit 140 may be the same as the data collection unit 110, the data preprocessing unit 120, or the data configuration unit 130. The model collection unit 140 may also perform a model selection function. In some cases, all acquired machine learning models are selected by the model collection unit 140 and sent to the model configuration unit 150 for further configuration. In other cases, the model collection unit does not select at least one of the acquired machine learning models. The model collection step in the model collection unit 140 may be performed before, after, or concurrently with the data collection step in the data collection unit 110. Acquired or selected machine learning models can be automatically acquired or selected by a program, manually acquired or selected by operation, or interactively acquired or selected through a user interface.

[0055] Model configuration unit 150 configures one or more selected models based on information about dedicated computing device 160. For example, a CNN model selected by model collection unit 140 might be suitable for predicting base calls using single-channel sequencing data, while dedicated computing device 160 is a GPU with tensor cores requiring eight input channels. In this case, the CNN model would need to be configured by model configuration unit 150 to accept eight-channel input data so that the CNN model can execute on the tensor cores of the GPU. In some cases, model configuration unit 150 may be the same as data collection unit 110, data preprocessing unit 120, data configuration unit 130, or model collection unit 140. Information about dedicated computing device 160 (such as in the above example where eight input channels are required for the data) can be sent to model configuration unit 150 via bus 170; alternatively, this information can be pre-acquired or determined by a general-purpose computing device and stored in local memory 155. The machine learning model is configured in model configuration unit 150 according to the program in local memory 155. In some embodiments, local memory 155 may be the same memory as in blocks 115, 125, 135, or 165.

[0056] The configured model and data are transmitted via bus 170 to a dedicated computing device 160 for performing computations, predictions, or classifications. The dedicated computing device 160 has at least one acceleration path that can use the configured model and data to perform computations, predictions, or classifications, and information about the dedicated computing device 160, including the acceleration path, is used to configure the raw input data and model in blocks 130 and 150. The information about the dedicated computing device 160 includes configuration parameters corresponding to the data size that enables the acceleration path of the dedicated computing device to run. For example, the dedicated computing device 160 may be a graphics processing unit (GPU) with at least one tensor core for faster matrix multiplication, and the configuration parameter may be determined to be 8, which is the number of input channels required by the tensor core. The dedicated computing device 160 also has a memory 165 for storing and processing the data and model. The memory 165 may be GPU memory. In some embodiments, the memory 165 may be the same memory as in blocks 115, 125, 135, or 155.

[0057] The output of the configured model using the configured data is obtained by a dedicated computing device 160 with memory 165 and sent to the output unit 180 via bus 170. In some cases, the output unit 180 may be the same as the data collection unit 110, data preprocessing unit 120, data configuration unit 130, model collection unit 140, model configuration unit 150, or dedicated computing device 160. The output can be provided automatically by a program, manually by operation, or interactively through a user interface.

[0058] III. Change the dimensions of the input data for the acceleration path

[0059] To address the incompatibility between trained models and input data and the prerequisites required to implement accelerated paths on dedicated computing devices, embodiments described herein disclose methods and systems for configuring model parameters and input data to be executed on different dedicated computing devices. The solution to the problem depends on the type of dedicated computing device and the type of deep learning model.

[0060] Figure 2 shows flowchart 200, which illustrates example methods for configuring raw input data and machine learning models according to various embodiments of the present invention.

[0061] At box 210, the raw input data is obtained. The raw input data can be sequencing data generated by data generation device 105 (e.g., a sequencing device, such as a nanopore device), as shown in Figure 1. The raw input data can also be other image data generated by an optical device. At box 210, the dimension set of the raw input data is also obtained. The dimension set can include a channel dimension with a number of C channels and a first length dimension corresponding to the height or width of the raw input data. In some cases, the dimension set may also include a batch number dimension with N batches.

[0062] A. Selection of Deep Learning Models

[0063] At box 220, a machine learning model with a function having a set of M model parameters is obtained. These M model parameters can be applied to at least one channel of the original input data. The machine learning model can be a deep learning network model. Deep learning models (deep learning network models) are typically used when dedicated computing power is required to achieve better performance. In some cases, machine learning models other than deep learning models can also be used. The term "machine learning model" as used in this paper can refer to a deep learning model, and vice versa.

[0064] Obtaining a machine learning model with a function may further include selecting one or more machine learning models from a model database, as described in more detail below. In some cases, a deep learning model is selected. The selection of a deep learning model may be based on research or business needs. The deep learning model may be predetermined and acquired by the model collection unit 140 in system 100 of Figure 1. This selection may also be based on specific needs and readjusted during or partially during execution. In some cases, the selection may be automatic or random. The selection may be adjusted interactively through a user interface. In some cases, obtaining a machine learning model with a function at box 220 is based on this selection. In some cases, the obtained machine learning model is a CNN model, and its function is a filter of the first layer of the CNN model.

[0065] B. Selection of Dedicated Computing Devices

[0066] The type of dedicated computing device determines how the machine learning model and raw input data are configured based on information about the dedicated computing device, including prerequisites for acceleration paths. This information is sometimes referred to as configuration parameters. For example, a GPU with Type I tensor cores may require an input channel size that is a multiple of 8, while another GPU with Type II tensor cores may require an input channel size that is a multiple of 16. In the first case, the configuration parameter is 8, and in the second case, it is 16. When implementing a machine learning model to be executed on a GPU with Type I tensor cores, the raw input data and the model need to be configured with an input channel size of 8. If the configuration requirement is not met, Type I tensor cores will not be fed, and the GPU will use non-accelerated cores for model execution. In this case, the computational efficiency is considered to be 4 to 8 times slower than that of tensor cores (MAndersch et al., Tensor Core DL Performance Guide ).

[0067] The selection of a dedicated computing device can be based on research or commercial needs. This selection can be predetermined and set as an input or default in the system shown in Figure 1. The selection can also be based on specific needs and readjusted during or partially during execution. In some cases, selection can be made automatically or randomly based on a hardware inventory. The selection can be adjusted interactively through a user interface.

[0068] At box 230, the configuration parameter K for the dedicated computing device is determined in parallel with the selection of the dedicated computing device. The configuration parameter K can be predetermined, set as an input or default in the system of Figure 1, either simultaneously with, before, or after the selection of the dedicated computing device. The configuration parameter K can be determined by the same computing device that obtained the raw input data. The configuration parameter K corresponds to the data size for which the acceleration path of the dedicated computing device is operational. Preferably, the selection of the dedicated computing device is first considered in relation to the optimal performance of the machine learning model, and then the configuration parameter K is determined based on the selection of the dedicated computing device. For example, when a GPU with tensor cores is selected as the dedicated computing device, the corresponding configuration parameter K can be determined to be 8. In some cases, the order can be reversed to avoid over-configuration regarding the selected machine learning model. The configuration parameter K can also be determined automatically or semi-automatically by the dedicated computing device, or alternatively, the configuration parameter K can be determined interactively through a user interface.

[0069] C. Data preprocessing and data configuration

[0070] At box 240, the raw input data can be directly configured based on the configuration parameter K determined at box 230. The configuration of the raw input data depends on the size and dimensions of the raw input data, as well as the size and dimensions of the configuration parameter K. The configuration of the raw input data includes scaling the channel dimension C using the configuration parameter K, and inversely scaling the first length dimension using the configuration parameter K, resulting in K x C channels. For example, when a GPU is selected as the dedicated computing device and the corresponding configuration parameter K is determined to be 8, if the raw input data is an RGB image with three channels (red, green, and blue), the raw input data is configured by reshaping each channel of the raw input data into 8 sub-channels (8 red sub-channels, 8 green sub-channels, and 8 blue sub-channels). This example can be used for image analysis.

[0071] Reshaping can be performed by dividing the data in each channel into 8 groups along the width of the original input data. In some cases, it can be performed by dividing the data in each channel into 8 groups along the height of the original input data. In some cases, it can be performed by dividing the data in each channel into 8 groups along dimensions other than the width and height of the original input data. In some embodiments, data preprocessing and configuration can be performed based on dimensions other than the channel dimension. The same or substantially similar methods can be performed based on any dimension of the original input data. For ease of expression, the dimension to which configuration applies is referred to as "channel" or "channel dimension".

[0072] The configuration of the raw input data does not need to be strictly followed, as the number of sub-channels is equal to the value of the configuration parameter K. In some cases, the raw input data is configured as a subset of data, and the number of subsets is equal to a multiple of the value of the configuration parameter K. For example, when K equals 8, the raw input data can be configured by reshaping each channel of the raw input data into any multiple of 16, 24, 32, or 8 sub-channels.

[0073] Sometimes, the raw input data is preprocessed before box 240. Sometimes, preprocessing of the raw input data is part of the configuration at box 240. The preprocessing of the raw input data depends on its size and quality. For example, if the raw input data is a batch of images of different sizes, preprocessing can resize the batch to the same size. The uniform size can be predetermined or determined based on the configuration parameter K. For example, if the number of pixels in the dimension to be reshaped is not a multiple of the value of the configuration parameter K, additional pixels with a value of 0 can be added to the raw input data to expand the number of pixels in the dimension to be reshaped to a multiple of 8. Other suitable methods can also be used to preprocess the raw input data.

[0074] D. Configuration of deep learning models

[0075] At box 250, the obtained machine learning model with a function is configured based on the configuration parameter K. The configuration of the model depends on the type of model. This configuration includes expanding the function to include at least K x M model parameters applied to at least K channels. The configuration of the model can be a configuration of the model parameters. In some cases, the configuration of the model can be a configuration of a subset of the model parameters. For example, when the obtained machine learning model is a CNN model with filters for different convolutional layers, the configuration of the CNN model can be a configuration of the filters for the first convolutional layer. In some cases, the disclosed techniques and methods can be applied to the configuration of the model with respect to the filters of the intermediate layers (e.g., any layer between the input and output layers) or the output layer of the deep learning model.

[0076] Filter configuration can include extended filters. If the original filter has M model parameters, the extended filter has at least K x M parameters. The extended filter can be applied to at least K channels. In some cases, filter configuration includes extending the filter into a sparse filter. In a sparse filter, each entry on the diagonal of one dimension corresponds to the original filter, and all other entries have zero values ​​(e.g., a Toeplitz matrix).

[0077] E. Execution of configured models and data

[0078] At box 260, the machine learning model configured in box 250 and the raw data configured in box 240 are sent to and executed by a dedicated computing device for computation, classification, and prediction. Due to the configuration occurring at boxes 240 and 250, computation, classification, and prediction can utilize the accelerated path of the dedicated computing device, and are computationally more efficient than on the non-accelerated path of a general-purpose computing device or a dedicated computing device. In some cases, computation, classification, and prediction performed on the accelerated path of the dedicated computing device also improves energy efficiency.

[0079] F. Output, Iteration, and Correction

[0080] At box 270, the output is provided by a dedicated computing device. The output of the deep learning model can take many different forms and can be used for further processing, prediction, or diagnostics. In some cases, the configurations at boxes 240 and 250 can be iterated based on research or business needs or based on the type of machine learning model. For example, when the machine learning model is a ResNet model, a second round of configuration of the ResNet model can be performed to configure filters in the second convolutional layer or skip layers. Although a ResNet model can have a multiple of 8 filters in its first convolutional layer, configuration of the model regarding its second convolutional layer can be performed when the number of filters in the first convolutional layer of the ResNet model is not a multiple of 8. In some cases, the configuration can be interactively readjusted through a user interface to achieve optimal performance. In some cases, the user of the user interface can correct the output.

[0081] Other techniques for this configuration include adding additional convolutional layers with the desired input and output channels, copying the input data to a padding buffer, and / or retraining the model with the desired input channels. These methods can be combined with the techniques described above and applied to different environments.

[0082] IV. Example Configuration of Data and Model

[0083] According to various embodiments of the invention, the following examples illustrate how data can be generated, collected, and configured, and how CNN models can be configured based on the requirements of GPU tensor cores. These examples also demonstrate data and model configurations in exemplary physical environments. It should be understood that the examples described herein are not intended to be exclusive, and any suitable methods and systems can be used to perform the same functions as the examples.

[0084] A. 8-channel convolution of the graphics processing unit

[0085] Figures 3 through 11 show examples of data and CNN models configured to be executed by a GPU that requires input data and a model with a number of channels that are multiples of 8 to be executed on its acceleration path (“8-channel GPU”), and the execution of a configured model using the configured data, as discussed in the flowchart of Figure 2.

[0086] 1. The raw input data and filters obtained for the CNN model

[0087] The first step involves obtaining the raw input data (e.g., box 210 in Figure 2) and the CNN model (e.g., box 220 in Figure 2) to be executed by the 8-channel GPU. The raw input data and the CNN model can have various sizes and dimensions. The obtained raw input data can be preprocessed before configuration (e.g., via data preprocessing unit 120 of system 100 in Figure 1). For example, when the raw input data is image data of varying sizes, they may need to be truncated and resized to have the same size in each of their dimensions, thus being configured into a batch. Other situations may exist where preprocessing of the raw input data is required. The CNN models are obtained using information about their filters (e.g., the function at box 220). For each CNN model, there should be at least one filter in each layer of the CNN model, and there should be multiple layers in the CNN model. Filters are matrices of various sizes and dimensions. Since the dimensions of the filters do not always meet the requirements of an 8-channel GPU, the filters also need to be configured before execution by the 8-channel GPU.

[0088] Figure 3A shows exemplary raw sequencing data obtained at box 210 in Figure 2. As can be seen from the figure, the raw sequencing data can be fluorescence signals generated from fluorescently stained nucleic acid material of the sample and has one, two, or four input channels. In this case, the raw input data may be difficult or impractical to process by a machine learning model and can be preprocessed as discussed in data preprocessing unit 120 in Figure 1.

[0089] Figure 3B illustrates preprocessed sequencing data. As an example, preprocessing of colored sequencing data may include denoising, color separation, baseline correction, and / or mobility shift correction. The criteria for these preprocessing functions can be preset, with the aim of preserving information as unbiased as that contained in the original input data. In some cases, preprocessed sequencing data will replace the initially obtained raw sequencing data and be configured and analyzed in subsequent steps.

[0090] Figures 4A and 4B show examples of visualizations of the raw input data and filters for a CNN model. The raw input data obtained at box 210 in Figure 2 can be in NHWC format, where N represents the number of raw input data in a batch (e.g., a batch of N images), H represents the height of the raw input data, W represents the width of the raw input data, and C represents the number of channels of the raw input data. For example, the raw input data has n -take- h -take- w -take- c The dimensions of the original input data. The height of the original input data can be the vertical dimension, while the width can be the horizontal dimension. In some cases, the H, W, and C dimensions can be switched. For example, the height of the original input data can be the horizontal dimension, while the width can be the vertical dimension. An exemplary CNN model is a machine learning model with a function having M model parameters. The function and value of M can depend on the filters of the CNN model. Examples of original input data include images, sequences (e.g., nucleic acid sequences), or signals generated during the sequencing process.

[0091] Figure 4A shows the raw image data 410 of the sample with filter 420 of a CNN model. The raw image data 410 can be generated by a data generation device 105 (such as a photon sensing device). The image captured by the data generation device 105 can be preprocessed and converted into raw sequencing data 410. In some cases, the raw image data 410 can be sequencing data based on fluorescence signals generated by a fluorescence microscopy sequencing device. The raw image data 410 can also be sequencing data generated by a nanopore-based sequencing device. The sequencing data can have a different number of channels than shown in Figure 4A. For example, sequencing data can have one, two, or four channels. The techniques, methods, systems, and examples disclosed herein are suitable for and can be applied to data with different dimensions. For ease of illustration, the examples discussed herein use data with three channels.

[0092] As shown in Figure 4A, the original image data 410 can be RGB image data. In this case, the number of input channels for the original image data 410 is 3. As mentioned above, the number of input channels can be different from 3; for example, the number of input channels can be 1, 2, or 4. Each unit of the original image data 410 can represent one pixel of the image data. As shown in Figure 4A, the NHWC format of this original image data 410 is 1-multi-1-multi- w - multiplied by -3, where wThe height is determined by the obtained image width. The height of the original image data 410 is shown as 1, and this height can be a number other than 1. When the height is 1, the number of dimensions can be identified as 2 (width dimension and channel dimension). When the height is not 1, the number of dimensions can be identified as 3 (height, width, and channel dimension).

[0093] In the example of Figure 4A, filter 420 has a size of 1 x 3 x 3. Each channel of filter 420 is adapted to determine a specific character of the same channel of image data 410. For example, filter f The 420p red channel can be used to determine the probability that a signal is red. (Filter) f The size of filter 420 can vary depending on the classification needs. When omitting the channel dimension, common filter sizes for sequencing data are 1 x 3, 1 x 5, and 1 x 7. The number of filter dimensions is not always suitable for computational efficiency when executed by a dedicated computing device using its acceleration path. In this example, filter 420 has three channels, while a GPU with tensor cores might require 8 input channels to utilize its acceleration path. In this case, filter 420, or the corresponding CNN model, needs to be configured with 8 or a multiple of 8 channels.

[0094] Figure 4B shows the raw 3D input data 430 of filter 440 with a CNN model. The raw input data 430 in Figure 4B can represent a 3D image with three channels (RGB). Cell 432 is a cell on the red channel of the raw input data 430. In some cases, each cell can represent more than one pixel of the image. For example, cell 432 in each channel of the raw input data 430 can represent 64 pixels (8 by 8), as shown in Figure 4B. In this case, the H, W, and C dimensions of the raw input data 430 are 64-by-64-by-3. The number 64 is for illustrative purposes only. The actual size or number of pixels of the raw input data can vary based on research or commercial needs. The HW dimensions of the raw input data 430 can be square or non-square rectangles.

[0095] As shown in Figure 4B, the filter f The HWC dimension of 440 is 3 x 3 x 3. The HW dimension of a 440 filter can also be a square or a non-square rectangle. When the channel dimension is omitted, commonly used filter sizes for 3D image data are 3 x 3, 5 x 5, and 7 x 7. fEach channel of 440 can be adapted to determine specific characteristics of the same channel. For example, a filter. f The 440 red channel can be used to detect the horizontal or vertical boundaries of a specific character on the red channel. As shown in the example, the filter... f 440 can also be viewed as a function of a CNN model with 27 model parameters (3 x 3 x 3). As discussed in Figure 4A, the number of dimensions of filter 440 is not always suitable for computational efficiency by dedicated computing devices using their acceleration paths.

[0096] 2. Apply the original input data to the CNN filters.

[0097] Figure 5 illustrates an example of convolving the original input data using the channels of filters in the convolutional layers of a CNN model. Matrix 510 can represent the red channel of cell 432 in Figure 4B, and matrix 520 can represent the red channel of filter 440. Here, matrix 520 can perform the function of detecting vertical boundaries on the red channel. Since matrix 520 is 3 x 3, a 3 x 3 submatrix of matrix 510 will be multiplied by matrix 520. For example, cell 532 in the resulting matrix 532 can be obtained by multiplying submatrix 512 by matrix 520, and cell 534 in the resulting matrix 532 can be obtained by multiplying submatrix 514 by matrix 520. The matrix multiplication performed here is a dot product, which multiplies the cells in the same column and row of each matrix and adds the multiplications of the cells to obtain the result.

[0098] As shown in Figure 5, typically for each channel of the original input data, there exists a corresponding filter channel in the CNN model to detect a specific character in the original input data on that channel. The same consideration should be made when implementing the next step of configuring the data and model; that is, the configuration of the data requires the configuration of the model. In some cases, more than one channel of the filter can correspond to a specific character in the original input data on that channel. In some cases, a three-channel filter is applied to three channels of original input data as a whole, and the resulting data has one channel. In some cases, more than one filter is applied in the convolutional layers of the CNN model, and the resulting data comes from each filter application.

[0099] Figure 6 illustrates an example of convolving the original input data using channels of multiple filters in a convolutional layer of a CNN model. The original input data 610 has three channels, and filters 620 and 640 are two different filters of the same dimension (3x3x3). In this example, each filter is applied to the original input data 610 as a whole, and each filter application yields single-channel result data, as shown in result data 630 and 640. In this case, submatrix 612 is multiplied by filter 620 (and filter 640), and a single value is obtained and recorded in cell 632 (and cell 652) of result data 630 (and result data 640). Result data 630 and 640 can be combined later and are two channels of the combined result data that will be processed by the CNN model in later layers.

[0100] The examples in Figures 5 and 6 illustrate that, in most cases, the number of channels in each filter is the same as the number of channels in the data to be processed by the CNN model. This means that CNN models are typically trained specifically for data with a particular number of input channels, and therefore the filters in the CNN model have a matching number of channels. When the original input data has a mismatched number of channels, either the CNN model cannot process them, or the CNN model must be configured to have a number of channels that matches the number of channels in the original input data.

[0101] 3. Data and model configuration

[0102] Executing CNN models on general-purpose computing devices can be time-consuming. Therefore, the trend in industry and research is to use dedicated computing devices to execute CNN models. In many embodiments, the dedicated computing device is a GPU. In this example, the dedicated computing device is a GPU with tensor cores, which requires 8 input channels. This means that executing standard three-channel RGB data on a CNN model trained on three-channel input data will not take advantage of the fast computation speed of a dedicated computing device. To achieve time and computational efficiency, configuration parameters are required. k To configure both the original input data and the corresponding CNN model. In this case, the configuration parameters of the dedicated computing device... k It was determined to be 8, corresponding to box 230 in Figure 2. Configuration parameters. k = 8 will be used in the following steps for configuring the original input data and the CNN model. For illustrative purposes, the examples shown in Figures 7A and 7B only show the configuration corresponding to the three-dimensional original input data 430, as shown in Figure 4B.

[0103] Figure 7A illustrates an exemplary visualization of the raw 3D input data 705 (raw input data 430 as shown in Figure 4B) configured for execution on an 8-channel GPU. This visualization is obtained as shown in Figure 4B. n -take- h -take- w -take- c (This is the original input data of size 1 x 64 x 64 x 3.) The original input data can then be configured to... n -take- h -take- w / k -take- ck (Here, the dimensions are 1 x 64 x 8 x 24), as shown in the configured input data 710 of Figure 7A.

[0104] In some cases, the configuration of the raw input data (shown in box 240 of Figure 2) is done by reshaping the raw input data 705. For example, the first 8 pixels of each row of the raw input data 705 are retained in the first three channels, and the next 8 pixels of each row of the raw input data 705 are sent to the next three channels (exemplarily shown in the dashed box in Figure 7A), and so on. The configuration by reshaping ensures that the configured input data has a number of channels equal to a multiple of the configuration parameter, which is 8 in this case. After configuration, the number of channels of the configured input data 710 is 24, which is a multiple of 8. Therefore, the input data is configured to take advantage of the GPU's acceleration path. In some cases, the GPU treats every three channels as a whole, and the configured input data makes the new number of channels 8.

[0105] Figure 7B illustrates another exemplary visualization of configuring the raw 3D input data 715 (raw input data 430 as shown in Figure 4B) to be suitable for execution on an 8-channel GPU. This is achieved by obtaining the visualization shown in Figure 4B. n -take- h -take- w -take- c After the raw input data (here is 1 x 64 x 64 x 3) is of size, each channel of the raw input data can be configured as follows: n -take- h -take- w / k (Here, the dimensions are 1 x 64 x 8), as shown in the configured input data 720 of Figure 7B.

[0106] In the example of Figure 7B, the first 8 pixels of each row of the red channel of the original input data 715 are retained in the first channel, and the next 8 pixels of each row of the original input data 715 are sent to the next three channels (shown in the dashed boxes in Figure 7B), and so on. The configuration performed through reshaping ensures that the configured input data has a number of channels equal to a multiple of the configuration parameter, which in this case is 8. After configuration, the number of channels of the configured input data 720 is 24 (the first eight channels correspond to the red channel in the original input data 715, and so on), which is a multiple of 8. Therefore, the input data is configured to utilize the GPU's acceleration path. In some cases, the GPU treats every eight channels as a batch, and each batch of configured input data results in a new number of channels of 8.

[0107] In some cases, the configuration of the original input data includes overlapping reshaping, meaning the configuration of the original input data includes padding. For example, when relevant information might be lost during reshaping, the channels of the configured input data can share the same information across different channels of the configured data. For instance, the information in the last two columns of the first channel of the configured input data can be the same as the information in the first two columns of the fourth channel of the configured input data. The size or repetition of the overlapping information depends on various factors, such as the filter size, information sensitivity, and the performance of the trained CNN model.

[0108] There are at least two padding modes for processing configured data, which can be used to execute a configured model on the configured data. A common mode is effective padding, which requires no additional padding of the configured data and assumes that the configured data can be completely covered by the configured filter. Another common mode is identical padding, which requires the size of the input data to be equal to the size of the output data. In this case, the configured input data is padded according to the size of the configured filter, and all padded values ​​are equal to zero.

[0109] Figure 8 illustrates three examples of configuring raw input data according to various embodiments. Matrix 810 represents the red channel of the exemplary raw input data. When the number of raw channels of the raw input data is doubled based on configuration parameters, each channel of the raw input data can be configured as two new channels. The first approach is to separate the data without padding, as shown in matrices 820 and 830. Matrix 820 is formed using the first four columns of matrix 810, and matrix 830 is formed using the last four columns of matrix 810. The second approach allows overlapping segments, which are padded using a portion of the raw input data, as shown in matrices 840 and 850. Matrix 840 is formed using the first five columns of matrix 810, and matrix 850 is formed using the last five columns of matrix 810. In this case, matrices 840 and 850 share information in the fourth and fifth columns of matrix 810; that is, matrices 840 and 850 overlap. The size of the overlapping information can vary. A third way to configure the raw input data is to use zero values ​​for padding, as shown in matrices 860 and 870. To preserve information on the edges of matrices 860 and 870, padding columns with values ​​equal to 0 are added to matrices 860 and 870. In some cases, more than one padding column can be added, and these values ​​can be different from 0. It should be understood that these three methods are illustrative, not exclusive. Different configuration methods can be used.

[0110] As discussed above, a CNN model can be configured when the number of channels in the configured input data does not match the number of channels in the filters of the trained CNN model. The configuration of the CNN model at box 250 in Figure 2 can be achieved by configuring filters. Filters can be configured by copying and / or readjusting, including resizing. For example, when the original input data is configured with 24 channels, a 3-by-3-by-3 filter in the corresponding CNN model can be configured by copying a three-channel filter to every three channels of a 24-channel filter with a size of 3-by-3-by-24. In this case, there are corresponding channels for the configured filters, making it feasible to execute the CNN model using the configured model.

[0111] Figures 9A and 9B illustrate two exemplary ways to configure filter 912 in a CNN model using configured input data for execution on an 8-channel GPU. The first way to configure filter 912 is to duplicate it in every three new channels. As shown in Figure 9A, a duplicate filter 910 can be configured to copy filter 912 in every three channels. That is, the first three channels of the duplicate filter 910, the next three (4th to 6th) channels of the duplicate filter 910, ..., and the last three (22nd to 24th) channels of the duplicate filter 910 are all identical to filter 912. This duplicate filter 910 can be used when the CNN model is set to perform a dot product on each channel of the filter, rather than performing a dot product on each filter as a whole (e.g., when performing a dot product on each channel of the filter, the resulting internal data has the same number of channels as the filter; conversely, when performing a dot product on each filter as a whole, the resulting internal data has one channel corresponding to each filter).

[0112] Figure 9B illustrates a second way of configuring the filter. In this configuration, a sparse filter is generated that replicates the filter in some of its units. As shown in Figure 9B, there are two ways to generate a sparse filter. The first way is to consider the three-channel filter 922 as a whole and configure it as a sparse filter 920, which replicates filter 922 diagonally, as indicated by the circles on the sparse filter 920. The two dimensions containing the diagonal have the same number, and this number is generally equal to the configuration parameter (here it is 8). The third dimension of the sparse filter 920 is equal to the number of filters in the layer being configured. For example, if configured in the first convolutional layer of a CNN model, and there are eight filters extracting features from the raw input data, then the third dimension of the sparse filter 920 is eight, as shown in Figure 9B. As a general setting for CNN models, the third dimension can be a multiple of 8.

[0113] The second way to configure the filter is by configuring the filter through each channel of the sparse filter 930, which replicates the filter's channel 932 diagonally, as shown by the circles on the sparse filter 930. For example, channel 932 represents the red channel of filter 922. In this sparse filter 930, the two dimensions containing the diagonal also have the same number, and this number is generally equal to the configuration parameter (here it is 8). The third dimension of the sparse filter 930 is equal to the number of filters in the layer being configured. For example, if configured in the first convolutional layer of a CNN model, and there are eight filters extracting features from the original input data, then the third dimension of the sparse filter 920 is eight. As a general setting for CNN models, the third dimension can be a multiple of 8. In this case, there might be three different sparse filters, one for each channel.

[0114] The second inverse configuration of the filters works in almost all cases, especially when the CNN model is set to perform dot multiplication on each filter globally, rather than on each channel of the filter. When executed on an 8-channel GPU, it provides at least 3x computational efficiency. Exemplary code is shown below.

[0115] pad_size = k

[0116] filterValNew = np.zeros([pad_size, num_channels_out, pad_size, num_channels_in, filter_height,filter_width]

[0117] for i in range(0, pad_size):

[0118] filterValNew[i, :, i,:, :, :] = filterVal

[0119] num_channels_out_padded = pad_size num_channels_out

[0120] biasValNew = np.tile(biasVal, pad_size)

[0121] When performing matrix multiplication using a GPU tensor core that requires 8 channels of input, the computation using the configured input data and sparse filters via the above process is faster than using the original input image and the original filters.f The computation is at least 3 times faster. It should be understood that these two methods are not mutually exclusive when configuring filters. Similar methods can be used to configure CNN models or filters.

[0122] 4. Execute the configured model on the configured data.

[0123] Figures 10 and 11 illustrate the execution of a configured model on configured data. More specifically, Figure 10 shows the execution using configured data 1010 (configured as shown in Figure 7A) and a configured model with a filter configured by replication, as shown in Figure 9A. Figure 11 shows the execution using configured data 1110 (configured as shown in Figure 7B) and a configured model with a sparse filter, as shown in Figure 9B.

[0124] In Figure 10, the configured data 1010 has replicated channels, such as the first red channel 1012, the first green channel 1014 (which is also the second channel of the configured data 1010), and the... m Red Channel 1016 and the n Blue channel 1018 (which is also the last channel of configured data 1010). The corresponding configured filter 1020 is a filter formed by replication, and configured filter 1020 has the same number of channels as configured data 1010. In order to execute the corresponding CNN model using configured data 1010, each channel of configured data 1010 is multiplied by the corresponding channel of configured filter 1020. For example, the first red channel 1012 is multiplied by the first red channel of configured filter 1020. The resulting internal data will have the same number of channels as configured data 1010 (and configured filter 1020).

[0125] In Figure 11, the configured data 1110 has a set of replicated channels, for example, the first few channels are red channels and the later few channels are green channels. In this case, filters with different configurations can be used. Configured filter 1120 is the filter that helps the red channels of the configured data 1110 to be executed by the corresponding CNN model, and configured filter 1130 is the filter that helps the blue channels of the configured data 1110 to be executed by the corresponding CNN model. When the corresponding CNN model is executed using the configured data 1110, a dot product is performed along the channel direction of the configured data 1110 (labeled in Figure 11) with the corresponding configured filters 1120 and 1130.

[0126] B. Sample system for the process

[0127] Figure 12 illustrates an example of a physical computing environment 1200 according to certain embodiments of the present invention. System 1210 is a data and model preparation system in which raw input data is obtained in module 1212 and can be preprocessed by data preprocessing module 1214, and a model can be selected from model database 1216. The raw input data may be sequencing data (such as sequencing pulse data), or the raw input data may be three-dimensional image data. The raw input data in module 1212 may be generated by data generation device 105 in Figure 1, or obtained by data collection unit 110. Module 1214 may correspond to data preprocessing unit 120 in Figure 1. Preprocessing may include denoising, color separation, baseline correction, mobility shift correction, resizing, remodeling, etc.

[0128] The model database 1216 may include only CNN models, only ANN models, or a combination of different types of deep learning or ML models. The model selection function in system 1210 can be performed by the model collection unit 140 in Figure 1. System 1210 can be implemented on a general-purpose computing device. In some cases, system 1210 may also be implemented on a dedicated computing device.

[0129] Module 1220 is a dedicated computing device information acquisition module. Module 1220 acquires information about the acceleration path of the dedicated computing device. For example, when the dedicated computing device is a GPU, this information may be the type of tensor core used by the GPU and the prerequisites for using the tensor core. This information may also include configuration parameters, which are the number of input channels required by the tensor core. For example, the tensor core of a GPU may require 8 input channels. In this case, the configuration parameter is 8. Module 1220 can perform the function at box 230 in Figure 2. Module 1220 may be an external module of system 1210. In some cases, module 1220 may be an internal module of system 1210.

[0130] System 1230 is a data and model configuration system, wherein data is configured in data configuration unit 1232 and the model is configured in model configuration unit 1234. Data configuration unit 1232 performs the same or similar functions required at block 240 in FIG. 2, and model configuration unit 1234 performs the same or similar functions required at block 250. Exemplary data configuration and model configuration processes can be found in FIGS. 7A-7B and FIG. 9A-9B. System 1230 can be implemented on a general-purpose computing device. In some cases, system 1230 can also be implemented on a dedicated computing device. System 1230 can be implemented on the same dedicated computing device as system 1210. In some cases, system 1230 can be implemented partly on a general-purpose computing device and partly on a dedicated computing device. For example, data configuration unit 1232 is implemented on a general-purpose computing device, while model configuration unit 1234 is implemented on a dedicated computing device.

[0131] System 1240 is an execution system implemented on a dedicated computing device. Configured input data is acquired by module 1242 and sent to module 1244, where a configured deep learning model is acquired for execution. Execution occurs on the acceleration path of the dedicated computing device. In the case that the dedicated computing device is a GPU, the acceleration path is a tensor core, and execution can achieve computational and energy efficiency. Output module 1250 can provide the output of the execution. Output module 1250 can be an internal module of system 1240. In some cases, systems 1210, 1230, and 1240 are implemented on the same dedicated computing device.

[0132] V. Computer Systems

[0133] Any computer system mentioned herein can utilize any suitable number of subsystems. An example of such a subsystem is shown in computer system 1300 of Figure 13. In some embodiments, the computer system includes a single computer device, wherein the subsystem may be a component of the computer device. In other embodiments, the computer system may include multiple computer devices, each being a subsystem with internal components. The computer system may include desktop and portable computers, tablets, mobile phones, and other mobile devices.

[0134] The subsystems shown in Figure 13 are interconnected via system bus 75. Additional subsystems are shown, such as printer 74, keyboard 78, storage device 79, monitor 76 (which is connected to display adapter 82), etc. Peripheral devices and input / output (I / O) devices connected to I / O controller 71 can be any number of devices known in the art, such as input / output (I / O) ports 77 (e.g., USB, FireWire). ® The system is connected to the computer system. For example, I / O port 77 or external interface 81 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect computer system 10 to a wide area network, such as the Internet, a mouse input device, or a scanner. Interconnection via system bus 75 allows the central processing unit 73 to communicate with each subsystem and control the execution of multiple instructions from system memory 72 or storage device 79 (e.g., a fixed disk, such as a hard disk drive, or optical disk), as well as the exchange of information between subsystems. System memory 72 and / or storage device 79 may contain computer-readable media. Another subsystem is a data collection device 85, such as a camera, microphone, accelerometer, etc. Any data mentioned herein can be output from one component to another and can be output to the user.

[0135] A computer system may include multiple identical components or subsystems, for example, connected together via an external interface 81, an internal interface, or a removable storage device that can be connected to or moved from one component to another. In some embodiments, the computer system, subsystem, or device may communicate via a network. In this case, one computer may be considered a client and another computer may be considered a server, with each computer being considered part of the same computer system. The client and server may each include multiple systems, subsystems, or components.

[0136] Various aspects of the embodiments may be implemented in a modular or integrated manner using hardware (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or computer software having a generally programmable processor, in the form of control logic. As used herein, processors include single-core processors, multi-core processors on the same integrated chip, or multiple processing units on a single circuit board or networked. Based on the disclosure and teachings provided herein, those skilled in the art will know and understand other ways and / or methods of implementing the embodiments of the present invention using hardware and combinations of hardware and software.

[0137] Any suitable computer language, such as Java, C, C++, C#, Objective-C, Swift, or scripting languages ​​such as Perl or Python, using, for example, conventional or object-oriented techniques, can be used to implement any software component or function described in this application as software code that is executed by a processor. The software code can be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media (such as hard disk drives or floppy disks), or optical media (such as optical discs (CDs) or DVDs (Digital Universal Disk), flash memory, etc.). The computer-readable medium can be any combination of such storage or transmission means.

[0138] Such programs can also be encoded and transmitted using carrier signals adapted for transmission over wired, optical, and / or wireless networks conforming to various protocols, including the Internet. Thus, computer-readable media can be created using data signals encoded with such programs. Computer-readable media encoded with program code can be packaged with compatible devices or provided separately from other devices (e.g., for download via the Internet). Any such computer-readable media can reside on or within a single computer product (e.g., a hard drive, CD, or an entire computer system) and can exist on or within different computer products within a system or network. The computer system may include monitors, printers, or other suitable displays for providing users with any of the results mentioned herein.

[0139] Any method described herein can be performed wholly or partially by a computer system including one or more processors, which can be configured to perform the steps. Therefore, embodiments may be directed to a computer system configured to perform the steps of any method described herein, and may have different components for performing corresponding steps or groups of corresponding steps. Although the steps are presented as numbered, the steps of the methods herein may be performed simultaneously or in different orders. Furthermore, some steps may be used in conjunction with some steps from other methods. Additionally, all or some steps may be optional. Furthermore, any step of any method may be performed using a module, unit, circuit, or other means for performing those steps.

[0140] The specific details of a particular embodiment may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the present invention. However, other embodiments of the invention may be directed to specific embodiments relating to each individual aspect, or specific combinations of these individual aspects.

[0141] For purposes of illustration and description, exemplary embodiments of the invention have been described above. It is not intended to exhaustively describe the invention or limit it to the precise forms described, and many modifications and variations are possible in accordance with the above teachings.

[0142] Unless otherwise specified, references to “a,” “an,” or “the” are intended to mean “one or more.” Unless otherwise specified, the use of “or” is intended to mean “includes or,” not “excludes or.” Referring to a “first” component does not necessarily require the provision of a second component. Furthermore, unless explicitly stated otherwise, references to “first” or “second” components do not limit the referenced component to a specific location.

[0143] All patents, patent applications, publications, and specifications mentioned herein are incorporated herein by reference in their entirety for all purposes. None of them are considered prior art.

Claims

1. A method for executing a machine learning model on a dedicated computing device, the method comprising: The first computing device obtains raw input data, which has a set of dimensions, including (1) a channel dimension with a number of C channels and (2) a first length dimension corresponding to the height or width of the raw input data; The first computing device obtains the machine learning model, which includes a function that applies a set of M model parameters to at least one channel of the original input data; The first computing device determines a configuration parameter K for the dedicated computing device, wherein the configuration parameter K corresponds to the following data size for which the acceleration path of the dedicated computing device is operable; The original input data is configured based on the configuration parameter K to obtain configured input data, wherein configuring the original input data includes scaling the number C channels in the channel dimension using the configuration parameter K, and inversely scaling the first length dimension using the configuration parameter K, thereby forming KxC channels; The machine learning model is configured based on the configuration parameter K to obtain a configured machine learning model, the configured machine learning model having a configured model dimension corresponding to the data size of the acceleration path, wherein configuring the machine learning model includes: The function is expanded to include at least KxM model parameters applied to at least K channels; The dedicated computing device uses the configured input data to execute the configured machine learning model with configured model parameters to obtain output data; and The output data is provided by the dedicated computing device.

2. The method of claim 1, further comprising (i) sequencing nucleic acid molecules obtained from a test sample using nanopore sequencing, or (ii) sequencing a collection of nucleic acid molecules using fluorescence microscopy sequencing to provide the raw input data.

3. The method of claim 1, further comprising preprocessing the original input data by the first computing device, wherein the preprocessing includes padding the original input data to satisfy a dimension based on the configuration parameter K.

4. The method of claim 1, wherein the machine learning model further comprises a second function that applies a set of N model parameters to at least one channel of internal data processed by the machine learning model, and wherein configuring the machine learning model further comprises extending the second function to include at least KxN model parameters applied to at least K channels of the internal data.

5. The method according to claim 1, wherein the machine learning model is a convolutional neural network (CNN) model.

6. The method of claim 5, wherein the function is a filter of the CNN model having C channels, and wherein extending the function includes generating a KxC channel filter, wherein each of the C channels of the KxC channel filter is the same as the filter having the C channels, and thus the filter having the C channels is copied K times in the KxC channel filter.

7. The method of claim 5, wherein the function is a filter of the CNN model having C channels, and wherein extending the function comprises generating a KxC channel filter, wherein each channel of the KxC channel filter has a larger size than the size of each channel of the filter having the C channels, and wherein the values ​​of the filter having the C channels are copied to a portion of the KxC channel filter, and the other portions of the KxC channel filter have values ​​equal to zero.

8. The method of claim 1, wherein the dedicated computing device is a graphics processing unit (GPU) and the acceleration path is one or more tensor cores.

9. The method of claim 1, wherein extending the function comprises copying the function K times.

10. A computer product comprising a non-transitory computer-readable medium storing a plurality of instructions, which, when executed, cause a computer system to perform the method according to any one of the preceding claims.

11. A system comprising: The computer product according to claim 2; as well as One or more processors for executing instructions stored on a computer-readable medium.

12. A system comprising tools for performing any of the methods described above.

13. A system comprising one or more processors configured to perform any of the methods described above.

14. A system comprising modules that perform the steps of any of the methods described above.

Images