Systems and methods for efficient model execution on machine-learning accelerators

Abstract

Methods, systems, and apparatus for reducing latency of configuration of image sensor and image signal processor. A computing system can include a machine learning (ML) processing engine that can process denoising diffusion ML models for execution on statically compiled ML accelerators. The system can determine that a partitioned graph representation, which includes connected subgraphs that each represent at least one layer of the neural network of the ML model, forms a directed acyclic graph. The system can insert cache nodes in the graph representation, where each cache node corresponds to a respective subgraph and is configured to cache output of the respective subgraph. The system can generate an execution dataflow graph including a plurality of iterations of the partitioned graph representation, where, at one or more iterations during model inference operations, the execution dataflow graph uses inputs from cache nodes and excludes execution of subgraphs corresponding to the cache nodes.

Images