Model Compilation for Streaming Execution Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming-based execution devices cannot run machine learning models as they cannot read instructions of an executable program, limiting their ability to perform computation tasks.
Innovation Solution
A model compiling method that acquires a model file corresponding to a machine learning model, parses it to obtain a computational graph, determines runtime and hardware configuration information for operators, and sends this information to a streaming-based execution device to perform computations, allowing the device to execute the model without compiling it into executable instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a machine learning model is compiled into instructions of an executable program, then the model can be executed on traditional execution devices, but streaming-based execution devices cannot read these instructions and thus cannot run the model
Solution Approach 1:
The patent changes the fundamental parameter of model representation from executable instructions to hardware configuration information. The compiler transforms the computational graph and operator parameters into hardware configuration information that streaming-based execution devices can directly read and execute, enabling compatibility with these specialized devices without requiring traditional instruction sets
Solution Approach 2:
The patent introduces hardware configuration information as an intermediary between the computational graph and the streaming-based execution device. This intermediary format serves as a bridge that translates abstract computational operations into hardware-specific configuration data that the execution device can process, eliminating the need for traditional instruction sets
2Reliability
If runtime information is determined for all operators in the computational graph, then complete hardware configuration can be provided, but this increases processing time and computational overhead
Solution Approach 1:
The patent segments the operators into two sets: a first set requiring runtime information determination and a second set that can be handled differently. This segmentation allows the system to determine hardware configuration information only for necessary operators, reducing unnecessary processing time while maintaining complete and accurate configuration for operators that require it
Solution Approach 2:
The patent applies partial action by determining runtime information only for the first set of operators rather than all operators in the computational graph. This selective approach provides sufficient hardware configuration information for execution while avoiding the time cost of processing every operator, achieving a balance between completeness and efficiency
Data Source
AI summary
A model compiling method and apparatus, and a model running system. The method includes: parsing a model file to obtain a first computational graph; determining runtime information of a first set of first operators according to a user input and the first computational graph; determining hardware configuration information of a first operator according to the runtime information of each first operator in the first set of first operators; and sending the hardware configuration information of the first operator to an execution device to cause the execution device to perform computation of the first operator.