Neural Network Runtime Parameter Updates via Metadata Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems, particularly deep neural networks, face limitations in updating parameters during runtime due to parameters being included in a compiled binary, and lack of direct access to the neural network in some environments, making it difficult to adapt to changing conditions without recompiling.
Innovation Solution
The system enables updating of neural network parameters, such as weights, during execution by generating metadata that allows for mutable parameters, facilitating updates without additional computing resources, using a neural network compiler and driver architecture that separates access and control layers, enabling runtime updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parameters are included in a compiled binary, then the neural network can be executed efficiently, but the parameters cannot be updated during runtime without recompiling
Solution Approach 1:
The patent segments the neural network parameters into two categories: immutable parameters that are compiled into the binary for efficient execution, and mutable parameters that are stored separately and can be updated at runtime. This segmentation allows the system to maintain execution efficiency while enabling parameter updates without recompiling.
Solution Approach 2:
The patent introduces dynamic parameter updating capability where mutable parameters can be modified during runtime through a driver interface. This allows the neural network to adapt to changing conditions without requiring a complete recompilation, resolving the contradiction between execution efficiency and adaptability.
2Ease of manufacture
If the neural network is compiled into a binary, then it can be deployed on resource-constrained devices, but direct access to update parameters is not possible
Solution Approach 1:
The patent introduces a driver interface as an intermediary between the compiled neural network binary and the parameter update mechanism. This driver provides a simplified interface that allows applications to update mutable parameters without needing direct access to the compiled binary's internal structure, thus maintaining deployment capability while improving ease of operation.
3Reliability
If parameters are stored in the compiled binary, then the system is self-contained, but adapting to changing conditions requires recompiling
Solution Approach 1:
The patent extracts mutable parameters from the compiled binary and stores them separately in a dedicated memory region. This extraction allows the system to maintain self-containment through the compiled binary while enabling rapid parameter updates by simply modifying the separate parameter storage without triggering a recompilation process.
Data Source
AI summary
The subject technology receives code corresponding to a neural network (NN) model and a set of weights for the NN model. The subject technology determines a set of layers that are mutable in the NN model. The subject technology determines information for mapping a second set of weights to the set of weights for the NN model. The subject technology generates metadata corresponding to the set of layers that are mutable, and the information for mapping the second set of weights to the set of weights for the NN model, wherein the generated metadata enables updating the set of layers that are mutable during execution of the NN model.