Gradient APIs for Split Client-Hosted Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hosting neural networks is resource-intensive and requires significant memory, leading to inefficiencies in hosting systems and client access methods.
Innovation Solution
Implementing an application programming interface (API) to manage gradient information and backpropagation between client and hosted neural networks, allowing for optimized resource allocation and specialized output calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If neural networks are hosted on remote systems, then client access to neural network functionality is enabled, but resource consumption and memory requirements increase significantly
Solution Approach 1:
The neural network is divided into two segments: a hosted segment running on remote system and a local segment running on client device. The local segment processes gradients and performs backpropagation locally, while only necessary inferencing computations are sent to the hosted segment. This segmentation reduces the resource burden on the hosted system while maintaining client accessibility.
Solution Approach 2:
The patent extracts the gradient computation and backpropagation functionality from the hosted system and places it in the local client segment. This extraction eliminates the need for the hosted system to process all neural network operations, significantly reducing memory and computational resource requirements while maintaining the hosted neural network's accessibility to clients.
2Extent of automation
If neural networks are hosted on remote systems, then centralized processing is achieved, but memory requirements increase to store network variations
Solution Approach 1:
The neural network is segmented between hosted and local segments, allowing the hosted system to maintain a single centralized network model while the local segment handles client-specific computations. This reduces the need to store multiple network variations in hosted memory.
Solution Approach 2:
Instead of storing multiple variations of the neural network in hosted memory, the patent uses a shared model where the hosted segment maintains the original network and communicates parameters to the local segment. The local segment creates functional copies through computation rather than storing duplicate network models, significantly reducing memory requirements.
3Use of energy by moving object
If all neural network processing is done locally, then resource consumption is reduced, but specialized calculations and optimized hosting benefits are lost
Solution Approach 1:
The neural network is segmented to perform different functions in different locations: local segment handles gradient computation and backpropagation to reduce resource consumption, while the hosted segment provides centralized model management and specialized calculations. This segmentation allows the system to achieve both resource efficiency and specialized processing capabilities.
Solution Approach 2:
The API serves as an intermediary between the local and hosted segments, enabling efficient communication and coordination. It allows the local segment to access hosted neural network functionality while maintaining local processing of gradients and backpropagation, thus achieving resource optimization without sacrificing specialized calculation capabilities.
Data Source
AI summary
Apparatuses, systems, and techniques to perform an API set to call to apply gradients of a neural network layer resulting from backpropagation. A processor performs a process to perform a set of one or more API calls by one or more API modules to allow for the performing of neural network layers hosted on client and service networks in an indicated order. For example, a client network may perform a set of layers, calling to a service network to continue inferencing, and call to receive the results of inferencing to resume inferencing within the client network.