Shared Neural Network Inference for Lower Mobile Compute Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inference computations for multiple neural networks on mobile devices are computationally expensive and energy-intensive, leading to battery life reduction, and existing methods do not efficiently utilize commonalities between networks for reduced computing and memory usage.
Innovation Solution
Identify and combine units within neural networks that are identical or similar based on predefined criteria, performing a single inference computation for these units on a common hardware platform, and processing the outputs in each network separately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If inference computations are performed separately for multiple neural networks, then each network can be processed independently, but computing expenditure and energy consumption increase significantly
Solution Approach 1:
The patent merges inference computations of multiple neural networks by identifying common units across networks and performing a single inference computation for these shared units. The method combines the computation results and distributes them back to the respective networks, thereby reducing redundant calculations and lowering energy consumption while maintaining independent network functionality.
Solution Approach 2:
The patent creates a universal inference computation mechanism that serves multiple neural networks simultaneously. By identifying units that are identical or similar across different networks based on predefined criteria, the system performs a single computation that can be universally applied to all networks sharing those units, reducing overall computational expenditure and energy usage.
2Adaptability or versatility
If multiple neural networks are run simultaneously on mobile devices, then diverse classification tasks can be performed, but battery life is reduced due to high computational expenditure
Solution Approach 1:
The patent enables multiple neural networks to share common inference computation resources by identifying and merging identical or similar units across networks. This reduces the overall computational load on mobile devices, allowing diverse classification tasks to be performed simultaneously without proportionally increasing energy consumption, thereby preserving battery life.
Solution Approach 2:
The patent performs inference computations only once for common units shared across multiple networks, rather than performing complete separate computations for each network. This partial computation approach maintains the versatility of running multiple networks while significantly reducing the total computational action required, thus conserving battery life.
3Reliability
If identical computation units are processed separately in multiple networks, then each network maintains independence, but memory space is wasted due to redundant computations
Solution Approach 1:
The patent merges the processing of identical computation units across multiple neural networks by performing a single inference computation for shared units. The computation results are then distributed back to the respective networks, reducing redundant memory usage while preserving network independence through the distribution mechanism.
Solution Approach 2:
The patent uses a copying mechanism where the result of a single inference computation for common units is copied and distributed to multiple neural networks that share those units. This approach reduces memory space requirements by avoiding redundant storage of identical computation results while maintaining the independence of each network through the copying process.
Data Source
AI summary
A method for the inference computation of a plurality of neural networks on a hardware platform. Each of the neural networks comprise a plurality of neurons, which respectively aggregate inputs into a network input using a transfer function characterized by weights and process this network input into an activation using an activation function. The method includes: identifying at least one unit, which comprises one or multiple transfer functions and/or complete neurons and exists in at least two of the networks in the same form or in a form that is similar according to a predefined criterion; performing a single inference computation for the unit on the hardware platform so that the unit provides a set of outputs; processing this set of outputs in the respective networks as an output of the unit. A method for the simultaneous execution of multiple applications is also provided.