Distributing Neural Network Computation Across Multiple Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks in mobile devices face challenges such as inference latency and vulnerability due to reliance on external cloud resources, leading to bottlenecks and security concerns, especially as user data increases.
Innovation Solution
A method utilizing multiple processors within an electronic device to distribute and perform computations for neural network layers, where one processor handles a first portion and another processor handles a second portion of the computation, with output values from each processor used as inputs for subsequent layers, optimizing resource usage and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If cloud resources are used for neural network inference, then processing power is improved, but inference latency increases and security vulnerabilities arise
Solution Approach 1:
The patent segments the neural network computation plan into multiple portions and distributes them across different processors (e.g., CPU, GPU, NPU) within the mobile device. This allows parallel processing of different neural network layers simultaneously, improving processing power while keeping data locally to reduce inference latency and eliminate security risks associated with cloud dependency.
2Power
If cloud resources are used for neural network inference, then processing capability is enhanced, but system reliability deteriorates due to network dependency
Solution Approach 1:
The patent enables the mobile device to perform neural network inference independently using its own hardware resources (CPU, GPU, NPU) without relying on external cloud services. The device self-manages the computation by distributing neural network layers across available processors, ensuring reliable operation even when network connectivity is unavailable.
3Productivity
If multiple processors are used for neural network computation, then processing speed is improved, but device complexity increases
Solution Approach 1:
The patent creates a universal neural network processing framework that can utilize multiple types of processors (CPU, GPU, NPU) for the same computational task. The system automatically selects and distributes neural network layers across available processors based on their capabilities, achieving high processing speed while maintaining flexibility and reducing the need for dedicated hardware for each processor type.
Data Source
AI summary
A method for processing an artificial network by an electronic device includes obtaining, by using a first processor and a second processor, a neural network computation plan for performing computation of a first neural network layer of the artificial neural network, performing a first portion of a computation of the first neural network layer by using the first processor, and performing a second portion of the computation of the first neural network layer by using the second processor based on the obtained neural network computation plan, obtaining a first output value based on a performance result of the first processor and a second output value based on a performance result of the second processor, and using the obtained first output value and the second output value as an input value of a second neural network layer of the artificial neural network.