Neural Network Weight Update Partitioning for Bandwidth Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In deep learning processes, memory resources and arithmetic units are often underutilized due to insufficient utilization of external memory bandwidth and inefficient arithmetic resource allocation, particularly when updating parameters in neural networks.
Innovation Solution
A semiconductor device comprising a first arithmetic part and a second arithmetic part, where the first part performs parameter updating operations and data transfer control, while the second part performs forward and backward operations, allowing efficient utilization of both memory and arithmetic resources by optimizing data transfer between internal and external memories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If parameters are input/output to/from external memory in processing layer algorithm, then data can be supplied to processors, but external memory bandwidth is underutilized because the number of parameters is smaller than the number of data pieces
Solution Approach 1:
The system divides processing into two separate arithmetic parts: one dedicated to parameter updating operations and another to forward/backward operations. This segmentation allows each part to handle different data types (parameters vs. data pieces) independently, enabling full utilization of external memory bandwidth while maintaining efficient parameter processing.
Solution Approach 2:
The first arithmetic part that handles parameter updating is designed to perform multiple functions including gradient calculation, parameter optimization, and weight updates. This multi-functional design allows the same hardware resource to handle both parameter processing and contribute to overall system throughput, improving memory utilization without sacrificing processing efficiency.
2Productivity
If parameters are updated in a semiconductor device including a large number of arithmetic units, then neural network training can be performed, but utilization efficiency of arithmetic units is reduced
Solution Approach 1:
The arithmetic units are segmented into two distinct groups: a first arithmetic part for parameter updating operations and a second arithmetic part for forward and backward propagation. This segmentation ensures that arithmetic units are specialized for their respective tasks, eliminating idle resources and improving overall utilization efficiency while maintaining full training capability.
Solution Approach 2:
A data transfer control unit acts as an intermediary between the two arithmetic parts, efficiently managing data flow and coordination. This intermediary ensures that both arithmetic parts operate simultaneously without resource conflicts, maximizing arithmetic unit utilization while enabling comprehensive neural network training.
3Manufacturing precision
If backward operation is executed to calculate gradient of weight, then weight can be updated, but communication bottlenecks occur due to sequential data transfer between arithmetic parts
Solution Approach 1:
The system enables continuous parallel operation where the first arithmetic part performs parameter updating while the second arithmetic part simultaneously executes forward and backward operations. The data transfer control unit maintains continuous data flow between parts, eliminating idle time and communication bottlenecks while ensuring accurate weight updates through coordinated operations.
Data Source
AI summary
A processing system includes a first chip including one or more first processors, and a second chip including one or more second processors. For a weight of a neural network to be updated, the one or more second processors execute a backward operation of the neural network and calculate a gradient of the weight, the calculated gradient is transferred to the one or more first processors, and the one or more first processors update the weight based on the calculated gradient. The one or more first processors transfer the updated weight to the second chip. The one or more first processors update the weight of the neural network in accordance with a method using multiple types of parameters.