Arithmetic Method for Convolutional Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large convolutional neural networks face increased power consumption due to the need to read and write data between internal and external memory, which is exacerbated by limited internal memory capacity, leading to higher costs.
Innovation Solution
An arithmetic method that selectively uses two modes for convolution operations based on input data size, either storing weight data in external memory and input data in internal memory, or vice versa, to minimize data transfer and calculate matrix products efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If internal memory capacity is increased to reduce power consumption, then power consumption is reduced, but manufacturing cost increases
Solution Approach 1:
The patent segments memory storage into two parts: internal memory for frequently accessed data (weight data) and external memory for less frequently accessed data (input data). This segmentation allows the system to maintain small internal memory capacity while still achieving efficient data access patterns that reduce overall power consumption without requiring expensive large-capacity internal memory.
Solution Approach 2:
The patent introduces external memory as an intermediary between the processing unit and the data sources. By using external memory as a mediator for storing input data and intermediate results, the system reduces the burden on internal memory, allowing small internal memory capacity to suffice while maintaining efficient data access that lowers power consumption.
2Loss of energy
If internal memory capacity is increased to reduce data transfer, then data transfer is reduced, but device complexity increases
Solution Approach 1:
The patent segments the memory system into internal and external components with distinct functional roles. Internal memory stores weight data that needs to be repeatedly accessed during convolution operations, while external memory stores input data and intermediate results. This segmentation optimizes data transfer efficiency without requiring the internal memory to handle all data storage, thus avoiding increased device complexity.
Solution Approach 2:
The patent dynamically manages data placement between internal and external memory based on access patterns. Frequently accessed weight data remains in internal memory, while input data and intermediate results are managed in external memory. This dynamic approach optimizes data transfer efficiency without requiring static over-provisioning of internal memory capacity, thereby avoiding increased device complexity.
3Adaptability or versatility
If data is read from external memory multiple times, then processing flexibility is improved, but power consumption increases
Solution Approach 1:
The patent performs preliminary actions by pre-loading weight data into internal memory before processing operations begin. This preliminary placement of frequently accessed data in fast internal memory allows multiple processing operations to proceed without repeated external memory accesses, significantly reducing power consumption while maintaining processing flexibility for different input data.
Solution Approach 2:
The patent enables continuous processing by maintaining weight data in internal memory throughout the convolution operations. This continuous availability of weight data in fast internal memory allows uninterrupted processing of different input data batches, maintaining processing flexibility while avoiding repeated power-consuming external memory accesses.
Data Source
AI summary
Provided is an arithmetic method of performing convolution operation in convolutional layers of a neutral network by calculating matrix products. The arithmetic method includes: determining, for each of the convolutional layers, whether an amount of input data to be inputted to the convolutional layer is smaller than or equal to a predetermined amount of data; selecting a first arithmetic mode and performing convolution operation in the first arithmetic mode, when the amount of input data is determined to be smaller than or equal to the predetermined amount of data in the determining; selecting a second arithmetic mode and performing convolution operation in the second arithmetic mode, when the amount of input data is determined to be larger than the predetermined amount of data in the determining; and outputting output data which is a result obtained by performing convolution operation.