Interpolation Down-Sampling in Neural Networks for Edge AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep neural networks (DNNs) are computationally demanding and energy-intensive, making them challenging to implement on electronic devices with limited hardware capabilities such as smartphones and IoT devices.
Innovation Solution
Implement a spatio-pyramidal neural network architecture that uses interpolation layers for down-sampling data instead of conventional pooling and convolutional layers with strides, allowing for smoother and more efficient data processing with reduced memory and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional pooling layers or convolutional layers with stride are used for down-sampling, then the neural network structure is simple and easy to implement, but the computational resources and energy consumption are high
Solution Approach 1:
The patent changes the fundamental parameter of down-sampling from discrete sampling (pooling) to continuous interpolation-based down-sampling. This parameter change transforms the computational nature from frequent memory accesses to more efficient interpolation calculations, reducing energy consumption while maintaining implementation feasibility
Solution Approach 2:
The patent substitutes the mechanical memory access operations in conventional pooling layers with mathematical interpolation operations. This substitution replaces the physically intensive memory read operations with computationally lighter interpolation calculations, thereby reducing energy consumption
2Ease of manufacture
If conventional pooling layers or convolutional layers with stride are used for down-sampling, then the neural network structure is simple, but the memory footprint is large
Solution Approach 1:
The patent changes the down-sampling parameter from discrete index-based sampling to continuous coordinate-based interpolation. This parameter change allows for more compact data representation and reduces the memory footprint by eliminating the need to store and access large intermediate data arrays that are characteristic of conventional pooling operations
Solution Approach 2:
The patent uses interpolation to create a continuous representation of data that can be down-sampled without copying large amounts of data to intermediate memory locations. This approach reduces memory footprint by performing calculations in-place or with minimal temporary storage requirements
3Ease of manufacture
If conventional pooling layers or convolutional layers with stride are used for down-sampling, then the implementation is straightforward, but the processing speed is slow due to frequent memory accesses
Solution Approach 1:
The patent substitutes mechanical memory access operations with mathematical interpolation computations. This substitution eliminates the bottleneck of frequent memory reads by performing down-sampling calculations directly from the data values, significantly improving processing speed while keeping the implementation relatively straightforward
Solution Approach 2:
The patent introduces interpolation as an intermediary computational step between the input data and the down-sampled output. This intermediary process allows for efficient calculation that avoids direct frequent memory accesses, acting as a mediator that transforms data in a computationally efficient manner
4Use of energy by moving object
If interpolation layers are used for down-sampling, then computational resources and memory footprint are reduced, but the network architecture becomes more complex
Solution Approach 1:
The patent designs the interpolation layer to serve multiple functions: down-sampling, feature extraction, and continuous spatial transformation. This multi-functionality reduces the need for separate pooling and convolutional layers, thereby reducing overall network architecture complexity while maintaining low energy consumption
5Quantity of substance
If interpolation layers are used for down-sampling, then memory footprint is reduced, but the computational algorithm becomes more complex
Solution Approach 1:
The patent changes the algorithm from discrete sampling with large memory buffers to continuous interpolation with in-place calculations. This parameter change reduces memory footprint by performing computations directly on the data values without requiring large intermediate storage, while the algorithmic complexity is managed through efficient interpolation formulations
Data Source
Figure 1
Figure 2
Figure 3a~3c
AI summary
The invention relates to a data processing apparatus (100) comprising processing circuitry (101) configured to implement a neural network, wherein the neural network comprises a plurality of processing layers for sequentially processing data, wherein, for down-sampling the data, the plurality of processing layers comprise a plurality of interpolation layers, wherein each interpolation layer of the plurality of processing layers (802a-k) is configured to process the data by determining a respective output data array based on a respective input data array using interpolation, wherein a spatial dimension of the respective output data array comprises less data points than the corresponding spatial dimension of the respective input data array. Moreover, the invention relates to a corresponding data processing method. The invention allows efficiently running neural networks on electronic devices with limited hardware capabilities, such as smartphones or other types of IoT devices.