AI Chip Segmentation for Edge Inference Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for accelerating convolutional neural network operations rely heavily on data center computing power or software optimizations, which are inefficient for edge devices in terms of power consumption and time, and do not effectively leverage the characteristics of convolutional neural networks.
Innovation Solution
The AI chip and convolutional neural network computing method divide convolution tasks into subtasks and assign them to parallel computing units for concurrent execution, with pooling operations performed separately, allowing for efficient matrix operations and reduced data transport, enabling faster and more power-efficient processing on edge devices without data center support.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If convolution tasks are executed using data center computing power or software optimizations, then computing capability is improved, but power consumption and processing time increase significantly for edge devices
Solution Approach 1:
The patent segments the convolution task into multiple subtasks and assigns them to different parallel computing units within the AI chip. This segmentation enables distributed processing that reduces the computational burden on single units, achieving high computing capability while maintaining low power consumption through efficient resource utilization across multiple specialized units.
Solution Approach 2:
The patent introduces a pooling computation unit as an intermediary component that receives results from parallel computing units and performs pooling operations. This intermediary structure optimizes data flow and reduces unnecessary data transport between computing units and memory, thereby reducing power consumption while maintaining processing efficiency.
2Power
If convolution tasks are executed using data center computing power or software optimizations, then computing capability is improved, but processing time increases significantly for edge devices
Solution Approach 1:
The patent divides the convolution task into multiple subtasks that can be executed concurrently by different parallel computing units. This segmentation enables parallel processing, significantly reducing processing time while maintaining high computing capability through coordinated execution of multiple subtasks.
Solution Approach 2:
The patent ensures continuous processing by having the pooling computation unit immediately process results as they become available from parallel computing units, without idle waiting time. This continuous action eliminates processing gaps and maintains high throughput, reducing overall processing time.
3Productivity
If convolution tasks are divided into subtasks and executed on parallel computing units, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
The patent designs parallel computing units with universal functionality to perform convolution, batch normalization, and non-linear computation operations. This multi-functionality reduces the need for specialized hardware for each operation type, thereby improving processing efficiency while controlling device complexity through resource consolidation.
Solution Approach 2:
Instead of having each computing unit perform all operations sequentially, the patent inverts the approach by having specialized parallel computing units handle specific operation types in parallel, with a separate pooling unit handling aggregation. This inversion improves efficiency through parallelization while managing complexity through functional specialization.
4Productivity
If pooling operations are performed separately from convolution operations, then processing efficiency is improved, but data transport requirements increase
Solution Approach 1:
The patent introduces a pooling computation unit as an intermediary that is directly connected to parallel computing units. This intermediary structure enables efficient data transfer by creating a dedicated data path for pooling operations, reducing the volume of data that needs to be transported through general-purpose memory interfaces and buses.
Solution Approach 2:
The patent merges the pooling computation unit with the parallel computing units into a unified architecture where data flow between convolution and pooling operations is optimized. This merging reduces redundant data transport by enabling direct data sharing between adjacent computational stages within the same processing pipeline.
Data Source
AI summary
Embodiments disclose an artificial intelligence chip and a convolutional neural network applied to the artificial intelligence chip comprising a processor, at least one parallel computing unit, and a pooling computation unit. The method comprises: dividing a convolution task into convolution subtasks and corresponding pooling subtasks; executing convolution subtasks at different parallel computing units, and performing convolution, batch normalization, and non-linear computing operation in a same parallel computing unit; sending an execution result of each parallel computing unit from executing the convolution subtask to the pooling computation unit for executing the corresponding pooling subtask; merging executing results of the pooling computation unit from performing pooling operations on the executing results outputted by respective convolution subtasks to obtain an execution result of the convolution task. This can reduce data transport, such that operations of the convolutional neural network may be accomplished with lower power consumption and less time in an edge device.