Crossbar Circuit for Unaligned Memory Access in Neural Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processors rely heavily on CPUs for operations, leading to significant bandwidth consumption and increased power usage due to the need for intermediate cache or buffer components, which are costly in terms of area and power.
Innovation Solution
A neural processor circuit with a crossbar circuit that directly connects a buffer memory to an array of neural engine circuits, allowing for unaligned memory access, re-alignment, or bypassing of input data during each operating cycle, thereby eliminating the need for intermediate cache components and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If intermediate cache or buffer components are used to enable unaligned memory access in neural network processors, then memory access flexibility is improved, but area consumption and power usage increase significantly
Solution Approach 1:
The patent extracts and eliminates the intermediate cache or buffer components from the neural network processor architecture. By using a crossbar circuit that directly connects memory to processing units, the system removes the need for separate cache structures, thereby reducing area consumption while maintaining memory access flexibility through the crossbar's routing capabilities
Solution Approach 2:
The crossbar circuit serves multiple functions simultaneously: it enables unaligned memory access, provides data routing between memory and processing units, and eliminates the need for dedicated cache structures. This multi-functionality allows the system to achieve memory access flexibility without requiring additional specialized components that would increase area consumption
2Adaptability or versatility
If intermediate cache or buffer components are used to enable unaligned memory access, then memory access flexibility is improved, but power consumption increases
Solution Approach 1:
The patent removes intermediate cache or buffer components that are power-consuming elements. By directly connecting memory to processing units through a crossbar circuit, the system eliminates the power overhead associated with maintaining and managing intermediate storage structures, thereby reducing overall power consumption while preserving memory access flexibility
Solution Approach 2:
The crossbar circuit performs multiple functions including enabling unaligned access and routing data directly between memory and processing units without requiring separate cache management logic. This consolidation reduces the power consumption associated with maintaining intermediate storage structures while preserving adaptability
3Device complexity
If CPU is used for all neural network operations, then system simplicity is maintained, but bandwidth consumption and power usage increase significantly
Solution Approach 1:
The patent segments the processing workload by introducing specialized neural processing units that handle neural network operations separately from the general-purpose CPU. This segmentation allows the CPU to maintain simplicity while dedicated hardware units handle power-intensive neural computations, thereby reducing overall power consumption without significantly increasing system complexity
Solution Approach 2:
The crossbar circuit acts as an intermediary that enables efficient data transfer between memory and neural processing units without involving the CPU. This mediator structure allows neural network operations to proceed with reduced CPU bandwidth consumption and lower power usage while maintaining relative system simplicity
Data Source
AI summary
Embodiments of the present disclosure relate to an unaligned memory access in a neural processor circuit. The neural processor circuit includes a crossbar circuit and a neural engine circuit coupled to the crossbar circuit. During each operating cycle of the neural processor circuit, the crossbar circuit receives a portion of input data, and re-aligns or bypasses the portion of input data. The neural engine circuit receives at least a portion of the re-aligned or bypassed portion of the input data, and performs a convolution operation on the received portion of re-aligned or bypassed portion of input data to generate output data.