Reconfigurable Neural CPU for Binary Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional heterogeneous architectures for embedded systems face challenges in optimizing end-to-end performance due to power and cost constraints, with neural network accelerators being underutilized and costly, leading to core under-utilization and limited performance improvements in resource-constrained devices.
Innovation Solution
A reconfigurable Neural CPU (NCPU) architecture that transitions between binary neural network accelerator mode and CPU mode, leveraging existing logic and memory to support both neural network inference and general-purpose CPU operations, achieving efficient core utilization and cost reduction by reusing SRAM as a data cache and implementing customized RISC-V instructions for seamless mode switching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a heterogeneous architecture with separate neural network accelerators is used, then machine learning inference capability is improved, but device area and cost increase
Solution Approach 1:
The patent implements a unified NCPU architecture where the same hardware resources (execution units, memory structures) serve both traditional CPU operations and binary neural network inference. The execution units can dynamically switch between RISC-V instruction execution and BNN MAC operations, while memory structures serve as either data cache or weight/image memory, eliminating the need for separate accelerator hardware and reducing overall device area.
Solution Approach 2:
The patent merges the CPU and neural network accelerator into a single integrated NCPU core. The control unit, execution units, and memory structures are combined into one unified architecture that can operate in either CPU mode or BNN accelerator mode, consolidating previously separate components into a single compact unit that reduces device area while maintaining both functionalities.
2Adaptability or versatility
If a heterogeneous architecture with separate neural network accelerators is used, then machine learning inference capability is improved, but energy consumption increases
Solution Approach 1:
The unified NCPU architecture allows the same execution units and memory to be shared between CPU and BNN operations. This eliminates redundant hardware in separate accelerators, reducing overall energy consumption. The dynamic reconfiguration capability ensures that energy-intensive components are only activated when needed for BNN inference, optimizing energy efficiency while maintaining ML capability.
Solution Approach 2:
By merging CPU and accelerator functions into one NCPU, the patent eliminates the energy overhead of maintaining separate accelerator hardware. The shared memory structures and execution units reduce redundant power consumption, and the unified control logic minimizes the energy required for mode switching and data transfer between separate components.
3Productivity
If a heterogeneous architecture is used, then specialized neural network processing is improved, but core utilization decreases
Solution Approach 1:
The NCPU employs dynamic reconfiguration capabilities that allow the execution units and memory structures to switch between CPU and BNN accelerator modes based on workload requirements. This dynamic adaptability ensures high core utilization by activating the appropriate functional mode when needed, preventing idle hardware resources while maintaining specialized neural network processing efficiency.
Solution Approach 2:
The unified execution units can perform both RISC-V instructions and BNN MAC operations, ensuring that the same hardware resources are fully utilized across different workload types. This multi-functionality eliminates idle time in specialized accelerator components by allowing them to serve general-purpose computing when BNN inference is not required, thereby improving overall core utilization.
4Adaptability or versatility
If memory is reconfigured between image/weight memories and data cache, then adaptability is improved, but switching complexity increases
Solution Approach 1:
The memory structures in the NCPU are designed with dynamic reconfiguration capabilities, allowing the same physical memory to be allocated as image memory, weight memory, or data cache based on the current operational mode. This dynamic memory management is controlled by mode-switching logic that automatically configures memory allocation when transitioning between CPU and BNN modes, providing flexibility without requiring complex manual reconfiguration.
Solution Approach 2:
The NCPU architecture pre-establishes memory allocation schemes for both CPU and BNN modes during design. When mode switching occurs, the memory structures are pre-configured to their appropriate functions through predefined control signals, eliminating the need for complex runtime reconfiguration logic and reducing switching complexity while maintaining adaptability.
Data Source
AI summary
Systems and methods for a unified reconfigurable neural central processing unit is provided. In one aspect, a neural central processing unit is in communication with a memory, wherein the neural central processing unit is configured to transition between a binary neural network accelerator mode and a central processing unit mode, wherein, in the binary neural network accelerator mode, the memory is configured as an image memory and weight memories, wherein, in the central processing unit mode, the memory is reconfigured, from the image memory and the weight memories, to a data cache.