AI Inference Device Using CNN ICs for Local Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI inference computing devices are limited by slow computational speed and high costs, making them impractical for processing large amounts of imagery data, and prior cloud solutions compromise data privacy and processing speed due to long-distance data transmission.
Innovation Solution
An AI inference computing device featuring a printed circuit board with CNN-based integrated circuits, wireless communication, and memory modules, allowing for local processing of imagery data with pre-trained filter coefficients and classification results, enabling efficient convolutional operations and reducing reliance on cloud-based systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If cloud-based solutions are used for AI inference, then computing power is improved, but data privacy is compromised and processing speed decreases
Solution Approach 1:
The system segments the AI inference functionality by separating the deep learning model (stored locally in non-volatile memory) from the processing engine (CNN-based IC), enabling local execution without cloud dependency. This segmentation allows the device to maintain computing power while eliminating transmission delays and privacy concerns.
Solution Approach 2:
The patent introduces an intermediary layer (the CNN-based IC with embedded non-volatile memory) that acts as a local proxy for cloud-based AI inference. This intermediary stores pre-trained models locally and performs inference computations on-device, mediating between the need for powerful computation and the requirement for fast, private processing.
2Power
If cloud-based solutions are used for AI inference, then computing power is improved, but data privacy is compromised
Solution Approach 1:
The system segments data handling by keeping sensitive input data and pre-trained models entirely within the local device boundary. The CNN-based IC processes data locally without transmitting it to external cloud servers, thus segmenting the computation from external networks and eliminating privacy exposure risks.
Solution Approach 2:
The device becomes self-sufficient for AI inference by embedding both the deep learning model and processing engine locally. It serves its own computational needs without external cloud dependency, performing inference privately on-device and eliminating the need to share data with external parties.
3Adaptability or versatility
If software-based CNN solutions are used, then implementation flexibility is improved, but computational speed and cost-effectiveness deteriorate
Solution Approach 1:
The patent replaces the mechanical/software-based CNN implementation with a dedicated hardware circuit (CNN-based IC). This substitution transforms the processing from general-purpose software execution to specialized hardware circuitry, dramatically improving computational speed while maintaining adaptability through programmable control logic.
Solution Approach 2:
The system changes the fundamental parameter of implementation from software to hardware. By transitioning from software-based CNN to hardware-based CNN IC, the patent achieves orders-of-magnitude improvement in computational speed and energy efficiency while preserving flexibility through configurable circuit design.
4Productivity
If dedicated CNN hardware is used, then computational speed is improved, but device complexity increases
Solution Approach 1:
The patent merges multiple functions into a single integrated CNN-based IC chip: the convolutional neural network processing engine, non-volatile memory for model storage, and control logic are combined in one device. This merging reduces overall system complexity by eliminating separate components while maintaining high computational speed.
Solution Approach 2:
The CNN-based IC is designed as a universal processing engine that can execute different deep learning models and operations. By creating a multi-functional hardware core that handles various CNN architectures and tasks, the patent reduces the need for multiple specialized devices, thereby managing complexity while delivering high performance.
Data Source
AI summary
An artificial intelligence inference computing device contains a printed circuit board (PCB) and a number of electronic components mounted thereon. Electronic components include a wireless communication module, a controller module, a memory module, a storage module and at least one cellular neural networks (CNN) based integrated circuit (IC) configured for performing convolutional operations in a deep learning model for extracting features out of input data. Each CNN based IC includes a number of CNN processing engines operatively coupled to at least one input/output data bus. CNN processing engines are connected in a loop with a clock-skew circuit. Wireless communication module is configured for transmitting pre-trained filter coefficients of the deep learning model, input data and classification results.