AI Inference Machine with PCIe Switch Task Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems face challenges in optimizing power usage and performance due to the complexity of AI and ML tasks, which often require computationally intensive processing, leading to instability and space constraints as more processing units are added.
Innovation Solution
A machine learning system with a central processor that creates AI-specific tasks and distributes them among multiple processing units via a PCIe switch, optimizing task distribution and power management using a multi-layered PCB and Edge TPUs for efficient AI computing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If more processing units are used to improve performance, then computational capability increases, but form factor and heat dissipation become problematic
Solution Approach 1:
The system segments processing units into multiple computing devices (first computing device with first processing units, second computing device with second processing units). Each device handles specific AI tasks independently, distributing the heat generation across separate physical units rather than concentrating it in a single processor, thus improving heat dissipation while maintaining computational capability.
Solution Approach 2:
The patent transitions from a single-processor vertical scaling approach to a multi-device horizontal scaling architecture. By distributing processing units across multiple computing devices connected via PCIe switches, the system expands the computational architecture across additional spatial dimensions, allowing better heat management while maintaining or increasing overall computational power.
2Productivity
If more processing units are added to increase computational power, then AI task processing improves, but system space requirements increase
Solution Approach 1:
The patent merges multiple computing devices into a unified AI processing system through PCIe switch interconnections. By combining first computing device, second computing device, and central processor into an integrated architecture, the system achieves high computational power for AI tasks while maintaining a compact form factor through standardized connection interfaces and shared system resources.
3Adaptability or versatility
If general-purpose processing units are used to handle varied workloads, then system versatility is maintained, but power and performance optimization becomes difficult
Solution Approach 1:
The patent applies local quality by assigning specific AI tasks to specific computing devices based on their capabilities and current load. The central processor intelligently distributes AI tasks to first or second computing devices depending on task requirements, allowing each processing unit to be optimized for its specific function while maintaining overall system versatility through the centralized task distribution architecture.
Data Source
Figure 1
Figure 2A~2B
Figure 2C
AI summary
A machine learning system comprising a power source, a central processor, and a first computing device. The central processor is configured to receive artificial-intelligence-specific models and creates at least one artificial-intelligence-specific task based on the artificial-intelligence-specific model. The first computing device is connected to the power source to receive power and the central processor to receive artificial-intelligence-specific tasks. The first computing device including a first base, a plurality of first processing units, and a first switch. The first base including a plurality of first traces. The first processing units are coupled with the first traces and configured to perform artificial-intelligence-specific tasks. The first switch is coupled with the first traces and connected to the first processing units via the first traces, wherein the first switch receives and distributes the artificial-intelligence-specific tasks amongst the first processing unit.