Weighted CPU AI Execution via Sparse Matrix Simplification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI technologies require high-end hardware for computations due to inefficient matrix optimization, leading to increased costs and cooling issues, and are unable to effectively distribute procedures across CPU threads, limiting the use of low-end hardware for AI development and applications.
Innovation Solution
A method that simplifies sparse matrices used by AI models through a matrix simplification procedure and applies transformed instructions to a weighted CPU, allowing it to distribute procedures across multiple threads based on weighting values, optimizing execution time and reducing computation resources needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If high-end hardware (high-end CPUs or GPUs) is used to execute AI computations, then computation speed and processing capability are improved, but hardware cost and cooling requirements increase
Solution Approach 1:
The patent transforms the instruction set to enable the CPU to execute AI computation instructions natively, changing the execution parameter from sequential single-thread mode to parallel multi-thread mode. This allows low-end CPUs to achieve AI computation capabilities previously requiring high-end hardware, reducing hardware cost while maintaining computation speed
Solution Approach 2:
The patent segments AI computation procedures into multiple independent threads that can be executed in parallel by multiple CPU cores. By dividing the computation workload across multiple threads with different weighting values, the system achieves high computation speed on low-end hardware without requiring expensive high-end CPUs or GPUs
2Ease of operation
If the CPU executes procedures in received order without optimization, then execution simplicity is maintained, but execution time increases and low-end hardware cannot complete computations efficiently
Solution Approach 1:
The patent dynamically adjusts the execution order of procedures by assigning different weighting values to different threads. The CPU scheduler uses these weighting values to determine execution priority and timing, allowing the system to optimize execution time adaptively while maintaining ease of operation through automatic scheduling without manual intervention
Solution Approach 2:
The patent performs preliminary transformation of the instruction set before execution, encoding threading and weighting information in advance. This preliminary action allows the CPU to execute procedures in optimized parallel order without requiring complex real-time scheduling decisions during execution, reducing execution time while maintaining simplicity
3Productivity
If SWAP operations are performed frequently to hand over procedures between threads, then procedure completion is achieved, but execution speed reduces due to additional time required
Solution Approach 1:
The patent merges multiple procedure execution streams into a unified thread execution model where procedures are distributed to threads based on weighting values. This merging eliminates the need for frequent SWAP operations between threads, as procedures are assigned to appropriate threads in advance and executed continuously, maintaining both procedure completion and high execution speed
Data Source
AI summary
A method for a CPU to execute artificial intelligence related processes is disclosed. The method includes: when executing TensorFlow on an electronic device, calling a corresponding AI model of TensorFlow according to content of program codes; determining and obtaining one or multiple sparse matrixes used by the AI model in performing calculations; executing a matrix simplifying procedure to the one or multiple sparse matrixes; executing an instruction transforming procedure to an instruction set applied for the AI model; issuing an instruction to a weighted CPU of the electronic device by the AI model according to transformed instruction set; and the weighted CPU, after receiving the instruction, averagely distributing multiple procedures indicated by the AI model to each of multiple threads of the weighted CPU to be respectively executed according to a weighting value of each of the multiple procedures.