Feature Vector Fusion for Faster CPU-GPU AI Reasoning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI model reasoning systems face delays due to high data transmission and compatibility issues between CPU and GPU, leading to slow reasoning speeds, especially in real-time applications like dialogue and recommendation systems.
Innovation Solution
A data processing method that generates an initial feature vector on a first processor for different types of data, followed by parallel feature extraction and fusion on a second processor, reducing data transmission and eliminating the need for separate quantization operations on both processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transmitted between CPU and GPU for model reasoning, then the system can process different types of data using specialized processors, but the reasoning speed decreases due to high data transmission overhead
Solution Approach 1:
The patent segments the data processing workflow into distinct stages: feature extraction on CPU, quantization on GPU, and reasoning on GPU. By dividing the task and keeping processed data on the GPU, it eliminates repeated data transmissions between CPU and GPU, thus improving reasoning speed while maintaining the ability to process different data types through specialized processors
Solution Approach 2:
The patent performs feature extraction as a preliminary action on the CPU before transferring data to the GPU. This preprocessing step converts raw data into feature vectors that are then quantized and kept on the GPU for subsequent reasoning operations, reducing the need for repeated CPU-GPU data exchanges and improving overall reasoning speed
2Reliability
If separate quantization operations are performed on both CPU and GPU, then each processor can optimize for its data types, but the device complexity increases
Solution Approach 1:
The patent merges the quantization operation to be performed exclusively on the GPU after feature extraction from the CPU. This consolidation eliminates the need for separate quantization operations on both processors, reducing device complexity and data transmission overhead while maintaining processing accuracy through the GPU's specialized architecture
3Loss of information
If large amounts of data are transmitted between processors, then complete information is available for reasoning, but the reasoning delay increases
Solution Approach 1:
The patent extracts only the essential feature vectors from the raw data during the feature extraction phase on the CPU. By taking out only the necessary information rather than transmitting complete raw data, it reduces data transmission volume and reasoning delay while maintaining data completeness needed for accurate model reasoning on the GPU
Data Source
AI summary
A data processing method including generating, by a first processor, an initial feature vector in response to a target reasoning task. The initial feature vector includes first and second feature bits corresponding to first and second data, respectively, in the target reasoning task. The and the second data are of different types. The method further includes determining, by a second processor, one or more first data features corresponding to the first data and one or more second data features corresponding to the second data, updating, by the second processor, the first feature bits and the second feature bits using the one or more first data features and the one or more second data features, respectively, to obtain a target feature vector, and performing model reasoning based on the target feature vector to obtain a target reasoning result corresponding to the target reasoning task.