AI Data Transfer Integrity Checks for Parallel Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity and size of data transmission and inference work in AI systems, particularly for Large Language Models (LLM), lead to data corruption issues due to error propagation and difficulty in identifying error occurrence locations, especially in parallel data processing across devices or within device memories.
Innovation Solution
A data processing method that includes adding detection data to transmission data, allowing for error detection by comparing receiving side data with the detection data to identify and correct errors in communication operations between devices or within device memories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted between devices or within device memories for AI inference operations, then processing capability and productivity are improved, but data corruption problems occur due to error propagation and transmission errors
Solution Approach 1:
The patent applies preliminary action by generating detection data (such as checksums or hash values) from source data before the DMA operation occurs. This detection data is stored in advance and later used to verify the integrity of destination data after transmission, enabling error detection before processing continues with potentially corrupted data.
Solution Approach 2:
The patent implements feedback by comparing the detection data generated from source data with detection data regenerated from destination data after the DMA operation. This feedback mechanism identifies whether data corruption occurred during transmission, allowing the system to detect and correct errors or retry the operation.
2Productivity
If parallel data processing is performed across multiple devices to increase throughput, then productivity is improved, but difficulty in identifying error occurrence locations increases
Solution Approach 1:
The patent applies segmentation by dividing the data transmission verification process into distinct segments: generating detection data at the source, transmitting data through DMA operations, regenerating detection data at the destination, and comparing the two. This segmented approach allows precise identification of where errors occur in the transmission chain, even in complex multi-device parallel processing scenarios.
3Reliability
If detection data is added to transmission data for error detection, then data integrity is improved, but device complexity increases
Solution Approach 1:
The patent uses detection data as an intermediary element that mediates between the source data and destination data. This intermediary serves as a verification mechanism without requiring complex real-time monitoring or control systems. The detection data (such as checksums) acts as a simple mathematical representation that can be generated and compared using standard processing operations, avoiding the need for specialized hardware or complex protocols.
Data Source
AI summary
A data processing method using a plurality of devices for Artificial Intelligence (AI) task, the method comprising: transmitting transmission data including detection data, determining whether an error has occurred in the transmission data based on the detection data, and when the error has not occurred in the transmission data, performing an operation task based on the transmission data, wherein receiving side detection data is derived based on the transmission data, and whether the error has occurred in the transmission data is determined based on the receiving side detection data and the detection data included in the transmission data.