Split AI Inference Result Aggregation Across Distributed Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing collaborative Artificial Intelligence (AI) systems face challenges in efficiently transmitting and combining partial results across distributed devices due to the lack of effective methods for intermediate data exchange and result aggregation.
Innovation Solution
A method for performing inference on a split AI model in a first device, transmitting intermediate data and identifiers to a second device, and aggregating results using corresponding identifiers to facilitate seamless collaboration between devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If intermediate data and results are transmitted between distributed devices for collaborative AI inference, then the AI model can be split across multiple devices to improve processing capability and distribution, but the complexity of data transmission and result aggregation increases
Solution Approach 1:
The patent introduces an intermediary device (second device) that receives intermediate data from the first device, performs additional inference operations, and then aggregates results. This intermediary acts as a mediator that simplifies the overall system architecture by centralizing the coordination of distributed inference tasks, thereby managing complexity while enabling distributed processing across multiple devices.
2Productivity
If partial results are transmitted between devices for aggregation, then collaborative inference can be performed on split AI models, but transmission overhead and communication requirements increase
Solution Approach 1:
The patent segments the AI model into multiple parts distributed across different devices, where each device performs inference on its assigned segment. The intermediate data and partial results are then transmitted and aggregated. This segmentation enables parallel processing and distributed computation, improving overall productivity while managing transmission overhead by only sending necessary intermediate representations rather than complete data sets.
3Reliability
If identifiers are used to associate and aggregate results from multiple devices, then result combination becomes more organized and accurate, but the data structure and processing complexity increases
Solution Approach 1:
The patent implements a feedback mechanism where identifiers are attached to intermediate data and transmitted between devices. The receiving device uses these identifiers to correctly associate and aggregate partial results from multiple sources. This feedback loop ensures that results are accurately matched to their corresponding input data, improving reliability while the identifier system provides a simple yet effective method for tracking data provenance through the distributed processing pipeline.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Procedures, methods, architectures, apparatuses, systems, devices, and computer program products for distributed artificial intelligence in which, in an embodiment, a first device performs inference of a first part of a split artificial intelligence, AI, model to obtain a first result and intermediate data for use in inference using a second part of the split AI model, transmits, to a second device, the intermediate data and an identifier, and information representative of the first result to be combined with a second result resulting from the inference using the second part of the AI model and the identifier. The second device performs inference of the second part of the split artificial intelligence, AI, model to obtain a second result, associates the second result with the first identifier, and aggregates the first result and the second result using the first identifier and the second identifier.