Split AI Inference Packetization for Partial and Intermediate Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI model splitting architectures fail to adequately transfer intermediate data required for consolidated inference results between endpoints, leading to incomplete calculations.
Innovation Solution
A method for configuring an AI model split between a wireless transmit/receive unit (WTRU) and a network, involving determining partial results data, combining them with intermediate data in a single data packet, and transmitting based on a sub-model configuration, with support for tensor structure elements and pre-processing of input data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If intermediate data is transmitted separately from partial results data, then data transmission completeness is improved, but transmission overhead and complexity increase
Solution Approach 1:
The patent combines partial results data and intermediate data into a single data packet for transmission. The data packet includes both the partial results from the first inference and the intermediate data required for the second inference, eliminating the need for separate transmission protocols and reducing overall transmission complexity while ensuring both data types are delivered together.
Solution Approach 2:
The data packet structure is designed to serve multiple functions simultaneously: it carries partial results data for consolidation, transports intermediate data for subsequent inference, and maintains compatibility with existing transmission protocols. This multi-functional design reduces the need for separate specialized transmission mechanisms.
2Productivity
If AI model inference is split between WTRU and network, then processing efficiency is improved, but data coordination complexity increases
Solution Approach 1:
The data packet acts as an intermediary structure that bridges the WTRU and network components. It encapsulates both partial results and intermediate data in a standardized format that can be seamlessly exchanged between the distributed inference components, simplifying the coordination complexity inherent in split inference architectures.
Solution Approach 2:
The AI model inference process is segmented into distinct parts executed at different locations (WTRU and network), with the data packet providing the coordination mechanism. This segmentation allows parallel processing while maintaining data integrity through the structured packet format that clearly delineates partial results from intermediate data.
3Loss of energy
If only intermediate data is transmitted between endpoints, then bandwidth usage is reduced, but inference result accuracy deteriorates
Solution Approach 1:
By merging partial results data and intermediate data into a single transmission packet, the system ensures that both critical data types are delivered together. This approach maintains inference accuracy by providing the receiving endpoint with both the intermediate data needed for continuation and the partial results needed for consolidation, without requiring separate transmissions that would increase bandwidth usage.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
In an embodiment, a method, implemented in a wireless transmit/receive unit, WTRU, comprises receiving a first message comprising first information indicating a configuration of splitting an AI model between the WTRU and a network; determining, based on the first message, an AI sub model configuration comprising a first sub model configuration for the WTRU and a second sub model configuration for the network; determining, from one or more nodes of the first sub model, one or more partial results data of inferenced input data; determining transmission of the one or more partial results data to the network, based on the sub AI model configuration; combining the one or more partial results data with intermediate data in a single data packet to be transmitted, based on the sub AI model configuration; and transmitting, to the network, a second message comprising the single data packet.