AI Model Split Point Selection for Edge Bandwidth and Power Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI/ML models require high computing power, memory, and power consumption, and their execution is limited by terminal device capabilities, leading to inefficient data processing and excessive bandwidth usage when offloaded to edge servers.
Innovation Solution
A method to determine a split point for AI/ML models based on network state, power consumption, and model attributes to optimize execution efficiency by distributing model parts between terminal devices and application servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If the AI/ML model is offloaded to an edge server, then computing power requirements and power consumption are reduced, but uplink transmission bandwidth is excessively occupied
Solution Approach 1:
The patent divides the AI/ML model into multiple segments and distributes them across different nodes (terminal device and edge server). By determining optimal split points based on network state, the system segments the model execution to balance local processing and cloud processing, thereby reducing the amount of data transmitted over uplink while still benefiting from reduced power consumption requirements at the terminal.
2Speed
If the AI/ML model is executed locally on the terminal device, then real-time processing is enabled, but computing power and memory requirements exceed device capabilities
Solution Approach 1:
The model is segmented into multiple parts that can be executed locally or remotely. The terminal device executes segments that fit within its computing capabilities while offloading heavier segments to the edge server, achieving a balance between processing speed and computing power requirements.
Solution Approach 2:
The system dynamically determines split points based on real-time network state information, allowing the division of model execution between terminal and server to adapt to changing conditions. This dynamic adjustment optimizes the balance between local processing speed and remote processing capabilities.
3Ease of operation
If the AI/ML model is completely offloaded to the edge server, then terminal device resources are preserved, but network transmission overhead increases significantly
Solution Approach 1:
Instead of complete offloading, the model is segmented with some parts executed locally and others remotely. This reduces the volume of data that needs to be transmitted to and from the edge server, as intermediate computations are performed locally at the terminal device.
4Loss of time
If more model layers are executed locally, then latency is reduced, but power consumption and computing power requirements increase
Solution Approach 1:
The system dynamically adjusts the number of model layers executed locally based on real-time power consumption information and network state. When power availability is high and network conditions are poor, more layers are executed locally to reduce latency. When power is constrained or network conditions are good, fewer layers are executed locally to save power.
Solution Approach 2:
The system changes the parameter of model execution distribution based on measured power consumption and network conditions. By adjusting the split point position, the system optimizes the trade-off between latency reduction (more local execution) and power consumption (less local execution).
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method and an apparatus for determining a split point are disclosed. The method includes: obtaining network state information, first power consumption information of a terminal device, and artificial intelligence model attribute information; and determining at least one split point from a plurality of split points based on the network state information, the first power consumption information, and the artificial intelligence model attribute information. According to the foregoing method, a split point is determined based on the network state information, the first power consumption information, the artificial intelligence model attribute information, or other information, so that the determined at least one split point can meet performance indicators such as a transmission rate, channel state information, and power consumption. In this way, a part, obtained through division based on the split point, of the artificial intelligence model can be executed on the terminal device, and another part is executed on an application server, so that an artificial intelligence model obtained through division can better adapt to terminal devices and application servers with different capabilities, and better meet a requirement of an artificial intelligence model computing service.