AI Model Split Point Selection Under Bandwidth and Power Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of determining an optimal split point for an artificial intelligence (AI)/machine learning (ML) model to improve execution efficiency is unresolved, particularly due to limited computing power, memory, and high power consumption in terminal devices, and excessive bandwidth usage when offloading to edge servers.
Innovation Solution
A method to determine a split point based on network state information, power consumption, and AI model attributes, ensuring the split point meets performance indicators such as transmission rate, channel state, and power consumption, allowing parts of the model to be executed on the terminal device and server efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the AI/ML model is offloaded to an edge server to enable processing, then the terminal device's computing power limitation is resolved, but excessively high uplink transmission bandwidth is occupied
Solution Approach 1:
The patent divides the AI/ML model into multiple parts and distributes them across different computing nodes (terminal device and edge server). This segmentation allows the system to balance computing requirements with bandwidth constraints by executing some model layers locally and others remotely, rather than offloading the entire model.
Solution Approach 2:
The patent enables different parts of the system to have different functional qualities - the terminal device executes parts of the model locally using its available computing resources, while the edge server handles other parts. This local quality approach optimizes resource utilization by matching computational tasks to appropriate execution locations based on local capabilities and constraints.
2Quantity of substance
If the AI/ML model is executed locally on the terminal device, then uplink transmission bandwidth is saved, but the terminal device cannot support real-time computing due to limited computing power
Solution Approach 1:
The patent segments the AI/ML model into multiple executable parts distributed between terminal device and edge server. This segmentation enables the system to achieve real-time computing by parallel execution of model parts across multiple nodes, compensating for the terminal device's limited local computing power while reducing bandwidth requirements compared to full offloading.
Solution Approach 2:
The patent merges the computing capabilities of the terminal device and edge server into a unified distributed computing system. By combining local execution with remote processing, the system achieves real-time computing performance that neither component could provide alone, while optimizing bandwidth utilization.
3Productivity
If the AI/ML model is split into multiple parts for distributed execution, then resource utilization is improved, but the complexity of determining optimal split points increases
Solution Approach 1:
The patent implements feedback mechanisms where the system evaluates the performance and resource consumption at different split points, then uses this information to determine optimal segmentation. The feedback loop allows the system to adaptively identify split points that maximize execution efficiency while minimizing complexity, based on actual runtime measurements and resource availability.
Solution Approach 2:
The patent changes key parameters such as model partitioning granularity, execution location, and data transmission frequency to optimize the balance between execution efficiency and system complexity. By dynamically adjusting these parameters based on network conditions, device capabilities, and task requirements, the system achieves high productivity without excessive complexity.
Data Source
AI summary
A method and an apparatus for determining a split point are disclosed. The method includes: obtaining network state information, first power consumption information of a terminal device, and artificial intelligence model attribute information; and determining at least one split point from a plurality of split points based on the network state information, the first power consumption information, and the artificial intelligence model attribute information. According to the foregoing method, a split point is determined based on the network state information, the first power consumption information, the artificial intelligence model attribute information, or other information, so that the determined at least one split point can meet performance indicators such as a transmission rate, channel state information, and power consumption.