Neural Network Sub-networks for Elastic Computing Power Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI inference systems have limited flexibility in adjusting computing power, which restricts their ability to dynamically allocate resources based on the need of prediction tasks, especially in scenarios requiring varying levels of resource utilization.
Innovation Solution
A neural network model comprising multiple sub-networks (n>2) is deployed on a computing platform, where a computing power coefficient determines the allocation of hardware resources, allowing for the dynamic selection and activation of sub-networks to adjust computing power levels, enabling flexible scaling of computing power from 1/n to n.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If computing power is fixed in AI inference systems, then system stability is maintained, but flexibility in adjusting computing power is limited
Solution Approach 1:
The neural network model is divided into multiple sub-networks (first sub-network, second sub-network, third sub-network, etc.) that can be independently selected and activated. This segmentation allows the system to flexibly adjust computing power by activating different combinations of sub-networks based on task requirements, rather than running the entire model or a fixed configuration.
Solution Approach 2:
The system dynamically adjusts the computing power configuration by selecting different subsets of sub-networks based on real-time task demands. The computing power coefficient determines which sub-networks are activated, enabling the system to transition between different operational states (full computing power, partial computing power, or minimal computing power) as needed.
2Productivity
If computing power is dynamically adjusted based on task needs, then resource allocation efficiency is improved, but the mechanism for adjustment becomes more complex
Solution Approach 1:
The system uses a computing power coefficient as a key parameter to control the activation state of different sub-networks. By changing this parameter, the system can smoothly transition between different computing power levels. The coefficient determines the proportion of sub-networks to activate, providing a simple yet effective mechanism for dynamic resource allocation without requiring complex control logic.
3Adaptability or versatility
If only a few levels of computing power are available, then system simplicity is maintained, but the space for elastically adjusting computing power is limited
Solution Approach 1:
The neural network is segmented into multiple independent sub-networks that can be selectively activated. This segmentation creates multiple discrete levels of computing power (e.g., 1/4, 2/4, 3/4, or 4/4 of total computing power), providing fine-grained control over resource allocation. The more sub-networks created, the more elastic the computing power adjustment becomes.
4Adaptability or versatility
If more sub-networks are created for elastic computing power adjustment, then computing power flexibility is improved, but model complexity increases
Solution Approach 1:
The model is divided into multiple sub-networks of relatively uniform structure and function. Each sub-network processes the same type of data and contributes to the final prediction result. This uniform segmentation reduces the complexity of managing diverse components while still achieving the goal of elastic computing power adjustment.
Solution Approach 2:
Multiple sub-networks are merged into a unified neural network model with a common architecture and training process. The sub-networks work together in parallel or sequence, sharing the same input and output interfaces. This merging approach allows the system to achieve complex functionality through simple, repeatable units.
Data Source
AI summary
Implementations of this specification provide prediction methods and apparatuses for adjusting computing power. One method comprises receiving a prediction request, wherein the prediction request comprises a sample to be tested, determining a computing power coefficient allocated to the prediction request, wherein the computing power coefficient indicates a proportion of hardware computing power resources allocated to the prediction request to total hardware computing power resources needed for a neural network model to run on a computing platform, determining k sub-networks inn sub-networks of the neural network model to be used for a present time based on the computing power coefficient, where n>2, and inputting the sample to be tested to the k sub-networks to obtain a prediction result.