AI Model Block Replacement for Inference During Download
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Upgrading AI models at user devices requires significant storage space, delaying availability of new functionalities and potentially introducing reliability or accuracy issues due to the need for complete download and deployment, which can impact other device functions.
Innovation Solution
A method for continuous deployment of AI models by gradually replacing current blocks with replacement blocks during download, allowing inference to be processed using received sequential blocks, reducing storage requirements and enabling access to the new model before complete download.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complete AI model download and deployment is required before use, then model reliability and accuracy are ensured, but storage space requirements increase and functionality availability is delayed
Solution Approach 1:
The AI model is divided into multiple blocks that can be downloaded and processed sequentially. The system processes inference requests using partial blocks as they become available, rather than waiting for the complete model. This segmentation allows the model to be deployed incrementally, reducing the time until functionality becomes available while maintaining reliability through continuous processing of available blocks.
Solution Approach 2:
The system performs preliminary actions by downloading and storing model blocks in advance, then immediately begins processing inference requests with the available blocks. This preliminary preparation of model components enables faster functionality availability without compromising the eventual complete model deployment, as blocks are ready before full download completes.
2Productivity
If complete AI model is stored before upgrade, then model functionality is available, but storage space occupied by current model cannot be reused
Solution Approach 1:
By segmenting the model into blocks, the system can delete current model blocks as replacement blocks are downloaded, rather than waiting for complete model replacement. This enables continuous storage optimization during the upgrade process, reducing the total storage space required compared to storing complete models simultaneously.
Solution Approach 2:
The system discards current model blocks progressively as replacement blocks are downloaded and processed. This incremental discarding and recovering of storage space allows efficient use of available storage during model upgrades, freeing up space sooner than traditional complete-download approaches.
3Quantity of substance
If sequential block processing is implemented, then storage requirements are reduced, but inference processing complexity increases
Solution Approach 1:
The system dynamically adapts its processing behavior based on the number of available model blocks. As blocks are downloaded and become available, the system progressively increases the complexity of processing by incorporating more blocks into inference requests. This dynamic approach manages processing complexity according to storage availability, rather than requiring full model processing capacity from the start.
Data Source
AI summary
A method and apparatus for gradual inference during continuous deployment of a replacement IA model replacing a current AI model is provided. Replacement blocks of the replacement AI model are provided to the computing device by an AI model provider directly or via one or more network element associated therewith to replace the current AI model. A computing device having the current AI model gradually receives replacement blocks and in response, deletes current blocks of the current AI model. Inference request can be processed gradually at the computing device as soon as at least the first replacement block is received thereat using received sequential replacement blocks to obtain a partial inference that is subsequently jointly processed at the AI model provider or one or more network element associated therewith to obtain an inference result, thereby enabling access to the replacement AI model for inference during its download at the computing device.