Distributed Inference Model Partitioning for Seamless Updates

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Inference models consume excessive computing resources and increase computational overhead in distributed environments, and updating these models disrupts continuous inference generation, posing challenges for seamless changeovers and maintaining redundancy.

Innovation Solution

The system partitions inference models into portions and distributes them across multiple data processing systems, using an update framework to ensure seamless changeovers and maintain redundancy levels, with metadata management to facilitate independent operation and efficient resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If inference models are hosted and executed by a single data processing system, then the system can maintain simplicity in model management, but the computing resource consumption increases excessively and computational overhead increases

Engineering Contradiction:

Improvemodel management complexityVSAvoidcomputing resource consumption

Core Design Contradiction:

Device complexityVSUse of energy by moving object

Solution Approach 1:

The inference model is partitioned into multiple portions and distributed across multiple data processing systems. Each system hosts and executes specific portions of the model, dividing the computational workload and reducing the resource consumption burden on any single system while maintaining overall model functionality.

Inventive Principle:

Principle #1Segmentation

2Use of energy by moving object

If inference models are updated in a distributed environment, then the system can reduce computing resource consumption through distribution, but updating the model disrupts continuous inference generation and makes seamless changeover difficult

Engineering Contradiction:

Improvecomputing resource consumptionVSAvoidcontinuous inference generation

Core Design Contradiction:

Use of energy by moving objectVSReliability

Solution Approach 1:

Updated portions of the inference model are prepared and validated in advance before being deployed to the distributed data processing systems. This preliminary preparation ensures that updates can be applied without disrupting ongoing inference operations, as the new portions are ready to seamlessly replace the old ones.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system maintains continuous inference generation during model updates by ensuring that updated model portions are ready and validated before deployment. The update framework coordinates the replacement of model portions across distributed systems in a way that preserves continuous inference capability without interruption.

Inventive Principle:

Principle #20Continuity of useful action

3Use of energy by moving object

If inference models are partitioned and distributed across multiple data processing systems, then computing resource consumption is reduced, but the system complexity for managing model updates and maintaining redundancy increases

Engineering Contradiction:

Improvecomputing resource consumptionVSAvoidupdate management complexity

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

An update framework acts as an intermediary between model developers and the distributed data processing systems. This framework manages the complex tasks of partitioning, distributing, validating, and coordinating model updates across multiple systems, thereby reducing the management burden and complexity for individual systems while maintaining efficient resource utilization.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Reliability

If redundancy levels are maintained in a distributed inference model environment, then the system can ensure continuous operation and reliability, but the device complexity for managing redundant model portions increases

Engineering Contradiction:

Improvecontinuous operationVSAvoidredundancy management complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Redundant portions of the inference model are segmented and distributed across multiple data processing systems. Each system hosts specific redundant model portions, and the update framework manages their coordination, allowing the system to maintain reliability through redundancy while distributing the management complexity across the distributed architecture rather than concentrating it in a single system.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11989172B2System and method for managing transition between inference models across multiple data processing systems

Publication Date: 2024.05.21 DELL PROD LP

AI summary

Methods and systems for seamlessly changing over between inference models are disclosed. The inference models may be distributed across multiple data processing systems. Provide a seamless changeover, updated inference models and original inference models may be managed in accordance with an update framework. The update framework may ensure that the original inference model continues to operate until all of the portions of the updated inference model are in place and ready to operate. During the update process, the update framework may ensure that redundancy goals continue to be met so that failures of some of the data processing systems are not be fatal to continued operation of at least one of the inference models, such as the original or updated inference model.