Distributed Inference Model Deployment Under Communication Disruptions

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems face challenges in efficiently managing inference model execution across distributed environments due to resource constraints and communication disruptions, affecting the speed and reliability of inference generation for downstream consumers.

Innovation Solution

The system partitions inference models into portions and distributes them across multiple data processing systems, dynamically adjusting deployment paths based on communication system information and consumer requirements to ensure compliance with inference generation thresholds.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the inference model is hosted by a single data processing system, then the system complexity is low, but the computing resources are insufficient to meet the speed and reliability requirements of downstream consumers

Engineering Contradiction:

Improveinference generation reliabilityVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The inference model is partitioned into multiple portions and distributed across multiple data processing systems. Each system hosts a subset of the model portions, enabling parallel processing and improving inference generation reliability while meeting downstream consumer requirements through distributed architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple data processing systems are combined to form a distributed inference model hosting environment. The systems work together as a unified infrastructure, with the inference model manager coordinating resource allocation and model deployment across all systems to achieve improved reliability and resource utilization.

Inventive Principle:

Principle #5Merging (Combining)

2Productivity

If the inference model is deployed across multiple data processing systems, then the computing resources are sufficient to meet consumer requirements, but the communication system disruptions affect inference generation speed

Engineering Contradiction:

Improveinference generation speedVSAvoidcommunication system reliability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system dynamically adjusts the deployment of inference model portions across data processing systems based on real-time communication system conditions. The inference model manager monitors communication reliability and reassigns model portions to alternative systems when disruptions occur, maintaining inference generation speed despite communication variability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes deployment parameters such as the location and distribution of inference model portions based on communication system status. When communication reliability deteriorates, the system modifies the deployment configuration to minimize the impact on inference generation speed, adapting to changing conditions in real-time.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If the inference model is dynamically redistributed across data processing systems, then the inference generation requirements are met, but the deployment complexity increases

Engineering Contradiction:

Improvedeployment adaptabilityVSAvoiddeployment complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The inference model manager implements continuous monitoring of communication system information and consumer requirements, using this feedback to dynamically adjust the deployment of inference model portions. The system evaluates current deployment effectiveness and reassigns model portions to optimize performance while adapting to changing conditions.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs automated self-adjustment of inference model deployment without requiring manual intervention. The inference model manager autonomously monitors system conditions, evaluates deployment effectiveness, and redistributes model portions across data processing systems based on real-time requirements and communication status.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS12619892B2System and method for managing inference model performance through proactive communication system analysis

Publication Date: 2026.05.05 DELL PROD LP

AI summary

Methods and systems for managing execution of inference models hosted by data processing systems are disclosed. To manage execution of inference models hosted by data processing systems, a system may include an inference model manager and any number of data processing systems. The inference model manager may communication system data for the communication system linking the data processing systems. The inference model manager may use the communication system data to determine whether the communication system meets inference generation requirements of the downstream consumer. If the communication system does not meet inference generation requirements of the downstream consumer, the inference model manager may obtain an inference generation plan to return to compliance with the inference generation requirements of the downstream consumer.