Distributed AI Platform Mesh for ML Workload Autoscaling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing AI platform services struggle to provide guaranteed autoscaling behavior and efficient machine learning capabilities in shared cluster setups due to performance issues and complex tenancy isolation requirements.

Innovation Solution

The implementation of a distributed AI platform mesh that includes a global control plane and worker planes, allowing tenants to connect their own worker planes for local management of machine learning workloads, thereby ensuring autoscaling and reducing load on the API server.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a shared cluster setup is used to provide AI platform services to multiple tenants, then resource utilization and cost efficiency are improved, but performance and autoscaling behavior cannot be guaranteed

Engineering Contradiction:

Improveresource utilizationVSAvoidautoscaling behavior guarantee

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system segments the AI platform into a control plane and multiple worker planes, where each worker plane is dedicated to specific tenants. This segmentation allows guaranteed autoscaling behavior for each tenant while maintaining overall resource utilization efficiency across the distributed architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a distributed dimension to the traditional shared cluster model by deploying worker planes across multiple locations. This dimensional change enables independent autoscaling control for each tenant while maintaining resource sharing benefits through the control plane coordination.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Device complexity

If a shared cluster setup is used to provide AI platform services to multiple tenants, then infrastructure sharing is improved, but tenancy isolation becomes complicated

Engineering Contradiction:

Improveinfrastructure sharingVSAvoidtenancy isolation

Core Design Contradiction:

Device complexityVSEase of operation

Solution Approach 1:

The system divides the infrastructure into distinct control plane and worker plane components, with worker planes being tenant-specific. This segmentation simplifies tenancy isolation by creating clear boundaries between tenants while maintaining infrastructure sharing through the centralized control plane.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The control plane acts as an intermediary between multiple tenants and the shared infrastructure. It manages resource allocation and coordination across tenant-specific worker planes, simplifying tenancy isolation while enabling infrastructure sharing without direct tenant-to-tenant complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If machine learning workloads are run in a shared cluster, then resource efficiency is improved, but latency increases due to network communication

Engineering Contradiction:

Improveresource efficiencyVSAvoidworkload latency

Core Design Contradiction:

ProductivityVSSpeed

Solution Approach 1:

The system segments workloads into those executed in the control plane versus those executed in tenant-specific worker planes. Latency-sensitive workloads are routed to local worker planes, while resource-intensive workloads utilize the shared control plane, optimizing both latency and resource efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by deploying worker planes close to tenants for latency-sensitive workloads, while maintaining centralized control for resource-efficient workloads. This creates different execution qualities based on workload characteristics, optimizing both speed and resource efficiency.

Inventive Principle:

Principle #3Local quality

4Extent of automation

If the API server handles all machine learning workloads in a shared cluster, then centralized management is improved, but server load increases

Engineering Contradiction:

Improvecentralized managementVSAvoidAPI server load

Core Design Contradiction:

Extent of automationVSPower

Solution Approach 1:

The system segments workload management between the control plane (centralized management) and worker planes (distributed execution). The API server handles only control plane workloads, while worker planes handle tenant-specific workloads, reducing API server load while maintaining centralized management for critical functions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts workload handling from the API server by implementing worker planes that independently manage tenant workloads. This extraction reduces API server load while maintaining centralized management capabilities for coordination and resource allocation.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS12307288B2Distributed AI platform mesh for machine learning workflows

Publication Date: 2025.05.20 SAP SE

AI summary

An AI platform mesh includes a machine learning control plane hosted in a container orchestration cluster. The machine learning control plane includes a worker plane manager and a set of machine learning services. One or more remote runtime environments configured to execute machine learning workloads can be registered with the machine learning control plane via the worker plane manager. The remote runtime environments can communicate with the set of machine learning services via the worker plane manager.