Same-Slot Auxiliary Instances for Rapid GPU Scrubbing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Access to GPUs for machine learning and high-performance computing is a major obstacle due to increased demand outpacing supply, leading to uncertainty and inefficiency in resource allocation, with long-term commitments being wasteful and expensive.

Innovation Solution

Implementing capacity blocks (CBs) that allow users to reserve GPU time and duration, ensuring predictable access without long-term commitments, and utilizing auxiliary compute instances for optimized startup and termination of user instances.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Stability of the object's composition

If long-term commitments are made for GPU access, then resource allocation stability is improved, but flexibility and cost efficiency deteriorate

Engineering Contradiction:

Improveresource allocation stabilityVSAvoidflexibility

Core Design Contradiction:

Stability of the object's compositionVSAdaptability or versatility

Solution Approach 1:

The system implements dynamic resource allocation where capacity blocks can be reserved for specific time windows rather than requiring long-term commitments. Users can launch instances within reserved capacity blocks during designated time periods, allowing both stable reservation and flexible usage without long-term binding.

Inventive Principle:

Principle #15Dynamics

2Quantity of substance

If GPU supply is increased to meet demand, then resource availability is improved, but resource utilization efficiency deteriorates due to waste

Engineering Contradiction:

ImproveGPU supplyVSAvoidresource utilization efficiency

Core Design Contradiction:

Quantity of substanceVSLoss of energy

Solution Approach 1:

The system uses periodic capacity blocks with specific time windows where GPUs are actively allocated to users. During non-active periods, resources are released back to the pool. This periodic allocation pattern ensures high utilization during active windows while avoiding waste during inactive periods, eliminating the need for continuous over-provisioning.

Inventive Principle:

Principle #19Periodic action

3Speed

If instance startup and termination processes are simplified, then operational speed is improved, but resource cleanup and verification quality deteriorates

Engineering Contradiction:

Improveinstance startup and termination speedVSAvoidresource cleanup and verification quality

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The system performs preliminary resource verification and setup during the instance launch process within capacity blocks. Health checks and resource validations are conducted before instances become fully operational, ensuring quality without requiring lengthy post-termination cleanup processes. This preliminary action approach maintains both speed and reliability.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12461836B1Rapid GPU scrubbing in a cloud provider network using same-slot auxiliary domains

Publication Date: 2025.11.04 AMAZON TECH INC

AI summary

Techniques for instance termination cleanup via auxiliary domains are described. A termination workflow is executed for a compute instance having graphics processing unit (GPU) access hosted in a cloud provider network. The termination workflow includes terminating the compute instance of the user, launching an auxiliary compute instance in the same slot, and executing a cleanup workflow by the auxiliary compute instance for the GPU. Upon the conclusion of the cleanup workflow, the state of the compute instance is updated to terminated, allowing for it to be relaunched.