Variable-Shape Tensor Processing With Dynamic Work Plans

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing machine learning (ML) methods struggle with inefficiencies when processing tensors of variable shapes, as traditional approaches either require inefficient recompilation for each shape or cumbersome padding, preventing effective application of graph-level optimizations.

Innovation Solution

A modifiable work plan is generated that supports a range of tensor shapes, allowing real-time modification based on actual tensor dimensions, enabling efficient processing of variable-shape tensors using computational engines.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional approaches are used to process tensors of variable shapes, then each shape requires separate handling, but this leads to inefficient recompilation for each shape

Engineering Contradiction:

Improvehandling of variable tensor shapesVSAvoidprocessing efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The work plan is designed to support multiple tensor shapes universally through range specifications. Instead of creating separate work plans for each shape, a single work plan can handle ranges of shapes by defining minimum and maximum dimensions, allowing the same computational graph to process tensors of varying sizes without recompilation.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system dynamically determines the actual tensor shape at runtime and selects or generates an appropriate work plan based on the input dimensions. This dynamic adaptation allows the system to handle variable shapes efficiently by matching the computational plan to the actual data dimensions without requiring pre-compilation for every possible shape.

Inventive Principle:

Principle #15Dynamics

2Adaptability or versatility

If traditional approaches are used to process tensors of variable shapes, then each shape requires separate handling, but this leads to cumbersome padding operations

Engineering Contradiction:

Improvehandling of variable tensor shapesVSAvoidcomputational overhead from padding

Core Design Contradiction:

Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The system changes the parameter representation from fixed shapes to shape ranges with minimum and maximum bounds. This allows the work plan to specify computational operations that adapt to the actual tensor dimensions within the range, eliminating the need for padding operations and associated computational overhead.

Inventive Principle:

Principle #35Parameter changes

3Manufacturing precision

If recompilation is performed for each tensor shape, then shape-specific optimization is achieved, but this increases processing time

Engineering Contradiction:

Improveshape-specific optimizationVSAvoidrecompilation time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The system performs preliminary preparation by defining work plans with shape ranges and placeholder dimensions before runtime execution. This preliminary structuring allows the computational graph to be prepared once for a range of shapes, avoiding the need for repeated recompilation while maintaining shape-specific optimization capabilities through runtime parameter substitution.

Inventive Principle:

Principle #10Preliminary action

4Productivity

If fixed work plans are used for tensor processing, then execution efficiency is improved, but this reduces flexibility for variable shapes

Engineering Contradiction:

Improveexecution efficiencyVSAvoidsupport for variable tensor shapes

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The work plan is segmented into fixed computational operations and variable parameters representing tensor dimensions. This segmentation allows the computational graph structure to remain fixed for efficiency while the dimensional parameters can vary to accommodate different tensor shapes, combining the benefits of both fixed and flexible approaches.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12406204B1Machine learning with variable-shape tensors

Publication Date: 2025.09.02 INTEL OVERSEAS FUNDING CORP

AI summary

An apparatus includes a memory and one or more processors. The memory is configured to store tensors for Machine Learning (ML) processing. The one or more processors are configured to receive a work plan associated with a subgraph of a ML graph of a ML model, the work plan supports processing of tensors having respective shapes in a selected range of shapes. A shape of a tensor specifies respective sizes of dimensions of that tensor. The one or more processors are further configured to receive from the memory an input tensor having an actual shape, to modify the work plan based on the actual shape to produce a modified work plan for processing the input tensor in accordance with the subgraph, and to process the input tensor in accordance with the subgraph by submitting the modified work plan for execution by one or more of the processors.