Tensor Broadcasting Offset Tables for Efficient Shape-Mismatch Operations

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing artificial intelligence models face challenges in performing tensor operations due to mismatched dimension sizes between tensors, requiring inefficient broadcasting operations to extend smaller dimensions to match larger ones, which can lead to increased computational complexity and resource utilization.

Innovation Solution

A processor-implemented method determines a broadcasting type based on the shapes of tensors, calculates respective offsets for each dimension, and performs operations by obtaining elements using these offsets, optimizing the broadcasting process through cumulative offset calculations and pre-generated programs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If broadcasting operations are performed to extend dimension sizes of tensors to match each other, then tensor operations can be executed between tensors of different shapes, but computational complexity and resource utilization increase

Engineering Contradiction:

Improvetensor operation compatibilityVSAvoidcomputational complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent pre-calculates and stores shape offset information and tensor offset information in offset tables before executing tensor operations. This preliminary action allows the broadcasting operation to directly retrieve offset values from pre-computed tables rather than calculating them during execution, significantly reducing computational complexity while maintaining adaptability to handle tensors of different shapes

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The offset tables are designed to automatically provide the necessary offset information for broadcasting operations based on tensor shapes and broadcasting types. The system uses the tensor shapes to directly index into pre-computed offset tables, allowing the operation to self-configure without requiring complex runtime calculations or manual intervention

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

Engineering Contradiction:

Improvetensor operation compatibilityVSAvoidresource utilization

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent pre-calculates offset information and stores it in compact offset tables during system initialization or before tensor operations. This eliminates the need to compute offsets during actual tensor operations, reducing runtime resource consumption while enabling flexible broadcasting between tensors of different shapes

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of physically replicating tensor data during broadcasting operations, the patent uses offset tables to virtually replicate the necessary information. The offset tables store computed offset values that allow the system to access and operate on the appropriate tensor elements without actually copying large amounts of data, thereby reducing memory resource utilization

Inventive Principle:

Principle #26Copying

3Adaptability or versatility

If traditional broadcasting operations are used to match dimension sizes, then tensors of different shapes can be operated on, but execution efficiency decreases

Engineering Contradiction:

Improvetensor operation compatibilityVSAvoidexecution efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent pre-computes shape offsets and tensor offsets for all possible dimension combinations and stores them in offset tables. During actual tensor operations, the system simply retrieves these pre-computed offsets based on the tensor shapes, which dramatically speeds up execution compared to computing offsets on-the-fly during broadcasting operations

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces the mechanical computation process of calculating offsets during broadcasting operations with a lookup process in pre-computed offset tables. This substitution of computational mechanics with table-based retrieval significantly improves execution efficiency while maintaining the ability to handle tensors of different shapes

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20250217675A1Method, device, and apparatus with tensor broadcasting operation

Publication Date: 2025.07.03 SAMSUNG ELECTRONICS CO LTD

AI summary

A processor-implemented method including, based on a first shape of a first tensor and a second shape of a second tensor used in an operation with the first tensor, determining a broadcasting type related to an extension of the first shape and the second shape, determining respective first shape offsets for each dimension of the first shape and respective second shape offsets for each dimension of the second shape based on the broadcasting type, determining respective first tensor offsets for each dimension of the first tensor and respective second tensor offsets for each dimension of the second tensor based on the broadcasting type, the respective first shape offsets for each dimension of the first shape, and the respective second shape offsets for each dimension of the second shape, determining a first offset of a first element included in the first tensor and a second offset of a second element included in the second tensor based on the respective first tensor offsets and the respective second tensor offsets, and performing the operation by obtaining the first element and the second element based on the first offset and the second offset.