Tensor Equivalence Detection for Neural Network Memory Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Tensors in neural networks consume significant memory and processing resources, leading to inefficiencies in memory and processing allocation.

Innovation Solution

A tensor processor is implemented to verify, determine, and establish equivalence between tensors, allowing for the removal of redundant tensor data and efficient storage by mapping equivalent tensors to common memory locations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If tensors are stored and processed in neural networks, then computational functionality is achieved, but memory consumption and processing resource usage increase significantly

Engineering Contradiction:

Improveneural network computational functionalityVSAvoidmemory consumption

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent merges equivalent tensors by identifying tensors with identical data values and mapping them to the same memory location. This combining of redundant tensor instances reduces memory consumption while preserving neural network computational functionality, as the merged tensors share the same underlying data storage.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent creates a universal memory mapping mechanism where a single memory location can serve multiple tensor identifiers simultaneously. The tensor equivalence detection system allows one stored tensor to fulfill the functional role of multiple equivalent tensors throughout the computational graph, achieving multi-functionality in memory allocation.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If tensors are stored and processed in neural networks, then computational functionality is achieved, but processing resource usage increases significantly

Engineering Contradiction:

Improveneural network computational functionalityVSAvoidprocessing resource usage

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent performs preliminary tensor equivalence detection and merging operations during the graph optimization phase before actual neural network execution. By identifying and consolidating equivalent tensors in advance, the system reduces the number of processing operations required during inference or training, thereby lowering processing resource usage and energy consumption.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If redundant tensor data is stored, then tensor equivalence detection can be performed, but memory consumption increases

Engineering Contradiction:

Improvetensor equivalence detection accuracyVSAvoidmemory consumption

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent extracts the essential identifying features of tensors (such as data values, shapes, and structural properties) into a compact representation for equivalence comparison. Instead of storing complete redundant tensor data, the system extracts key characteristics that enable accurate equivalence detection with minimal memory overhead.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS20240070450A1Tensor processing for neural network

Publication Date: 2024.02.29 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to establish a correspondence between at least a plurality of tensors. In at least one embodiment, information is caused to be stored from one of two or more different tensors having one or more variable dimensions, the one of the two or more different tensors is to represent the two or more different tensors.