Tensor Broadcasting Within AI Operations to Cut Processing Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing environments face limitations in performing tensor computations due to the need for separate processing and additional operations when input tensors have differing dimensions, leading to reduced performance and increased complexity.
Innovation Solution
A computer program product and method that includes dimension broadcasting within a single instruction for artificial intelligence processing operations, allowing tensors with differing dimensions to be aligned, thereby reducing the need for separate processing and increasing processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate processing operations are used for tensors with differing dimensions, then dimension compatibility is achieved, but processing speed and efficiency deteriorate
Solution Approach 1:
The patent combines dimension broadcasting with AI processing operations into a single unified operation. Instead of performing dimension alignment separately and then executing AI operations, the system integrates both steps, allowing the broadcasting to occur implicitly during the AI processing itself. This merging eliminates the need for separate processing stages, thereby improving processing speed while maintaining dimension compatibility.
Solution Approach 2:
The system performs dimension broadcasting in advance by preparing broadcasted versions of tensors with index size one before executing AI processing operations. This preliminary action ensures that all dimension mismatches are resolved beforehand, allowing the subsequent AI operations to proceed without additional dimension-checking overhead, thus improving overall processing efficiency.
2Productivity
If multiple processor invocations are used for dimension alignment and AI operations, then processing accuracy is maintained, but system resource use increases
Solution Approach 1:
The patent merges dimension broadcasting and AI processing into a single processor invocation. By integrating these operations, the system eliminates the overhead of multiple function calls, memory accesses, and processor state transitions that would occur with separate operations. This single unified operation reduces system resource consumption including CPU cycles, memory bandwidth, and energy usage while maintaining processing accuracy.
3Reliability
If dimension broadcasting is performed separately before AI operations, then tensor compatibility is ensured, but processing time increases
Solution Approach 1:
The system performs dimension broadcasting as a preliminary action by pre-computing broadcasted tensor versions and storing them in memory before AI processing. This allows the main AI operations to proceed directly with compatible tensors without runtime dimension checking or dynamic broadcasting, thereby ensuring tensor compatibility while minimizing processing time loss.
Solution Approach 2:
The patent creates copied and broadcasted versions of input tensors with index size one, expanding them to match the dimensions of other input tensors. These pre-generated copies are then used directly in AI processing operations, eliminating the need for runtime dimension transformation and ensuring compatibility without time penalty during the actual computation.
Data Source
AI summary
Tensor processing with broadcasting includes obtaining a first input tensor, the first input tensor including a dimension of index size one, and performing an artificial intelligence processing operation using the first input tensor and a second input tensor to obtain results, the second input tensor including a dimension of index size greater than one, where performing the artificial intelligence processing operation includes broadcasting the dimension of index size one of the first input tensor to match the dimension of index size greater than one of the second input tensor.