Constellation Feature Matching for Low-Memory Edge AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional feature matching processes in machine learning are computationally intensive, inefficient, and have a substantial memory footprint, serving as a bottleneck for performance in terms of latency and power consumption, and are unsupported by edge devices.
Innovation Solution
The use of constellation sets for feature matching, which involves shifting target tensors based on offsets, generating intermediate tensors through elementwise multiplication, and aggregating flattened tensors to create a correlation tensor, eliminating the need for convolution-based kernel striding and memory-inefficient transformer-based token searching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional feature matching methods are used, then feature matching can be performed, but computational expense and processing time increase substantially
Solution Approach 1:
The patent segments the feature matching process into distinct tensor operations: shifting the target tensor by offset values, elementwise multiplication with the reference tensor, and aggregation of results. This segmentation enables more efficient computation by breaking down the complex matching operation into manageable steps that can be optimized independently.
Solution Approach 2:
The patent changes the computational parameters by using a fixed set of offset values to shift the target tensor, replacing the traditional convolutional approach. This parameter-based shifting method reduces computational complexity while maintaining matching accuracy, directly addressing the contradiction between speed and computational expense.
2Productivity
If conventional feature matching methods are used, then feature matching can be performed, but memory footprint increases substantially
Solution Approach 1:
The patent extracts only the necessary computational elements for feature matching by using a fixed set of offset values and performing elementwise multiplication. This extraction approach eliminates the need to store and process large convolutional kernels or transformer attention matrices, significantly reducing memory footprint while maintaining matching efficiency.
Solution Approach 2:
By changing from convolutional kernel operations to parameter-based tensor shifting with fixed offsets, the patent reduces the quantity of data that needs to be stored in memory. The fixed offset parameters require minimal storage compared to traditional convolutional weights or transformer attention mechanisms.
3Measurement precision
If conventional feature matching methods are used, then feature matching can be performed, but latency increases
Solution Approach 1:
The patent performs preliminary action by pre-defining a fixed set of offset values that are used to shift the target tensor. This preliminary setup eliminates the need for complex runtime computations to determine shift amounts, reducing processing latency while preserving matching accuracy through the carefully selected offset values.
Solution Approach 2:
The patent changes the time-consuming convolutional or transformer-based operations into faster parameter-based tensor shifting. By using fixed offset parameters and elementwise multiplication, the computation time is significantly reduced while maintaining the precision needed for accurate feature matching.
4Productivity
If conventional feature matching methods are used, then feature matching can be performed, but power consumption increases
Solution Approach 1:
The patent segments the feature matching into efficient tensor operations that can be executed with lower power consumption. By dividing the operation into shifting, elementwise multiplication, and aggregation steps, the system can optimize power usage at each stage, making feature matching viable for edge devices with limited power resources.
Solution Approach 2:
The patent changes from power-intensive convolutional and transformer operations to more efficient parameter-based tensor shifting. This parameter change reduces the computational load and associated power consumption, enabling feature matching to run on energy-constrained edge devices while maintaining acceptable performance.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. In an example method, a reference tensor corresponding to a reference image and a target tensor corresponding to a target image are accessed. A constellation set is used to perform feature matching for the reference and target tensors while processing data using the artificial neural network, where the constellation set comprises a set of offsets. The feature matching includes, for each respective offset of the first constellation set, shifting the target tensor based on the respective offset, generating a respective intermediate tensor based on elementwise multiplying the reference tensor and the shifted target tensor, and generating a respective flattened tensor based on flattening the respective intermediate tensor. The respective flattened tensors are aggregated to generate a correlation tensor for the reference and target tensors.