TopK Vector Sorting via Register-Based Insertion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for performing TopK operations in electronic devices, such as machine learning applications, are inefficient due to excessive use of memory and processing power, particularly when dealing with large vectors, as they require repeated comparisons and data movements, leading to computational inefficiencies.
Innovation Solution
The proposed solution leverages the arithmetic logic units (ALUs) and registers within a hardware-based system to perform TopK operations efficiently by reading the vector once and using on-chip memories for intermediate processing, where the top K values are tracked and updated in registers, reducing unnecessary data movements and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional sequential methods are used to identify top K values by repeatedly finding maximum values, then the top K values can be obtained in sorted order, but computation efficiency deteriorates due to repeated instructions and wasted computational power on sorting non-top-K elements
Solution Approach 1:
The patent divides the vector into multiple segments or blocks that can be processed in parallel. Each segment is processed independently to identify local top-K candidates, which are then merged and sorted to obtain the final top-K results. This segmentation enables parallel processing while reducing the computational burden compared to processing the entire vector sequentially.
Solution Approach 2:
The patent performs preliminary sorting or filtering of the vector elements before the final top-K selection. By pre-processing the data to organize elements in a more favorable structure (such as partial sorting or creating candidate lists), the subsequent top-K identification requires fewer comparisons and operations, thereby improving overall computation efficiency.
2Measurement precision
If full sorting of vector elements is performed to identify top K values, then the top K values are obtained in sorted order, but computational resources are wasted on sorting elements that are not in the top K
Solution Approach 1:
The patent extracts only the necessary information for identifying top-K values without performing a complete sort of all elements. By using selection algorithms or heap-based approaches, the method directly extracts the top-K elements from the vector without ordering the remaining elements, thereby eliminating wasted computational effort on non-top-K elements while maintaining sorted order accuracy for the extracted top-K values.
3Ease of manufacture
If sequential maximum identification is used to obtain top K values, then the method is simple to implement, but data movement increases due to repeated reading and comparison operations
Solution Approach 1:
The patent implements a continuous processing pipeline where vector elements are read once and processed through a series of operations (comparison, selection, and insertion into the top-K structure) in a single pass. This continuous action eliminates the need to repeatedly read and re-read the same data elements, significantly reducing data movement volume while maintaining implementation feasibility through standardized processing steps.
Data Source
AI summary
A method includes receiving a TopK instruction to sort a highest K elements of a vector data. A first K elements of the vector data are sorted and stored in a first register. Another element of the vector data is read and determined whether it has a value that is greater than or is within a range of values of the first K elements. A position of the another element within the first K elements is determined if the another element has a value within that is within the range of values. A subset of the elements of the first K elements that are smaller than the another element are shifted down after determining the position of the another element in the first K elements. The another element is inserted in the determined position after the shifting. The process is repeated for each remaining element of the vector data.