Neural Network Accelerator Bitonic Sorting for Deterministic Argsort
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network accelerators (NNAs) lack the facility to sort data elements efficiently, relying on CPUs for sorting operations which are non-deterministic and inefficient, especially for functions like non-maximum suppression and Argsort that require sorting steps.
Innovation Solution
A method utilizing a bitonic sorting algorithm that compares pairs of elements in parallel within the NNA, using elementwise operations to generate a result array with deterministic time complexity, eliminating the need for dedicated sorting hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sorting is performed using a CPU with quicksort algorithm, then sorting functionality is achieved, but the time taken is non-deterministic and depends on data order
Solution Approach 1:
The sorting problem is segmented into multiple independent comparison operations that can be executed in parallel. Instead of a sequential quicksort algorithm, the patent divides the sorting task into parallel comparison stages where multiple element pairs are compared simultaneously, converting a sequential process into a parallel one to achieve deterministic timing.
Solution Approach 2:
The patent transforms the static sequential quicksort process into a dynamic parallel processing system. By utilizing the NNA's capability to perform multiple elementwise operations simultaneously, the system adapts the sorting approach to leverage parallel execution, making the sorting time deterministic rather than dependent on data order.
2Productivity
If sorting is performed by CPU externally or integrated within NNA, then sorting function is achieved, but it is inefficient and not ideal for NNA architecture
Solution Approach 1:
The patent makes the NNA universally capable of performing sorting operations by using its existing elementwise operation capabilities. Instead of requiring dedicated sorting hardware, the NNA uses its general-purpose computational units to perform sorting, making the sorting function multi-functional and eliminating the need for separate sorting hardware.
Solution Approach 2:
The NNA performs sorting operations using its own computational resources and elementwise operation capabilities, rather than relying on external CPU assistance. This self-service approach allows the NNA to handle sorting tasks independently, improving efficiency by eliminating inter-device data transfer and utilizing its inherent parallel processing strength.
3Reliability
If worst-case scenario time is allocated for sorting, then all sorting cases can be handled, but the time allowed is longer than necessary for non-worst case scenarios
Solution Approach 1:
The patent performs preliminary actions by organizing data into parallel comparison groups before the actual sorting execution. By pre-arranging the comparison structure and utilizing the deterministic nature of parallel operations, the system can allocate exact time requirements without needing to reserve time for worst-case sequential scenarios, as the parallel structure guarantees consistent execution time.
Data Source
AI summary
A method of comparing a plurality of elements in a first array, using a neural network accelerator having fixed-function hardware, the method including the steps of generating a second array, the second array having the position of each pair of elements to be compared swapped, comparing respective elements of the first array and the second array to generate a third array to identify which of the respective elements of the first and second array is larger or smaller and generating a result array, using at least the third array, by using a fourth predetermined array, the fourth predetermined array indicating the position in the result array of the larger and the smaller of each element of each pair of elements.