Hardware Accelerator for Text Log Querying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems for querying large text datasets are inefficient, requiring high energy and hardware costs due to the need for scale-out processing on large clusters, and struggle with ad-hoc queries and regular expression matching, which are resource-intensive and slow.
Innovation Solution
A programmable hardware accelerator architecture that splits queries into partial programs and uses multiple query algorithms to process different portions of the data in parallel, employing bit-split state machines and content addressable memory to efficiently match patterns and perform arithmetic operations, allowing for early rejection of non-matching data and concurrent processing of multiple characters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If scale-out processing on large clusters is used to process vast text collections, then processing capability is improved, but energy consumption and hardware costs increase significantly
Solution Approach 1:
The patent segments the text processing task into distinct functional units implemented as hardware accelerators. The Aho-Corasick pattern matching machine is divided into state machine units, transition table storage, and output generation components. This segmentation allows parallel processing of multiple patterns simultaneously, achieving high text processing capability while consuming less energy than scale-out cluster approaches.
2Adaptability or versatility
If conventional CPU-based approaches are used for ad-hoc queries on large datasets, then flexibility is maintained, but processing speed decreases
Solution Approach 1:
The patent implements dynamically reconfigurable hardware accelerators that can adapt to different query patterns and data structures. The state machine transition tables can be loaded with different patterns, and the hardware configuration can be adjusted based on the specific ad-hoc query requirements. This dynamic adaptability maintains query flexibility while achieving processing speeds orders of magnitude faster than conventional CPU approaches.
3Quantity of substance
If bit-split state machines are used to reduce storage requirements, then storage cost decreases, but system complexity increases
Solution Approach 1:
The patent applies bit-splitting technology that changes the representation parameter of state machine transitions from full-byte encoding to bit-level encoding. Each state transition is represented by individual bits that can be combined to form the complete transition address. This parameter change reduces storage requirements from storing complete transition tables to storing separate bit maps, achieving compact storage while managing complexity through systematic bit manipulation operations.
Data Source
AI summary
A hardware accelerator 2 for performing queries into, for example, an indexed text log files is formed of plurality of hardware execution units (text engines) 4, each executing a partial query program upon the same full set of input data. These partial query programs may switch between different query algorithms on up to a per-character basis. The sequence of data when loaded into a buffer memory 16 for querying may be searched for delimiters as the data is loaded. The hardware execution units may support a number match program instruction which serves to identify a numeric variable, and to determine a value of that numeric variable located at a variable position within a sequence of characters being queried.