Speculative Regex Execution in Database Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Regular expression (regex) matching in database management systems (DBMS) is a compute-heavy task that consumes significant memory and processing resources, especially with larger input strings and more complex regex patterns, and existing hardware acceleration solutions are either expensive or lack flexibility.
Innovation Solution
Generating specialized machine bytecode for regex patterns and input strings at runtime, using optimized execution logic based on descriptors, and converting regex automata to handle encoded strings without decoding, to reduce query execution time and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If regex matching is performed on encoded strings by decoding first, then memory space is reduced, but compute resources increase due to decoding overhead
Solution Approach 1:
The patent introduces an intermediary encoded-automaton that operates directly on encoded string representations without requiring full decoding. This intermediary structure bridges the gap between encoded storage and regex matching requirements, allowing the automaton to process encoded tokens and transitions that correspond to patterns in the original decoded string, thereby avoiding the computationally expensive decoding step while maintaining memory efficiency
Solution Approach 2:
The patent transforms the regex automaton into an encoded-automaton by changing the parameter representation from decoded characters to encoded tokens. The automaton's transition function is modified to operate on encoded string tokens rather than decoded characters, and the pattern representation is transformed to work with the encoding scheme, enabling direct matching on compressed data
2Speed
If regex operations are performed on fast access memory, then matching speed improves, but resource consumption increases
Solution Approach 1:
The patent segments the regex matching process into distinct phases: encoding-aware automaton construction, encoded token processing, and transition execution. By organizing the automaton into discrete states and transitions that operate on encoded tokens, the system enables efficient processing that reduces the need to load entire decoded strings into fast access memory, thereby maintaining matching speed while reducing resource consumption
3Speed
If hardware acceleration is used for regex operations, then execution speed improves, but cost and flexibility decrease
Solution Approach 1:
The patent implements a dynamic regex engine that can adapt its behavior based on the encoding scheme and input characteristics. The encoded-automaton construction and execution are performed dynamically at runtime, allowing the system to optimize for different encoding schemes (e.g., dictionary encoding, run-length encoding) and input patterns without requiring dedicated hardware for each specific case, thereby maintaining flexibility while achieving high execution speeds
Data Source
AI summary
Techniques are described to improve the performance of regular expression (regex) evaluation in a database management system (DBMS) by a speculative execution of a regex engine. In an embodiment, the DBMS determines properties of strings and generates descriptors for the string input data. Based on the descriptors, the regex engine validates an existing assertion for the execution logic to, at least in part, evaluate the regex pattern on the strings. Based on validating the existing assertion for selecting the execution logic, evaluating the regex pattern on the strings.