Dynamic Lexer Object Construction for Adaptive Text Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional formal grammars are inflexible and require redefinition or recompilation to accommodate different field separators, special character sets, and token variations in character sequences, making them inefficient for dynamic pattern recognition across various file types.
Innovation Solution
Dynamic lexer object construction allows for the explicit and dynamic manipulation of grammar rules at runtime using APIs, enabling the adjustment of separators, character sets, and tokens without recompilation, through match and unmatch functions integrated into the lexer class.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional formal grammars are used, then grammar definition is simple and static, but adaptability to different file types and separators is poor
Solution Approach 1:
The patent implements dynamic lexer object construction where grammar rules are not fixed but can be modified at runtime. The lexer class allows dynamic adjustment of separators, character sets, and tokens through methods like `setSeparator`, `addToken`, and `removeToken`, enabling the same grammar framework to adapt to different file types without redefinition.
Solution Approach 2:
The patent changes parameters of the grammar rule dynamically by allowing modification of separators, character sets, and token patterns at runtime. The lexer object can change its internal parameters (separator character, allowed character sets) based on the input file type, enabling one grammar to handle multiple formats by parameter adjustment rather than redefinition.
2Adaptability or versatility
If formal grammar is adjusted for different separators and character sets, then adaptability improves, but recompilation and redefinition are required
Solution Approach 1:
The patent eliminates the need for recompilation by implementing dynamic modification of lexer objects at runtime. Methods such as `setSeparator` and `addToken` allow the grammar to be adjusted immediately without restarting or recompiling the parser, thus saving time while maintaining adaptability to different separators and character sets.
Solution Approach 2:
The patent uses a template-based lexer class that can be instantiated multiple times with different parameters. Instead of recompiling the entire grammar, the system creates new lexer objects or modifies existing ones with different separator and character set parameters, effectively copying the grammar structure with adjusted parameters rather than recompiling from scratch.
3Adaptability or versatility
If static grammar rules are used, then implementation is simple, but flexibility to accommodate various input strings is limited
Solution Approach 1:
The patent makes the grammar dynamic by providing public methods in the lexer class that allow runtime modification of separators, character sets, and tokens. This enables flexible adaptation to various input contexts while maintaining ease of operation through intuitive API methods like `setSeparator`, `addToken`, and `removeToken` that simplify the adjustment process.
Solution Approach 2:
The lexer object serves itself by providing self-modifying capabilities through built-in methods. The system can adjust its own grammar rules without external intervention or complex configuration processes, making the adaptation process simple and straightforward while maintaining high flexibility for various input strings.
Data Source
AI summary
In one example in accordance with the present disclosure, a method for dynamic lexer object construction includes receiving a grammar rule including a variable and a Boolean function to be performed on a character sequence using the variable. The method includes receiving the character sequence and a value corresponding to the character sequence and constructing a lexer object that includes the value as the variable used by the Boolean function. The method includes parsing the character sequence using the lexer object.