Dispatcher Parser Thread Event Queue Rule Versioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parsing technologies face performance degradation due to the need for locking or restarting parsing processes to maintain consistency when updating or changing parsing rules, which hinders efficient data processing.
Innovation Solution
A computer program that utilizes a dispatcher thread to receive and match events with parsing rules, storing them in an event queue, and a parser thread to process events based on these rules, with reference counters managing rule updates and storage to prevent performance degradation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If locking is used to maintain consistency when updating parsing rules, then consistency is maintained, but performance degrades due to occupation of operation time
Solution Approach 1:
The system divides parsing rules into multiple versions (current version and updated version) and uses version numbers to manage transitions. This segmentation allows the parser to work with the current version while updates are prepared in the background, eliminating the need for locking and avoiding performance degradation.
Solution Approach 2:
Updated parsing rules are prepared and validated in advance before being activated. The system performs preliminary checks to ensure the updated rule is valid, then atomically switches to the new version. This preliminary action prevents inconsistencies without requiring locks during the actual parsing operations.
2Reliability
If parsing process is stopped to update parsing rules, then consistency is maintained, but performance degrades due to stopping and re-starting
Solution Approach 1:
The system implements dynamic version switching where the parser can continuously process events while the parsing rule version is updated. By using version numbers and allowing the parser to dynamically adapt to the current version, the system maintains consistency without stopping the parsing process, thus avoiding time loss.
Solution Approach 2:
The parsing process continues uninterrupted during rule updates. The system maintains a continuous stream of event processing by allowing the parser to operate on the current version while updates are prepared and validated in the background, ensuring no useful action is lost during transitions.
3Productivity
If reference to parsing rule is made during updates, then performance improves, but consistency may be compromised
Solution Approach 1:
The system introduces version numbers as an intermediary between the parser and parsing rules. The version number acts as a mediator that allows the parser to reference rules during updates without compromising consistency. By checking the version number, the system ensures that only valid, complete rule sets are applied, maintaining both performance and consistency.
Data Source
AI summary
Disclosed is a computer program stored in a computer readable storage medium including encoded commands. When the computer program is executed by one or more processors of a computer system, the computer program enables the one or more processors to perform operations for parsing, and the operations include: receiving, by a dispatcher thread, an event; matching, by the dispatcher thread, information on a parsing rule corresponding to the event with the event, and storing the event and the matched information on the parsing rule in an event queue; fetching, by a parser thread, the event and the information on the parsing rule from the event queue; and processing, by the parser thread, the event according to the parsing rule corresponding to the event based on the information on the parsing rule.