Real-Time Event Record Extraction With Editable Rule Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid increase in machine-generated data has made it difficult to efficiently analyze large, unstructured datasets due to the challenges in determining effective extraction rules, leading to improper or ineffective data processing and value omission.
Innovation Solution
A system that enables real-time display of event records with an indication of previously provided extraction rules, allowing users to automatically generate or manually enter extraction rules, and emphasizes relevant text fields using techniques like highlighting or dimming, facilitating efficient data analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If extraction rules are manually determined for large unstructured datasets, then data processing accuracy improves, but time consumption and operational complexity increase significantly
Solution Approach 1:
The system enables self-service by automatically generating extraction rules through machine learning algorithms that learn from user interactions and data patterns, eliminating the need for manual rule creation while maintaining high extraction accuracy
Solution Approach 2:
The system implements feedback mechanisms where user corrections and interactions with extracted data are continuously used to refine and improve extraction rules, enabling the system to learn and adapt over time while reducing manual intervention requirements
2Loss of information
If traditional data analysis methods are used on large unstructured datasets, then data processing completeness improves, but analysis efficiency deteriorates
Solution Approach 1:
The system replaces traditional mechanical data analysis methods with automated machine learning-based extraction systems that can process large volumes of unstructured data efficiently while maintaining comprehensive data coverage through continuous learning and adaptation
3Ease of operation
If extraction rules are created without real-time feedback, then rule development simplicity improves, but rule effectiveness deteriorates
Solution Approach 1:
The system provides real-time feedback on extraction rule performance by displaying highlighted extracted values and enabling user corrections, which are immediately used to refine rules, ensuring both ease of operation and high effectiveness through continuous iterative improvement
Data Source
AI summary
Embodiments are directed towards real time display of event records with an indication of previously provided extraction rules. A plurality of extraction rules may be provided to the system, such as automatically generated and/or user created extraction rules. These extraction rules may include regular expressions. A plurality of event records may be displayed to the user, such that text in a field defined by an extraction rule is emphasized in the display of the event record. The same emphasis may be provided for text in overlapping fields, or the emphasis may be somewhat different for different fields. The user interface may enable a user to select a portion of text of an event record, such as by rolling-over or clicking on an emphasized part of the event record. By selecting the portion of the event record, the interface may display each extraction rule associated with the selected portion.