Dynamic Drill-Down Nested Graphical Interfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing and searching massive quantities of machine data generated from diverse sources, such as system logs, network packet data, and sensor data, is challenging due to the vast amount of data and varying formats, leading to inefficiencies in data retrieval and analysis.
Innovation Solution
A data intake and query system that uses a late-binding schema to process and store machine data as events, allowing for flexible schema definition and extraction rules application at search time, enabling field-searchability and efficient retrieval of specific data items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If pre-processing data to extract specified data items for efficient retrieval, then retrieval efficiency is improved, but data flexibility and analysis capability are reduced
Solution Approach 1:
The system performs preliminary actions by ingesting and storing all raw machine data without pre-processing or filtering. The data is maintained in its original format with minimal transformation, allowing for flexible analysis later. Schema definition and data extraction occur at search time rather than ingestion time, enabling adaptability while maintaining retrieval efficiency through on-demand processing.
Solution Approach 2:
The patent implements dynamic schema definition that occurs at search time rather than being static at ingestion time. The system dynamically determines which data items to extract and how to structure them based on the specific search query requirements. This dynamic approach allows the system to adapt to different analysis needs while maintaining efficient retrieval for each specific query type.
2Adaptability or versatility
If storing massive quantities of raw data, then data analysis flexibility is improved, but data retrieval and search performance deteriorate
Solution Approach 1:
The system performs preliminary data ingestion and storage of all raw machine data without pre-processing. The heavy lifting of data extraction, transformation, and structuring is deferred to search time. This preliminary storage of raw data maintains flexibility while the on-demand processing at search time ensures retrieval performance by only processing the specific data items needed for each query.
Solution Approach 2:
The patent introduces an intermediary layer in the form of a search-time processing system that acts as a mediator between the stored raw data and the analysis requirements. This intermediary dynamically extracts and structures data based on search queries, bridging the gap between storing all raw data and retrieving specific items efficiently, without requiring pre-processing of all data.
3Productivity
If pre-extracting and storing specified data items, then retrieval efficiency is improved, but data completeness and insight potential are reduced
Solution Approach 1:
The system performs the preliminary action of ingesting and storing all raw machine data in its complete form without extraction or filtering. This ensures no data is lost during pre-processing. The extraction and structuring of specific data items is deferred to search time, maintaining data completeness while enabling efficient retrieval and analysis of all available data based on specific analytical needs.
Data Source
AI summary
Data (e.g., security data) is presented at increasing levels of detail in a drill down on graphical displays. The data can be retrieved pursuant to a request, such as a search. A first graphical display corresponds to a first level of the detail. A first actionable graphic is displayed, which, when activated, presents a second graphical display with more detail. Thus, nested graphical displays are provided. The detailed data may be retrieved in response to a search, with only the high-level data graphically displayed first, with the additional data and stored templates be ready to instantly produce the nested, detailed charts upon use activation.