Multi-Query GUI With Late-Binding Schema for Machine Data

Solution Overview

Problem

Existing tools lack the ability to efficiently search and analyze large sets of diverse data, particularly raw machine data, in a visually intuitive and flexible manner, leading to challenges in deriving insights from vast amounts of data stored in IT environments.

Innovation Solution

A data intake and query system utilizing a late-binding schema and flexible schema to process and search machine data, enabling field-searchability and allowing extraction rules to be defined at search time, with a metadata catalog to manage and refine field extraction rules.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If pre-processing data to extract specified data items is performed, then retrieval efficiency of extracted data is improved, but flexibility to analyze all generated data is lost

Engineering Contradiction:

Improveretrieval efficiencyVSAvoidflexibility to analyze data

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary indexing of data fields during data ingestion, creating search-ready data structures without extracting or filtering the data. This allows fast retrieval when needed while preserving the ability to analyze all raw data through flexible query language, resolving the contradiction between pre-processing benefits and analysis flexibility.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes the parameter of data organization from static pre-extracted fields to dynamic field extraction at query time. The schema is applied late-binding, allowing the same raw data to be indexed differently for different queries, thus maintaining both retrieval efficiency and analytical flexibility.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If data is stored in raw format for later analysis, then flexibility of analysis is improved, but search and analysis speed decreases

Engineering Contradiction:

Improveflexibility of analysisVSAvoidsearch and analysis speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system performs preliminary indexing of data fields during data ingestion, creating search-ready data structures without extracting or filtering the data. This allows fast retrieval when needed while preserving the ability to analyze all raw data through flexible query language, resolving the contradiction between pre-processing benefits and analysis flexibility.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system introduces an intermediary indexing layer between raw data storage and query processing. This intermediary structure enables fast search operations on indexed fields while the raw data remains accessible for flexible analysis, mediating between the need for speed and flexibility.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If field extraction rules are defined at search time, then adaptability to different data structures is improved, but system complexity increases

Engineering Contradiction:

Improveadaptability to data structuresVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system changes the parameter of data organization from static pre-extracted fields to dynamic field extraction at query time. The schema is applied late-binding, allowing the same raw data to be indexed differently for different queries, thus maintaining both retrieval efficiency and analytical flexibility.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system makes the field extraction process dynamic by applying schemas at query time rather than during data ingestion. This allows the extraction rules to adapt to different data structures on-demand, managing complexity through lazy evaluation and caching of extraction results.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20250252135A1Executing one query based on results of another query

Publication Date: 2025.08.07 CISCO TECHNOLOGY INC

AI summary

Systems and methods are disclosed for performing multiple queries in a single graphical user interface (GUI) displayed in a client browser. The client browser causes the display of a first user interface field in a first area of the GUI, where the first user interface field can be used to enter or edit a first query. The client browser also causes first query results generated by a data intake and query system executing the first query to be displayed in the first area. The client browser further causes the display of a second user interface field in a second area of the GUI, where the second user interface field can be used to enter or edit a second query. The client browser also causes second query results generated by the data intake and query system executing the second query to be displayed in the second area.