Late-Binding Schema for Machine Data Analysis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Analyzing and searching massive quantities of diverse machine data generated by various components in IT environments is challenging due to the vast amount of data types and formats, leading to inefficiencies in data retrieval and analysis.

Innovation Solution

The implementation of an event-based data intake and query system that uses a late-binding schema to process, index, and store machine data, allowing for flexible extraction and search of data at search time, enabling the use of a common information model across disparate data sources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If data is pre-processed with specified data items extracted and stored, then retrieval efficiency is improved, but data completeness and flexibility for future analysis deteriorates

Engineering Contradiction:

Improvedata retrieval speedVSAvoiddata analysis flexibility

Core Design Contradiction:

SpeedVSAdaptability or versatility

Solution Approach 1:

The patent applies preliminary action by pre-processing machine data to extract specified data items and store them in a data system before actual analysis needs arise. This allows efficient retrieval when those specific items are needed, while the pre-processing framework is designed to be adaptable to future analysis requirements.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements dynamics by making the data extraction and storage process adaptive rather than static. The system can dynamically adjust which data items are extracted and stored based on anticipated analysis needs, and can modify extraction rules as new analysis requirements emerge, balancing retrieval efficiency with future flexibility.

Inventive Principle:

Principle #15Dynamics

2Ease of operation

If visualizations are used to represent data, then data understandability is improved, but the representation of total data characteristics deteriorates

Engineering Contradiction:

Improvedata understandabilityVSAvoiddata characteristic completeness

Core Design Contradiction:

Ease of operationVSLoss of information

Solution Approach 1:

The patent applies partial action by using visualizations to represent only the most important or relevant characteristics of machine data, rather than attempting to display all data attributes. This makes the data understandable to users while accepting that some data characteristics are necessarily omitted from the visual representation.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11829378B1Automated generation of insights for machine generated data

Publication Date: 2023.11.28 CISCO TECHNOLOGY INC

AI summary

A data processing platform generates visualizations for data streams to visually represent a portion of data in the data stream. The platform performs an analysis of a change in values of data contained in the data stream and generates, using a result of the analysis, metadata identifying an insight into the data in the data stream. The insight indicates a characteristic of the change in values. A natural language representation of the insight is generated using the metadata and output for display in association with the visualization.