Semantic Layer for Machine Data Interpretation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The overwhelming volume, complexity, and variability of machine data generated by IT systems pose challenges for software developers, system administrators, and business users in understanding and organizing this data for troubleshooting, security analysis, and business performance measurement, as conventional tools are not designed to handle the velocity, volume, and variability of this data effectively.
Innovation Solution
A method and system for processing machine data that involves receiving and interpreting machine data from various sources, mapping it to a matching entry type in a database, assigning semantic events, and storing these events with associated attributes in a knowledge base, enabling effective representation and analysis of machine data across multiple systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional monitoring and analysis tools are used to process machine data, then the tools are simple and familiar, but they cannot handle the variety, velocity, volume, or variability of machine data effectively
Solution Approach 1:
The patent introduces a semantic layer as an intermediary between machine data sources and analysis tools. This semantic layer includes a schema that defines entry types, attributes, and relationships, allowing conventional tools to process machine data effectively by translating and structuring the data in a standardized format that both preserves complexity handling capability and maintains tool simplicity
2Quantity of substance
If machine data is collected from multiple information systems and domains, then the data volume and insights increase, but the ability to effectively organize and understand the data decreases
Solution Approach 1:
The patent creates a universal schema that can handle multiple types of machine data from different information systems and domains through a single standardized framework. The schema defines entry types and attributes that can accommodate various data formats and sources, allowing the same organizational structure to universally process diverse data volumes without increasing organizational complexity
3Adaptability or versatility
If machine data formats change and become more unpredictable, then the data reflects real-world variability, but conventional tools cannot keep up with changing formats
Solution Approach 1:
The patent implements a dynamic schema that can adapt to changing machine data formats while maintaining processing speed. The schema allows for flexible definition and modification of entry types and attributes, enabling the system to accommodate unpredictable format changes without sacrificing processing productivity, as the semantic layer absorbs format variability rather than requiring tool redesign
Data Source
AI summary
Improved techniques for processing machine data are disclosed. Embodiments are operable to receive machine data input, interpret its meaning, and then represent that meaning in a knowledge base that grows over time with each new entry. The knowledge base enables extension of both syntax and lexicon, which are the main determinants of meaning. As new entries are added, the knowledge in the knowledge base grows. Over time, the knowledge base acquires more meaning. The disclosed machine data processing system includes entry type recognition, mapping entry types to semantic events, and building entries in the knowledge base based on the semantic event-entry type mapping. Data generated by this process may be used to conduct searches for patterns of semantic events across multiple different machine data sources. This information may then be used to perform useful work such as detecting security threats, identifying operational problems, or tracking customer purchases, etc.