Data Model Selection for Event-Based Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search tools are inadequate for efficiently searching large volumes of machine-generated data, particularly in handling time-stamped records and event-based searching, requiring users to navigate through repetitive data sets and often necessitating high domain knowledge for query generation.
Innovation Solution
The development of a data modeling system that generates semantic meaning for both structured and unstructured data, allowing for event-based searching without modifying the original data, using data models composed of objects with fields, filters, and constraints, and enabling users to create reports and query strings based on these models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If users search large volumes of machine-generated data using traditional keyword-based search tools, then they can retrieve some results, but the search efficiency is low and requires high domain knowledge for query generation
Solution Approach 1:
The patent introduces an intermediary layer (data model with event types, parameters, and relationships) between the raw machine data and the user query. This intermediary enables users to search using natural language or simplified criteria without needing to understand the complex underlying data structure, thereby improving both search efficiency and ease of operation
Solution Approach 2:
The patent replaces the mechanical keyword-matching system with an intelligent event-based search system that uses data models, event types, and semantic relationships. This substitution allows the system to automatically understand and process search intent, eliminating the need for users to manually construct complex queries with domain knowledge
2Measurement precision
If users manually examine large data sets to identify search criteria, then they can find relevant information, but the time required is excessive
Solution Approach 1:
The patent performs preliminary action by pre-defining data models, event types, and relationships before the search process. This preparation work organizes the data structure in advance, allowing users to immediately perform accurate searches without needing to manually examine and analyze the raw data, thereby achieving high search accuracy while minimizing time loss
3Adaptability or versatility
If search tools use only keyword-based searching, then the implementation is simple, but the ability to handle event-based and time-stamped machine data is insufficient
Solution Approach 1:
The patent segments the search system into distinct modular components: data models for structure definition, event types for classification, parameters for filtering, and relationships for contextual linking. This segmentation enables the system to handle complex machine data with event-based and time-stamped information while keeping each component manageable and the overall system organized
Data Source
AI summary
Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.