Field Name Recommendation for Unstructured Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of monitoring and managing web services across disparate systems and layers in network architecture makes it difficult to maintain high service performance and user experience, especially when dealing with unstructured data like log messages or business transaction stack traces, where extracting relevant field names is challenging without manual input and requires expertise in database formats and syntax.
Innovation Solution
A system that recommends field names and types for unstructured data by applying heuristics, such as context and pattern recognition, allowing users to select data portions and automatically identify field names, types, and values, reducing manual effort and improving data extraction efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual extraction of field names from unstructured data is performed, then data extraction accuracy can be maintained, but user effort and time consumption increase significantly
Solution Approach 1:
The system enables self-service by automatically recommending field names, field types, and regular expressions without requiring manual user input. The heuristic analysis engine autonomously processes unstructured data strings and generates extraction configurations, eliminating the need for users to manually specify field names while maintaining high extraction accuracy.
Solution Approach 2:
The system performs preliminary action by pre-analyzing unstructured data strings and pre-generating recommended field names, field types, and regular expressions before the user needs them. This advance processing allows users to directly apply pre-computed extraction rules without performing manual analysis, significantly reducing their time and effort.
2Measurement precision
If expertise in database formats and syntax is required for field extraction, then extraction quality can be maintained, but accessibility and ease of operation decrease
Solution Approach 1:
The system introduces an intermediary heuristic analysis engine that translates complex database format requirements and syntax rules into automatic field name recommendations. This intermediary layer handles the complexity of database formats and regular expression syntax, presenting only simple field name selections to users and thereby maintaining extraction quality while improving accessibility.
Solution Approach 2:
The system performs self-service by automatically generating field names and regular expressions based on analyzed data patterns, eliminating the need for users to possess expertise in database formats and syntax. The heuristic engine independently handles all complex formatting requirements, making the tool accessible to users without specialized knowledge while maintaining high extraction quality.
3Ease of operation
If automated field name recommendation is implemented, then user effort is reduced, but system complexity increases
Solution Approach 1:
The system segments the automated field name recommendation process into distinct modular components: a heuristic analysis engine for pattern recognition, a field name generator for creating recommendations, and a regular expression generator for extraction rule creation. This segmentation manages system complexity by organizing functions into independent, maintainable modules while delivering comprehensive automated recommendations to users.
Data Source
AI summary
In one aspect, a system for recommending a field name in data is disclosed. The system includes a processor; a memory; and one or more modules stored in the memory and executable by a processor to perform operations. The operations performed include: receive a user selection from choices of data; convert the selected data into a searchable format; receive a user selection of a portion of the selected data; apply heuristics to the portion; and recommend a field name and a field type based on the heuristics applied to the portion.