Prediction Models for Matching Data Analytics Task Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Organizations face challenges in efficiently retrieving relevant data for data analytics tasks due to complex data structures and murky data, leading to subjective analysis and significant time consumption, especially when analysts lack experience with the organization's data set.
Innovation Solution
Utilizing prediction models to match stored parameters to a new data analytics task by employing unsupervised, semi-supervised, or supervised machine learning models to analyze historical data analytics tasks and provide selective matches to users, enhancing data exploration and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If analysts manually select data from complex data structures, then they can retrieve relevant data for analysis, but the process consumes significant time and requires extensive experience
Solution Approach 1:
The system pre-processes and stores metadata about data structures, relationships, and historical analytics tasks before they are needed. When a new analytics task is received, the prediction model quickly retrieves and matches against this pre-organized information, avoiding time-consuming manual exploration of complex data structures while maintaining high accuracy in identifying relevant data.
2Measurement precision
If analysts rely on team members for data selection, then they can overcome lack of experience, but the process becomes subjective and time-consuming
Solution Approach 1:
The prediction model enables the analytics task system to automatically select relevant data without human intervention. The model learns from historical analytics tasks and data patterns, then autonomously identifies and retrieves appropriate data structures, tables, and fields for new analytics tasks, eliminating subjective human judgment and manual effort while improving accessibility for users of all experience levels.
3Measurement precision
If extensive experience is required for data selection, then data retrieval accuracy improves, but the system becomes less accessible to less experienced analysts
Solution Approach 1:
The prediction model acts as an intermediary between the analytics task requirements and the complex data structures. It translates high-level task descriptions into specific data selection decisions by learning from historical patterns, thereby bridging the gap between user intent and accurate data retrieval without requiring users to have deep domain knowledge or experience with the data architecture.
Data Source
AI summary
Aspects of the present disclosure are directed to prediction models that match stored parameters to a new data analytics task. A data analytics task can refer to a task that analyzes and queries data sources to retrieve target data. Example parameters for a data analytics task include a task description, assigned individuals, relevant databases, relevant data tables, search queries, data table joins, etc. Implementations can store historical data analytics tasks and their task parameters. Prediction model(s) can be provided new data analytics task parameters and predict matching historical data analytics tasks for the new task. The matching data analytics tasks can then be analyzed to select data tables common among them, and the common data tables can be provided to a user. Implementations can also group additional parameters according to a selected common data table and provide the grouped additional parameters to the user.