Data Distillery Signal Detection Without Predefined Objectives
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data mining methodologies require a predefined and specific objective, limiting their ability to uncover the full potential value of large volumes of data, especially when there is a limited understanding of the data.
Innovation Solution
A computer-implemented method that collects data from multiple sources, filters viable data sources, prioritizes discovery objectives, enriches data using machine learning mechanisms and subject matter expertise, and visually displays extracted signals to facilitate understanding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional data mining methodologies are used with predefined objectives, then analysis precision is improved, but adaptability to explore full data potential is worsened
Solution Approach 1:
The system dynamically adjusts analysis objectives based on data characteristics and user feedback during the exploration process. The analytical framework evolves from predefined static objectives to adaptive, data-driven discovery objectives that emerge through iterative interaction between the system and user.
Solution Approach 2:
The analysis process is divided into multiple stages: initial data exploration, hypothesis generation, targeted analysis, and validation. This segmentation allows the system to first explore data broadly without predefined constraints, then focus on specific objectives identified during exploration, resolving the contradiction between precision and adaptability.
2Measurement precision
If complete understanding of data is required before evaluation, then measurement precision is improved, but time required for preparation is worsened
Solution Approach 1:
The system performs preliminary data exploration and characterization actions automatically before formal evaluation. It generates initial insights about data structure, patterns, and quality metrics without requiring complete understanding or manual preparation, enabling rapid start of analysis while maintaining precision through automated intelligence.
Solution Approach 2:
The system serves itself by automatically generating data characterizations, identifying patterns, and proposing evaluation objectives without requiring extensive manual preparation or complete data understanding upfront. This self-service capability reduces preparation time while maintaining evaluation accuracy through automated analytical processes.
3Productivity
If specific objective is defined for data evaluation, then analysis focus is improved, but ability to uncover hidden value is worsened
Solution Approach 1:
The system continuously incorporates feedback from data exploration results to refine and expand analysis objectives. As patterns and insights are discovered during initial exploration, these feedback loops trigger additional targeted analyses that uncover hidden values while maintaining efficiency through focused subsequent processing.
Solution Approach 2:
The system transitions from single-objective analysis to multi-dimensional exploration by automatically identifying and analyzing multiple potential objectives simultaneously. This dimensional expansion allows comprehensive value discovery across different analysis perspectives while maintaining productivity through parallel processing of multiple objectives.
Data Source
AI summary
Computer-implemented methods, systems and products for analytics and discovery of patterns or signals. The method includes a set of operations or steps, including collecting data from a plurality of data sources, the data having a plurality of associated data types, and filtering the collected data based on identifying viable data sources from which the data is collected. The method further includes prioritizing discovery objectives based on analyzing the filtering results, and enriching the filtered collected data from viable data sources according to the prioritized discovery objectives. The method further includes extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input, and graphically displaying the extracted signals in a meaningful way to a human operator such that the human operator is enabled to understand importance of extracted signals.