Cognitive Learning Lifecycle for Dark Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently processing and analyzing large volumes of complex data, including 'dark data,' which is often neglected or underutilized, making it difficult to extract actionable insights.
Innovation Solution
A cognitive information processing system comprising a processor, data bus, and non-transitory computer-readable storage medium with computer program code that performs cognitive learning operations through a cognitive inference and learning system, applying various techniques like semantic analysis, goal optimization, and natural language processing to generate cognitive insights from diverse data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data processing approaches are used, then processing simplicity is maintained, but processing efficiency and capability to handle big data deteriorate
Solution Approach 1:
The patent segments the cognitive learning process into distinct lifecycle phases (data collection, data preparation, modeling, evaluation, deployment) with specialized components for each phase. This segmentation allows the system to handle complex big data processing through modular, manageable stages rather than attempting monolithic processing.
Solution Approach 2:
The patent introduces a cognitive inference and learning system as an intermediary layer between raw data and actionable insights. This intermediary system includes specialized components (data collectors, preprocessors, model trainers, evaluators) that mediate the complex transformations required to convert diverse data formats into meaningful patterns and predictions.
2Loss of information
If dark data is collected and processed, then actionable insights are improved, but data processing complexity increases
Solution Approach 1:
The patent applies preliminary data preparation and preprocessing operations before main processing. The data preparation phase includes cleaning, normalization, and feature extraction that prepares dark data for subsequent analysis, making it more amenable to processing and reducing complexity in later stages.
Solution Approach 2:
The patent replaces traditional mechanical data processing methods with cognitive computing techniques including natural language processing, semantic analysis, and machine learning. These cognitive methods automatically interpret and derive meaning from unstructured dark data without requiring manual processing rules.
3Loss of information
If cognitive learning techniques are applied, then insight generation is improved, but computational resources required increase
Solution Approach 1:
The patent implements iterative learning where the cognitive system processes data in incremental batches rather than all at once. The model is trained progressively through multiple epochs, allowing computational resources to be reused across iterations rather than requiring peak resources for complete reprocessing.
Solution Approach 2:
The patent incorporates feedback mechanisms where model performance is continuously evaluated and used to adjust processing parameters. The evaluation phase provides feedback to the modeling phase, enabling the system to optimize computational resource usage by stopping training when performance plateaus or by adjusting model complexity based on data characteristics.
Data Source
AI summary
A cognitive information processing system comprising a processor; a data bus coupled to the processor; and a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for: receiving data from a plurality of data sources; processing the data from the plurality of data sources to perform a cognitive learning operation, the processing being performed via a cognitive inference and learning system, the cognitive learning operation comprising a plurality of cognitive learning operation lifecycle phases, the cognitive learning operation applying a cognitive learning technique to generate a cognitive learning result; and, updating a destination based upon the cognitive learning result.