Cognitive Data Management for Dark Data Insight Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently processing and analyzing large volumes of big data, particularly 'dark data,' which is often neglected or underutilized, making it difficult to extract actionable insights in a timely manner.
Innovation Solution
A cognitive information processing system that receives data from multiple sources, including public and private data sources, using a cognitive data management module to access and provide information to an inference and learning system, employing techniques like semantic analysis, goal optimization, collaborative filtering, and natural language processing to generate cognitive insights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data processing approaches are used to handle big data, then data storage and basic processing can be achieved, but the ability to efficiently analyze and extract actionable insights from dark data deteriorates
Solution Approach 1:
The patent segments data into multiple categories including dark data, structured data, unstructured data, and semi-structured data. This segmentation allows the system to apply different processing approaches to different data types, enabling efficient analysis of dark data while maintaining overall system organization and scalability.
Solution Approach 2:
The patent introduces a cognitive data management module as an intermediary between raw data sources and the inference/learning system. This module performs semantic analysis, goal optimization, and collaborative filtering to transform raw data into actionable insights, bridging the gap between data storage and insight extraction.
2Loss of information
If comprehensive data collection from multiple sources is implemented, then data completeness and analysis potential improve, but system complexity and processing difficulty worsen
Solution Approach 1:
The patent implements a universal cognitive data management module that handles multiple data types (structured, unstructured, semi-structured) from various sources (public, private, social, device) through a single integrated system. This multi-functional approach maintains data completeness while avoiding the complexity of separate processing systems for each data type.
Solution Approach 2:
The patent changes the processing parameters dynamically based on data characteristics. The cognitive data management module adjusts processing approaches according to data type, source, and analysis goals, enabling comprehensive data collection while maintaining manageable system complexity through adaptive parameter adjustment.
3Loss of information
If advanced cognitive processing techniques are applied to dark data, then actionable insight extraction improves, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary processing of dark data through the cognitive data management module, which conducts semantic analysis and goal optimization in advance. This preliminary action prepares data for faster subsequent analysis by the inference and learning system, reducing overall processing time while maintaining comprehensive insight extraction.
Solution Approach 2:
The patent applies cognitive processing techniques selectively to the most valuable dark data based on analysis goals. Rather than processing all dark data uniformly, the system identifies and prioritizes high-value data segments for advanced processing, achieving actionable insights with reduced processing time and computational resources.
Data Source
AI summary
A method for receiving a plurality of types of data within a cognitive information processing system environment comprising: receiving data from a plurality of data sources, the plurality of data sources comprising a public data source and a private data source; accessing information from the plurality of data sources via a cognitive data management module; and, providing the information to an inference and learning system.