Cognitive Graph Insight Engine for Big Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently processing and extracting insights from large volumes of big data, particularly 'dark data,' which includes neglected or underutilized information, due to difficulties in capture, curation, storage, search, sharing, and visualization.
Innovation Solution
A cognitive information processing system comprising an insight/learning engine that applies operations to a target cognitive graph to generate cognitive insights, utilizing processes like semantic analysis, goal optimization, collaborative filtering, common sense reasoning, natural language processing, summarization, and entity resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data processing approaches are used to handle big data, then data can be processed using conventional tools, but processing efficiency becomes insufficient for large volumes of data within tolerable time intervals
Solution Approach 1:
The patent segments the cognitive processing workload by implementing a cognitive graph structure that divides data relationships into discrete nodes and edges. This allows parallel processing of different graph operations (traversals, queries, analyses) simultaneously, dramatically improving processing efficiency for large datasets while maintaining manageable time intervals
Solution Approach 2:
The patent introduces a cognitive graph as an intermediary data structure between raw big data and analysis operations. This cognitive graph pre-structures relationships and patterns, serving as a mediator that accelerates subsequent queries and analyses, reducing processing time without sacrificing comprehensive data analysis capability
2Loss of information
If dark data is collected and stored, then potential insights are available, but data becomes difficult to access, curate, and utilize effectively
Solution Approach 1:
The cognitive graph structure serves multiple functions simultaneously: it stores dark data, pre-computes relationships, enables pattern recognition, and provides structured access mechanisms. This multi-functionality reduces the complexity of data management by consolidating multiple operations into a unified framework that handles both structured and unstructured data uniformly
Solution Approach 2:
The system performs preliminary actions by pre-processing data into cognitive graph structures during data ingestion, pre-computing relationships and patterns before they are needed for analysis. This preliminary structuring of dark data makes subsequent access and utilization significantly easier, reducing the complexity of retrieving and analyzing stored information
Data Source
AI summary
An apparatus for use within a cognitive information processing system comprising: an insight/learning engine, the insight/learning engine encapsulating an operation, the operation being applied to a target cognitive graph to generate a cognitive insight.