Data Analytics System for Intelligent Storage Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing large volumes of big data is complex and time-consuming due to its voluminous and unstructured nature, posing challenges in organization, storage, and processing, including issues like missing data, inaccurate algorithms, and privacy violations.
Innovation Solution
A data analytics system that extracts data from various sources, ranks it, identifies pattern changes, and applies rules for purging and masking to reduce the data volume stored, using advanced analytics and databases to uncover hidden patterns and trends while ensuring efficient processing and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all big data is stored and analyzed, then complete data analysis capability is improved, but storage space requirements and processing complexity increase significantly
Solution Approach 1:
The patent extracts only the essential and relevant data from big data sets for storage, rather than storing all data. This is achieved through intelligent data reduction techniques that identify and retain only critical information, thereby reducing storage space requirements while maintaining data completeness for analysis purposes
Solution Approach 2:
The patent applies different processing and storage strategies to different portions of data based on their importance and characteristics. Critical data is stored in detail while less important data is aggregated or summarized, creating a hierarchical storage structure that optimizes both storage efficiency and analytical capability
2Loss of information
If all big data is stored and analyzed, then data analysis capability is improved, but processing time and complexity increase
Solution Approach 1:
The system extracts only essential data elements for storage and analysis, eliminating redundant and non-critical data. This extraction process significantly reduces the volume of data requiring processing while preserving the completeness of information needed for meaningful analysis
Solution Approach 2:
The patent performs preliminary data processing, filtering, and reduction before storage, organizing data in advance into optimized structures. This preliminary action reduces the complexity and time required for subsequent analysis operations by having data ready in an optimized state
3Volume of stationary object
If data is reduced through purging and masking, then storage efficiency is improved, but data accuracy and completeness may deteriorate
Solution Approach 1:
The patent applies selective purging and masking strategies where only non-essential or redundant data portions are reduced, while critical data maintains its full detail and precision. This localized approach ensures data accuracy is preserved for important information while achieving storage efficiency through reduction of less critical data
Data Source
AI summary
The system and method of the present disclosure relates to technology for reducing the amount of data stored in a storage system by processing subsets of data stored in data sources using advanced analytics. The process generally includes extracting data from data sources for analysis by ranking the data, marking the data, identifying pattern changes in the data, comparing pattern changes in the data and purging and/or masking the data for storage. The system also includes databases for storing and defining rules, patterns, policies and classification data to be applied to the data from the data sources and analytics to apply the rules, patterns, policies and classification information on the data. As a result, the data stored in the data sources is reduced, and processing efficiency is increased.