Database Data Usage Analytics Engine for Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems lack a standardized process for analyzing data usage patterns, leading to inefficient data storage management, where unused data is often deleted or archived inconsistently, and System Administrations and Database Administrators must manually manage storage, resulting in high costs and resource wastage.
Innovation Solution
A data usage analytics engine that processes database queries to identify usage metrics, generates data usage patterns, and determines optimal storage configurations, automatically offloading unused data to low-cost storage services, thereby reducing overall costs and improving data management efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual data storage management is implemented by System Administrators and Database Administrators, then data can be stored and accessed, but significant manual effort and time are required to identify and manage data usage trends
Solution Approach 1:
The system implements self-service by automatically analyzing data usage patterns through parsing database queries and generating usage metrics without human intervention. The data usage analytics engine autonomously identifies hot, warm, and cold data categories and determines optimal storage configurations, eliminating the need for manual analysis by administrators.
Solution Approach 2:
The patent replaces manual mechanical processes with automated computational systems. Instead of administrators manually reviewing and managing storage, a computer processor executes automated query parsing, metrics generation, and storage optimization algorithms to manage data lifecycle automatically.
2Reliability
If data is stored on local platforms for legal and business requirements, then data preservation is achieved, but storage costs increase and data access efficiency decreases for frequently used data
Solution Approach 1:
The system applies local quality by storing different types of data in different locations based on their specific access patterns. Hot data (frequently accessed) is stored on local high-performance storage, while cold data (rarely accessed) is moved to remote or archival storage, optimizing both access efficiency and storage cost for each data category.
Solution Approach 2:
The patent implements dynamic storage management where data locations are not fixed but change based on usage patterns. The system continuously monitors query patterns and automatically relocates data between local and remote storage as access patterns evolve, ensuring optimal storage configuration changes over time.
3Productivity
If comprehensive data usage analysis is implemented to identify usage patterns, then optimal storage configuration can be determined, but system complexity increases
Solution Approach 1:
The analytics engine is segmented into distinct functional modules: query parsing component, metrics generation component, pattern analysis component, and storage recommendation component. Each module performs a specific function in the data usage analysis pipeline, making the overall complex system manageable through functional decomposition.
Data Source
AI summary
An embodiment of the present invention is directed to implementing a data usage analytics engine for database systems. An embodiment of the present invention is directed to implementing a Data Usage Analysis engine that receives queries (e.g., SQL queries), tables (e.g., Internal Catalog tables) and/or other data formats as input. An embodiment of the present invention may then parse the queries and identify various data usage patterns. This may include details concerning what tables are used, how much data is queried at what intervals, frequency of querying along what attributes are used in the queries and/or other usage details in various levels of granularity.