Data Set Manager Reasoning Over Metadata for Cloud Policy Enforcement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data management techniques in cloud infrastructure are inadequate in handling data provenance, versioning, volatility, derivation, indexing, materialization, and state, making it difficult to assess, enforce, and audit data policies and rules, leading to inaccurate assumptions and performance issues.
Innovation Solution
A data set manager that performs reasoning over metadata, comprising a metadata capture module, a reasoning module, and an action recommendation module, to characterize data sets and their relationships, and drive actions based on explicit policies and rules, ensuring accurate and efficient data management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data management techniques are used in cloud infrastructure, then system complexity is reduced and ease of operation is maintained, but data management accuracy and reliability deteriorate due to fragmented approaches to handling data provenance, versioning, volatility, derivation, indexing, materialization and state
Solution Approach 1:
The patent introduces a data set manager as an intermediary component that sits between the complex cloud infrastructure and the data processing elements. This manager captures metadata about data sets and performs reasoning operations on this metadata to enforce policies and constraints, thereby improving data management reliability without requiring changes to the underlying complex infrastructure.
Solution Approach 2:
The system implements feedback mechanisms by capturing metadata about data set properties and relationships, then using reasoning operations to determine whether policies are satisfied. This feedback loop enables continuous monitoring and enforcement of data management rules, improving reliability through systematic validation rather than ad-hoc checks.
2Ease of operation
If policies and rules are hidden in procedural code and schedules, then device complexity is reduced, but ease of operation deteriorates as policies become difficult to assess, enforce and audit
Solution Approach 1:
The patent extracts policy definitions and data management rules from embedded procedural code and schedules, placing them into explicit metadata structures that can be captured and reasoned about. This extraction makes policies accessible and auditable without requiring changes to the operational code that implements them.
Solution Approach 2:
The data set manager acts as an intermediary that separates policy enforcement from operational code. By capturing metadata about data sets and performing reasoning operations independently of the procedural code, the system makes policies assessable and enforceable without increasing code complexity.
3Measurement precision
If optimistic or pessimistic assumptions are made about data, then device complexity is reduced, but measurement precision deteriorates leading to inaccurate data management decisions
Solution Approach 1:
The patent replaces mechanical assumptions about data (optimistic or pessimistic guesses) with a reasoning system that derives data characteristics from captured metadata. Instead of assuming data properties, the system uses logical reasoning operations on metadata to determine properties such as currency, completeness, and suitability for specific uses.
Solution Approach 2:
The system enables data sets to effectively describe themselves through captured metadata about their properties and relationships. The reasoning module then uses this self-provided information to make accurate determinations about data suitability, eliminating the need for external assumptions about data characteristics.
Data Source
AI summary
A data set manager is configured to interact with data processing elements of an information processing system. The data set manager comprises a metadata capture module configured to access or otherwise obtain metadata characterizing data sets associated with the data processing elements, a reasoning module configured to perform one or more reasoning operations on the metadata, and an action recommendation module configured to identify one or more recommended actions for the data processing elements based at least in part on results of the reasoning operations. The metadata characterizes properties of the data sets and relationships among the data sets, and may be defined in accordance with at least one of a specified ontology and a specified class. The data set manager and associated data processing elements may be implemented, by way of example, in cloud infrastructure of a cloud service provider, or on another type of processing platform.