Resource Management System for Data Warehouse Concurrency Control

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Managing access to data in data warehouses is challenging due to concurrency issues and the need to interface with multiple data sets managed by different providers, leading to conflicts and inefficiencies in resource acquisition.

Innovation Solution

A resource management system that generates tickets for resource requests, queries each resource provider to determine availability, and either grants access if all resources are available or queues the request if not, preventing deadlocks and allowing for priority-based queuing to optimize resource allocation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple users access the same data simultaneously in a data warehouse, then user productivity and data utilization are improved, but data consistency and reliability deteriorate due to concurrency issues

Engineering Contradiction:

Improveuser productivityVSAvoiddata consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary actions by checking resource availability and acquiring locks before allowing data access. The resource manager verifies that required resources are available and prevents concurrent access by acquiring exclusive locks on data pages, thus maintaining data consistency while enabling productive multi-user access.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The resource manager acts as an intermediary between multiple users and the data warehouse resources. It mediates access requests by managing locks and resource allocation, ensuring that data consistency is maintained while allowing multiple users to productively access and modify data.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If a resource management system checks availability of all resources before granting access, then data reliability and consistency are improved, but the time required for resource allocation increases

Engineering Contradiction:

Improvedata consistencyVSAvoidresource allocation time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary resource availability checks and lock acquisitions in a single atomic operation before granting access. By checking all required resources upfront and acquiring necessary locks in advance, the system ensures data consistency while minimizing the time resources are locked, thus reducing overall allocation time.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The resource management process is segmented into distinct phases: resource checking, lock acquisition, and access granting. This segmentation allows the system to efficiently manage the time-consuming availability checks separately from the actual data access operations, reducing the perceived allocation time while maintaining reliability.

Inventive Principle:

Principle #1Segmentation

3Reliability

If the system queues resource requests that cannot be immediately granted, then resource allocation reliability is improved, but the time delay for request fulfillment increases

Engineering Contradiction:

Improveresource allocation reliabilityVSAvoidrequest fulfillment time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system implements periodic action by checking the status of queued requests at regular intervals and attempting to grant access when resources become available. This periodic checking mechanism maintains reliable resource allocation by ensuring requests are fulfilled in order while minimizing delays through continuous monitoring and immediate grant when possible.

Inventive Principle:

Principle #19Periodic action

4Adaptability or versatility

If the resource management system interfaces with multiple different data providers, then system versatility and adaptability are improved, but system complexity increases due to different interface techniques

Engineering Contradiction:

Improvesystem versatilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The resource manager is designed with universal functionality to interface with multiple different data providers through a standardized interface. It can acquire locks and manage resources across various data warehouses and providers using a unified approach, thus achieving system versatility without proportionally increasing complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The resource manager serves as an intermediary layer between the user applications and multiple diverse data providers. It abstracts the complexity of different provider interfaces by implementing a unified lock management mechanism that works across all providers, thereby maintaining system versatility while managing complexity centrally.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS9229949B2Extensible and generic framework for managing resources for data warehousing loads

Publication Date: 2016.01.05 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Methods, systems, and computer program products are provided for managing resources. A request is received for a plurality of resources from a requester. A ticket is generated for the request that indicates the resources. For each resource, a resource provider for the resource is queried to determine whether the resource can be acquired. The ticket is queued if any of the resources cannot be acquired. The ticket is granted if all of the resources can be acquired.