Distributed Processing System Cache Validation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Distributed key value stores face challenges in maintaining data consistency and scalability while reducing the cost of cache updates and transaction re-executions, as existing methods either increase cache update costs or fail to ensure data consistency due to cache staleness.

Innovation Solution

A distributed processing system that uses optimistic exclusion and caching, where each server independently updates its cache only when necessary, by comparing transaction data identification information to determine if the cache data is up-to-date, and re-executes transactions if optimistic exclusion fails, thereby reducing both cache update and re-execution costs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If cache update is performed before every transaction execution, then data consistency is improved, but processing speed and scalability deteriorate

Engineering Contradiction:

Improvedata consistencyVSAvoidprocessing speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent applies partial action by updating the cache only when necessary (when optimistic exclusion fails or cache is stale) rather than before every transaction. This selective approach maintains data consistency while avoiding unnecessary cache update operations that would reduce processing speed and scalability.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system performs preliminary optimistic exclusion checks and cache validation before full transaction execution. By checking whether cache data is current and whether optimistic exclusion can succeed beforehand, the system avoids unnecessary cache updates while maintaining data consistency.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If pessimistic exclusion is used for transaction control, then data consistency is improved, but scalability deteriorates

Engineering Contradiction:

Improvedata consistencyVSAvoidscalability

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent inverts the traditional pessimistic exclusion approach by using optimistic exclusion as the default mechanism. Instead of locking data before access (pessimistic), the system allows concurrent access and validates consistency afterward (optimistic), thereby improving scalability while maintaining data consistency through validation.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

Each server independently manages its own cache and transaction execution without requiring centralized locking coordination. This self-service approach allows servers to operate autonomously, improving scalability while maintaining data consistency through local validation and re-execution mechanisms.

Inventive Principle:

Principle #25Self-service

3Productivity

If optimistic exclusion is used for transaction control, then scalability is improved, but data consistency deteriorates

Engineering Contradiction:

ImprovescalabilityVSAvoiddata consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system implements feedback mechanisms by checking whether optimistic exclusion succeeded and whether cache data is current after transaction execution. When validation fails, the system triggers re-execution with cache updates, thereby maintaining data consistency while preserving the scalability benefits of optimistic exclusion.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system dynamically adjusts its behavior based on cache validity and optimistic exclusion results. Rather than using a fixed approach, the system transitions between using cached data and updating from persistent storage based on real-time validation outcomes, maintaining both scalability and data consistency.

Inventive Principle:

Principle #15Dynamics

4Productivity

If cache is used for high-speed processing, then processing speed is improved, but cache update cost increases

Engineering Contradiction:

Improveprocessing speedVSAvoidcache update cost

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS10284649B2Distributed processing system

Publication Date: 2019.05.07 NEC CORP

AI summary

The server device includes a cache, and identification information of data used in a previously executed transaction. The server device compares identification information of data to be used in a transaction received from a client, with the identification information held by it. When the comparison result shows a mismatch, the server device executes the transaction after updating the cache by using data acquired from a persistent storage device, while when the comparison result shows a match, the server device executes the transaction without updating the cache. Then, the server device determines whether optimistic exclusion succeeded or failed, and in the case of failure, re-executes the transaction after updating the data in the cache by using the data acquired from the persistent storage device.