Centralized Data Management in Distributed Cloud Services

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods fail to provide a continuous and complete process for managing data related to functional and non-functional organizations across distributed storage servers, leading to challenges in timely and trackable data management and deletion.

Innovation Solution

A centralized architecture is introduced, featuring an aggregating server that communicates with distributed publishing and subscribing servers, enabling continuous data management and deletion processes. This architecture includes metadata segmentation, state-indicating cursors, and a cloud data pipeline for efficient data processing and monitoring.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a distributed storage architecture is used to store data across multiple geographical locations, then data availability and accessibility are improved, but data management complexity and tracking difficulty increase

Engineering Contradiction:

Improvedata accessibilityVSAvoiddata management complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a centralized control system that acts as an intermediary between distributed storage nodes. This control system coordinates data placement, migration, and deletion operations across the distributed architecture, providing a unified management interface that simplifies tracking and management while maintaining the benefits of distributed storage accessibility.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent segments data management operations into distinct components: metadata management, data placement policies, migration orchestration, and deletion coordination. This segmentation allows each component to be managed independently, reducing overall system complexity while maintaining distributed architecture benefits.

Inventive Principle:

Principle #1Segmentation

2Reliability

If data is continuously moved across distributed storage servers due to tenant migrations and instance refreshes, then data freshness and functionality are improved, but data loss risk and management overhead increase

Engineering Contradiction:

Improvedata freshnessVSAvoiddata loss risk

Core Design Contradiction:

ReliabilityVSLoss of substance

Solution Approach 1:

The patent implements backup and verification mechanisms before data migration operations. Data is validated and backed up prior to movement, creating a safety cushion that prevents data loss during continuous data movement operations caused by tenant migrations and instance refreshes.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Solution Approach 2:

The patent incorporates feedback loops that continuously monitor data integrity and migration status. This feedback mechanism detects and corrects issues during data movement, ensuring data freshness while minimizing loss risk through real-time validation and error correction.

Inventive Principle:

Principle #23Feedback

3Ease of operation

If existing data management methods are used without a centralized framework, then system simplicity is maintained, but data deletion completeness and tracking capability deteriorate

Engineering Contradiction:

Improvesystem simplicityVSAvoiddata deletion completeness

Core Design Contradiction:

Ease of operationVSManufacturing precision

Solution Approach 1:

The centralized control system serves as an intermediary that coordinates deletion operations across distributed storage nodes. This intermediary ensures complete and trackable data deletion by managing the entire deletion process centrally, while individual storage nodes simply execute deletion commands without requiring complex local decision-making logic.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12339867B1Data management in a large scale distributed cloud service

Publication Date: 2025.06.24 SALESFORCE INC

AI summary

A computer-implemented method is disclosed for storing source data related to targeted and untargeted tenants on several publishing servers. The method includes sending metadata related to a first part of the source data related to the targeted tenants from the publishing servers to an aggregating server and storing the metadata in a first database including metadata segments arranged in a first sequenced array and having corresponding state-indicating cursors arranged in a second sequenced array beginning with a first cursor and ending with a last cursor. The method also includes storing the second sequenced array on a second database, querying the first cursor of the second sequenced array from a subscribing server including targeted data related to at least a part of the tenants, querying the metadata segment corresponding to the first cursor, and performing a predetermined operation on at least a part of the targeted data.