Ring-Based Microservice Workload Failover for Data Integrity

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In batch processing systems using microservices, parallel threads can cause duplicate entries or corrupted data in destination datastores due to the failure of a thread requiring manual intervention and impacting the data pipeline system.

Innovation Solution

Implementing a ring architecture where microservice containers are deployed to process data records in a circular configuration, allowing the next container to automatically take over if one fails, ensuring seamless data processing without manual intervention.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If parallel threads are used to process data faster, then processing speed is improved, but data integrity deteriorates due to duplicate entries or corrupted data when threads fail

Engineering Contradiction:

Improveprocessing speedVSAvoiddata integrity

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent segments the data processing workload into distinct partitions, where each microservice container is assigned a specific partition to process. This segmentation prevents parallel threads from processing the same data records simultaneously, eliminating duplicate entries and data corruption while maintaining processing speed through parallel execution of different partitions.

Inventive Principle:

Principle #1Segmentation

2Reliability

If manual intervention is implemented to handle failed threads, then data integrity is maintained, but system downtime increases and productivity decreases

Engineering Contradiction:

Improvedata integrityVSAvoidsystem availability

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements a self-service mechanism where the ring architecture automatically detects when a microservice container fails and redistributes its data partition to another container in the ring. This automatic failover eliminates the need for manual intervention, maintaining data integrity while ensuring continuous system operation and high availability.

Inventive Principle:

Principle #25Self-service

3Reliability

If ring architecture is implemented for automatic failover, then system reliability is improved, but architectural complexity increases

Engineering Contradiction:

Improvefault toleranceVSAvoidarchitectural complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies universality by making each microservice container in the ring architecture capable of processing any data partition, not just its assigned partition. This multi-functionality simplifies the failover mechanism, as any container can take over failed workloads without requiring complex specialization or configuration, thereby achieving fault tolerance with manageable architectural complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12504997B2Ring architecture-based workload distribution in a microservice computing environment

Publication Date: 2025.12.23 DELL PROD LP

AI summary

Techniques are disclosed for managing workloads in data processing systems. For example, a method determines a set of containers for processing a given workload of data records, wherein each container of the set of containers is configured to process a given subset of the data records. The method causes deployment of the set of containers in a ring configuration to process the given workload of data records such that at least one of the containers in the ring configuration automatically processes one or more data records of a subset of data records assigned to another container in the ring configuration that becomes inactive.