Fault-Tolerant Event Processing Platform Architecture

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional event processing platforms face challenges in handling large volumes of event notifications, leading to outdated information, computational strain, server crashes, and potential data loss due to the high cost of processing, which can result in system unavailability and loss of notifications.

Innovation Solution

A fault-tolerant event notification processing platform is implemented, featuring edge nodes, log aggregators, processing nodes, and a key-value data store, where event notifications are stored temporarily at edge nodes and backed up, allowing for real-time processing and recovery from faults without interrupting user experience.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If conventional event processing platforms process large volumes of event notifications, then information completeness is improved, but system reliability deteriorates due to server crashes and resource unavailability

Engineering Contradiction:

Improveinformation completenessVSAvoidsystem reliability

Core Design Contradiction:

Loss of informationVSReliability

Solution Approach 1:

The system segments the event processing workload across multiple independent processing nodes instead of relying on a single centralized system. Each node can independently receive, process, and store event notifications, ensuring that failures in one node do not cause system-wide crashes. This segmentation maintains information completeness while improving system reliability through distributed architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements preliminary buffering and persistent storage of event notifications before full processing completes. Event notifications are immediately captured and stored in a resilient manner, ensuring data is preserved even if processing nodes fail. This preliminary action guarantees information completeness is maintained regardless of subsequent system failures.

Inventive Principle:

Principle #10Preliminary action

2Loss of time

If event notifications are processed in real-time, then information freshness is improved, but computational strain increases leading to resource unavailability

Engineering Contradiction:

Improveinformation freshnessVSAvoidcomputational strain

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The processing workload is divided into smaller tasks distributed across multiple processing nodes, allowing real-time processing of event notifications without overloading a single system. This segmentation enables the platform to maintain information freshness while managing computational strain through parallel processing capabilities.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Processing nodes are designed with multi-functionality, capable of performing various processing operations on different types of event notifications. This universality allows the system to efficiently handle diverse data streams with optimized resource utilization, maintaining real-time processing capability while reducing overall computational strain through intelligent resource allocation.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If computational resources are increased to handle large-scale event processing, then processing capacity is improved, but system complexity increases making fault tolerance harder to maintain

Engineering Contradiction:

Improveprocessing capacityVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system uses segmentation to divide the large-scale processing capacity into modular, independent processing nodes. Each node maintains simplified operational logic while the collective system achieves high processing capacity. This modular approach actually reduces overall system complexity by making fault isolation and management easier, as failures can be contained to individual nodes rather than propagating system-wide.

Inventive Principle:

Principle #1Segmentation

4Reliability

If event notifications are stored at multiple components for fault tolerance, then system reliability is improved, but data storage requirements increase

Engineering Contradiction:

Improvefault toleranceVSAvoiddata storage requirements

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system merges storage capabilities across multiple processing nodes, where each node maintains event notifications locally. This combining approach provides fault tolerance through redundancy while optimizing storage utilization by allowing shared access to distributed data stores, reducing total storage requirements compared to fully replicated systems.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS10169166B2Real-time fault-tolerant architecture for large-scale event processing

Publication Date: 2019.01.01 BEIJING CHUANGXIN JOURNEY NETWORK TECH CO LTD

AI summary

Techniques described herein include an event notification processing platform configured to process large-scale event notifications in relative real time. The platform may receive event notifications from multiple sources and publish them to an event stream, or log. The platform may subsequently process each notification at a processing module according to one or more sets of rules and the processed information may be made available via a data store. Rule sets may be selected based on the type of event received by the platform. A backup data store may record event notifications as they are received or at periodic intervals. Event notification data may also be stored at multiple levels of the platform, so that in the case of a failure of one or more components of the platform, data may continue to be processed.