IoT Event Processing Framework with Dynamic Task Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stream processing systems face challenges in providing real-time analytics with the accuracy and completeness of batch processing systems, and they are not cost-effective due to resource wastage and latency issues, especially in handling varying computational demands and large data volumes from IoT applications.
Innovation Solution
A stream processing framework that combines batch and streaming processing modes to deliver contextual responses with low latency, using a state processing network that implements multi-step event monitoring and triggers based on time and event transitions, and allows for concurrent and multiplexed processing to adapt to varying computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stream processing systems use default schedulers with round-robin task distribution, then tasks are evenly distributed among computation resources, but substantial computation time and resources are lost during assignment and re-assignment, introducing significant latency
Solution Approach 1:
The patent implements dynamic task scheduling that adapts to varying computational demands in real-time. The system monitors resource availability and task requirements, dynamically assigning tasks to optimal computation resources rather than using static round-robin distribution. This dynamic approach eliminates resource wastage and reduces latency by ensuring tasks are assigned to available resources immediately when they become free.
Solution Approach 2:
The system performs preliminary actions by pre-allocating computation resources and maintaining readiness lists of available resources before tasks arrive. When tasks need to be executed, the system can immediately assign them to pre-ready resources without waiting for resource allocation, significantly reducing assignment latency and improving overall productivity.
2Measurement precision
If batch processing systems are used for analytics, then accuracy and completeness of analytics are achieved, but loading and processing requirements take hours to run, providing real-time results is not possible
Solution Approach 1:
The patent segments the analytics processing into two distinct modes: batch processing mode for accurate and complete analytics on historical data, and stream processing mode for real-time analytics on incoming events. The system can selectively use batch processing when accuracy is critical and stream processing when real-time results are needed, effectively resolving the contradiction between processing speed and analytical precision.
Solution Approach 2:
The system changes processing parameters dynamically based on query requirements. For real-time analytics queries, the system switches to stream processing parameters with lower latency tolerance. For comprehensive analytics queries, it switches to batch processing parameters with higher accuracy requirements. This parameter adaptation allows the same system to optimize for different performance characteristics as needed.
3Ease of operation
If non-experts want to combine and harness big data computing and analytics, then domain knowledge can be applied directly, but reliance on expensive technical consultants is required
Solution Approach 1:
The patent implements a self-service platform where non-expert users can directly configure and execute big data analytics queries using intuitive interfaces. The system automatically handles complex task scheduling, resource allocation, and data processing without requiring technical consultants. Users can define their analytics needs through simple parameter inputs, and the system autonomously manages the entire processing pipeline, eliminating dependency on expensive technical expertise.
Data Source
AI summary
The technology disclosed relates to processing events generated by Internet of Things (IoT) devices. In particular, it relates to storing a machine-readable declarative specification of stateful event processing of an automated multi-step progression of monitoring of Internet of Things (IoT) devices that generate events. It includes compiling into tangible memory, in response to the declarative specification, a state processing network that implements a multi-step progression of monitoring events generated by the IoT devices. The state processing network implements both the time based transition triggers and the event based transition triggers after being initiated by the IoT devices. Further, data structures, which record monitoring status of particular IoT devices, are compiled. Finally, the state processing network further selects an alternative action to trigger, and triggers the selected action using at least some data from a particular event being processed.