Stream Processing Dispatch Adapting Parallelism to Thread Availability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing stream processing systems face inefficiencies in scaling resources and managing computational workloads, leading to increased costs and latency due to their inability to adapt to varying computational requirements and availability, particularly in environments with constrained resources.

Innovation Solution

A stream processing framework that adapts its behavior by comparing available physical threads to logically parallel threads, allowing for concurrent processing when sufficient threads are available and multiplexing batches sequentially when resources are limited, thereby optimizing resource utilization and reducing strain on shared resources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If stream processing systems use fixed resource allocation and round-robin scheduling, then system simplicity is maintained, but resource utilization efficiency deteriorates and latency increases

Engineering Contradiction:

Improveresource utilization efficiencyVSAvoidscheduling mechanism complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system dynamically adjusts the degree of parallelism based on available computational resources at runtime. The dispatch process compares available physical threads against required logical threads and adapts processing mode (concurrent vs. multiplexed) accordingly, allowing the system to optimize resource utilization without fixed resource allocation

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the operational parameter of parallelism degree from a fixed value to a dynamically adjustable parameter. By modifying the degree of parallelism based on resource availability metrics, the system achieves efficient resource utilization while maintaining operational simplicity through automated parameter adjustment

Inventive Principle:

Principle #35Parameter changes

2Reliability

If the system allocates dedicated computation resources for each task sequence, then task processing reliability is improved, but resource waste increases and cost effectiveness deteriorates

Engineering Contradiction:

Improvetask processing reliabilityVSAvoidcomputation resource waste

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

Physical computation threads are designed to serve multiple task sequences simultaneously through multiplexed processing. A single physical thread can handle multiple batches from different task sequences sequentially, making the computation resources universal and eliminating waste associated with dedicated allocation

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system merges multiple task sequences into a shared pool of computational resources. By combining resource allocation for multiple tasks into a unified dispatch mechanism, the system ensures reliable task processing while minimizing resource waste through efficient sharing

Inventive Principle:

Principle #5Merging (Combining)

3Speed

If the system uses concurrent processing for all batches, then processing speed is improved, but resource strain increases and system stability deteriorates under constrained resources

Engineering Contradiction:

Improvebatch processing speedVSAvoidsystem stability under constrained resources

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The system dynamically switches between concurrent processing mode and multiplexed processing mode based on real-time resource availability. When physical threads are sufficient, concurrent processing maximizes speed; when resources are constrained, multiplexed processing maintains system stability by preventing resource exhaustion

Inventive Principle:

Principle #15Dynamics

4Productivity

If the system multiplexes all batches sequentially, then resource utilization is improved, but processing latency increases and productivity deteriorates

Engineering Contradiction:

Improveoverall system throughputVSAvoidbatch processing latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system dynamically selects between concurrent and multiplexed processing modes based on the ratio of available physical threads to required logical threads. This dynamic adaptation ensures that latency is minimized when resources permit while maintaining high overall throughput by switching to multiplexed mode when resources are constrained

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10198298B2Handling multiple task sequences in a stream processing framework

Publication Date: 2019.02.05 SALESFORCE INC

AI summary

The technology disclosed improves existing streaming processing systems by allowing the ability to both scale up and scale down resources within an infrastructure of a stream processing system. In particular, the technology disclosed relates to a dispatch system for a stream processing system that adapts its behavior according to a computational capacity of the system based on a run-time evaluation. The technical solution includes, during run-time execution of a pipeline, comparing a count of available physical threads against a set number of logically parallel threads. When a count of available physical threads equals or exceeds the number of logically parallel threads, the solution includes concurrently processing the batches at the physical threads. Further, when there are fewer available physical threads than the number of logically parallel threads, the solution includes multiplexing the batches sequentially over the available physical threads.