Low Latency Gateway for Asynchronous Orchestration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Orchestration engine systems face latency issues due to garbage collection in virtual machines, which can lead to service level agreement breaches and transaction failures in online transaction systems.
Innovation Solution
Implementing a low latency gateway system that uses direct memory management by pre-allocating memory for data pipelines, reducing memory creation and cleanup overhead, and employing a memory allocation component to sequentially store and reuse memory blocks for transaction data, thereby minimizing latency and improving cache utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a gateway processes large numbers of transactions using traditional memory management in a virtual machine environment, then the system can handle high transaction volumes, but latency is introduced due to garbage collection operations
Solution Approach 1:
The patent segments memory management by creating separate memory pools for different data types (request data, response data, payload data) and using dedicated buffers for each segment. This segmentation allows independent management of memory regions, enabling the system to process high transaction volumes while avoiding global garbage collection pauses that affect the entire memory space.
Solution Approach 2:
The patent implements preliminary action by pre-allocating memory buffers and pools before transaction processing begins. Memory is allocated in advance for request queues, response queues, and payload storage, eliminating the need for dynamic memory allocation during transaction processing. This pre-allocation strategy prevents latency caused by runtime memory management operations.
2Ease of operation
If traditional memory allocation and cleanup operations are performed for each transaction, then memory can be dynamically managed, but memory creation and cleanup overhead increases latency
Solution Approach 1:
The patent implements self-service memory management where memory buffers are automatically reused within their respective pools without requiring external allocation or cleanup operations. When a buffer is released, it returns to its pool for automatic reuse by the next transaction of the same type. This self-service mechanism eliminates the overhead of dynamic memory allocation and deallocation while maintaining flexibility.
Solution Approach 2:
The patent applies discarding and recovering by implementing circular buffers and reusable memory pools where released memory is immediately recovered and made available for reuse. Instead of freeing memory to the system heap, buffers are returned to their designated pools, ensuring continuous availability without allocation overhead. This approach maintains memory flexibility while eliminating repeated allocation/cleanup cycles.
3Adaptability or versatility
If dynamic memory allocation is used for transaction processing, then memory can be adapted to varying transaction sizes, but memory management complexity increases
Solution Approach 1:
The patent implements dynamics by creating adaptive memory pools that can dynamically allocate buffers of varying sizes within predetermined ranges. Each pool is configured with minimum and maximum buffer sizes, allowing flexible allocation based on transaction requirements while maintaining structured management. This dynamic approach provides adaptability without the complexity of fully dynamic memory allocation.
Data Source
AI summary
Systems and techniques for providing a low latency gateway for an asynchronous orchestration engine using direct memory are presented. A system can directly allocate an array memory space within a first data structure for transaction data associated with transaction requests for an online transaction system. The system can sequentially store respective data threads of the transaction data into respective memory blocks of the array memory space within the first data structure. The system can also sequentially separate the memory blocks of the array memory space within the first data structure into data channels for storage in a second data structure. Furthermore, the system can respectively format data channels and convert the data channels into communication pathways for the online transaction system based on at least one serialization technique for transmission to one or more memories of a virtual machine of the online transaction system.