Adaptive Function Call Batching for Interconnect Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transactional API protocols face performance bottlenecks due to high latency and network resource usage when multiple function calls are invoked over a network or high-latency interconnect, leading to undesirable delays and increased data transmission.
Innovation Solution
The implementation of named batches, where repeated sequences of function calls are identified, templatized, and stored as symbolic names with placeholders for variable arguments, allowing subsequent occurrences to be transmitted efficiently, reducing data transfer by sending only the symbolic name and argument values across the interconnect.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If multiple function calls are transmitted individually over the interconnect, then the application can maintain simple transmission logic, but the latency and data transmission volume increase significantly
Solution Approach 1:
The patent combines multiple individual function call transmissions into a single batch transmission. The batch manager groups multiple function calls that would otherwise be sent separately into one consolidated transmission unit, reducing the number of interconnect transactions and thereby reducing latency while maintaining manageable transmission logic through the batching mechanism.
Solution Approach 2:
The patent performs preliminary actions by pre-processing and grouping function calls into batches before transmission. The batch manager prepares the batch of function calls in advance, organizing them into a structured format with placeholders for variable arguments, so that when transmission occurs, it can be done efficiently as a single unit rather than multiple separate transmissions.
2Productivity
If multiple function calls are transmitted individually over the interconnect, then each function call can be processed independently, but the network resource usage and data transmission volume increase
Solution Approach 1:
The patent uses copying by creating a template representation of repeated function call sequences. When the same sequence of function calls is detected, instead of transmitting the full data again, the system transmits a reference or placeholder that points to the stored template, significantly reducing data transmission volume while allowing the executer to reconstruct the original function calls independently.
Solution Approach 2:
The patent changes parameters by replacing actual argument values with placeholders in repeated function call sequences. The variable arguments are substituted with placeholder tokens, transforming the transmission from carrying full data to carrying compact references, thereby reducing data transmission volume while preserving the ability to process each function call with its actual parameters at the executer side.
3Quantity of substance
If repeated sequences of function calls are identified and templatized, then data transmission is reduced, but the system complexity increases due to batch management and template creation
Solution Approach 1:
The patent implements self-service by enabling the batch manager to automatically detect, analyze, and create templates for repeated function call sequences without external intervention. The system monitors the command stream, identifies patterns, and autonomously generates templatized versions, reducing the need for manual configuration or complex external batch management mechanisms.
Solution Approach 2:
The patent uses feedback by continuously monitoring the command stream for repeated sequences and adjusting batch management strategies based on observed patterns. The system feeds back information about detected sequences to the batch manager, which then creates or updates templates accordingly, allowing the system to adapt to usage patterns and optimize batching dynamically without increasing inherent complexity.
4Productivity
If placeholders are used for variable arguments in templatized batches, then transmission efficiency improves, but the risk of incorrect parameter substitution increases
Solution Approach 1:
The patent applies segmentation by separating the function call structure from the variable arguments. The templatized batch contains placeholders that represent specific argument positions, and the system carefully manages the substitution process by matching placeholders with corresponding actual arguments, ensuring that each placeholder receives the correct parameter value and maintaining reliability while improving transmission efficiency.
Data Source
AI summary
Embodiments described herein are generally directed to improving performance of a transactional API protocol by adaptively optimizing function call performance at runtime. In an example, a command stream is monitored that includes function calls associated with the transactional API to be carried out by an executer on behalf of an application. An amount of data transmitted over an interconnect between the application and the executer is reduced by: (i) identifying a sequence of multiple of the function calls that represents a batch and satisfies a set of one or more criteria; (ii) creating a template of the batch having a symbolic name and including placeholders for a subset of variable arguments of the multiple of the function calls; and (ii) after observing a subsequent occurrence of the sequence within the command stream, transmitting via the interconnect the symbolic name and values for the subset of variable arguments.