Batch Mode Execution for Remote Service Calls in Database Engines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing machine learning techniques in practice is challenging due to algorithmic and mathematical complexities, requiring significant specialized knowledge and being resource-intensive, with models often focused on specific use cases and environments, making them difficult to adapt and deploy effectively, especially when integrated with traditional software engineering and relational databases.
Innovation Solution
A batch mode execution method for calls to remote services, including a batch manager that processes multiple rows as a single request, optimizing network connections and computational efficiency by using batch iterators and operators to offload processing-intensive tasks like machine learning inference to scalable services, allowing query services to be hosted on commodity hardware while improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional row-by-row processing is used in database engines, then simplicity of implementation is maintained, but performance and resource utilization deteriorate when calling remote services
Solution Approach 1:
The patent segments the processing workflow into distinct batch mode operators that handle different aspects of batch processing (batch collection, batch execution, result aggregation), allowing complex remote service calls to be broken down into manageable, specialized components that can be optimized independently
Solution Approach 2:
The patent merges multiple row processing operations into a single batch mode execution unit, combining N rows into one request to remote services. This merging approach amortizes network connection costs and improves throughput by processing multiple data items in parallel rather than sequentially
2Adaptability or versatility
If machine learning services are integrated into traditional database systems, then functional capability is improved, but infrastructure complexity and specialized knowledge requirements increase
Solution Approach 1:
The patent creates a universal batch mode execution framework that can handle various types of remote service calls (machine learning inference, data processing, analytics) through a common architecture. This multi-functional approach allows traditional database systems to integrate diverse services without requiring separate infrastructure for each service type
Solution Approach 2:
The patent introduces batch mode operators as intermediary components between the database engine and remote services. These operators serve as a mediation layer that translates database queries into batch requests suitable for remote services, handling protocol conversion, batching, and result aggregation, thereby simplifying integration without exposing infrastructure complexity to end users
3Productivity
If network calls are made for each individual row, then data processing accuracy is maintained, but network costs and processing time increase significantly
Solution Approach 1:
The patent merges N individual row processing operations into a single batch request, combining multiple data items into one network call. This merging strategy maintains data processing accuracy by ensuring each row is still processed individually by the remote service, while dramatically reducing network overhead by amortizing connection establishment and protocol handling costs across multiple rows
Solution Approach 2:
The patent enables continuous batch processing where the batch mode execution framework maintains persistent network connections and continuously streams batches of rows to remote services. This continuous action eliminates the start-stop pattern of individual row processing, keeping network resources utilized efficiently throughout the query execution period
Data Source
AI summary
Techniques for batch mode execution for calls to remote services are described. A method of batch mode execution for calls to remote services may include generating, by a query service of a provider network, a query plan to optimize a query for batch processing of data, the query plan including at least a function reference to a function provided by at least one service of the provider network, executing the query plan to invoke the function associated with the function reference, wherein a batch function generates a request including a batch of service calls to be processed by the at least one service, sends the request including the batch of service calls to the at least one service, and obtains a plurality of machine learning responses from the at least one service, and generating a query response based on the plurality of responses.