Database Query Processing with Asynchronous ML Inference Batching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing machine learning techniques in practical applications is challenging due to algorithmic complexities, requiring significant specialized knowledge and being difficult to integrate with traditional data processing systems like relational databases, making it resource-intensive and environment-specific.
Innovation Solution
Integrating machine learning inference calls into database queries, such as SQL queries, using a provider network that allows users to utilize machine learning services without requiring specialized knowledge, by decoupling query processing and inference using an asynchronous request handler and local machine learning models, reducing the need for network calls and enabling batch processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If machine learning inference calls are integrated into database queries, then the ease of operation is improved, but the device complexity increases
Solution Approach 1:
The system is divided into distinct modules: a query processing component that handles SQL queries, an inference component that executes machine learning models, and an integration layer that coordinates between them. This segmentation allows each component to specialize in its function while maintaining overall system manageability despite the increased complexity.
2Manufacturing precision
If machine learning techniques are implemented with specialized knowledge requirements, then the manufacturing precision is improved, but the ease of operation deteriorates
Solution Approach 1:
The patent introduces an intermediary layer that translates high-level SQL queries into machine learning inference calls. This mediator handles the complexity of model selection, parameter configuration, and inference execution, allowing users to achieve accurate results without needing specialized machine learning knowledge. The intermediary abstracts the technical complexity while preserving precision.
3Adaptability or versatility
If network calls are used for machine learning inference, then the adaptability is improved, but the productivity deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-loading machine learning models into local memory and preparing inference pipelines in advance. This allows the system to maintain adaptability by supporting multiple models while improving productivity through reduced network latency during actual query execution. The preliminary preparation of models and inference configurations enables faster processing without sacrificing versatility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for making machine learning inference calls for database query processing are described. In some embodiments, a method of making machine learning inference calls for database query processing may include generating a first batch of machine learning requests based at least on a query to be performed on data stored in a database service, wherein the query identifies a machine learning service, sending the first batch of machine learning requests to an input buffer of an asynchronous request handler, the asynchronous request handler to generate a second batch of machine learning requests based on the first batch of machine learning requests, and obtaining a plurality of machine learning responses from an output buffer of the asynchronous request handler, the machine learning responses generated by the machine learning service using a machine learning model in response to receiving the second batch of machine learning requests.