In-Memory Data Retrieval Bypassing SQL Query Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Standard Structured Query Language (SQL) database queries are resource-intensive and slow, particularly when dealing with real-time data acquisition and display, as they require time-consuming parsing, optimization, and engine processing, leading to increased bandwidth and processing time.
Innovation Solution
Implementing a SQL bypass query mechanism that uses in-memory retrieval to directly access and update data, allowing only changed data values to be transmitted to client devices, which then update the user interface accordingly, reducing the amount of data transmitted and processing required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If standard SQL database queries are used for real-time data acquisition, then data can be retrieved from the database, but the process is slow and resource-intensive due to parsing, optimization, and engine processing requirements
Solution Approach 1:
The patent extracts the data retrieval operation from the complex SQL query processing system and implements a direct in-memory access mechanism. Instead of going through the full SQL parsing, optimization, and engine processing pipeline, the system directly queries the data structure in memory, eliminating unnecessary processing steps and achieving faster data retrieval with reduced computational complexity
Solution Approach 2:
The patent creates a copy of the database data in memory that can be directly accessed without going through the SQL query processing pipeline. This in-memory copy allows for rapid data retrieval by bypassing the complex SQL engine, while the original database remains intact for persistence and ACID compliance
2Loss of information
If all data is transmitted to client devices for real-time updates, then complete data information is available, but bandwidth usage increases and processing time is extended
Solution Approach 1:
The patent extracts only the changed data portions from the complete data set and transmits them to client devices. Instead of sending all data to ensure completeness, the system identifies and transmits only the specific data elements that have changed, maintaining data integrity while significantly reducing bandwidth consumption and client-side processing requirements
Solution Approach 2:
The patent applies different transmission strategies to different portions of data based on their change status. Changed data is transmitted in full detail, while unchanged data is not transmitted at all. This localized approach ensures that clients receive complete information about changes while avoiding unnecessary transmission of static data, optimizing both bandwidth usage and information completeness
3Reliability
If standard SQL queries are used for each data update, then data accuracy is maintained, but processing time increases and real-time performance is degraded
Solution Approach 1:
The patent performs preliminary actions by maintaining the database data in memory in a structured format that is ready for direct access. This pre-arranged in-memory representation allows for immediate data retrieval and change detection without requiring time-consuming SQL query execution at the moment of update, thus maintaining data accuracy while reducing update processing time
Solution Approach 2:
The patent implements continuous monitoring of data changes in memory without interrupting the data flow or requiring periodic SQL query resets. The system continuously compares new data against the in-memory copy, identifies changes, and transmits them immediately, maintaining both data accuracy and real-time performance through uninterrupted processing
Data Source
AI summary
An apparatus includes a processor and a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to perform operations including receiving a request for object identifiers, the request associated with a unique session, sending an object identifier, the object identifier associated with an object from a list of objects each having a corresponding object identifier, receiving a request for data, the request referencing the object identifier and a data value associated with the object, performing an in-memory retrieval for the data value associated with the object identifier, and transmitting the data value.