Extensible Stream Objects for External Data Querying

The patent segments the large table into multiple smaller partitions or chunks that can be independently processed. Instead of scanning the entire table, the query system divides the data into manageable segments, processes only relevant segments based on query predicates, and combines results. This segmentation enables parallel processing and reduces the time required to execute queries on very large datasets.

The patent extracts and utilizes metadata about data partitions, such as minimum and maximum values for different columns. This metadata allows the system to determine which partitions contain data relevant to the query without scanning all data. By extracting this summary information, the system avoids processing irrelevant partitions, significantly reducing computing resource consumption while maintaining query accuracy.

The patent introduces stream objects as intermediary components that bridge the database system and external data stream platforms. These stream objects encapsulate the complexity of external stream processing, providing a standardized interface for data ingestion and querying. By using this intermediary layer, the system can leverage external streaming capabilities without directly managing the underlying platform complexity, thus improving productivity while controlling system complexity.