Techniques for generating and operating on in-memory datasets

Abstract

In various embodiments, a consumer generates and operates with an in-memory dataset. In operation, a read state application copies a snapshot associated with a source dataset from a file store to a random access memory (RAM) to generate an in-memory dataset. Notably, the source dataset has a first size, and the in-memory dataset has a second size that is smaller than the first size. The read state application then performs a read-only operation on the in-memory dataset to access one or more data values that are stored in the in-memory dataset and associated with the source dataset. When the one or more data values are accessed, the second data size of the in-memory dataset is maintained. Advantageously, some consumers that are unable to store the entire source dataset in are able to store the entire snapshot in RAM.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims the priority benefit of the United States Provisional Patent Application having Ser. No. 62 / 405,157 (Attorney Docket Number NETF / 0140USL) and filed on Oct. 6, 2016. This application also claims priority benefit to United States Provisional Patent Application having Ser. No. 62 / 425,767 (Attorney Docket Number NETF / 0142USL) and filed on Nov. 23, 2016. This application also claims priority benefit to United States Provisional Patent Application having Ser. No. 62 / 428,640 (Attorney Docket Number NETF / 0142USL02) and filed on Dec. 1, 2016. This application also claims priority benefit to United States Provisional Patent Application having Ser. No. 62 / 432,048 (Attorney Docket Number NETF / 0142USL3) and filed on Dec. 9, 2016. The subject matter of these related applications is hereby incorporated herein by reference.BACKGROUND OF THE INVENTIONField of the Invention[0002]Embodiments of the present invention relate generally ...

AI Technical Summary

Benefits of technology

This patent discusses a method for creating and accessing a dataset in memory. By using a snapshot of a source dataset, the method allows for faster access to data and decreases latency compared to accessing data from a remote location. This method has technical effects in improving the speed and efficiency of accessing and processing data in memory.

Problems solved by technology

One limitation of storing a dataset in RAM is that, over time, the size of the dataset typically increases.

As the size of the dataset increases, the amount of RAM required to store the dataset increases and may even exceed the storage capacity of the RAM included in a given server.

Further, because of bandwidth limitations, both the time required to initially copy the dataset to the RAM and the time required to subsequently update the copy of the dataset increase.

One drawback of this approach, however, is that the latencies associated with accessing the dataset from the remote location may increase the time required for one or more applications to respond to end-user requests to unacceptable levels.

However, oftentimes the time and effort required to redesign the access patterns of the applications is prohibitive.

Images