Storage internal index construction method for improving query and storage performance of novel memory

Abstract

The invention discloses a storage internal index construction method for improving query and storage performance of a novel memory. The method includes: when RDF graph data is stored, sequencing the RDF graph data, then dividing the sequenced RDF graph data into different pages, establishing a skip list capable of realizing binary search for the pages, wherein the structure of nodes in the skip list is (k, v) form, k corresponds to node information, and v corresponds to a logic address; then, storing the ordered RDF graph data into a solid state disk (SSD); and establishing a mapping table of the physical address and the logic address of the RDF graph data storage by utilizing the FTL, and storing the mapping table. According to the method, the SSD can be quickly searched and efficiently managed, and the method has a very strong competitive advantage.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a method for constructing an in-storage index that improves the query and storage performance of a new type of storage. Background technique [0002] Common approaches to storing RDF graph data include memory-based storage. This method mainly puts the graph data in memory, and can use page-level mapping and block-level mapping when reading data. The page-level mapping is based on the page of flash memory (flash) as the mapping granularity. Users access solid-state drives (SSDs) through logical page addresses (LBAs). SSDs read and write based on physical page addresses (PBAs). When users write a logical page of data, the SSD master will find a physical page Write data, and establish a mapping between physical pages and logical pages through the mapping table, and physical pages can be mapped to any logical page, but the mapping information needs to be saved for later readin...

AI Technical Summary

Problems solved by technology

[0003] However, with the development of the Internet, huge RDF graph data needs to occupy a large amount of memory, which means that a large amount of memory is required to store mapping information. When a certain graph data needs to be queried, it also takes a long time to traverse the mapping table, resulting in Low efficiency; when the device restarts, it needs to scan the entire flash to build the mapping information table, which further leads to a waste of time

[0004] Aiming at the problem that the page-level mapping index requires a large amount of memory, block-level mapping can be used to store RDF graph data, and the related triplet graph information can be stored in the block SSD. A mapping table needs to be established in the SSD to store physical The mapping relationship between blocks and logical blocks, assuming that each block contains 512 pages, then the index established by the mapping becomes 1 / 512 of the page-level mapping, although the memory occupied by the RDF mapping is greatly reduced, but the block The location of the data page is fixed. When a data page needs to be updated, the entire data block needs to be updated, so there is no efficient page-level mapping.

[0005] Additionally, to prevent workloads with a large number of small updates, the entire flash is written regardless of whether the page is full, which increases write amplification and makes block-level mapping generally inefficient

[0006] With the development of the Internet, RDF data has increased significantly. Therefore, how to effectively store and manage massive RDF data has become a huge challenge. Although many methods have been proposed, such as RDF-3X, RDF Cube, etc., but Slow query speed is a fatal weakness. How to improve RDF storage efficiency and query speed in the case of large amounts of data has become the key to research

Images