SS Table file data processing method and system

A technology for file data and processing methods, applied in the field of communication, can solve problems such as waste of storage space, and achieve the effect of reducing the amount of stored data

Inactive Publication Date: 2013-07-24
ALIBABA GRP HLDG LTD
View PDF4 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present application provides a data processing method and system based on the SSTable file format to solve the problem of wasting storage space in the existing SSTable file data storage mechanism

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • SS Table file data processing method and system
  • SS Table file data processing method and system
  • SS Table file data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Existing SSTable files only support sparse format storage, but in many cases, business data is actually dense structured data that requires dense format storage. The so-called dense format storage refers to: when storing row data, the non-empty column in the row stores the column value, and the empty column is filled with a null value object, that is, each column in the row must store a column value. For example, for the business data migrated from the database MySql or Oracle, the number of columns in each table is determined, the column changes are infrequent, most of the columns in the table are not empty, and the business often reads and writes the entire row. For this kind of application, adopting the sparse format storage method of the existing SSTable file will cause relatively large waste of space, and the reading and writing efficiency is not high.

[0030] In order to solve the above problems, the embodiment of the present application improves the storage form...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an SS Table file data processing method and an SS Table file data processing system, wherein an SS Table file is provided with a framework table, a column sequence and column attribute information of row data of the SS Table file are defined in the framework table. The method comprises the following steps of reading row data to be written into the SS Table file; writing a row major key into the SS Table file, and writing each column data of the read row data corresponding to the row major key according to the column sequence and the column attribute information of the row data which is defined in the framework table of the SS Table file. When the method and the systemare adopted to carry out data processing on the SS Table file, data reading and writing processing can be carried out according to the framework table, so that storage of row data only is realized by only storing the row major key and column values, the column values are stored according to the prescribed sequence in the framework table, and information such as column names or line IDsis needed to be stored, so that data storage volume can be reduced.

Description

technical field [0001] The present application relates to the field of communication technology, in particular to a method and system for processing SSTable file data. Background technique [0002] SSTable (Sorted String Table, sorted string table), which is essentially a file format for storing ordered Key-Value data on disk, each SSTable file consists of multiple blocks, once written, Cannot be modified, only read. [0003] At present, the SSTable format is commonly used in distributed databases to store ordered Key-Value data. Key-Value data is stored in the form of strings. Each SSTable contains many rows. A certain number of rows constitute a block, and each row contains a Primary key (RowKey) and row data. Row data generally includes several columns, each column is named as the primary key Key, and the column value is Value. figure 1 A row data storage format of an SSTable file is shown. [0004] SSTable files generally only support sparse storage formats. When wri...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 庄明强
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap