Data processing method and device

A data processing and data technology, applied in the computer field, can solve the problems of slow reading of hot data and affect the overall performance of the system, and achieve the effect of speeding up access and improving system performance.

Active Publication Date: 2016-03-30
HUAWEI CLOUD COMPUTING TECH CO LTD +1
View PDF5 Cites 45 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] However, in practical applications, there are some frequently used data (called hot data), which may be moved to lower-level storage space during compaction, resulting in the reading of these hot data become slower, which in turn affects the overall performance of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device
  • Data processing method and device
  • Data processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0065] It should be noted that the data processing methods in the embodiments of the present invention are all applied to storage systems that store data according to the LSMTree structure.

[0066] In the storage system of LSMTree, the storage format of data is a key-value structure, that is, the data stored in LSMTree contains two parameters: key value and value. Value is the data value of the data, and the key value is the index parameter of the dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Embodiments of the invention disclose a data processing method and device. The data processing method and device are used for carrying out heat statistics on heat data and redistributing the data according to the takeover of the heat statistics. The method disclosed by the embodiments of the invention comprises the following steps: determining first data to be merged in a storage space corresponding to a first target layer of an LSM Tree, wherein a key value of the first data is a first key value; determining second data whose key value is the first key value in the storage space corresponding to a second target layer of the LSM Tree; determining the first data as recently received heat data; deleting the second data from the storage space corresponding to the second target layer; migrating the first data from the storage space corresponding to the first target layer into the storage space corresponding to the second target layer; and re-writing the first data in the internal memory space of a storage system.

Description

technical field [0001] The invention relates to the field of computers, in particular to a data processing method and device. Background technique [0002] Log-structured merge-tree (LSMTree, Log-structuredmerge-tree) is a structure widely used in the field of data storage. LSMTree has made the following design for the performance impact of random I / O on persistent storage: [0003] 1. Hierarchical placement of data: data is initially written into the memory space of the storage system (a system that stores data in an LSMTree structure), and then gradually written from top to bottom to other hard disk drives (HDD, HardDiskDrive) or In persistent storage space such as solid state disk (SSD, SolidStateDrive). [0004] 2. The data is updated in different places, and the data of the key-value structure is written in the form of log. [0005] 3. The background process of the system continuously merges the data from the high level to the low level (ie from level0 to levelN), de...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 贾士博熊劲岳银亮刘扬宽
Owner HUAWEI CLOUD COMPUTING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap