A data processing method and related equipment

A data processing and metadata technology, applied in the storage field, can solve the problems of log structure merge tree large merge overhead, inability to meet the requirements of storage performance, limited number of times of hard disk erase and write, etc., to improve the lower layer capacity and watt record storage capacity , the effect of increasing sequential read and write operations and reducing IO overhead

Active Publication Date: 2021-08-17
WENHUA UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In terms of software, under the background of big data, the traditional file system or block storage can no longer meet the storage performance requirements of most applications, and the key-value storage based on the log-structured merge tree has become a large-scale data center due to its excellent service quality. The main storage method of the application
[0004] Although solid-state drives, tile-record disks, and log-structure merged tree structures all have such obvious advantages, they also have their own defects, for example: the number of erasing and writing of solid-state disks is limited, first erase and then write; Amplification problem; and log-structured merge tree huge merge overhead

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing method and related equipment
  • A data processing method and related equipment
  • A data processing method and related equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0063] The embodiment of the present application provides a data processing method and related equipment, which can comprehensively utilize the advantages of solid-state hard disks, tile-record disks, and log-structured merge trees to optimize storage of disk data.

[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is no...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the present application provides a data processing method and related equipment, which can comprehensively utilize the advantages of solid-state equipment, tile recording disks, and log-structured merge trees to optimize the stored data. The method includes: acquiring target data; determining a target ordered key-value pair table corresponding to the target data; determining a second ordered key-value pair table according to the target ordered key-value pair table; The value pair table determines the first ordered key-value pair table; merges the second ordered key-value pair table into the first ordered key-value pair table; when the merged first ordered key-value pair table When the table meets the preset condition, write the merged first ordered key-value pair table and the third ordered key-value pair table as an array set into the target dynamic area whose size dynamically changes in the tile record disk .

Description

technical field [0001] The present application relates to the field of storage, and in particular to a data processing method and related equipment. Background technique [0002] With the rapid development of the information age, data in various fields has shown explosive growth. According to the research report of IDC, from 2010 to 2020, the amount of global data will increase by 50 times, and the annual growth rate of the total amount of global data will maintain about 50% in the future. Among them, the amount of data in my country will reach 8.6ZB, accounting for about 21%. In addition, delay-sensitive interactive Internet applications (such as Weibo, Twitter, and Facebook) are gradually increasing. While requiring high response speed, such applications also encourage users to publish and share files and pictures. Therefore, the storage system has been challenged unprecedentedly in terms of capacity and performance. [0003] In terms of hardware, among the various capa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06
CPCG06F3/061G06F3/0616G06F3/0634G06F3/064G06F3/068
Inventor 詹玲俞侃王中华姚婷万继光
Owner WENHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap