Optimization method and system for data layout

A technology of data layout and optimization method, applied in the direction of memory address/allocation/relocation, input/output to record carrier, etc., which can solve the problems of discrete distribution of continuous data stream, system read performance degradation, and system sequential read performance reduction. , to achieve the effect of improving sequentiality, improving read performance, and improving sequential read and write performance

Active Publication Date: 2014-03-26
INST OF INFORMATION ENG CAS
View PDF3 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the appearance of repeated data blocks causes the continuous data flow to be distributed discretely on the disk storage medium, causing a large number of random read operations during the system read process, which seriously reduces the sequential read performance of the system.
The technical solution disclosed in this patent application cannot solve the problem of system read performance degradation caused by the discrete distribution of data blocks

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Optimization method and system for data layout
  • Optimization method and system for data layout
  • Optimization method and system for data layout

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] The principles and features of the present invention are described below in conjunction with the accompanying drawings, and the examples given are only used to explain the present invention, and are not intended to limit the scope of the present invention.

[0055] like figure 1 As shown, it is a data layout optimization method described in Embodiment 1 of the present invention, which specifically includes the following steps:

[0056] Step 1: Receive the continuous data stream input by the business system;

[0057] Step 2: Divide the data stream into multiple data blocks that are continuous without crossover, and save the data blocks into the cache;

[0058] Step 3: Determine whether the data block to be processed in the cache reaches the upper limit of the cache, if yes, go to the next step; otherwise, jump to step 1;

[0059] Step 4: Calculate the hash value of all data blocks in the cache, and use each hash value as the fingerprint of the corresponding data block;...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an optimization method for data layout. The optimization method comprises the following steps: 1, receiving continuous data flow input by a service system; 2, dividing the data flow into a plurality of continuous data blocks without intersection; 3, judging whether the data blocks to be processed reach the upper limit of caching, if yes, carrying out the next step, and if not, skipping to the step 1; 4, obtaining fingerprints of all the data blocks in caching through calculation; 5, matching the fingerprints of the data blocks in caching and fingerprints in a fingerprint list, marking the matched data blocks as repeated data blocks, and storing the fingerprints which are not matched into the fingerprint list; 6, upgrading the physical location information of the data blocks in a data block information table; and 7, storing the corresponding data blocks according to the physical location information of the data blocks in the data block information table. According to the optimization method, the sequence of data is improved, discretization of data layout is lowered, and the sequential reading and writing performance of a repeated data deleting system is improved.

Description

technical field [0001] The invention relates to a data layout optimization method and system. Background technique [0002] With the explosive growth of data volume, the cost of data storage and management is getting higher and higher. Since deduplication technology can effectively reduce data volume and storage space requirements, it has become a research hotspot in the storage field. [0003] With the development of deduplication technology, it is gradually applied to the primary storage system. However, due to the characteristics of deduplication technology, it also has a certain impact on the read performance of the storage system under the advantage of reducing the amount of data storage. Aiming at the problem of data deduplication performance optimization, Chinese patent application publication number 102385554A disclosed an optimization method of a deduplication system on March 21, 2012. Using this method, the system can be based on the current load and capacity of ea...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F12/06G06F12/08G06F3/06
Inventor 周晓阳王树鹏张永铮吴广君王海平李斌斌
Owner INST OF INFORMATION ENG CAS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap