File storage dynamic aggregation based optimization method

An optimization method and dynamic aggregation technology, applied in the fields of instrumentation, computing, electrical digital data processing, etc., can solve problems that are not necessarily the best overall benefit

Active Publication Date: 2013-03-20
STATE GRID CORP OF CHINA +1
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There is such a problem in this model when evaluating the value of data replication: nodes often make decisions according to their own interests, so they may not necessarily get the best global benefits

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • File storage dynamic aggregation based optimization method
  • File storage dynamic aggregation based optimization method
  • File storage dynamic aggregation based optimization method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] An optimization method based on dynamic aggregation of file storage according to the present invention. By dynamically combining files, the optimization method obtains copies of files frequently accessed by nodes and packs and stores them in local nodes, thereby reducing the number of times nodes read files remotely, reducing User data access delay, the specific steps are as follows:

[0039] 1) Determine whether the data overlaps based on the data master, and check whether the request file sent by the user is in the local node. If there is data overlap, the request file exists in the local node; if there is no data overlap, go to step 2);

[0040] Suppose there are n nodes in the system , each node contains m files, the user sends a file request to a node when accessing, assuming that the request contains multiple files, the request can be expressed as a file collection .

[0041] at node There is a data master , then by generated collection is the object t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a file storage dynamic aggregation based optimization method. In the optimization method, a copy of a file which is frequently accessed by a node is acquired and is packed and stored at a local node by dynamically combining the file, so that frequency of remotely reading the file by the node is reduced and user data access delay is reduced. The method comprises the following steps of: 1, determining whether data is overlapped on the basis of a data original, detecting whether a request file sent by a user is in a local node, if the data is overlapped, judging that the request file is in the local node, if the data is not overlapped, turning into the step 2; 2, detecting whether sufficient storage space is locally available to accommodate a request file based on a copy placement optimization algorithm, if yes, selecting a remote node with minimum performance cost to obtain a request file copy, is not, turning into the step 3, wherein the cost comprises file transmission time and queuing time on the remote node; and 3, based on a storage space optimization algorithm, calculating times t1 of reading the request file by the node, calculating times t2 of readingless files from all files stored in the local node, if t1 is more than t2, deleting the t2 and clearing space for storing the request file. The cost of the copy selected to be replicated by the method is the minimum at the present and the cost is also the minimum in later time.

Description

technical field [0001] The invention belongs to the field of distributed storage, in particular to an optimization method based on file storage dynamic aggregation. Background technique [0002] Distributed storage has established an integrated storage, processing, access, transmission and service architecture and heterogeneous distribution environment for massive data. Data replication can not only improve the efficiency of data access, but also improve the load balance and reliability of the system. [0003] The traditional replication method is to replicate a data master to replicas at different sites. The reason for this situation is that some past or present replication requirements and processing procedures are relatively simple. With the development of data replication technology, the situation has undergone many changes. Different replication source copies can be divided on the same data source object, and these different replication source copies have their own mult...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 王纪军唐巍洪昕
Owner STATE GRID CORP OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap