Fuzzy balance method and system for distributed storage pool
A technology of distributed storage and balancing system, applied in the field of balancing of distributed storage pools, can solve the problems of single point failure, affecting the stability of data service quality system, unbalanced load of distributed system, etc., to achieve two-dimensional distribution balance Effect
Inactive Publication Date: 2016-09-28
INST OF HIGH ENERGY PHYSICS CHINESE ACAD OF SCI
View PDF2 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The purpose of the present invention is to provide a fuzzy balance method for distributed storage pools, which is used to solve the problem that the existing distributed storage systems do not realize storage resource balance in the true sense, and the number of files in different storage pools varies greatly, resulting in The unbalanced load of the distributed system causes a single point of failure, which affects the quality of data service and the stability of the system
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0010] An embodiment of the fuzzy balance method of the distributed storage pool of the present invention mainly includes:
[0011] A. Classify all files in each storage pool by size;
[0012] One embodiment of the classification method is to set multiple size intervals, and the coverage areas of the multiple size intervals do not overlap with each other, and all the files in each storage pool are in one-to-one correspondence with the file size space according to the size, that is, they are located in the same file Files within the size space range are of the same file type, while files located within different file size space ranges are of different file types.
[0013] B. Divide storage pools into migration-in pools and migration-out pools, including:
[0014] According to the average available space and the current available space of the storage pool, determine whether a certain storage pool belongs to the moving-in pool or the moving-out pool, specifically:
[0015] Calc...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a distributed type storage pool fuzzy balancing method and system. The method includes the steps of classifying all files in each storage pool according to the sizes, calculating available space in each storage pool and average available space, taking the storage pools with the available space larger than the average available space as immigration pools, taking the storage pools with the available space smaller than the average available space as emigration pools, calculating the file proportion of each class of files in each storage pool, setting an immigration proportion and an emigration proportion, transferring the files in each emigration pool to each immigration pool, enabling an absolute value of the difference between the available space of each storage pool and the average available space to be smaller than or equal to an available space deviation threshold value after file transferring is completed, stopping transferring one class of files out of one emigration pool if the file proportion of the class of files in the emigration pool is smaller than the emigration proportion, and stopping transferring one class of files into one immigration pool if the file portion of the class of files in the immigration pool is larger than the immigration proportion.
Description
technical field [0001] The invention relates to the technical field of distributed storage, in particular to a method and system for balancing a distributed storage pool. Background technique [0002] In a large-scale distributed storage system, the required data storage units (storage pools) are often gradually added according to the growth of specific needs. This usage pattern leads to an imbalance in the available space of the new and old storage pools. At the same time, in most usage scenarios, the size of data written to the storage system varies greatly (ranging from several KB to several GB). [0003] Data Migration is a mechanism commonly used in large data environments to improve data service efficiency. The main purpose of data migration is to dynamically adjust the distribution of data with changes in user access patterns, so that the load of each data storage unit is balanced, and at the same time, each item of data can obtain better input / output performance. ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/50
Inventor 伍文静陈刚程耀东
Owner INST OF HIGH ENERGY PHYSICS CHINESE ACAD OF SCI