Distributed file system index calculation method and device and electronic equipment

A technology of distributed files and computing methods, applied in the field of databases, can solve the problem of heavy workload of servers, and achieve the effects of alleviating the heavy workload, reducing the amount of migration, and saving evenly

Pending Publication Date: 2022-04-01
TIANJIN TIANDI WEIYE INFORMATION SYST INTEGRATION CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a distributed file system index calculation method, device and ele

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system index calculation method and device and electronic equipment
  • Distributed file system index calculation method and device and electronic equipment
  • Distributed file system index calculation method and device and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] Such as figure 2 As shown, in the embodiment of the present invention, the distributed file system index calculation method includes the following steps:

[0049] S11. Parameter acquisition step: acquire the number of virtual nodes to be allocated, the number of redundant fragments, and the system parameters of the distributed file system.

[0050]This step can be calculated based on experience: distribute 40 hash points per 1 TB of storage space, and estimate the rated number of hash points for the entire volume. For example, if a user is allocated a maximum space of 20TB, the number of hash points of this volume is 800, and the number of virtual nodes (Vnodes) is 800. The number of redundant fragments depends on the erasure protection method: P=K+M, where K is the number of original data blocks, and M is the number of redundant data blocks.

[0051] In this embodiment, the distribution of the virtual nodes of the volume is calculated, and the calculated hash functi...

Embodiment 2

[0076] Such as image 3 As shown, in the embodiment of the present invention, the distributed file system index calculation method includes the following steps:

[0077] S21. Parameter obtaining step: obtaining the number of virtual nodes to be allocated, the number of redundant fragments, and the system parameters of the distributed file system.

[0078] For example, the number of virtual nodes V=800, the number of redundant slices P=10, the system parameters include server parameter server, group parameter group, disk parameter disk, the weight group_weight of each disk group and the weight disk_weight of each disk.

[0079] S22. The loop argument vn.

[0080] The vn in this step represents the serial number of the virtual node. For example, the initial value of vn is 0, and the maximum value is 800.

[0081] S23. The first lottery step: based on the serial number of the current virtual node and system parameters, use the first lottery function to calculate the disk group ...

Embodiment 3

[0114] Such as Figure 7 As shown, the embodiment of the present invention provides a distributed file system index computing device, including:

[0115] The parameter acquisition module 1 is configured to acquire the number of virtual nodes to be allocated, the number of redundant fragments, and the system parameters of the distributed file system.

[0116] The virtual node calculation module 2 is configured to use a lottery algorithm to calculate the disk group where each virtual node is located.

[0117] The redundant slice calculation module 3 is configured to use a lottery algorithm to calculate the disk where each redundant slice is located.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a distributed file system index calculation method and device and electronic equipment, belongs to the technical field of databases, and solves the problem that the workload of a server in an existing distributed file system is large. The method comprises the following steps: a parameter acquisition step: acquiring the number of virtual nodes to be distributed, the number of redundant fragments and system parameters of the distributed file system; a virtual node calculation step: calculating a disk group where each virtual node is located by using a lot drawing algorithm; and a redundant fragment calculation step: calculating the disk where each redundant fragment is located by using a lot drawing algorithm.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to a distributed file system index calculation method, device and electronic equipment. Background technique [0002] In the design of a distributed file system, a file is divided into K original data blocks, and then M redundant data blocks are generated using an erasure code algorithm to form K+M fragmented data with redundancy, and then these Fragmented data is distributed on different disks in the disaster recovery domain to achieve decentralized disaster recovery capabilities. As the capacity of the distributed file system continues to increase, the metadata for file indexes will become more and more, and the cost of managing huge metadata is also gradually increasing, so the development direction of the distributed file system is to use consistency Hashing technology to replace the metadata of the file. Hash technology can use the file name to calculate the hash hash value...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/13G06F16/182
Inventor 于宏志张岩郗胜奎齐东伟王朝昕黄艳
Owner TIANJIN TIANDI WEIYE INFORMATION SYST INTEGRATION CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap