Unlock instant, AI-driven research and patent intelligence for your innovation.

A hdfs-based file storage method, device and distributed file system

A file storage and file technology, applied in instrumentation, computing, electrical and digital data processing, etc., can solve problems such as low new disks and concentrated I/O operations, optimize load, reduce utilization differences, and improve storage and storage efficiency. The effect of quality of service

Active Publication Date: 2017-06-06
BEIJING QIHOO TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Even in the homogeneous state, when the load is high and a node is replaced or a new disk is added, the utilization rate of the new disk will be much lower than that of the old disk, so that a large number of I / O operations are still concentrated on the old disk

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A hdfs-based file storage method, device and distributed file system
  • A hdfs-based file storage method, device and distributed file system
  • A hdfs-based file storage method, device and distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0044] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] In response to the file storage request from the client, HDFS divides the requested file into multiple data blocks, and then stores them in multiple storage devices under the HDFS data node. The data storage of HDFS includes two levels. The first level is to select data nodes, and the second level is to select specific storage D from the selected data nodes. 1 ,D 2 ,...D n Used for data block storage. The present inventio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a file storage method based on an HDFS (Hadoop Distributed File System). The method is used for partitioning a file into a plurality of data blocks and storing into storages of data nodes of the HDFS in response to a file storage request coming from a client. The method comprises the following steps: residual space acquisition: acquiring the residual storage spaces of the storages; data node selection: selecting data nodes for the data blocks; storage selection: selecting a storage for storing each data block in a plurality of storages of the selected data nodes, wherein the storage selection step comprises selecting one storage serving as a target storage of each data block according to a preset rule based on the residual storage spaces of the storages. The invention further provides a file storage control device based on the HDFS and an HDFS distributed file system.

Description

technical field [0001] The invention relates to the field of mass data storage, in particular to a file storage method and device based on a distributed file system. Background technique [0002] With the continuous development of science and technology, the era of massive data has arrived. Large international companies such as Google, Amazon, IBM, and Microsoft have invested a lot of scientific research in this field and proposed a variety of innovative massive data management technologies. These research work mainly focus on the three levels of storage layer, computing layer and interface layer. figure 1 Shows the relevant layers involved in massive data processing techniques. Among them, technologies represented by Google's distributed file system GFS and parallel programming framework MapReduce have become mainstream technologies for massive data storage and analysis. Based on the design ideas of GFS and MapReduce, the Hadoop project of the open source community Apach...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 赵彦荣郭东东赵健博
Owner BEIJING QIHOO TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com