Unlock instant, AI-driven research and patent intelligence for your innovation.

A disk fault detection method and system based on hdfs

A fault detection and disk technology, which is applied in the detection of faulty computer hardware, input/output to record carriers, etc., can solve the problems of manual disk detection, low efficiency, and low real-time performance, and achieve cost-saving effects

Active Publication Date: 2019-05-14
FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The technical problem to be solved by the present invention is to provide a disk failure detection method based on HDFS, which solves the problems that the existing disk damage needs manual detection, low real-time performance and low efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A disk fault detection method and system based on hdfs
  • A disk fault detection method and system based on hdfs
  • A disk fault detection method and system based on hdfs

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] In order to describe the technical content, structural features, achieved goals and effects of the present invention in detail, the following will be described in detail in conjunction with the embodiments and accompanying drawings.

[0031] Hadoop Distributed File System (HDFS) is designed as a distributed file system suitable for running on commodity hardware. HDFS is part of the Apache HadoopCore project, which is a highly fault-tolerant system suitable for deployment on inexpensive machines. HDFS can provide high-throughput data access and is very suitable for applications on large-scale data sets.

[0032] In view of the fact that the existing hadoop fails to visually display the damaged disk, users cannot timely understand and optimize the system resource status, design regular access based on the original hadoop accessible URL, and read the HDFS configuration disk information for calculation, directly destroy the damaged disk Disk information, displayed in the m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an HDFS (Hadoop Distributed File System) based disk fault detection method and system. The method comprises the following steps of acquiring port information of all DataNode components in a hadhoop cluster, and storing the port information; the DataNode components being configured with corresponding disks and obtaining disk information corresponding to the DataNode components according to the port information; and regularly sending a request to each DataNode component, the DataNode component receiving the request and returning response information to a request end, the request end querying the port information of the DataNode component, corresponding to the returned response information, and obtaining the port information of the DataNode component not returning the response information, wherein the disk of the DataNode component not returning the response information is in a damaged state. By monitoring multi-task parallel triggering of a system, accessing to all DataNode components in the hadhoop cluster and summarizing results, a manager is not required to pay attention to a realization process and only needs to view a final detection result of a monitoring end.

Description

technical field [0001] The invention relates to the field of computer storage, in particular to an HDFS-based disk failure detection method and system. Background technique [0002] With the vigorous rise of interactive websites for ordinary Internet users such as social networks, Weibo, and location services, such as Google, Facebook, Twitter, and domestic Renren, Weibo, etc., providing Internet-based services to hundreds of millions of users Interactive services with wireless networks, Internet users all over the world conduct various interactions every day, and produce various data at any time. The amount of these data is several times the amount of data in the stand-alone era. [0003] In order to store these data, various Internet companies have established huge data centers all over the world, and the number of hosts in a single data center ranges from hundreds to tens of thousands. In the data center, the data storage needs to be stored on the disk eventually, so the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/22G06F3/06
Inventor 丁小亮白默王雷
Owner FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD