Method and system for fast detection and recovery of node failure based on hdfs

A recovery method and system recovery technology, which is applied in the transmission system, digital transmission system, data exchange network, etc., can solve problems that cannot meet the needs of system security, and achieve the effect of not increasing network bandwidth and reducing system risk

Active Publication Date: 2020-01-03
EISOO SOFTWARE
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing HDFS detection and recovery mechanism still cannot meet the needs of system security

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for fast detection and recovery of node failure based on hdfs
  • Method and system for fast detection and recovery of node failure based on hdfs
  • Method and system for fast detection and recovery of node failure based on hdfs

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

[0035] It should be noted that the diagrams provided in the following embodiments are only schematically illustrating the basic ideas of the present invention, and only the components related to the present invention are shown in the diagrams rather than the number, shape and shape of the compo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a fast detection and recovery method and system for node failure based on HDFS. The method comprises the following steps: setting a dangerous node time interval, if heartbeat information of a data node is not detected within the dangerous node time interval, determining the data node as a dangerous node, and adding the dangerous node in a dangerous data node queue; finding identical data blocks in data nodes in the dangerous data node queue; and setting a recovery priority according to the occurrence times of the identical data blocks, adding the identical data blocks in a to-be-recovered queue according to the recovery priority to preferentially recover the identical data blocks. According to the fast detection and recovery method and system for the node failure based on the HDFS provided by the invention, the dangerous data blocks are recovered in advance on the premise of guaranteeing the availability of the data blocks of the data nodes and basically increasing no network bandwidth, so that the system risks are reduced.

Description

technical field [0001] The invention relates to the technical field of storage clusters, in particular to an HDFS-based fast detection and recovery method and system for node failure. Background technique [0002] With the continuous improvement of the informatization level of the whole society, social life is gradually inseparable from computer systems. At present, computer systems have fully entered the era of "cloud computing" and "big data", and it has become an inevitable trend for large-scale server clusters to respond to massive requests and process massive data. Interconnect independent and dispersed computers through the network to make them a cluster, and manage them in a single system mode macroscopically. To the user, the cluster looks like a stand-alone server. [0003] However, cluster systems with high-performance computing capabilities are far from meeting the needs of such large-scale applications, and must also be able to provide durable, efficient and st...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L12/24H04L12/26H04L29/08
CPCH04L41/0654H04L41/0668H04L43/10H04L67/1097
Inventor 席刚
Owner EISOO SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap