Check patentability & draft patents in minutes with Patsnap Eureka AI!

How to improve the availability of web_hdfs under Hadoop dual-machine cluster

An availability and clustering technology, applied in the computer field, can solve the problems of increasing application end node switching test network overhead, increasing program resource overhead, increasing file operation failures, etc., so as to reduce the risk of file operation failures, reduce network resource overhead and program The effect of resource overhead

Active Publication Date: 2021-05-18
福建星瑞格软件有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In view of this kind of operation failure, the traditional method is to visit and test each node one by one through Web_HDFS; but this method has the following problems: 1. To visit and test each node one by one, adding the application node switching test 2. Increase the risk of file operation failure due to network disconnection when the application side switches nodes; 3. Increase the program resource overhead caused by repeated testing when the application side uses Web_HDFS

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • How to improve the availability of web_hdfs under Hadoop dual-machine cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] Please refer to figure 1 As shown, the present invention promotes the preferred embodiment of the method for Web_HDFS usability under Hadoop two-machine cluster, comprises the following steps:

[0025] Step S10, create the zookeeper session of Hadoop; Hadoop is a software framework that can carry out distributed processing to a large amount of data;

[0026] Step S20, start the cache function of the active node, obtain the active node and the standby node from the zookeeper session and cache them; start the update function of the active node, call the file management service to visit the active nodes one by one, verify the operability of the active node, And update the operability information to the application side;

[0027] Step S30, the application obtains the active node from the cache, calls Web_HDFS to perform file operations, and if the file operation fails, obtains operable active nodes according to the operability information to perform file operations.

[00...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a method for improving the usability of Web_HDFS under the Hadoop dual-machine cluster, comprising the following steps: step S10, creating a zookeeper session of Hadoop; step S20, starting the caching function of the active node, and obtaining the active node and the standby node from the zookeeper session And cache; start the update function of the active node, call the file management service to visit the active node one by one, verify the operability of the active node, and update the operability information to the application end; step S30, the application end obtains the activity from the cache The node calls Web_HDFS to perform file operations. If the file operation fails, obtain operable active nodes based on the operability information to perform file operations. The invention has the advantages of reducing network overhead and program resource consumption caused by frequently invoking nodes to test availability; improving the availability of Hadoop dual-machine clusters and reducing the failure rate of file operations.

Description

technical field [0001] The invention relates to the field of computers, in particular to a method for improving the availability of Web_HDFS under Hadoop dual-machine clusters. Background technique [0002] When the application-side program uses Web_HDFS to operate files in a Hadoop dual-machine cluster, it needs to obtain the active node for file operations. When the obtained node is a standby node, some file operations may fail. In view of the failure of this kind of operation, the traditional method is to visit and test each node one by one through Web_HDFS; but this method has the following problems: 1. To visit and test each node one by one, adding the switch test of the application end node 2. Increase the risk of file operation failure due to network disconnection when the application side switches nodes; 3. Increase the program resource overhead caused by repeated testing when the application side uses Web_HDFS. Contents of the invention [0003] The technical pro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L12/24H04L12/26H04L29/08
CPCH04L41/0663H04L43/0817H04L67/1097
Inventor 林其锦
Owner 福建星瑞格软件有限公司
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com