Spark cluster outage self-protection and incoming telegram self-starting method

A self-starting and self-protecting technology, applied in the field of big data, can solve problems such as damaged data and clusters, damaged hardware devices, manual startup, etc., to achieve the effect of complete functions, high customization, and ensuring safety and stability

Active Publication Date: 2017-08-04
NORTHWEST UNIV(CN)
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

On the one hand, if there is a sudden power failure, even if there is an emergency power supply, if the cluster cannot respond in time, after the emergency power supply is exhausted, it will still cause inconsistencies in the cluster, which will damage the data and the cluster itself, and even directly damage the hardware equipment; On the one hand, after the mains power is restored, the operation and maintenance personnel need to manually start the spark cluster, which increases the labor cost, and it is difficult to guarantee the timeliness

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Spark cluster outage self-protection and incoming telegram self-starting method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0025] This embodiment provides a self-protection method for spark cluster power failure and self-starting method, which mainly involves selectively enabling the emergency power function for some devices in the network topology, while other devices are directly powered by the mains, and Cooperate with the corresponding software to achieve the goal, including the following steps:

[0026] Step 1, prepare the required components, including: multiple hosts, switches, routers, UPS;

[0027] Step 2, such as figure 1 As shown, connect all the hosts in the prepared components to the switch, and connect the LAN port of the router to the switch;

[0028] The host and switch are powered by UPS, while the router is powered by commercial power; and the present invention has no restrictions on UPS and no networking requirements, and only needs to ensure that all designated devices have emergency power.

[0029] All the hosts, switches and routers constitute a local area network;

[0030...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a spark cluster outage self-protection and incoming telegram self-starting method. The method comprises the steps of connecting all host computers with exchangers separately, wherein LAN of routers is connected with the exchangers; the host computers and the exchangers are all supplied with power through a UPS, and the routes are supplied with power through mains supply; all the host computers, the exchangers and the routers form a local area network; multiple host computers form a cluster through the exchangers; any of the host computers is set to be a main node, and other host computers are set to be subsidiary nodes; the main code detects whether or not the routers are online every fixed time period, the mains supply is regarded as being switched off when the several subsidiary codes detect that the routers are offline, then a spark cluster is closed, a hadoop cluster is then closed, and finally all the host computers are closed, so that cluster software and hardware are protected from being damaged; if the mains supply is resumed, the routers are automatically started, the main node is awakened through the local area network, and subsequently the main node automatically opens the cluster, so that the purpose of cluster incoming telegram self-starting is reached.

Description

technical field [0001] The invention belongs to the technical field of big data, and in particular relates to a method for self-protection after power failure of a spark cluster and self-starting when power is received. Background technique [0002] Spark is a big data computing platform that includes many sub-projects. It has become the top open source project of the Apache Foundation, has a huge community support, and its technology is gradually becoming mature. And in terms of performance, Spark stands out among many big data analysis and processing platforms due to its advantages of fast speed, ease of use, versatility, integration, and lightweight fast processing. However, there is still a lot of room for optimization in the operation and maintenance of the spark cluster. On the one hand, if there is a sudden power failure, even if there is an emergency power supply, if the cluster cannot respond in time, after the emergency power supply is exhausted, it will still cau...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F1/30H04L12/26H04W84/12
CPCH04L43/0811H04L43/0876H04L43/10H04W84/12G06F1/30
Inventor 胡景钊冯筠马凯歌
Owner NORTHWEST UNIV(CN)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap