Railway power distribution network mass information stream processing method based on Storm and Kafka message communication

A technology of message communication and processing method, which is applied in the field of massive information flow processing of railway distribution network, can solve problems such as not being able to meet real-time processing requirements, and achieve the effect of improving overall processing efficiency and high concurrent processing capability

Inactive Publication Date: 2018-02-16
EAST CHINA JIAOTONG UNIVERSITY
View PDF5 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] With the rapid development of intelligent railway dispatching and monitoring in my country, the rapid increase in the amount of monitoring data has made the traditional monitoring information processing methods unable to meet the real-time processing requirements of thi...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Railway power distribution network mass information stream processing method based on Storm and Kafka message communication
  • Railway power distribution network mass information stream processing method based on Storm and Kafka message communication

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The technical solution of this patent will be described in further detail below in conjunction with specific embodiments.

[0024] See Figure 1-2 , The method for processing massive information flow of railway distribution network based on Storm and Kafka message communication, the specific steps are as follows:

[0025] (1) The collection and transmission stage of a large amount of real-time monitoring data of the distribution network

[0026] In the Linux system, the Kafka message middleware uses asynchronous transmission to obtain real-time monitoring data of the railway distribution network monitoring system, and the collected real-time monitoring data of different topics (such as: analog quantity, status quantity, etc.) are first cached in Kafka messages In the server, several Storm nozzle components actively pull information on different topics from the message server in a subscription mode, and the Storm cluster performs streaming computing processing of the monitorin...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a railway power distribution network mass information stream processing method based on Storm and Kafka message communication. The method comprises the steps that Kafka message-oriented middleware is set up in a computer with a Linux operating system, integration with an existing power distribution monitoring system is realized through the Kafka message-oriented middleware,and a large amount of real-time monitoring data collected by the monitoring system is transmitted to a Storm cluster by use of a published/subscribed message transmission mode of the monitoring system. Through the processing method, a large amount of the monitoring data can be directly saved into an HBase database for distributed storage after being processed quickly and parallelly by nodes in the Storm cluster, and mass monitoring data of a railway power distribution network is processed through the distributed computer cluster. According to the method, a cloud computing technology combiningKafka and Storm is adopted to process the data, millisecond-level processing delay, distributed storage and query of the mass real-time monitoring data of the railway power distribution network are realized, and the overall processing efficiency of scheduling monitoring big data is improved.

Description

Technical field [0001] The invention relates to the technical field of power distribution network monitoring, in particular to a method for processing massive information flow of a railway power distribution network based on Storm and Kafka message communication. Background technique [0002] With the rapid development and scale expansion of modern electrified railways in my country, the dispatch and monitoring of the railway distribution network is now mostly based on the panoramic real-time data processing of the grid operation. As the distribution system is connected to a variety of new intelligent equipment, it also integrates A kind of information management system, such as power distribution energy management system, railway power supply dispatching and monitoring data center, etc. Higher sampling frequency requires dispatching and monitoring to quickly process the huge amount of real-time monitoring data. In general, consider resident real-time monitoring data in a memory d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30G06F9/54G06Q50/06
CPCG06F9/546G06F16/221G06F16/245G06F16/24568G06F16/27G06F16/28G06F2209/547G06Q50/06
Inventor 屈志坚王冬王群峰彭翔
Owner EAST CHINA JIAOTONG UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap