A Storm-based industrial signaling data stream type computing framework

Abstract

The invention provides a Storm-based industrial signaling data stream type computing framework. Due to the difficulties of large scale, miscellaneous types and low quality of current industrial big data and a complex mode during stream data mining, a framework which reasonably utilizes a limited memory when a data mining algorithm is executed so as to obtain more data at one time is designed. In order to mine valuable information from industrial signaling data existing in a stream form, a Storm is utilized to mine a real-time data stream. For different real-time data mining tasks, the same type of data sources may be used, Spout sharing of the data sources of different Topology is achieved through Kafka message middleware, and reliable transmission of messages, message returning and the like are guaranteed. According to the Storm-based industrial signaling data stream type computing framework, the effectiveness and the high efficiency of information extraction in the industrial signaling data mining process can be effectively improved, the current situation that industrial big data are miscellaneous in type and low in quality is changed, and the requirements of intelligent industrial production and development are met.

Description

technical field [0001] The invention relates to the fields of big data storage, data search, data mining, and stream data processing, and in particular relates to a Storm-based industrial signaling data stream computing framework. Background technique [0002] Under the development requirements of the Industry 4.0 era, industrial big data is mainly divided into machine data, external third-party data and business related business data. Based on the characteristics of the industry, the large-scale time-series data collected during the production process presents three major difficulties: large scale, miscellaneous types, and low quality, which will also affect the collection of industrial data. [0003] The characteristics of streaming data mining determine that it is more complicated than traditional data mining, because streaming data is continuously generated, and the size of memory is limited, so it is impossible to put all the collected data in memory to wait for mining,...

AI Technical Summary

Problems solved by technology

Based on the characteristics of the industry, the large-scale time-series data collected during the production process presents three major difficulties: large scale, miscellaneous types, and low quality, which will also affect the collection of industrial data

[0003] The characteristics of streaming data mining determine that it is more complicated than traditional data mining, because streaming data is continuously generated, and the size of memory is limited, so it is impossible to put all the collected data in memory to wait for mining, but only real-time processing

Images