Kinetic energy data processing system and method based on Spark computing framework

A data processing system and computing framework technology, applied in database management systems, manufacturing computing systems, data processing applications, etc., can solve the problems of reduced accuracy of result data, low real-time data retrieval, and inability to collect data, etc., to achieve reduction Data access I/O, optimization of data processing algorithms, and overall performance improvement

Pending Publication Date: 2020-05-19
CHINA TOBACCO ZHEJIANG IND
View PDF14 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] (1) Due to the huge amount of data and the need to connect multi-directional system data, it is not possible to collect these data well;
[0004] (2) A large amount of data will also introduce noise data that does not meet the quality requirements due to diversity, complexity, ...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Kinetic energy data processing system and method based on Spark computing framework
  • Kinetic energy data processing system and method based on Spark computing framework
  • Kinetic energy data processing system and method based on Spark computing framework

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] In order for the relevant technicians to understand the technical solutions of the present invention more clearly, the present invention will be further described in detail with reference to the accompanying drawings.

[0029] Referring to Figure 1, this embodiment discloses a kinetic energy data processing system based on the Spark computing framework, which includes three modules: a data acquisition module, a data processing analysis module, and a data storage module. The data acquisition module is configured to access multiple data The interface of the collection terminal realizes the collection and integration of data in each link of the power system, and opens up data interaction with existing information systems, including MES systems, energy management systems, centralized control systems, and OPC-based data acquisition services; the data processing analysis module Including data access module, data cache module and real-time calculation module, real-time calculation...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a kinetic energy data processing system based on a Spark computing framework. The kinetic energy data processing system comprises a data acquisition module used for monitoringkinetic energy data of an industrial workshop, a data processing and analyzing module used for carrying out data computing on data streams issued to a Kafka message queue, and a distributed data storage module. Considering the variety of the data acquisition terminals, the Kafka cluster is provided with functional interfaces capable of being accessed to various data acquisition terminals, so thata flexible and extensible data acquisition system is established, and the problem of access of a plurality of data terminals is avoided. And secondly, as the amount of detection data needing to be processed is large, the stream data from the Kafka message queue is processed by adopting stream calculation, and the calculation performance can be greatly improved. And thirdly, the real-time performance of processing and analyzing the data according to the decision requirement of the industrial workshop on equipment operation is high, so that a distributed storage mode is adopted for the data.

Description

Technical field [0001] The invention relates to the technical field of kinetic energy data processing for various equipment in a workshop, and in particular to a kinetic energy data processing system and method based on a Spark computing framework. Background technique [0002] The equipment in the power workshop of a cigarette factory will generate massive amounts of kinetic energy data. In order to reduce production costs and improve production quality, it is necessary to collect, process and analyze these kinetic energy data. In the process of rapid growth of big data technology, there will be the following problems: [0003] (1) Due to the huge amount of data and the need to connect multi-directional system data, these data cannot be collected well; [0004] (2) A large amount of data will also introduce noise data that does not meet the quality requirements due to diversity, complexity, and various uncertain factors such as channel quality during the transmission process, such ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/27G06F16/25G06F16/215G06F9/54G06Q50/04
CPCG06F16/27G06F16/215G06F16/252G06F9/546G06Q50/04G06F2209/548Y02P90/30
Inventor 叶志晖王文娟张成挺钱杰
Owner CHINA TOBACCO ZHEJIANG IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap