Real-time intelligent processing framework based on storm streaming data

A streaming data and framework technology, applied in the direction of electrical digital data processing, special data processing applications, database management systems, etc., to achieve the effects of improving intelligence and executable efficiency, reducing development cycle and cost, and simplifying complex problems

Active Publication Date: 2016-04-20
HANGZHOU YUANCHENG TECH CO LTD
View PDF5 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] At the same time, for the decision makers at the upper level, when performing data query work, complex code operations create a certain gap between them and the data, making it difficult to perform good operations and analysis on the data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time intelligent processing framework based on storm streaming data
  • Real-time intelligent processing framework based on storm streaming data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] In order to further understand the present invention, the preferred embodiments of the present invention are described below in conjunction with examples, but it should be understood that these descriptions are only to further illustrate the features and advantages of the present invention, rather than limiting the claims of the present invention.

[0040] The present invention provides a real-time intelligent processing framework for streaming data based on Storm, which relies on the data setting and function selection of the front-end user, and writes it into a formatted configuration file to generate the complex topology of the back-end Storm and mass flow data processing, as attached figure 1 As shown, it specifically includes the following steps:

[0041] First, the front-end sets the data source and data format, and at the same time sets the SQL-like filtering method as a specific functional operator, and uses SQL to specify the expression and write it into the co...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a real-time intelligent processing framework based on storm streaming data. The real-time intelligent processing framework comprises the following steps of setting a front end data structure and a filtering condition via a demand configuration layer, analyzing data of configuration files including SQL instructions via a data structure analyzing layer, analyzing function operators of topology of the storm via an operator function topology layer, generating codes, compling and packaging via a system finishing layer, handing in and executing the finial topology, storing result data in a data storage via a data image layer, and real-time generating data images and displaying the data images to a decision maker. The beneficial effects of the real-time intelligent processing framework based on storm streaming data are that interaction with upper-level decision makers can be well achieved; related topology can be dynamically and intelligently generated according to the complicate data processing function, so code re-usability can be improved and developing period and cost can be greatly reduced; system intelligent property and executable efficiency can be improved; a user image can be displayed in real time; and quick and strong support is provided for decision-making by decision makers.

Description

technical field [0001] The invention relates to the field of intelligent processing of streaming data in big data, in particular to a Storm-based big data processing platform for intelligently completing streaming data processing. Background technique [0002] In today's era of rapid development of the knowledge economy, data acquisition and analysis has increasingly become a magic weapon for understanding society and people's trends. With the use of a large number of PC and mobile client applications, it is possible to obtain a large amount of data. Therefore, people's focus is also increasingly focused on how to quickly query and analyze data. In the ever-expanding mass data, streaming data occupies a considerable proportion. The processing of streaming data has also received more and more attention. Streaming data has the characteristics of large data volume, fast speed, and continuous arrival, which makes it difficult for many traditional database technologies and bat...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/24568G06F16/2471G06F16/25
Inventor 金苍宏刘泽民吴明晖应晶
Owner HANGZHOU YUANCHENG TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap