Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Big data self-adaption topology processing method based on storm

A processing method and adaptive technology, applied in the field of big data processing, can solve the problem that the data processing method cannot meet the application scenarios of big data stream computing, and achieve the effect of enhancing pertinence and ensuring timeliness

Active Publication Date: 2016-07-13
南京乐图软件技术有限公司
View PDF5 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is that traditional data processing methods cannot meet the application scenarios of large data flow calculations

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data self-adaption topology processing method based on storm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] The storm adopted in the present invention is a real-time, distributed and highly fault-tolerant computing system. Storm can process large batches of data, and it can also make the processing more real-time under the premise of ensuring high reliability; that is, all information will be processed. Storm also has the characteristics of fault tolerance and distributed computing, which allows Storm to be extended to different machines for large-scale data processing.

[0020] The storm computing system platform is mainly divided into three layers, which are user interaction layer, logic processing layer and execution layer. Among them, the user interaction layer is mainly responsible for the task of interacting with the user, including the user's statement input, the call of the system file, and the display of the result. The statement list obtained by analyzing the data statement is passed to the next layer for processing; the logic processing layer , is responsible for ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a big data self-adaption topology processing method based on a storm.The method comprises the steps of user request monitoring, request information analyzing, topology logic generating and data flow query.Operation operator are selected according to different needs of different users by monitoring the customized service request of the users in real time, services are dynamically customized, topology logic of storm recognizable operation is generated, and therefore dynamic customizing and real-time processing of the data processing services are achieved; the corresponding operators are conveniently selected through an operator execution tree, the optimal operation execution topology meeting the customizing requests of the users is formed, the pertinence of data flow query and mining is enhanced, and the timeliness of data is guaranteed as much as possible.

Description

technical field [0001] The invention relates to a big data processing method, in particular to a storm-based big data adaptive topology processing method. Background technique [0002] In many fields such as telecommunications, the Internet, the Internet of Things, and finance, data presents the characteristics of rapid growth, low unit value density, rapid model changes, and value reduction with the passage of processing time. That is to say, the data scale is large and the growth rate is fast, emphasizing the fast processing time, and it is necessary to dynamically select services from the changeable modes to analyze and mine the data in real time. Since data flow has characteristics different from traditional static data (ordered, high-speed, boundless, and rapidly changing), traditional data processing methods cannot meet the application scenarios of large data flow computing. Query and mining processing give us a new research topic and bring new challenges to the resea...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/24568G06F16/2465
Inventor 陆莉莉何淼季一木史海峰谈杰凌宝慧武文扬
Owner 南京乐图软件技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com