Check patentability & draft patents in minutes with Patsnap Eureka AI!

A storm-based stream data regular matching method

A matching method and stream data technology, applied in the computer field, can solve the problem that stream data processing only supports simple regular matching, etc., and achieve the effect of improving regular matching efficiency, improving efficiency and reducing the amount of transmitted data.

Active Publication Date: 2020-10-30
BEIJING SCISTOR TECH
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In order to improve the efficiency of stream data regular matching and solve the problem that only simple regular matching is supported in stream data processing, the present invention provides a Storm-based stream data regular matching method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A storm-based stream data regular matching method
  • A storm-based stream data regular matching method
  • A storm-based stream data regular matching method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0017] The present invention will be further described in detail with reference to the accompanying drawings and embodiments.

[0018] The present invention combines the Storm stream processing technology with the regular matching technology, and according to the effective combination of the Tuple processing structure in Storm and the regular matching data packet load, the processing speed of the entire stream processing cluster can be improved, and the regular expression matching efficiency can be improved.

[0019] The present invention adopts Kafka message queue in the data cache module to store cache data. The main function of Storm is to perform stream-based real-time computing. It processes the data streams that are always generated very quickly. However, most of the data is not a uniform data stream, but sometimes more and sometimes less. Batch processing in this case is not suitable, so Kafka is introduced as a message queue, which works perfectly with Storm, so that s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a streaming data regular matching method based on Storm, and belongs to the technical field of computers. The method comprises the following steps: establishing a real-time processing cluster, using a Kafka cluster as a data caching module, serializing original data, packaging the serialized data into a Message, and loading the Message into a Kafka message queue; subscribingdata of a certain Topic in the Kafka, filling the obtained Message data into a unit Tuple of a Storm in sequence, and directly sending the Tuple to a calculation operator Bolt; Aand carrying out unpacking on the Tuple by the Bolt, unpacking the obtained Message data, carrying out deserialization on the Message data, and carrying out pattern matching on the effective data blocks subjected to deserialization. According to the method and the device, the batch processing of data transmission in the Storm cluster is ensured, the transmission efficiency of the data in the Storm real-time computingcluster is improved, and the regular matching efficiency is improved.

Description

technical field [0001] The invention belongs to the technical field of computers, relates to Internet data processing, and in particular to a regular matching technology based on Storm for real-time processing of stream data. Background technique [0002] With the rapid development of the Internet, network information is increasing exponentially, so that the amount of data to be detected and the rule data of regular expressions increase dramatically. At the same time, in the processing business of a large amount of network data, data such as messages often need to be processed in real time, which poses a huge challenge to the real-time matching performance of the regular expression matching technology. At present, the research on regular expression matching technology mainly focuses on the matching efficiency and space storage when converting it into an automaton for matching. However, regular expressions can only support simple fuzzy matching and screening in real-time proc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/958G06F16/957
Inventor 王振宇孟宪文李斌斌
Owner BEIJING SCISTOR TECH
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com