A massive data processing method and system

A mass data processing and mass data technology, applied in the field of automation, can solve the problems that personnel cannot guarantee the quality of analysis, analysis experience cannot be effectively accumulated and inherited, and the efficiency of large repetitions is low.

Active Publication Date: 2020-10-27
RUN TECH CO LTD BEIJING
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The existing fully manual protocol analysis work method has problems such as large volume and low repetition efficiency, frequent turnover of personnel cannot guarantee the quality of analysis, and analysis experience cannot be effectively accumulated and passed on.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A massive data processing method and system
  • A massive data processing method and system
  • A massive data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] Please refer to the attached figure 1 , is a schematic flowchart of a massive data processing method provided in Embodiment 1 of the present invention, the method is applicable to the scenario of massive network data processing, the method is executed by a massive data processing system, and the system can be implemented by software and / or hardware , integrated inside the data analysis tool. The method specifically includes the following steps:

[0045] S110. Obtain keywords input by the user.

[0046] S120. Obtain corresponding first data after encoding the keyword according to a preset encoding manner.

[0047] Specifically, the keywords provided by the user are converted and stored in various common encoding types, such as utf8, unicode, url, base64, md5, etc. The data encoding process can be, for example, as figure 2 As shown, it is a schematic diagram of the data encoding structure, where encode1, encode2, etc. are encoded strings, code is the string before en...

Embodiment 2

[0073] Please refer to the attached Figure 5 , which is a schematic structural diagram of a massive data processing system provided in Embodiment 2 of the present invention, the system specifically includes the following modules:

[0074] A keyword acquisition module 210, configured to acquire keywords input by the user;

[0075] A keyword encoding module 220, configured to encode the keyword according to a preset encoding method to obtain corresponding first data;

[0076] Data acquisition module 230, for acquiring mass data;

[0077] A data decoding module 240, configured to decode the massive data according to a corresponding decoding method to obtain corresponding second data;

[0078] The result output module 250 is configured to match the first data and the second data according to the encoding method and the corresponding decoding method, and output a matching result.

[0079] Preferably, the device also includes:

[0080] A result association module, configured to...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a massive data treatment method and a massive data treatment system; the method includes steps of acquiring key words input by a user; according to the preset coding method, coding the key words and then acquiring corresponding first data; acquiring massive data; decoding the massive data according to corresponding decoding method and then acquiring corresponding second data; according to the coding method and the corresponding decoding method, matching first data and the second data, and outputting the matching result. Based on the method and the system, various treated valuable information can be extracted from massive data efficiently through realizing the automatic data treatment work, a lot of repeat indexing analysis works can be completed, the analysis quality is guaranteed and the analysis experience can be absorbed; besides, the method and the system can continuously improve the analysis accuracy, and make up the blind zone and difference of the knowledge analysis of people with different levels.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of automation, and in particular, to a massive data processing method and system. Background technique [0002] There are a large number of historically accumulated network data packets in our daily work environment, and there are a large number of real-time network data streams in the actual network environment, and the information we want will appear in these data at any time. How to efficiently extract this information and the context of this information has become an urgent need for many people, and this information may be stored in the original message in various forms, such as encoded in unicode+base64, and then compressed by zlib, such as encrypted by md5. To detect such information and its context from massive data, protocol analysis is required. The protocol data analysis work is the basis of the protocol analysis. Whether the protocol analysis is comprehensive and in-dept...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/9535
CPCG06F16/1748
Inventor 朱立业
Owner RUN TECH CO LTD BEIJING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap