Method and system for carrying out multi-dimensional statistic analysis on large number of DNS journals

A multi-dimensional statistical analysis and log technology, applied in the field of data statistics, can solve the problems of difficulty in multi-dimensional statistical analysis of big data, decreased data processing efficiency, and low data storage efficiency, so as to improve data processing efficiency, improve stability, and reduce work. load effect

Active Publication Date: 2016-06-01
SURFILTER NETWORK TECH
View PDF6 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The disadvantage of this solution is that: with the large increase in the size of the data in the database, the efficiency of data storage will gradually decrease, and the efficiency of data processing will drop sharply. It will also become more and more difficult for multi-dimensional statistical analysis of big data.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for carrying out multi-dimensional statistic analysis on large number of DNS journals
  • Method and system for carrying out multi-dimensional statistic analysis on large number of DNS journals
  • Method and system for carrying out multi-dimensional statistic analysis on large number of DNS journals

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] In order to solve the defect in the prior art that the data processing and data statistical efficiency of the data statistical scheme based on the traditional database or file index method gradually declines with the increase of the size of the stored data, the main innovation points of the present invention are:

[0050] 1. Split the heavy and massive DNS log data processing task into the first task queue (subjob1,...subjobM) composed of M subtasks, and divide each subtask under the first task queue (subjob1,...subjobM) It is further divided into a second task queue (task1, task2...taskN) composed of N execution tasks (ie, DNS log deduplication tasks), and the split M*N execution tasks are reasonably allocated to M*N sub-servers 3022 under the system of the present invention, when each sub-server 3022 under the system of the present invention receives an execution task, it downloads separately from the cloud server 200 the DNS log data required for processing its execut...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and system for carrying out multi-dimensional statistic analysis on a large number of DNS journals. The system comprises a plurality of DNS data collection modules connected successively, a DNS data summarization center, a cloud server, a DNS journal processing center, a DNS journal statistic analysis center and a database. The DNS data summarization center is used for receiving and summarizing DNS journal data collected by the plurality of DNS data collection modules, and uploading the summarized lots of DNS journal data termly to the cloud server. The DNS journal processing center is used for receiving a DNS journal data processing task submitted by the cloud server, downloading the large amount of DNS journal data from the cloud server, and performing the distributed processing on the large amount of DNS journal data. The DNS journal statistic analysis center is used for performing statistics and analysis on the processed DNS journal data based on given multiple dimensions, generating a multi-dimensional DNS journal data statistics and analysis result, and storing the obtained DNS journal data statistics and analysis result into the database.

Description

technical field [0001] The present invention relates to the technical field of data statistics, and more specifically, relates to a method and system for performing multidimensional statistical analysis on massive DNS logs. Background technique [0002] In the field of data statistics, data statistics schemes based on database or file indexes are widely used. When the amount of data in this solution is small, the usual practice is to store the data first, then update the data, and finally perform statistics and analysis on the updated data based on different dimensions. The disadvantage of this solution is that with the large increase in the size of the data stored in the database, the efficiency of data storage will gradually decrease, and the efficiency of data processing will drop sharply. It will also become more and more difficult for multi-dimensional statistical analysis of big data. Contents of the invention [0003] The technical problem to be solved by the prese...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/26H04L12/24H04L29/08
Inventor 胡文鹏刘臣唐新民景晓军沈智杰
Owner SURFILTER NETWORK TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap