System and method for high-speed memory and distributed type processing of massive logs

A distributed processing, high-speed memory technology, applied in the field of communications, can solve problems such as low efficiency, and achieve the effects of reducing costs, improving processing speed, and high scalability

Active Publication Date: 2014-01-22
BEIJING CAPITEK
View PDF5 Cites 56 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, for log archiving and backup, especially when providing external log quer...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • System and method for high-speed memory and distributed type processing of massive logs
  • System and method for high-speed memory and distributed type processing of massive logs

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] In order to make the purpose and technical solution of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0017] see figure 1 , introducing the system structure of the present invention through high-speed memory and distributed processing of massive logs: multiple log generation devices and a log management server are provided, wherein:

[0018] Multiple log generation devices are distributed in each network element, and are used to normalize the format of various original logs, then cache them in high-speed memory components, and periodically upload the associated index files generated by them to the logs Management server. The log generation device has the following three components:

[0019] Original log generation module: used to generate original logs of various business applications for the telecommunications network;

[0020] Log preprocessing module: responsible for r...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a system and a method for high-speed memory and distributed type processing of massive logs. The system comprises a plurality of log generating devices and a log management server, wherein the log generating devices are dispersed in all network elements and are used for carrying out format normalization on various original logs, caching the original logs into high-speed memory components and uploading the original logs to the log management server periodically; each log generation device is respectively provided with three parts, i.e., an original log generation module, a log pre-processing module and a high-speed memory component. The log management server is responsible for unified derivation and filing storage of the logs beyond the caching period in the high-speed memory components distributed in all the log generating devices so as to be used for inquiring and analyzing the logs by other systems. The system and the method disclosed by the invention have the beneficial effects that a plurality of types of logs can be processed in real time with high-efficiency and low cost, and both the expandability and the actionability are better. Compared with the mode of adopting a relational database to store the logs, the system and the method disclosed by the invention have the advantages that the processing cost is reduced, the bottleneck problem of the database with large data volume is also solved, and the efficiency of querying and analyzing the logs is improved.

Description

technical field [0001] The invention relates to a communication log processing technology, more precisely, to a system and method for distributing and processing massive logs through high-speed memory, and belongs to the field of communication technology. Background technique [0002] In the service network of a telecom operator, the communication service log for network managers to understand the service usage status is an indispensable part. Especially with the continuous evolution and development of factors such as network diversification, IP, and business openness, the security threats faced by telecommunications equipment are also increasing. It is possible to record and analyze logs of user business usage in real time, and to discover and locate network The role in the problem is also becoming more and more important. Moreover, with the rapid expansion of network services and the increasing number of users, the types and quantities of telecommunications network equipm...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/24H04L29/08
Inventor 陈涛谭立彦李学春张云生王菲
Owner BEIJING CAPITEK
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap