A system and method for distributing and processing massive logs through high-speed memory
A distributed processing and high-speed memory technology, applied in the field of communication, can solve the problems of low efficiency and achieve the effects of cost reduction, high implementability, and good promotion and application prospects
Active Publication Date: 2016-08-17
BEIJING CAPITEK
View PDF5 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, for log archiving and backup, especially when providing external log query and analysis, this processing method must poll the message queue, which is inefficient
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0016] In order to make the purpose and technical solution of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings.
[0017] see figure 1 , introducing the system structure of the present invention through high-speed memory and distributed processing of massive logs: multiple log generation devices and a log management server are provided, wherein:
[0018] Multiple log generation devices are distributed in each network element, and are used to normalize the format of various original logs, then cache them in high-speed memory components, and periodically upload the associated index files generated by them to the logs Management server. The log generation device has the following three components:
[0019] Original log generation module: used to generate original logs of various business applications for the telecommunications network;
[0020] Log preprocessing module: responsible for r...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
A system and method for distributing and processing massive logs through high-speed memory, the system is composed of multiple log generation devices and a log management server: the log generation devices are scattered in each network element, and are used to store various original logs After the format is normalized, it is cached in the high-speed memory component and uploaded to the log management server periodically; the log generation device has three components: the original log generation module, the log preprocessing module and the high-speed memory component. The log management server is responsible for uniformly exporting, archiving and storing the logs that exceed the cache period in the high-speed memory components distributed in each log generating device, so that other systems can query and analyze the logs. The invention can process multiple types of logs in real time with high efficiency and low cost, and has good scalability and implementability. Compared with using a relational database to store logs, it reduces processing costs, solves the database bottleneck problem of large amounts of data, and improves the efficiency of querying and analyzing logs.
Description
technical field [0001] The invention relates to a communication log processing technology, more precisely, to a system and method for distributing and processing massive logs through high-speed memory, and belongs to the field of communication technology. Background technique [0002] In the service network of a telecom operator, the communication service log for network managers to understand the service usage status is an indispensable part. Especially with the continuous evolution and development of factors such as network diversification, IP, and business openness, the security threats faced by telecommunications equipment are also increasing. It is possible to record and analyze logs of user business usage in real time, and to discover and locate network The role in the problem is also becoming more and more important. Moreover, with the rapid expansion of network services and the increasing number of users, the types and quantities of telecommunications network equipm...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L12/24H04L29/08
Inventor 陈涛谭立彦李学春张云生王菲
Owner BEIJING CAPITEK