Log analysis method, device and system and readable storage medium
An analysis method and analysis system technology, applied in the fields of devices, log analysis methods, systems and readable storage media, can solve problems such as troublesome deployment and maintenance, high real-time requirements, and small log volume, and achieve lightweight, The effect of fast transmission and high concurrency
Pending Publication Date: 2022-03-11
HANGZHOU ANHENG INFORMATION TECH CO LTD
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
When deploying the system, the requirements for the deployment environment are relatively high, and multi-node deployment is often required. Deployment and maintenance are relatively troublesome. It needs to be realized with the help of more message middleware, etc., and the log volume is not large, the real-time requirements are high, and the log system is not For business scenarios such as core businesses or small and medium-sized enterprises, using the above technologies for log collection and analysis is too heavyweight
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0046] Existing log collection systems are generally implemented using technologies such as ELK (which is a collection of Logstash, Elasticsearch, and Kibana open source software), Flume (a distributed, reliable, and highly available massive log aggregation system), although the functions are complete, However, the technical threshold and deployment environment requirements are relatively high, and the maintenance cost is relatively high. Developers are required to master a variety of technologies during development, which is not out of the box for those who are not majors in big data. When deploying the system, the requirements for the deployment environment are relatively high, and multi-node deployment is often required. Deployment and maintenance are relatively cumbersome, and more message middleware is required for implementation. Although the traditional log system has complete functions, it is often too heavy for business scenarios such as small log volume, high real-ti...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a log analysis method, device and system and a readable storage medium, and the method is applied to a log processing center, and comprises the steps: receiving a log which is detected on a local disk of a log detection end and is transmitted by Netty; writing the log into a local disk of a log processing center, obtaining a newest log from the local disk of the log processing center, and processing the newest log according to a local ClickHouse to obtain a first processed log; and the first processing log is sent to a local ClickHouse, and the first processing log is analyzed by the local ClickHouse. According to the technical scheme disclosed by the invention, the complexity of log analysis is reduced through high transmission and high concurrency performance of Netty, strong storage capability and analysis capability of ClickHouse and log storage on a local disk, the real-time performance of log analysis is improved, and log analysis is realized in a lightweight manner.
Description
technical field [0001] The present application relates to the technical field of log analysis, and more specifically, to a log analysis method, device, system and readable storage medium. Background technique [0002] Logs are a very broad concept in computer systems, and any program may output logs. Logs record various things that happen on the device and the network every day. Therefore, the status of each device and the entire network can be understood by analyzing the logs. [0003] At present, the log collection system is generally implemented using technologies such as ELK and Flume. Although the functions are complete, the technical threshold and deployment environment requirements are relatively high, and the maintenance cost is relatively high. Developers are required to master a variety of technologies during development, which is not out of the box for those who are not majors in big data. When deploying the system, the requirements for the deployment environmen...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/34G06F16/178G06F16/18G06F16/174
CPCG06F11/3476G06F16/1794G06F16/1815G06F16/1744
Inventor 倪山登范渊刘博
Owner HANGZHOU ANHENG INFORMATION TECH CO LTD