Method and system for showing and querying logs in real time

A log and server technology, applied in the field of big data processing, can solve the problems of real-time log display and filter query shortcomings, can not meet the real-time log timeliness well, achieve rich operation options, large query flexibility, and improve user experience Effect

Inactive Publication Date: 2017-01-04
杭州玳数科技有限公司
View PDF4 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Traditional ELK uses Logstash to complete log collection and write it into Elasticsearch, and then obtains data from Elasticsearch and displays it through Kibana, while Kibana focu

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for showing and querying logs in real time
  • Method and system for showing and querying logs in real time
  • Method and system for showing and querying logs in real time

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiment is only one embodiment of the present invention, not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without making creative efforts all belong to the protection scope of the present invention.

[0031] The invention discloses a method for real-time display and query of logs, see figure 1 As shown, the method includes the following steps:

[0032] Step S1: Collect specific log information through the LOG collection server Logstash;

[0033] Step S2: The Tornado server reads the log message records from the message system Kafka through the multi-thread concurrent reading mode, and provides filtering services;

[0034] Step S3: The Tornado server uses high-performance Web...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a method and a system for showing and querying logs in real time. The method comprises steps as follows: log messages are collected by a LOG collecting server Logstash and sent to a message system server Kafka in a message mode; a Tornado server reads log message records from the message system Kafka in a multi-thread concurrent reading mode and provides filter service; the Tornado server adopts a high-performance Websocket long link to send log messages to a foreground terminal Livetail quickly; the foreground terminal Livetail displays the log messages in real time in a specific manner and provides filtering and searching functions. The collected log messages are directly written into the storage management system Kafka, then read by the Tornado concurrently and sent to the front-end server Livetail with data filtering and querying functions by using efficient WebSocket, so that information time delay is reduced, the timeliness of the log messages is greatly improved, a filtering mechanism is provided at the server end, the messages are effectively pushed, the safety of the log messages is guaranteed, defects of existing systems are greatly overcome, and the user experience is improved.

Description

technical field [0001] The present invention relates to a method and system for real-time display and query of logs, in particular to a method and system for real-time display and query of logs in the field of big data processing. Background technique [0002] In the context of the big data era, how to quickly classify and efficiently and accurately search real-time log data has gradually become an important factor affecting the productivity of log management. If you can manage massive real-time log data and at the same time refine and classify the data according to customer needs and then push them accurately, you can not only reduce user waiting time, improve production efficiency, but also effectively ensure the security of user data. [0003] Real-time log ELK includes log collection and real-time display and query of logs. Traditional ELK uses Logstash to complete log collection and write it into Elasticsearch, and then obtains data from Elasticsearch and displays it t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/156G06F16/168G06F16/1815
Inventor 唐云飞宁海元张新铭杨思枢阚海明
Owner 杭州玳数科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap