Log storage method and device and log query method and device

A log and storage technology, applied in the field of log processing, can solve the problems of high consumption, affecting normal business processing, and low efficiency of log query, and achieve the effect of improving efficiency

Pending Publication Date: 2020-09-22
BEIJING KINGSOFT CLOUD NETWORK TECH CO LTD
View PDF9 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Since the above-mentioned log query method is to scan and traverse each business node in the entire distributed system, the scanning process needs to consume a large amount of cpu (Central Processing Unit, central processing unit) and memory, which affects the processing of normal business, and the efficiency of log query Very low, time-consuming is often hours

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log storage method and device and log query method and device
  • Log storage method and device and log query method and device
  • Log storage method and device and log query method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0108] As an implementation manner of the embodiment of the present invention, the above method may also include:

[0109] Periodically clean up and / or maintain the stored log files.

[0110] In order to improve the performance of the log collection node and avoid resource waste, the log collection node can regularly clean up log files, delete expired historical data, and ensure the speed of log file storage and query. At the same time, the log collection node can also regularly maintain logs according to actual needs, so as to ensure smooth log file storage and query.

[0111] It can be seen that in this embodiment, the log collection node can regularly clean up and / or maintain the stored log files, which can ensure smooth log file storage and query, improve the performance of the log collection node, and avoid waste of resources.

[0112] Corresponding to the above log storage method, an embodiment of the present invention further provides a log query method, and the log qu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a log storage method and device and a log query method and device, the log storage method is applied to a service node in a distributed system, the distributedsystem further comprises a log collection node, and the method comprises the steps of: generating a log file according to the service processing situation; and sending the log file to a correspondinglog collection node according to a pre-established mapping relationship between the service node and the log collection node, so that the log collection node stores the log file; and establishing anindex for the stored log file. Thus, when log query is carried out, as the log file is stored in the log collection node, only log query needs to be carried out in the log collection node, processingof normal services by the service node is not influenced, and meanwhile, thousands of nodes in the distributed system do not need to be traversed, so that the log query efficiency is greatly improved.

Description

technical field [0001] The present invention relates to the technical field of log processing, in particular to a log storage method and device, and a log query method and device. Background technique [0002] With the development of the big data technology industry, large-scale clustered distributed systems have become the basic components of big data technology. A distributed system is often composed of thousands of business devices, that is to say, it has thousands of business nodes. The running logs of computer programs in these business nodes are distributed and the number is very large. Since the log can record the status of the computer program at runtime, the log can be analyzed to determine some problems in the business nodes, so log query is very important for the operation and maintenance of large-scale clustered distributed systems. [0003] At present, the log query for distributed systems is based on batch operation instructions such as ansible automatic opera...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/13G06F16/14G06F16/17G06F16/18
CPCG06F16/134G06F16/14G06F16/1734G06F16/1815
Inventor 赵宇徐寅斐侯雪峰
Owner BEIJING KINGSOFT CLOUD NETWORK TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap