Check patentability & draft patents in minutes with Patsnap Eureka AI!

A log processing method and system

A technology for processing systems and processing methods, which is applied in the field of log processing methods and systems, can solve problems such as complex implementation, achieve the effects of reducing packet loss rate, improving log processing efficiency, and simplifying log processing procedures

Active Publication Date: 2020-09-08
NEW H3C TECH CO LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this processing method needs to monitor the resource utilization rate of each container in real time, which is more complicated to implement

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A log processing method and system
  • A log processing method and system
  • A log processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0037] The preset ratio threshold of the cache queue is 75%, and the number of currently started containers for processing logs is 3, and the containers use the active pull method to pull logs from the cache queue for processing.

[0038] The control module periodically (fixed period or non-fixed period) monitors the cache ratio of the cache queue. Taking the non-fixed period (for example, gradually shortening the monitoring period with the increase of the cache ratio) as an example, when the cache ratio of the cache queue is detected to be greater than 75% for the first time, it is used as the starting time point of this statistics, for example, as shown in Table 1 As shown, when the buffer ratio 78% greater than 75% is detected for the first time, it is recorded as the starting time point (0th second) of this statistics; if the monitoring period at this time is 5 seconds, the next statistical time point is At the 5th second from the starting time point, the cache ratio at th...

Embodiment 2

[0043] The preset ratio threshold of the cache queue is 75%, and the number of currently started containers for processing logs is 3, and the containers use the active pull method to pull logs from the cache queue for processing.

[0044] The control module periodically monitors the cache ratio of the cache queue. Taking a fixed period (for example, the monitoring period is 5 seconds) as an example, when the cache ratio of the cache queue is greater than 75%, the number of times the cache ratio is continuously greater than 75% is counted, as shown in Table 2.

[0045] cache ratio Number of statistics 78％ 1 85％ 2 80％ 3 78％ 4 82％ 5 86％ 6

[0046] Table 2

[0047] If the preset number of times threshold is 5 times, the current statistical number of times (6 times) is greater than the number of times threshold (5 times), and the cache ratio of the cache queue is always in a high state, and there is a risk of packet loss. Therefore, th...

Embodiment 3

[0049] The number of containers currently started for processing logs is 3, and the containers use the active pull method to pull logs from the cache queue for processing.

[0050] The control module periodically monitors the cache ratio of the cache queue. Taking the non-fixed cycle as an example, count the duration of the continuous increase of the cache ratio, as shown in Table 3. A certain monitoring time point is used as the starting time point of this statistics, for example, as shown in Table 3, the time point at which the buffer ratio is 50% is monitored as the starting time point of this statistics, and is recorded as the 0th second; if The monitoring period at this time is 5 seconds, then the next statistical time point is the 5th second from the starting time point, and the cache ratio at the 5th second of monitoring is 52%, which is greater than the previous statistical time point (0th second) Therefore, the current statistical cache ratio continues to increase fo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a log processing method and a log processing system. The method comprises the steps of determining a cache ratio of a cache queue by a control module, wherein the cache ratio isthe ratio of the number of cache logs in the cache queue accounting for the maximum number of the logs allowed to cache in the cache queue, and the logs are pulled from the cache queue by a containerin the log processing system and are processed; when the cache ratio satisfies the starting condition of the container, informing a container management module in the log processing system of starting a new container to participate in log processing. Through monitoring the state of the cache queue, the log processing capability of the log processing system is dynamically adjusted; roll polling isnot needed for the resource state of each container in the whole processing course, thus the processing flow is simplified.

Description

technical field [0001] The invention relates to the technical field of network communication, in particular to a log processing method and system. Background technique [0002] When the log processing system processes logs, if the processing efficiency is not high, packet loss and other situations may occur. At present, the log processing system generally monitors the resource utilization (for example, CPU, memory utilization, etc.) of the container (the process running in the container to process the log), and when the resource utilization of the container reaches the preset utilization threshold, increase A new container to improve the log processing efficiency of the log processing system and reduce the packet loss rate. However, this processing method needs to monitor the resource utilization rate of each container in real time, which is complicated to implement. Contents of the invention [0003] The purpose of the present invention is to provide a log processing me...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/30G06F11/34
Inventor 张闯
Owner NEW H3C TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com