Adaptive method of line pool in log collection system

An acquisition system and self-adaptive technology, applied in transmission systems, digital transmission systems, electrical components, etc., to achieve fast processing effects

Active Publication Date: 2017-05-17
706 INST SECOND RES INST OF CHINAAEROSPACE SCI & IND
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, threads are also used as system resources, and it is not an ideal choice to generate a new thread for each log processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Adaptive method of line pool in log collection system
  • Adaptive method of line pool in log collection system
  • Adaptive method of line pool in log collection system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Next, we will observe the working effect of using the thread pool adaptive method proposed by the present invention in the log collection system through experiments.

[0037] Deploy an adaptive thread pool in the log collection system. In order to facilitate observation, all parameters related to time are enlarged, and the time unit of millisecond level is increased to second level. According to the hard disk parameters used in the experiment, the actual time required to calculate an IO operation is 14 milliseconds, so the experimental parameter T IO = 14 seconds, the theoretical log processing time T E = 1 second. Therefore, the sending rate of the log sender is divided into four stages, namely slow, slow, fast, and fast, as shown in Table 1.

[0038] Table 1 Log sender sending rate and sending number

[0039] speed Number of logs slow Send a log every 2 seconds 20 slower Send a log every 1.5 seconds 20 faster Send a log every ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a self-adaption method of a thread pool of a log collection system and relates to self-adaption of related thread pools in log collection systems. An event-driven checker is added to the log collection system, and the thread pool is established by setting a threshold value alpha, theoretical time TE needed for log processing and log processing time tolerance beta which are related to a task queue. The process that the system receives each log is an event, and the checker is driven to detect whether the size of the current task queue exceeds alpha. Alpha is equal to two. If the size of the current task queue exceeds alpha, a thread heuristic method is started and comprises the following steps that the equation shown in the specification is calculated, tn is defined as the number of threads needing to be added theoretically, TE is defined as theoretical time needed for log processing, AVGTQ is defined as the mean interval of the logs reaching the system alpha times recently, and PT is defined as the number of threads in the thread pool at the moment; then, the value of tn and the value of alpha are judged, and the smaller value is selected as the number of threads needing to added really; finally, non-core threads or logs with idle waiting time being TE are created for cache processing. According to the method, the thread pool is adjusted dynamically, and rapid processing of the logs is guaranteed.

Description

technical field [0001] The invention mainly relates to a log collection system, and mainly considers the self-adaptation problem of the thread pool in the log collection system. Background technique [0002] With the development of information technology, organizations have deployed a variety of network devices and application servers for their internal network needs. These application servers and network devices generate a large number of logs every day, which record the operation of the devices and the access operations of users to the application servers. You can monitor network operations by analyzing logs, and optimize network devices and application servers. [0003] At present, since there is no unified format specification requirement for logs, which is not conducive to further analysis work, it is necessary to process logs, thus resulting in a log collection system. However, with the increase of network traffic, the log sending speed of log senders such as network...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/34H04L12/24
Inventor 孙宇王斌王润高高景生
Owner 706 INST SECOND RES INST OF CHINAAEROSPACE SCI & IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap