Self-adaption method of thread pool of log collection system

An acquisition system and self-adaptive technology, applied in transmission systems, digital transmission systems, electrical components, etc., to achieve fast processing effects

Active Publication Date: 2015-03-25
706 INST SECOND RES INST OF CHINAAEROSPACE SCI & IND
View PDF3 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, threads are also used as system resources, and it is not an ideal choice to generate a new thread for each log processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Self-adaption method of thread pool of log collection system
  • Self-adaption method of thread pool of log collection system
  • Self-adaption method of thread pool of log collection system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Below we will observe the working effect of using the thread pool adaptive method proposed by the present invention in the log collection system through experiments.

[0037] The adaptive thread pool is deployed in the log collection system, and all time-related parameters are enlarged for the convenience of observation, and the time unit of the millisecond level is raised to the second level. According to the hard disk parameters used in the experiment, the actual time required for an IO operation is calculated to be 14 milliseconds, so the experimental parameter T IO = 14 seconds, log processing theoretical time T E = 1 second. Therefore, the sending rate of the log sender is divided into four stages, which are slow, slow, fast, and fast, as shown in Table 1.

[0038] Table 1 Log sender sending rate and number of sending

[0039]

speed

Number of logs

slow

Send a log every 2 seconds

20

slower

Send a log every 1.5 secon...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a self-adaption method of a thread pool of a log collection system and relates to self-adaption of related thread pools in log collection systems. An event-driven checker is added to the log collection system, and the thread pool is established by setting a threshold value alpha, theoretical time TE needed for log processing and log processing time tolerance beta which are related to a task queue. The process that the system receives each log is an event, and the checker is driven to detect whether the size of the current task queue exceeds alpha. Alpha is equal to two. If the size of the current task queue exceeds alpha, a thread heuristic method is started and comprises the following steps that the equation shown in the specification is calculated, tn is defined as the number of threads needing to be added theoretically, TE is defined as theoretical time needed for log processing, AVGTQ is defined as the mean interval of the logs reaching the system alpha times recently, and PT is defined as the number of threads in the thread pool at the moment; then, the value of tn and the value of alpha are judged, and the smaller value is selected as the number of threads needing to added really; finally, non-core threads or logs with idle waiting time being TE are created for cache processing. According to the method, the thread pool is adjusted dynamically, and rapid processing of the logs is guaranteed.

Description

technical field [0001] The invention mainly relates to a log collection system, and mainly considers the self-adaptive problem of the thread pool in the log collection system. Background technique [0002] With the development of information technology, various organizations have deployed a variety of network devices and application servers for their internal network needs. These application servers and network devices generate a large number of logs every day, and these logs record the running conditions of the devices and the access operations of users to the application servers. You can monitor network operation by analyzing logs, and optimize network devices and application servers. [0003] At present, because the log does not have a unified format specification requirement, which is not conducive to further analysis work, it is necessary to process the log, and thus the log collection system has emerged. However, with the increase of network traffic, the log sending ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/34H04L12/24
Inventor 孙宇王斌王润高高景生
Owner 706 INST SECOND RES INST OF CHINAAEROSPACE SCI & IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap