System monitoring data collecting method with dynamically controllable traffic

A technology for monitoring data and traffic dynamics, applied in hardware monitoring and other directions, can solve problems such as the overall performance and stability of the system, and achieve the effect of ensuring quality

Active Publication Date: 2014-03-26
SHANGHAI STOCK EXCHANGE
View PDF2 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is to solve the technical problem that large-scale data collection and landing will have a significant impact on the overall performance and stability of the system during the collection process of the system status data of the transaction real-time system, and realize the large-scale data collection and landing. A dynamic and controllable system monitoring data acquisition method that not only ensures the smooth operation of the real-time system, but also can effectively meet the collection and distribution of characteristic data.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • System monitoring data collecting method with dynamically controllable traffic
  • System monitoring data collecting method with dynamically controllable traffic
  • System monitoring data collecting method with dynamically controllable traffic

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] figure 1 It is the monitoring data asynchronous acquisition and processing structure diagram of the present invention. The working principle of this part is that by calling the Linux system thread interface, the monitoring module starts a separate monitoring data processing thread (ProcessThread) after initialization to be responsible for the multi-target processing output of system data , each time the main program calls the data acquisition interface, it actually only caches the system state data internally to a data output request queue in memory, and does not generate actual IO operations; then, after the acquisition interface completes the queue operation, it will Return immediately and let the main program continue to execute its real-time application logic, thus essentially avoiding the danger of the IO that may be caused by the synchronous operation to hang the main program.

[0034] The monitoring data processing thread (ProcessThread) continuously polls the da...

Embodiment 2

[0037] The collection interface of the monitoring data collection library will check the occurrence frequency of system monitoring events in real time and dynamically adjust the flow of collected data when performing data enqueuing operations, so as to avoid the impact of a large amount of data IO operations on the overall performance of the system. This part is designed as follows: the system sets three parameters, namely, the flow control detection cycle (rateControlInterval), the data output request number threshold (rateControlCount) and the flow control execution cycle (rateControlDuration). The flow control algorithm first counts the number of data requests arriving in the queue during the flow control detection period. Once the number of requests reaches or exceeds the request threshold, the flow control mechanism is immediately activated and lasts until the end of the flow control execution period. Therefore, a total cycle of dynamic flow control can be divided into two...

Embodiment 3

[0039] In the flow control execution cycle, the collection interface of the monitoring data collection library no longer adds the newly arrived data to the data output request queue, but merges the same characteristic data through the pre-established hash table during initialization, so as to reduce the actual data. traffic target. You need to define the concept of "same characteristic data" (that is, Hash Key) according to your own business. For example, for a securities trading system, if the monitoring data comes from the same PBU or product number, then it can be defined as the same characteristic data. The data merging mechanism is: for a feature data type that has not appeared before, add an item in the e hash table; otherwise, only increase the count of the same record type.

[0040] Take the monitoring of system errors as an example: when a large number of errors burst out, the acquisition interface writes each piece of error data with the same characteristic data of t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of data collection and traffic control, in particular to a system monitoring data collecting method with dynamically controllable traffic. The system monitoring data collecting method with the dynamically controllable traffic mainly comprises three parts, namely monitoring data asynchronous collection and processing, traffic control starting detection and traffic control data combination. According to the monitoring data asynchronous collection and processing method, by calling a system thread interface, an independent monitoring data processing thread is started to take charge of multi-target processing output of system data after a monitoring module is initialized; the total period of dynamitic traffic control can be divided into two parts, namely a traffic control detection part Td and a traffic control execution part Te. Compared with the prior art, the system monitoring data collecting method with the dynamically controllable traffic has the advantages that through a traffic speed weight control algorithm, stable operation of a key real-time system is guaranteed, the collecting requirement of the monitoring data of different monitoring targets are effectively met, and the quality of ground data is guaranteed through a combination method of feature data.

Description

[technical field] [0001] The invention relates to the fields of data collection and flow control, in particular to a system monitoring data collection method with dynamically controllable flow. [Background technique] [0002] The system status data acquisition module can record the running status of the program, the site of the error and other data in real time, and output the collected data to multiple targets such as screens, files and databases, so as to help developers and operation and maintenance personnel understand the system status and audit operations process, troubleshooting error causes, archiving system data, etc. Therefore, in the development and operation of key real-time systems, especially for trading systems with high availability goals, a good design and implementation of data acquisition modules is not only conducive to code reuse, but also has important significance for the stable and efficient operation of the system . [0003] Today's mainstream moni...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/30
Inventor 金鑫武剑锋王泊刘凯陈雷王程程惠敏顺黄寅飞白硕
Owner SHANGHAI STOCK EXCHANGE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap