SNMP-based database index monitoring method, device, equipment and medium

Abstract

The application discloses a database index monitoring method and device based on SNMP, equipment and medium, relates to the technical field of database monitoring, and comprises the following steps: automatically loading and initializing SNMP when an Agent process is started; receiving monitored database indexes and mapping to form extended OIDs; forming an inner loop control mechanism based on a PID control algorithm and a cache queue length, forming an outer loop control mechanism based on network state parameters, and adjusting the index sending rate to send database monitoring indexes according to the double-loop index sending rate. By embedding SNMP in the Agent process, binding the SNMP life cycle with the Agent, and then mapping the database indexes to form the extended OIDs which can be dynamically extended. The double-loop control mechanism dynamically adjusts the sending frequency from the double-layer control mechanism of network load and adaptation, and solves the problems of complicated configuration, difficult expansion and data delay in traditional monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database monitoring, and in particular to a database index monitoring method and device based on SNMP, equipment and medium. BACKGROUND

[0002] With the deep development of information technology, the data storage and management scale of enterprises and organizations grows exponentially, and the monitoring of database and system index data has become a core technology practice to ensure the stability and performance of data infrastructure. As the nerve center of digital operation, the continuous and healthy operation of various databases and systems is directly related to business continuity and service quality. Real-time monitoring and periodic maintenance of key indicators (such as CPU utilization, disk I / O, transaction processing efficiency, etc.) can not only provide early warning of potential risks, but also optimize resource allocation through performance trend analysis, laying a technical foundation for the safe and efficient operation of data assets in the digital transformation of enterprises.

[0003] In the actual application scene of monitoring indicators, the traditional monitoring method based on SNMP protocol faces multiple technical bottlenecks. First, in the complexity of the configuration link, SNMP agent deployment needs to manually complete port opening, OID encoding mapping and permission authentication, etc. operations, which consumes manpower and material resources, and the long-term exposed port is more likely to become a breakthrough for OID traversal attack, unauthorized access and other security threats; second, the index expansion is insufficient, when the business demand changes, the monitoring indicators need to be added or modified, the monitoring component code needs to be modified and the service needs to be restarted; in addition, the timeliness of the data is not good, when using fixed polling interval combined with passive caching strategy, all data indicators are cached based on the same polling interval, which cannot distinguish between key indicators such as database connection number and ordinary indicators, resulting in delay in abnormal early warning of core indicators, and lack of dynamic adjustment mechanism in network congestion; finally, the real-time performance, flexibility and other indicators of database index monitoring are difficult to meet the growing monitoring needs of enterprises. SUMMARY

[0004] The embodiment of the present application provides a database index monitoring method, device, equipment and medium based on SNMP to solve the technical problems of insufficient flexibility and real-time performance of database index monitoring.

[0005] In a first aspect, the embodiment of the present application provides a database index monitoring method based on SNMP, comprising:

[0006] S101, through the ApplicationListener event listening mechanism, automatically loading the configuration file and initializing the SNMP engine when the Agent process starts, and constructing the MIB tree structure;

[0007] S102, receiving the monitored database index, mapping the monitored database index into an OID extension field according to a preset mapping rule, forming an extended OID according to the OID extension field, associating the extended OID with the monitored database index, and registering the extended OID to a memory MIB tree;

[0008] S103, based on a PID control algorithm, establishing an inner loop control mechanism of the index sending rate according to the length of the cache queue, establishing an outer loop control mechanism of the index sending rate based on network state parameters, adjusting the index sending rate according to the inner loop control mechanism and the outer loop control mechanism at the same time, and forming a double-loop index sending rate combining the inner loop and the outer loop;

[0009] S104, sending the database monitoring index in the MIB tree to the monitoring server according to the double-loop index sending rate.

[0010] Further, the S101 comprises:

[0011] Using the Spring container, defining the SNMP instance through code annotation when the Agent process is started, and defining the calling interface and message distributor of the SNMP instance in the Agent process;

[0012] Based on the OID starting value, a readable and writable MIB table model is established, variable row objects are created in the table according to the number of OID starting values, and the variable row objects are cached locally and registered to the SNMP server according to the KeyName of each row.

[0013] Further, the S102 comprises:

[0014] By scanning the code annotation, the metadata of the code annotation is extracted, the monitored database index is mapped into an instance field, an index field, a grouping field and an attribute field according to a preset mapping rule, and is spliced with an OID starting value to form an extended OID;

[0015] The extended OID is associated with the monitored database index and is registered to the memory MIB tree.

[0016] Further, the association of the extended OID with the monitored database index and the registration of the extended OID to the memory MIB tree comprise:

[0017] Using dynamic proxy technology, MIB entities are created according to the extended OID, the data type and the access authority of the monitored database index, and the MIB entities are registered to the memory MIB tree by using the MIB service interface of SNMP4J.

[0018] Further, the S103 comprises:

[0019] Based on the PID control algorithm, the index transmission rate is calculated according to the length of the buffer queue, the target transmission rate is generated, and an inner-loop control mechanism for the index transmission rate is established according to the target transmission rate and the length of the buffer queue.

[0020] Based on network state parameters, a correction factor is calculated according to network latency, packet loss rate and available bandwidth utilization. The correction factor is used to adjust the index transmission rate and establish an outer loop control mechanism for the index transmission rate.

[0021] The index transmission rate is adjusted simultaneously using both inner-loop and outer-loop control mechanisms to form a dual-loop index transmission rate.

[0022] Furthermore, the inner ring adjustment mechanism includes:

[0023] Based on the PID control algorithm, the proportional term is calculated according to the current real-time actual queue length, the integral term is calculated according to the historical queue length, and the derivative term is calculated according to the historical queue length change rate. The target transmission rate is calculated using the proportional term, integral term, and derivative term.

[0024] The queue length adjustment parameter is calculated based on the current real-time actual queue length and the preset queue capacity threshold. The inner loop rate adjustment value is generated based on the target transmission rate and the queue length adjustment parameter, forming an inner loop control mechanism for the target transmission rate.

[0025] Furthermore, the method also includes:

[0026] The particle swarm optimization algorithm (PSO) is used to dynamically optimize the PID control algorithm, forming an optimized inner-loop control mechanism.

[0027] Secondly, embodiments of the present invention provide a database metric monitoring device based on SNMP, comprising:

[0028] The SNMP embedded module is used to automatically load the configuration file, initialize the SNMP engine, and build the MIB tree structure when the Agent process starts, through the ApplicationListener event listening mechanism.

[0029] The OID dynamic extension module is used to receive the monitored database metrics and map them into extended fields of OID according to the preset mapping rules to form extended OIDs, associate them with the monitored database metrics, and register them to the in-memory MIB tree.

[0030] The indicator transmission rate control module is used to form a dual-loop indicator transmission rate adjustment mechanism, which is based on the inner loop control mechanism established by the PID control algorithm according to the length of the buffer queue, and the outer loop control mechanism established by the network state parameters.

[0031] The monitoring index sending module is configured to send the database monitoring index in the MIB tree to the monitoring server according to a double-ring index sending rate.

[0032] In a third aspect, an electronic device includes:

[0033] one or more processors;

[0034] a memory device storing one or more programs,

[0035] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described SNMP-based database index monitoring method.

[0036] In a fourth aspect, a storage medium containing computer executable instructions for performing the above-described SNMP-based database index monitoring method when executed by a computer processor is provided.

[0037] The embodiment of the application provides a database index monitoring method, device and equipment based on SNMP and a medium, the method embeds an SNMP protocol stack in an agent process in the form of a native component by using a Spring container, and integrates an SNMP interface in the agent process by using dependency injection; application program listens to events, loads a configuration file and initializes an SNMP engine automatically when the agent process starts, obtains database indexes and generates instances through the integrated interface, and further constructs an MIB tree. According to a preset mapping rule, the database indexes at the business level are mapped into extended fields of OIDs to form extended OIDs that can be dynamically extended, and then the extended OIDs are associated with the database indexes and registered to the MIB tree. According to the length of a cache queue, an inner loop control mechanism is formed, and together with an outer loop control mechanism formed by network state parameters, a double-loop control mechanism is formed to adjust the index sending rate of the database indexes. Through deep integration at the process level, the life cycle of the SNMP agent is strictly bound to the agent, which can avoid interference of the agent function on the running of the agent main body; the extended OIDs formed through path mapping can dynamically adjust the OID format according to the actual use scene, automatically convert the business index name into a standard number, and can automatically complete OID mapping allocation, MIB tree registration and permission binding and other processes, thereby saving the tedious operation of manually writing MIB files or maintaining OID mapping tables; the inner loop control mechanism based on the PID control algorithm and the cache queue length and the outer loop control mechanism based on the network state parameters can dynamically adjust the sending frequency from the double-layer control mechanism of the network load and adaptation, guarantee the timeliness and stability of the database index sending, effectively solve the core problems of complex configuration, difficult expansion and data delay in the traditional monitoring, improve the adaptability of the database monitoring scene through pure data-driven adaptive adjustment, and reduce or even avoid the risk caused by code intrusion. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are incorporated herein for illustrating the specific embodiments of the present application and explaining the present application, and are not intended to define improper limitations of the present application. In the drawings:

[0039] Figure 1 A flowchart of a database index monitoring method based on SNMP according to the first embodiment of the present application;

[0040] Figure 2 A flowchart of a database index monitoring method based on SNMP according to the second embodiment of the present application;

[0041] Figure 3 A flowchart of a database index monitoring method based on SNMP according to the third embodiment of the present application;

[0042] Figure 4A structural schematic diagram of a database index monitoring device based on SNMP according to Embodiment Four of the present application;

[0043] Figure 5 A structural diagram of an electronic device according to Embodiment Five of the present application. DETAILED DESCRIPTION

[0044] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for the purpose of interpretation of the present application and are not limiting of the present application. In addition, it should be noted that, for the purpose of description, only the parts related to the present application are shown in the accompanying drawings and not all the structures.

[0045] Database indexes are various system state parameters of a database system during operation, and the working state of the database can be better mastered by monitoring the database indexes, and potential threats or abnormal states of the database can be warned. In a traditional SNMP monitoring scenario, there are problems of complicated configuration and complex port management, and the traditional agent is also prone to becoming a target of port scanning and DDoS attack due to long-term independent listening to a fixed port. Especially in the database monitoring scenario of multiple instances and multiple indexes, the poor expansibility of data indexes becomes a core problem of limiting business dynamic adjustment. When a database index to be monitored needs to be added or modified, not only the SNMP agent configuration file needs to be manually modified, the index collection logic needs to be redefined, but also the MIB file needs to be written or adjusted, the OID needs to be allocated for the new index and registered in the MIB tree, and the service needs to be restarted each time the index is changed, which is complicated and has poor business adaptability. At the same time, there are key indexes such as the number of database connections in the database indexes, and if the polling or polling cache combined sending scheme is used for the key indexes and other ordinary indexes, the abnormal warning of the core indexes is prone to lag, and there is also a lack of network congestion dynamic adjustment mechanism, which will cause data accumulation or packet loss once the bandwidth fluctuates, and the importance difference of indexes and network environment self-adaptability are also lacking. Therefore, a database index monitoring method with safety, strong expansibility and strong adaptability is needed.

[0046] Embodiment One

[0047] Figure 1 A flowchart of a database index monitoring method based on SNMP according to Embodiment One of the present application, in which SNMP is embedded in the Agent process, and the extensible dynamic OID is automatically generated according to the preset rules and mapped to the MIB tree, and finally the index sending rate is adjusted according to the cache queue length and network environment parameters, and the specific steps include the following steps:

[0048] S101, through the ApplicationListener event listening mechanism, automatically load configuration file and initialize SNMP engine when the Agent process starts, and build MIB tree structure.

[0049] Through the deep integration of process level, the SNMP protocol stack is embedded into the Agent process in the form of native components by using Spring container, and the SNMP protocol stack is seamlessly integrated with the Agent process by dependency injection, so that the SNMP agent becomes an integral part of the process. When the Agent process starts, with the help of ApplicationListener application event listening mechanism, by automatically loading configuration file and initializing SNMP engine, through the integrated interface, the Agent process can call SNMP protocol to collect database indicators and generate instances, and then build MIB tree; When the Agent process stops, the resource release is completed synchronously, the life cycle of the SNMP agent is strictly bound with the Agent, which can avoid the interference of the agent function to the Agent main body running. At the same time, according to the required monitoring index content and quantity, the MIB tree structure is constructed, which is used to store the database index objects to be monitored, and sent to the monitoring server in the subsequent process.

[0050] S102, receiving the monitored database indicators, mapping the monitored database indicators to OID extension fields according to the preset mapping rule, and forming an extended OID according to the OID extension field, associating the extended OID with the monitored database indicators and registering to the memory MIB tree.

[0051] Through path mapping technology, the business level database indicators (such as indicator name) are automatically mapped to instance number, indicator number and other extension fields, and the extended OID is formed on the basis of the OID starting number, so that the OID can automatically adapt to the database indicator monitoring requirements of multiple scenes and multiple instances. After the OID is associated with the monitored database indicators, it is registered to the memory MIB tree, and then the instances in the MIB tree can be sent to the indicator monitoring server through the cache queue, and the potential threat and abnormal risk of the database are warned. The tedious operation of manually writing MIB file or maintaining OID mapping table is saved, and the YAML / JSON declarative configuration file can also be combined, the user only needs to declare business parameters such as indicator name and collection frequency in the configuration, and the system can automatically complete the whole process of OID allocation, MIB tree registration and permission binding, simplifying the configuration process of new monitoring indicators to pure business parameter filling.

[0052] S103, based on the PID control algorithm, the length of the cache queue is used to establish the inner loop control mechanism of the index sending rate, the network state parameters are used to establish the outer loop control mechanism of the index sending rate, the index sending rate is adjusted based on the inner loop control mechanism and the outer loop control mechanism, and a double-loop index sending rate combining the inner loop and the outer loop is formed.

[0053] Based on the PID control algorithm, the length of the cache queue is used as a real-time feedback signal to accurately calculate and adjust the index sending rate, and the index sending rate is further adjusted in combination with the length of the cache queue, forming an inner loop control mechanism of the index sending rate. Meanwhile, based on the real-time collected network state parameters (such as RTT, packet loss rate, available bandwidth), a rate correction factor is calculated, forming an outer loop control mechanism, which is secondarily adjusted based on the inner loop control mechanism, taking into account short-term load response and long-term network adaptation, which can effectively avoid network congestion or delay aggravation, through pure data-driven adaptive adjustment, ensuring the timeliness of database index sending, and can be widely applied to various database monitoring scenarios.

[0054] S104, according to the double-loop index sending rate, the database monitoring indexes in the MIB tree are sent to the monitoring server.

[0055] According to the real-time dynamic adjustment of the double-loop index sending rate formed by the cache queue length and the network environment, the database monitoring index instances associated with the OID in the MIB tree are sent to the monitoring server, the monitoring server can identify the content of the database monitoring index according to the OID, and then analyze the database state according to the identified content, detect the abnormal state of the database, and also can prewarn the potential risks according to the change of the index.

[0056] The embodiment embeds the SNMP protocol stack in the form of a native component in the Agent process by using the Spring container, and integrates the SNMP interface in the Agent process by using dependency injection; the application program listens to events, automatically loads the configuration file and initializes the SNMP engine when the Agent process starts, obtains the database indicators through the integrated interface and generates instances, and then constructs the MIB tree. According to the preset mapping rule, the business-level database indicators are mapped into the extension field of the OID to form an extended OID that can be dynamically extended, and then the extended OID is associated with the database indicators and registered to the MIB tree. According to the length of the cache queue, an inner ring control mechanism is formed, which together with the outer ring control mechanism formed by the network state parameters forms a double ring control mechanism to adjust the indicator sending rate of the database indicators. Through the deep integration at the process level, the life cycle of the SNMP agent is strictly bound to the Agent, which can avoid the interference of the agent function on the running of the Agent main body; the extended OID formed through path mapping can dynamically adjust the OID format according to the actual use scene, automatically convert the business indicator name into a standard number, and can automatically complete the OID mapping allocation, MIB tree registration and permission binding processes, thereby saving the tedious operation of manually writing MIB files or maintaining the OID mapping table; the inner ring control mechanism based on the PID control algorithm and the length of the cache queue and the outer ring control mechanism based on the network state parameters can dynamically adjust the sending frequency from the double-layer control mechanism of network load and adaptation, guarantee the timeliness and stability of the database indicator sending, effectively solve the core problems of complex configuration, difficult expansion and data delay in traditional monitoring, improve the adaptability of the database monitoring scene through pure data-driven adaptive adjustment, and reduce or even avoid the risk caused by code invasion.

[0057] Embodiment two

[0058] Figure 2 The flowchart of the SNMP-based database indicator monitoring method according to Embodiment Two of the application, which is optimized on the basis of the above-mentioned embodiment. In this embodiment, S101 is specifically optimized as follows:

[0059] Using the Spring container, the SNMP instance is defined through code annotation when the Agent process starts, and the calling interface and message distributor of the SNMP instance are defined in the Agent process.

[0060] Based on the OID starting value, a readable and writable MIB table model is established, variable row objects are created in the table according to the number of OID starting values, and the variable row objects are cached locally and registered to the SNMP server according to the KeyName of each row.

[0061] Correspondingly, the SNMP-based database indicator monitoring method provided by the embodiment specifically comprises:

[0062] S201, using Spring container, defining SNMP instance through code annotation when Agent process starts, and defining calling interface and message dispatcher of SNMP instance in Agent process.

[0063] Using the automatic assembly mechanism of Spring Boot of Spring container, defining SNMP instance, TransportMapping transmission mapping interface called by Agent process, and MessageDispatcher message dispatcher for allocating database index to instance in Agent process using @Configuration and @Bean annotation when Agent process starts. Through reading annotation and automatic loading when Agent process starts, the SNMP protocol stack is seamlessly integrated in Agent process, and corresponding functional modules are automatically created and initialized when process starts, which tightly binds SNMP agent service with life cycle of Agent process, isolates with external environment, starts with Agent and terminates with Agent.

[0064] S202, establishing readable and writable MIB table model based on OID starting value, creating variable row object in table according to number of OID starting value, and caching variable row object in local and registering to SNMP server according to KeyName of each row.

[0065] First, a readable and writable MIB table model for storing OID is established, MOTable and MOMutableColumn are used by SnmpAgent to establish based on OID starting value (for example, 1.3.6.1.4.1.9999), then DefaultMOMutableRow2PC is used by SnmpAgentProxy to create variable row object in table, the variable row object is used to store OID and index value of monitoring index, the variable row object is a row object whose field number and field content can be changed, each row is used to store OID and specific index value of one database index. Then, cache to local according to KeyName of each row, and call registerSnmpAgent() function to register all MIB variable row objects in cache to SNMP server. Through converting business data (such as SnmpData) into mapping relationship with OID, and dynamically registering MIB object to SNMP service through cache mechanism, automatic construction and management of runtime MIB tree are realized. Through using doWrite(Map<String, List <string>The method realizes the support capability of the automatic expansion mechanism of the row object, especially when the number of the database indexes to be monitored after the addition of the database indexes to be monitored or the switching of the monitoring scene is greater than the current target row, the new variable row object can be dynamically created to be inserted into the MIB table and the cache, so as to ensure the consistency of the data.

[0066] In addition, in the security layer, the system comprehensively supports the SNMPv2 and SNMPv3 protocols, including the USM user authentication (SHA / MD5) and encryption (DES / AES and its extension), multiple users with different security policies are added by rewriting the addUsmUser(USM usm) method, and the authentication password, encryption algorithm and key are specified; at the same time, the VACM model is combined to realize fine-grained access control, multiple access groups are created by the addViews(VacmMIB vacm) method, and the corresponding OID tree view permissions (read / write / notify) are allocated to them, and the access range of each node is accurately controlled. In the network and isolation design, the system initializes the UDP transport mapping through the initTransportMappings() function, the default listening address is read from the custom configuration file by the constructor, the custom configuration file is a network environment configuration file defined according to the current transmission situation, and dynamic port binding is realized in combination with the SNMPv3 security model. The SNMP4J underlying implementation can also be encapsulated inside SnmpAgentProxy through class loading isolation, only exposing necessary operation interfaces (such as register(SnmpData snmpData), write(SnmpData snmpData), etc.) to the outside, and hiding the underlying implementation details of the SNMP protocol stack; through the thread pool isolation mechanism, the ScheduledExecutorService independent thread pool set by SnmpAgent during initialization handles the proxy tasks, limits the concurrent resources, and prevents service crashes caused by high concurrency; and through the Cache cache management strategy, the MIB object is prevented from being created and destroyed frequently, and the cache information is prevented from being obtained by reflection and other methods from the outside, which improves the performance and enhances the context isolation capability. The life cycle of the Agent service is driven by the initSnmpAgent() function, which completes the SNMP engine initialization, configuration loading, context registration, service startup and other steps in turn, and all operations are recorded through the log to record the key events, which is convenient for subsequent debugging and operation and maintenance.

[0067] S203, receiving the monitored database indexes, mapping the monitored database indexes into OID extension fields according to a preset mapping rule, forming an extension OID according to the OID extension fields, associating the extension OID with the monitored database indexes, and registering the extension OID to the memory MIB tree.

[0068] Specifically, by scanning the code annotation, the metadata of the code annotation is extracted, the monitored database index is mapped into instance fields, index fields, grouping fields and attribute fields by using a preset mapping rule, and is spliced with an OID starting value to form an extended OID.

[0069] For the scalability problem of the monitored database indicators, the path mapping between the indicator name and OID of the database indicators is performed through the syntax parser. The mapping relationship can be cached through ConcurrentHashMap to support high-frequency queries. The standard OID prefix of the database indicators is used as the starting number (for example, 1.3.6.1.4.1.9999.1), and the hierarchical mapping rule including the "starting number, instance number, indicator number, grouping number, attribute number" is established to avoid OID allocation confusion and hierarchical relationship ambiguity, so that the newly added database indicators can quickly integrate into the existing monitoring system. Through the dynamic binding of the custom mapping field of the Agent configuration file and the collector port number (for example, port 10000 is mapped to 100, corresponding to the OID segment 1.3.6.1.4.1.9999.1.100), the standardized indicator name (such as "GBase8s_Er_Cluster_Server") is associated with the exclusive indicator number (such as 1) to form the extended OID in the form of 1.3.6.1.4.1.9999.1.100.1.1, 1.3.6.1.4.1.9999.1.100.1.2, which can dynamically expand the field. For example, the ConfigParse parser is used to parse the database indicator configuration, the path of the YAML configuration file snmp-config.yml is obtained from AppConfigProperties based on Spring dependency injection, the mapping relationship between the database indicator number and the database instance number under the database instanceNumbers node (format: List<Map<String, Integer>>) is parsed using the SnakeYAML library, and the YAML configuration file is used to query which database indicators are monitored under each database instance. The existence verification of the configuration stream, the data type forced verification and the default processing of the null value are also included in the parsing process, and finally the mapping list of the indicator number and the instance number of each indicator is generated. Through the annotation-driven mode, the collection class annotated by the developer through the @MetricDefinition annotation (such as @MetricDefinition(metricName="GBase8s_Er_Cluster_Server")) is recognized, the BeanPostProcessor mechanism of the Spring container can automatically scan and extract the metadata of the annotation in the tree pooling stage of the Agent process, and the metadata is associated with the above generated indicator number and instance number to generate the extended field of the OID, form the instance number, indicator number, grouping number and attribute number of each database indicator, and splice with the starting number of the OID, and finally generate the complete extended OID and register it to the MetricRegistry.In the dynamic generation process of OID, manual configuration or service restart is no longer needed, and the operation of extending the index is simplified from the "code development level" to the "configuration declaration level", and only the mapping relationship between the database instance and the database index needs to be declared in the YAML configuration file.

[0070] Using dynamic proxy technology, MIB entities are created according to the extended OID, the data type and access permission of the monitored database index, and the MIB entities are registered to the memory MIB tree using the MIB service interface of SNMP4J.

[0071] The SnmpMibProxy proxy class for managing the MIB tree is generated by dynamic proxy technology, and the node entity MibNode of the MIB tree containing the extended OID, the index data type and the index access permission is created according to the metadata of the database index. The variable row object in the MIB table is inserted into the node entity of the MIB tree, and then the MIB entity is registered to the memory MIB tree using the MibServer interface of SNMP4J. When binding the permission of the node entity in the MIB tree, the RBAC policy can be integrated to associate the SNMPv2 and SNMPv3 user groups according to the identification number of the database instance, so that the MIB tree supports configuration hot update and incremental MIB loading, and manual maintenance of the OID mapping table is no longer needed. Only by configuring the driver and automatic parsing, the hot update of the monitored index can be realized, and the business department can independently extend the database index dimension to be monitored, which ensures the consistency and security of the index monitoring in the multi-instance and multi-index scenario, and reduces the system maintenance cost.

[0072] S204, based on the PID control algorithm, an inner loop control mechanism of the index sending rate is established according to the length of the cache queue, an outer loop control mechanism of the index sending rate is established based on the network state parameter, the index sending rate is adjusted according to the inner loop control mechanism and the outer loop control mechanism, and a double-loop index sending rate combined with the inner loop and the outer loop is formed.

[0073] S205, according to the double-loop index sending rate, the database monitoring index in the MIB tree is sent to the monitoring server.

[0074] The embodiment embeds the SNMP protocol stack in the form of a native component into the agent through process-level deep integration and dependency injection, strictly binds the life cycle of the SNMP agent to the agent, maps the database indicators at the service level to the extension fields of the OIDs through path mapping, splices the extension OIDs with the starting numbers of the standard OIDs to form the extended OIDs, dynamically extends the database indicators according to the actual application scenarios, no longer needs to manually write the MIB file or maintain the OID mapping table, and combines the YAML declaration configuration file to automatically complete the OID allocation and MIB tree registration, shields the details of the SNMP underlying protocol, and especially when the database indicators that need to be monitored are added, only the business parameters need to be filled in, thereby shortening the online cycle of the added monitoring indicators.

[0075] Embodiment three

[0076] Figure 3 A flowchart of a database indicator monitoring method based on SNMP according to the embodiment three of the application, the embodiment is optimized based on the above-mentioned embodiment, in the embodiment, S103 is specifically optimized as:

[0077] Based on the PID control algorithm, the target sending rate is generated according to the length of the cache queue, and the inner loop control mechanism of the indicator sending rate is established according to the target sending rate and the length of the cache queue.

[0078] Based on the network state parameters, the correction factor is calculated according to the network delay, the packet loss rate and the available bandwidth utilization, the indicator sending rate is adjusted by using the correction factor, and the outer loop control mechanism of the indicator sending rate is established.

[0079] The inner loop control mechanism and the outer loop control mechanism are used to adjust the indicator sending rate at the same time, and the double-loop indicator sending rate is formed.

[0080] Correspondingly, the database indicator monitoring method based on SNMP provided by the embodiment specifically includes:

[0081] S301, automatically loading a configuration file and initializing an SNMP engine when an agent process is started through an ApplicationListener event listening mechanism, and constructing a MIB tree structure.

[0082] S302, receiving a monitored database indicator, mapping the monitored database indicator to an OID extension field according to a preset mapping rule, forming an extended OID according to the OID extension field, associating the extended OID with the monitored database indicator, and registering the extended OID to a memory MIB tree.

[0083] S303, based on the PID control algorithm, calculates the target transmission rate according to the length of the buffer queue, generates the target transmission rate, and establishes an inner-loop control mechanism for the target transmission rate based on the target transmission rate and the length of the buffer queue.

[0084] To precisely control the transmission rhythm of indicator data, a classic PID control algorithm is used. Based on the length of the buffer queue as a reference value, the proportional term, integral term, and micro-sharing are calculated separately and combined to generate the target transmission rate. Then, based on the length of the buffer queue, it is determined whether the buffer queue exceeds the preset limit. Together with the target transmission rate, they form an inner-loop control mechanism for real-time adjustment of the indicator transmission rate.

[0085] One optional implementation of this embodiment is that the inner loop control mechanism includes:

[0086] Based on the PID control algorithm, the proportional term is calculated according to the current real-time actual queue length, the integral term is calculated according to the historical queue length, and the derivative term is calculated according to the historical queue length change rate. The target transmission rate is then calculated using the proportional term, integral term, and derivative term.

[0087] The classic PID control algorithm consists of three core parts: a proportional term (P), an integral term (I), and a derivative term (D). The proportional term adjusts the output proportionally according to the current error, providing a fast response. Its calculation formula is: ,in This is the proportionality coefficient, which determines the strength of the system's response to the current error. express The deviation between the actual queue length and the target value at any given time is used for rapid response to instantaneous fluctuations; the integral term accumulates and sums historical errors to eliminate the system's steady-state error, and the calculation formula is as follows. ,in These are integral coefficients used to eliminate steady-state errors. Indicates time τ The error value at which, τ It is an integral dummy variable, representing the time from the initial time 0 to the current time. τ Any past moment between; It's about time. τ The small increment indicates that the integral is a continuous accumulation of the error over time; the differential term makes advance predictions and adjustments based on the rate of change of the error, suppressing rapid changes in the error in advance, as shown in the formula: ,in These are the differential coefficients. In an infinitesimal time interval Within, error The minute changes Representing an infinitesimal increment in time, it is used to define the limiting process of the derivative, helping to suppress overshoot and predict trend changes. The formula for calculating the target transmission rate is:

[0088]

[0089] in, Indicates the target transmission rate. Indicates time.

[0090] The queue length adjustment parameter is calculated based on the current real-time actual queue length and the preset queue capacity threshold. The inner loop rate adjustment value is generated based on the target transmission rate and the queue length adjustment parameter, forming an inner loop control mechanism for the target transmission rate.

[0091] Furthermore, based on preset thresholds and real-time queue length, it can be determined whether the real-time cache queue length is too long or too short, and the sending rate can be further adjusted. A LinkedBlockingQueue queue can be used in the kernel cache to monitor the queue length in real time. The preset thresholds refer to the balance between data generation and sending rates in historical data. For example, an upper threshold H (typically 70% of the queue capacity) and a lower threshold L (typically 30% of the queue capacity) can be set. When the queue length > H, the data sending rate increases to 1.5 times the normal rate; when the queue length < L, the sending rate decreases to 0.6 times the normal rate; when it is between H and L, the normal rate is maintained. This allows for further adjustment of the target sending rate based on the target sending rate, forming an inner-loop control mechanism for the target sending rate.

[0092] Optionally, the particle swarm optimization algorithm (PSO) can be used to dynamically optimize the PID control algorithm, forming an optimized inner-loop control mechanism.

[0093] To improve the adaptability of PID in complex scenarios, the system further employs the particle swarm optimization (PSO) algorithm. Dynamic optimization is performed. These three parameters are used as the particle's three-dimensional spatial position, and the velocity and position are updated in each iteration using the following formula:

[0094]

[0095]

[0096] in, Inertial weights are used to control the balance between particle exploration and development. Indicates the first Particles in time The speed of the particle determines its direction and distance of movement in the search space; These are learning factors, representing the influence of individual experience and group experience, respectively. represents the position of the th particle at time , which is the coordinate of the particle in the search space, corresponding to the parameters of the PID controller is a random number in the interval [0, 1]; is the individual optimal solution of the i th particle is the global optimal solution of the entire population. The PSO optimization algorithm evaluates the effect of each set of parameters by defining a fitness function (such as minimizing the sum of squared errors MSE or maximizing throughput stability), and continuously iterates and updates during the running process, finally converging to the optimal parameter combination under the current environment; the iteration process can be executed periodically in the background, or triggered according to the system state, so as to realize the real-time adaptive adjustment of the PID parameters, and significantly improve the response ability and stability of the system under the condition of burst traffic or network fluctuation.

[0097] S304, based on the network state parameters, according to the network delay, packet loss rate and available bandwidth utilization, the correction factor is calculated, and the index sending rate is adjusted by using the correction factor, and the outer loop control mechanism of the index sending rate is established.

[0098] By collecting network state parameters, the index sending rate is adjusted according to the state of the transmission network. By sending a probe packet to the remote host once every 500 milliseconds, and recording the sending and receiving time stamps and response state, the key performance indicators of the current network are calculated: RTT (round trip time), packet loss rate and available bandwidth utilization. When it is detected that the packet loss rate exceeds 5% or the bandwidth utilization is higher than 80%, the system determines that the current network is in a congested or unstable state, and the rate adjustment mechanism will be triggered to avoid packet loss or retransmission problems caused by too fast data sending. For example, based on the weighted moving average algorithm, the collected network indicators are smoothed, and the RTT, packet loss rate and bandwidth utilization collected in the last n times are taken as input data. The calculation formula of the weighted average value of the packet loss rate

[0099]

[0100] wherein, , , represents the packet loss rate collected in the last three times, , , represents the weight . Then the weighted average value obtained is compared with the preset threshold value to generate the rate correction factor of the packet loss rate, if the weighted packet loss rate exceeds 5%, it means that there is a significant packet loss risk in the network, then​​ 0.5 if the bandwidth utilization is higher than 80%, indicating that the network is close to saturation 0.7; keep the rate correction factor unchanged in other cases , that is, no adjustment is made to the rate. The index sending rate is adjusted by the correction factor, forming an outer loop control mechanism for the index sending rate.

[0101] S305, the index sending rate is adjusted by the inner loop control mechanism and the outer loop control mechanism at the same time, forming a double-loop index sending rate.

[0102] The outer loop control mechanism of the correction factor is combined with the inner loop control mechanism of the queue length to multiply the PID control algorithm, realize the double-loop index sending rate of the inner loop control mechanism and the outer loop control mechanism adjusting the index sending rate at the same time, realize the early prediction and active response to network congestion through continuous perception of network state and trend analysis of historical data, not only improve the adaptability of the system in the network fluctuation environment, but also effectively guarantee the stable transmission of monitoring data in complex network conditions, further enhance the reliability and robustness of the overall monitoring system.

[0103] S306, according to the double-loop index sending rate, the database monitoring index in the MIB tree is sent to the monitoring server.

[0104] In this embodiment, based on the PID control algorithm, the cache queue length is taken as the feedback signal, and the cache queue length is taken as the basic parameter of the control index sending rate to calculate the target sending rate. At the same time, according to whether the cache queue length exceeds the preset limit, the target sending rate is adjusted again, forming an inner loop control mechanism. At the same time, network environment parameters are continuously collected, and a correction factor is calculated according to the network environment, forming an outer loop control mechanism. The inner loop control mechanism and the outer loop control mechanism jointly act on the index sending rate, forming a double-loop adjustment mechanism for adjusting the index sending rate. According to the double-loop index sending rate formed by the double-loop adjustment mechanism, the database monitoring index in the MIB tree is sent to the monitoring server. By taking the cache queue length as the core of the system-level feedback, the index sending rate is adaptively adjusted in a pure data-driven manner. The double-loop mechanism combining the inner loop and the outer loop cleverly separates the short-term load response and the long-term network adaptation function, improves the adaptability and stability of the system in the case of burst traffic or network fluctuation, and can effectively avoid data retention in complex network environments such as network congestion or delay aggravation. It also has the characteristics of non-service intrusion, and only needs to adjust the index sending rate by adjusting the parameters calculated by data-driven calculation, without modifying the code.

[0105] Embodiment four

[0106] Figure 4 The structure diagram of the SNMP-based database index monitoring device is described in Embodiment Four of the present application. In this embodiment, the SNMP-based database index monitoring device comprises:

[0107] The SNMP embedded module 810 is configured to automatically load the configuration file and initialize the SNMP engine and build the MIB tree structure through the ApplicationListener event listening mechanism when the Agent process is started.

[0108] The OID dynamic expansion module 820 is configured to receive the monitored database index and map the monitored database index into the expansion field of the OID according to the preset mapping rule to form the expansion OID, associate the expansion OID with the monitored database index, and register the expansion OID to the memory MIB tree.

[0109] The index sending rate control module 830 is configured to adjust the index sending rate based on the PID control algorithm according to the inner loop control mechanism established based on the length of the cache queue and the outer loop control mechanism established based on the network state parameter.

[0110] The monitoring index sending module 840 is configured to send the database monitoring index in the MIB tree to the monitoring server according to the double-loop index sending rate.

[0111] The embodiment loads the configuration file and initializes the SNMP engine at the start of the Agent process through the time monitoring mechanism of the SNMP embedded module, embeds the SNMP protocol stack in the Agent process seamlessly, the OID dynamic expansion module maps the received database index to the extended field of the OID to form an extended OID with the starting number of the OID, and registers the extended OID associated with the database index to the MIB tree, the index sending rate control module establishes an inner loop control mechanism based on the PID control algorithm and the length of the cache queue and an outer loop control mechanism based on the network state parameter to form a double-loop index sending rate, and the monitoring index sending module sends the database index in the MIB tree to the monitoring server according to the double-loop index sending rate. Through the deep integration at the process level, the life cycle of the SNMP agent is strictly bound to the Agent, which can avoid the interference of the agent function on the running of the Agent main body; the extended OID formed through the path mapping can dynamically adjust the OID format according to the actual use scene, automatically convert the business index name into a standard number, and automatically complete the OID mapping allocation, MIB tree registration and permission binding processes, thereby saving the tedious operation of manually writing the MIB file or maintaining the OID mapping table; the inner loop control mechanism based on the PID control algorithm and the length of the cache queue and the outer loop control mechanism based on the network state parameter can dynamically adjust the sending frequency from the network load and the double-layer control mechanism, guarantee the timeliness and stability of the database index sending, effectively solve the core problems of the traditional monitoring, such as complicated configuration, difficult expansion and data delay, improve the adaptability of the database monitoring scene through the pure data-driven adaptive adjustment, and reduce or even avoid the risk caused by the code invasion.

[0112] The SNMP-based database index monitoring device provided by the embodiment of the application can execute the SNMP-based database index monitoring method provided by any embodiment of the application, has the function modules and beneficial effects corresponding to the execution method.

[0113] Embodiment five

[0114] Figure 5 A structural diagram of an electronic device according to the fifth embodiment of the application, Figure 5 A block diagram of an exemplary electronic device 12 suitable for implementing embodiments of the application is shown. Figure 5 The electronic device 12 shown is merely one example, and should not be taken as limiting the functionality or applicability of embodiments of the application.

[0115] As shown in Figure 5 The electronic device 12 is shown as a general computing device. Components of the electronic device 12 can include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that connects the various system components, including the system memory 28 and the processing unit 16.

[0116] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures. By way of example, these architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0117] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that is accessible by electronic device 12 and includes both volatile and non-volatile media, removable and non-removable media.

[0118] System memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 5 not shown, is typically provided as residual storage across electronic device 12, and can be used for storing data that is both received as well as data that is generated by the processor 20. Although Figure 5 not shown, can also be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and can also be provided for reading from and writing to a removable, non-volatile optical disk such as a CD-ROM and DVD-ROM disk, respectively. In these instances, each can also be connected to bus 18 by one or more data media interfaces. As will be further depicted and described below, memory 28 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0119] Program / utility 40, having a set (at least one) of program modules 42, can be stored in, for example, system memory 28 by way of example, such as an operating system, one or more application programs, other program modules, and program data, each of which implements at least a portion of a network environment. Program modules 42 generally carry out the functions and / or methodologies of embodiments of the application as described herein.

[0120] The electronic device 12 can also be in communication with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; can also be in communication with one or more devices that enable a user to interact with the electronic device 12 / server / computer; and / or can be in communication with any devices (such as a network card, a modem, etc.) that enable the electronic device 12 to communicate with one or more other computing devices. Such communication can be facilitated by an Input / Output (I / O) interface 22. Still yet, the electronic device 12 can be in communication with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter 20. As Figure 5 illustrated, the network adapter 20 is in communication with the other components of the electronic device 12 through a bus 18. It should be understood that although not shown, other hardware and / or software components that are Figure 5 described in connection with the electronic device 12 can be used in connection with the electronic device 12. These components, some of which are not shown, include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0121] The processing unit 16 can execute a program from the system memory 28 in order to perform various functions as well as to process data, such as implementing the SNMP-based database metric monitoring method provided by embodiments of the present application.

[0122] Embodiment Six

[0123] Embodiment Six of the present application also provides a storage medium containing computer-executable instructions that, when executed by a computer processor, perform the SNMP-based database metric monitoring method provided by the above-described embodiments.

[0124] The computer storage media of embodiments of the present application can be any available media that is accessible by a computer. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, the computer-readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0125] A computer readable signal medium can include a propagated data signal with computer executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that can be involved in

[0126] The code can be transmitted in any coded or modular form, including, but not limited to wireless, wired, optical, etc. or any suitable combination thereof.

[0127] The computer program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In an embodiment, electronic circuitry including, for example, programmable logic circuitry, application specific circuitry, or field programmable gate array (FPGA) circuitry, includes the circuitry required to implement the present application, as well as

[0128] It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the application should, therefore, be determined not with reference to the above description, but instead with reference to the appended claims, along with their full scope of equivalents.< / string>

Claims

1. A method for monitoring database metrics based on SNMP, characterized in that, The application comprises the following steps: S101, automatically loading a configuration file and initializing an SNMP engine and building a MIB tree structure through an ApplicationListener event listening mechanism when an Agent process is started; S102, receiving monitored database indicators, mapping the monitored database indicators into OID extension fields according to preset mapping rules, forming an extension OID according to the OID extension fields, associating the extension OID with the monitored database indicators, and registering the extension OID to a memory MIB tree; S103, based on a PID control algorithm, establishing an inner loop control mechanism of an indicator sending rate according to the length of a cache queue, establishing an outer loop control mechanism of the indicator sending rate based on network state parameters, adjusting the indicator sending rate according to the inner loop control mechanism and the outer loop control mechanism, and forming a double-loop indicator sending rate combining the inner loop and the outer loop; S104, sending the database monitoring indicators in the MIB tree to a monitoring server according to the double-loop indicator sending rate.

2. The method of claim 1, wherein, The S101 comprises the following steps: defining an SNMP instance through code annotation when the Agent process is started by using a Spring container, and defining a calling interface and a message distributor of the SNMP instance in the Agent process; establishing a readable and writable MIB table model based on an OID starting value, creating variable row objects in the table according to the number of the OID starting values, and caching the variable row objects locally and registering the variable row objects to an SNMP server according to the KeyName of each row.

3. The method of claim 1, wherein, The S102 comprises the following steps: extracting metadata of the code annotation by scanning the code annotation, mapping the monitored database indicators into instance fields, indicator fields, grouping fields and attribute fields according to preset mapping rules, and splicing the monitored database indicators with the OID starting values to form an extension OID; associating the extension OID with the monitored database indicators and registering the extension OID to the memory MIB tree.

4. The method of claim 3, wherein, The step of associating the extension OID with the monitored database indicators and registering the extension OID to the memory MIB tree comprises the following steps: creating a MIB entity according to the extension OID, the data type and the access authority of the monitored database indicators by using a dynamic proxy technology, and registering the MIB entity to the memory MIB tree by using a MIB service interface of SNMP4J.

5. The method of claim 1, wherein, The S103 comprises the following steps: based on a PID control algorithm, calculating an indicator sending rate according to the length of a cache queue, generating a target sending rate, and establishing an inner loop control mechanism of the indicator sending rate according to the target sending rate and the length of the cache queue; based on network state parameters, calculating a correction factor according to network delay, packet loss rate and available bandwidth utilization, adjusting the indicator sending rate by using the correction factor, and establishing an outer loop control mechanism of the indicator sending rate; adjusting the indicator sending rate by using the inner loop control mechanism and the outer loop control mechanism to form a double-loop indicator sending rate.

6. The method of claim 5, wherein, The inner loop control mechanism comprises the following steps: based on a PID control algorithm, calculating a proportional term according to the actual real-time queue length, calculating an integral term according to the historical queue length, calculating a differential term according to the historical queue length change rate, and calculating a target sending rate by using the proportional term, the integral term and the differential term; The queue length adjustment parameter is calculated according to the actual queue length in real time and a preset queue capacity threshold, and the inner loop rate adjustment value is generated according to the target sending rate and the queue length adjustment parameter, so as to form an inner loop control mechanism of the index sending rate.

7. The method of claim 6, wherein, The method further comprises: The PID control algorithm is dynamically optimized by using a particle swarm optimization (PSO) algorithm to form an optimized inner loop control mechanism.

8. A database indicator monitoring device based on SNMP, characterized in that, The method comprises: An SNMP embedded module is configured to automatically load a configuration file and initialize an SNMP engine and build a MIB tree structure when an Agent process is started through an ApplicationListener event monitoring mechanism; An OID dynamic expansion module is configured to receive monitored database indexes and map them into extended fields of OIDs according to preset mapping rules to form extended OIDs, associate the extended OIDs with the monitored database indexes, and register the extended OIDs to a memory MIB tree; An index sending rate control module is configured to adjust the index sending rate based on an inner loop control mechanism established by the PID control algorithm according to the length of a cache queue and an outer loop control mechanism established based on network state parameters; An index sending module is configured to send the database monitoring indexes in the MIB tree to a monitoring server according to the double-loop index sending rate.

9. An electronic device, comprising: The electronic device comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the SNMP-based database index monitoring method as claimed in any one of claims 1-7. 10.A storage medium containing computer executable instructions for performing the SNMP-based database index monitoring method as claimed in any one of claims 1-7 when executed by a computer processor.

Images