Database index monitoring method, device and equipment based on SNMP (Simple Network Management Protocol) and medium

Abstract

The invention discloses a database index monitoring method, device and equipment based on an SNMP (Simple Network Management Protocol) and a medium, and relates to the technical field of database monitoring, and the method comprises the following steps: automatically loading and initializing the SNMP when an Agent process is started; receiving a monitored database index and mapping the monitored database index to form an extended OID; and forming an inner ring control mechanism based on a PID control algorithm and the length of the cache queue, forming an outer ring control mechanism based on the network state parameters, and adjusting the index sending rate according to the double-ring index sending rate to send the database monitoring index. An SNMP is embedded in an Agent process, the life cycle of the SNMP is bound with the Agent, and then database indexes are mapped to form an extended OID which can be dynamically extended. The double-loop control mechanism dynamically adjusts the sending frequency from the network load and the adaptive double-loop control mechanism, and the problems of tedious configuration, difficult expansion and data delay in traditional monitoring are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database monitoring, and in particular to a database index monitoring method and device based on SNMP, equipment and medium. BACKGROUND

[0002] With the deep development of information technology, the data storage and management scale of enterprises and organizations grows exponentially, and the monitoring of database and system index data has become a core technology practice to ensure the stability and performance of data infrastructure. As the nerve center of digital operation, the continuous and healthy operation of various databases and systems is directly related to business continuity and service quality. Real-time monitoring and periodic maintenance of key indicators (such as CPU utilization, disk I / O, transaction processing efficiency, etc.) can not only provide early warning of potential risks, but also optimize resource allocation through performance trend analysis, laying a technical foundation for the safe and efficient operation of data assets in the digital transformation of enterprises.

[0003] In the actual application scene of monitoring indicators, the traditional monitoring method based on SNMP protocol faces multiple technical bottlenecks. First, in the complexity of the configuration link, SNMP agent deployment needs to manually complete port opening, OID encoding mapping and permission authentication, etc. operations, which consumes manpower and material resources, and the long-term exposed port is more likely to become a breakthrough for OID traversal attack, unauthorized access and other security threats; second, the index extensibility is insufficient, when the business demand changes and new or modified monitoring indicators are needed, the monitoring component code needs to be modified and the service needs to be restarted; in addition, the timeliness of the data is not good, when using fixed polling interval combined with passive caching strategy, all data indicators are cached based on the same polling interval at the same time, which cannot distinguish between key indicators such as database connection number and ordinary indicators, resulting in delay in abnormal early warning of core indicators, and lack of dynamic adjustment mechanism in network congestion; finally, the real-time performance, flexibility, etc. of database index monitoring cannot meet the growing monitoring needs of enterprises. SUMMARY

[0004] The embodiments of the present application provide a database index monitoring method, device, equipment and medium based on SNMP to solve the technical problems of insufficient flexibility and real-time performance of database index monitoring.

[0005] In a first aspect, the embodiments of the present application provide a database index monitoring method based on SNMP, comprising: S101, through the ApplicationListener event listening mechanism, automatically loading the configuration file and initializing the SNMP engine when the Agent process starts, and constructing the MIB tree structure; S102, receive the monitored database index, map the monitored database index to an OID extension field according to a preset mapping rule, form an extended OID according to the OID extension field, associate the extended OID with the monitored database index, and register the extended OID to a memory MIB tree; S103, based on a PID control algorithm, establish an inner loop control mechanism of the index sending rate according to the length of the cache queue, establish an outer loop control mechanism of the index sending rate based on network state parameters, adjust the index sending rate according to the inner loop control mechanism and the outer loop control mechanism, and form a double-loop index sending rate combining the inner loop and the outer loop; S104, send the database monitoring index in the MIB tree to a monitoring server according to the double-loop index sending rate.

[0006] Further, the S101 comprises: Using a Spring container, defining an SNMP instance by code annotation when an Agent process is started, and defining a calling interface and a message distributor of the SNMP instance in the Agent process; Based on an OID starting value, establishing a readable and writable MIB table model, creating a variable row object in the table according to the number of OID starting values, and caching the variable row object locally and registering the variable row object to an SNMP server according to the KeyName of each row.

[0007] Further, the S102 comprises: By scanning the code annotation, extracting metadata of the code annotation, mapping the monitored database index to an instance field, an index field, a grouping field, and an attribute field according to a preset mapping rule, and splicing the monitored database index with an OID starting value to form an extended OID; Associate the extended OID with the monitored database index and register the extended OID to a memory MIB tree.

[0008] Further, the S102 comprises: Using dynamic proxy technology, creating a MIB entity according to the extended OID, the data type of the monitored database index, and access permission, and registering the MIB entity to the memory MIB tree by using a MIB service interface of SNMP4J.

[0009] Further, the S103 comprises: Based on a PID control algorithm, calculating the index sending rate according to the length of the cache queue, generating a target sending rate, and establishing an inner loop control mechanism of the index sending rate according to the target sending rate and the length of the cache queue; Based on network state parameters, according to network delay, packet loss rate and available bandwidth utilization, a correction factor is calculated, the index sending rate is adjusted by using the correction factor, and an outer loop control mechanism of the index sending rate is established; The inner loop control mechanism and the outer loop control mechanism are used to adjust the index sending rate simultaneously, and a double-loop index sending rate is formed.

[0010] Further, the inner loop adjustment mechanism comprises: Based on a PID control algorithm, a proportional term is calculated according to the current actual queue length, an integral term is calculated according to the historical queue length, and a differential term is calculated according to the historical queue length change rate, and the target sending rate is calculated by using the proportional term, the integral term and the differential term; According to the current actual queue length and the preset queue capacity threshold, a queue length adjustment parameter is calculated, an inner loop rate adjustment value is generated according to the target sending rate and the queue length adjustment parameter, and an inner loop control mechanism of the index sending rate is formed.

[0011] Further, the method further comprises: A particle swarm optimization algorithm PSO is used to dynamically optimize the PID control algorithm, and an optimized inner loop control mechanism is formed.

[0012] In a second aspect, an embodiment of the present application provides a database index monitoring device based on SNMP, comprising: An SNMP embedded module is used to automatically load a configuration file and initialize an SNMP engine and build a MIB tree structure through an ApplicationListener event listening mechanism when an Agent process is started; An OID dynamic expansion module is used to receive a monitored database index and map it into an expansion field of an OID according to a preset mapping rule to form an expansion OID, associate the monitored database index, and register it to a memory MIB tree; An index sending rate control module is used to establish an inner loop control mechanism based on a PID control algorithm according to the length of a cache queue, and form a double-loop index sending rate with an outer loop control mechanism based on network state parameters to adjust the index sending rate; A monitoring index sending module is used to send the database monitoring index in the MIB tree to a monitoring server according to the double-loop index sending rate.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising: One or more processors; A storage device is used to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned SNMP-based database index monitoring method.

[0014] In a fourth aspect, an embodiment of the present application provides a storage medium containing computer executable instructions for executing the above-mentioned SNMP-based database index monitoring method when executed by a computer processor.

[0015] The SNMP-based database index monitoring method, device, equipment and medium provided by the embodiment of the present application embed the SNMP protocol stack in the form of a native component in the Agent process by using the Spring container, and integrate the SNMP interface in the Agent process by using dependency injection; the application program listens to events, automatically loads the configuration file and initializes the SNMP engine when the Agent process starts, obtains the database index through the integrated interface and generates an instance, and then constructs the MIB tree. According to the preset mapping rule, the business-level database index is mapped to the extension field of the OID to form an extended OID that can be dynamically extended, and then the extended OID is associated with the database index and registered to the MIB tree. According to the length of the cache queue, an inner loop control mechanism is formed, and together with the outer loop control mechanism formed by the network state parameter, a double-loop control mechanism is formed to adjust the index sending rate of the database index. Through the deep integration at the process level, the life cycle of the SNMP agent is strictly bound to the Agent, which can avoid the interference of the agent function on the running of the Agent main body; the extended OID formed by the path mapping can dynamically adjust the OID format according to the actual use scene, automatically convert the business index name into a standard number, and can automatically complete the OID mapping allocation, MIB tree registration and permission binding processes, etc., saving the tedious operation of manually writing MIB files or maintaining the OID mapping table; the inner loop control mechanism based on the PID control algorithm and the cache queue length and the outer loop control mechanism of the network state parameter can dynamically adjust the sending frequency from the double-layer control mechanism of the network load and adaptation, guarantee the timeliness and stability of the database index sending, effectively solve the core problems of complex configuration, difficult expansion and data delay in traditional monitoring, improve the adaptability of the database monitoring scene through pure data-driven adaptive adjustment, and reduce or even avoid the risk caused by code intrusion. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application and are incorporated herein for a purpose of explanations. The illustrative embodiments of the present application, together with their description, are employed to explain the application in detail. Figure 1 A flowchart of an SNMP-based database index monitoring method according to the first embodiment of the present application; Figure 2 A flow chart of a database index monitoring method based on SNMP according to Embodiment Two of the present application; Figure 3 A flow chart of a database index monitoring method based on SNMP according to Embodiment Three of the present application; Figure 4 A structural schematic diagram of a database index monitoring device based on SNMP according to Embodiment Four of the present application; Figure 5 A structural diagram of an electronic device according to Embodiment Five of the present application. DETAILED DESCRIPTION

[0017] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0018] Database indexes are various system state parameters of a database system during runtime, and through monitoring the database indexes, the working state of the database can be better mastered, and potential threats or abnormal states of the database can be warned. In a traditional SNMP monitoring scenario, there are problems of complicated configuration and complex port management, and there also exist problems that the traditional agent long-term independently listens to a fixed port, and is extremely easy to become a target of port scanning and DDoS attack. Especially in the face of a database monitoring scenario of multiple instances and multiple indexes, the poor expansibility of the data indexes becomes a core problem of limiting business dynamic adjustment. When a database index to be monitored needs to be added or modified, not only the SNMP agent configuration file needs to be manually modified, the index collection logic needs to be redefined, but also the MIB file needs to be written or adjusted, the OID needs to be allocated for the new index and registered in the MIB tree, and the service needs to be restarted each time the index is changed, which is complicated in operation and poor in business adaptability. At the same time, there are key indexes such as the number of database connections in the database indexes, and if the polling or polling cache combined sending scheme is used for the key indexes and other ordinary indexes, the abnormal early warning of the core indexes is likely to lag, and there also lacks a network congestion dynamic adjustment mechanism, and once the bandwidth fluctuates, data accumulation or packet loss will be caused, and there are deficiencies in the importance difference of the indexes and the network environment self-adaptability. Therefore, a database index monitoring method with safety, strong expansibility and strong adaptability is needed.

[0019] Embodiment One Figure 1The flow chart of the database index monitoring method based on SNMP according to the first embodiment of the present application is shown in the figure, wherein the SNMP is embedded in the Agent process, and the extensible dynamic OID is automatically generated according to the preset rule and mapped to the MIB tree, and finally the index sending rate is adjusted according to the buffer queue length and the network environment parameters, and the specific steps include the following steps: S101, through the ApplicationListener event listening mechanism, the configuration file is automatically loaded and the SNMP engine is initialized when the Agent process is started, and the MIB tree structure is constructed.

[0020] Through the deep integration of the process level, the SNMP protocol stack is embedded in the Agent process in the form of a native component by using the Spring container, the SNMP protocol stack is seamlessly integrated with the Agent process by dependency injection, so that the SNMP agent becomes an integral part of the process. When the Agent process is started, the ApplicationListener application event listening mechanism is used to automatically load the configuration file and initialize the SNMP engine, the Agent process can call the SNMP protocol to collect database indexes and generate instances through the integrated interface, and then the MIB tree is constructed. When the Agent process is stopped, the resource release is completed synchronously, the life cycle of the SNMP agent is strictly bound with the Agent, and the interference of the agent function on the Agent main body operation can be avoided. At the same time, the MIB tree structure is constructed according to the index content and quantity to be monitored, which is used to store the database index objects to be monitored and sent to the monitoring server in the subsequent process.

[0021] S102, receiving the monitored database indexes, mapping the monitored database indexes to the OID extension field according to the preset mapping rule, and forming the extended OID according to the OID extension field, associating the extended OID with the monitored database indexes and registering them to the memory MIB tree.

[0022] Through the path mapping technology, the business-level database index (for example, index name) is automatically mapped to extended fields such as instance number, index number, etc. The extended OID is formed on the basis of the OID starting number, so that the OID can automatically adapt to the database index monitoring requirements of multiple scenarios and multiple instances. After the OID is associated with the monitored database index, it is registered to the memory MIB tree. Then, the instances in the MIB tree can be sent to the index monitoring server through the cache queue, and the potential threats and abnormal risks of the database are warned. The tedious operation of manually writing MIB files or maintaining OID mapping tables is omitted. In addition, the YAML / JSON declarative configuration file can be combined. The user only needs to declare the index name, collection frequency and other business parameters in the configuration, and the system can automatically complete the OID allocation, MIB tree registration and permission binding whole process. The configuration process of the new monitoring index is simplified to pure business parameter filling.

[0023] In S103, based on the PID control algorithm, an inner loop control mechanism of the index sending rate is established according to the length of the cache queue, an outer loop control mechanism of the index sending rate is established based on network state parameters, the index sending rate is adjusted according to the inner loop control mechanism and the outer loop control mechanism at the same time, and a double-loop index sending rate combining the inner loop and the outer loop is formed.

[0024] Based on the PID control algorithm, the length of the cache queue is taken as a real-time feedback signal to accurately calculate and adjust the index sending rate. In addition, the length of the cache queue is used to further adjust the index sending rate, forming an inner loop control mechanism of the index sending rate. Meanwhile, based on the real-time collected network state parameters (such as RTT, packet loss rate, available bandwidth), a rate correction factor is calculated, forming an outer loop control mechanism. On the basis of the inner loop control mechanism, secondary adjustment is performed, taking into account short-term load response and long-term network adaptation, which can effectively avoid network congestion or delay aggravation. Through pure data-driven adaptive adjustment, the sending timeliness of the database index is guaranteed, and it can be widely used in various database monitoring scenarios.

[0025] In S104, according to the double-loop index sending rate, the database monitoring index in the MIB tree is sent to the monitoring server.

[0026] According to the double-loop index sending rate formed by the length of the cache queue and the network environment, the database monitoring index instance associated with the OID in the MIB tree is sent to the monitoring server. The monitoring server can identify the content of the database monitoring index according to the OID, and then analyze the database state according to the identified content to detect the abnormal state of the database. In addition, potential risks can be warned according to the change of the index.

[0027] The embodiment embeds the SNMP protocol stack in the form of a native component in the Agent process by using the Spring container, and integrates the SNMP interface in the Agent process by using dependency injection; the application program listens to events, automatically loads the configuration file and initializes the SNMP engine when the Agent process starts, obtains the database indicators through the integrated interface and generates instances, and then constructs the MIB tree. According to the preset mapping rule, the business-level database indicators are mapped into the extension field of the OID to form an extended OID that can be dynamically extended, and then the extended OID is associated with the database indicators and registered in the MIB tree. According to the length of the cache queue, an inner ring control mechanism is formed, and together with the outer ring control mechanism formed by the network state parameters, a double ring control mechanism is formed to adjust the indicator sending rate of the database indicators. Through the deep integration at the process level, the life cycle of the SNMP agent is strictly bound to the Agent, which can avoid the interference of the agent function on the running of the Agent main body; the extended OID formed through path mapping can dynamically adjust the OID format according to the actual use scene, automatically convert the business indicator name into a standard number, and can automatically complete the OID mapping allocation, MIB tree registration and permission binding processes, thereby saving the tedious operation of manually writing MIB files or maintaining the OID mapping table; the inner ring control mechanism based on the PID control algorithm and the length of the cache queue and the outer ring control mechanism based on the network state parameters can dynamically adjust the sending frequency from the double-layer control mechanism of the network load and adaptation, guarantee the timeliness and stability of the database indicator sending, effectively solve the core problems of complex configuration, difficult expansion and data delay in traditional monitoring, improve the adaptability of the database monitoring scene through pure data-driven adaptive adjustment, and reduce or even avoid the risk caused by code intrusion.

[0028] Embodiment Two Figure 2 A flowchart of a database indicator monitoring method based on SNMP according to Embodiment Two of the present application, which is optimized on the basis of the above-mentioned embodiment. In this embodiment, S101 is specifically optimized as follows: Using the Spring container, the SNMP instance is defined through code annotation when the Agent process starts, and the calling interface and message distributor of the SNMP instance are defined in the Agent process. Based on the OID starting value, a readable and writable MIB table model is established, variable row objects are created in the table according to the number of OID starting values, and the variable row objects are cached locally and registered in the SNMP server according to the KeyName of each row.

[0029] Correspondingly, the database indicator monitoring method based on SNMP provided in the embodiment specifically comprises the following steps: S201, using Spring container, defining SNMP instance by code annotation when Agent process starts, and defining calling interface and message dispatcher of SNMP instance in Agent process.

[0030] Using the automatic assembly mechanism of Spring Boot of Spring container, defining SNMP instance, TransportMapping transmission mapping interface of Agent process calling SNMP, and MessageDispatcher message dispatcher of database index allocation to instance in Agent process by using @Configuration and @Bean annotation when Agent process starts. By reading annotation and automatic loading when Agent process starts, seamless integration of SNMP protocol stack in Agent process is realized, and corresponding functional modules are automatically created and initialized when process starts, which tightly binds SNMP agent service with life cycle of Agent process, isolates with external environment, starts with Agent and terminates with Agent.

[0031] S202, based on OID starting value, establishing readable and writable MIB table model, creating variable row object in table according to OID starting value number, and caching variable row object in local according to KeyName of each row and registering to SNMP server.

[0032] First, a readable and writable MIB table model for storing OID is established, MOTable and MOMutableColumn are used by SnmpAgent to establish based on OID starting value (for example, 1.3.6.1.4.1.9999), then DefaultMOMutableRow2PC is used by SnmpAgentProxy to create variable row object in table, the variable row object is used to store OID and index value of monitoring index, the variable row object is a row object whose field number and field content can be changed, each row is used to store OID and specific index value of one database index. Then, cache to local according to KeyName of each row, and call registerSnmpAgent() function to register all cached MIB variable row objects to SNMP server. By converting business data (such as SnmpData) into mapping relationship with OID, and dynamically registering MIB object to SNMP service through cache mechanism, automatic construction and management of runtime MIB tree are realized. By using doWrite(Map<String, List <string>The method realizes the support capability of the automatic expansion mechanism of the row object, especially when the number of the database indexes to be monitored after the addition of the database indexes to be monitored or the switching of the monitoring scene is greater than the current target row, the new variable row object can be dynamically created to be inserted into the MIB table and the cache, so as to ensure the consistency of the data.

[0033] In addition, in the security layer, the system comprehensively supports the SNMPv2 and SNMPv3 protocols, including the USM user authentication (SHA / MD5) and encryption (DES / AES and its extension), multiple users with different security policies are added by rewriting the addUsmUser(USM usm) method, and the authentication password, encryption algorithm and key are specified; at the same time, the VACM model is combined to realize fine-grained access control, multiple access groups are created by the addViews(VacmMIB vacm) method, and the corresponding OID tree view permissions (read / write / notify) are allocated to them, and the access range of each node is accurately controlled. In the network and isolation design, the system initializes the UDP transport mapping through the initTransportMappings() function, the default listening address is read from the custom configuration file by the constructor, the custom configuration file is a network environment configuration file defined according to the current transmission situation, and dynamic port binding is realized in combination with the SNMPv3 security model. The SNMP4J underlying implementation can also be encapsulated in SnmpAgentProxy by class loading isolation, only exposing necessary operation interfaces (such as register(SnmpData snmpData), write(SnmpData snmpData), etc.) to the outside, and hiding the underlying implementation details of the SNMP protocol stack; through the thread pool isolation mechanism, the ScheduledExecutorService independent thread pool set by SnmpAgent during initialization handles the proxy tasks, limits the concurrent resources, and prevents service crashes caused by high concurrency; and through the Cache cache management strategy, the MIB object is prevented from being created and destroyed frequently, and the cache information is prevented from being obtained by reflection and other methods from the outside, which improves the performance and enhances the context isolation capability. The life cycle of the Agent service is driven by the initSnmpAgent() function, which completes the SNMP engine initialization, configuration loading, context registration, service startup and other steps in turn, and all operations are recorded through the log to record the key events, which is convenient for subsequent debugging and operation and maintenance.

[0034] S203, receiving the monitored database indexes, mapping the monitored database indexes into OID extension fields according to a preset mapping rule, forming an extended OID according to the OID extension fields, associating the extended OID with the monitored database indexes, and registering the extended OID to the memory MIB tree.

[0035] Specifically, by scanning the code annotation, the metadata of the code annotation is extracted, the monitored database index is mapped into instance fields, index fields, grouping fields and attribute fields by using a preset mapping rule, and is spliced with an OID starting value to form an extended OID.

[0036] For the scalability problem of the monitored database indicators, the path mapping between the indicator name and OID of the database indicators is performed through the syntax parser. The mapping relationship can be cached through ConcurrentHashMap to support high-frequency queries. The standard OID prefix of the database indicators is used as the starting number (for example, 1.3.6.1.4.1.9999.1), and the hierarchical mapping rule including the "starting number, instance number, indicator number, grouping number, attribute number" is established to avoid OID allocation confusion and hierarchical relationship ambiguity, so that the newly added database indicators can quickly integrate into the existing monitoring system. Through the dynamic binding of the custom mapping field of the Agent configuration file and the collector port number (for example, port 10000 is mapped to 100, corresponding to the OID segment 1.3.6.1.4.1.9999.1.100), the standardized indicator name (such as "GBase8s_Er_Cluster_Server") is associated with the exclusive indicator number (such as 1) to form the extended OID in the form of 1.3.6.1.4.1.9999.1.100.1.1, 1.3.6.1.4.1.9999.1.100.1.2, which can dynamically expand the field. For example, the ConfigParse parser is used to parse the database indicator configuration, the path of the YAML configuration file snmp-config.yml is obtained from AppConfigProperties based on Spring dependency injection, the mapping relationship between the database indicator number and the database instance number under the database instanceNumbers node (format: List<Map<String, Integer>>) is parsed using the SnakeYAML library, and the YAML configuration file is used to query which database indicators are monitored under each database instance. The existence verification of the configuration stream, the data type forced verification and the default processing of the null value are also included in the parsing process, and finally the mapping list of the indicator number and the instance number of each indicator is generated. Through the annotation-driven mode, the collection class annotated by the developer through the @MetricDefinition annotation (such as @MetricDefinition(metricName="GBase8s_Er_Cluster_Server")) is recognized, the BeanPostProcessor mechanism of the Spring container can automatically scan and extract the metadata of the annotation in the tree pooling stage of the Agent process, and the metadata is associated with the above generated indicator number and instance number to generate the extended field of the OID, form the instance number, indicator number, grouping number and attribute number of each database indicator, and splice with the starting number of the OID, and finally generate the complete extended OID and register it to the MetricRegistry.In the dynamic generation process of OID, manual configuration or service restart is no longer needed, and the operation of extending the index is simplified from the "code development level" to the "configuration declaration level", and only the mapping relationship between the database instance and the database index needs to be declared in the YAML configuration file.

[0037] Using dynamic proxy technology, MIB entities are created according to the extended OID, the data type and access permission of the monitored database index, and the MIB entities are registered to the memory MIB tree using the MIB service interface of SNMP4J.

[0038] The SnmpMibProxy proxy class for managing the MIB tree is generated by dynamic proxy technology, and the node entity MibNode of the MIB tree containing the extended OID, the index data type and the index access permission is created according to the metadata of the database index. The variable row object in the MIB table is inserted into the node entity of the MIB tree, and then the MIB entity is registered to the memory MIB tree using the MibServer interface of SNMP4J. When binding the permission of the node entity in the MIB tree, the RBAC policy can be integrated to associate the SNMPv2 and SNMPv3 user groups according to the identification number of the database instance, so that the MIB tree supports configuration hot update and incremental MIB loading, and manual maintenance of the OID mapping table is no longer needed. Hot update of the monitored index can be realized by configuration driving and automatic parsing, and the business department can independently extend the database index dimension to be monitored, which ensures the consistency and security of the index monitoring in the multi-instance and multi-index scenario, and reduces the system maintenance cost.

[0039] S204, based on the PID control algorithm, an inner loop control mechanism of the index sending rate is established according to the length of the cache queue, an outer loop control mechanism of the index sending rate is established based on the network state parameter, the index sending rate is adjusted according to the inner loop control mechanism and the outer loop control mechanism, and a double-loop index sending rate combining the inner loop and the outer loop is formed.

[0040] S205, according to the double-loop index sending rate, the database monitoring index in the MIB tree is sent to the monitoring server.

[0041] The embodiment embeds the SNMP protocol stack in the form of a native component into the agent through process-level deep integration and dependency injection, strictly binds the life cycle of the SNMP agent to the agent, maps the database indicators at the service level to the extension fields of the OIDs through path mapping, splices the extended OIDs with the starting number of the standard OIDs to form the extended OIDs, dynamically extends the database indicators as required in the actual application scene, no longer needs to manually write MIB files or maintain the OID mapping table, and automatically completes the OID allocation and MIB tree registration in combination with the YAML declaration configuration file, shields the details of the SNMP underlying protocol, and especially when the database indicators that need to be monitored are added, only the business parameters need to be filled in, thereby shortening the online cycle of the added monitoring indicators.

[0042] Embodiment three Figure 3 A flowchart of the SNMP-based database indicator monitoring method described in Embodiment Three of the application, the embodiment is optimized on the basis of the above-mentioned embodiment, in this embodiment, S103 is specifically optimized as: Based on the PID control algorithm, the target sending rate is generated by calculating the indicator sending rate according to the length of the cache queue, and the inner loop control mechanism of the indicator sending rate is established according to the target sending rate and the length of the cache queue. Based on the network state parameters, the correction factor is calculated according to the network delay, the packet loss rate and the available bandwidth utilization, the indicator sending rate is adjusted by using the correction factor, and the outer loop control mechanism of the indicator sending rate is established. The inner loop control mechanism and the outer loop control mechanism are used to adjust the indicator sending rate at the same time, and a double-loop indicator sending rate is formed.

[0043] Correspondingly, the SNMP-based database indicator monitoring method provided by the embodiment specifically comprises: S301, automatically loading a configuration file and initializing an SNMP engine when an Agent process is started, and constructing a MIB tree structure through an ApplicationListener event listening mechanism.

[0044] S302, receiving a monitored database indicator, mapping the monitored database indicator to an OID extension field according to a preset mapping rule, forming an extended OID according to the OID extension field, associating the extended OID with the monitored database indicator, and registering the extended OID to a memory MIB tree.

[0045] S303, based on the PID control algorithm, the target sending rate is generated by calculating the indicator sending rate according to the length of the cache queue, and the inner loop control mechanism of the indicator sending rate is established according to the target sending rate and the length of the cache queue.

[0046] To precisely control the transmission rhythm of indicator data, a classic PID control algorithm is used. Based on the length of the buffer queue as a reference value, the proportional term, integral term, and micro-sharing are calculated separately and combined to generate the target transmission rate. Then, based on the length of the buffer queue, it is determined whether the buffer queue exceeds the preset limit. Together with the target transmission rate, they form an inner-loop control mechanism for real-time adjustment of the indicator transmission rate.

[0047] One optional implementation of this embodiment is that the inner loop control mechanism includes: Based on the PID control algorithm, the proportional term is calculated according to the current real-time actual queue length, the integral term is calculated according to the historical queue length, and the derivative term is calculated according to the historical queue length change rate. The target transmission rate is then calculated using the proportional term, integral term, and derivative term.

[0048] The classic PID control algorithm consists of three core parts: a proportional term (P), an integral term (I), and a derivative term (D). The proportional term adjusts the output proportionally to the current error, providing a fast response. Its calculation formula is: ,in This is the proportionality coefficient, which determines the strength of the system's response to the current error. express The deviation between the actual queue length and the target value at any given time is used for rapid response to instantaneous fluctuations; the integral term accumulates and sums historical errors to eliminate the system's steady-state error, and the calculation formula is as follows. ,in These are integral coefficients used to eliminate steady-state errors. Indicates time τ The error value at which, τ It is an integral dummy variable, representing the time from the initial time 0 to the current time. τ Any past moment between; It's about time. τ The small increment indicates that the integral is a continuous accumulation of the error over time; the differential term makes advance predictions and adjustments based on the rate of change of the error, suppressing rapid changes in the error in advance, as shown in the formula: ,in The differential coefficients are... In an infinitesimal time interval Within, error The minute changes Representing an infinitesimal increment in time, it is used to define the limiting process of the derivative, helping to suppress overshoot and predict trend changes. The formula for calculating the target transmission rate is:

[0049] in, Indicates the target transmission rate. Indicates time.

[0050] The queue length adjustment parameter is calculated based on the current real-time actual queue length and the preset queue capacity threshold. The inner loop rate adjustment value is generated based on the target transmission rate and the queue length adjustment parameter, forming an inner loop control mechanism for the target transmission rate.

[0051] Furthermore, based on preset thresholds and real-time queue length, it can be determined whether the real-time cache queue length is too long or too short, and the sending rate can be further adjusted. A LinkedBlockingQueue queue can be used in the kernel cache to monitor the queue length in real time. The preset thresholds refer to the balance between data generation and sending rates in historical data. For example, an upper threshold H (typically 70% of the queue capacity) and a lower threshold L (typically 30% of the queue capacity) can be set. When the queue length > H, the data sending rate increases to 1.5 times the normal rate; when the queue length < L, the sending rate decreases to 0.6 times the normal rate; when it is between H and L, the normal rate is maintained. This allows for further adjustment of the target sending rate based on the target sending rate, forming an inner-loop control mechanism for the target sending rate.

[0052] Optionally, the particle swarm optimization algorithm (PSO) can be used to dynamically optimize the PID control algorithm, forming an optimized inner-loop control mechanism.

[0053] To improve the adaptability of PID in complex scenarios, the system further employs the particle swarm optimization (PSO) algorithm. Dynamic optimization is performed. These three parameters are used as the particle's three-dimensional spatial position, and the velocity and position are updated in each iteration using the following formula:

[0054]

[0055] in, Inertial weights are used to control the balance between particle exploration and development. Indicates the first Particles in time The speed of the particle determines its direction and distance of movement in the search space; These are learning factors, representing the influence of individual experience and group experience, respectively. Indicates the first Particles in time The position is the particle's coordinate in the search space, corresponding to the parameters of the PID controller. ; A random number within the interval [0,1]; For particles i the individual optimal solution; The global optimal solution for the entire population. The PSO optimization algorithm evaluates the effect of each set of parameters by defining a fitness function (such as minimizing the mean square error MSE or maximizing throughput stability), and continuously iterates and updates during the running process, finally converging to the optimal parameter combination under the current environment; the iteration process can be executed regularly in the background, or triggered according to the system state, so as to realize the real-time adaptive adjustment of the PID parameters, and significantly improve the response ability and stability of the system under the condition of burst traffic or network fluctuation.

[0056] S304, based on the network state parameters, according to the network delay, packet loss rate and available bandwidth utilization, a correction factor is calculated, and the index sending rate is adjusted by using the correction factor, and an outer loop control mechanism of the index sending rate is established.

[0057] By collecting network state parameters, the index sending rate is adjusted according to the state of the transmission network. By sending a probe packet to the remote host every 500 milliseconds, and recording the sending and receiving time stamps and the response state, the key performance indicators of the current network are calculated: RTT (round trip time), packet loss rate and available bandwidth utilization. When it is detected that the packet loss rate exceeds 5% or the bandwidth utilization is higher than 80%, the system determines that the current network is in a congested or unstable state, and the rate adjustment mechanism will be triggered to avoid packet loss or retransmission problems caused by too fast data sending. For example, based on the weighted moving average algorithm, the collected network indicators are smoothed, and the RTT, packet loss rate and bandwidth utilization collected in the last n times are taken as input data. The weighted average value of the packet loss rate is The calculation formula is:

[0058] wherein, , , The packet loss rate collected in the last three times is represented by , , The weight is represented by . Then the weighted average value calculated is compared with the preset threshold to generate the rate correction factor of the packet loss rate If the weighted packet loss rate exceeds 5%, it means that there is a significant packet loss risk in the network, so the rate correction factor of the packet loss rate is set to 0.5; if the bandwidth utilization is higher than 80%, it means that the network is close to saturation, so the rate correction factor of the bandwidth utilization is set to 0.7; in other cases, the rate correction factor is kept, that is, the rate is not adjusted. The index sending rate is adjusted by the correction factor to form an outer loop control mechanism of the index sending rate.

[0059] S305, the index sending rate is adjusted by the inner loop control mechanism and the outer loop control mechanism at the same time to form a double-loop index sending rate.

[0060] The inner loop control mechanism and the outer loop control mechanism are combined with the queue length by the outer loop control mechanism and the PID control algorithm to multiply, the double-loop index sending rate of adjusting the index sending rate by the inner loop control mechanism and the outer loop control mechanism at the same time is realized, the network congestion is realized in advance by continuously sensing the network state and combining the trend analysis of historical data. The prediction and active response not only improve the adaptability of the system in the network fluctuation environment, but also effectively guarantee the stable transmission of the monitoring data in the complex network condition, and further enhance the reliability and robustness of the whole monitoring system.

[0061] S306, according to the double-loop index sending rate, the database monitoring index in the MIB tree is sent to the monitoring server.

[0062] In this embodiment, based on the PID control algorithm, the cache queue length is taken as the feedback signal, the cache queue length is taken as the basic parameter of the control index sending rate, the target sending rate is calculated, and then the target sending rate is adjusted again according to whether the cache queue length exceeds the preset limit, forming an inner loop control mechanism; at the same time, network environment parameters are continuously collected, and a correction factor is calculated according to the network environment, forming an outer loop control mechanism; the inner loop control mechanism and the outer loop control mechanism jointly act on the index sending rate, forming a double-loop adjustment mechanism for adjusting the index sending rate, and the database monitoring index in the MIB tree is sent to the monitoring server according to the double-loop index sending rate formed by the double-loop adjustment mechanism. By taking the cache queue length as the system-level feedback core, the index sending rate is adaptively adjusted in a pure data-driven manner, the short-term load response and the long-term network adaptation function are ingeniously separated by the double-loop mechanism combining the inner loop and the outer loop, the adaptability and stability of the system in the case of burst traffic or network fluctuation are improved, and data retention in complex network environments such as network congestion or delay aggravation can be effectively avoided; it also has the characteristics of non-service intrusion, and only the adjustment parameter calculated by data-driven adjustment is needed to adjust the index sending rate, without modifying the code.

[0063] Embodiment four Figure 4 The structure diagram of the database index monitoring device based on SNMP described in embodiment four of the application, in this embodiment, the database index monitoring device based on SNMP comprises: The SNMP embedded module 810 is used for automatically loading a configuration file and initializing an SNMP engine and building an MIB tree structure through an ApplicationListener event listening mechanism when an Agent process is started. The OID dynamic expansion module 820 is configured to receive the monitored database indexes and map the monitored database indexes into an expansion field of an OID according to a preset mapping rule to form an expansion OID, associate the monitored database indexes with the expansion OID, and register the expansion OID to the memory MIB tree; The index sending rate control module 830 is configured to adjust the index sending rate based on a PID control algorithm and an inner loop control mechanism established according to the length of the cache queue, and an outer loop control mechanism established based on network state parameters to form a double-loop index sending rate. The monitored index sending module 840 is configured to send the database monitored indexes in the MIB tree to the monitoring server according to the double-loop index sending rate.

[0064] In the embodiment, the SNMP embedded module automatically loads a configuration file and initializes an SNMP engine at the start of the Agent process through a time listening mechanism, seamlessly embeds an SNMP protocol stack in the Agent process, the OID dynamic expansion module maps the monitored database indexes into an expansion field of an OID after receiving the monitored database indexes, forms an expansion OID with an OID starting number, and registers the expansion OID to the MIB tree after associating the database indexes with the expansion OID, the index sending rate control module forms a double-loop index sending rate based on a PID control algorithm and an inner loop control mechanism established according to the length of the cache queue, and an outer loop control mechanism established based on network state parameters, and the monitored index sending module sends the database indexes in the MIB tree to the monitoring server according to the double-loop index sending rate. Through deep integration at the process level, the life cycle of the SNMP agent is strictly bound to the Agent, which can avoid interference of the agent function on the running of the Agent main body, the expansion OID formed through path mapping can dynamically adjust the OID format according to the actual use scene, automatically convert the business index name into a standard number, and automatically complete OID mapping allocation, MIB tree registration, and permission binding processes, thereby saving the cumbersome operation of manually writing MIB files or maintaining an OID mapping table, the inner loop control mechanism based on the PID control algorithm and the cache queue length and the outer loop control mechanism based on the network state parameter can dynamically adjust the sending frequency from the network load and the double-layer control mechanism, guarantee the timeliness and stability of the database index sending, effectively solve the core problems of complex configuration, difficult expansion, and data delay in the traditional monitoring, improve the adaptability of the database monitoring scene through pure data-driven adaptive adjustment, and reduce or even avoid the risk caused by code invasion.

[0065] The SNMP-based database index monitoring device provided in the embodiment of the application can execute the SNMP-based database index monitoring method provided in any embodiment of the application, has the function modules and beneficial effects corresponding to the execution method.

[0066] Embodiment five Figure 5 A structural diagram of an electronic device according to Embodiment five of the application, Figure 5 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 5 The electronic device 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0067] like Figure 5 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0068] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0069] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0070] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 As not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0071] Program / utility 40 having a set of program modules 42 can be stored in system memory 28 by way of example, such program modules 42 include an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, which may

[0072] Electronic device 12 can also communicate with one or more external devices 14 such as a keyboard or pointing device, a display 24, etc.; one or more devices that enable a user to interact with electronic device 12; and / or one or more devices (e.g., network card, modem, etc.) that enable electronic device 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 22. Still yet, electronic device 12 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through network adapter 20. As Figure 5 illustrated, network adapter 20 communicates with the other components of electronic device 12 via bus 18. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with electronic device 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc. Figure 5

[0073] Processing unit(s) 16 can execute instructions, for example, for the operating system, the one or more application programs, other program modules, and program data, and perform various other functions as described in the embodiments of the present application.

[0074] Embodiment Six Embodiment Six of the present application also provides a storage medium containing computer executable instructions, which when executed by a computer processor, are used to perform the SNMP-based database metric monitoring method as provided by the above embodiments.

[0075] ​The computer storage media of the embodiments of the present application can adopt any combination of one or more computer-readable media. The computer-readable media can be computer-readable signal media or computer-readable storage media. The computer-readable storage media may, for example, but is not limited to, an electrical, a magnetic, an optical, an electromagnetic, an infrared, or a semiconductor system, device, or apparatus, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage media can be any tangible medium that contains or stores a program in a form that can be used by or in connection with an instruction execution system, apparatus, or device.

[0076] The computer-readable signal media can include a computer-readable program code in a baseband or propagated as a carrier wave in a propagation medium. Such a propagated signal can take a wide variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium that can be used to carry or store computer program code for use by or in connection with an instruction execution system, apparatus, or device.

[0077] Program code embodied on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the above.

[0078] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments of the present application, electronic mail (email) can be utilized as the distrusting mechanism to effectuate exchange of information.

[0079] Note that the above merely describes preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.< / string>

Claims

1. A database metric monitoring method based on SNMP, characterized in that, include: S101 automatically loads the configuration file and initializes the SNMP engine and builds the MIB tree structure when the Agent process starts, through the ApplicationListener event listening mechanism; S102: Receive the monitored database metrics, map the monitored database metrics to OID extension fields according to preset mapping rules, form extended OIDs based on OID extension fields, associate the extended OIDs with the monitored database metrics, and register them to the memory MIB tree. S103, based on PID control algorithm, establishes an inner loop control mechanism for index transmission rate based on the length of the buffer queue, and establishes an outer loop control mechanism for index transmission rate based on network state parameters. The index transmission rate is adjusted simultaneously according to the inner loop control mechanism and the outer loop control mechanism to form a dual-loop index transmission rate that combines the inner and outer loops. S104, based on the dual-ring indicator sending rate, sends the database monitoring indicators in the MIB tree to the monitoring server.

2. The method according to claim 1, characterized in that, S101 includes: Using the Spring container, an SNMP instance is defined via code annotations when the Agent process starts, and the SNMP instance's calling interface and message dispatcher are defined in the Agent process. A readable and writable MIB table model is established based on the OID start value. Variable row objects are created in the table according to the number of OID start values. The variable row objects are cached locally and registered with the SNMP server according to the KeyName of each row.

3. The method according to claim 1, characterized in that, S102 includes: By scanning code annotations, the metadata of the code annotations is extracted. Using preset mapping rules, the monitored database metrics are mapped to instance fields, metric fields, grouping fields, and attribute fields, and then concatenated with the OID start value to form an extended OID. Associate the extended OID with the monitored database metrics and register it to the in-memory MIB tree.

4. The method according to claim 3, characterized in that, The step of associating extended OIDs with monitored database metrics and registering them to the in-memory MIB tree includes: Using dynamic proxy technology, MIB entities are created based on extended OIDs, the data types of monitored database metrics, and access permissions. The MIB entities are then registered to the in-memory MIB tree using the SNMP4J MIB service interface.

5. The method according to claim 1, characterized in that, S103 includes: Based on the PID control algorithm, the index transmission rate is calculated according to the length of the buffer queue, the target transmission rate is generated, and an inner-loop control mechanism for the index transmission rate is established according to the target transmission rate and the length of the buffer queue. Based on network state parameters, a correction factor is calculated according to network latency, packet loss rate and available bandwidth utilization. The correction factor is used to adjust the index transmission rate and establish an outer loop control mechanism for the index transmission rate. The index transmission rate is adjusted simultaneously using both inner-loop and outer-loop control mechanisms to form a dual-loop index transmission rate.

6. The method according to claim 5, characterized in that, The inner loop control mechanism includes: Based on the PID control algorithm, the proportional term is calculated according to the current real-time actual queue length, the integral term is calculated according to the historical queue length, and the derivative term is calculated according to the historical queue length change rate. The target transmission rate is calculated using the proportional term, integral term, and derivative term. The queue length adjustment parameter is calculated based on the current real-time actual queue length and the preset queue capacity threshold. The inner loop rate adjustment value is generated based on the target transmission rate and the queue length adjustment parameter, forming an inner loop control mechanism for the target transmission rate.

7. The method according to claim 6, characterized in that, The method further includes: The particle swarm optimization algorithm (PSO) is used to dynamically optimize the PID control algorithm, forming an optimized inner-loop control mechanism.

8. A database indicator monitoring device based on SNMP, characterized in that, include: The SNMP embedded module is used to automatically load the configuration file, initialize the SNMP engine, and build the MIB tree structure when the Agent process starts, through the ApplicationListener event listening mechanism. The OID dynamic extension module is used to receive the monitored database metrics and map them into extended fields of OID according to the preset mapping rules to form extended OIDs, associate them with the monitored database metrics, and register them to the in-memory MIB tree. The indicator transmission rate control module is used to form a dual-loop indicator transmission rate adjustment mechanism, which is based on the inner loop control mechanism established by the PID control algorithm according to the length of the buffer queue, and the outer loop control mechanism established by the network state parameters. The monitoring metric sending module is used to send the database monitoring metrics in the MIB tree to the monitoring server according to the dual-ring metric sending rate.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the SNMP-based database metric monitoring method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the SNMP-based database metric monitoring method as described in any one of claims 1-7.

Images