Monitoring method and device, electronic equipment, storage medium and program product

Abstract

The application provides a monitoring method and device, electronic equipment, a storage medium and a program product. The method is applied to a lower-level server in a monitoring system. When a plug-in uploaded by a user is obtained, a plug-in table is updated based on the plug-in. If it is confirmed that the plug-in type of the plug-in belongs to a derived object generation type, a derived arrangement table is updated based on the plug-in. The plug-in table is scanned every preset scanning period, so as to identify a new plug-in uploaded in the current scanning period. According to a script file of the new plug-in, registration processing is performed, so as to create a plug-in instance. The plug-in instance is used to obtain index data or derived object data that needs to be collected according to rules defined in the script file, and report the data to a higher-level system. The plug-in table and / or the derived arrangement table are updated based on the plug-in uploaded by the user, and the latest uploaded new plug-in is identified by regularly scanning the plug-in table, and the registration processing of the new plug-in is performed, so that the hot loading of plug-in extension is realized, and service restart is not needed.

Description

Technical Field

[0001] This invention relates to the field of data monitoring technology, and more specifically, to a monitoring method, device, electronic device, storage medium, and program product. Background Technology

[0002] With the expansion of enterprise IT (Information Technology) infrastructure and the widespread adoption of microservice architectures, the demand for monitoring distributed systems is increasing. Current mainstream monitoring systems generally adopt a three-tier architecture of "agent-based data collection + central aggregation + visual alerts." In this architecture, lower-level nodes periodically acquire basic resource metrics such as CPU, memory, disk, and network by deploying lightweight data collection agents, and report the data to the central server via HTTP (Hypertext Transfer Protocol), Kafka, or dedicated protocols. The central server is responsible for data aggregation, storage, and alert determination, and finally, the monitoring center provides a visual interface.

[0003] To support monitoring of heterogeneous components such as databases, middleware, and self-developed services, existing monitoring systems typically provide plugin extension mechanisms. However, these mechanisms require plugins to be deployed and configured before the monitored service is started. This means that if new monitoring targets are added or the collection logic is adjusted, the monitored service process must be restarted, which cannot meet the requirement of "zero downtime operation and maintenance" in a high-availability production environment. Summary of the Invention

[0004] The purpose of this invention is to provide a monitoring method, device, electronic device, storage medium, and program product to improve the problems existing in the prior art.

[0005] The embodiments of the present invention can be implemented as follows: [Invention Solution] Compared with existing technologies, embodiments of the present invention provide a monitoring method, device, electronic device, storage medium, and program product. This method is applied to a lower-level server in a monitoring system. The monitoring system includes a database and an upper-level system that are communicatively connected to the lower-level server. The database includes a plugin table and a derived orchestration table corresponding to the lower-level server. The method is as follows: when a user-uploaded plugin is obtained, the plugin table is updated based on the plugin; if it is confirmed that the plugin type belongs to the derived object generation type, the derived orchestration table is updated based on the plugin; the plugin table is scanned every preset scanning cycle to identify new plugins uploaded in the current scanning cycle; registration processing is performed according to the script file of the new plugin to create a plugin instance; the plugin instance is used to obtain the required indicator data or derived object data according to the rules defined in the script file and report the data to the upper-level system. The present invention updates the plugin table and / or derived orchestration table based on user-uploaded plugins, and identifies the latest uploaded new plugins by periodically scanning the plugin table and performs new plugin registration processing, achieving hot loading of plugin extensions without requiring service restarts. Attached Figure Description

[0006] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0007] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention.

[0008] Figure 2 This is a flowchart illustrating a monitoring method provided in an embodiment of the present invention.

[0009] Figure 3 This is an overall application example diagram provided for an embodiment of the present invention.

[0010] Figure 4 This is a schematic diagram of a monitoring device provided in an embodiment of the present invention.

[0011] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0013] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0014] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0015] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0016] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0017] In existing technologies, to support monitoring of heterogeneous components such as databases, middleware, and self-developed services, existing monitoring systems typically provide plugin extension mechanisms. For example, Zabbix allows users to write Shell / Python scripts as "UserParameters" to execute external commands; Prometheus relies on users to develop independent Exporter processes to expose HTTP metric interfaces; and some commercial APM tools (such as New Relic) provide SDKs for developers to implement tracking. However, the above solutions share the following common limitations: First, plugin extensions lack runtime dynamism. Most systems require plugins to be deployed and configured before service startup. Adding new monitoring targets or adjusting data collection logic necessitates restarting the project or main service process, failing to meet the "zero-downtime operation and maintenance" requirement of high-availability production environments. Even if some systems support hot script reloading (e.g., executing external scripts via scheduled directory scanning), their execution context remains isolated from the main application, preventing them from participating in business logic, sharing memory state, or calling internal services as Spring Beans.

[0018] Secondly, monitoring capabilities are limited to the data acquisition end, and the data link is not programmable. Existing plug-in mechanisms almost entirely focus on "how to acquire data," lacking control over "how to transmit" and "how to consume" data. For example, when the Kafka link is interrupted or the central server is unreachable, users cannot temporarily define alternative transmission channels (such as switching to MQTT or local cache replay); nor can they bypass the central server and directly deliver data of specific monitored objects to designated topics in the monitoring center to reduce latency. This tightly coupled "acquisition-transmission" design severely limits the system's adaptability in complex network environments such as hybrid cloud and edge computing.

[0019] Secondly, there is a lack of ability to logically combine and derive models of monitored objects and metrics. Existing systems typically treat each collected item as an independent atomic metric, failing to support users in orchestrating multiple raw metrics (such as "CPU utilization <80%", "disk remaining >10%", "API latency <500ms") into a composite monitoring object (such as "service health") through custom logic. Even when some platforms provide composite conditions in alarm rules, the result is only a boolean alarm signal, unable to generate new, structured monitoring entities that can be referenced by subsequent links, resulting in the inability to embed business semantics into the monitoring system.

[0020] Finally, dynamic plugins suffer from a lack of security and manageability. While allowing users to upload arbitrary scripts increases flexibility, it easily introduces security risks (such as executing system commands or deleting files) or stability issues (such as memory leaks or infinite loops). Existing solutions generally lack sandbox isolation, access control, dependency version management, and circuit breaker mechanisms, making them difficult to implement securely in enterprise environments. Furthermore, operational capabilities such as canary releases, performance monitoring, and version rollbacks for plugins are virtually nonexistent.

[0021] Based on the discovery of the aforementioned technical problems, the inventors, through creative labor, proposed the following technical solutions to solve or improve these problems. It should be noted that the deficiencies in the solutions of the prior art are all results derived by the inventors after practical experience and careful research. Therefore, the discovery process of the aforementioned problems and the solutions proposed in the embodiments of this application below should be considered contributions made by the inventors to this application during the inventive process, and should not be construed as technical content known to those skilled in the art.

[0022] The application scenarios of this invention will be introduced first.

[0023] Please see Figure 1 The monitoring system includes multiple subordinate servers and a superior system connected to each subordinate server. The superior system includes a central server and a monitoring center. Each subordinate server, central server, and monitoring center can access the database in the system.

[0024] Each subordinate server contains a monitoring program for data collection. The database includes a plugin table and a derived orchestration table for each subordinate server. The plugin table manages user-customized, hot-reloadable extension plugins for subordinate servers, enabling the collection of more data. The derived orchestration table manages each derivation path that aggregates lower-level metrics into higher-level business objects.

[0025] The following uses a lower-level server and its associated plugin table and derived orchestration table in the database as an example to introduce the monitoring method provided by the embodiments of the present invention.

[0026] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a monitoring method provided in an embodiment of the present invention. The subject executing this method can monitor a lower-level server in the system. The method includes the following steps S201 to S204.

[0027] S201. When a user uploads a plugin, update the plugin table based on the plugin.

[0028] S202. If it is confirmed that the plugin type belongs to the derived object generation type, then update the derivation orchestration table based on the plugin.

[0029] S203. Scan the plugin table once every preset scanning cycle to identify new plugins uploaded in the current scanning cycle.

[0030] S204. Based on the script file of the new plugin, perform the registration process to create a plugin instance; the plugin instance is used to obtain the required indicator data or derived object data according to the rules defined in the script file and report the data to the superior system.

[0031] When a user-uploaded plugin is received, the plugin table is updated accordingly, thus incorporating the plugin's lifecycle into the scope of database transaction management. However, during this process, the plugin has not yet been loaded and executed; it exists only as a metadata entity in the database awaiting activation.

[0032] The plugins uploaded by users have their own plugin types (such as monitoring extension types and derived object generation types). If it is confirmed that the plugin type of the user-uploaded plugin belongs to the derived object generation type, then it is also necessary to update the derivation orchestration table based on the plugin.

[0033] Simultaneously, the plugin table will be scanned every preset scanning cycle (e.g., 30 seconds) to identify new plugins uploaded during the current scanning cycle. Then, the newly identified plugins need to be registered and activated. The scanning process can be initiated by a scheduled task module built into the lower-level server. This database-driven plugin metadata management and dynamic lifecycle control solves the core shortcomings of traditional plugins, such as reliance on the file system, inability to centrally start and stop, and inconsistent cluster states, achieving dynamic plugin expansion with "configuration as code, and activation upon startup and shutdown."

[0034] The monitoring method provided by this invention first updates the plugin table and / or derived orchestration table based on the plugins uploaded by users, and then identifies the latest uploaded plugins by periodically scanning the plugin table and registering the new plugins, thereby realizing hot loading of plugin extensions without the need for service restart.

[0035] Optionally, the monitoring program installed on the lower-level server is a lightweight service process embedded on the operating system, which involves the initialization and running phases. Therefore, the monitoring method may also include the following steps S101~S102.

[0036] S101. When the monitoring program runs for the first time, it performs registration processing based on each pre-set native monitoring object.

[0037] In this embodiment, when the monitoring program runs for the first time, the plugin table and derived orchestration table in the database are empty or contain only default placeholders, indicating that no user-uploaded plugins have been loaded yet. However, the monitoring program has several native monitoring objects that need to be registered during the first run in order to enable them to perform monitoring.

[0038] For example, suppose the pre-defined native monitoring objects include the following three types: (1) Host object: represents the current server host, and its ID format can be: {MAC address}_host). (2) Disk object: Each disk partition corresponds to a Disk object, and its ID format can be: {MAC address}_disk_{device}; (3) NetworkInterface object: Each network card corresponds to a NetworkInterface object, and its ID format can be: {MAC address}_nic_{name}.

[0039] The specific operation of registration processing can be to write the unique identifier, type label, parent reference relationship and initial metadata field of the above three types of objects into the monitoring object registration center in memory, and assign a lifecycle management handle to them, so that each native monitoring object becomes the anchor entity for subsequent indicator collection and data modeling.

[0040] S102. According to the preset collection rules, periodically collect each preset native monitoring indicator under each native monitoring object and report the data to the superior system.

[0041] In this embodiment, the monitoring program has default settings for the collection rules of each native monitoring object. Therefore, after each native monitoring object is registered, the program periodically collects each preset native monitoring indicator under each native monitoring object according to the preset collection rules and reports the data to the superior system.

[0042] The data collection process can be driven by a built-in scheduled task, which calls the oshi-core library at fixed time intervals (e.g., every 30 seconds) to execute underlying system calls. This retrieves the CPU usage (cpu_usage) and memory usage (memory_used_percent) for the Host object, the disk read / write bytes (disk_read_bytes, disk_write_bytes) for the Disk object, and the network transmit / receive bytes (network_in_bytes, network_out_bytes) for the NetworkInterface object. During each data reporting cycle, all collected results are encapsulated into a structured monitoring snapshot using a unified JSON schema, containing standard fields such as id, parentId, type, and properties. This snapshot is then continuously pushed to the upper-level system via HTTP or Kafka protocols.

[0043] Optionally, the implementation process of step S201 above may include the following sub-steps S2011 to S2013: S2011. When a user uploads a plugin and it is determined that the user has set the plugin status to enabled, assign a plugin ID to the plugin.

[0044] In this embodiment, users can enable the plugin by setting its status to 1 or true through a web interface or other controlled entry point plugin script file. The script file can be Groovy script source code text that has passed syntax validation and sandbox security pre-checks. Next, a globally unique plugin ID is assigned to the plugin.

[0045] S2012. Extract the plugin type and dependency information from the plugin's script file, and perform a hash calculation on the script file to obtain the script hash.

[0046] Then, the script header comments or specific metadata blocks can be statically parsed to identify the plugin type (plugin_type field). Its value is strictly limited to a predefined enumeration set, including categories such as DataCollectorPlugin (data collection plugin), DataDispatcherPlugin (data delivery plugin), and DataConsumerPlugin (data consumption plugin). Simultaneously, third-party library dependencies declared in the script are parsed and expressed in a structured JSON format, such as {“kafka-clients”: “3.4.0”, “mqtt-client”: “2.8.0”}. Subsequently, a SHA-1 hash operation is performed on the complete script content, and the resulting script hash is used in subsequent scanning cycles to determine whether a plugin is registered. In this process, the plugin type determines the plugin's role and invocation context within the system architecture, while the dependency information provides a prerequisite for subsequent ClassLoader isolation and shared pool construction.

[0047] S2013. Based on plugin ID, plugin type, script file, plugin status, script hash, and dependency information, add a new entry to the plugin table.

[0048] In this embodiment, updating the plugin table can be performed as follows: An INSERT statement is executed on the plugin table (i.e., the monitor_plugin table) within a relational database transaction. The fields written strictly correspond to the table structure definition: the plugin_id field carries the assigned plugin identifier, the plugin_type field records the parsed plugin type, the script_content field persists the full text of the original script file as a TEXT type, the enabled field is set to true or 1, the version_hash field contains the calculated script hash, and the dependencies field stores the parsed JSON dependency declaration. Therefore, each entry in the plugin table includes the plugin ID, plugin type, script file, plugin status, script hash, and dependency information.

[0049] It's understandable that if the uploaded plugin is for expanding monitoring, then it can be divided into two types: Type 1: Extending the Metrics of an Existing Monitoring Object: Users can add a new monitoring metric, DISK_FREE_GB (remaining disk space), to an existing Disk object by uploading a Groovy plugin file, such as disk_free_space.groovy. This plugin does not create a new monitoring object; instead, it reuses the existing Disk object ID and attributes to directly add the new metric data. Once the plugin is enabled, the system will automatically include the newly added metric in the current monitoring snapshot during the next data collection cycle.

[0050] Type 2, Adding New Monitoring Objects and Metrics: When monitoring entirely new entity types, such as Docker containers, users can upload corresponding Groovy plugin files, such as docker_container_monitor.groovy. This plugin not only defines a new monitoring object type, Container, but also configures multiple monitoring metrics for it, such as CPU utilization and memory usage. After the plugin is registered, in each collection cycle, the system will generate the corresponding monitoring object and metrics based on the plugin's definition and add them to the monitoring snapshot.

[0051] Optionally, taking the case where the plugin status can be either 1 or 0 as an example, where 1 indicates that the plugin is enabled and 0 indicates that the plugin is disabled, then the implementation process of "scanning the plugin table once to identify new plugins uploaded in the current scanning period" in step S203 above may include the following sub-steps S2031~S2032: S2031, find each entry to be scanned in the plugin table where the plugin status is 1; S2032. For each entry to be scanned, if the script hash in the entry to be scanned is different from all the script hashes cached in memory, then the script file in the entry to be scanned is determined to be a new plugin for the current scan cycle.

[0052] In this embodiment, a query statement can be used to filter out only the entries in the plugin table containing active plugins with an enabled field value of 1, excluding all redundant entries that are disabled, deleted, or inactive due to initialization. The script hash of each registered plugin is cached in memory. Therefore, the script hash of each active plugin is compared with the cached script hashes. If a match is found, it indicates that the plugin is registered and running. If all match, it indicates that the plugin is a new plugin uploaded within the current scanning period, and the loading and registration process will be triggered.

[0053] Optionally, the implementation process of step S204 above may include the following sub-steps S2041~S204: S2041. Call the Groovy compiler to dynamically compile the script file and obtain the Class object of the Java class; S2042. Verify whether the Class object implements the predefined interface and whether it declares valid Spring annotations; S2043. If so, the Spring container is invoked to register a GenericBeanDefinition based on the Class object to complete the Bean initialization and obtain the plugin instance. S2044. Retrieve the plugin ID and script hash of the new plugin from the plugin table, and cache the plugin ID and script hash in memory.

[0054] In this embodiment, the registration process can specifically be: (1) Use a separate GroovyClassLoader instance to dynamically compile the script files of the new plugin and generate Class objects; (2) Verify whether the Class object implements a predefined interface (such as DataCollectorPlugin) and check whether it declares a valid Spring annotation (such as @Component); (3) Construct a Bean definition through GenericBeanDefinition, set its beanClass to the compiled Groovy class, and register it to BeanDefinitionRegistry; then call applicationContext.getBean() to get the plugin instance and trigger Spring lifecycle callbacks (including @PostConstruct); the plugin instance is not an independent thread or isolated script process, but a native Spring component that shares infrastructure such as transaction management, connection pool, and event bus with the main application.

[0055] (4) Explicitly call AutowireCapableBeanFactory.autowireBean(instance) to ensure that dependency injection annotations such as @Autowired are effective, and store the plugin ID and the current script hash in the memory cache (ConcurrentHashMap) for comparison in the next scan cycle.

[0056] Optionally, users can delete a plugin or set its plugin status to 0 or false (indicating it is disabled) through the web interface. Then, the following will be executed: (1) Call beanFactory.destroySingleton(beanName) to destroy the Bean instance and trigger the @PreDestroy callback (e.g., shut down Kafka Producer); (2) Call beanRegistry.removeBeanDefinition(beanName) to remove the definition from the Spring container; (3) Remove the plugin ID and script hash of the plugin from memory.

[0057] This enables dynamic uninstallation and resource reclamation of plugins, ensuring no memory leaks, no residual threads, and no connection leaks, thus meeting the stability requirements for long-term operation.

[0058] Optionally, to break the strong coupling between "acquisition and transmission" and give users complete control over the data flow, the monitoring system predefines three abstract interfaces: (1) DataCollectorPlugin: Define collect(): List <monitoringmetric>This method is used to return a list of indicators collected in the current period. (2) DataDispatcherPlugin: Defines the dispatch(Snapshot snapshot): void method, which allows rewriting the data delivery logic (such as modifying the Kafka topic or switching protocols); (3) DataConsumerPlugin: Define the consume(Record record): void method, which is used to perform custom processing on the consumer side (such as aggregation, noise reduction, and writing to a dedicated DB).

[0059] All plugins can be deeply integrated by injecting other beans in the Spring context (such as KafkaTemplate, DataSource, RestTemplate) via @Autowired.

[0060] Among them, the DataCollectorPlugin interface can dynamically and hot-load plugins for monitoring extended types; the DataDispatcherPlugin interface can dynamically and hot-load plugins for delivery logic modification types; and the DataConsumerPlugin interface can dynamically and hot-load plugins for derived object generation types.

[0061] Among them, for plugins that modify delivery logic, during their operation after registration, they can periodically check the availability of the main data link (such as Kafka Broker connectivity). Once the main link failure is confirmed, they can automatically switch to the backup data link (such as switching to MQTT or local cache replay), achieving link self-healing without manual intervention.

[0062] Optionally, for high-priority monitoring objects (such as core asset change services), users can write plugins to deliver their data directly to a dedicated Kafka topic (such as critical-service-metrics) in the monitoring center, bypassing the central server's aggregation layer and greatly reducing end-to-end latency.

[0063] Optionally, by utilizing the DataConsumerPlugin interface, this invention can also model task flow orchestration and derived monitoring objects based on DAG, thereby achieving a leap from atomic metrics to business semantic objects.

[0064] The derived orchestration table, namely the task_flow table, can include the following four fields: (1) flow_id: orchestration ID; (2) nodes: A JSON array of nodes. Each node in the array is identified by plugin_id or native_collector_key, which reflects a plugin or native collector related to a native monitoring object. (3) edges: A JSON-formatted relational array that defines the dependencies between nodes (e.g., ["cpu_check", "disk_check"] → "health_evaluator"); (4) output_object_type: Derived object type (e.g., SERVICE_HEALTH).

[0065] Therefore, each orchestration table entry in the derived orchestration table includes an orchestration ID, a node array, a relation array, and a derived object type. Each orchestration table entry has a corresponding entry in the plugin table, and each orchestration table entry can be regarded as a data acquisition DAG. Therefore, the TaskOrchestrator deployed in the system can execute the following steps S301~S302 in each acquisition cycle.

[0066] S301. For each enabled orchestration entry, sequentially call the instances corresponding to each node in the node array of the orchestration entry, excluding the last node, to collect data and obtain basic data.

[0067] If the node is a native collector, its standard collection method is called; if the node is a plugin, the corresponding Bean is obtained from the SpringContext and collect() is called.

[0068] S302. Based on the basic data and the relation array in the arrangement table, call the plugin instance corresponding to the last node in the node array of the arrangement table according to the preset derivation rules, output the current indicator value of the derived object, and report the data to the superior system.

[0069] In this embodiment, the plugin instance corresponding to the last node in the node array can output the current indicator value of the derived object according to the preset derivation rules in the script file. The preset derivation rules can involve comparing multiple original indicators and outputting the current indicator value of the derived object according to certain requirements.

[0070] For example, assuming "CPU utilization < 80%", "disk remaining > 10%", and "API latency < 500ms", the custom orchestrated composite monitoring object "Service Health" should currently have a score of 70. It should be noted that this example is merely illustrative and is not intended to be limiting.

[0071] It is understandable that derived objects automatically inherit the parentId of child objects (if all child objects share the same parent ID), forming a tree-like topology; the system assigns a globally unique ID to it, and the generation rule is: flow_id + "" + current timestamp (milliseconds) + "" + the first 8 bits of the SHA-1 hash value of the child object ID set (e.g., iot_health_flow_1735000000000_a3f9b2c1), ensuring no conflicts in a distributed environment.

[0072] Furthermore, derived objects can be incorporated into a unified data model: the generated derived objects are encapsulated together with the original metrics, participate in subsequent standardization, transmission, and storage, and can be independently queried, alerted, or used as input for higher-level orchestration in the monitoring center. Thus, the monitoring system of this invention possesses the ability to express complex business states (such as "service health" and "edge node availability score"), providing high-level feature inputs for the definition of subsequent links or lower-level monitoring objects and metrics.

[0073] The overall application example diagram of this invention, based on the above content, can also be referred to. Figure 3 As shown.

[0074] Optionally, to ensure the security and controllability of dynamic plugins in the production environment, the monitoring program has sandbox isolation, access control, dependency version management, and an exception circuit breaker mechanism: (1) Sandbox execution environment: Before the Groovy script is executed, an independent thread context is created and a custom SecurityManager is installed to restrict the following high-risk operations: 1) Runtime.exec(), System.exit(), File.delete(); 2) Reflection access to non-whitelisted classes (such as sun.misc.Unsafe, java.lang.Shutdown); 3) Network connections are restricted to accessing only pre-configured service endpoints (such as Kafka Broker, central server address list); 4) Creating new threads or thread pools is prohibited to prevent resource exhaustion attacks. SecurityManager is uninstalled immediately after the script execution is complete to avoid affecting the main application.

[0075] (2) Dependency isolation and shared pool: The dependencies declared by the plugin are resolved by PluginDependencyResolver. The system maintains a shared ClassLoader pool and caches loaded libraries by GAV (GroupId:ArtifactId:Version) to avoid duplicate loading and version conflicts.

[0076] (3) Circuit Breaker and Canary Release: 1) Circuit Breaker Mechanism: If the number of exceptions thrown by the plugin exceeds the threshold (e.g., 5 times / hour), the database enabled = false will be automatically updated and an alarm will be pushed; 2) Canary Release: The new plugin is initially enabled = false and marked is_canary = true; the operation and maintenance personnel can specify some node IDs and temporarily overwrite them with enabled = true on these nodes; the system collects its performance indicators and necessary business data indicators, such as waveform data and asset data collected by the system, and after verifying that there are no abnormalities and a fixed threshold period (e.g., 4 hours), it can be fully enabled.

[0077] It should be noted that the execution order of each step in the above method embodiments is not limited to that shown in the attached figures, and the execution order of each step shall be subject to the actual application situation.

[0078] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: 1. Achieve zero-downtime dynamic scaling of monitoring capabilities, completely solving the problem of hot plugin updates: Compared to systems like Zabbix and Prometheus, which require restarting the project or main process to load new plugins, this invention achieves runtime loading, unloading, and updating of plugins through a database-driven plugin metadata management mechanism and dynamic mapping from Groovy to Spring Beans. In production environment testing, after operations personnel upload and enable a new plugin via the Web UI, the system automatically completes compilation, registration, and initialization within 30 seconds (default scan cycle), and takes effect in the next collection cycle. When a plugin is disabled, related resources (such as Kafka Producer connections) are immediately released, with no memory leaks or thread remnants. In a cluster with 50 subordinate nodes, the plugin status of all nodes remains strongly consistent, achieving 100% deployment consistency, completely eliminating service interruptions caused by plugin changes, and meeting industrial-grade high availability requirements.

[0079] 2. Breaking the strong coupling between "collection and transmission" to build a programmable, self-healing, and low-latency data link: Addressing the issues of fixed data links and data loss upon failure in traditional monitoring systems, this invention empowers users with full-link programming capabilities for data flow through the DataDispatcherPlugin and DataConsumerPlugin interfaces. Real-world testing shows that when the primary Kafka link is simulated to fail, the system's built-in health detector automatically switches to the backup MQTT plugin after three consecutive failures (approximately 15 seconds), restoring the data delivery success rate from 0% to 99.8%. Simultaneously, configuring a direct delivery plugin for core services reduces end-to-end latency from an average of 1.8 seconds to 0.35 seconds, meeting millisecond-level real-time monitoring requirements. This mechanism improves data reliability by over 40% in complex scenarios such as hybrid cloud and weak network edge environments.

[0080] 3. Achieving a leap from atomic metrics to business semantic objects to support intelligent operation and maintenance decisions: Unlike existing systems that only provide isolated metrics, this invention uses a DAG task flow orchestration engine to integrate multiple raw metrics into structured derived monitoring objects. For example, CPU, disk, and API latency are orchestrated into an "iot-service-health" object, whose health score (0–100) can be directly used for alarms, capacity planning, or automated scaling.

[0081] 4. In response to the security and stability risks brought about by dynamic scripts, this invention achieves defense in depth through a four-fold mechanism of "sandbox + dependency isolation + circuit breaking + grayscale".

[0082] In order to perform the corresponding steps in the above method embodiments and various possible implementations, an implementation of a monitoring device is given below.

[0083] Please see Figure 4 , Figure 4 A schematic diagram of the monitoring device provided in an embodiment of the present invention is shown. The monitoring device 200 is applied to a lower-level server in a monitoring system. The monitoring system includes a database and an upper-level system that are communicatively connected to the lower-level server. The database includes a plugin table and a derived orchestration table corresponding to the lower-level server. The monitoring device 200 includes: a plugin upload module 210, a plugin scanning module 220, and a plugin registration module 230.

[0084] The plugin upload module 210 is used to update the plugin table based on the plugins when a user uploads a plugin. The plugin upload module 210 is also used to update the derivation orchestration table based on the plugin if it is confirmed that the plugin type belongs to the derived object generation type. The plugin scanning module 220 is used to scan the plugin table once every preset scanning cycle to identify new plugins uploaded in the current scanning cycle. The plugin registration module 230 is used to perform registration processing based on the script file of the new plugin to create a plugin instance; the plugin instance is used to obtain the required indicator data or derived object data according to the rules defined in the script file and report the data to the superior system.

[0085] Optionally, the lower-level server has monitoring software installed, and the monitoring device 200 may further include: The initialization module is used to perform registration processing based on each pre-defined native monitoring object when the monitoring program runs for the first time. The native data acquisition module is used to periodically collect data on each preset native monitoring indicator under each native monitoring object according to preset acquisition rules and report the data to the superior system.

[0086] Optionally, the plugin upload module 210 can be used to: when a user uploads a plugin and it is determined that the user has set the plugin status to enabled, assign a plugin ID to the plugin; extract the plugin type and dependency information from the plugin's script file, and perform a hash calculation on the script file to obtain a script hash; and add a new entry to the plugin table based on the plugin ID, plugin type, script file, plugin status, script hash, and dependency information.

[0087] Optionally, each entry in the plugin table includes a plugin status and a script hash. The plugin scanning module 220 can specifically be used to: find each entry in the plugin table whose plugin status is 1; for each entry, if the script hash in the entry is different from all the script hashes cached in memory, then the script file in the entry is determined to be a new plugin detected in the current scanning cycle.

[0088] Optionally, the plugin registration module 230 can be used to: call the Groovy compiler to dynamically compile the script file to obtain the Class object of the Java class; verify whether the Class object implements the predefined interface and whether it declares a valid Spring annotation; if so, call the Spring container to register GenericBeanDefinition based on the Class object to complete the Bean initialization and obtain the plugin instance; retrieve the plugin ID and script hash of the new plugin from the plugin table, and cache the plugin ID and script hash in memory.

[0089] Optionally, each orchestration entry in the derived orchestration table includes an orchestration ID, a node array, a relation array, and a derived object type. The monitoring device 200 may also include a composite derivation module, used for: for each enabled orchestration entry, sequentially calling the instances corresponding to each node in the node array of the orchestration entry (excluding the last node) to collect data and obtain basic data; one node in the node array is a plugin or a native collector related to a native monitoring object; based on the basic data and the relation array in the orchestration entry, calling the plugin instance corresponding to the last node in the node array of the orchestration entry according to preset derivation rules, outputting the current indicator value of the derived object, and reporting the data to the superior system.

[0090] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the monitoring device 200 described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0091] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 300 includes a processor 310, a memory 320, and a bus 330, with the processor 310 connected to the memory 320 via the bus 330.

[0092] The memory 320 can be used to store software programs or firmware, for example, the software program or firmware corresponding to the monitoring device 200 described above. The processor 310 executes various functional applications and data processing by running the software program stored in the memory 320 to implement the monitoring method provided in the embodiments of the present invention.

[0093] The memory 320 may be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), FLASH (Flash Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0094] The processor 310 can be an integrated circuit chip with signal processing capabilities, capable of executing software programs, such as the software program corresponding to the monitoring device 200. The processor 310 can be a general-purpose processor, including: CPU (Central Processing Unit), NP (Network Processor), SoC (System on Chip), etc.; it can also be: DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0095] Understandable. Figure 5 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown. Figure 5 The components shown can be implemented using hardware, software, or a combination thereof.

[0096] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the monitoring method disclosed in the above embodiments. The computer-readable storage medium can be, but is not limited to, various media capable of storing program code, such as a USB flash drive, external hard drive, ROM, RAM, PROM, EPROM, EEPROM, FLASH disk, or optical disk.

[0097] This invention also provides a computer program product that, when executed by a processor, implements the monitoring method disclosed in the above embodiments.

[0098] In summary, this invention provides a monitoring method, device, electronic device, storage medium, and program product. The method is applied to a lower-level server in a monitoring system. The monitoring system includes a database and an upper-level system that are communicatively connected to the lower-level server. The database includes a plugin table and a derived orchestration table corresponding to the lower-level server. The method is as follows: when a user-uploaded plugin is obtained, the plugin table is updated based on the plugin; if it is confirmed that the plugin type belongs to the derived object generation type, the derived orchestration table is updated based on the plugin; the plugin table is scanned every preset scanning cycle to identify new plugins uploaded in the current scanning cycle; registration processing is performed according to the script file of the new plugin to create a plugin instance; the plugin instance is used to obtain the required indicator data or derived object data according to the rules defined in the script file and report the data to the upper-level system. This invention updates the plugin table and / or derived orchestration table based on user-uploaded plugins, and identifies the latest uploaded new plugins by periodically scanning the plugin table and performs new plugin registration processing, achieving hot loading of plugin extensions without requiring service restarts.

[0099] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.< / monitoringmetric>

Claims

1. A monitoring method, characterized in that, A lower-level server used in a monitoring system, the monitoring system including a database and a higher-level system that are communicatively connected to the lower-level server, the database including a plug-in table and a derived orchestration table corresponding to the lower-level server; The method includes: When a user uploads a plugin, the plugin table is updated based on that plugin. If it is confirmed that the plugin type belongs to the derived object generation type, then update the derived orchestration table based on the plugin; The plugin table is scanned once every preset scanning cycle to identify new plugins uploaded in the current scanning cycle; According to the script file of the new plugin, a registration process is performed to create a plugin instance; the plugin instance is used to obtain the required indicator data or derived object data according to the rules defined in the script file and report the data to the superior system.

2. The monitoring method according to claim 1, characterized in that, The lower-level server is equipped with a monitoring program. Before the step of updating the plugin table based on the plugin when a user-uploaded plugin is received, the method further includes: When the monitoring program runs for the first time, it performs registration processing based on each pre-defined native monitoring object. According to the preset data collection rules, each preset native monitoring indicator under each native monitoring object is periodically collected and reported to the superior system.

3. The monitoring method according to claim 1, characterized in that, The step of updating the plugin table based on the plugin when a user-uploaded plugin is obtained includes: When a user uploads a plugin and it is determined that the user has set the plugin status to enabled, a plugin ID is assigned to the plugin. Extract the plugin type and dependency information from the plugin's script file, and perform a hash calculation on the script file to obtain the script hash; Based on the plugin ID, the plugin type, the script file, the plugin status, the script hash, and the dependency information, add a new entry to the plugin table.

4. The monitoring method according to claim 1, characterized in that, Each entry in the plugin table includes the plugin status and script hash; The step of scanning the plugin table to identify new plugins uploaded during the current scanning period includes: Find each entry in the plugin table whose plugin status is 1 that needs to be scanned; For each of the entries to be scanned, if the script hash in the entry to be scanned is different from all the script hashes cached in memory, then the script file in the entry to be scanned is determined to be a new plugin for the current scan cycle.

5. The monitoring method according to claim 1, characterized in that, The step of registering the new plugin based on its script file to create a plugin instance includes: The Groovy compiler is invoked to dynamically compile the script file, resulting in a Class object of a Java class. Verify whether the Class object implements the predefined interface and whether it declares valid Spring annotations; If so, the Spring container is invoked to register a GenericBeanDefinition based on the Class object to complete the Bean initialization and obtain the plugin instance; The plugin ID and script hash of the new plugin are obtained from the plugin table, and the plugin ID and script hash are cached in memory.

6. The monitoring method according to claim 1, characterized in that, Each entry in the derived orchestration table includes an orchestration ID, a node array, a relation array, and a derived object type; the method further includes: For each enabled orchestration entry, the instances corresponding to each node in the node array of the orchestration entry, excluding the last node, are called sequentially to collect data and obtain basic data; each node in the node array is a native collector related to a plugin or native monitoring object. Based on the basic data and the relation array in the orchestration table, the plugin instance corresponding to the last node in the node array of the orchestration table is called according to the preset derivation rules, outputs the current indicator value of the derived object, and reports the data to the superior system.

7. A monitoring device, characterized in that, A lower-level server used in a monitoring system, the monitoring system including a database and a higher-level system that are communicatively connected to the lower-level server, the database including a plug-in table and a derived orchestration table corresponding to the lower-level server; The device includes: The plugin upload module is used to update the plugin table based on the plugins uploaded by users. The plugin upload module is also used to update the derived orchestration table based on the plugin if it is confirmed that the plugin type belongs to the derived object generation type. The plugin scanning module is used to scan the plugin table once every preset scanning cycle to identify new plugins uploaded in the current scanning cycle. The plugin registration module is used to perform registration processing according to the script file of the new plugin to create a plugin instance; the plugin instance is used to obtain the required indicator data or derived object data according to the rules defined in the script file and report the data to the superior system.

8. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a software program, and the processor executes the software program when the electronic device is running to implement the monitoring method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the monitoring method according to any one of claims 1-6.

10. A program product, characterized in that, When the program product is executed by the processor, it implements the monitoring method according to any one of claims 1-6.

Images