Configurable Dores system monitoring method, device, equipment and medium

Abstract

The invention discloses a configurable Dores system monitoring and conversion method, device and system and a medium. The method comprises the following steps: deploying an Agent for configuring a monitoring system on each data node of a database system; when the Agent is started, collecting index data according to a preset time period, and transmitting the index data to a configuration monitoring system; the configuration monitoring system receives the index data, analyzes and processes the index data according to an alarm rule, and outputs alarm information; and when the alarm information accords with an alarm rule, triggering an alarm and carrying out alarm processing. According to the method, the Agent of the monitoring system is deployed on each data node of the database system, dynamic monitoring is carried out on the database system, the operation and maintenance efficiency is improved, the system fault time is shortened, and the operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and is applied to scenarios such as finance or medical treatment, and in particular to a configurable Doris system monitoring method, device, equipment and storage medium. Background Art

[0002] Doris (database system) is a high-performance, real-time analytical database built on the MPP (Massively Parallel Processing) architecture. It can quickly return query results under massive data with a sub-second response time. Doris supports high-concurrency point query scenarios and high-throughput complex analysis scenarios, and is suitable for a variety of usage scenarios such as report analysis, ad hoc query, unified data warehouse construction, data lake federated query acceleration, etc.

[0003] In the operation and maintenance of the big data Doris platform, monitoring and analyzing large-scale data systems is one of the daily tasks. For configurable systems, there will be obvious behavioral changes when the system performance changes from stable to abnormal, or when it recovers from abnormal to stable. These performance changes can be identified by extracting and analyzing system behavior, logs, and indicators. Rapidly identifying the change in system performance status has a wide range of application value in the system operation and maintenance industry, and it can help enterprises improve system stability and operation and maintenance efficiency.

[0004] In traditional system monitoring solutions, administrators often face complex, lengthy and cumbersome monitoring settings. Users must configure monitoring rules layer by layer and gradually adjust parameters according to the guidance to achieve monitoring, which often leads to a poor administrator experience. Traditional manual monitoring settings are facing increasingly severe challenges. In addition, due to the lack of an efficient doris monitoring configuration system, a large number of monitoring tasks rely on manual analysis, resulting in valuable human resources being invested in simple, repetitive and low-value labor. This situation not only limits the improvement of the overall operation and maintenance efficiency of the data center, but also leads to high operating costs, while affecting the stability and reliability of the system. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a configurable Doris system monitoring method, device, equipment and storage medium to solve the problems of high operating cost, low efficiency and impact on system stability and reliability of traditional system monitoring methods.

[0006] In order to solve the above technical problems, the embodiment of the present invention provides a configurable Doris system monitoring method, which adopts the following technical solution:

[0007] A configurable Doris system monitoring method comprises the following steps:

[0008] Deploy the Agent that configures the monitoring system on each data node of the database system;

[0009] When the Agent is started, it collects indicator data according to a preset time period and transmits the indicator data to a configuration monitoring system;

[0010] The configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rules, and outputs alarm information;

[0011] When the alarm information meets the alarm rules, an alarm is triggered and alarm processing is performed.

[0012] In one embodiment, before deploying the Agent of the configuration monitoring system on each data node of the database system, the process further includes:

[0013] Configuring an instance name for each data node of the database system;

[0014] When the data node contains a data unit, an instance name is configured for the data node;

[0015] When the data node contains several data units, an instance name is configured for each data unit.

[0016] In one embodiment, when the Agent is started, collecting indicator data according to a preset time period and transmitting the indicator data to a configuration monitoring system includes:

[0017] When the Agent is started, the Agent is used as a monitoring node to send a registration request to the registration center;

[0018] When the Agent is successfully registered, the monitoring system is configured to determine the node to be monitored according to the Agent and the monitoring task;

[0019] The Agent collects the index data of the nodes to be monitored according to a preset time period, and transmits the index data to the configuration monitoring system.

[0020] In one embodiment, when the Agent is started, the Agent is used as a monitoring node to send a registration request to the registration center, and further includes:

[0021] The registration center receives the registration request from the Agent and stores the registration information in the service directory;

[0022] The monitoring system is configured to update the monitoring nodes and monitoring strategies according to the registration information.

[0023] In one embodiment, the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information, including:

[0024] The configuration monitoring system matches the indicator data with the alarm rules;

[0025] When the indicator data meets the alarm rules, the alarm information is output.

[0026] In one embodiment, the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information, and further includes:

[0027] When the indicator data exceeds the alarm threshold, an alarm message is output;

[0028] The configuration monitoring system displays the indicator data and the alarm information on a screen.

[0029] In one embodiment, before the configuration monitoring system updates the monitoring node and the monitoring strategy according to the registration information, the configuration monitoring system includes:

[0030] Configuring a registration center service discovery module in the configuration monitoring system;

[0031] When the registration center service discovery module detects that the registration center receives the registration request, it sends a request signal to the configuration monitoring system;

[0032] The configuration monitoring system receives the request signal and obtains the registration information of the registration center.

[0033] In order to solve the above technical problems, the embodiment of the present invention also provides a configurable Doris system monitoring device, which adopts the following technical solution:

[0034] A configurable Doris system monitoring device, comprising:

[0035] The monitoring deployment module is used to deploy and configure the monitoring system agent on each data node of the database system;

[0036] A data collection module, used for collecting indicator data according to a preset time period when the Agent is started, and transmitting the indicator data to a configuration monitoring system;

[0037] An alarm judgment module is used for the configuration monitoring system to receive the indicator data, analyze and process the indicator data according to the alarm rules, and output alarm information;

[0038] The alarm processing module is used to trigger an alarm and perform alarm processing when the alarm information meets the alarm rules.

[0039] In order to solve the above technical problems, an embodiment of the present invention further provides a computer device, which adopts the following technical solution:

[0040] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned configurable Doris system monitoring method when executing the computer-readable instructions.

[0041] In order to solve the above technical problems, an embodiment of the present invention further provides a computer-readable storage medium, which adopts the following technical solution:

[0042] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the configurable Doris system monitoring method as described above.

[0043] Compared with the prior art, the embodiments of the present invention mainly have the following beneficial effects:

[0044] The configurable Doris system monitoring method described in the embodiment of the present invention is implemented by deploying an Agent of the configuration monitoring system on each data node of the database system; when the Agent is started, the index data is collected according to a preset time period, and the index data is transmitted to the configuration monitoring system; the configuration monitoring system receives the index data, analyzes and processes the index data according to the alarm rules, and outputs alarm information; when the alarm information meets the alarm rules, an alarm is triggered and an alarm process is performed. The present invention implements dynamic monitoring of the database system, improves operation and maintenance efficiency, reduces system failure time, and reduces operating costs by deploying an Agent of the configuration monitoring system on each data node of the database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the scheme in the present invention, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a flow chart of an embodiment of a configurable Doris system monitoring method according to the present invention;

[0047] Figure 2 is a flow chart of another embodiment of a configurable Doris system monitoring method according to the present invention;

[0048] Figure 3 is a structural schematic diagram of an embodiment of a configurable Doris system monitoring device according to the present invention;

[0049] Figure 4 FIG. 1 is a schematic structural diagram of an embodiment of a computer device according to the present invention. DETAILED DESCRIPTION

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present invention or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0051] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0053] It should be noted that the configurable Doris system monitoring method provided in the embodiment of the present invention is generally executed by a server / terminal device, and accordingly, the configurable Doris system monitoring device is generally arranged in a server / terminal device.

[0054] refer to Figure 1 , shows a flow chart of an embodiment of a configurable Doris system monitoring method according to the present invention. The configurable Doris system monitoring method comprises the following steps:

[0055] S1 deploys the agent that configures the monitoring system on each data node of the database system;

[0056] S2 When the Agent is started, it collects indicator data according to a preset time period and transmits the indicator data to the configuration monitoring system;

[0057] S3: the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information;

[0058] S4: When the alarm information meets the alarm rule, an alarm is triggered and alarm processing is performed.

[0059] In this embodiment, an agent (Agent) of the configuration monitoring system (CMS, Configuration Monitoring System), such as NodeExporter, is deployed on each data node of the database system (Doris). When the Agent is started, it automatically registers itself through the API of the registration center and becomes a monitoring target in the monitoring system. After successful registration, the Agent will collect indicator data according to a preconfigured period (for example, every 2 minutes), and organize the collected data and send it to the monitoring main server. This data transmission step is usually implemented through HTTP (Hypertext Transfer Protocol) or other network protocols.

[0060] After the monitoring master server receives the data, it may store it in a time series database for further analysis and processing. The configuration monitoring system receives these indicator data and analyzes and processes the data according to the set alarm rules to output alarm information. The system can also visualize the collected data and evaluate whether the data meets the preset alarm rules, triggering alarms when necessary.

[0061] The continuous operation of Agent ensures that the monitoring system can monitor the status of Doris database or other services in real time or near real time. In this way, the monitoring system can automatically collect, analyze and respond to monitoring data, thereby improving operation and maintenance efficiency and system stability.

[0062] In one embodiment, before deploying the Agent of the configuration monitoring system on each data node of the database system, the process further includes:

[0063] Configuring an instance name for each data node of the database system;

[0064] When the data node contains a data unit, an instance name is configured for the data node;

[0065] When the data node contains several data units, an instance name is configured for each data unit.

[0066] In this embodiment, the configuration monitoring system (CMS) usually configures an instance name for each data node of the database system. If a data node only runs one service or component that needs to be monitored, then the node may only be assigned one instance name for identification. However, if multiple independent services or components are deployed on a data node, each service or component should have a unique instance name to ensure that they can be distinguished and monitored separately.

[0067] The primary purpose of an instance name is to identify a specific service or component, not the physical node itself. Therefore, even if multiple services are deployed on the same physical server, each service should have its own instance name. "Service or component" in this context may refer to the following:

[0068] The database instance itself, especially if the database system supports running multiple instances on the same node.

[0069] Different services within the database, such as query services, transaction services, etc., need to be monitored separately.

[0070] Other components of the database system include cache, connection pool, logging system, etc.

[0071] By configuring a unique instance name for each service or component, CMS can more accurately monitor and manage each part of the database system.

[0072] Specifically, each service or component that needs to be monitored is assigned a unique instance name in the CMS. This instance name is used to identify and distinguish different monitoring targets in the system. When the monitoring agent is started and registered with the registration center, it will carry or associate its instance name in the registration center. In this way, the CMS can automatically discover and identify specific monitoring targets through registration information. CMS uses instance names to manage monitoring configurations, including setting monitoring parameters, thresholds, alarm rules, etc. The agent collects relevant data according to the configured time period and sends the data back to the CMS. In the CMS, data is usually classified by instance name to ensure that the CMS can correctly associate the data with specific services or components. When the monitoring conditions meet the alarm threshold, the CMS will determine the service or component with the problem based on the instance name and include this key information in the alarm notification issued. This mechanism ensures that the operation and maintenance team can quickly identify the problem and take appropriate measures.

[0073] The instance name has the following functions:

[0074] (1) The instance name of each database node provides a unique identifier that helps distinguish different nodes in the same database system;

[0075] (2) The instance name allows CMS to monitor each database node in detail, distinguish different services or components running on the same node, and collect and analyze specific performance indicators for each node;

[0076] (3) By configuring an instance name for each node, the administrator can set independent monitoring configuration and alarm rules for each node in CMS;

[0077] (4) When an alarm is triggered, the instance name can help quickly locate the specific database node where the problem occurs and clearly indicate it in the monitoring report;

[0078] (5) In the monitoring dashboard, data can be aggregated or separated based on instance name.

[0079] (6) As the database system expands, new nodes can be added and assigned new instance names to achieve comprehensive monitoring of the entire database system.

[0080] By configuring an instance name for each node, service, or component in the database system, CMS can provide more detailed and targeted monitoring, thereby enhancing the maintainability and stability of the entire database system.

[0081] In one embodiment, when the Agent is started, collecting indicator data according to a preset time period and transmitting the indicator data to a configuration monitoring system includes:

[0082] When the Agent is started, the Agent is used as a monitoring node to send a registration request to the registration center;

[0083] When the Agent is successfully registered, the monitoring system is configured to determine the node to be monitored according to the Agent and the monitoring task;

[0084] The Agent collects the index data of the nodes to be monitored according to a preset time period, and transmits the index data to the configuration monitoring system.

[0085] In this embodiment, reference may be made to Figure 2 When the agent in the monitoring system starts, it first sends a registration request to the registration center, indicating that it wants to join the monitoring system as a new monitoring node. The registration center, as the core of service discovery and registration, receives the registration request of the agent, updates the service directory, and records all available monitoring nodes.

[0086] After successful registration, the result will notify the Configuration Monitoring System (CMS) that there are new monitoring nodes available for configuration and monitoring. CMS determines the nodes that need to be monitored based on the registered Agent information and predefined monitoring tasks, which may involve matching Agent attributes and monitoring policies to determine appropriate monitoring behavior.

[0087] According to the registration information and CMS configuration, the Agent starts to collect the indicator data of the node at a preset time period (such as every minute or every 5 minutes), including CPU usage, memory usage, disk I / O, network traffic, etc. The collected indicator data is then transmitted to the CMS.

[0088] CMS is responsible for receiving and processing this data, and storing, analyzing and visualizing it as needed. Based on the information provided by the Agent and its own monitoring strategy, CMS performs monitoring tasks, including real-time monitoring, performance analysis, trend prediction, alarm triggering, etc. The entire process is automated to ensure real-time and continuous monitoring.

[0089] Agent registration, data collection and transmission, and CMS processing and response are seamlessly connected. With the registration of new agents and the possible deregistration of old agents, CMS can dynamically adjust the monitoring scope to adapt to changes in IT infrastructure.

[0090] Throughout the process, it is crucial to ensure data security and transmission reliability. Communication between the registration center and the CMS may require encryption and authentication mechanisms. Through this mechanism, the configuration monitoring system can efficiently manage and monitor various nodes and services in the IT infrastructure to ensure system stability and performance.

[0091] In one embodiment, when the Agent is started, the Agent is used as a monitoring node to send a registration request to the registration center, and further includes:

[0092] The registration center receives the registration request from the Agent and stores the registration information in the service directory;

[0093] The monitoring system is configured to update the monitoring nodes and monitoring strategies according to the registration information.

[0094] In this embodiment, the Agent in the monitoring system automatically sends a registration request to the registration center when it is started, intending to register itself as a monitoring target node. This process is fully automated and does not require manual intervention. After receiving the registration request from the Agent, the registration center stores its information in the service directory. This information may include the Agent's identifier, IP address, port number, and the type of resource being monitored.

[0095] The service directory maintained by the registry records a list of all registered services and agents, which can be queried by the configuration monitoring system (CMS). CMS regularly queries the service directory of the registry to discover new monitoring targets or update the status of existing targets. Based on the information obtained from the registry, CMS updates its internal monitoring node list to ensure that all nodes that need to be monitored are included.

[0096] Based on the registration information and business rules, CMS adjusts or sets monitoring policies, which may include setting data collection frequency, defining monitoring indicators, configuring alarm thresholds, etc. Automated processes ensure that CMS can respond to changes in the service catalog in real time. The service discovery mechanism enables CMS to identify and start monitoring newly registered services based on the information in the registry.

[0097] When the status of the Agent or the monitoring target changes, the registry updates the service catalog, and the CMS dynamically adjusts its monitoring behavior accordingly. This decentralized registration and discovery mechanism enhances the flexibility and scalability of the monitoring system, allowing the system to easily adapt to environmental changes. The interaction between the registry and the CMS ensures that the monitoring system can dynamically manage and monitor the entire infrastructure without manual intervention, thereby improving operation and maintenance efficiency and automation.

[0098] In one embodiment, the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information, including:

[0099] The configuration monitoring system matches the indicator data with the alarm rules;

[0100] When the indicator data meets the alarm rules, the alarm information is output.

[0101] In this embodiment, reference Figure 2In the indicator alarm module, CMS allows administrators to set a series of alarm rules, which specify the specific conditions for triggering alarms. For example, you can set an alarm to be triggered when the CPU usage exceeds 90%. The alarm platform will automatically and continuously monitor the collected indicator data and match it with the predefined alarm rules. The system will automatically check each indicator to determine whether it meets the conditions for triggering an alarm. Once the indicator data successfully matches the alarm rule, such as the database response time exceeds the preset threshold, the alarm platform will trigger the corresponding notification mechanism according to the configuration. If the phone alarm rule is matched, the system will automatically dial the preset phone number and convey the alarm information in the form of voice broadcast. Similar to the phone alarm, when the indicator data meets the email alarm rule, the alarm platform will send an email to the preset email address. These emails will contain detailed information about the alarm, including the time when the alarm occurred, the name of the indicator that triggered the alarm, the specific value, the possible impact, and the recommended action steps. In addition to phone and email alarms, the alarm platform may also support other notification methods, such as SMS, instant messaging, or application push notifications, to ensure that the alarm information can be conveyed to the relevant personnel in a timely and effective manner.

[0102] The Configuration Monitoring System (CMS) evaluates indicator data in real time according to preset rules. The system continuously checks the latest collected data to ensure that any abnormal situations can be responded to in a timely manner. When a metric exceeds the preset safety range or threshold, the CMS will automatically trigger an alarm without manual intervention.

[0103] The purpose of intelligent alarm management is to reduce false positives and missed positives. The system sends a notification only after confirming the accuracy of the alarm through analysis. All alarm events are recorded in detail in the system, including the time of the alarm, the indicators involved, the thresholds triggered, and the notification method, etc. These records are convenient for subsequent analysis and auditing. Once the alarm is triggered, the operation and maintenance team can respond quickly based on the alarm information to diagnose and repair the problem. If the alarm is not handled in time, the system may automatically upgrade the alarm level according to the configured policy and notify the relevant personnel through different channels or more urgent methods. After the problem is resolved, the system may also send an alarm recovery notification to inform the relevant personnel that the system has returned to normal. Such an alarm mechanism ensures that when key indicators are abnormal, the relevant personnel can be notified in time and take corresponding measures, thereby minimizing the impact of system failures and improving the stability and reliability of the system.

[0104] In one embodiment, the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information, and further includes:

[0105] When the indicator data exceeds the alarm threshold, an alarm message is output;

[0106] The configuration monitoring system displays the indicator data and the alarm information on a screen.

[0107] In this embodiment, reference Figure 2 The indicator display module of CMS, the user interface (UI) of CMS not only displays real-time indicator data and alarm information, but also provides a variety of visualization elements such as charts, tables, trend lines, etc., so that operation and maintenance personnel can monitor the system status in real time and respond to abnormalities quickly. Alarm information is clearly displayed on the visual dashboard of CMS, including detailed contextual information such as severity, duration, and scope of impact. Users can interact with alarms, such as confirming alarms, updating status, and even directly triggering automated responses. In addition, operation and maintenance personnel can update alarm thresholds or monitoring configurations through the UI as needed to adapt to system changes or optimize monitoring strategies.

[0108] Furthermore, the Configuration Monitoring System (CMS) is equipped with a user interface (UI), which not only displays indicator data and alarm information in real time, but also provides a variety of visualization elements such as charts, tables, trend lines, etc., allowing users to intuitively monitor system status and quickly respond to abnormal situations. Key functions include:

[0109] Alarm threshold detection: The system continuously monitors indicator data and compares it with the preset alarm threshold.

[0110] Alarm information output: When indicator data exceeds the threshold, the system generates an alarm message and notifies the user through multiple channels.

[0111] Real-time screen display: The UI displays real-time data and alarms, providing detailed contextual information such as the severity, duration, and impact of the alarm.

[0112] Interactivity: Users can interact with alerts to acknowledge alerts, update status, or trigger automated responses.

[0113] Custom configuration: Operation and maintenance personnel can update alarm thresholds or monitoring configurations as needed to optimize monitoring strategies.

[0114] Data visualization: The monitoring dashboard built with front-end technology is customized according to user needs to display different monitoring data and information.

[0115] Cluster status display: Displays the overall operating status of the entire IT infrastructure or service cluster, such as resource utilization and system load.

[0116] Component performance monitoring: Displays performance metrics of components or services, such as response time and throughput.

[0117] Host performance monitoring: Monitor the performance of a single host or server, such as CPU, memory, disk, and network I / O.

[0118] Real-time data stream: The dashboard obtains the latest monitoring data through real-time data connection with the configuration module.

[0119] Preconfigured dashboards: Provide templates that users can use to quickly start monitoring and adjust as needed.

[0120] Interactive interface: supports click, hover and other operations to obtain more detailed information or perform specific operations.

[0121] Data filtering and sorting: Users can filter and sort displayed data to quickly locate problems or analyze performance.

[0122] Alert integration: Integrate alert information in data visualization and display alerts on the dashboard when thresholds are exceeded.

[0123] Multi-dimensional view: supports viewing data from different dimensions such as time range, service type, geographic location, etc.

[0124] Sharing and Subscriptions: Users can share dashboards or subscribe to updates on specific metrics to stay in sync.

[0125] Mobile access: Supports access to the dashboard on mobile devices, allowing users to monitor system status anytime and anywhere.

[0126] Through these functions, CMS provides users with a comprehensive, real-time monitoring solution to help users promptly detect and respond to potential problems and ensure system stability and performance.

[0127] In one embodiment, before the configuration monitoring system updates the monitoring node and the monitoring strategy according to the registration information, the configuration monitoring system includes:

[0128] Configuring a registration center service discovery module in the configuration monitoring system;

[0129] When the registration center service discovery module detects that the registration center receives the registration request, it sends a request signal to the configuration monitoring system;

[0130] The configuration monitoring system receives the request signal and obtains the registration information of the registration center.

[0131] In this embodiment, the CMS is configured with a registry service discovery module, so that it can automatically discover all registered services and start collecting metrics of these services. Service discovery is a key function in modern monitoring systems, especially in cloud-based and distributed environments.

[0132] Through the configured registry service discovery module, CMS can automatically detect all services registered to the registry. This means that CMS does not need to manually configure the monitoring information of each service. The registry acts as a hub for service registration and discovery. When a new service or monitoring agent starts and registers itself, the registry records this information. The service catalog maintained by the registry is dynamically updated as services are registered and deregistered.

[0133] CMS will periodically query or listen to this directory to obtain the latest service status information. The service discovery mechanism reduces the need to manually add and configure monitoring targets, thereby reducing the workload of operation and maintenance and reducing human errors. In a dynamic environment where services change frequently, service discovery ensures that CMS can keep abreast of all currently active services, no matter when they are started or stopped.

[0134] Once CMS identifies new services through service discovery, it can immediately start collecting performance metrics for these services for real-time monitoring. For newly discovered services, CMS can automatically apply preset monitoring templates or policies, or dynamically generate monitoring configurations based on the type and properties of the service. The registry may perform health checks on registered services, and CMS can adjust monitoring priorities or alarm policies based on this health status information.

[0135] Service discovery mechanisms are usually decentralized, allowing services to register and deregister independently of the CMS, which in turn discovers these changes through the registry. As the system scales and new services are added, service discovery mechanisms support the CMS to horizontally scale its monitoring capabilities without making large-scale modifications to the existing architecture. Service discovery is the foundation for implementing automated, resilient, and highly available monitoring systems, enabling the CMS to intelligently adapt to the ever-changing IT environment.

[0136] Furthermore, this embodiment also includes a configuration module, which serves as a configuration management database for storing information about IT infrastructure. Configuration management is used to store configuration information of the monitoring system, including monitoring targets, thresholds, alarm rules, etc.

[0137] When a new monitoring instance is automatically registered through the CMS and the registration center, the relevant configuration information can also be automatically updated to the configuration module. The configuration module is usually a database or similar storage system that is specifically used to store the configuration information of the IT infrastructure. The stored information includes but is not limited to the detailed information of the monitoring target, the threshold of the performance indicator, the alarm rules, and other related monitoring settings.

[0138] The configuration module records the identification information and attributes of all monitoring targets, such as database instances, servers, applications, etc., and associates them with their instance names. The thresholds of performance indicators define the conditions for triggering alarms, and these thresholds are stored by the configuration module so that the monitoring system can compare the collected data and determine whether to trigger an alarm.

[0139] The configuration module also contains alert rules based on specific conditions, such as when an indicator exceeds a threshold, falls below a certain value, or changes in rate. When a new monitoring instance is automatically registered with the registration center through CMS, the configuration module can be automatically updated to reflect the configuration information of the new monitoring instance. The configuration module can automatically update the configuration information to ensure that the monitoring strategy is consistent with the latest status of the IT infrastructure.

[0140] The configuration module is tightly integrated with the service discovery mechanism so that when a service is registered with the registry, relevant information is automatically synchronized to the configuration module. This allows administrators to develop and adjust monitoring policies as needed, including adding new monitoring targets, modifying thresholds, or updating alarm rules.

[0141] The configuration module provides a user interface that enables administrators to easily view, edit and manage configuration information. It ensures data consistency of all monitoring configurations, prevents monitoring problems caused by configuration errors, and can record the configuration change history to help audit and ensure compliance. With these functions, the configuration module becomes an indispensable part of the monitoring system, providing strong support for automated and effective management of IT infrastructure.

[0142] The embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0143] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, highly configurable Doris system monitoring technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0144] In an embodiment of the present invention, an Agent of a configuration monitoring system is deployed on each data node of a database system; when the Agent is started, index data is collected according to a preset time period, and the index data is transmitted to the configuration monitoring system; the configuration monitoring system receives the index data, analyzes and processes the index data according to an alarm rule, and outputs an alarm message; when the alarm message meets the alarm rule, an alarm is triggered and an alarm process is performed. The present invention implements dynamic monitoring of the database system, improves operation and maintenance efficiency, reduces system failure time, and reduces operating costs by deploying an Agent of a configuration monitoring system on each data node of the database system.

[0145] Further references Figure 3 , as a response to the above Figure 3 The present invention provides an embodiment of a configurable Doris system monitoring device, which is similar to Figure 3 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0146] like Figure 3 As shown, the configurable Doris system monitoring device described in this embodiment includes: a monitoring deployment module 11, a data collection module 12, an alarm judgment module 13 and an alarm processing module 14. Among them:

[0147] The monitoring deployment module 11 is used to deploy the Agent of the configuration monitoring system on each data node of the database system;

[0148] A data collection module 12, used for collecting indicator data according to a preset time period when the Agent is started, and transmitting the indicator data to a configuration monitoring system;

[0149] The alarm judgment module 13 is used for the configuration monitoring system to receive the indicator data, analyze and process the indicator data according to the alarm rules, and output alarm information;

[0150] The alarm processing module 14 is used to trigger an alarm and perform alarm processing when the alarm information meets the alarm rules.

[0151] like Figure 3 As shown, the monitoring deployment module 11, the data collection module 12, the alarm judgment module 13 and the alarm processing module 14 are connected in sequence. The module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of configurable Doris system monitoring and conversion than the program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0152] In one embodiment, the configurable Doris system monitoring device further includes:

[0153] The instance name configuration module is used to configure an instance name for each data node of the database system; when the data node contains one data unit, the instance name is configured for the data node; when the data node contains several data units, the instance name is configured for each data unit.

[0154] In one embodiment, the data collection module 12 includes:

[0155] A registration unit, configured to send a registration request to the registration center using the Agent as a monitoring node when the Agent is started;

[0156] A monitoring node confirmation unit, used for configuring the monitoring system to determine the node to be monitored according to the Agent and the monitoring task when the Agent is successfully registered;

[0157] The data collection and transmission unit is used for the Agent to collect the index data of the node to be monitored according to a preset time period, and transmit the index data to the configuration monitoring system.

[0158] In one embodiment, the registration unit further includes:

[0159] A registration information storage unit, used for the registration center to receive the registration request of the Agent and store the registration information in the service directory;

[0160] The monitoring update unit is used to configure the monitoring system to update the monitoring nodes and monitoring strategies according to the registration information.

[0161] In one embodiment, the alarm determination module 13 includes:

[0162] A rule matching unit, used for the configuration monitoring system to match the indicator data with the alarm rule;

[0163] The alarm output unit is used to output alarm information when the indicator data meets the alarm rules.

[0164] In one embodiment, the alarm determination module 13 further includes:

[0165] The information display unit is used to output alarm information when the indicator data exceeds the alarm threshold; the configuration monitoring system displays the indicator data and the alarm information on the screen.

[0166] In one embodiment, the registration unit includes:

[0167] Configuring a registration center service discovery module in the configuration monitoring system;

[0168] When the registration center service discovery module detects that the registration center receives the registration request, it sends a request signal to the configuration monitoring system;

[0169] The configuration monitoring system receives the request signal and obtains the registration information of the registration center.

[0170] In the above embodiment, the present invention discloses a configurable Doris system monitoring device, which belongs to the field of data processing technology. The present invention deploys an Agent of the configuration monitoring system on each data node of the database system; when the Agent is started, the index data is collected according to a preset time period, and the index data is transmitted to the configuration monitoring system; the configuration monitoring system receives the index data, analyzes and processes the index data according to the alarm rules, and outputs alarm information; when the alarm information meets the alarm rules, an alarm is triggered and an alarm process is performed. The present invention implements dynamic monitoring of the database system, improves operation and maintenance efficiency, reduces system failure time and reduces operating costs by deploying an Agent of the configuration monitoring system on each data node of the database system.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0172] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0173] To solve the above technical problems, the embodiment of the present invention also provides a computer device. Figure 4 , Figure 4This is a basic structural block diagram of the computer device in this embodiment.

[0174] The computer device 8 includes a memory 8a, a processor 8b, and a network interface 8c that are interconnected through a system bus. It should be noted that the figure only shows a computer device 8 with components 8a-8c, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0175] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0176] The memory 8a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 8a can be an internal storage unit of the computer device 8, such as a hard disk or memory of the computer device 8. In other embodiments, the memory 8a can also be an external storage device of the computer device 8, such as a plug-in hard disk equipped on the computer device 8, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 8a can also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 8a is generally used to store the operating system and various application software installed on the computer device 8, such as a computer-readable instruction of a configurable Doris system monitoring method, etc. In addition, the memory 8a can also be used to temporarily store various types of data that have been output or are to be output.

[0177] The processor 8b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other configurable Doris system monitoring chip in some embodiments. The processor 8b is generally used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is used to run the computer-readable instructions or process data stored in the memory 8a, such as computer-readable instructions for running the configurable Doris system monitoring method.

[0178] The network interface 8c may include a wireless network interface or a wired network interface. The network interface 8c is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0179] The computer device proposed in this embodiment belongs to the field of data processing technology and is applied in scenarios such as finance or medical treatment. The present invention deploys an Agent of the configuration monitoring system on each data node of the database system; when the Agent is started, it collects indicator data according to a preset time period and transmits the indicator data to the configuration monitoring system; the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rules, and outputs alarm information; when the alarm information meets the alarm rules, an alarm is triggered and alarm processing is performed. The present invention implements dynamic monitoring of the database system, improves operation and maintenance efficiency, reduces system failure time and reduces operating costs by deploying an Agent of the configuration monitoring system on each data node of the database system.

[0180] The present invention also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the configurable Doris system monitoring method as described above.

[0181] The computer-readable storage medium proposed in this embodiment belongs to the field of data processing technology and is applied in scenarios such as finance or medical treatment. The present invention deploys an Agent of the configuration monitoring system on each data node of the database system; when the Agent is started, it collects indicator data according to a preset time period and transmits the indicator data to the configuration monitoring system; the configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rules, and outputs alarm information; when the alarm information meets the alarm rules, an alarm is triggered and alarm processing is performed. The present invention implements dynamic monitoring of the database system, improves operation and maintenance efficiency, reduces system failure time and reduces operating costs by deploying an Agent of the configuration monitoring system on each data node of the database system.

[0182] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0183] It should be noted that if software tools or components other than those of the company appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The embodiments described above are only some embodiments of the present invention, not all embodiments. The preferred embodiments of the present invention are given in the accompanying drawings, but they do not limit the scope of the invention. The present invention can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive. Although the present invention is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific embodiments, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of the present invention, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present invention.

Claims

1. A configurable Doris system monitoring method, characterized in that: The steps include: Deploy the Agent that configures the monitoring system on each data node of the database system; When the Agent is started, it collects indicator data according to a preset time period and transmits the indicator data to a configuration monitoring system; The configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rules, and outputs alarm information; When the alarm information meets the alarm rules, an alarm is triggered and alarm processing is performed.

2. The configurable Doris system monitoring method according to claim 1, characterized in that: Before deploying the Agent of the configuration monitoring system on each data node of the database system, the method further includes: Configuring an instance name for each data node of the database system; When the data node contains a data unit, an instance name is configured for the data node; When the data node contains several data units, an instance name is configured for each data unit.

3. The configurable Doris system monitoring method according to claim 2, characterized in that: When the Agent is started, the indicator data is collected according to a preset time period and the indicator data is transmitted to the configuration monitoring system, including: When the Agent is started, the Agent is used as a monitoring node to send a registration request to the registration center; When the Agent is successfully registered, the monitoring system is configured to determine the node to be monitored according to the Agent and the monitoring task; The Agent collects the index data of the nodes to be monitored according to a preset time period, and transmits the index data to the configuration monitoring system.

4. The configurable Doris system monitoring method according to claim 3, characterized in that: When the Agent is started, the Agent is used as a monitoring node to send a registration request to the registration center, and further includes: The registration center receives the registration request from the Agent and stores the registration information in the service directory; The monitoring system is configured to update the monitoring nodes and monitoring strategies according to the registration information.

5. The configurable Doris system monitoring method according to claim 4, characterized in that: The configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information, including: The configuration monitoring system matches the indicator data with the alarm rules; When the indicator data meets the alarm rules, the alarm information is output.

6. The configurable Doris system monitoring method according to claim 5, characterized in that: The configuration monitoring system receives the indicator data, analyzes and processes the indicator data according to the alarm rule, and outputs alarm information, and also includes: When the indicator data exceeds the alarm threshold, output alarm information; The configuration monitoring system displays the indicator data and the alarm information on a screen.

7. The configurable Doris system monitoring method according to claim 6, characterized in that: Before the configuration monitoring system updates the monitoring nodes and monitoring strategies according to the registration information, the configuration monitoring system includes: Configuring a registration center service discovery module in the configuration monitoring system; When the registration center service discovery module detects that the registration center receives the registration request, it sends a request signal to the configuration monitoring system; The configuration monitoring system receives the request signal and obtains the registration information of the registration center.

8. A configurable Doris system monitoring device, characterized in that: include: The monitoring deployment module is used to deploy and configure the monitoring system agent on each data node of the database system; A data collection module, used for collecting indicator data according to a preset time period when the Agent is started, and transmitting the indicator data to a configuration monitoring system; An alarm judgment module is used for the configuration monitoring system to receive the indicator data, analyze and process the indicator data according to the alarm rules, and output alarm information; The alarm processing module is used to trigger an alarm and perform alarm processing when the alarm information meets the alarm rules.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the configurable Doris system monitoring method as claimed in any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the configurable Doris system monitoring method according to any one of claims 1 to 7 are implemented.

Images