Cross-cluster infrastructure automatic monitoring method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a cross-cluster infrastructure automatic monitoring method and device, equipment and a storage medium. The method comprises the following steps: deploying a preset data acquisition component in each infrastructure of each cluster; acquiring operation data of each infrastructure of each cluster through a preset data acquisition component, and filling the operation data of the infrastructures into a preset monitoring template; performing standardization processing on the operation data in each preset monitoring template to obtain standardized data; and receiving the input operation data access request, and displaying the corresponding standardized data based on the display rule in the operation data access request, so that the stability, reliability and efficiency of system operation in a multi-cluster environment can be improved by monitoring the operation data of each infrastructure of each cluster under a cross-cluster condition.

Description

Technical Field

[0001] The present application relates to the field of big data analysis technology, and in particular, to a method, apparatus, computer device and storage medium for automated monitoring of cross-cluster infrastructure. Background Art

[0002] With the rapid development of cloud computing and big data technologies, more and more companies are starting to build multi-cluster environments to meet the high availability and scalability of their businesses. However, cross-cluster infrastructure management and monitoring has become a challenge.

[0003] The monitoring solutions in related technologies are often designed for a single cluster, and it is difficult to effectively cover multiple clusters and achieve unified management and monitoring. For example, there is a lack of a unified monitoring view, it is difficult to process and analyze massive data in real time, and there are problems such as inconsistent alarm strategies or complex configurations.

[0004] The monitoring schemes in related technologies may not update monitoring information in time due to low data collection frequency or processing delays, resulting in delayed fault detection and processing (results), and failure to respond quickly to system abnormalities, affecting service stability and user experience. Current monitoring may focus on core systems or key services, while ignoring other non-core but equally important infrastructure components. The lack of comprehensive coverage of multi-cluster and cross-region infrastructure may result in failure to timely discover and handle problems in certain regions or clusters. Existing monitoring may only focus on some basic performance indicators, such as CPU usage, memory usage, etc., while ignoring some key business indicators or user experience indicators. The setting of monitoring indicators may not be detailed enough to accurately reflect the actual operation status or potential problems of the system. The monitoring of infrastructure in a multi-cluster environment is independent of each other, making it difficult to conduct centralized and unified monitoring management, resulting in decentralized and inefficient operation and maintenance operations, making it difficult to conduct efficient global monitoring and troubleshooting, and failing to meet unified operation and maintenance requirements. Cross-cluster infrastructure may involve a variety of different operating systems, middleware, databases and other components. The complexity and diversity of these components pose great challenges to automated collection and monitoring. Summary of the invention

[0005] Embodiments of the present application provide a method, apparatus, computer device, and storage medium for automated monitoring of cross-cluster infrastructure.

[0006] A first aspect of an embodiment of the present application provides a method for automated monitoring of cross-cluster infrastructure, including:

[0007] Deploy preset data collection components in each infrastructure of each cluster;

[0008] The operation data of each infrastructure of each cluster is collected through the preset data collection component, and the operation data of the infrastructure is filled into the preset monitoring template;

[0009] Standardize the operation data in each preset monitoring template to obtain standardized data;

[0010] An input operation data access request is received, and corresponding standardized data is displayed based on a display rule in the operation data access request.

[0011] In an optional embodiment of the present application, the step of deploying a preset data collection component in each infrastructure of each cluster includes:

[0012] Building a central monitoring platform, wherein the central monitoring platform is used to centrally store, process and display data collected from each infrastructure of different clusters;

[0013] Select the corresponding preset data collection component according to the type of each infrastructure in each cluster, and ensure that the preset data collection component supports the automatic discovery mechanism.

[0014] In an optional embodiment of the present application, the selecting of a corresponding preset data collection component according to the type of each infrastructure of each cluster to ensure that the preset data collection component supports an automatic discovery mechanism includes:

[0015] According to the type of each infrastructure in each cluster, use the preset automated deployment tool to obtain the data collection components corresponding to the current infrastructure from the component database of the central monitoring platform;

[0016] Deploy the acquired data collection components to a corresponding infrastructure, and configure the infrastructure according to a configuration template corresponding to the infrastructure;

[0017] The data acquisition component deployed to the infrastructure is configured so that the data acquisition component is connected to the central monitoring platform, and corresponding data acquisition sources and data acquisition targets are set on the data acquisition component.

[0018] In an optional embodiment of the present application, the method further includes:

[0019] Configure the auto-discovery mechanism for each cluster;

[0020] Based on the automatic discovery mechanism, the infrastructure in the cluster is automatically discovered using a preset cluster management tool, and the discovered infrastructure information is stored in the configuration management database of the central monitoring platform.

[0021] In an optional embodiment of the present application, after automatically discovering the infrastructure in the cluster using the preset cluster management tool, the method further includes:

[0022] Determining whether a data collection component corresponding to the infrastructure has been deployed on the discovered infrastructure;

[0023] For infrastructures where data collection components have been deployed, the current infrastructure information is stored in the configuration management database of the central monitoring platform;

[0024] For infrastructures that have not deployed data collection components, the preset data collection components are first deployed in the current infrastructure, and then the information of the deployed infrastructure is stored in the configuration management database of the central monitoring platform.

[0025] In an optional embodiment of the present application, when the number of the same type of infrastructure in the same cluster exceeds one, the deploying of a preset data collection component in each cluster includes:

[0026] For the same type of infrastructure in the same cluster, if the physical locations of all infrastructures are distributed within a preset range, or the network conditions of the infrastructure do not meet the preset network requirements, a data acquisition server is set between the central monitoring platform and the infrastructure, and the data acquisition server is used to collect the operating data of all infrastructures and perform preliminary processing. All infrastructures are connected to the data acquisition server, and the data acquisition server is connected to the central monitoring platform, wherein data acquisition components are deployed on each infrastructure and data acquisition server;

[0027] If the distribution of the physical locations of all infrastructures exceeds the preset range, or the network conditions of the infrastructures meet the preset network requirements, a data acquisition server and a data acquisition component are deployed on each infrastructure. The data acquisition server is used to perform preliminary processing on the local data collected by the data acquisition component and send the processed data to the central monitoring platform.

[0028] In an optional embodiment of the present application, the operation data of all infrastructures are collected and preliminarily processed, including:

[0029] Obtain the operating data of all infrastructures and extract the feature vectors of the pre-processed operating data;

[0030] Build an operation data sensitivity scoring model to evaluate the sensitivity of operation data and classify it, and encrypt sensitive data based on the classification results;

[0031] The exchanged data is decrypted and stored as operating data, and a visual interface is built to display the operating data in real time.

[0032] A second aspect of an embodiment of the present application provides a cross-cluster infrastructure automated monitoring device, including:

[0033] A deployment module is used to deploy preset data collection components in each infrastructure of each cluster;

[0034] A filling module is used to collect the operation data of each infrastructure of each cluster through a preset data collection component, and fill the operation data of the infrastructure into a preset monitoring template;

[0035] The standardization module is used to standardize the operation data in each preset monitoring template to obtain standardized data;

[0036] The display module is used to receive an input operation data access request and display corresponding standardized data based on a display rule in the operation data access request.

[0037] According to a third aspect of an embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above methods for automated monitoring of cross-cluster infrastructure are implemented.

[0038] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any of the above methods for automated monitoring of cross-cluster infrastructure are implemented.

[0039] The above technical solution provided by the embodiment of the present application has at least some or all of the following advantages compared with the prior art:

[0040] The cross-cluster infrastructure automated monitoring method described in the embodiment of the present application deploys a preset data collection component in each infrastructure of each cluster; collects the operating data of each infrastructure of each cluster through the preset data collection component, and fills the operating data of the infrastructure into a preset monitoring template; standardizes the operating data in each preset monitoring template to obtain standardized data; receives an input operating data access request, and displays corresponding standardized data based on the display rules in the operating data access request. By monitoring the operating data of each infrastructure of each cluster under the cross-cluster, the stability, reliability and efficiency of the system operation in a multi-cluster environment can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 A flowchart of a method for automated monitoring of cross-cluster infrastructure provided by an embodiment of the present application;

[0043] Figure 2 A flowchart for collecting operation data of each infrastructure of each cluster provided in one embodiment of the present application;

[0044] Figure 3 A flowchart of a proxy configuration provided for an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of a cross-cluster infrastructure automation monitoring device provided by an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0047] See also Figure 1 The cross-cluster infrastructure automated monitoring method provided in the embodiment of the present application includes the following steps 100 to 400:

[0048] Step 100: deploying a preset data collection component in each infrastructure of each cluster;

[0049] Step 200: collect the operation data of each infrastructure of each cluster through a preset data collection component, and fill the operation data of the infrastructure into a preset monitoring template;

[0050] Step 300: Standardize the operation data in each preset monitoring template to obtain standardized data;

[0051] Step 400: Receive an input operation data access request, and display corresponding standardized data based on a display rule in the operation data access request.

[0052] In an optional embodiment of the present application, in step 100, the step of deploying a preset data collection component in each infrastructure of each cluster includes:

[0053] Building a central monitoring platform, wherein the central monitoring platform is used to centrally store, process and display data collected from each infrastructure of different clusters;

[0054] Select the corresponding preset data collection component according to the type of each infrastructure in each cluster, and ensure that the preset data collection component supports the automatic discovery mechanism.

[0055] In an optional embodiment of the present application, the selecting of a corresponding preset data collection component according to the type of each infrastructure of each cluster to ensure that the preset data collection component supports an automatic discovery mechanism includes:

[0056] According to the type of each infrastructure in each cluster, use the preset automated deployment tool to obtain the data collection components corresponding to the current infrastructure from the component database of the central monitoring platform;

[0057] Deploy the acquired data collection components to a corresponding infrastructure, and configure the infrastructure according to a configuration template corresponding to the infrastructure;

[0058] The data acquisition component deployed to the infrastructure is configured so that the data acquisition component is connected to the central monitoring platform, and corresponding data acquisition sources and data acquisition targets are set on the data acquisition component.

[0059] In an optional embodiment of the present application, the method further includes:

[0060] Configure the auto-discovery mechanism for each cluster;

[0061] Based on the automatic discovery mechanism, the infrastructure in the cluster is automatically discovered using a preset cluster management tool, and the discovered infrastructure information is stored in the configuration management database of the central monitoring platform.

[0062] In an optional embodiment of the present application, after automatically discovering the infrastructure in the cluster using the preset cluster management tool, the method further includes:

[0063] Determining whether a data collection component corresponding to the infrastructure has been deployed on the discovered infrastructure;

[0064] For infrastructures where data collection components have been deployed, the current infrastructure information is stored in the configuration management database of the central monitoring platform;

[0065] For infrastructures that have not deployed data collection components, the preset data collection components are first deployed in the current infrastructure, and then the information of the deployed infrastructure is stored in the configuration management database of the central monitoring platform.

[0066] In an optional embodiment of the present application, when the number of the same type of infrastructure in the same cluster exceeds one, the deploying of a preset data collection component in each cluster includes:

[0067] For the same type of infrastructure in the same cluster, if the physical locations of all infrastructures are distributed within a preset range, or the network conditions of the infrastructure do not meet the preset network requirements, a data acquisition server is set between the central monitoring platform and the infrastructure, and the data acquisition server is used to collect the operating data of all infrastructures and perform preliminary processing. All infrastructures are connected to the data acquisition server, and the data acquisition server is connected to the central monitoring platform, wherein data acquisition components are deployed on each infrastructure and data acquisition server;

[0068] If the distribution of the physical locations of all infrastructures exceeds the preset range, or the network conditions of the infrastructures meet the preset network requirements, a data acquisition server and a data acquisition component are deployed on each infrastructure. The data acquisition server is used to perform preliminary processing on the local data collected by the data acquisition component and send the processed data to the central monitoring platform.

[0069] In an optional embodiment of the present application, when there are a large number of the same type of infrastructure in the same cluster, how to deploy data collection components,

[0070] First, choose between centralized agent deployment and distributed deployment. Centralized agent deployment: If there are many infrastructures in the cluster, but the physical locations are relatively concentrated, or the network conditions are poor, you can consider using a centralized data acquisition server. The server is responsible for collecting data from all infrastructures and performing preliminary processing. This method may require the deployment of collection components on each infrastructure separately, and also on the agent server; distributed deployment: If the cluster is large, the infrastructure is widely distributed, and the network conditions are good, it is recommended to adopt a distributed deployment method. Deploy data collection agents (Agents) on each infrastructure or a group of infrastructures. These agents are responsible for the collection and preliminary processing of local data, and then send the data to the central server for aggregation and analysis. This method can improve the flexibility and scalability of data collection, while reducing dependence on network bandwidth.

[0071] Second, use standardized configurations to ensure that all infrastructures of the same type use the same configuration template. This can be achieved through configuration management. Standardized configurations help reduce problems caused by inconsistent configurations and simplify the update and maintenance process.

[0072] Third, automated deployment, using automated tools or scripts to automatically deploy data collection components. For example, in a K8s environment, you can use Helm charts (Helm charts is a packaging format for K8s resources that simplifies application deployment and management) to deploy data collection components; in other environments, you can use Ansible playbook (an automated script written in YAML for configuring, deploying, and orchestrating applications) or other similar automated tools.

[0073] Fourth, dynamic configuration. If the number of infrastructure in the cluster changes frequently (for example, in a cloud-native environment), you should consider using dynamic configuration methods to automatically discover newly added nodes and automatically install and configure data collection components on these nodes. This can be achieved through service discovery mechanisms (such as Consul, etcd, etc.).

[0074] Fifth, centralized management: For large-scale clusters, use a centralized configuration management and service registration system to centrally control the configuration and status of data collection components, which is convenient for monitoring and troubleshooting.

[0075] Sixth, resource isolation ensures that data acquisition components do not take up too many resources and affect the performance of the host or application. You can assign fixed resource limits to data acquisition components or use lighter data acquisition tools in resource-intensive applications.

[0076] In an optional embodiment of the present application, how to deploy data collection components on different infrastructures in the same cluster:

[0077] First, determine the type of infrastructure: clarify what types of infrastructure are included in the cluster, such as physical servers, virtual machines, database servers, network devices (switches, routers), storage devices, etc.

[0078] Second, analyze monitoring requirements: For each infrastructure, analyze its key performance indicators (KPIs) and monitoring requirements. For example, a physical server may need to monitor CPU usage, memory usage, disk I / O, etc.; a database server may need to monitor database query performance, number of connections, etc.; a network device may need to monitor network traffic, packet loss rate, etc.;

[0079] Third, distributed deployment: Deploy dedicated data collection agents or lightweight monitoring services on each infrastructure. These agents or services will be responsible for collecting data from their respective infrastructures and sending them to the central monitoring server for centralized processing;

[0080] Fourth, centralized processing and analysis: Establish a central monitoring server to receive data from various infrastructures and conduct unified processing, analysis and storage. The central monitoring server should have strong data processing capabilities and high availability to ensure the real-time and accuracy of data.

[0081] Fifth, security and permission management: Implement strict security measures to ensure data transmission and storage security during data collection. Set reasonable permission management strategies to limit access to data collection components and monitoring data.

[0082] In an optional embodiment of the present application, for infrastructures in different clusters, data collection components are deployed through the following steps:

[0083] First, build a central monitoring platform (agent gateway): Select or build a central monitoring platform to centrally store, process, and display collected data. Ensure that the monitoring platform supports multi-tenancy, permission management, and scalability to manage data from multiple clusters.

[0084] Second, data collection component deployment: select appropriate data collection components according to the infrastructure type and ensure that the data collection components support automatic discovery and configuration.

[0085] a. Based on the infrastructure information, use automated deployment tools to pull the image or software package of the data collection component from the central warehouse.

[0086] b. Deploy the data collection components to the corresponding infrastructure and configure them according to the configuration template.

[0087] c. Configure the data collection component to connect to the central monitoring platform (agent gateway) and set the correct data collection source and target.

[0088] Third, automated collection implementation: Use the cloud service provider’s cluster management tools to automatically discover the infrastructure in the cluster, and store the discovered infrastructure information (such as IP address, port, type, etc.) in a configuration management database or key-value storage.

[0089] In an optional embodiment of the present application, the operation data of all infrastructures are collected and preliminarily processed, including:

[0090] Obtain the operating data of all infrastructures and extract the feature vectors of the pre-processed operating data;

[0091] Build an operation data sensitivity scoring model to evaluate the sensitivity of operation data and classify it, and encrypt sensitive data based on the classification results;

[0092] The exchanged data is decrypted and stored as operating data, and a visual interface is built to display the operating data in real time.

[0093] In an optional embodiment of the present application, a feature vector of the preprocessed running data is extracted by presetting an autoencoder network model.

[0094] In an optional embodiment of the present application, the construction of the application system information data sensitivity scoring model to evaluate the application system information data sensitivity and classify it refers to collecting historical application system information data and extracting historical feature vectors, calculating the mean of the historical feature vectors and setting them as the benchmark data vector;

[0095] Collect application system information data in real time and extract real-time feature vectors. Use the K-means clustering analysis algorithm to cluster the real-time feature vectors, and select the center point of each cluster as the reference data vector.

[0096] Combining the RBF kernel function with the integral, the cumulative similarity A(x) between the benchmark data vector and the feature vector of the application system information data is calculated. The formula is:

[0097]

[0098] Where x is the feature vector of application system information data, x0 is the reference data vector, and x i is the historical feature vector of the i-th application system information data;

[0099] Perform logarithmic transformation B(x) on the accumulated similarity A, the formula is:

[0100] B(x)=log(1+A(x));

[0101] The hyperbolic tangent function is introduced to smooth the accumulated similarity of the reference feature vector to obtain the accumulated similarity C(x) after smoothing. The formula is:

[0102]

[0103] Where M is the number of reference eigenvectors, x j is the jth reference eigenvector;

[0104] A sensitivity scoring model is constructed to evaluate the sensitivity score S(x) of the feature vector of application system information data. The formula is:

[0105]

[0106] Collect the sensitivity scores of historical application system information data and set an evaluation threshold. Compare the sensitivity scores of the application system information data feature vectors with the evaluation threshold. If the sensitivity score of the application system information data feature vector is greater than and equal to the evaluation threshold, it is determined to be sensitive data. If the sensitivity score of the application system information data feature vector is less than the evaluation threshold, it is determined to be ordinary data.

[0107] In an optional embodiment of the present application, encrypting the sensitive data of the application system information data based on the classification result refers to randomly generating a key K using a random number generator;

[0108] Use exponential function and sine function to perform nonlinear transformation on sensitive data to obtain the nonlinear transformation result H(x′), the formula is:

[0109]

[0110] Where x′ is sensitive data;

[0111] The sensitive data is transformed nonlinearly using logarithmic function and smooth inverse tangent function to obtain the multi-level nonlinear transformation result E(x′), the formula is:

[0112] E(x')=tan -1 (x' 2 k+log(x'+k));

[0113] Construct the encryption formula:

[0114] S(x′)=H(x′)+E(x′);

[0115] Bring sensitive data into the encryption formula for encryption.

[0116] In an optional embodiment of the present application, decrypting the exchanged data to store the application system information data refers to receiving the transmitted encrypted data and the corresponding key K;

[0117] Use the decryption formula to decrypt the received data and obtain the decrypted sensitive data O(S). The formula is:

[0118]

[0119] The collected application system information and sensitive data generated by the analysis are stored in the database, the application system information and sensitive data generated by the analysis in the database are backed up in the cloud, and the backup data are regularly checked for integrity.

[0120] In an optional embodiment of the present application, see Figure 2In step 200, the operation data of each infrastructure of each cluster is collected by a preset data collection component, including:

[0121] Automatically collect the operation data of each infrastructure in each cluster through preset data collection components;

[0122] According to the correspondence between the operation data type and the collection type in the preset collection management, the collection type of the collected operation data is determined, wherein the collection type includes real-time data collection, timed data collection and event-triggered data collection,

[0123] Real-time data collection is one of the main collection methods of the platform. It uses efficient data transmission protocols to ensure the accuracy and real-time nature of the data, and collects the operation data of the infrastructure in real time through the agent nodes deployed in each cluster. These data include but are not limited to key indicators such as server load, network bandwidth utilization, and health status of storage devices of infrastructure such as server hardware, network devices, storage area network storage devices, cloud services, and power supplies.

[0124] Scheduled data collection is an important means for the platform to periodically monitor the infrastructure. According to the preset time interval (such as hourly, daily, weekly, etc.), the agent node will automatically trigger the data collection task to collect the operation data within a specific time period. This method is suitable for those data items that do not need real-time monitoring but are periodically concerned.

[0125] When a specific event occurs in the infrastructure (such as equipment failure, network interruption, etc.), the platform will trigger the event collection mechanism. The agent node will immediately collect data related to the event and send the data to the central server for further processing and analysis. This method can ensure that relevant data can be quickly obtained when a critical event occurs, providing support for rapid response and fault handling.

[0126] Collection management can configure collection objects, collect various local client data through dedicated proxy plug-ins, simple network management protocol, Java management extension, intelligent platform management interface and other technologies, send them to the server, and store the data in the database. Users can see the monitoring data based on the front-end page. Collection management can directly use monitoring templates, and indicator data has been defined in the templates. When adding collection data, you can define whether to open asset records, which will provide basic data for asset management, integrate business and monitoring, and avoid the problem of mismatch between business data and monitoring object data.

[0127] In an optional embodiment of the present application, in step 300, the operation data in each preset monitoring template is standardized to obtain standardized data, including:

[0128] The operating data in each preset monitoring template (such as CPU usage, memory usage, network traffic, disk I / O, etc.) is integrated into a unified format through standardization processing, which is used as standardized data.

[0129] In an optional embodiment of the present application, before displaying the corresponding standardized data based on the display rule in the running data access request, the method further includes:

[0130] Build a unified visual monitoring interface for users to view and manage the status of all clusters through a single portal, including resource usage, anomaly detection, historical trend analysis, etc., to quickly grasp the overall operation status and identify potential problem areas;

[0131] Customized visualization instrument view, using data visualization technology, displays data to users in the form of curve graphs, bar graphs, pie charts, etc., so that users can intuitively understand the distribution and change trend of data. It allows users to customize monitoring views according to their needs, including selecting which indicators to display and how to layout charts, etc., to meet users' personalized monitoring needs;

[0132] Monitoring data query allows you to intuitively and simply view the indicator data of the monitored object, and update the latest data in real time. It also provides charts and historical data for viewing, traces the cause of the alarm and analyzes it. Through data, you can understand the utilization of existing resources, discover resource bottlenecks and waste, and thus optimize resource allocation and improve resource utilization efficiency.

[0133] In an optional embodiment of the present application, the method further includes:

[0134] Update the status information of each cluster in real time to ensure the consistency and accuracy of the monitored operation data;

[0135] Automatically adjust resource allocation according to business needs and cluster resource conditions to improve resource utilization;

[0136] When a failure occurs, quickly locate the cause of the problem and initiate appropriate recovery measures.

[0137] In an optional embodiment of the present application, the method further includes:

[0138] Draw a topology diagram for the visual network structure of each infrastructure in each cluster, graphically displaying each node in the network structure and the connection relationship between them, making the network structure and connection method clear at a glance;

[0139] Analyze and evaluate network performance through topology diagrams to help identify possible bottlenecks or failure points;

[0140] Plan network expansion with topology diagrams to ensure new devices and connections can be effectively integrated into the existing network structure.

[0141] When a network problem occurs, the topology map can be used to quickly identify the root cause of the problem, such as hardware or software failure, thereby speeding up problem resolution.

[0142] Simplify the configuration process through topology diagrams, which make it easier to understand and remember the configuration of network components through intuitive graphical forms.

[0143] In an optional embodiment of the present application, the method further includes:

[0144] Provide alarm rule setting services, such as triggering an alarm when a certain indicator exceeds or falls below a certain threshold;

[0145] Once an abnormal situation is detected, the alarm system will quickly issue an alarm and notify the operation and maintenance personnel through various means such as email station reminders and text messages. The problem can be quickly located and diagnosed through detailed alarm information.

[0146] Save historical alarm information for operation and maintenance personnel to query and analyze. By analyzing historical alarm data, potential problems in the system can be discovered and prevented and optimized in advance.

[0147] In an optional embodiment of the present application, see Figure 3 , the method further comprises:

[0148] A unified configuration interface is defined through agent configuration management to receive configuration requests from users. These interfaces include but are not limited to: basic information configuration of agent nodes (such as IP address, port number, operating system type, etc.), data collection configuration (such as collection items, collection frequency, data format, etc.), alarm policy configuration (such as alarm threshold, alarm method, etc.), etc. By defining a unified interface, unified configuration of agent nodes in different clusters, different operating systems, and different network environments can be achieved.

[0149] The automated deployment tool of the agent configuration management is used to realize the automated deployment of the agent node. After receiving the configuration request from the user, the module will generate the corresponding deployment script according to the parameters defined in the configuration interface, and push the script to the target agent node for execution through the automated deployment tool. During the deployment process, the module will monitor the deployment progress and results in real time to ensure that the agent node can be successfully deployed and started.

[0150] The following is an application example of the cross-cluster infrastructure automated monitoring method of the present application, comprising the following steps:

[0151] The first step is to deploy data collection components: deploy data collection components in each cluster and configure corresponding collection rules and parameters. According to the type, environment and monitoring requirements of the cluster, select appropriate data collection components and formulate deployment strategies for data collection components to ensure the efficiency and stability of data collection. Data collection components usually refer to software tools or service modules designed to automatically collect monitoring data from cross-cluster infrastructure (such as physical servers, virtual machines, containers, databases, network devices, etc.). These components are the data source of the monitoring platform and are responsible for collecting, extracting and forwarding various monitoring indicators and log information, and converting them into formats that can be used for analysis, alarms and visualization. These components are usually lightweight agents that can run on different computing nodes (such as servers, virtual machines or containers) and can communicate with the backend of the monitoring system. Non-agent components can connect to servers through protocols to collect data from infrastructure. Through these protocols, infrastructure data collection can be achieved without installing additional management software on servers or other network devices. This method simplifies the management process, improves efficiency, and reduces maintenance costs. At the same time, these protocols also provide interoperability for devices between different manufacturers, allowing administrators to use unified tools to manage the entire network environment.

[0152] The second step is to configure the automatic discovery mechanism: To ensure that the platform can discover the infrastructure in the cluster in real time, you need to configure the automatic discovery mechanism. Set the scanning cycle and scanning range of the automatic discovery mechanism to ensure that the platform can discover the infrastructure in the cluster in real time.

[0153] Step 3: Start data collection: After the data collection component and automatic discovery mechanism are configured, the data collection process can be started. Start the data collection component to start collecting monitoring data from each cluster, ensure that the component can operate normally and start collecting monitoring data, and send the data to the central storage system.

[0154] Step 4: Configure data analysis and display components: Configure the display rules and styles of data analysis and display components according to user needs. Clarify the display requirements and monitoring goals, including the monitoring indicators to be displayed, chart types, and the time range of data display, and configure the display rules of data analysis and display components.

[0155] Step 5: User access and monitoring: Users can view the operating status and performance indicators of each cluster through a unified monitoring view. The platform also provides an alarm function. When the monitoring data exceeds the preset threshold, an alarm will be automatically triggered to notify the user.

[0156] The cross-cluster infrastructure automated monitoring method of the present application has a unified monitoring architecture for cluster infrastructures in different regions, different cloud environments, or different technology stacks, solving the problem that traditional monitoring systems are difficult to monitor multi-cluster resources from a global perspective, and realizing automated collection of cross-regional and cross-cluster hybrid infrastructures. Regardless of where the infrastructure is located or whether it belongs to different clusters or platforms, the platform can perform unified and automated data collection, greatly improving the efficiency and scope of monitoring, and automatically adjusting the data collection frequency and resource allocation according to the cluster load, ensuring that the monitoring system itself does not become a burden on the cluster, while optimizing the overall monitoring efficiency.

[0157] The cross-cluster infrastructure automated monitoring method of the present application automates the monitoring of cross-regional and cross-cluster hybrid infrastructure, covering multiple types of monitoring data indicators such as server hardware, network equipment, storage area network storage equipment, cloud services, power supply, etc., to achieve comprehensive, efficient and automated monitoring management, understand the utilization of existing resources, discover resource bottlenecks and waste, thereby optimizing resource allocation, improving resource utilization efficiency, and greatly improving the monitoring scope and efficiency. The comprehensive monitoring capability ensures that no matter where the infrastructure is distributed, it can be effectively included in the monitoring scope, greatly improving the efficiency and accuracy of monitoring. Through automated data collection and monitoring, the manual participation links are significantly reduced, and the workload of operation and maintenance personnel is reduced. The unified visual interface enables the operation and maintenance team to quickly identify problems and understand the health status of the cluster, so as to respond faster and improve operation and maintenance efficiency. It can instantly discover performance bottlenecks and abnormal behaviors, automatically trigger early warnings, and help the operation and maintenance team take action before problems affect the business, reduce the impact of failures, and improve service quality.

[0158] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0159] See also Figure 4 An embodiment of the present application provides a cross-cluster infrastructure automated monitoring device 400, including:

[0160] A deployment module 410, for deploying a preset data collection component in each infrastructure of each cluster;

[0161] A filling module 420 is used to collect the operation data of each infrastructure of each cluster through a preset data collection component, and fill the operation data of the infrastructure into a preset monitoring template;

[0162] The standardization module 430 is used to perform standardization processing on the operation data in each preset monitoring template to obtain standardized data;

[0163] The display module 440 is used to receive an input operation data access request, and display corresponding standardized data based on a display rule in the operation data access request.

[0164] For the specific definition of the above-mentioned device 400, please refer to the definition of the pseudo-random sequence generation method above, which will not be repeated here. Each module in the above-mentioned device 400 can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0165] The cross-cluster infrastructure automated monitoring device of the present application solves the problem of infrastructure management and monitoring in a multi-cluster environment. Through automated collection mechanisms, rich monitoring indicators, data processing and analysis, and cross-cluster collaborative management, it achieves comprehensive monitoring of infrastructure such as servers, networks, and storage in the cluster. It also provides accurate early warnings and fault handling suggestions through intelligent analysis algorithms, greatly improving the operation and maintenance efficiency and stability of the cluster, providing users with efficient and reliable monitoring services, and ensuring the stable operation of the business.

[0166] The cross-cluster infrastructure automated monitoring device of the present application designs corresponding monitoring strategies and collection solutions for different types of components, comprehensively covers all objects that need to be monitored, realizes real-time collection and efficient transmission of cross-cluster data, and ensures the timeliness and accuracy of monitoring data; at the same time, by expanding the monitoring scope, covering all key infrastructure components, and eliminating monitoring blind spots; in the setting of monitoring indicators, performance indicators, business indicators and user experience indicators will be comprehensively considered to ensure the comprehensiveness and richness of monitoring data, simplify the configuration and management process of the monitoring platform through unified management and intelligent analysis functions, improve the intelligence level of monitoring, and solve the complexity and diversity of cross-cluster infrastructure by optimizing system architecture and data processing algorithms, ensuring the stability and scalability of the monitoring platform.

[0167] The cross-cluster infrastructure automated monitoring device of this application improves data collection efficiency: the use of automated collection technology can achieve fast and accurate data capture of multiple cluster infrastructures. Through intelligent algorithms and efficient data processing mechanisms, the platform can significantly reduce manual intervention, reduce data collection costs, and improve data accuracy and real-time performance; achieve comprehensive monitoring: not only can it monitor traditional infrastructure indicators such as CPU, memory, disk usage, etc., but it can also comprehensively monitor network status, application performance, security events, etc. This cross-cluster and cross-level monitoring capability enables a comprehensive grasp of the operating status of the entire system, and timely discovers and handles potential problems; enhances cross-cluster data consistency and integration capabilities: unified data models and protocols ensure that data from different clusters can be collected, integrated and analyzed under unified standards, providing a global perspective monitoring view, which is convenient for operation and maintenance personnel to fully understand and manage the status of the entire distributed system; real-time warning and fault location: with powerful data analysis capabilities, it can monitor the operating status of the infrastructure in real time and trigger warnings according to preset thresholds or rules. Once a failure or abnormal situation occurs, the problem can be quickly located, and detailed fault information and handling suggestions can be provided to users, thereby shortening the fault recovery time and reducing the risk of system downtime; Visual management and operation: Provides a rich visual interface and tools, allowing users to intuitively understand the operating status of the cluster infrastructure. At the same time, it also supports custom monitoring items, alarm rules, etc. to meet the personalized needs of different users. In addition, through visual operations, administrators can configure, manage and optimize the system more conveniently; Flexible expansion and integration: Taking into account the actual needs and scenario differences of different users, the platform has designed a flexible expansion mechanism that can easily integrate third-party monitoring tools or systems. In addition, it also supports multi-tenant mode, which can meet the resource sharing and isolation needs between different organizations or departments.

[0168] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a pseudo-random sequence generation method as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the pseudo-random sequence generation method as described above is implemented.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above pseudo-random sequence generation method can be implemented.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0175] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for automated monitoring of cross-cluster infrastructure, characterized in that: include: Deploy preset data collection components in each infrastructure of each cluster; The operation data of each infrastructure of each cluster is collected through the preset data collection component, and the operation data of the infrastructure is filled into the preset monitoring template; Standardize the operation data in each preset monitoring template to obtain standardized data; An input operation data access request is received, and corresponding standardized data is displayed based on a display rule in the operation data access request.

2. The method according to claim 1, characterized in that The deployment of preset data collection components in each infrastructure of each cluster includes: Building a central monitoring platform, wherein the central monitoring platform is used to centrally store, process and display data collected from each infrastructure of different clusters; Select the corresponding preset data collection component according to the type of each infrastructure in each cluster, and ensure that the preset data collection component supports the automatic discovery mechanism.

3. The method according to claim 2, characterized in that The method of selecting a corresponding preset data collection component according to the type of each infrastructure of each cluster to ensure that the preset data collection component supports the automatic discovery mechanism includes: According to the type of each infrastructure in each cluster, use the preset automated deployment tool to obtain the data collection components corresponding to the current infrastructure from the component database of the central monitoring platform; Deploy the acquired data collection components to a corresponding infrastructure, and configure the infrastructure according to a configuration template corresponding to the infrastructure; The data acquisition component deployed to the infrastructure is configured so that the data acquisition component is connected to the central monitoring platform, and corresponding data acquisition sources and data acquisition targets are set on the data acquisition component.

4. The method according to claim 2, characterized in that: The method further comprises: Configure the auto-discovery mechanism for each cluster; Based on the automatic discovery mechanism, the infrastructure in the cluster is automatically discovered using a preset cluster management tool, and the discovered infrastructure information is stored in the configuration management database of the central monitoring platform.

5. The method according to claim 4, characterized in that After automatically discovering the infrastructure in the cluster using the preset cluster management tool, the method further includes: Determining whether a data collection component corresponding to the infrastructure has been deployed on the discovered infrastructure; For infrastructures where data collection components have been deployed, the current infrastructure information is stored in the configuration management database of the central monitoring platform; For infrastructures that have not deployed data collection components, the preset data collection components are first deployed in the current infrastructure, and then the information of the deployed infrastructure is stored in the configuration management database of the central monitoring platform.

6. The method according to claim 2, characterized in that When there are more than one infrastructure of the same type in the same cluster, the deployment of a preset data collection component in each cluster includes: For the same type of infrastructure in the same cluster, if the physical locations of all infrastructures are distributed within a preset range, or the network conditions of the infrastructure do not meet the preset network requirements, a data acquisition server is set between the central monitoring platform and the infrastructure, and the data acquisition server is used to collect the operating data of all infrastructures and perform preliminary processing. All infrastructures are connected to the data acquisition server, and the data acquisition server is connected to the central monitoring platform, wherein data acquisition components are deployed on each infrastructure and data acquisition server; If the distribution of the physical locations of all infrastructures exceeds the preset range, or the network conditions of the infrastructures meet the preset network requirements, a data acquisition server and a data acquisition component are deployed on each infrastructure. The data acquisition server is used to perform preliminary processing on the local data collected by the data acquisition component and send the processed data to the central monitoring platform.

7. The method according to claim 6, characterized in that The operation data of all infrastructures are collected and processed initially, including: Obtain the operating data of all infrastructures and extract the feature vectors of the pre-processed operating data; Build an operation data sensitivity scoring model to evaluate the sensitivity of operation data and classify it, and encrypt sensitive data based on the classification results; The exchanged data is decrypted and stored as operating data, and a visual interface is built to display the operating data in real time.

8. A cross-cluster infrastructure automated monitoring device, characterized in that: include: A deployment module is used to deploy preset data collection components in each infrastructure of each cluster; A filling module is used to collect the operation data of each infrastructure of each cluster through a preset data collection component, and fill the operation data of the infrastructure into a preset monitoring template; The standardization module is used to standardize the operation data in each preset monitoring template to obtain standardized data; The display module is used to receive an input operation data access request and display corresponding standardized data based on a display rule in the operation data access request.

9. A computer device comprising: It comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the cross-cluster infrastructure automatic monitoring method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cross-cluster infrastructure automated monitoring method according to any one of claims 1 to 7 are implemented.

Images