Multi-cloud monitoring method and system, electronic device and computer-readable storage medium

Abstract

Disclosed in embodiments of the present application are a multi-cloud monitoring method and system, an electronic device, and a computer-readable storage medium. The method comprises: acquiring account information of a plurality of cloud accounts in a plurality of cloud platforms, and acquiring instance information of cloud services provided by the cloud platforms for the cloud accounts; and when a monitoring tag in the instance information represents monitoring a target cloud account, acquiring monitoring data of a cloud service corresponding to the target cloud account. In this way, unified supervision can be performed on a plurality of cloud platforms.

Description

Multi-cloud monitoring methods, systems, electronic devices, and computer-readable storage media

[0001] This application claims priority to Chinese patent application filed on January 2, 2025, application number 2025100059249, entitled "Monitoring Method, System, Electronic Device and Computer-Readable Storage Medium for Multi-Cloud Applications", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of cloud service technology, specifically to a multi-cloud monitoring method, system, electronic device, and computer-readable storage medium. Background Technology

[0003] With the development of cloud computing, many cloud computing platforms have emerged, and in some large-scale businesses, multi-cloud hybrid scenarios have appeared to support business systems. Technical issues

[0004] However, the relevant technologies only support monitoring a single cloud account on a single cloud platform, and each cloud platform has its own monitoring methods, making it impossible to uniformly supervise multiple cloud platforms, resulting in scattered monitoring data. Technical solutions

[0005] This application provides a multi-cloud monitoring method, system, electronic device, and computer-readable storage medium, which can uniformly supervise multi-cloud platforms and identify target cloud accounts that need to be monitored through monitoring tags, thereby saving monitoring resources by monitoring only the target cloud accounts.

[0006] In a first aspect, embodiments of this application provide a multi-cloud monitoring method, including:

[0007] Obtain account information for multiple cloud accounts across multiple cloud platforms, and obtain instance information for cloud services provided by the cloud platforms to the cloud accounts;

[0008] The monitoring tag in the instance information indicates that when monitoring the target cloud account, monitoring data of the cloud service corresponding to the target cloud account is obtained.

[0009] In one embodiment, the monitoring data is obtained by calling a cloud monitoring interface; the method further includes:

[0010] In response to the call request from the cloud monitoring interface, update the number of requests and the request time stored in the database;

[0011] The script checks the query rate per second based on the number of requests and the request time stored in the database.

[0012] The call request is rejected when the number of requests and the request time exceed the query rate limit per second.

[0013] In one embodiment, obtaining account information for multiple cloud accounts across multiple cloud platforms includes:

[0014] The configuration management database is invoked, and account information of multiple cloud accounts in multiple cloud platforms is collected according to the configuration fields of the configuration management database.

[0015] In one embodiment, obtaining instance information of cloud services provided by the cloud platform for the cloud account includes:

[0016] Based on the account information of the cloud account, the instance interface of the cloud service provided by the cloud platform for the cloud account is called to obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

[0017] In one embodiment, obtaining instance information of cloud services provided by the cloud platform for the cloud account includes:

[0018] Start monitoring threads for each of the aforementioned cloud accounts;

[0019] Based on the monitoring thread, obtain instance information of the cloud services provided by the cloud platform for the cloud account.

[0020] In one embodiment, when the monitoring tag in the instance information indicates that the target cloud account is being monitored, obtaining the monitoring data of the cloud service corresponding to the target cloud account includes:

[0021] When the monitoring tag indicates that the target cloud account is being monitored, the structure required to call the cloud monitoring interface of the cloud platform is assembled according to the calling rules of the cloud platform where the target cloud account is located.

[0022] Based on the structure, the cloud monitoring interface of the cloud platform is called to obtain the monitoring data.

[0023] In one embodiment, obtaining the monitoring data of the cloud service corresponding to the target cloud account includes:

[0024] Obtain the configuration file for monitoring metrics of the cloud service corresponding to the target cloud account; the configuration file includes the monitoring metrics that need to be monitored.

[0025] Read the monitoring metrics of the cloud service corresponding to the target cloud account from the configuration file;

[0026] Obtain the monitoring data corresponding to the monitoring indicators from the various monitoring data of the cloud service corresponding to the target cloud account.

[0027] In one embodiment, obtaining the monitoring data of the cloud service corresponding to the target cloud account includes:

[0028] Call the cloud monitoring interface of the cloud platform to obtain the monitoring data to be processed returned by the cloud monitoring interface;

[0029] The monitoring data to be processed is assembled into target monitoring data that can be identified by a multi-cloud monitoring system;

[0030] The monitoring data capture job configured based on the multi-cloud monitoring system captures the target monitoring data.

[0031] Secondly, embodiments of this application provide a multi-cloud monitoring system, including:

[0032] The information acquisition module is used to acquire account information of multiple cloud accounts in multiple cloud platforms, and to acquire instance information of cloud services provided by the cloud platforms to the cloud accounts;

[0033] The data acquisition module is used to acquire monitoring data of the cloud service corresponding to the target cloud account when the monitoring tag in the instance information represents monitoring of the target cloud account.

[0034] In one embodiment, the monitoring data is obtained by calling a cloud monitoring interface; the system further includes:

[0035] The update module is used to update the number of requests and the request time stored in the database in response to the call request of the cloud monitoring interface;

[0036] The inspection module is used to check the query rate per second based on the number of requests and the request time stored in the database, using a script.

[0037] The rejection module is used to reject the call request when the number of requests and the request time exceed the query rate limit per second.

[0038] In one embodiment, the information acquisition module includes:

[0039] The data collection unit is used to call the configuration management database and collect account information of multiple cloud accounts in multiple cloud platforms according to the configuration fields of the configuration management database.

[0040] In one embodiment, the information acquisition module includes:

[0041] The instance interface calling unit is used to call the instance interface of the cloud service provided by the cloud platform for the cloud account based on the account information of the cloud account, so as to obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

[0042] In one embodiment, the information acquisition module includes:

[0043] The startup unit is used to start monitoring threads for each of the cloud accounts.

[0044] The information acquisition unit is used to acquire instance information of the cloud service provided by the cloud platform to the cloud account based on the monitoring thread.

[0045] In one embodiment, the data acquisition module includes:

[0046] The structure assembly unit is used to assemble the structure required to call the cloud monitoring interface of the cloud platform according to the calling rules of the cloud platform where the target cloud account is located when the monitoring tag represents monitoring the target cloud account;

[0047] The interface calling unit is used to call the cloud monitoring interface of the cloud platform based on the structure to obtain the monitoring data.

[0048] In one embodiment, the data acquisition module includes:

[0049] The file acquisition unit is used to acquire the configuration file of the monitoring metrics of the cloud service corresponding to the target cloud account; the configuration file includes the monitoring metrics that need to be monitored.

[0050] The reading unit is used to read the monitoring metrics of the cloud service corresponding to the target cloud account from the configuration file;

[0051] The monitoring data acquisition unit is used to acquire the monitoring data corresponding to the monitoring indicators from the various monitoring data of the cloud service corresponding to the target cloud account.

[0052] In one embodiment, the data acquisition module includes:

[0053] The data acquisition unit is used to call the cloud monitoring interface of the cloud platform to obtain the monitoring data to be processed returned by the cloud monitoring interface.

[0054] A data assembly unit is used to assemble the monitoring data to be processed into target monitoring data that can be identified by a multi-cloud monitoring system.

[0055] A data capture unit is used to capture monitoring data based on the multi-cloud monitoring system configuration, and to capture the target monitoring data. Thirdly, embodiments of this application also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps in the aforementioned multi-cloud monitoring method.

[0056] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the aforementioned multi-cloud monitoring method.

[0057] Fifthly, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described in embodiments of this application. Beneficial effects

[0058] In summary, this embodiment of the application can obtain account information for multiple cloud accounts across multiple cloud platforms, as well as instance information of cloud services provided by each cloud platform to each cloud account. Based on the monitoring tags in the instance information, it can be determined whether to monitor the cloud account. When the monitoring tag indicates that the cloud account is to be monitored, the cloud monitoring interface of the cloud platform where the cloud account resides can be called to perform monitoring, thereby obtaining monitoring data. In this way, unified monitoring of multiple cloud accounts across multiple cloud platforms is achieved, reducing the maintenance cost of cloud account monitoring. Furthermore, the target cloud account to be monitored can be identified through monitoring tags, and monitoring is only performed on the target cloud account, thus filtering out cloud accounts that do not need to be monitored and saving monitoring resources. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 is a schematic diagram of the steps of a multi-cloud monitoring method provided in an embodiment of this application;

[0061] Figure 2 is a schematic diagram of another step of the multi-cloud monitoring method provided in an embodiment of this application;

[0062] Figure 3 is a topology diagram of a multi-cloud monitoring method provided in an embodiment of this application;

[0063] Figure 4 is a schematic diagram of the structure of a multi-cloud monitoring system provided in an embodiment of this application;

[0064] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Embodiments of the present invention

[0065] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0066] The terms used in the embodiments of this application are explained below.

[0067] Exporter: A software tool used to export data from different systems or services to a monitoring system, enabling the monitoring system to collect and analyze data from various sources. Exporter can be used to acquire data from various cloud platforms in a multi-cloud scenario. Prometheus: An open-source system monitoring and alerting toolkit primarily used for monitoring cloud-native applications and infrastructure. Prometheus can collect real-time metric data from multiple sources and is easy to use, scalable, and customizable. One embodiment of this application uses Prometheus to build a multi-cloud monitoring system.

[0068] Multi-cloud refers to using cloud services from multiple cloud platforms, each with different cloud service providers. Multi-cloud allows for cost optimization by selecting services from different providers. However, multi-cloud environments are complex, management is difficult, and integrating data across different cloud platforms presents significant challenges.

[0069] Cloud monitoring interface: An open interface provided by the cloud platform that exposes managed cloud service monitoring data. It allows users and third-party applications to access and manipulate cloud monitoring data, such as metrics and logs. Based on the cloud monitoring interface, monitoring data can be accessed and analyzed. The cloud monitoring interface can be integrated with third-party tools, enabling organizations to correlate monitoring data with other systems.

[0070] Python: A programming language.

[0071] Prometheus-client: A library for generating and exporting data in applications. It can export data in Prometheus format so that the data can be collected and processed by Prometheus.

[0072] Configuration Management Database (CMDB): A platform used to store all components and their relationships within an information technology (IT) infrastructure. CMDB can automatically discover and record IT assets, model the dependencies and connectivity between IT assets, and track and manage changes occurring within the IT infrastructure.

[0073] Example: In object-oriented programming, a class is an abstract data type. A class defines the attributes and behaviors of an object, and an instance is a concrete object created based on a class.

[0074] In one embodiment, as shown in Figure 1, a multi-cloud monitoring method is provided. Although the logical order is shown in the schematic diagram, in some cases, the steps shown or described can be performed in a different order than that shown in the figures. Specifically, this multi-cloud monitoring method can be applied to a server, which can be a physical server or a cloud server providing various cloud services. It is worth noting that this application does not limit the number of servers; any number of servers can be used depending on implementation needs. For example, a server can be a single server or a server cluster composed of multiple servers, etc.

[0075] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0076] According to the multi-cloud monitoring method shown in Figure 1, the method includes at least steps S110 to S120, which are described in detail below: In step S110, account information of multiple cloud accounts in multiple cloud platforms is obtained, as well as instance information of cloud services provided by the cloud platform for the cloud account is obtained.

[0077] In step S120, when the monitoring tag in the instance information indicates that the target cloud account is being monitored, the monitoring data of the cloud service corresponding to the target cloud account is obtained.

[0078] For multiple cloud accounts in multiple regions within a multi-cloud scenario, it is possible to obtain the account information for each cloud account on each cloud platform. A cloud account is a user's identity credential used to access and manage cloud services. Users log in to the cloud service platform using their cloud account to utilize various cloud service resources. Cloud account information includes, but is not limited to: username, account ID, region, access key, secret key (AKSK), and whether monitoring is enabled on the cloud platform.

[0079] Cloud service providers may divide their infrastructure into different regions for operation to reduce latency, improve availability, and enhance fault tolerance. Monitoring requirements for cloud accounts may differ across regions, thus necessitating the determination of the region to which a cloud account belongs. Whether monitoring is enabled on the cloud platform hosting the cloud account can be pre-configured by the user based on actual needs. If the cloud account information indicates that monitoring is not enabled on the cloud platform hosting the cloud account, then monitoring of all cloud accounts on that platform is unnecessary, saving resources.

[0080] Based on the obtained cloud account information, the instance interface for calling cloud services provided by the cloud platform for the cloud account can be retrieved. The Software Development Kit (SDK) can be found in the official documentation of the cloud platform. The SDK encapsulates the Application Programming Interface (API) of the cloud services. The specific name, parameters, and usage methods of the instance interface for calling cloud services provided by the cloud platform for the cloud account can be obtained from the SDK. After authentication using the cloud account information, calling the instance interface for the cloud services provided by the cloud platform for the cloud account will return the instance information.

[0081] The instance interface defines how the cloud platform interacts with the cloud account; the instance message of the cloud service provided by the cloud platform to the cloud account is the result returned by calling the instance interface, which includes the configuration information and running status of the cloud service.

[0082] Cloud services are services provided by cloud platforms via the Internet, including various functions such as computing power, storage, and database management. Specifically, they may include, but are not limited to: SaaS (Software as a Service), IaaS (Infrastructure as a Service), PaaS (Platform as a Service), and / or DaaS (Data as a Service).

[0083] The cloud platform provides instance information for cloud services to cloud accounts, including monitoring tags. These tags indicate whether the cloud account is being monitored. User-defined monitoring tags are also supported.

[0084] A separate monitoring thread is started for each cloud account, and this thread is automatically destroyed when the cloud account goes offline. The monitoring thread can obtain the monitoring tags from the instance information of the cloud services provided by the cloud platform for the cloud account, and determine whether to monitor the cloud account based on the monitoring tags.

[0085] The target cloud account is the cloud account that needs to be monitored among multiple cloud accounts. Users can pre-set monitoring tags to represent the cloud accounts that need to be monitored. When the monitoring tag indicates that the target cloud account is to be monitored, the monitoring thread can obtain the calling rules of the cloud monitoring interface in the cloud platform, and assemble the structure required to call the cloud monitoring interface of the cloud platform according to the calling rules. It obtains the various monitoring indicators corresponding to the cloud service that need to be monitored, and based on the assembled structure, calls the cloud monitoring interface of the cloud platform to monitor the target cloud account, obtaining the monitoring data corresponding to each monitoring indicator. Using the technical solution of this application embodiment, account information of multiple cloud accounts in multiple cloud platforms can be obtained, as well as instance information of cloud services provided by each cloud platform to each cloud account; based on the monitoring tags in the instance information, it can be determined whether to monitor the cloud account. When the monitoring tag indicates that the cloud account is to be monitored, the cloud monitoring interface of the cloud platform where the cloud account is located can be called to perform monitoring, thereby obtaining monitoring data. In this way, unified monitoring of multiple cloud accounts on multiple cloud platforms is achieved, reducing the maintenance cost of monitoring cloud accounts. Furthermore, the target cloud accounts that need to be monitored can be identified through monitoring tags, and only the target cloud accounts are monitored, thus filtering out cloud accounts that do not need to be monitored and saving monitoring resources.

[0086] Based on the above technical solution, as an embodiment, step S110 may include: calling the configuration management database and collecting account information of multiple cloud accounts in multiple cloud platforms according to the configuration fields of the configuration management database. Obtaining the account information of the cloud accounts can be done by obtaining the account information of each cloud account online each time the cloud account is monitored; alternatively, the account information of the cloud accounts can be obtained in advance based on the configuration management database and cached in a database (Redis), and the pre-cached account information can be retrieved from Redis when the cloud account needs to be monitored.

[0087] The configuration management database is pre-configured with fields including the required cloud account information. Based on these fields, the database can automatically retrieve account information for multiple cloud accounts across multiple cloud platforms. When a new cloud account is added, the database can also automatically retrieve its information for subsequent automatic monitoring.

[0088] Using Python, you can periodically call the configuration management database to retrieve account information from multiple cloud accounts across multiple cloud platforms, collected according to configuration fields. This collected account information is then cached in Redis. When the account information is needed, it is directly retrieved from Redis.

[0089] The technical solution adopted in this application embodiment allows configuration fields of the configuration management database to be configured, enabling the database to automatically obtain account information from multiple cloud accounts on a multi-cloud platform. By periodically calling the configuration management database and caching the obtained cloud account information, updates to the account information can be achieved. Furthermore, when the account information is needed subsequently, it can be directly retrieved from the cached database, thereby improving monitoring efficiency. The configuration management database can automatically obtain account information for newly added cloud accounts, reducing the maintenance and monitoring costs for these accounts.

[0090] Based on the above technical solution, as an embodiment, step S110 may include: calling the instance interface of the cloud service provided by the cloud platform for the cloud account according to the account information of the cloud account, and obtaining the instance information of the cloud service provided by the cloud platform for the cloud account.

[0091] Obtaining instance information of cloud services provided by the cloud platform for the cloud account can be done in two ways: either online, retrieving instance information each time the cloud account is monitored; or pre-retrieving instance information and caching it in Redis, retrieving the pre-cached instance information from Redis when monitoring the cloud account is required. Authentication and authorization are performed using the cloud account's AKSK to establish a connection with the cloud platform. The instance interface of the cloud services provided by the cloud platform for the cloud account is called, the returned information is parsed, and the instance information of the cloud services is extracted from the response. The instance information of the cloud services includes configuration information and running status.

[0092] Cloud service instance information is cached in Redis. When the instance information of a cloud service is needed, it can be retrieved directly from the database.

[0093] The technical solution adopted in this application embodiment can pre-obtain and cache instance information of cloud services. When instance information of cloud services is needed later, it can be directly retrieved from the cached database, thereby improving monitoring efficiency. Based on the above technical solution, as an embodiment, step S110 may include: starting a monitoring thread for each of the cloud accounts; and based on the monitoring thread, obtaining instance information of the cloud services provided by the cloud platform to the cloud account.

[0094] A separate monitoring thread is started for each cloud account, and this thread is automatically destroyed when the cloud account goes offline. This monitoring thread can obtain instance information of cloud services provided by the cloud platform to the cloud account, or retrieve pre-cached instance information of cloud services from the database.

[0095] The monitoring thread can also determine whether to monitor the cloud account based on the monitoring tags in the obtained cloud service instance information. If the monitoring tag in the cloud service instance information indicates that monitoring the cloud account is not required, it can be skipped directly. If the monitoring tag in the cloud service instance information indicates that monitoring the cloud account is required, the cloud monitoring interface is called to obtain the monitoring data of the cloud service.

[0096] Using the technical solution of this application embodiment, multiple monitoring threads can execute monitoring tasks in parallel and access the cached database simultaneously without complex inter-process communication. The multiple monitoring threads are independent of each other; an error in one monitoring thread will not affect the execution of other monitoring threads, thus achieving error isolation.

[0097] Based on the above technical solution, as an embodiment, step S120 may include: when the monitoring tag indicates that the target cloud account is being monitored, assembling the structure required to call the cloud monitoring interface according to the calling rules of the cloud platform where the target cloud account is located; and calling the cloud monitoring interface of the cloud platform based on the structure to obtain the monitoring data.

[0098] Each cloud platform has its own calling rules. You can obtain the calling rules of the cloud monitoring interface of each cloud platform through the official documentation of each cloud platform, the information and tools provided by the cloud platform for developers to manage and develop cloud services, the software development kit (SDK) provided by the cloud platform, and / or the customer support channels of the cloud platform.

[0099] When monitoring tags indicate the need to monitor cloud accounts, the monitoring thread can be used to obtain the calling rules of the cloud monitoring interface in the cloud platform where the cloud account to be monitored is located, and the structure required to call the cloud monitoring interface of the cloud platform can be assembled according to the calling rules.

[0100] Based on the assembled structure, the cloud monitoring interface of the corresponding cloud platform can be called to obtain the returned monitoring data. Using the technical solution of this application embodiment, cloud accounts that do not need monitoring can be filtered out by monitoring tags. Only the cloud platform where the cloud account to be monitored resides needs to be assembled to call the cloud monitoring interface. Then, the cloud monitoring interface can be called based on the assembled structure to obtain monitoring data, thereby saving monitoring costs.

[0101] Based on the above technical solution, as an embodiment, step S120 may include: obtaining a configuration file of monitoring metrics for the cloud service corresponding to the target cloud account; the configuration file includes monitoring metrics that need to be monitored; reading the monitoring metrics of the cloud service corresponding to the target cloud account from the configuration file; and obtaining the monitoring data corresponding to the monitoring metrics from the various monitoring data of the cloud service corresponding to the target cloud account.

[0102] The monitoring metrics in the configuration file for the cloud service corresponding to the target cloud account are pre-configured, and users can modify the monitoring metrics in the configuration file to remove unnecessary monitoring metrics.

[0103] In one embodiment, the configuration file can be a Yaml (a human-readable data serialization format) configuration file, which has the advantage of allowing users to quickly view and modify configuration information.

[0104] The system reads the monitoring metrics of the cloud service corresponding to the cloud account from the configuration file, calls the cloud monitoring interface of the cloud platform, and obtains the monitoring data corresponding to each monitoring metric.

[0105] The technical solution adopted in this application embodiment allows users to modify the monitoring metrics in the configuration file. When the cost of cloud monitoring is high, unnecessary monitoring metrics can be removed to save costs. The configuration file is a YAML configuration file, which has the advantages of easy modification of configuration information, readability, and support for multiple data types. Based on the above technical solution, as an embodiment, the monitoring data is obtained by calling the cloud monitoring interface; as shown in Figure 2, the method may further include steps S130 to S150.

[0106] In step S130, in response to the call request of the cloud monitoring interface, the number of requests and the request time stored in the database are updated;

[0107] In step S140, the query rate per second is checked based on the number of requests and the request time stored in the database using a script.

[0108] In step S150, the call request is rejected when the number of requests and the request time exceed the query rate limit per second.

[0109] In scenarios involving multiple cloud platforms, multiple cloud accounts, and multiple regions, large-scale monitoring may encounter rate throttling issues with cloud monitoring interfaces, leading to data loss. To address this issue, a queries-per-second (QPS) limit can be imposed on calls to the cloud monitoring interface. The QPS limit can be configured according to actual needs.

[0110] In one embodiment, the database can be Redis, and the script can be a Lua script. Lua is a lightweight, high-performance scripting language. Lua script interpreters execute quickly, enabling rapid response and processing of various tasks, making it suitable for performance-critical scenarios. Redis supports executing Lua scripts on the server side, allowing multiple Redis commands to be combined into an atomic operation, improving performance, ensuring data consistency, and reducing overhead.

[0111] Configure a key-value pair in the Redis database to store the current call count and timestamp of the cloud monitoring interface. For example, you can use a fixed prefix plus the current number of seconds as the key, and the call count corresponding to the current number of seconds as the key-value pair, setting an expiration time to ensure that the old counter is automatically cleared at the end of each second.

[0112] The Lua script responds to the cloud monitoring API call request, retrieves the key corresponding to the current timestamp, and creates a new key if the key does not exist and initializes it to 0. The Lua script increments the value corresponding to the key, representing the call count within the current second. The Lua script checks if the call count exceeds the set QPS limit. If the limit is exceeded, an error message is returned; otherwise, a success message is returned. Upon receiving an error message, the monitoring thread can wait a fixed amount of time before re-initiating the cloud monitoring API call request.

[0113] In another embodiment, the database can be a relational database, storing the identifier, request time, and number of requests for the cloud monitoring interface. In response to a cloud monitoring interface request, a Python script is used to perform a database update operation, and a database query is used to check the query rate per second (QPS). All records in the database within the most recent second are queried, and the total number of requests is calculated, i.e., the QPS. If the QPS exceeds the QPS limit, the request is rejected.

[0114] The technical solution adopted in this application introduces a query rate limit per second, which can reasonably control the call frequency of the cloud monitoring interface according to actual business needs and the carrying capacity of the cloud platform. By limiting the query rate per second of the cloud monitoring interface, the problem of cloud platform rate limiting of the cloud monitoring interface that may exist in large-scale monitoring scenarios is solved. This ensures that the monitoring task can successfully obtain the required data under normal circumstances, while avoiding triggering the cloud platform's rate limiting measures due to excessively frequent calls to the cloud monitoring interface.

[0115] Based on the above technical solution, as an embodiment, obtaining the monitoring data of the cloud service corresponding to the target cloud account may include: calling the cloud monitoring interface to obtain the monitoring data to be processed returned by the cloud monitoring interface; assembling the monitoring data to be processed into target monitoring data identified by the multi-cloud monitoring system; and capturing the target monitoring data based on the monitoring data capture job configured by the multi-cloud monitoring system.

[0116] By calling the cloud monitoring interface of the cloud platform according to the read monitoring metrics, the pending Exporter monitoring data corresponding to the monitoring metrics returned by the cloud monitoring interface can be obtained. Exporter monitoring data is cloud platform monitoring data and may not be directly recognized and processed by the multi-cloud monitoring system. Therefore, the pending monitoring data needs to be first assembled into target monitoring data that can be recognized by the multi-cloud monitoring system. This target monitoring data can then be captured by the monitoring data capture job configured in the multi-cloud monitoring system, and based on the captured target monitoring data, the cloud accounts that need to be monitored can be monitored.

[0117] In one embodiment, the multi-cloud monitoring system can be a Prometheus system. The Prometheus client can be used to assemble the exporter monitoring data to be processed into target monitoring data that the Prometheus system can recognize and process. The Prometheus system is configured with Jobs, which can be used to capture target monitoring data.

[0118] Based on the acquired target monitoring data, the multi-cloud monitoring system can perform real-time performance testing and analysis, fault diagnosis and early warning, capacity optimization, security monitoring and compliance checks on the cloud services provided by the cloud platform to the cloud account, so as to better manage and optimize the multi-cloud environment and ensure the stable operation and continuous development of business.

[0119] Figure 3 is a topology diagram of a multi-cloud monitoring method provided in an embodiment of this application. As shown in Figure 3, the multi-cloud exporter can actively obtain the account information of the cloud account and the instance information of the cloud service from Redis. The account information of the cloud account is obtained from the configuration management database and cached in Redis, while the instance information of the cloud service is the information returned by calling the instance interface of the cloud service based on the account information of the cloud account in Redis.

[0120] If the monitoring tag in the cloud service instance information indicates that monitoring of the cloud account is not required, it can be skipped. If the monitoring tag in the cloud service instance information indicates that monitoring of the cloud account is required, pre-configured monitoring metrics are read from the Yaml configuration file. The structure is assembled according to the cloud platform's cloud monitoring interface calling rules. Based on the structure, the cloud platform's cloud monitoring interface is called to obtain the monitoring data corresponding to the monitoring metrics. To prevent rate limiting by the cloud platform when calling the cloud monitoring interface, a query rate limit can be imposed on the query rate per second when calling the cloud platform's cloud monitoring interface.

[0121] After the multi-cloud exporter acquires monitoring data, Prometheus cannot directly process this data. Instead, it uses Prometheus-client to assemble the data into target monitoring data that Prometheus can recognize and process. Prometheus then scrapes monitoring data from the multi-cloud exporter to provide unified monitoring of multi-cloud accounts across multiple regions on a multi-cloud platform. This includes real-time performance monitoring and analysis, fault diagnosis and early warning, capacity optimization, security monitoring, and compliance checks of cloud services provided to these accounts. This allows for better management and optimization of the multi-cloud environment, ensuring stable business operation and sustainable development.

[0122] To facilitate better implementation of the multi-cloud monitoring method of this application, this application also provides a multi-cloud monitoring system based on the above-described multi-cloud monitoring method. The meanings of the terms used are the same as in the multi-cloud monitoring method described above, and specific implementation details can be found in the descriptions of the method embodiments.

[0123] Please refer to Figure 4, which is a schematic diagram of the structure of a multi-cloud monitoring system provided in an embodiment of this application. The multi-cloud monitoring system includes:

[0124] The information acquisition module 401 is used to acquire account information of multiple cloud accounts in multiple cloud platforms, and to acquire instance information of cloud services provided by the cloud platforms for the cloud accounts;

[0125] The data acquisition module 402 is used to acquire monitoring data of the cloud service corresponding to the target cloud account when the monitoring tag in the instance information represents monitoring of the target cloud account.

[0126] In one embodiment, the monitoring data is obtained by calling a cloud monitoring interface; the system further includes an update module, used to update the number of requests and the request time stored in the database in response to the call request of the cloud monitoring interface;

[0127] The inspection module is used to check the query rate per second based on the number of requests and the request time stored in the database, using a script.

[0128] The rejection module is used to reject the call request when the number of requests and the request time exceed the query rate limit per second.

[0129] In one embodiment, the information acquisition module 401 includes:

[0130] The data collection unit is used to call the configuration management database and collect account information of multiple cloud accounts in multiple cloud platforms according to the configuration fields of the configuration management database.

[0131] In one embodiment, the information acquisition module 401 includes:

[0132] The instance interface calling unit is used to call the instance interface of the cloud service provided by the cloud platform for the cloud account based on the account information of the cloud account, so as to obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

[0133] In one embodiment, the information acquisition module 401 includes:

[0134] The startup unit is used to start monitoring threads for each of the cloud accounts.

[0135] The information acquisition unit is used to acquire instance information of the cloud service provided by the cloud platform to the cloud account based on the monitoring thread.

[0136] In one embodiment, the data acquisition module 402 includes:

[0137] The structure assembly unit is used to assemble the structure required to call the cloud monitoring interface of the cloud platform according to the calling rules of the cloud platform where the target cloud account is located when the monitoring tag represents monitoring the target cloud account;

[0138] The interface calling unit is used to call the cloud monitoring interface of the cloud platform based on the structure to obtain the monitoring data.

[0139] In one embodiment, the data acquisition module 402 includes:

[0140] The file acquisition unit is used to acquire the configuration file of the monitoring indicators of the cloud service corresponding to the target cloud account; the configuration file includes the monitoring indicators that need to be monitored.

[0141] The reading unit is used to read the monitoring metrics of the cloud service corresponding to the target cloud account from the configuration file;

[0142] The monitoring data acquisition unit is used to acquire the monitoring data corresponding to the monitoring indicators from the various monitoring data of the cloud service corresponding to the target cloud account.

[0143] In one embodiment, the data acquisition module 402 includes:

[0144] The pending data acquisition unit is used to call the cloud monitoring interface of the cloud platform to obtain the pending monitoring data returned by the cloud monitoring interface;

[0145] A data assembly unit is used to assemble the monitoring data to be processed into target monitoring data that can be identified by a multi-cloud monitoring system.

[0146] The data capture unit is used for capturing monitoring data in a multi-cloud-based monitoring system configuration, and capturing the target monitoring data.

[0147] The technical solution of this application embodiment can obtain account information of multiple cloud accounts across multiple cloud platforms, as well as instance information of cloud services provided by each cloud platform to each cloud account. Based on the monitoring tags in the instance information, it can be determined whether to monitor the cloud account. When the monitoring tag indicates that the cloud account is to be monitored, the cloud monitoring interface of the cloud platform where the cloud account resides can be called to perform monitoring, thereby obtaining monitoring data. In this way, unified monitoring of multiple cloud accounts across multiple cloud platforms is achieved, reducing the maintenance cost of cloud account monitoring. Furthermore, the target cloud account to be monitored can be identified through monitoring tags, and monitoring is only performed on the target cloud account, thus filtering out cloud accounts that do not need to be monitored and saving monitoring resources.

[0148] For specific limitations regarding multi-cloud monitoring systems, please refer to the limitations on multi-cloud monitoring methods described above, which will not be repeated here. Each module in the aforementioned multi-cloud monitoring system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0149] In addition, this application also provides an electronic device, as shown in Figure 5, which illustrates the structural schematic diagram of the electronic device involved in this application. Specifically:

[0150] The electronic device may include components such as a processor 501 with one or more processing cores and a memory 502 of one or more computer-readable storage media. Those skilled in the art will understand that the electronic device structure shown in FIG5 does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0151] The processor 501 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 501.

[0152] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0153] In one embodiment, the electronic device further includes a power supply 503 that supplies power to the various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power equipment debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0154] In one embodiment, the electronic device may further include an input unit 504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0155] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 502 according to the following instructions, and the processor 501 runs the applications stored in the memory 502, thereby implementing the steps in any of the multi-cloud monitoring methods provided in the embodiments of this application.

[0156] Those skilled in the art will understand that the structure shown in Figure 5 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0157] In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods described in any embodiment of this application.

[0158] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of this application.

[0159] In some embodiments, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the methods described in any embodiment of this application.

[0160] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0161] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0162] Therefore, this application provides a computer-readable storage medium storing a computer program that can be loaded by a processor to perform the steps in any of the multi-cloud monitoring methods provided in this application.

[0163] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0164] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0165] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the multi-cloud monitoring methods provided in this application, the beneficial effects that any of the multi-cloud monitoring methods provided in this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0166] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0167] The above provides a detailed description of a multi-cloud monitoring method, system, electronic device, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A multi-cloud monitoring method, wherein, include: Obtain account information for multiple cloud accounts across multiple cloud platforms, and obtain instance information for cloud services provided by the cloud platforms to the cloud accounts; The monitoring tag in the instance information indicates that when monitoring the target cloud account, monitoring data of the cloud service corresponding to the target cloud account is obtained.

2. The method according to claim 1, wherein, The monitoring data is obtained by calling the cloud monitoring interface; the method also includes: In response to the call request from the cloud monitoring interface, update the number of requests and the request time stored in the database; The script checks the query rate per second based on the number of requests and the request time stored in the database. The call request is rejected when the number of requests and the request time exceed the query rate limit per second.

3. The method according to claim 1, wherein, The acquisition of account information for multiple cloud accounts across multiple cloud platforms includes: The configuration management database is invoked, and account information of multiple cloud accounts in multiple cloud platforms is collected according to the configuration fields of the configuration management database.

4. The method according to claim 3, wherein, The method further includes: The fields of the configuration management database are pre-configured, including the account information of the required cloud account; The configuration management database automatically retrieves account information for multiple cloud accounts across multiple cloud platforms according to the configuration fields. When a new cloud account is added, the configuration management database automatically obtains the account information of the cloud account.

5. The method according to claim 1, wherein, The acquisition of account information for multiple cloud accounts across multiple cloud platforms includes: The account information of the cloud account is obtained in advance based on the configuration management database, and the account information of the cloud account is cached in the database; When monitoring cloud accounts, pre-cached account information is retrieved from the database.

6. The method according to claim 1, wherein, The acquisition of account information for multiple cloud accounts across multiple cloud platforms includes: When monitoring the cloud account, the account information of the cloud account is obtained online.

7. The method according to claim 1, wherein, The step of obtaining instance information of cloud services provided by the cloud platform for the cloud account includes: Based on the account information of the cloud account, the instance interface of the cloud service provided by the cloud platform for the cloud account is called to obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

8. The method according to claim 7, wherein, The method of calling the instance interface of the cloud service provided by the cloud platform for the cloud account includes: The software development kit (SDK) is found in the official documentation of the cloud platform, which encapsulates the application programming interfaces (APIs) for cloud services. Obtain from the software development kit the specific name, parameters, and usage method of the instance interface for calling the cloud services provided by the cloud platform for the cloud account.

9. The method according to claim 7, wherein, The instance interface defines the way the cloud platform interacts with the cloud account; The instance message of the cloud service includes the configuration information and running status information of the cloud service.

10. The method according to claim 1, wherein, The step of obtaining instance information of cloud services provided by the cloud platform for the cloud account includes: Start monitoring threads for each of the aforementioned cloud accounts; Based on the monitoring thread, obtain instance information of the cloud services provided by the cloud platform for the cloud account.

11. The method according to claim 10, wherein, The monitoring thread is automatically destroyed after the cloud account is taken offline.

12. The method according to claim 1, wherein, When the monitoring tag in the instance information indicates that the target cloud account is being monitored, the monitoring data of the cloud service corresponding to the target cloud account is obtained, including: When the monitoring tag indicates that the target cloud account is being monitored, the structure required to call the cloud monitoring interface of the cloud platform is assembled according to the calling rules of the cloud platform where the target cloud account is located. Based on the structure, the cloud monitoring interface of the cloud platform is called to obtain the monitoring data.

13. The method according to claim 1, wherein, The acquisition of monitoring data for the cloud service corresponding to the target cloud account includes: Obtain the configuration file for monitoring metrics of the cloud service corresponding to the target cloud account; the configuration file includes the monitoring metrics that need to be monitored. Read the monitoring metrics of the cloud service corresponding to the target cloud account from the configuration file; Obtain the monitoring data corresponding to the monitoring indicators from the various monitoring data of the cloud service corresponding to the target cloud account.

14. The method according to claim 1, wherein, The acquisition of monitoring data for the cloud service corresponding to the target cloud account includes: Call the cloud monitoring interface of the cloud platform to obtain the monitoring data to be processed returned by the cloud monitoring interface; The monitoring data to be processed is assembled into target monitoring data that is identified by the multi-cloud monitoring system; the target monitoring data is captured based on the monitoring data capture job configured by the multi-cloud monitoring system.

15. The method according to claim 14, wherein, The cloud monitoring interface is an open interface provided by the cloud platform for exposing monitoring data of managed cloud services.

16. The method according to claim 1, wherein, The account information of the cloud account includes: account name, account ID, region, access key and secret key, as well as whether the cloud platform on which it is located has monitoring enabled.

17. The method according to claim 1, wherein, The step of obtaining instance information of cloud services provided by the cloud platform for the cloud account includes: Instance information of the cloud service is obtained in advance and cached in the database; When monitoring the cloud account, the instance information of the pre-cached cloud service is retrieved from the database.

18. A multi-cloud monitoring system, wherein, include: The information acquisition module is used to acquire account information of multiple cloud accounts in multiple cloud platforms, and to acquire instance information of cloud services provided by the cloud platforms to the cloud accounts; The data acquisition module is used to acquire monitoring data of the cloud service corresponding to the target cloud account when the monitoring tag in the instance information represents monitoring of the target cloud account.

19. An electronic device, wherein, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multi-cloud monitoring method as described in any one of claims 1 to 17.

20. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the multi-cloud monitoring method as described in any one of claims 1 to 17.

Images