Multi-cloud monitoring method and system, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a multi-cloud monitoring method and system, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining the account information of a plurality of cloud accounts in a plurality of cloud platforms, and obtaining the instance information of cloud services provided by the cloud platforms for the cloud accounts; and when the monitoring label in the instance information represents that a target cloud account is monitored, obtaining monitoring data of a cloud service corresponding to the target cloud account. Therefore, according to the scheme, unified supervision can be carried out on the multiple cloud platforms, the target cloud account needing to be monitored can be determined through the monitoring label, only the target cloud account is monitored, and monitoring resources are saved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of cloud service technology, and specifically to a multi-cloud monitoring method, system, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of cloud computing, many cloud computing platforms have emerged. In some large businesses, multi-cloud hybrid scenarios have emerged to support business systems. However, related technologies only support monitoring of a single cloud account on a single cloud platform, and each cloud platform has its own monitoring method, which makes it impossible to uniformly monitor multiple cloud platforms, resulting in decentralized monitoring data. Summary of the invention

[0003] The embodiments of the present application provide a multi-cloud monitoring method, system, electronic device, and computer-readable storage medium, which can uniformly supervise the multi-cloud platform, and can determine the target cloud account that needs to be monitored through monitoring tags, and only monitor the target cloud account, saving monitoring resources.

[0004] In a first aspect, an embodiment of the present application provides a multi-cloud monitoring method, including:

[0005] Obtain account information of multiple cloud accounts in multiple cloud platforms, and obtain instance information of cloud services provided by the cloud platforms for the cloud accounts;

[0006] When the monitoring tag in the instance information indicates that the target cloud account is to be monitored, monitoring data of the cloud service corresponding to the target cloud account is obtained.

[0007] In one embodiment, the monitoring data is obtained by calling a cloud monitoring interface; the method further includes:

[0008] In response to the call request of the cloud monitoring interface, updating the request number and request time stored in the database;

[0009] Performing a query rate per second check on the number of requests and the request time stored in the database based on a script;

[0010] When the number of requests and the request time exceed the query rate limit per second, the call request is rejected.

[0011] In one embodiment, obtaining account information of multiple cloud accounts in multiple cloud platforms includes:

[0012] A configuration management database is called, and account information of a plurality of cloud accounts in a plurality of cloud platforms is collected according to configuration fields of the configuration management database.

[0013] In one embodiment, obtaining instance information of a cloud service provided by the cloud platform for the cloud account includes:

[0014] According to the account information of the cloud account, the instance interface of the cloud service provided by the cloud platform for the cloud account is called to obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

[0015] In one embodiment, obtaining instance information of a cloud service provided by the cloud platform for the cloud account includes:

[0016] Initiate a monitoring thread for each of the cloud accounts;

[0017] Based on the monitoring thread, instance information of the cloud service provided by the cloud platform for the cloud account is obtained.

[0018] In one of the embodiments, when the monitoring tag in the instance information indicates that the target cloud account is monitored, obtaining monitoring data of the cloud service corresponding to the target cloud account includes:

[0019] When the monitoring tag represents monitoring of the target cloud account, assembling a structure required for calling a cloud monitoring interface of the cloud platform according to a calling rule of the cloud platform where the target cloud account is located;

[0020] Based on the structure, the cloud monitoring interface of the cloud platform is called to obtain the monitoring data.

[0021] In one embodiment, obtaining monitoring data of the cloud service corresponding to the target cloud account includes:

[0022] Obtaining a configuration file of monitoring indicators of the cloud service corresponding to the target cloud account; the configuration file includes monitoring indicators that need to be monitored;

[0023] Reading, from the configuration file, monitoring indicators of the cloud service corresponding to the target cloud account;

[0024] The monitoring data corresponding to the monitoring indicator in each monitoring data of the cloud service corresponding to the target cloud account is obtained.

[0025] In one embodiment, obtaining monitoring data of the cloud service corresponding to the target cloud account includes:

[0026] Calling the cloud monitoring interface of the cloud platform to obtain the monitoring data to be processed returned by the cloud monitoring interface;

[0027] Assembling the monitoring data to be processed into target monitoring data identified by the multi-cloud monitoring system;

[0028] A monitoring data capture operation configured based on the multi-cloud monitoring system captures the target monitoring data.

[0029] In a second aspect, an embodiment of the present application provides a multi-cloud monitoring system, including:

[0030] An information acquisition module, used to acquire account information of multiple cloud accounts in multiple cloud platforms, and to acquire instance information of cloud services provided by the cloud platforms for the cloud accounts;

[0031] The data acquisition module is used to acquire monitoring data of the cloud service corresponding to the target cloud account when the monitoring tag in the instance information indicates that the target cloud account is monitored.

[0032] In one embodiment, the monitoring data is obtained by calling a cloud monitoring interface; the system further includes:

[0033] An updating module, used to update the number of requests and request time stored in the database in response to a call request of the cloud monitoring interface;

[0034] A checking module, used for checking the query rate per second on the number of requests and the request time stored in the database based on a script;

[0035] The rejection module is used to reject the call request when the number of requests and the request time exceed the query rate limit per second.

[0036] In one embodiment, the information acquisition module includes:

[0037] The collection unit is used to call the configuration management database and collect the account information of the multiple cloud accounts in the multiple cloud platforms according to the configuration fields of the configuration management database.

[0038] In one embodiment, the information acquisition module includes:

[0039] The instance interface calling unit is used to call the instance interface of the cloud service provided by the cloud platform for the cloud account according to the account information of the cloud account, and obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

[0040] In one embodiment, the information acquisition module includes:

[0041] A starting unit, used to start a monitoring thread for each of the cloud accounts;

[0042] An information acquisition unit is used to acquire instance information of the cloud service provided by the cloud platform for the cloud account based on the monitoring thread.

[0043] In one embodiment, the data acquisition module includes:

[0044] a structure assembling unit, configured to assemble a structure required for calling a cloud monitoring interface of the cloud platform according to a calling rule of the cloud platform where the target cloud account is located when the monitoring tag represents monitoring of the target cloud account;

[0045] The interface calling unit is used to call the cloud monitoring interface of the cloud platform based on the structure to obtain the monitoring data.

[0046] In one embodiment, the data acquisition module includes:

[0047] A file acquisition unit, used to acquire a configuration file of monitoring indicators of the cloud service corresponding to the target cloud account; the configuration file includes monitoring indicators that need to be monitored;

[0048] A reading unit, configured to read the monitoring indicator of the cloud service corresponding to the target cloud account from the configuration file;

[0049] The monitoring data acquisition unit is used to acquire the monitoring data corresponding to the monitoring indicator from the various monitoring data of the cloud service corresponding to the target cloud account.

[0050] In one embodiment, the data acquisition module includes:

[0051] The to-be-processed data acquisition unit is used to call the cloud monitoring interface of the cloud platform to obtain the to-be-processed monitoring data returned by the cloud monitoring interface;

[0052] A data assembling unit, used for assembling the monitoring data to be processed into target monitoring data identified by the multi-cloud monitoring system;

[0053] The data capture unit is used to capture the target monitoring data based on the monitoring data capture operation configured by the multi-cloud monitoring system.

[0054] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the above-mentioned multi-cloud monitoring method when executed by the processor.

[0055] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned multi-cloud monitoring method are implemented.

[0056] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0057] In summary, in the embodiments of the present application, account information of multiple cloud accounts in multiple cloud platforms can be obtained, as well as instance information of cloud services provided by each cloud platform for each cloud account; based on the monitoring tag in the instance information, it can be determined whether to monitor the cloud account. When the monitoring tag indicates that the cloud account is to be monitored, the cloud monitoring interface of the cloud platform where the cloud account is located can be called to monitor, thereby obtaining monitoring data. In this way, unified monitoring of multiple cloud accounts on multiple cloud platforms is achieved, reducing the maintenance cost of cloud account monitoring, and the target cloud account that needs to be monitored can be determined by the monitoring tag, and only the target cloud account is monitored, thereby filtering out cloud accounts that do not need to be monitored and saving monitoring resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 This is a schematic diagram of the steps of a multi-cloud monitoring method provided by an embodiment of the present application;

[0060] Figure 2 This is another step schematic diagram of the multi-cloud monitoring method provided by one embodiment of the present application;

[0061] Figure 3 It is a topological diagram of a multi-cloud monitoring method provided by an embodiment of the present application;

[0062] Figure 4 It is a structural diagram of a multi-cloud monitoring system provided by an embodiment of the present application;

[0063] Figure 5 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0065] The terms involved in the embodiments of the present application are explained below.

[0066] Exporter: A software tool used to export data from different systems or services to a monitoring system, enabling the monitoring system to collect and analyze data from various sources. Exporter can be used to obtain data from various cloud platforms in a multi-cloud scenario.

[0067] Prometheus: An open source system monitoring and alarm toolkit, mainly used to monitor cloud native applications and infrastructure. Prometheus can collect real-time indicator data from multiple sources and has the advantages of being easy to use, scalable and customizable. In one embodiment of the present application, a multi-cloud monitoring system is built based on Prometheus.

[0068] Multi-cloud: refers to the use of cloud services from multiple cloud platforms, each of which has a different corresponding cloud service provider. Multi-cloud allows you to choose cloud services from different cloud service providers to achieve cost optimization. However, the multi-cloud environment is complex, and it is difficult to manage multiple clouds. Integrating data between different cloud platforms is a big challenge.

[0069] Cloud monitoring interface: An open interface provided by the cloud platform to expose the monitoring data of hosted cloud services. It allows users and third-party applications to access and manipulate cloud monitoring data, such as metrics and logs. Based on the cloud monitoring interface, monitoring data can be accessed and analyzed. The cloud monitoring interface can be integrated with third-party tools, enabling organizations to correlate monitoring data with other systems.

[0070] Python: A development language.

[0071] Prometheus-client: A library for generating and exporting data in applications. It can export data to Prometheus format so that the data can be collected and processed by Prometheus.

[0072] Configuration Management Database (CMDB): A platform for storing all components and their relationships in the information technology (IT) infrastructure. CMDB can automatically discover and record IT assets, model the dependencies and connectivity between IT assets, and track and manage changes in the IT infrastructure.

[0073] Example: In object-oriented programming, a class is an abstract data type that defines the properties and behaviors of an object, and an instance is a specific object created based on the class.

[0074] In one embodiment, Figure 1 As shown, a multi-cloud monitoring method is provided. Although a logical order is shown in the step diagram, in some cases, the steps shown or described may be performed in an order different from that shown in the figure. Specifically, the multi-cloud monitoring method can be applied to a server, wherein the server can be a physical server or a cloud server that provides various cloud services. It is worth noting that the present application does not limit the number of servers. According to the implementation needs, there can be any number of servers. For example, the server can be a single server or a server cluster composed of multiple servers, and so on.

[0075] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0076] according to Figure 1 The multi-cloud monitoring method shown in the figure includes at least steps S110 to S120, which are described in detail as follows:

[0077] In step S110, account information of multiple cloud accounts in multiple cloud platforms is obtained, and instance information of cloud services provided by the cloud platforms for the cloud accounts is obtained;

[0078] In step S120, when the monitoring tag in the instance information indicates that the target cloud account is to be monitored, monitoring data of the cloud service corresponding to the target cloud account is obtained.

[0079] For multiple cloud accounts in multiple regions in a multi-cloud scenario, you can obtain the account information of each cloud account in each cloud platform. A cloud account is an identity credential used by users to access and manage cloud services. Users log in to the cloud service platform through a cloud account and use various cloud service resources. The account information of a cloud account includes but is not limited to: account name, account ID, region, access key and secret key (AKSK), and whether monitoring is enabled on the cloud platform.

[0080] Cloud service providers may divide their infrastructure into different regions for operation to reduce latency, improve availability and fault tolerance, etc. The monitoring requirements of cloud accounts in different regions may be different, so it is necessary to obtain the region to which the cloud account belongs.

[0081] Whether the cloud platform where the cloud account is located is monitored can be pre-configured by the user according to actual needs. When the account information of the cloud account indicates that the cloud platform where the cloud account is located is not monitored, it is not necessary to monitor each cloud account of the cloud platform to save resources.

[0082] Based on the account information of the cloud account obtained, you can obtain the instance interface for calling the cloud service provided by the cloud platform for the cloud account. Look for the software development kit (SDK) in the official documents of the cloud platform. The software development kit encapsulates the application programming interface (API) of the cloud service. From the software development kit, you can obtain the specific name, parameters, and usage methods of the instance interface for calling the cloud service provided by the cloud platform for the cloud account. After using the account information of the cloud account for identity authentication, call the instance interface of the cloud service provided by the cloud platform for the cloud account, and you can get the returned instance information.

[0083] Among them, the instance interface defines the way the cloud platform interacts with the cloud account; the instance message of the cloud service provided by the cloud platform to the cloud account is the result returned by calling the instance interface, including the configuration information and running status of the cloud service.

[0084] Cloud services are services provided by cloud platforms through the Internet, including computing power, storage, database management and other functions. They may include but are not limited to: SaaS (Software as a Service), IaaS (Infrastructure as a Service), PaaS (Platform as a Service) and / or DaaS (Data as a Service).

[0085] The instance information of cloud services provided by the cloud platform for cloud accounts includes monitoring tags, which can indicate whether the cloud account is monitored. Monitoring tags can be customized by users.

[0086] A monitoring thread is started independently for each cloud account, and the monitoring thread can be automatically destroyed after the cloud account goes offline. The monitoring thread can obtain the monitoring tag in the instance information of the cloud service provided by the cloud platform for the cloud account, and determine whether to monitor the cloud account based on the monitoring tag.

[0087] The target cloud account is the cloud account that needs to be monitored among multiple cloud accounts. The user can set the monitoring tag that represents the cloud account that needs to be monitored in advance. When the monitoring tag represents the monitoring of the target cloud account, the monitoring thread can obtain the calling rules of the cloud monitoring interface in the cloud platform, and assemble the structure required to call the cloud monitoring interface of the cloud platform according to the calling rules. Obtain the various monitoring indicators that need to be monitored corresponding to the cloud service, call the cloud monitoring interface of the cloud platform based on the assembled structure, monitor the target cloud account, and obtain the monitoring data corresponding to the returned various monitoring indicators.

[0088] By adopting the technical solution of the embodiment of the present application, it is possible to obtain the account information of multiple cloud accounts in multiple cloud platforms, as well as the instance information of the cloud services provided by each cloud platform for each cloud account; based on the monitoring tag in the instance information, it can be determined whether to monitor the cloud account. When the monitoring tag indicates that the cloud account is to be monitored, the cloud monitoring interface of the cloud platform where the cloud account is located can be called to monitor, thereby obtaining monitoring data. In this way, unified monitoring of multiple cloud accounts on multiple cloud platforms is achieved, reducing the maintenance cost of cloud account monitoring, and the target cloud account that needs to be monitored can be determined by the monitoring tag, and only the target cloud account is monitored, thereby filtering out cloud accounts that do not need to be monitored and saving monitoring resources.

[0089] Based on the above technical solution, as an embodiment, step S110 may include: calling a configuration management database, and collecting account information of multiple cloud accounts in multiple cloud platforms according to configuration fields of the configuration management database.

[0090] To obtain the account information of a cloud account, the account information of each cloud account can be obtained online each time the cloud account is monitored; or the account information of the cloud account can be obtained in advance based on the configuration management database, and the account information of the cloud account can be cached in the database (Redis). When the cloud account needs to be monitored, the pre-cached account information can be obtained from Redis.

[0091] The fields of the configuration management database are configured in advance, and the configuration fields include the account information of the required cloud accounts. The configuration management database can automatically obtain the account information of multiple cloud accounts in multiple cloud platforms according to the configuration fields. When a new cloud account is added, the configuration management database can also automatically obtain the account information of the cloud account, so as to automatically monitor the newly added cloud account later.

[0092] The Python language can be used to periodically call the configuration management database to obtain the account information of multiple cloud accounts in multiple cloud platforms collected by the configuration management database according to the configuration fields, and cache the collected cloud account information in Redis.

[0093] When the account information of the cloud account is needed, the pre-cached account information of the cloud account is directly obtained from Redis.

[0094] By adopting the technical solution of the embodiment of the present application, the configuration fields of the configuration management database can be configured so that the configuration management database automatically obtains the account information of multiple cloud accounts in the multi-cloud platform. By periodically calling the configuration management database and caching the account information of the acquired cloud accounts, the account information of the cloud accounts can be updated, and when the account information of the cloud accounts is needed later, the account information of the cloud accounts can be directly obtained from the cached database, thereby improving the efficiency of monitoring. The configuration management database can automatically obtain the account information of the newly added cloud accounts, reducing the maintenance and monitoring costs of the newly added cloud accounts.

[0095] Based on the above technical solution, as an embodiment, step S110 may include: according to the account information of the cloud account, calling the instance interface of the cloud service provided by the cloud platform for the cloud account, and obtaining the instance information of the cloud service provided by the cloud platform for the cloud account.

[0096] To obtain the instance information of the cloud service provided by the cloud platform for the cloud account, the instance information of the cloud service can be obtained online each time the cloud account is monitored; or the instance information of the cloud service can be obtained in advance and cached in Redis. When the cloud account needs to be monitored, the pre-cached instance information of the cloud service can be obtained from Redis.

[0097] Use the cloud account's AKSK for authentication and authorization to establish a connection with the cloud platform. Call the cloud service instance interface provided by the cloud platform for the cloud account, get the returned information, parse the returned information, and extract the cloud service instance information from the response information. The cloud service instance information includes the cloud service configuration information and operation status.

[0098] Cache the instance information of the cloud service to Redis. When you need to obtain the instance information of the cloud service, you can directly obtain the cached instance information from the database.

[0099] By adopting the technical solution of the embodiment of the present application, the instance information of the cloud service can be acquired in advance and cached. When the instance information of the cloud service is needed subsequently, the instance information of the cloud service can be directly acquired from the cached database, thereby improving the monitoring efficiency.

[0100] Based on the above technical solution, as an embodiment, step S110 may include: starting a monitoring thread for each of the cloud accounts; based on the monitoring thread, obtaining instance information of the cloud service provided by the cloud platform for the cloud account.

[0101] A monitoring thread is started independently for each cloud account, and the monitoring thread is automatically destroyed when the cloud account goes offline. The monitoring thread can obtain the instance information of the cloud service provided by the cloud platform for the cloud account, or obtain the instance information of the cloud service cached in advance from the database.

[0102] The monitoring thread can also determine whether to monitor the cloud account based on the monitoring tag in the acquired cloud service instance information. When the monitoring tag in the cloud service instance information indicates that the cloud account does not need to be monitored, it can be skipped directly. When the monitoring tag in the cloud service instance information indicates that the cloud account needs to be monitored, the cloud monitoring interface is called to obtain the monitoring data of the cloud service.

[0103] By adopting the technical solution of the embodiment of the present application, multiple monitoring threads can execute monitoring tasks in parallel and access the cached database at the same time without complex inter-process communication. Multiple monitoring threads are independent of each other, and the error of one monitoring thread will not affect the execution of other monitoring threads, so error isolation can be achieved.

[0104] Based on the above technical solution, as an embodiment, step S120 may include: when the monitoring tag represents the monitoring of the target cloud account, assembling the structure required for calling the cloud monitoring interface according to the calling rules of the cloud platform where the target cloud account is located; based on the structure, calling the cloud monitoring interface of the cloud platform to obtain the monitoring data.

[0105] Each cloud platform has its own calling rules. You can obtain the calling rules of the cloud monitoring interface in each cloud platform through the official documents of each cloud platform, the information and tools of calling rules provided by the cloud platform to developers for cloud service management and development, the software development kit (SDK) provided by the cloud platform and / or the customer support channels of the cloud platform.

[0106] When the monitoring tag indicates that a cloud account needs to be monitored, the monitoring thread can be used to obtain the calling rules of the cloud monitoring interface in the cloud platform where the cloud account to be monitored is located, and assemble the structure required to call the cloud monitoring interface of the cloud platform according to the calling rules.

[0107] Based on the assembled structure, the cloud monitoring interface of the cloud platform corresponding to the structure can be called to obtain the returned monitoring data.

[0108] By adopting the technical solution of the embodiment of the present application, cloud accounts that do not need to be monitored can be filtered out through monitoring tags. It is only necessary to assemble the structure required for calling the cloud monitoring interface for the cloud platform where the cloud accounts that need to be monitored are located, so that the cloud monitoring interface can be called according to the assembled structure to obtain monitoring data, thereby saving monitoring costs.

[0109] Based on the above technical solution, as an embodiment, step S120 may include: obtaining a configuration file of monitoring indicators of the cloud service corresponding to the target cloud account; the configuration file includes monitoring indicators that need to be monitored; reading the monitoring indicators of the cloud service corresponding to the target cloud account from the configuration file; obtaining the monitoring data corresponding to the monitoring indicators from each monitoring data of the cloud service corresponding to the target cloud account.

[0110] The monitoring indicators in the configuration file of the monitoring indicators of the cloud service corresponding to the target cloud account are pre-configured, and the monitoring indicators in the configuration file support modification by the user to remove unnecessary monitoring indicators.

[0111] In one embodiment, the configuration file may be a YAML (a human-readable data serialization format) configuration file. The YAML configuration file has the advantage of being convenient for users to quickly view and modify configuration information.

[0112] Read the monitoring indicators of the cloud service corresponding to the cloud account from the configuration file, call the cloud monitoring interface of the cloud platform, and obtain the monitoring data corresponding to each monitoring indicator.

[0113] By adopting the technical solution of the embodiment of the present application, the monitoring indicators in the configuration file support user modification, and when the cost of cloud monitoring is high, unnecessary monitoring indicators can be removed to save costs. The configuration file is a Yaml configuration file, which can be used to conveniently modify configuration information, has the advantages of readability and support for multiple data types.

[0114] Based on the above technical solution, as an embodiment, the monitoring data is obtained by calling the cloud monitoring interface; Figure 2 As shown, the method may further include steps S130 to S150.

[0115] In step S130, in response to the call request of the cloud monitoring interface, the request number and request time stored in the database are updated;

[0116] In step S140, a query rate per second is checked on the number of requests and the request time stored in the database based on a script;

[0117] In step S150, when the number of requests and the request time exceed the query rate limit per second, the call request is rejected.

[0118] In scenarios with multiple cloud platforms and multiple cloud accounts in multiple regions, large-scale monitoring may result in the cloud platform limiting the flow of cloud monitoring interfaces, resulting in monitoring data loss. To solve this technical problem, you can set a query per second (QPS) limit for calling the cloud monitoring interface. The query per second limit can be set according to actual needs.

[0119] In one embodiment, the database may be Redis, and the script may be a Lua script. Lua is a lightweight, high-performance scripting language. The interpreter of the Lua script executes quickly, can respond to and process various tasks quickly, and is suitable for scenarios with high performance requirements. Redis supports the execution of Lua scripts on the server side, which allows multiple Redis commands to be combined into an atomic operation, improving performance, ensuring data consistency, and reducing overhead.

[0120] Set a key-value pair in the Redis database and determine a Redis key-value pair to store the current number of calls and timestamps of the Cloud Monitoring interface. For example, you can use a fixed prefix plus the current number of seconds as the key, the number of calls corresponding to the current number of seconds as the key value, and set an expiration time to ensure that the old counter is automatically cleared at the end of each second.

[0121] The Lua script responds to the call request of the cloud monitoring interface and obtains the key corresponding to the current timestamp. If the key does not exist, a new key is created and initialized to 0. The Lua script increases the value corresponding to the key, indicating that the number of calls in the current second is increased by one. The Lua script checks whether the number of calls exceeds the set QPS limit. If the limit is exceeded, an error message is returned; if the limit is not exceeded, a success message is returned. When the monitoring thread receives an error message, it can wait for a fixed time and then initiate a call request to the cloud monitoring interface again.

[0122] In another embodiment, the database may be a relational database, which stores the identifier, request time, and number of requests of the call request of the cloud monitoring interface. In response to the call request of the cloud monitoring interface, a Python script is used to perform a database update operation, and a database query is used to perform a query rate per second check, query all records in the database within the last second, and calculate the total number of requests, that is, the query rate per second. When the query rate per second exceeds the query rate per second limit, the call request is rejected.

[0123] The technical solution of the embodiment of the present application is adopted, and the query rate per second limit is introduced, which can reasonably control the calling frequency of the cloud monitoring interface according to the actual business needs and the carrying capacity of the cloud platform. By limiting the query rate per second of the cloud monitoring interface, the problem of the cloud platform limiting the flow of the cloud monitoring interface in large-scale monitoring scenarios is solved, ensuring that the monitoring task can smoothly obtain the required data under normal circumstances, and avoiding the triggering of the cloud platform's flow limiting measures due to excessively frequent calls to the cloud monitoring interface.

[0124] Based on the above technical solution, as an embodiment, obtaining the monitoring data of the cloud service corresponding to the target cloud account may include: calling the cloud monitoring interface to obtain the monitoring data to be processed returned by the cloud monitoring interface; assembling the monitoring data to be processed into target monitoring data identified by the multi-cloud monitoring system; and capturing the target monitoring data based on the monitoring data capture operation configured by the multi-cloud monitoring system.

[0125] By calling the cloud monitoring interface of the cloud platform according to the read monitoring indicators, the Exporter monitoring data to be processed corresponding to the monitoring indicators returned by the cloud monitoring interface can be obtained. The Exporter monitoring data is the monitoring data of the cloud platform, which may not be directly recognized and processed by the multi-cloud monitoring system. Therefore, it is necessary to first organize the monitoring data to be processed into target monitoring data recognized by the multi-cloud monitoring system, so as to capture the target monitoring data based on the monitoring data capture job configured by the multi-cloud monitoring system, and monitor the cloud account that needs to be monitored based on the captured target monitoring data.

[0126] In one embodiment, the multi-cloud monitoring system may be a Prometheus system, and the Prometheus-client may be used to assemble the to-be-processed Exporter monitoring data into target monitoring data that can be identified and processed by the Prometheus system. A Job is configured in the Prometheus system, and the target monitoring data may be captured based on the Job.

[0127] Based on the acquired target monitoring data, the multi-cloud monitoring system can conduct real-time performance detection and analysis, fault diagnosis and early warning, capacity optimization, security monitoring and compliance checks on the cloud services provided by the cloud platform to the cloud account, so as to better manage and optimize the multi-cloud environment and ensure the stable operation and sustainable development of the business.

[0128] Figure 3 Schematic diagram of the topology of a multi-cloud monitoring method provided by an embodiment of the present application. Figure 3As shown in the figure, the multi-cloud exporter can actively obtain the account information of the cloud account and the instance information of the cloud service from Redis. The account information of the cloud account is obtained from the configuration management database and cached in Redis, and the instance information of the cloud service is the returned information obtained by calling the instance interface of the cloud service based on the account information of the cloud account in Redis.

[0129] When the monitoring tag representation in the instance information of the cloud service does not need to monitor the cloud account, it can be skipped directly. When the monitoring tag representation in the instance information of the cloud service needs to monitor the cloud account, read the pre-configured monitoring indicators from the Yaml configuration file. Assemble the structure according to the calling rules of the cloud platform's cloud monitoring interface. Based on the structure, call the cloud platform's cloud monitoring interface to obtain the monitoring data corresponding to the monitoring indicators. Among them, when calling the cloud platform's cloud monitoring interface, in order to prevent the cloud platform from limiting the flow, the query rate per second for calling the cloud platform's cloud monitoring interface can be limited.

[0130] After the multi-cloud exporter obtains the monitoring data, Prometheus cannot directly process the monitoring data obtained by the multi-cloud exporter. You can use Prometheus-client to assemble the monitoring data obtained by the multi-cloud exporter into target monitoring data that Prometheus can identify and process. Prometheus grabs the monitoring data from the multi-cloud exporter to uniformly monitor the multi-cloud accounts in multi-region scenarios of the multi-cloud platform based on the monitoring data, and conducts real-time performance detection and analysis, fault diagnosis and early warning, capacity optimization, security monitoring and compliance inspection of the cloud services provided by the cloud platform to the cloud account, so as to better manage and optimize the multi-cloud environment and ensure the stable operation and sustainable development of the business.

[0131] In order to better implement the multi-cloud monitoring method of the present application, the present application also provides a multi-cloud monitoring system based on the multi-cloud monitoring method. The meanings of the terms are the same as those in the multi-cloud monitoring method, and the specific implementation details can refer to the description in the method embodiment.

[0132] See also Figure 4 , Figure 4 : is a schematic diagram of the structure of a multi-cloud monitoring system provided in an embodiment of the present application, and the multi-cloud monitoring system includes:

[0133] The information acquisition module 401 is used to acquire account information of multiple cloud accounts in multiple cloud platforms, and to acquire instance information of cloud services provided by the cloud platforms for the cloud accounts;

[0134] The data acquisition module 402 is configured to acquire monitoring data of a cloud service corresponding to a target cloud account when the monitoring tag in the instance information indicates that the target cloud account is monitored.

[0135] In one embodiment, the monitoring data is obtained by calling a cloud monitoring interface; the system further includes:

[0136] An updating module, used to update the number of requests and request time stored in the database in response to a call request of the cloud monitoring interface;

[0137] A checking module, used for checking the query rate per second on the number of requests and the request time stored in the database based on a script;

[0138] The rejection module is used to reject the call request when the number of requests and the request time exceed the query rate limit per second.

[0139] In one embodiment, the information acquisition module 401 includes:

[0140] The collection unit is used to call the configuration management database and collect the account information of the multiple cloud accounts in the multiple cloud platforms according to the configuration fields of the configuration management database.

[0141] In one embodiment, the information acquisition module 401 includes:

[0142] The instance interface calling unit is used to call the instance interface of the cloud service provided by the cloud platform for the cloud account according to the account information of the cloud account, and obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

[0143] In one embodiment, the information acquisition module 401 includes:

[0144] A starting unit, used to start a monitoring thread for each of the cloud accounts;

[0145] An information acquisition unit is used to acquire instance information of the cloud service provided by the cloud platform for the cloud account based on the monitoring thread.

[0146] In one embodiment, the data acquisition module 402 includes:

[0147] a structure assembling unit, configured to assemble a structure required for calling a cloud monitoring interface of the cloud platform according to a calling rule of the cloud platform where the target cloud account is located when the monitoring tag represents monitoring of the target cloud account;

[0148] The interface calling unit is used to call the cloud monitoring interface of the cloud platform based on the structure to obtain the monitoring data.

[0149] In one embodiment, the data acquisition module 402 includes:

[0150] A file acquisition unit, used to acquire a configuration file of monitoring indicators of the cloud service corresponding to the target cloud account; the configuration file includes monitoring indicators that need to be monitored;

[0151] A reading unit, configured to read the monitoring indicator of the cloud service corresponding to the target cloud account from the configuration file;

[0152] The monitoring data acquisition unit is used to acquire the monitoring data corresponding to the monitoring indicator from the various monitoring data of the cloud service corresponding to the target cloud account.

[0153] In one embodiment, the data acquisition module 402 includes:

[0154] The to-be-processed data acquisition unit is used to call the cloud monitoring interface of the cloud platform to obtain the to-be-processed monitoring data returned by the cloud monitoring interface;

[0155] A data assembling unit, used for assembling the monitoring data to be processed into target monitoring data identified by the multi-cloud monitoring system;

[0156] The data capture unit is used to capture the target monitoring data based on the monitoring data capture operation configured by the multi-cloud monitoring system.

[0157] By adopting the technical solution of the embodiment of the present application, it is possible to obtain the account information of multiple cloud accounts in multiple cloud platforms, as well as the instance information of the cloud services provided by each cloud platform for each cloud account; based on the monitoring tag in the instance information, it can be determined whether to monitor the cloud account. When the monitoring tag indicates that the cloud account is to be monitored, the cloud monitoring interface of the cloud platform where the cloud account is located can be called to monitor, thereby obtaining monitoring data. In this way, unified monitoring of multiple cloud accounts on multiple cloud platforms is achieved, reducing the maintenance cost of cloud account monitoring, and the target cloud account that needs to be monitored can be determined by the monitoring tag, and only the target cloud account is monitored, thereby filtering out cloud accounts that do not need to be monitored and saving monitoring resources.

[0158] For the specific definition of the multi-cloud monitoring system, please refer to the definition of the multi-cloud monitoring method above, which will not be repeated here. Each module in the above multi-cloud monitoring system can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0159] In addition, the present application also provides an electronic device, such as Figure 5 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:

[0160] The electronic device may include one or more processors 501 of processing cores and one or more computer-readable storage media memories 502 and other components. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0161] The processor 501 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, the processor 501 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0162] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0163] In one embodiment, the electronic device further includes a power supply 503 for supplying power to various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 503 can also include one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, power status indicators and other arbitrary components.

[0164] In one embodiment, the electronic device may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0165] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 501 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 501 will run the application program stored in the memory 502, thereby implementing the steps in any of the multi-cloud monitoring methods provided in the embodiments of the present application.

[0166] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in any embodiment of the present application is implemented.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the present application is implemented.

[0169] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which implement the method described in any embodiment of the present application when executed by a processor.

[0170] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0171] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0172] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any multi-cloud monitoring method provided in the present application.

[0173] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0174] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0175] Since the instructions stored in the computer-readable storage medium can execute the steps in any multi-cloud monitoring method provided in the present application, the beneficial effects that can be achieved by any multi-cloud monitoring method provided in the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0176] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0177] The above is a detailed introduction to a multi-cloud monitoring method, system, electronic device and computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A multi-cloud monitoring method, characterized in that: include: Obtain account information of multiple cloud accounts in multiple cloud platforms, and obtain instance information of cloud services provided by the cloud platforms for the cloud accounts; When the monitoring tag in the instance information indicates that the target cloud account is to be monitored, monitoring data of the cloud service corresponding to the target cloud account is obtained.

2. The method according to claim 1, characterized in that: The monitoring data is obtained by calling a cloud monitoring interface; the method further includes: In response to the call request of the cloud monitoring interface, updating the request number and request time stored in the database; Performing a query rate per second check on the number of requests and the request time stored in the database based on a script; When the number of requests and the request time exceed the query rate limit per second, the call request is rejected.

3. The method according to claim 1, characterized in that The obtaining of account information of multiple cloud accounts in multiple cloud platforms includes: A configuration management database is called, and account information of a plurality of cloud accounts in a plurality of cloud platforms is collected according to configuration fields of the configuration management database.

4. The method according to claim 1, characterized in that: The obtaining instance information of the cloud service provided by the cloud platform for the cloud account includes: According to the account information of the cloud account, the instance interface of the cloud service provided by the cloud platform for the cloud account is called to obtain the instance information of the cloud service provided by the cloud platform for the cloud account.

5. The method according to claim 1, characterized in that The obtaining instance information of the cloud service provided by the cloud platform for the cloud account includes: Initiate a monitoring thread for each of the cloud accounts; Based on the monitoring thread, instance information of the cloud service provided by the cloud platform for the cloud account is obtained.

6. The method according to claim 1, characterized in that When the monitoring tag in the instance information indicates that the target cloud account is monitored, obtaining monitoring data of the cloud service corresponding to the target cloud account includes: When the monitoring tag represents monitoring of the target cloud account, assembling a structure required for calling a cloud monitoring interface of the cloud platform according to a calling rule of the cloud platform where the target cloud account is located; Based on the structure, the cloud monitoring interface of the cloud platform is called to obtain the monitoring data.

7. The method according to claim 1, characterized in that The obtaining monitoring data of the cloud service corresponding to the target cloud account includes: Obtaining a configuration file of monitoring indicators of the cloud service corresponding to the target cloud account; the configuration file includes monitoring indicators that need to be monitored; Reading, from the configuration file, monitoring indicators of the cloud service corresponding to the target cloud account; The monitoring data corresponding to the monitoring indicator in each monitoring data of the cloud service corresponding to the target cloud account is obtained.

8. The method according to claim 1, characterized in that The obtaining monitoring data of the cloud service corresponding to the target cloud account includes: Calling the cloud monitoring interface of the cloud platform to obtain the monitoring data to be processed returned by the cloud monitoring interface; Assembling the monitoring data to be processed into target monitoring data identified by the multi-cloud monitoring system; Based on the monitoring data capture operation configured by the multi-cloud monitoring system, the target monitoring data is captured.

9. A multi-cloud monitoring system, characterized in that: include: An information acquisition module, used to acquire account information of multiple cloud accounts in multiple cloud platforms, and to acquire instance information of cloud services provided by the cloud platforms for the cloud accounts; The data acquisition module is used to acquire monitoring data of the cloud service corresponding to the target cloud account when the monitoring tag in the instance information indicates that the target cloud account is monitored.

10. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the multi-cloud monitoring method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the multi-cloud monitoring method according to any one of claims 1 to 8 are implemented.

Images