Multi-cluster data query method, device, monitoring platform and storage medium

Abstract

The embodiment of the present application is applicable to the field of big data technology, and provides a multi-cluster data query method, device, monitoring platform and storage medium. The method is applied to the monitoring platform, including: obtaining an access request; the access request includes the cluster identifier of the cluster to be accessed; the monitoring platform is used to monitor clusters in multiple special zone platforms, and each special zone platform monitors the cluster data of at least one cluster; according to the cluster identifier of the cluster to be accessed and the cluster identifier corresponding to each cluster, the cluster with the same cluster identifier as the cluster to be accessed is determined as the cluster to be accessed, and the special zone platform containing the cluster to be accessed is determined as the target special zone platform; according to the cluster identifier, the preset cluster data query address in the target special zone platform is modified to generate a target query address; according to the target query address, the cluster data in the cluster to be accessed is queried. The above method can improve the efficiency of querying cluster data from multiple special zone platforms.

Description

Technical Field

[0001] The present application belongs to the field of big data technology, and in particular relates to a multi-cluster data query method, device, monitoring platform and storage medium. Background Art

[0002] With the development of cluster technology, cloud computing systems usually use cluster technology to communicate and connect loosely integrated clusters (computer software and / or hardware) in cloud computing systems, so that the clusters in the cloud computing system can collaborate highly closely to complete computing work.

[0003] Taking Ping An Cloud System as an example, Ping An Cloud System is a highly available and elastically scalable cloud-hosted Elastic Search service built based on open source Elastic Search. Among them, Ping An Cloud System includes multiple zone platforms of different specifications, and each zone platform includes multiple clusters. For each cluster, the cluster data in the cluster contains the Ping An Cloud System and user behavior recording tools, which play a decisive role in monitoring cluster operation and troubleshooting cluster failures.

[0004] However, with the massive growth of applications and services deployed in the Ping An Cloud system, multiple clusters will inevitably be formed in the Ping An Cloud system according to business and needs. However, the cluster data of multiple clusters is usually scattered, making it difficult to obtain and aggregate them in a unified manner. When a Ping An Cloud system fails, if it is necessary to view the cluster data of each cluster to troubleshoot the cluster with the problem, the operation and maintenance personnel will need to spend too much time repeatedly logging into the dedicated platform corresponding to different clusters to obtain the corresponding cluster data, which is time-consuming and laborious. Summary of the invention

[0005] The embodiments of the present application provide a multi-cluster data query method, device, monitoring platform and storage medium, which can solve the problem that operation and maintenance personnel need to spend too much time repeatedly logging into the dedicated platform corresponding to different clusters before obtaining the corresponding cluster data.

[0006] In a first aspect, an embodiment of the present application provides a multi-cluster data query method, which is applied to a monitoring platform. The method includes:

[0007] Obtain an access request; the access request includes a cluster identifier of a cluster to be accessed; the monitoring platform is used to monitor clusters in multiple zone platforms, and each zone platform monitors cluster data of at least one cluster;

[0008] According to the cluster identifier of the cluster to be visited and the cluster identifier corresponding to each cluster, a cluster having the same cluster identifier as the cluster to be visited is determined as the cluster to be visited, and a zone platform including the cluster to be visited is determined as the target zone platform;

[0009] Modify the preset cluster data query address in the target zone platform according to the cluster identifier to generate a target query address;

[0010] Query the cluster data in the cluster to be accessed according to the target query address.

[0011] In a second aspect, an embodiment of the present application provides a multi-cluster data query device, which is applied to a monitoring platform, and the device includes:

[0012] The acquisition module is used to obtain the access request; the access request includes the cluster identifier of the cluster to be accessed; the monitoring platform is used to monitor the clusters in multiple zone platforms, and each zone platform monitors the cluster data of at least one cluster;

[0013] A target zone platform determination module is used to determine, based on the cluster identifier of the cluster to be visited and the cluster identifier corresponding to each cluster, a cluster with the same cluster identifier as the cluster to be visited as the cluster to be visited, and to determine the zone platform including the cluster to be visited as the target zone platform;

[0014] A generation module modifies a preset cluster data query address in a target zone platform according to a cluster identifier to generate a target query address;

[0015] The query module is used to query the cluster data in the cluster to be accessed according to the target query address.

[0016] In a third aspect, an embodiment of the present application provides a monitoring platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of the first aspect described above when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method of the first aspect described above is implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a monitoring platform, enables the monitoring platform to execute the method of the first aspect.

[0019] Compared with the prior art, the beneficial effect of the embodiment of the present application is that when the operation and maintenance personnel need to obtain the cluster data in any cluster, the monitoring platform can obtain the access request containing the cluster identifier input by the operation and maintenance personnel, so as to determine the target zone platform containing the cluster to be accessed from multiple zone platforms according to the cluster identifier. In addition, the monitoring platform can also use the cluster identifier to determine the specific storage address of the cluster data of the cluster to be accessed in the zone platform, so as to modify the preset cluster data query address of the pre-set target zone platform and generate a target query address. At this time, the generated target query address includes not only the access domain name for accessing the zone platform, but also the specific storage address of the cluster data of the cluster to be accessed in the zone platform. In this way, the monitoring platform can directly obtain the corresponding cluster data from multiple zone platforms according to the target query address, without having to repeatedly log in to different zone platforms before obtaining the corresponding cluster data, thereby improving the efficiency of the operation and maintenance personnel in querying cluster data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a structural diagram of a monitoring platform provided by an embodiment of the present application;

[0022] Figure 2 This is a flowchart of a multi-cluster data query method provided in an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of an implementation method of S101 of a multi-cluster data query method provided in an embodiment of the present application;

[0024] Figure 4 is a flowchart of an implementation of a multi-cluster data query method provided by another embodiment of the present application;

[0025] Figure 5 It is a structural diagram of a multi-cluster data query device provided by an embodiment of the present application;

[0026] Figure 6 It is a structural diagram of a monitoring platform provided in another embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0028] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] The multi-cluster data query method provided in the embodiment of the present application can be applied to a monitoring platform, wherein the monitoring platform is mainly used to monitor the cluster data of each cluster in multiple dedicated area platforms.

[0031] Specifically, refer to Figure 1 Each zone platform has an independent data storage sub-unit for collecting and summarizing the cluster data of each cluster in each zone platform. The cluster data includes but is not limited to the physical resources, container resources, and cluster logs of each cluster during cluster operation, which are not limited to this.

[0032] Specifically, Figure 1 The box in each zone platform shows only the structure of one zone platform, which contains a data storage subunit (promtheus component). Among them, the zone platform mainly collects the above cluster data through the following components.

[0033] For example, the zone platform can collect operating system level indicators such as node CPU, memory, disk, network, etc. through the node collector. Figure 1 The box where the operating system layer is located contains various indicators. Not all indicators are shown. The container level indicators such as container CPU, network, and disk usage are collected through the container collector, that is, Figure 1 The various indicators contained in the box where the container layer is located are not all shown; the robust indicators of platform-level service status such as the management platform registration center, management console, and constructor are collected through the program coordination service collector and the network detection collector, that is, Figure 1The various indicators contained in the box where the platform service layer is located are not all shown. Among them, the zone platform can also collect key indicator data such as index rate, query rate, index delay, query delay, data volume, number of shards, number of query queues, number of query rejections, number of write queues, number of write rejections, etc. of each cluster through the search engine service collector.

[0034] For the monitoring platform, an independent data storage unit (promtheus component) is also built inside it. Specifically, the data storage unit in the monitoring platform and the data storage sub-units in each zone platform are all promtheus components. In addition, the promtheus component has a federation mechanism, which allows a data storage unit to obtain cluster data of each indicator from another data storage sub-unit, so as to achieve unified management of cluster data of each cluster in each zone platform.

[0035] The monitoring platform also includes self-developed diagnostic alarm components and visual dashboards. The self-developed diagnostic alarm components can read the indicator data in the data storage subunit to implement alarms for key abnormal indicators that affect the high availability of the ES cluster, such as the cluster's CPU usage, cluster health status, node write queue accumulation, node addition and reduction, etc. At the same time, the cluster can be fully diagnosed and a diagnostic report can be quickly issued. The diagnostic report includes but is not limited to reports at the cluster level, node level, and index level.

[0036] For example, at the cluster level, the report includes but is not limited to: the diagnosis results of indicators such as cluster health status, cluster shard migration, and cluster storage capacity; at the node level, the report includes but is not limited to: the diagnosis results of indicators such as the number of data node shards and the hardware configuration of each node; at the index level, the report includes but is not limited to: the diagnosis results of indicators such as whether the primary shard settings are reasonable, whether the shard data is uniform, and whether the number of fields is reasonable.

[0037] Among them, the visualization dashboard is used to uniformly display the cluster data of each cluster, so that the operation and maintenance personnel can locate the problems in the cluster from the data of each cluster. Among them, the visualization dashboard can specifically be a grafanadashboard component. Exemplarily, the cluster data can be a cluster log, which includes but is not limited to Debug logs, Info logs, Warn logs, Error logs and cluster slow logs. When the cluster logs are obtained, the operation and maintenance personnel can also select different cluster logs and display them in the visualization dashboard, or display all cluster logs to assist the operation and maintenance personnel in locating the problems in the cluster.

[0038] Among them, in this embodiment, a multi-cluster data query method is used to query the cluster data of each cluster in each special zone platform, without having to repeatedly log in to different special zone platforms to query the cluster data of each cluster in different special zone platforms, thereby improving the query efficiency of cluster data by operation and maintenance personnel.

[0039] Specific examples are as follows:

[0040] See also Figure 2 , Figure 2 A flowchart of a multi-cluster data query method provided in an embodiment of the present application is shown, and the method includes the following steps:

[0041] S101, the monitoring center obtains an access request; the access request includes a cluster identifier of a cluster to be accessed; the monitoring platform is used to monitor clusters in multiple zone platforms, and each zone platform monitors cluster data of at least one cluster.

[0042] In one embodiment, the access request is a request for querying cluster data in the cluster to be accessed. Since the monitoring platform monitors clusters in multiple zone platforms at the same time, in order to determine the cluster to be accessed from multiple clusters, the access request also needs to carry the cluster identifier of the cluster to be accessed.

[0043] The access request includes but is not limited to multiple types of requests such as GET request, POST request or PUT request. In this embodiment, the access request may specifically be a POST type request.

[0044] In one embodiment, the cluster table identifier is used to distinguish each cluster and has unique identification. The cluster identifier includes but is not limited to identifiers in the form of numbers, letters, etc., which are not limited to this.

[0045] Specifically, refer to Figure 3 The monitoring platform can generate an access request through the following sub-steps S1011-S1012:

[0046] S1011. The monitoring platform obtains query information input by the user; the query information at least includes a cluster identifier.

[0047] S1012. The monitoring platform generates an access request according to the cluster identifier and sends it to the gateway.

[0048] In one embodiment, the query information may be information input by the operation and maintenance personnel on the monitoring platform, and the information may include at least a cluster identifier. In a specific embodiment, if the operation and maintenance personnel know the specific query address of the cluster data of the cluster to be accessed, the query address may be directly input on the monitoring platform. Then, the monitoring platform obtains the cluster data from the corresponding data storage subunit according to the query address using the federation rules of the data storage unit and the data storage subunit.

[0049] In one embodiment, the access request may be generated by an HTTP protocol debugging proxy tool in the monitoring platform. Specifically, the proxy tool may be a Fiddler tool.

[0050] Exemplarily, the type of access request is a POST type request. The Fiddler tool can automatically generate a POST request based on the file (query information) uploaded by the operation and maintenance personnel. Specifically, Fiddler can generate a random string as a "split boundary" in the request header of the POST request. After that, the monitoring platform can write the random string into the Content-Type field in the HTTP protocol, and write the cluster identifier in the request body, generate a POST request and pass it to the gateway. After that, when parsing the POST request, the gateway can parse the field information in the POST request according to the "split boundary".

[0051] The access request may be an asynchronous JavaScript and XML (Asynchronous JavaScript and XML, AJAX) POST request. By using the AJAX POST type of access request, there is no need to add additional browser plug-ins to the monitoring platform, thus reducing the number of plug-ins used in the monitoring platform.

[0052] In one embodiment, the gateway is a network connector in the monitoring platform, which can realize network interconnection at a level above the network layer. Specifically, the gateway can process a received access request of a certain protocol type, obtain the information (cluster identifier) ​​in the access request, and repackage the received information so that the packaged information can adapt to the protocol requirements of the target platform (i.e., the dedicated platform containing the cluster to be accessed).

[0053] S102: The monitoring platform determines the cluster with the same cluster identifier as the cluster to be visited as the cluster to be visited, and determines the zone platform including the cluster to be visited as the target zone platform, based on the cluster identifier of the cluster to be visited and the cluster identifier corresponding to each cluster.

[0054] S103: The monitoring platform modifies the preset cluster data query address in the target zone platform according to the cluster identifier of the cluster to be accessed, and generates a target query address.

[0055] In one embodiment, the cluster identifier is unique. Therefore, after determining the cluster to be visited according to the cluster identifier, the monitoring platform can also determine the target zone platform including the cluster to be visited according to the clusters monitored by each zone platform.

[0056] It should be noted that since the clusters monitored by each zone platform may handle different services and functions, the data storage subunits of each zone platform are independent. In addition, in order to avoid cluster data leakage, the query address of the cluster data contained in each zone platform is different.

[0057] In the application, the preset cluster data query address in the target zone platform can be: the access address set for the data storage subunit in the target zone platform. Specifically, the preset cluster data query address is the access domain name of the target zone platform. Among them, each zone platform corresponds to an access domain name, and a zone platform usually contains multiple clusters, and the cluster data of each cluster is stored in different locations of the data storage subunit in the zone platform. The monitoring platform can only access the data storage subunit through the access domain name, but the cluster data of each specific cluster in the data storage subunit needs to be further accessed by the rewritten target query address.

[0058] Based on this, the monitoring platform also needs to flexibly generate a target query address for querying the cluster data of the accessed cluster according to the cluster identifier of the access request.

[0059] In a specific embodiment, referring to Figure 4 The monitoring platform can modify the preset cluster data query address through the following steps S131-S138 to obtain the target query address:

[0060] First, the monitoring platform includes a data storage unit for storing cluster data, and each zone platform includes an independent data storage sub-unit for storing cluster data of each cluster in the zone platform; based on this, the monitoring platform needs to process the cluster data of each data storage sub-unit in each zone platform in advance to pre-set the preset cluster data query address corresponding to each zone platform, specifically:

[0061] S131. The monitoring platform stores the cluster data of each data storage subunit in the data storage unit.

[0062] In one embodiment, the data storage subunit and the data storage unit are both stored in the above pair. Figure 1This is explained in the explanation of , and will not be explained again. It should be noted that the monitoring platform can store the cluster data of each data storage sub-unit in the data storage unit by: directly storing the cluster data in the data storage sub-units under different special zone platforms in the data storage unit respectively; or the data storage unit can call the cluster data in the data storage sub-unit. At this time, because the data storage unit has a federal mechanism, it can be realized that a data storage unit obtains the cluster data of each indicator from another data storage sub-unit. Therefore, it can be considered that the acquisition method through the above-mentioned federal mechanism also belongs to storing the cluster data in each data storage sub-unit in another form in the data storage unit, so that the monitoring platform can implement the above-mentioned multi-cluster query method.

[0063] S132. The monitoring platform sets the initial query address of each data storage subunit in the data storage unit respectively; the initial query address includes an access domain name for accessing the data storage subunit.

[0064] It should be noted that no matter which storage method is used, the monitoring platform needs to predetermine the initial query address of each data storage subunit in the data storage unit. That is, when the monitoring platform accesses the cluster data of the cluster to be accessed through the data storage unit, it needs to pre-access the dedicated platform containing the cluster to be accessed, and then it can query the cluster data of the cluster to be accessed from the cluster data of multiple clusters contained in the dedicated platform.

[0065] Therefore, it can be considered that the above initial query address at least includes the access domain name used to access the data storage subunit, that is, it can also be considered as the access domain name for accessing the dedicated platform including the data storage subunit.

[0066] S133. The monitoring platform adds a variable address for accessing cluster data in the access domain name to generate a preset cluster data query address of the zone platform; the variable address is used to perform variable replacement according to the cluster identifier of the cluster to be accessed to generate a target query address.

[0067] In one embodiment, after setting the access domain name, only the cluster data of each cluster under the zone platform can be queried. However, each cluster data usually includes a large amount of data. If all are displayed in the monitoring platform, the cluster data displayed on the visualization dashboard will appear redundant, which is not conducive to the operation and maintenance personnel to analyze the status of each cluster. Therefore, the monitoring platform can also add a variable address for specifically accessing the cluster data of a certain cluster in the access domain name, so as to further query the cluster data to be accessed from multiple cluster data.

[0068] The variable address is set in the preset cluster data query address in the form of a variable, and is used to perform variable replacement according to the cluster identifier to generate a final target query address.

[0069] Exemplarily, the monitoring platform can determine the target storage address of the cluster data of the to-be-accessed cluster in the data storage subunit of the zone platform according to the cluster identifier, and then replace the variable address with the target storage address to generate a target query address.

[0070] Based on the above description, it can be understood that when generating the target query address, the monitoring platform also needs to obtain the storage address of the cluster data of the cluster to be accessed in the corresponding data storage subunit. Therefore, after storing the cluster data of each data storage subunit in the data storage unit in S131, the monitoring platform also needs to perform the following steps S134-S136 on the cluster data in advance, so that the storage address of the cluster data of the cluster to be accessed in the corresponding data storage subunit can be obtained at any time:

[0071] S134. For any dedicated zone platform, if the dedicated zone platform includes multiple clusters, the monitoring platform determines that the cluster data of each cluster is respectively stored in the storage address of the data storage subunit of the dedicated zone platform.

[0072] S135. The monitoring platform establishes a mapping relationship between each storage address and a cluster identifier.

[0073] S136. The monitoring platform caches the mapping relationship to the data storage unit.

[0074] In one embodiment, the cluster data of each cluster under the same zone platform is different, so the storage address of the cluster data of each cluster in the data storage subunit should also be different. Based on this, when each cluster has a unique cluster identifier, for the storage address of any cluster data in the data storage subunit, the monitoring platform can also respectively establish a mapping relationship between the cluster identifier and the corresponding storage address, and then cache the mapping relationship in the data storage unit of the monitoring platform.

[0075] Based on the above description, after the monitoring platform completes the setting of the preset cluster data query address of the zone platform and preprocesses the cluster data, it can generate the target query address through the following steps, as detailed below:

[0076] S137: The monitoring platform determines, according to the mapping relationship, a target storage address of the cluster data corresponding to the cluster identifier of the cluster to be accessed in the data storage subunit.

[0077] S138. The monitoring platform replaces the variable address in the preset cluster data query address with the target storage address to generate a target query address.

[0078] In one embodiment, the above S136 has explained that the mapping relationship is stored in the data storage unit and can be obtained by the monitoring platform at any time. Based on this, the monitoring platform can directly determine the target storage address of the corresponding cluster data in the data storage subunit according to the mapping relationship.

[0079] It should be noted that the monitoring platform can obtain the mapping relationship in the data storage unit in the following manner: Specifically, after the monitoring platform generates an access request according to the cluster identifier, the access request can be sent to the gateway in the monitoring platform, and then the mapping relationship can be obtained from the data storage unit using a preset script in the gateway.

[0080] Specifically, the functions that the above-mentioned preset script needs to execute may include: a parsing function of parsing the access request to obtain a cluster identifier, an acquisition function of obtaining a mapping relationship from a data storage unit, and a modification function of replacing the variable address in the preset cluster data query address with the target storage address to generate a target query address. Therefore, in this embodiment, the script language of the above-mentioned preset script can specifically be the LUA language. Among them, the purpose of sampling the preset script of the LUA language is that when the preset script is embedded in the application (monitoring platform), it can provide flexible expansion and customization functions for the application (monitoring platform), and it is easier to understand and maintain. Therefore, when the gateway receives the access request, it triggers the preset script to execute the above functions to generate the target query address.

[0081] Based on the above description, the target query address generated at this time not only includes the access domain name used to access the dedicated zone platform, but also includes the target storage address of the cluster data of the cluster to be accessed in the data storage subunit from the cluster data of multiple clusters stored in the data storage subunit of the dedicated zone platform.

[0082] Then, S104, the monitoring platform queries the cluster data in the to-be-accessed cluster according to the target query address. In this way, the monitoring platform can directly obtain the corresponding cluster data from multiple clusters under each zone platform according to the cluster identifier, without having to repeatedly log in to different zone platforms to obtain the corresponding cluster data.

[0083] In this embodiment, when the operation and maintenance personnel need to obtain the cluster data in any cluster, the monitoring platform can obtain the access request containing the cluster identifier input by the operation and maintenance personnel to determine the target zone platform containing the cluster to be accessed from multiple zone platforms according to the cluster identifier. In addition, the monitoring platform can also use the cluster identifier to determine the specific storage address of the cluster data of the cluster to be accessed in the zone platform, so as to modify the preset cluster data query address of the pre-set target zone platform and generate a target query address. At this time, the generated target query address includes not only the access domain name used to access the zone platform, but also the specific storage address of the cluster data of the cluster to be accessed in the zone platform. In this way, the monitoring platform can directly obtain the corresponding cluster data from multiple zone platforms according to the target query address, without having to repeatedly log in to different zone platforms to obtain the corresponding cluster data, thereby improving the efficiency of the operation and maintenance personnel in querying cluster data.

[0084] See also Figure 5 , Figure 5 1 is a structural block diagram of a multi-cluster data query device provided in an embodiment of the present application. The multi-cluster data query device in this embodiment includes modules for executing Figures 2 to 4 For details, please refer to the steps in the corresponding embodiment. Figures 2 to 4 as well as Figures 2 to 4 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 5 The multi-cluster data query device 500 may include: an acquisition module 510, a target zone platform determination module 520, a generation module 530 and a query module 540, wherein:

[0085] The acquisition module 510 is used to obtain an access request; the access request includes a cluster identifier of a cluster to be accessed; the monitoring platform is used to monitor clusters in multiple zone platforms, and each zone platform monitors cluster data of at least one cluster.

[0086] The target zone platform determination module 520 is used to determine the cluster with the same cluster identifier as the cluster to be visited as the cluster to be visited, and determine the zone platform including the cluster to be visited as the target zone platform based on the cluster identifier of the cluster to be visited and the cluster identifier corresponding to each cluster.

[0087] The generation module 530 modifies the preset cluster data query address in the target zone platform according to the cluster identifier to generate a target query address.

[0088] The query module 540 is used to query the cluster data in the cluster to be accessed according to the target query address.

[0089] In one embodiment, the monitoring platform includes a data storage unit for storing cluster data, and each zone platform includes an independent data storage subunit for storing cluster data of each cluster in the zone platform; wherein the data storage unit and the data storage subunit are both promtheus components; the multi-cluster data query device 500 also includes:

[0090] The storage module is used to store the cluster data of each data storage subunit into the data storage unit.

[0091] The setting module is used to respectively set the initial query address of each data storage subunit in the data storage unit; the initial query address includes an access domain name for accessing the data storage subunit.

[0092] Add a module to add a variable address for accessing cluster data in the access domain name to generate a preset cluster data query address for the zone platform; the variable address is used to replace the variable according to the cluster identifier of the cluster to be accessed to generate a target query address.

[0093] In one embodiment, the multi-cluster data query device 500 further includes:

[0094] The storage address determination module is used for determining, for any dedicated area platform, if the dedicated area platform includes multiple clusters, the storage address where the cluster data of each cluster is stored in the data storage subunit of the dedicated area platform.

[0095] The establishment module is used to respectively establish a mapping relationship between each storage address and a cluster identifier.

[0096] The cache module is used to cache the mapping relationship to the data storage unit.

[0097] In one embodiment, the generating module 530 is further configured to:

[0098] According to the mapping relationship, the target storage address of the cluster data corresponding to the cluster identifier of the cluster to be accessed in the data storage subunit is determined; the variable address in the preset cluster data query address is replaced by the target storage address to generate the target query address.

[0099] In one embodiment, the multi-cluster data query device 500 further includes a gateway, and the acquisition module 510 is further used to:

[0100] Obtain query information input by the user; the query information at least includes a cluster identifier; generate an access request according to the cluster identifier and send it to the gateway.

[0101] In one embodiment, the multi-cluster data query device 500 further includes:

[0102] The mapping relationship acquisition module is used to acquire the mapping relationship from the data storage unit according to a preset script in the gateway after sending an access request to the gateway.

[0103] In one embodiment, the multi-cluster data query device 500 further includes:

[0104] The display module is used to visualize the cluster data.

[0105] When it is understood that Figure 5 In the structural block diagram of the multi-cluster data query device shown in FIG. Figures 2 to 4 The steps in the corresponding embodiments, and for Figures 2 to 4 Each step in the corresponding embodiment has been explained in detail in the above embodiment. Figures 2 to 4 as well as Figures 2 to 4 The relevant descriptions in the corresponding embodiments are not repeated here.

[0106] Figure 6 : is a structural block diagram of a monitoring platform provided by an embodiment of the present application. Figure 6 As shown, the monitoring platform 600 of this embodiment includes: a processor 610, a memory 620, and a computer program 630 stored in the memory 620 and executable on the processor 610, such as a program of a multi-cluster data query method. When the processor 610 executes the computer program 630, the steps in each embodiment of the multi-cluster data query method described above are implemented, such as Figure 2 Alternatively, the processor 610 implements the above when executing the computer program 630 Figure 5 The functions of each module in the corresponding embodiment are, for example, Figure 5 For details on the functions of modules 510 to 540, please refer to Figure 5 Related description in the corresponding embodiment.

[0107] Exemplarily, the computer program 630 may be divided into one or more modules, one or more modules are stored in the memory 620, and are executed by the processor 610 to implement the multi-cluster data query method provided in the embodiment of the present application. One or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 630 in the monitoring platform 600. For example, the computer program 630 may implement the multi-cluster data query method provided in the embodiment of the present application.

[0108] The monitoring platform 600 may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art will appreciate that Figure 6It is only an example of the monitoring platform 600 and does not constitute a limitation of the monitoring platform 600. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the monitoring platform may also include input and output devices, network access devices, buses, etc.

[0109] The processor 610 may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0110] The memory 620 may be an internal storage unit of the monitoring platform 600, such as a hard disk or memory of the monitoring platform 600. The memory 620 may also be an external storage device of the monitoring platform 600, such as a plug-in hard disk, a smart memory card, a flash memory card, etc. equipped on the monitoring platform 600. Furthermore, the memory 620 may also include both an internal storage unit of the monitoring platform 600 and an external storage device.

[0111] An embodiment of the present application provides a monitoring platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-cluster data query method as described in each of the above embodiments is implemented.

[0112] An embodiment of the present application provides a computer-readable storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-cluster data query method in each of the above embodiments is implemented.

[0113] An embodiment of the present application provides a computer program product. When the computer program product is run on a monitoring platform, the monitoring platform executes the multi-cluster data query method in each of the above embodiments.

[0114] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A multi-cluster data query method, characterized in that: Applied to a monitoring platform, the method comprises: Obtain an access request; the access request includes a cluster identifier of a cluster to be accessed; the monitoring platform is used to monitor clusters in multiple zone platforms, each of the zone platforms monitors cluster data of at least one cluster, and each zone platform is respectively equipped with an independent data storage subunit for collecting and summarizing cluster data of each cluster in each zone platform, the cluster data includes physical resources, container resources of each cluster, and cluster logs when the cluster is running; the data storage unit in the monitoring platform and the data storage subunits in each zone platform are both promtheus components, and the promtheus component has a federation mechanism, which allows a data storage unit to obtain cluster data of various indicators from another data storage subunit; According to the cluster identifier of the cluster to be visited and the cluster identifier corresponding to each of the clusters, a cluster having the same cluster identifier as the cluster to be visited is determined as the cluster to be visited, and a zone platform including the cluster to be visited is determined as the target zone platform; Modify the preset cluster data query address in the target zone platform according to the cluster identifier of the cluster to be accessed to generate a target query address; Querying cluster data in the to-be-accessed cluster according to the target query address; The monitoring platform includes a data storage unit for storing cluster data, and each of the dedicated area platforms includes an independent data storage subunit for storing cluster data of each cluster in the dedicated area platform; Before obtaining the access request, the method further includes: storing the cluster data of each of the data storage subunits into the data storage unit; The initial query address of each data storage subunit in the data storage unit is respectively set; the initial query address includes an access domain name for accessing the data storage subunit; Add a variable address for accessing the cluster data in the access domain name to generate a preset cluster data query address of the zone platform; the variable address is used to perform variable replacement according to the cluster identifier of the cluster to be accessed to generate the target query address; After storing the cluster data of each of the data storage subunits in the data storage unit, the method further includes: For any of the dedicated area platforms, if the dedicated area platform includes a plurality of clusters, determining that the cluster data of each cluster is respectively stored at a storage address of a data storage subunit of the dedicated area platform; Establishing a mapping relationship between each storage address and the cluster identifier respectively; The mapping relationship is cached in the data storage unit.

2. The multi-cluster data query method according to claim 1, characterized in that: The step of modifying the preset cluster data query address in the target zone platform according to the cluster identifier of the cluster to be accessed to generate a target query address includes: Determine, according to the mapping relationship, a target storage address of the cluster data corresponding to the cluster identifier of the cluster to be accessed in the data storage subunit; The target storage address is used to replace the variable address in the preset cluster data query address to generate the target query address.

3. The multi-cluster data query method according to claim 2, characterized in that: The monitoring platform also includes a gateway; The obtaining access request includes: Acquire query information input by a user; the query information at least includes the cluster identifier; Generate the access request according to the cluster identifier, and send it to the gateway; Before determining the target storage address of the cluster data corresponding to the cluster identifier of the to-be-accessed cluster in the data storage unit according to the mapping relationship, the method further includes: After sending the access request to the gateway, the mapping relationship is obtained from the data storage unit according to a preset script in the gateway.

4. The multi-cluster data query method according to claim 3, characterized in that: After querying the cluster data in the to-be-accessed cluster according to the target query address, the method further includes: The cluster data is visualized.

5. A multi-cluster data query device, characterized in that: Applied to a monitoring platform, the device comprises: An acquisition module is used to obtain an access request; the access request includes a cluster identifier of a cluster to be accessed; the monitoring platform is used to monitor clusters in multiple zone platforms, each of which monitors cluster data of at least one cluster, and each zone platform is respectively equipped with an independent data storage subunit for collecting and summarizing cluster data of each cluster in each zone platform, wherein the cluster data includes physical resources, container resources of each cluster, and cluster logs when the cluster is running; the data storage unit in the monitoring platform and the data storage subunits in each zone platform are both promtheus components, and the promtheus component has a federation mechanism, which allows a data storage unit to obtain cluster data of various indicators from another data storage subunit; a target zone platform determination module, configured to determine, according to the cluster identifier of the cluster to be visited and the cluster identifier corresponding to each of the clusters, a cluster having the same cluster identifier as the cluster to be visited as the cluster to be visited, and to determine the zone platform including the cluster to be visited as the target zone platform; A generation module, which modifies a preset cluster data query address in the target zone platform according to the cluster identifier to generate a target query address; A query module, used for querying the cluster data in the cluster to be accessed according to the target query address; The monitoring platform includes a data storage unit for storing cluster data, and each of the dedicated area platforms includes an independent data storage subunit for storing cluster data of each cluster in the dedicated area platform; Before obtaining the access request, the method further includes: storing the cluster data of each of the data storage subunits into the data storage unit; The initial query address of each data storage subunit in the data storage unit is respectively set; the initial query address includes an access domain name for accessing the data storage subunit; Add a variable address for accessing the cluster data in the access domain name to generate a preset cluster data query address of the zone platform; the variable address is used to perform variable replacement according to the cluster identifier of the cluster to be accessed to generate the target query address; After storing the cluster data of each of the data storage subunits in the data storage unit, the method further includes: For any of the dedicated area platforms, if the dedicated area platform includes a plurality of clusters, determining that the cluster data of each cluster is respectively stored at a storage address of a data storage subunit of the dedicated area platform; Establishing a mapping relationship between each storage address and the cluster identifier respectively; The mapping relationship is cached in the data storage unit.

6. A monitoring platform, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images