Grafana-based cloud application visual monitoring method

Abstract

The invention discloses a Grafana-based cloud application visual monitoring method, which is characterized in that a cloud platform and a unified index acquisition platform are combined, data are obtained through intranet access, application monitoring is completed by means of a simplified graphical interface, a Web Api interface of Grafana is utilized to generate a visual display big disk, and a corresponding permission system is synchronously constructed. And integrating with a cloud application deployment platform, and default-configuring an overall resource monitoring large disk. According to the visualized monitoring method, Grafana complex operation is simplified in an application layer, the visualized monitoring configuration difficulty is reduced, seamless connection with an application deployment platform is achieved, a user focuses on an application exposure index, additional access to a visualized platform is not needed, Grafana operation and authority management do not need to be worried about, the security of sensitive indexes is fully guaranteed by means of intranet access, and the user experience is improved. And the sensitive data leakage risk is greatly reduced.

Description

Technical Field

[0001] The present invention relates to a cloud application visualization display technology, and in particular to a cloud application visualization monitoring method based on Grafana. Background Art

[0002] In the current era, cloud-native technology is booming, and it is a major trend for applications to move from traditional deployment to cloud deployment and operation. For applications running on the cloud, it is of utmost importance to clearly understand the operation of the application, and business applications need to clearly understand the business situation through business indicators. Therefore, visualization of monitoring indicators for cloud applications is one of the important technologies to meet these needs.

[0003] Grafana is a data visualization tool that is widely recognized by the business, but there is no mature solution for combining it with cloud platforms. Currently, most cloud platforms deploy Grafana applications directly on cloud platforms for visualization monitoring, and users are required to manually configure them. This requires users to have relevant knowledge of the indicator collection process and the specific usage of Grafana. However, it is difficult for users to be fully familiar with its complex operations.

[0004] For example, a monitoring method for server performance in a cloud environment and container performance on a node (CN201711111884) performs visual monitoring on a cloud platform, but the visualization configuration is still manually configured based on the user's exposure to the native Grafana platform, and does not simplify related functions and permission management.

[0005] The monitoring indicators and business indicators of applications are extremely sensitive data, so the access addresses cannot be easily exposed. Different user roles have different control permissions for cloud applications. These permissions have different operation permissions for visualized monitoring data, so synchronization of permission management is crucial. However, there is currently no unified solution for synchronizing user data and permissions between cloud platforms and Grafana applications.

[0006] Therefore, a more efficient Grafana-based solution for visual monitoring of cloud applications is needed. Summary of the invention

[0007] The problem to be solved by the present invention is to solve the problem of visualization of monitoring data of cloud applications, and to provide a visualization solution for monitoring of a cloud platform to simplify the complex configuration of cloud applications when using Grafana, and to better combine the advantages of unified collection of cloud platform monitoring indicators, and strengthen the unified authority management of Grafana and cloud platform users.

[0008] In order to solve the problem of sensitive data exposure, synchronize user data and manage permissions between cloud platform users and Grafana users, build a simplified indicator data model, and reduce the complexity of configuring the visualization platform, the present invention provides a method for visual monitoring of cloud applications based on Grafana, which is characterized by specifically including the following steps:

[0009] Step 1: Build and deploy a unified Grafana application for cloud applications and configure related parameters. The Grafana application serves as the entry point for all cloud applications to visually monitor the disk access and provides a basis for the system to call its Web API.

[0010] Step 2: Obtain the user permissions of the cloud application and the different management permissions of the user on the application, which are used to synchronize user data and permissions in the Grafana application. The permissions that users in the cloud platform can use to call APIs in the Grafana system are consistent with their control over the project in the cloud platform, which is used to implement dual permission management for users.

[0011] Step 3: Build and configure data structure parameters for displaying relevant monitoring indicators in the cloud platform monitoring configuration interface. The data structure parameters include visualization indicator parameters, which include chart type, indicator name and operation function. Localizing complex dial data structures can simplify the configuration method, and the data is stored in the local database, reducing the risk of loss of Grafana system configuration data.

[0012] Step 4: The system converts the data structure parameters described in step 3 into specific Grafana configurations and generates relevant visual monitoring dashboards in Grafana through Web API.

[0013] The step 1 includes:

[0014] Step 1-1: Configure the unified indicator data source access address on the cloud, collect the monitoring indicator data of the cloud application to the unified data indicator storage platform, and provide a unified data acquisition entry for the visual monitoring dashboard. At the same time, Grafana accesses the relevant data source through the internal network of the cloud platform to reduce the risk of data exposure.

[0015] Step 1-2: Configure the Web API address that the system calls uniformly and the related token call authentication to provide basic call configuration for the subsequent use of Web API to create, modify, and destroy the disk.

[0016] In step 2, the synchronization of user data and permissions includes: obtaining user data of cloud applications, dividing user operation permissions for the visual monitoring dashboard according to different user permissions for cloud applications, and creating, granting permissions and deleting user data synchronization operations through scheduled tasks.

[0017] In step 3, the data structure parameters of the monitoring indicators of the cloud application include: specifying the name and type of the monitoring indicator; specifying the display pattern of the monitoring indicator, the pattern includes a time series diagram and a pie chart, etc.; the processing function corresponding to the monitoring indicator, the processing function includes summation, averaging and aggregation; specifying the filtering label of the monitoring indicator, and storing the data in the local database at the same time, reducing the risk of loss of Grafana system configuration data.

[0018] Specifically, step 4 includes: the system converts the simplified indicator data structure parameter configuration to a complex Grafana configuration item according to the user, generates a relevant customized visual monitoring dashboard on Grafana by calling the relevant Web API, and generates an exclusive jump link for the dashboard in the system for jump access.

[0019] The beneficial effects achieved by the present invention are as follows: a scheme and system for displaying the visualization of monitoring indicators based on the application configuration method combined with the Grafana software is realized, the complicated operation of Grafana is shielded at the application level, the configuration difficulty of application visualization monitoring is reduced, and the system is integrated with the application deployment platform. Users do not need to access the visualization platform for cloud applications separately and do not need to pay attention to any Grafana-related operations and user authority management. They only need to pay attention to the monitoring indicators exposed by the application itself to realize the visualization monitoring of the application. With the help of the internal access method of the cloud application, the problem of sensitive indicator exposure is minimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of an exemplary embodiment of the present invention;

[0021] Figure 2 The figure is a flow chart of an overall scheme of an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose and technical solution of the present invention more clearly described, the present invention is described in detail below with reference to the accompanying drawings and specific implementations:

[0023] The present invention provides a solution and system design for visual monitoring of cloud applications based on Grafana, which implements a solution and system for visual monitoring of cloud applications by a cloud platform, such as Figure 1 As shown in the figure, the main process of the solution architecture includes the following steps:

[0024] Step 1: Build and deploy a unified Grafana application for cloud applications and configure related parameters. The Grafana application serves as the entry point for all cloud application visualization dashboards and provides a basis for the system to call its Web API.

[0025] The step 1 further includes:

[0026] Step 1-1: Deploy the Grafana application and configure the relevant cloud data collection address: Configure the unified indicator data source access address on the cloud. The monitoring indicator data of the cloud application should be collected to the same data indicator storage platform, and then provide a unified data acquisition entry for the visual monitoring dashboard. At the same time, Grafana accesses the relevant data source through the internal network of the cloud platform to reduce the risk of data exposure.

[0027] Step 1-2: Configure the Web API access address of Grafana and the API token call authentication for unified system calls on the platform, and provide basic call configuration for subsequent use of Web API to create, modify and destroy the dashboard, user authority division, etc.

[0028] Step 2: Synchronize the permission control of cloud platform users and Grafana users: obtain the user permissions of the cloud application and the different management permissions of the user to the application, which are used to synchronize user data and permissions in the Grafana application, and realize dual permission management for users. The control rights of users in the cloud platform over the project are consistent with the permissions of the APIs that can be called in the Grafana system. In step 2, the user data synchronization needs to obtain the user data of the cloud application, and divide the user's operation permissions for the visualization dashboard according to the different permissions of the user to the cloud application. The synchronization operation of user data is created, authorized, deleted, etc. through scheduled tasks. Through the processing of scheduled tasks, the concurrency of Grafana requests can be greatly reduced.

[0029] Step 3: Build and configure relevant simplified indicator data for the application, and store it in the database: Build and configure the data structure parameters for displaying relevant monitoring indicators in the cloud platform monitoring configuration interface. The data structure parameters include visualization indicator parameters, and the visualization indicator parameters include chart type, indicator name and operation function. Localizing the complex dial data structure can simplify the configuration method, and the data is stored in the local database, reducing the risk of loss of Grafana system configuration data.

[0030] In step 3, the monitoring indicator data structure parameters of the cloud application should at least include: 1. the name and type of the specified monitoring indicator 2. the display pattern of the specified indicator (such as: time series diagram, pie chart, etc.), 3. the processing function of related indicators (such as: sum, average, aggregation, etc.), 4. the specified label screening of the related indicator cluster.

[0031] Step 4: Generate relevant dashboards and jump links in Grafana based on the simplified configuration. At the same time, through scheduled tasks, you can directly rebuild the lost dashboard data in Grafana through local configuration data: the system parses the relevant monitoring indicator data parameters of the cloud application into Grafana-related complex configurations, and builds relevant visualization dashboards in the Grafana application through Web API, and generates a dedicated access jump link for each visualization monitoring dashboard, and generates a general monitoring panel for each cloud application.

[0032] Specifically, the system converts the simplified indicator data structure parameter configuration by the user into a complex Grafana configuration item, generates a customized visual monitoring dashboard on Grafana by calling the relevant Web API, and generates a dedicated jump link for the dashboard in the system. This method extracts the complex Grafana chart configuration into a clear and simple configuration type. Users do not need to perform visual configuration in a complex graphical interface, which simplifies the configuration method of the visual dashboard. At the same time, through scheduled tasks, the lost dashboard can be directly rebuilt in the Grafana system through local configuration data.

[0033] like Figure 2 In an exemplary embodiment shown, the cloud platform uses Prometheus data as an indicator collection platform to provide a data source for the Grafana system. Since the two systems are deployed in the cloud platform at the same time, data only needs to be obtained using the local intranet. The GrafanaApi address exposure function is used to expose the WebApi service, reducing the risk of data exposure. At the same time, the relevant Api token authentication is configured on the cloud platform to provide relevant basic configuration for subsequent Web Api call functions.

[0034] The management of projects and applications is isolated. Different applications in Grafana belong to different organizational domains, which naturally forms the isolation of Grafana domains and user permissions. The user permissions and projects of the two platforms are synchronized. Secondly, different applications have independent control permissions in the same domain, which more effectively combines the user's permission control of projects and applications in the cloud platform to achieve dual control of cloud platform projects and user permissions. At the same time, according to the survival and stop of projects and applications, the relevant dashboard data is automatically deleted and modified to reduce the query load of the data source.

[0035] Set up a relevant simplified visualization configuration page on the cloud platform, inject the simplified local configuration directly into Grafana through the ability of Web Api, build a corresponding visualization page dashboard, and generate relevant jump access links. After the configuration is completed, the relevant data is stored in MySQL (locally in the database) to reduce the risk of data loss. The present invention provides an implementation scheme and system design for visual monitoring of cloud applications based on Grafana, which shields the complicated operation of Grafana at the application level, reduces the configuration difficulty of application visual monitoring, and integrates with the application deployment platform. Users do not need to access the visualization platform for cloud applications separately and do not need to pay attention to any Grafana-related operations and user authority management. They only need to pay attention to the monitoring indicators exposed by the application itself to achieve visual monitoring of the application. With the help of the internal access method of the cloud application, the problem of sensitive indicator exposure is minimized.

[0036] The present invention may also have many other implementation modes. The above embodiments do not limit the present invention in any way. Without departing from the spirit and essence of the present invention, technicians familiar with the field may make various corresponding changes and modifications based on the present invention. All other improvements and applications made to the above embodiments in an equivalent transformation manner should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for visual monitoring of cloud applications based on Grafana, characterized in that: Includes the following: Step 1: Build and deploy a unified Grafana application for cloud applications and configure related parameters. The Grafana application serves as the entry point for all cloud applications to visually monitor the disk access and provides a basis for the system to call its Web API. Step 2: Obtain the user permissions of the cloud application and the different management permissions of the user on the application, which are used to synchronize user data and permissions in the Grafana application. The permissions that users in the cloud platform can use to call APIs in the Grafana system are consistent with their control over the project in the cloud platform, which is used to implement dual permission management of users; Step 3: Construct and configure data structure parameters for displaying relevant monitoring indicators in the cloud platform monitoring configuration interface, wherein the data structure parameters include visualization indicator parameters, and the visualization indicator parameters include chart type, indicator name, and operation function; Step 4: The system converts the data structure parameters described in step 3 into a specific Grafana configuration and generates a related visual monitoring dashboard in Grafana through the Web API.

2. According to claim 1, a method for visual monitoring of cloud applications based on Grafana is characterized in that: The step 1 includes: Step 1-1: Configure the unified indicator data source access address on the cloud, collect the monitoring indicator data of the cloud application to the unified data indicator storage platform, and provide a unified data acquisition entry for the visual monitoring dashboard. At the same time, Grafana accesses the relevant data source through the internal network of the cloud platform; Step 1-2: Configure the Web API address that the system calls uniformly and the related token call authentication to provide basic call configuration for the subsequent use of Web API to create, modify, and destroy the disk.

3. The method for visual monitoring of cloud applications using Grafana according to claim 2, characterized in that: In step 2, the synchronization of user data and permissions includes: obtaining user data of cloud applications, dividing user operation permissions for the visual monitoring dashboard according to different user permissions for cloud applications, and creating, granting permissions and deleting user data synchronization operations through scheduled tasks.

4. The method for visual monitoring of cloud applications using Grafana according to claim 3, characterized in that: In step 3, the data structure parameters of the monitoring indicators of the cloud application include: specifying the name and type of the monitoring indicator; specifying the display pattern of the monitoring indicator, the pattern includes a time series diagram and a pie chart, etc.; the processing function corresponding to the monitoring indicator, the processing function includes summation, averaging and aggregation; specifying the filtering label of the monitoring indicator, and storing the data in the local database at the same time.

5. The method for visual monitoring of cloud applications using Grafana according to claim 4, characterized in that: Specifically, step 4 includes: the system converts the simplified indicator data structure parameter configuration to a complex Grafana configuration item according to the user, generates a relevant customized visual monitoring dashboard on Grafana by calling the relevant Web API, and generates an exclusive jump link for the dashboard in the system for jump access.

Images