Method and system for realizing chart query based on standardized access process

Abstract

The invention provides a method and a system for realizing chart query based on a standardized access process, and belongs to the technical field of monitoring data processing and visualization. The system formulates a unified data format and interface protocol standard, accesses data sources such as a database, a log system or a message queue according to the standard, and realizes the chart query. And preprocessing and converting the acquired data, and then writing the preprocessed and converted data into Prometheus. A query request containing PromQL and interface parameters is received, unit conversion is executed according to the interface parameters, query adaptation is completed, a graph is generated according to user configuration after a query result is obtained, and single-graph full-screen, multi-graph column dividing and hovering prompt are supported; and real-time query and update according to the refresh parameters are supported. And the alarm rule, the recording rule, the grabbing configuration and the sending strategy are uniformly managed based on the configuration file.

Description

Technical Field

[0001] This invention belongs to the field of monitoring data processing and visualization technology, and in particular relates to a method and system for implementing chart query based on a standardized access process. Background Technology

[0002] Prometheus, as a monitoring and alerting tool, is widely used for the collection and processing of multidimensional data, but it still has objective limitations in terms of chart query capabilities. First, existing Prometheus systems typically lack unified standards and processes for accessing different data sources, often requiring customized development and configuration for different data sources. This leads to high system integration complexity, increased development and maintenance costs, and reduced system flexibility and scalability.

[0003] Secondly, chart queries typically rely on PromQL. However, due to significant differences in data formats and structures across various data sources, directly using PromQL for queries may encounter issues such as unsupported operators or data type mismatches, limiting chart query capabilities and making it difficult for users to fully utilize data for visualization and analysis. Furthermore, existing technologies also suffer from a lack of flexibility in chart display, insufficient personalization, and inadequate real-time update support, further restricting the application scope and user experience of the Prometheus system.

[0004] Based on the above issues, there is a need for a Prometheus chart query solution that can access multi-source data in a standardized manner and provide a more adaptable and flexible mechanism in the query and display process, so as to reduce integration costs and improve the query and display experience. Summary of the Invention

[0005] In view of this, the present invention aims to propose a method and system for implementing chart query based on a standardized access process, so as to at least solve one of the problems in the background art.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A method for implementing chart querying based on a standardized access process, applied to the Prometheus monitoring system, includes: Establish a unified data source access standard, which shall include at least data format specifications and interface protocol specifications; Based on the data source access standard, the system receives the data source access configuration and completes the legality verification. It then establishes a data acquisition connection with the target data source according to the interface protocol and collects raw monitoring data. The original monitoring data is preprocessed and transformed into a time-series data format that can be parsed by Prometheus and written into the Prometheus time-series database. Receive a chart query request, which includes at least a PromQL query statement and interface parameters related to the query; Based on the interface parameters and the data source access standard, the PromQL query statement is subjected to query adaptation processing. The query adaptation processing includes at least operator support processing, unit conversion processing and data type matching processing to obtain the adapted query statement. The adapted query statement is executed to obtain the query results, and the target chart is generated and displayed based on the query results.

[0007] Furthermore, the data format specification at least specifies one or more of the data structure, encoding method, data type definition, numerical precision, and text character set; the interface protocol specification at least specifies one or more of the data transmission method, request and response data structure, authentication and authorization mechanism, and error handling method, and the request at least includes a data source identifier and data request range information.

[0008] Furthermore, the preprocessing and transformation includes at least: uniformly mapping fields from different data sources to generate time-series samples containing indicator names, tag key-value sets, timestamps, and indicator values; performing data type conversion on the indicator values ​​to meet Prometheus's parsing requirements; and writing the time-series samples into the Prometheus time-series database.

[0009] Furthermore, the unit conversion process includes: parsing the target measurement unit identifier from the interface parameters; when the target measurement unit identifier is a byte unit, multiplying the indicator value by 1024 according to a preset conversion rule; when the target measurement unit identifier is a bit unit, multiplying the indicator value by 1000 according to a preset conversion rule; and performing a consistent conversion on the operation items involving different units before or during the query.

[0010] Furthermore, the operator support processing includes: identifying operators in the PromQL query statement that do not match the target data source data type based on the data type and preset operator mapping rules, and replacing the mismatched operators with equivalent operators that can be executed by Prometheus; and or, splitting the PromQL query statement into intermediate statistical statements and a final query statement, wherein the intermediate statistical statements are used to generate record rule results, and the final query statement completes the chart query based on the record rule results.

[0011] Furthermore, generating and displaying the target chart includes: selecting a chart type based on the data characteristics of the query results; receiving the user's chart display configuration to complete chart style customization and layout customization, wherein the chart display configuration includes at least one or more of the following: indicator selection, resource selection, product selection, and title content; the layout customization supports at least single chart full-screen display and multi-chart column display; and outputting detailed data information of the data point when the user's pointer hovers over the data point.

[0012] Furthermore, the chart query request further includes a refresh parameter, which is used to indicate a real-time query mode; in the real-time query mode, the adapted query statement is periodically executed according to the refresh parameter, and the target chart is refreshed with the updated query results.

[0013] Furthermore, the Prometheus monitoring system also executes a configuration file-driven management process, which includes at least alarm rule management, logging rule management, capture configuration management, and sending policy management. The management process is used to load the corresponding rules or configurations in the form of configuration files and apply them to the Prometheus monitoring system.

[0014] Furthermore, the target data source includes at least one or more of a database, a log system, and a message queue; establishing a data acquisition connection includes calling the access adaptation component corresponding to the target data source according to the interface protocol to complete data acquisition and format conversion.

[0015] Furthermore, this solution discloses a Prometheus system that implements chart query capabilities based on a standardized access process, including: a data source access module, used to define a unified access standard and access data sources according to the access standard, and used to perform preprocessing and transformation on the accessed data; a chart query module, used to receive query requests and execute queries based on PromQL, and supports real-time queries to update chart data; and a user interface module, used to provide a human-computer interaction interface for data source access configuration, query statement writing, and chart display.

[0016] Furthermore, the chart query module includes a query adaptation submodule, which is used to perform unit conversion processing based on interface parameters, and to perform operator support processing and data type matching processing based on data type, so as to output the adapted PromQL query statement.

[0017] Furthermore, this solution discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described thereon.

[0018] Compared with existing technologies, the method and system for implementing chart query based on a standardized access process described in this invention have the following advantages: (1) This invention standardizes the data source access process, establishes a unified data format and interface protocol, and enables different data sources to access Prometheus in a consistent manner, reducing the customized development work for a single data source and improving the system's scalability and flexibility. (2) This invention optimizes the Prometheus query mechanism to enable the query mechanism to support more types of operators and data types, alleviate the query difficulties caused by differences in the format and structure of different data sources, improve query accuracy and efficiency, and reduce the probability of query failure due to unsupported operators or mismatched data types. (3) This invention performs unit conversion based on interface parameters, automatically converts data of different units before and during the query, ensures unit consistency and improves the comparability of results, and is especially suitable for data source scenarios such as storage where bytes and bits are mixed, avoiding misjudgment caused by inconsistent units; (4) This invention supports the customization of configuration items such as indicators, resources, products and titles through a flexible chart display mechanism, supports single full-screen and multi-column layout, and provides an interactive method of mouse hover data point prompts, making the data presentation more intuitive and improving the efficiency of user analysis and investigation. (5) This invention achieves the integration and automation of management actions through configuration file-based management mechanisms such as alarm rule management, record rule management, capture configuration management and sending strategy management, which simplifies maintenance operations and improves system maintainability and ease of use; (6) The system level of the present invention has a user-friendly interface and real-time update capability, and can update the chart data in real time as requested after the chart query results are generated, thereby enhancing the user's operating experience. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram of the method described in an embodiment of the present invention. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0021] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0022] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0023] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Example 1: A Chart Query Method Based on Standardized Access Procedures This embodiment is implemented in a Prometheus monitoring system environment, which includes a data source access module, a chart query module, and a user interface module. The data source access module is used to define a unified access standard and complete data preprocessing and transformation; the chart query module executes queries based on PromQL and supports real-time queries; the user interface module provides a human-computer interaction interface for access configuration, query statement writing, and chart display.

[0025] Step one: Define a unified data source access standard. The access standard should include at least two parts: data format specifications and interface protocol specifications. Regarding data format specifications, constraints should be imposed on data structures, encoding methods, data type definitions, numerical precision, and text character sets to ensure the system can uniformly identify and process data from different data sources. Regarding interface protocol specifications, the data transmission methods, request and response formats, authentication and authorization mechanisms, and error handling methods should be clearly defined. It should also be stipulated that request messages must include at least the data source identifier and the data request scope to ensure that different data sources can access the Prometheus system according to the same communication rules.

[0026] Step two: Connect to the target data source and collect data according to the access standard. The data source access module reads or receives the data source access configuration and verifies its validity according to the data format specifications and interface protocol specifications. After successful verification, the data source access module establishes a connection with the target data source and collects raw monitoring data. The target data source may include databases, log systems, and message queues, etc. The system uses a unified format and protocol to identify, parse, and access different data sources.

[0027] Step 3 involves preprocessing and transforming the collected data before writing it into Prometheus. The data source access module performs preprocessing on the raw monitoring data, including field identification and mapping, data type normalization, and timestamp and tag information organization. This converts data from different sources into a time-series data format that Prometheus can parse and process, and writes it into the Prometheus time-series data store to ensure that subsequent PromQL queries can be executed on data of the same format.

[0028] Step four: Receive chart query requests and perform query adaptation. Users write query statements using PromQL and initiate chart query requests. The chart query module receives the query request, which includes at least the PromQL query statement and interface parameters related to the query. Considering that differences in data structures from different data sources may lead to unsupported operators or data type mismatches, in this embodiment, the chart query module performs adaptation processing on the PromQL query before execution. This enables the query mechanism to support more types of operators and data types, meeting diverse query needs and improving query accuracy and efficiency.

[0029] Step 5: Perform unit conversion optimization based on interface parameters. In the query adaptation process, the chart query module reads the unit information from the interface parameters and performs conversion according to preset unit conversion rules to ensure data consistency in units. For example, when similar storage metrics are measured in bytes and bits in different data sources, the system can convert the units before querying based on the interface parameters and automatically convert them during the query process when different unit operations are involved, ensuring calculation accuracy. The default conversion for byte units is multiplying by 1024, and for bit units, it's multiplying by 1000. This unit conversion optimization enhances the comparability and usability of query results.

[0030] Step six: Execute the PromQL query and output the query results. The chart query module retrieves the corresponding data results from Prometheus based on the adapted query statement; when necessary, it can also select appropriate result aggregation or serialization methods according to the data characteristics to facilitate subsequent chart rendering and display.

[0031] Step 7: Generate and display the target chart. The chart query module selects the chart type based on the data characteristics of the query results and renders the data into a chart for display by the user interface module. Users can customize the chart style, layout, and interaction methods according to their needs. The customization includes at least the selection of indicators, resources, products, and title content; the layout method supports at least single chart full-screen display and multi-chart column display; and it supports data point hints, displaying detailed data information for the corresponding data point when the mouse hovers over it, to enhance the user's ability to explore and analyze the data.

[0032] Step 8: Real-time Query and Chart Refresh. In one implementation, the chart query module supports real-time querying, enabling it to update chart data in real time based on user requests. Specifically, the chart query request can carry refresh parameters. The chart query module periodically triggers query execution according to the refresh parameters and refreshes the displayed chart with the updated query results, thereby meeting the need for continuous observation of real-time monitoring indicators.

[0033] Step nine, configuration-based integrated management. In this embodiment, the system's alarm rule management, logging rule management, capture configuration management, and sending policy management are all managed uniformly based on configuration files. Through the configuration file loading and activation mechanism, the rule and policy maintenance process is simplified, reducing maintenance costs caused by manual operation and dispersed configurations, and improving system maintainability and usability.

[0034] Example 2: Prometheus System Example As described in the above embodiments, the present invention also provides a Prometheus system that implements chart query capabilities based on a standardized access process. The system includes at least a data source access module, a chart query module, and a user interface module. The data source access module defines a unified access standard and implements data preprocessing and transformation; the chart query module implements chart queries based on PromQL and supports real-time query updates; the user interface module provides a visual interface for data source access configuration, query statement writing, and chart display, facilitating users to complete access and query operations.

[0035] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for implementing chart query based on a standardized access process, applied to the Prometheus monitoring system, characterized in that: include: Establish a unified data source access standard, which shall include at least data format specifications and interface protocol specifications; Based on the data source access standard, the system receives the data source access configuration and completes the legality verification. It then establishes a data acquisition connection with the target data source according to the interface protocol and collects raw monitoring data. The original monitoring data is preprocessed and transformed into a time-series data format that can be parsed by Prometheus and written into the Prometheus time-series database. Receive a chart query request, which includes at least a PromQL query statement and interface parameters related to the query; Based on the interface parameters and the data source access standard, the PromQL query statement is subjected to query adaptation processing. The query adaptation processing includes at least operator support processing, unit conversion processing and data type matching processing to obtain the adapted query statement. The adapted query statement is executed to obtain the query results, and the target chart is generated and displayed based on the query results.

2. The method according to claim 1, characterized in that, The data format specification shall at least define one or more of the following: data structure, encoding method, data type definition, numerical precision, and text character set; the interface protocol specification shall at least define one or more of the following: data transmission method, data structure of request and response, authentication and authorization mechanism, and error handling method, and the request shall at least include data source identifier and data request scope information.

3. The method according to claim 1, characterized in that, The unit conversion process includes: parsing the target measurement unit identifier from the interface parameters; when the target measurement unit identifier is a byte unit, multiplying the indicator value by 1024 according to a preset conversion rule; when the target measurement unit identifier is a bit unit, multiplying the indicator value by 1000 according to a preset conversion rule; and performing a consistent conversion on the operation items involving different units before or during the query.

4. The method according to claim 1, characterized in that, The operator support processing includes: identifying operators in the PromQL query statement that do not match the target data source data type based on data type and preset operator mapping rules, and replacing the mismatched operators with equivalent operators that can be executed by Prometheus; and or, splitting the PromQL query statement into intermediate statistical statements and a final query statement, wherein the intermediate statistical statements are used to generate record rule results, and the final query statement completes the chart query based on the record rule results.

5. The method according to claim 1, characterized in that, The process of generating and displaying the target chart includes: selecting the chart type based on the data characteristics of the query results; receiving the user's chart display configuration to complete the chart style customization and layout customization, wherein the chart display configuration includes at least one or more of the following: indicator selection, resource selection, product selection, and title content; the layout customization supports at least single chart full-screen display and multi-chart column display; and outputting the detailed data information of the data point when the user's pointer hovers over the data point.

6. The method according to claim 1, characterized in that, The chart query request further includes refresh parameters, which are used to indicate a real-time query mode. In the real-time query mode, the adapted query statement is periodically executed according to the refresh parameters, and the target chart is refreshed with the updated query results.

7. The method according to claim 1, characterized in that, The Prometheus monitoring system also executes a configuration file-driven management process, which includes at least alarm rule management, logging rule management, capture configuration management, and sending policy management. The management process is used to load the corresponding rules or configurations in the form of configuration files and apply them to the Prometheus monitoring system.

8. The method according to claim 1, characterized in that, The target data source includes at least one or more of a database, a log system, and a message queue; establishing a data acquisition connection includes calling the access adaptation component corresponding to the target data source according to the interface protocol to complete data acquisition and format conversion.

9. A Prometheus system that implements chart query capabilities based on a standardized access process, characterized in that, include: The data source access module is used to define a unified access standard and access the data source according to the access standard, and to perform preprocessing and transformation on the accessed data; The chart query module is used to receive query requests and execute queries based on PromQL, and supports real-time queries to update chart data; The user interface module provides a human-computer interaction interface for data source access configuration, query statement writing, and chart display.

10. The system according to claim 9, characterized in that, The chart query module includes a query adaptation submodule, which is used to perform unit conversion processing based on interface parameters and to perform operator support processing and data type matching processing based on data type, so as to output the adapted PromQL query statement.

Images