Microservices architecture-based monitoring and early warning method

Abstract

The present invention relates to a microservices architecture-based monitoring and early warning method, comprising: S1: a metric collection agent collecting metric data during the operation of a microservices system in real time; S2: the metric collection agent transmitting the metric data to a monitoring system; S3: the monitoring system storing and analyzing the received metric data; S4: displaying a query result and an analysis result by means of a visualization tool; and S5: predicting the development trend of the metric data on the basis of the received metric data, and if the predicted metric data exceeds a set metric data threshold limit value, carrying out early warning. The present invention achieves comprehensive monitoring of metrics across the entire microservices system and uses an efficient and rational approach for collection, transmission and storage.

Description

A monitoring and early warning method based on microservice architecture Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a monitoring and early warning method, terminal equipment, and storage medium based on a microservice architecture. Background Art

[0002] With the rapid development of information technology and the widespread adoption of microservices, the microservices architecture has been widely adopted in modern application development. Microservices divide a large application system into multiple small, independent services, each running in an independent process. This architecture offers high cohesion and low coupling, significantly improving system maintainability and scalability. However, the application of microservices also presents new challenges, such as inter-service dependencies, resource allocation for service calls, and monitoring and risk warning for hardware metrics and software service performance.

[0003] The existing monitoring system has the following technical problems:

[0004] (1) Existing monitoring systems focus on monitoring hardware operating indicators or software operating status. There is no complete monitoring ecosystem that can include all aspects of the microservice architecture.

[0005] (2) Existing monitoring systems generally require the monitored resources or services to regularly report specified monitoring indicators, which imposes certain transformation and adaptation costs on the services.

[0006] (3) After obtaining the monitoring data, there is no effective use and analysis of the monitoring data. Most of the data is only displayed by time or other dimensions. It is impossible to automatically issue early warnings based on preset early warning formulas or prediction algorithm models, and to discover and deal with potential risks in advance.

[0007] Therefore, proposing a monitoring and early warning method based on microservice architecture is of great significance to improving the reliability and stability of the entire microservice system.

[0008] Summary of the Invention

[0009] In order to solve the above problems, the present invention proposes a monitoring and early warning method, terminal device and storage medium based on microservice architecture.

[0010] The specific plan is as follows:

[0011] A monitoring and early warning method based on a microservice architecture includes the following steps:

[0012] S1: The indicator collection agent collects indicator data in real time during the operation of the microservice system;

[0013] S2: The indicator collection agent transmits the indicator data to the monitoring system;

[0014] S3: The monitoring system stores and analyzes the received indicator data;

[0015] S4: Display query results and analysis results through visualization tools;

[0016] S5: Predict the development trend of the indicator data based on the received indicator data. If the predicted indicator data exceeds the set indicator data threshold limit, issue an early warning.

[0017] Furthermore, the collected indicator data includes hardware, network, operating system, infrastructure components, common middleware and application microservice levels; for hardware level collection, standard interfaces provided by the operating system or special hardware monitoring tools are used for collection; for network level collection, network card interfaces or network monitoring tools and standard network commands are used for collection; for operating system level collection, system interfaces of the operating system are used for collection; for application microservice level collection, indicator interfaces provided by the microservice framework or standard feature interfaces of the development language are used for acquisition.

[0018] Furthermore, in the process of transmitting the indicator data to the monitoring system, the monitoring system adopts a pull mode to obtain the indicator data; the indicator collection agent assembles the indicator data into an indicator data structure according to a set data format.

[0019] Furthermore, the monitoring system adopts the pull mode to obtain indicator data as follows: the indicator collection agent assembles the indicator data into an indicator data structure according to the set data format; after configuring the service address and data frequency of the indicator collection agent for the indicator data to be pulled in the monitoring system, the indicator data structure is obtained from the corresponding indicator collection agent, and the indicator data structure is parsed according to the set data format to obtain the indicator data.

[0020] Furthermore, the process of transmitting indicator data to the monitoring system is implemented through the HTTP protocol.

[0021] Furthermore, the indicator data received by the monitoring system is stored in a time series database.

[0022] Furthermore, one or more of the following three methods are used for presentation:

[0023] (1) Use multiple dimensions for display, including time dimension, node dimension, geographical dimension and business field dimension;

[0024] (2) Display by aggregating multiple indicator data, using the sum, average, maximum, minimum, interpolation, function calculation, or custom function;

[0025] (3) Combine and display multiple indicator data.

[0026] Furthermore, the early warning methods include: SMS notification, email notification, phone notification, DingTalk notification, WeChat reminder and enterprise WeChat alarm.

[0027] Furthermore, the indicator data threshold limits include the threshold limits of a single indicator and the threshold limits of a combination expression of multiple indicators.

[0028] The present invention adopts the above technical solution, which can comprehensively monitor the indicators related to the entire microservice system and use efficient and reasonable methods to collect, transmit and store them. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] FIG1 is a schematic diagram of an application system based on a microservice architecture in an embodiment of the present invention.

[0030] FIG2 is a flow chart of a method according to an embodiment of the present invention.

[0031] FIG3 is a flow chart showing the early warning judgment in this embodiment. DETAILED DESCRIPTION

[0032] To further illustrate various embodiments, the present invention provides accompanying drawings. These drawings form part of the present disclosure and are primarily used to illustrate the embodiments and, in conjunction with the relevant description in the specification, to explain the operating principles of the embodiments. By referring to these drawings, those skilled in the art will be able to understand other possible implementations and the advantages of the present invention.

[0033] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0034] Microservices architecture is an architectural concept (see Figure 1). It aims to decouple the overall system by dividing services into modules and decomposing functionality into discrete services. Compared to traditional monolithic applications, decomposing an application into several or even dozens of functional microservices allows each microservice to be independently developed and iterated using different languages ​​and frameworks. Services communicate with each other using REST or RPC, providing faster development efficiency and more flexible service support. A microservices system refers to an application system developed based on the microservices architecture. Microservice components include the middleware used in the microservices architecture, such as configuration and registration centers, caches, databases, API gateways, message queues, circuit breakers, and rate limiters, as well as the microservices formed by the various functional modules.

[0035] An embodiment of the present invention provides a monitoring and early warning method based on a microservice architecture, as shown in FIG2 , the method comprising the following steps:

[0036] S1: The indicator collection agent collects indicator data in real time during the operation of the microservice system.

[0037] The collection of monitoring indicators is the first step and foundation for establishing a monitoring system. It is primarily responsible for collecting relevant indicator data at all levels during the operation of the microservice system. In this embodiment, the component that completes the monitoring indicator collection task is called the "Indicator Collection Agent", which is deployed at each monitoring target.

[0038] The indicator collection agent can collect indicators at different levels, such as hardware, network, operating system, infrastructure components, common middleware, and application microservices. Specifically, the following methods can be used to collect indicators at different levels:

[0039] (1) For the collection of hardware-level indicators, standard interfaces provided by the operating system or specialized hardware monitoring tools can be used for collection. The main indicators include but are not limited to basic hardware machine information, CPU usage, memory usage and usage rate, disk usage and usage rate, disk read and write speed, I / O frequency and time consumption, GPU health status and codec usage rate.

[0040] (2) For the collection of network-level indicators, network card interfaces or network monitoring tools and standard network commands provided by the operating system (such as netstat and top) can be used for collection. The main indicators include but are not limited to the network card data flow per unit time, network upload and download flow per second, socket connection information and details of different network protocols, packet sending and receiving rates, and packet loss and error rates of sending and receiving.

[0041] (3) For the collection of operating system level indicators, the system interface and command line tools of the operating system can be used for collection. The main indicators include but are not limited to the overall system load rate, system process data, file system read and write status and frequency, file descriptor open status and context switch count.

[0042] (4) Collection of indicators at the infrastructure component level. Infrastructure components generally have their own operational indicator exposure function, which is connected to the corresponding indicator exposure function through HTTP or TCP protocol to obtain the corresponding indicator data. Infrastructure components mainly include various service platforms, resource scheduling platforms, virtualization platforms, data storage components, etc. These platforms or components basically have standard monitoring indicators, which are integrated into this embodiment as one of the components of the indicator source.

[0043] (5) Regarding the collection of common middleware-level indicators, middleware generally has its own operational indicator exposure function. The corresponding indicator data is obtained by connecting to the corresponding indicator exposure function through HTTP or TCP protocol. A complete microservice architecture system usually includes common middleware such as load balancing, gateway, service registration and discovery, unified configuration center, cache, search, message queue, etc. Each middleware has its own operational monitoring and performance indicators. This embodiment integrates it as one of the components of the indicator source.

[0044] (6) For the collection of indicators at the application microservice level, the indicator interface provided by the microservice framework or the standard feature interface of the development language can be used to obtain them. This embodiment uses the JAVA development language as an example to illustrate the core indicators that microservices need to pay attention to. The main indicators include but are not limited to process startup time and running time, number of queries per second QPS, number of transactions per second TPS, interface response time RT, total number of requests per second, number of successful / failed requests per second, average time per request, number of processes, total number of threads, total number of threads in each state, file descriptor size, number of loaded classes, JVM memory allocation data, heap and non-heap memory usage ratio, heap memory allocation details, non-heap memory allocation details, number of garbage collections, frequency of recycling per unit time, distribution of garbage collection pause time, etc.

[0045] By collecting core monitoring indicators at each of the above levels, you can comprehensively monitor the operational status of the entire microservice system. Any abnormality in any level of indicators can manifest as a disruption in the microservice system at the upper application level. By comprehensively checking indicators, you can quickly locate abnormal data, analyze the root cause of the problem, and resolve faults and issues more quickly.

[0046] After the indicator collection agent obtains the indicator data of the monitoring target, in order to meet the unified and multi-dimensional storage and analysis requirements, it is necessary to design a set of standard indicator data label formats in the indicator collection component. In this embodiment, the indicator data label format includes:

[0047] Metric name: If the metric is an instantaneous value, simply name it "metric." If it is a calculation or aggregation type, end the metric name with the corresponding calculation or aggregation method, such as _total, _counter, or _avg. Explanations of the metric name can be annotated in the form of comments.

[0048] Indicator value: the collected indicator data.

[0049] Timestamp ts: The time when the indicator is reported, which serves as the cornerstone of time series data.

[0050] Service name: The service name must be globally unique. It is generally composed of the hardware category and model or the system name plus the module name. The final service name is mainly used to distinguish the source of the indicator.

[0051] Instance name: In order to improve the system's carrying capacity, a microservice architecture generally deploys multiple instances of a service. You can directly use the machine name or IP as the instance name.

[0052] Service Type (job): The service type primarily identifies the component to which the metric belongs. For example, all MySQL monitoring data is uniformly tagged with "job = mysql," and all Redis monitoring data is uniformly tagged with "job = redis." For business microservice modules, the service module name can be used as the tag.

[0053] Regional availability zone: The regional information of the indicator machine is placed in the label. If a problem occurs in a zone, it is very easy to see which specific zone has abnormal indicator data, and then quickly switch the flow.

[0054] Environment type env: The environment type is an optional tag used to identify the production environment or the test environment. If the monitoring system is deployed only in the production environment, this tag is optional. However, if the monitoring system is reused in multiple environments, using env to distinguish the environments is essential.

[0055] S2: The indicator collection agent transmits the indicator data to the monitoring system.

[0056] Monitoring indicator transmission is the second step in the monitoring system, primarily responsible for transmitting the various indicator data collected by the indicator collection agent. In this embodiment, the component that performs monitoring indicator transmission is referred to as the "indicator transmission component." Generally speaking, data acquisition can be performed using either a pull or push model. The pull model involves the monitoring system periodically acquiring indicator data from the indicator collection agent, while the push model involves the indicator collection agent or the monitored target itself proactively pushing the collected indicator data to the monitoring system. Prior art generally only uses the push model to push monitoring indicator data to the monitoring system. This approach imposes significant modification costs on the monitored target. If a unified data push format is required, custom development is generally required on the monitored target side, making it less suitable for rapid integration. Therefore, this embodiment primarily utilizes a pull model for data acquisition. The most significant advantage of the pull model is decoupling. After acquiring various data, the indicator collection agent assembles it into an indicator data structure using a standard, universal data format. The monitoring system simply needs to define the service address and data frequency for pulling the "indicator collection agent" to obtain all indicator data transmitted using the standard indicator data structure. The data can then be parsed according to the standard structure, eliminating the need for custom parsing methods due to varying data structures in different software or tools. At the same time, if the monitoring system itself goes down or the service is unstable, no request will be initiated to obtain data, and there will be no impact on the monitoring target end.

[0057] Furthermore, to ensure reliable transmission of indicator data, reliable transmission protocols and mechanisms must be employed, such as TCP, HTTP, and message queues. In this embodiment, HTTP is used. There's no need to maintain a heartbeat between the indicator collection agent and the monitoring system. Instead, the address of the indicator collection agent is automatically and dynamically discovered based on a configuration file-based watch mechanism. This decouples dependencies on the monitoring target and eliminates any additional pressure on it.

[0058] An example of obtaining monitoring data via HTTP is as follows:

[0059] Get http(s): / / ip:port / metric

[0060] {"metric":"xxxxxxx","ts":"2023-05-0101:02:03.999","value":123.4,

[0061] "service": "my-service", "instance": "my-service01", "job": "mysql", "zone": "beiji ng", "env": "prod"}.

[0062] S3: The monitoring system stores and analyzes the received indicator data.

[0063] Storage and analysis is the third step in the monitoring system, which is mainly responsible for storing and analyzing the indicator data transmitted by the "Indicator Transmission Component".

[0064] In this embodiment, the component that performs the indicator storage and analysis tasks is called the "storage and analysis component." Monitoring indicator data generally evolves over time, with the data body structure rarely changing. Each data item carries a strict timestamp. Based on this data characteristic, a time series database can be used for storage. A time series database is a database management system specifically designed for storing, managing, and processing time series data. Time series data refers to data arranged in chronological order. Its main characteristic is that the data changes over time, and data operations are essentially only inserts, with almost no updates or deletes. Popular time series databases include InfluxDB, OpenTSDB, and TDengine. The main advantage of time series databases over relational databases is that indicator data can be compressed by timestamp column during storage to reduce storage space and improve query efficiency.

[0065] Since the indicator data obtained by the indicator collection agent has a unified indicator data label format, the storage table structure of the time series database is also very unified, which is more convenient for compression processing and query efficiency improvement.

[0066] Time series libraries generally include data analysis capabilities. While storing indicator data, they can simultaneously analyze and store the analysis results. During analysis, they employ data analysis methods and algorithms, such as statistical and machine learning techniques, to analyze and mine indicator data. This analysis can extract valuable information and patterns, providing more comprehensive and multi-dimensional data support for subsequent query analysis, presentation, and trend warnings.

[0067] S4: Display query results and analysis results through visualization tools.

[0068] Displaying the queried data in the form of a chart is more vivid and intuitive, and can better observe the data change history and trend. In this embodiment, the component that completes the display task of the indicator query results and analysis results is called the "query analysis display component". If you need to manually view the data through the interface or API of the time series database every time, this is very inconvenient. It can neither reduce the workload of manual operation nor show the development trend of the data. Therefore, in this embodiment, the interface or SQL query of the time series database is connected to professional chart display tools, such as ECharts, Grafana, etc. The specific display forms include but are not limited to pie charts, histograms, curve charts, line charts, dot plots, dashboards, etc. Multi-dimensional combined data can also be compared and displayed. Based on the unified data format defined in the first step, it can be conveniently displayed from the time dimension, service dimension, instance dimension, regional dimension, and business service dimension. The display of the visualization tool can help users understand the current operating status and potential problems of the system.

[0069] Specifically, presentations can be made using multiple dimensions, multiple aggregations, and multiple indicator aggregations. Multi-dimensionality means that indicator data can be queried and analyzed from different perspectives and dimensions, such as time, node, region, and business domain. Multiple aggregations allow for aggregation operations on multiple indicator data, such as summation, average, maximum, minimum, interpolation, function calculations, and various custom function algorithms. Multi-indicator aggregation allows for the combined display of multiple indicator data to more comprehensively reflect the system's operational status and compare similar indicators.

[0070] S5: Predict the development trend of the indicator data based on the received indicator data. If the predicted indicator data exceeds the set indicator data threshold, issue an early warning.

[0071] The component that performs indicator trend warning tasks in this embodiment is called the "trend warning component." The monitoring system can monitor the system's real-time operating status. It can also predict potential or impending faults by monitoring data trends. This allows for proactive troubleshooting and resolution before actual failures occur, preventing online service interruptions that could impact system performance.

[0072] The following describes the early warning judgment process with reference to FIG3 .

[0073] In the trend warning component, you can configure warning rules for each single indicator as needed. You can also configure combined expression rules as needed, that is, the values ​​obtained by combining, calculating, and aggregating the instantaneous values ​​of multiple indicators for warning, so as to achieve more comprehensive system security redline control. Configuration rules are directly configured using configuration files. For example, the configuration is organized in yaml format for easy direct management and viewing. You can also configure various automated configuration management tools to automatically update configuration rules and make them effective in real time. Among them, the single indicator threshold limit refers to setting a threshold limit for a single indicator. For example, an alarm is triggered when the CPU usage exceeds 80%. The combined expression threshold limit refers to setting a threshold limit for multiple indicators. For example, an alarm is triggered when the CPU usage exceeds 80% and the memory usage exceeds 70%.

[0074] After receiving new indicator data, the trend warning component determines whether an alert rule (including single-indicator rules and combined expression rules) has been configured for that indicator. If a single-indicator rule exists, it directly calculates whether the rule should be triggered. A combined expression rule requires a comprehensive calculation based on the recent data of other indicators. It should be noted that the collection periods of multiple indicators in a combined expression may not coincide, so it is necessary to reasonably assess how many units of time can be considered as data at the same point in time for calculation, such as within 1 second. Once the alert rule is triggered, the pre-set alarm template is called to fill in the relevant parameters and issue an alert notification.

[0075] Early warning notifications can be sent to system personnel through physical alarms or to system administrators through various IM mechanisms, such as SMS, email, phone, DingTalk, WeChat, and WeChat for Business alerts. Upon receiving the notifications, system administrators can promptly review and analyze the warning indicators, identify potential issues, and take appropriate measures to address them, preventing them from escalating or even causing system crashes.

[0076] This embodiment establishes a comprehensive monitoring and early warning system for the core indicators of the entire microservice architecture system, including hardware, network, operating system, middleware, microservices, etc., to facilitate system maintenance personnel to have a more comprehensive grasp of the system's operating health status, eliminate system risks in advance, reduce the probability of system failures, and greatly improve the operational stability and operation and maintenance efficiency of the microservice system.

[0077] The embodiments of the present invention have the following beneficial effects:

[0078] 1) Comprehensively collect metrics across various layers of the microservice architecture, including hardware, network, operating system, infrastructure components, common middleware, and application microservices. The collected metrics are not limited to a specific piece of hardware, software, or service, but provide a comprehensive overview of the health of the entire system.

[0079] 2) The pull mode is mainly used to obtain data instead of the push mode of traditional technology, which completely decouples the monitoring target and the monitoring system, and prevents the service stability of the monitoring system itself from affecting the normal operation of the monitoring target service. The active on-demand data acquisition of the pull mode can greatly reduce the pressure on the monitoring target service itself compared to the periodic polling and push of the push mode.

[0080] 3) Visualizing monitoring data and issuing real-time warnings based on pre-set trend rules maximizes the value of monitoring data and improves the efficiency of system maintenance personnel. This even allows for unmanned operation, with early warnings and remote access for troubleshooting and processing, reducing labor costs.

[0081] Although the present invention has been particularly shown and described in conjunction with preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made to the present invention without departing from the spirit and scope of the invention as defined in the appended claims, and all such changes are within the scope of protection of the present invention.

Claims

1. A monitoring and early warning method based on a microservices architecture, characterized in that, It includes the following steps: S1: The metric collection agent collects metric data in real time during the operation of the microservice system; S2: The metric collection agent transmits the metric data to the monitoring system; S3: The monitoring system stores and analyzes the received metric data; S4: The query results and analysis results are displayed through visualization tools; S5: Predict the development trend of the metric data based on the received metric data. If the predicted metric data exceeds the set threshold limit of the metric data, an alarm is given.

2. The monitoring and warning method based on the microservice architecture according to claim 1, characterized in that: The collected metric data includes the hardware, network, operating system, infrastructure components, common middleware, and application microservice levels. For the collection at the hardware level, the standard interfaces provided by the operating system or specialized hardware monitoring tools are used for collection; for the collection at the network level, the network card interface or network monitoring tools and standard network commands are used for collection; for the collection at the operating system level, the system interfaces and command-line tools of the operating system are used for collection; For the collection at the application microservice level, the metric interfaces provided by the microservice framework or the standard feature interfaces of the development language are used to obtain.

3. The monitoring and early warning method based on the microservice architecture according to claim 1, characterized in that: The metric collection agent converts the collected metric data into a unified metric data label format. The fields included in the metric data label format are: metric name, metric value, timestamp, service name, instance name, service type, and regional availability zone.

4. The monitoring and early warning method based on the microservice architecture according to claim 1, wherein: During the process of transmitting the metric data to the monitoring system, the monitoring system uses the pull mode to obtain the metric data; the metric collection agent assembles the metric data into a metric data structure according to the set data format.

5. The monitoring and warning method based on the microservice architecture according to claim 4, characterized in that: The method for the monitoring system to obtain the metric data in the pull mode is: the metric collection agent assembles the metric data into a metric data structure according to the set data format; after configuring the service address and data frequency of the metric collection agent for the metric data to be pulled in the monitoring system, obtain the metric data structure from the corresponding metric collection agent, and parse the metric data structure according to the set data format to obtain the metric data.

6. The monitoring and warning method based on the microservice architecture according to claim 1, wherein: The process of transmitting the metric data to the monitoring system is implemented through the HTTP protocol.

7. The monitoring and warning method based on the microservice architecture according to claim 1, wherein: The metric data received by the monitoring system is stored using a time series database.

8. The monitoring and early warning method based on a microservices architecture according to claim 1, wherein: One or more of the following three methods are used for display: (1) Display using multiple dimensions, including time dimension, node dimension, regional dimension, and business domain dimension; (2) Display by aggregating multiple metric data. The aggregation methods are summation, average value, maximum value, minimum value, interpolation, function calculation, or custom function; (3) Display by combining multiple metric data.

9. The monitoring and early warning method based on the microservice architecture according to claim 1, characterized in that: The alarm methods include: SMS notification, email notification, phone notification, DingTalk notification, WeChat reminder, and enterprise WeChat alarm.

10. The monitoring and early warning method based on the microservice architecture according to claim 1, characterized in that: The threshold limits of the metric data include the threshold limits of a single metric and the threshold limits of a combined expression of multiple metrics.

Images