Application running situation awareness method and system based on micro-service platform

Abstract

The invention discloses an application running situation awareness method and system based on a micro-service platform, and relates to the technical field of data service value evaluation and decision optimization, and the method comprises the steps: collecting data, and carrying out the first tracking; carrying out first analysis and fault positioning; and carrying out visual monitoring and intelligent alarm based on an automatic operation and maintenance strategy. According to the application running situation awareness method based on the micro-service platform, efficient operation and maintenance of the system can be ensured from distributed data collection to link tracking, potential problems can be found in time, and faults can be rapidly positioned and repaired. Through accurate data acquisition and link tracking, a high-quality data basis is provided for subsequent performance analysis and fault positioning. And multi-dimensional performance monitoring is combined with an intelligent fault detection technology, so that the system problem identification efficiency is greatly improved, and the operation and maintenance difficulty is reduced. The method has better effects in the aspects of identification efficiency, efficiency and fault repair accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of data service value assessment and decision optimization, and specifically to an application operation situation awareness method and system based on a microservice platform. Background Art

[0002] With the rapid development of big data technology and cloud computing technology, enterprises and organizations are making more and more in-depth use of data. As an important support for the digital transformation of enterprises, data services have gradually become the core business module of various platforms. Current data service platforms usually rely on distributed computing and microservice architecture, which makes real-time data collection, processing and management more efficient. However, with the increase in data volume and system complexity, how to scientifically and effectively evaluate the value of data services and make optimization decisions has become a key direction for technological development. In order to meet this challenge, traditional data analysis methods have gradually transformed into intelligent ones, especially the application of neural network models such as multilayer perceptrons, which provides new possibilities for the value evaluation of data services. With its nonlinear mapping ability and ability to characterize complex relationships, multilayer perceptrons have gradually become an important tool for data analysis and optimization decision-making.

[0003] Although the existing technology has provided a variety of methods for performance monitoring and decision optimization of data services, there are still many shortcomings when facing complex distributed data service systems. First, traditional data collection and monitoring methods often only monitor a single performance indicator, and cannot fully cover multi-dimensional data such as CPU, memory, network traffic, logs, etc., resulting in a lack of sufficient information support in performance bottleneck analysis. Secondly, most of the existing link tracking methods rely on manual settings and static analysis, which is difficult to dynamically adapt to complex microservice call links, making fault location and bottleneck analysis inefficient. In addition, the existing decision optimization methods usually rely on fixed rules and preset parameters, and cannot make full use of advanced machine learning algorithms such as multi-layer perceptrons to intelligently analyze and predict data, limiting the system's adaptive optimization capabilities in a dynamic environment. Therefore, how to build a data service system with multi-dimensional performance monitoring, intelligent fault detection and decision optimization has become a problem that needs to be solved urgently in the existing technology. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing data service monitoring and optimization methods are unable to comprehensively collect multi-dimensional data, have low fault location efficiency, and lack intelligent decision-making optimization, as well as the problem of how to achieve efficient and intelligent data service value assessment and optimization decision-making.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for sensing application operation status based on a microservice platform, comprising collecting data for first tracking; performing first analysis and fault location; and performing visual monitoring and intelligent alarm based on automated operation and maintenance strategies.

[0007] As a preferred solution of the application operation situation awareness method based on the microservice platform described in the present invention, wherein: the data collection for the first tracking includes deploying a data collector and collecting data through the collector.

[0008] As a preferred solution of the application operation situation awareness method based on the microservice platform described in the present invention, wherein: the first tracking of the collected data includes tracking and recording the complete path of each service request flowing through the system.

[0009] As a preferred solution of the application operation situation awareness method based on the microservice platform described in the present invention, the first analysis and fault location include entering data collection and first tracking completion, and the first analysis of system performance.

[0010] As a preferred solution of the application operation situation awareness method based on the microservice platform described in the present invention, wherein: the first analysis and fault location include real-time fault detection by analyzing the data after the first analysis is completed.

[0011] As a preferred solution of the application operation situation awareness method based on the microservice platform described in the present invention, wherein: the visual monitoring and intelligent alarm based on the automated operation and maintenance strategy includes introducing the automated operation and maintenance strategy;

[0012] When the system detects performance issues, predefined recovery actions are automatically executed.

[0013] As a preferred solution of the application operation situation awareness method based on the microservice platform of the present invention, wherein: the visual monitoring and intelligent alarm based on the automated operation and maintenance strategy includes viewing the health status of the system in real time through the dashboard, and deeply analyzing the operation data of the service through click operations;

[0014] Set alarm rules. When performance indicators exceed preset thresholds, alarm notifications are automatically triggered to inform operation and maintenance personnel of the problem.

[0015] Another object of the present invention is to provide an application operation situation awareness system based on a microservice platform, which can solve the problem of low fault location efficiency in current data service monitoring and optimization methods through performance analysis and fault location.

[0016] As a preferred solution of the application operation situation awareness system based on the microservice platform described in the present invention, it includes: a data collection and first tracking module, a first analysis and fault location module, and an automated operation and maintenance and visualization management module; the data collection and first tracking module is used to collect data to achieve full life cycle management of each request; the first analysis and fault location module is used to predict performance problems; the automated operation and maintenance and visualization management module is used to respond to and handle problems.

[0017] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a step of an application operation situation awareness method based on a microservice platform.

[0018] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for realizing application operation situation awareness based on a microservice platform.

[0019] Beneficial effects of the present invention: The application operation situation awareness method based on the microservice platform provided by the present invention realizes all-round intelligent monitoring and optimization of the microservice system from distributed data collection to link tracking, and then to multi-dimensional performance analysis and intelligent fault detection. These steps work together to not only ensure the efficient operation and maintenance of the system, but also to timely discover potential problems, quickly locate and repair faults, and realize an operation and maintenance management platform with high automation and intelligence. Through accurate data collection and link tracking, a high-quality data foundation is provided for subsequent performance analysis and fault location. At the same time, multi-dimensional performance monitoring combined with intelligent fault detection technology greatly improves the efficiency of system problem identification and reduces the difficulty of operation and maintenance. This innovative method not only improves the stability of the system, but also greatly reduces human intervention, ensuring that applications under the distributed microservice architecture can run stably and efficiently in a complex and changeable environment. The present invention achieves better results in terms of recognition efficiency, efficiency and fault repair accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0021] Figure 1 An overall flow chart of an application operation situation awareness method based on a microservice platform provided for the first embodiment of the present invention.

[0022] Figure 2An overall flow chart of an application operation situation awareness system based on a microservice platform provided for the third embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0024] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides an application operation situation awareness method based on a microservice platform, comprising:

[0025] S1: Collect data for the first tracking.

[0026] Going further, distributed data collection, first of all, in the microservice platform, multiple data collectors need to be deployed to collect runtime data from different service nodes. These collectors can be deployed in different infrastructure environments such as Kubernetes Pods, virtual machines, physical servers, etc. The content of data collection includes:

[0027] Collect resource utilization of each service, such as CPU, memory, and disk I / O;

[0028] Use eBPF (Extended Berkeley Packet Filter) technology to collect traffic information, including uplink and downlink rates, latency, and packet loss rates between services;

[0029] Collect log information generated during the operation of microservices for subsequent analysis.

[0030] This distributed data collector runs in the background in a non-intrusive manner without affecting system performance. Through eBPF technology, efficient data collection can be achieved at the kernel level while ensuring the real-time performance of the system and the accuracy of the data.

[0031] It should be noted that the first tracking can be of various types, such as link tracking technology, log tracking based on event-driven architecture, etc. In this embodiment, the link tracking technology is used to track and record the complete path of each service request flowing through the system. The specific steps are as follows:

[0032] Generation and propagation of Trace ID and Span ID: When the system receives a user request, it generates a unique Trace ID and generates a Span ID for each sub-request during the request propagation process. These IDs will be associated with the service call link of the request to form a complete tracing path;

[0033] Recording and storage of link data: When a request flows between various service nodes, the system will record key information such as the processing time, response time, status code, etc. of each service to form a complete call link. Through the complete tracking of the call link, the running path of each request can be clearly displayed, helping operation and maintenance personnel to analyze the interactive dependencies between services and quickly locate anomalies or performance bottlenecks.

[0034] This link tracking method based on interceptor or JavaAgent technology realizes the full life cycle management of each request, ensuring real-time monitoring and troubleshooting of complex call relationships in the microservice system.

[0035] In another feasible embodiment, the first tracking adopts methods such as log tracking based on event-driven architecture, including: In some scenarios, call link tracking can be achieved through **Event-Driven Architecture (EDA)**. In an event-driven system, services communicate asynchronously through message queues (such as Kafka, RabbitMQ), and the event stream of each service can be recorded in the log system. By analyzing and correlating these logs, cross-service call link tracking can be achieved. Suitable for microservice tracking in asynchronous communication scenarios, especially event-driven systems. Able to monitor and track the flow path of messages with low latency, combined with log analysis tools (such as ELK stack) for detailed analysis. Relatively flexible, easy to capture complex asynchronous call relationships.

[0036] S2: Perform the first analysis and fault location.

[0037] Furthermore, the first analysis can be of various types, such as multi-dimensional performance analysis, a unified platform for distributed tracking and indicator collection, and other methods. In this embodiment, after multi-dimensional performance analysis, data collection and link tracking are completed, the multi-dimensional analysis stage of system performance is entered. The present invention monitors and analyzes the following key performance indicators in real time:

[0038] Network performance analysis: By analyzing the collected network traffic data, the system can monitor indicators such as the uplink and downlink rates, latency, throughput, and packet loss rate between various services, and identify potential bottlenecks or anomalies in the network. With the help of eBPF technology, the system can monitor and analyze network traffic in real time with minimal overhead;

[0039] Service performance analysis: The system tracks each service’s response time, throughput, error rate, and other metrics. With this data, the system can determine the health of each service and identify which services may have performance issues, such as services with long response times or abnormally high error rates.

[0040] In addition, the multi-dimensional analysis of service performance includes:

[0041] CPU, memory and other resource utilization analysis: By analyzing system resource utilization, resource bottlenecks can be discovered in a timely manner to prevent performance degradation.

[0042] Request analysis: Analyze the request volume, request latency, success rate and other dimensions between services to identify high-load or inefficient service nodes.

[0043] It should be noted that in another feasible embodiment, the first analysis adopts methods such as a unified platform for distributed tracing and indicator collection, including: through a unified observability platform such as OpenTelemetry, link tracing, logging and indicator collection can be performed simultaneously. Unlike traditional multi-dimensional analysis, OpenTelemetry integrates multiple data sources, which can not only monitor the performance of services, but also unify link tracing, logs and indicators into a unified framework for analysis. This approach allows for more comprehensive system observation and provides a unified view of system behavior.

[0044] Beneficial effects:

[0045] The problem of fragmentation of monitoring data in different dimensions in traditional analysis methods is eliminated, and the unified collection framework can make the relationship between data sources clearer.

[0046] Provides standardized interfaces and is easy to integrate with a variety of tools (such as Prometheus, Jaeger, and Zipkin).

[0047] Suitable for complex distributed systems that require multi-level monitoring and real-time data correlation analysis.

[0048] Furthermore, after the performance analysis is completed, the system analyzes the data through intelligent algorithms and detects faults or abnormalities in real time. The specific steps include:

[0049] Fault detection and anomaly identification: By introducing machine learning algorithms, the system can automatically detect anomalies in the system based on the differences between historical data and real-time data. The anomaly detection model learns the normal operating mode of the system and issues alarms for data fluctuations, abnormal peaks, etc. For example, when the response time of a service suddenly increases or the error rate reaches a certain threshold, the system can automatically identify it as a fault and generate an alarm;

[0050] Intelligent location of fault source: Combining the results of link tracking and log analysis, the system can quickly locate the specific location where the fault occurred. By analyzing the complete service call link, the system can identify which node or service has an abnormality at a specific time point, helping operation and maintenance personnel to quickly determine the source of the fault and reduce troubleshooting time.

[0051] S3: Visual monitoring and intelligent alarm based on automated operation and maintenance strategies.

[0052] Furthermore, in order to improve the stability and operation efficiency of the system, the present invention introduces an automated operation strategy. When the system detects an abnormality or performance problem, it can automatically perform predefined recovery operations, including:

[0053] Automatic fault recovery: When a service fails or performance degrades, the system can automatically take action, such as restarting the failed service, expanding service instances, adjusting load balancing policies, etc. This automated recovery operation ensures that the system can quickly recover to normal when a failure occurs, reducing system downtime;

[0054] Automated configuration management: The system can be combined with the CI / CD (continuous integration / continuous delivery) pipeline to achieve automated deployment and configuration management of applications. When a new version of a service is launched, the system can automatically update and replace the service instance and ensure that the normal operation of other services is not affected during operation.

[0055] Through these automated operation and maintenance methods, the system can efficiently solve operation and maintenance problems without relying on human intervention, improving the availability and reliability of the system.

[0056] It should be noted that the present invention provides a powerful visual management function to help operation and maintenance personnel intuitively and comprehensively understand the operating status of the system. The visualization part is achieved by the following means:

[0057] Real-time monitoring dashboard: The system integrates visualization tools such as Grafana and Kibana to display monitored service performance, link calls, network status and other data in the form of charts. Operation and maintenance personnel can view the health status of the system in real time through the dashboard, and deeply analyze the operation data of a specific service through click operations;

[0058] Intelligent alarm system: The system supports setting alarm rules. When certain key performance indicators (such as CPU usage, response time, error rate, etc.) exceed the preset threshold, the system will automatically trigger an alarm notification to inform the operation and maintenance personnel of the problem. The alarm notification can be sent via email, SMS or corporate WeChat, and contains detailed fault information and recommended treatment measures.

[0059] Example 2, an embodiment of the present invention, provides an application operation situation awareness method based on a microservice platform. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0060] First, this experiment aims to verify the application effect of a data service value assessment and decision optimization method based on multi-layer perceptron in the microservice platform, especially through distributed data collection, link tracking, performance analysis and fault location, automated operation and maintenance strategy and other steps to optimize and improve the system. This experiment was conducted on a microservice platform with multiple service nodes, involving six key service nodes, named service A, service B, service C, service D, service E and service F.

[0061] In the first phase, distributed data collection technology was used to collect the operating status and performance data of each service, involving key indicators such as CPU utilization, memory usage, disk I / O performance, network latency, request throughput, and error rate. The data collector was deployed in Kubernetes Pods and obtained network traffic data through eBPF technology to ensure efficient collection without affecting system performance.

[0062] Then, through link tracking technology, the flow path of each user request is recorded in detail. The system generates a Trace ID and Span ID for each request to ensure that the flow process of the request between each node can be fully recorded. The response time and processing status of each node are clearly displayed through link tracking to help analyze system bottlenecks and abnormal points.

[0063] In the second stage, the data is passed to the multi-dimensional performance analysis platform for processing. Network performance analysis helps the system identify communication bottlenecks between services, while CPU and memory utilization analysis provides warnings of resource bottlenecks. At the same time, by comparing and analyzing the response time and error rate of requests, potential high-load service nodes can be identified.

[0064] Finally, under the automated operation and maintenance strategy, when the system detects that some services have a long response time or a high error rate, it automatically triggers recovery operations, such as expanding service instances or redistributing loads. Through the intelligent alarm system, operation and maintenance personnel can obtain the system operation status in real time and take quick measures based on the alarm prompts.

[0065]

[0066] By analyzing the table data, we can draw the following conclusions:

[0067] First, the CPU and memory usage show that the CPU usage (85%) and memory usage (2048MB) of service D are significantly higher than other services, indicating that the service occupies a large amount of resources in the system. The associated throughput (130requests / sec) is low, while the error rate (2.5%) is relatively high, indicating that service D may be one of the bottlenecks of the system. The high resource usage and inefficient response of the service indicate that its performance may be limited by internal resource configuration and cannot handle its assigned load. Therefore, the automated operation and maintenance strategy should be optimized for service D, such as increasing its processing capacity by increasing service instances or redistributing loads.

[0068] On the other hand, service A has a relatively balanced performance, with a CPU usage of 45%, a memory usage of 512MB, a network latency of only 10ms, a throughput of 200 requests / sec, and an error rate of only 0.5%. This shows that service A has good performance when handling the current load and can be used as a reference node with better performance.

[0069] By comparing the network latency and throughput of service B and service E, we found that the network latency of service B is as high as 30ms and the throughput is low (150 requests / sec), while service E, although with high memory usage, has a network latency of only 25ms and a throughput of 160 requests / sec. This indicates that service B may have a network performance bottleneck, and it is recommended to adjust its network configuration to reduce latency and improve throughput.

[0070] Comprehensive analysis of the data in the table shows that the method of the present invention can effectively collect and analyze multi-dimensional data of service performance, helping to quickly identify bottleneck nodes in the system. Compared with traditional manual performance monitoring, the present invention greatly improves the efficiency of locating performance problems through distributed data collection and intelligent link tracking. The introduction of automated operation and maintenance strategies can respond quickly when abnormalities occur in high-load services, reducing system downtime and the risk of performance degradation. These advantages are fully reflected in the experiment, demonstrating the innovation and practicality of the present invention in a complex microservice environment.

[0071] Example 3, reference Figure 2 , is an embodiment of the present invention, which provides an application operation situation awareness system based on a microservice platform, including a data collection and first tracking module, a first analysis and fault location module, and an automated operation and maintenance and visualization management module.

[0072] The data collection and first tracking module is used to collect data to achieve full life cycle management of each request. The first analysis and fault location module is used to predict performance issues. The automated operation and maintenance and visual management module is used to respond to and handle problems.

[0073] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0074] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0075] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0076] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0077] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for sensing application operation status based on a microservice platform, characterized in that: include: Collect data for first tracking; Conduct the first analysis and fault location; Perform visual monitoring and intelligent alarms based on automated operation and maintenance strategies.

2. The method for sensing application operation status based on a microservice platform according to claim 1, characterized in that: The collecting of data for the first tracking includes deploying a data collector and collecting data through the collector.

3. The method for sensing application operation status based on a microservice platform according to claim 2, characterized in that: The collecting data for the first tracking includes tracking and recording the complete path of each service request flowing through the system.

4. The method for sensing application operation status based on a microservice platform according to claim 3, characterized in that: The first analysis and fault location include entering data collection and first tracking completion, and a first analysis of system performance.

5. The method for sensing application operation status based on a microservice platform according to claim 4, characterized in that: The performing of the first analysis and fault location includes, after the first analysis is completed, analyzing the data to detect the fault in real time.

6. The method for sensing application operation status based on a microservice platform according to claim 5, characterized in that: The visual monitoring and intelligent alarm based on the automated operation and maintenance strategy includes introducing the automated operation and maintenance strategy; When the system detects performance issues, predefined recovery actions are automatically executed.

7. The method for sensing application operation status based on a microservice platform according to claim 6, characterized in that: The visual monitoring and intelligent alarm based on the automated operation and maintenance strategy includes real-time viewing of the system health status through the dashboard and in-depth analysis of the service operation data through click operations; Set alarm rules. When performance indicators exceed preset thresholds, alarm notifications are automatically triggered to inform operation and maintenance personnel of the problem.

8. A system using the method for sensing application operation status based on a microservice platform as claimed in any one of claims 1 to 7, characterized in that: It includes data collection and first tracking module, first analysis and fault location module, and automated operation and visual management module; The data collection and first tracking module is used to collect data and realize the full life cycle management of each request; The first analysis and fault location module is used to predict performance problems; The automated operation and maintenance and visual management module is used to respond to and handle problems.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the application operation situation awareness method based on the microservice platform described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the application operation situation awareness method based on a microservice platform described in any one of claims 1 to 7 are implemented.

Images