Micro-service cloud platform transverse network performance real-time monitoring operation and maintenance method and system

Abstract

The invention discloses a micro-service cloud platform transverse network performance real-time monitoring operation and maintenance method and system, and relates to the technical field of operation and maintenance, and the method comprises the steps: collecting flow information among a container Pod, a node Node and a host in a micro-service cloud platform, and constructing an application service topological graph, a node topological graph and a host topological graph; based on the collected data, the system carries out network flow index analysis and network fault alarm analysis to help to identify abnormal links and network load abnormalities; network flow information is acquired in real time through a plurality of acquisition components of a virtual machine and a container environment, and link faults are automatically identified and alarmed. According to the method, the accuracy of network fault diagnosis is improved, data support is provided for network optimization and resource allocation, the fault processing time is shortened, the service interruption risk caused by network problems is reduced, the fault response speed is improved, and stable operation and service continuity of a micro-service cloud platform are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of operation and maintenance technology, and specifically to a method and system for real-time monitoring and operation of horizontal network performance of a microservice cloud platform. Background Art

[0002] In today's era of rapid development of information technology, operation and maintenance technology, as a key supporting computing model, has profoundly changed the way enterprise IT infrastructure is built and operated, especially the microservice cloud platform. The container orchestration system based on Kubernetes has become a key infrastructure to support this modern architecture. The widespread application of microservices has promoted the transformation of service delivery models, but it has also brought new challenges to network performance monitoring and operation and maintenance. In the microservice cloud platform, the network traffic information between containers (Pod) and nodes (Node) has become an important indicator reflecting the health of the system. Real-time monitoring and analysis of this information is crucial to ensuring service quality and user experience.

[0003] However, the existing network performance monitoring technology still exposes many shortcomings when responding to the actual needs of the microservice cloud platform. Traditional monitoring tools are difficult to adapt to the dynamics and complexity of the microservice environment, resulting in the constructed topology map being often static and unable to be updated in real time, unable to accurately reflect the real-time status of the microservice, and the real-time and accuracy of the monitoring data are low. The existing monitoring technology lacks sufficient depth in network traffic data analysis, and the fault alarm system has obvious defects in accuracy and timeliness. Faced with the complex link relationships in the microservice cloud platform, the existing technology is difficult to effectively identify and alarm network faults, resulting in insufficient timeliness in fault location and processing, affecting the continuity of services and user experience. The monitoring system generally lacks an automated fault response mechanism and relies too much on manual intervention, which not only greatly increases the operation and maintenance costs, but also reduces the efficiency of fault handling. These technical bottlenecks seriously limit the service capabilities of the microservice cloud platform and the problem-solving capabilities of the operation and maintenance team, and cannot meet the urgent needs of the modern cloud computing environment for efficient and intelligent operation and maintenance. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing operation and maintenance technical methods have the problems that the existing microservice cloud platform network performance monitoring methods have low real-time topology map construction, insufficient traffic indicator analysis capabilities, poor real-time link fault identification and alarm mechanism, as well as how to achieve all-round, real-time, automated network performance monitoring and operation and maintenance.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform, comprising collecting traffic information between container Pods, nodes and hosts in the microservice cloud platform to build application service topology maps, node topology maps and host topology maps; based on the collected data, the system performs network traffic indicator analysis and network fault alarm analysis to help identify abnormal links and network load anomalies; through a variety of collection components in virtual machines and container environments, network traffic information is collected in real time, and link failures are automatically identified and alarmed.

[0007] As a preferred solution of the real-time monitoring and operation and maintenance method of the horizontal network performance of the microservice cloud platform described in the present invention, the construction of the application service topology diagram includes link traffic information such as the upstream and downstream rates, average response delay, throughput, traffic size, packet loss rate and packet error rate between container Pods, and identifies high-latency, high-load and abnormal service links through screening conditions.

[0008] As a preferred solution of the real-time monitoring and operation and maintenance method of the horizontal network performance of the microservice cloud platform described in the present invention, wherein: the construction of the node topology map includes displaying the calling relationship between different nodes based on the application service topology map and the deployment relationship between the nodes and the containers, analyzing the performance indicators of the node links through traffic data, and evaluating the latency and packet loss through virtual network observation.

[0009] As a preferred solution of the real-time monitoring and operation and maintenance method of the horizontal network performance of the microservice cloud platform described in the present invention, wherein: the construction of the host topology map includes displaying the traffic information between the hosts through the node topology and the relationship between the node and the host, helping to analyze the network packet loss and delay from the physical link level, discovering abnormal load links, and providing support data for resource allocation.

[0010] As a preferred solution of the real-time monitoring and operation and maintenance method of horizontal network performance of the microservice cloud platform described in the present invention, the network traffic indicator analysis includes traffic indicator ranking, distribution display and historical trend analysis, the traffic ranking is displayed in the form of bar charts and pie charts, and the trend analysis shows the traffic changes in the form of line charts and bar charts.

[0011] As a preferred solution of the real-time monitoring and operation and maintenance method of horizontal network performance of the microservice cloud platform described in the present invention, the identification of abnormal links and network load anomalies includes the system counting and displaying network fault alarm information through different alarm levels such as emergency, important, and general, drilling down layer by layer from the physical network to the virtual network, and helping to troubleshoot the fault points of network packet loss and delay.

[0012] As a preferred solution of the real-time monitoring and operation and maintenance method of the horizontal network performance of the microservice cloud platform described in the present invention, the automatic identification and alarm of link failures include the system running on the KVM host machine through the Scapy collection component to collect the network traffic of the virtual machine, running in the Kubernetes environment through the Cilium component to collect the network traffic of the Pod and the node, and performing traffic collection in the Calico and GlobalRouter network environments.

[0013] Another object of the present invention is to provide a microservice cloud platform horizontal network performance real-time monitoring operation and maintenance system, which can analyze network traffic indicators and network fault alarms based on the collected data, help identify abnormal links and network load anomalies, and solve the problems of insufficient network performance monitoring and untimely alarms in current operation and maintenance technologies.

[0014] As a preferred solution of the real-time monitoring and operation and maintenance system for horizontal network performance of the microservice cloud platform described in the present invention, it includes: a topology construction module, a traffic index analysis and alarm module, and a link monitoring and automatic alarm module; the topology construction module is used to build an application service topology map, a node topology map and a host topology map by collecting traffic information between container Pods, nodes Nodes and hosts in the microservice cloud platform; the traffic index analysis and alarm module is used to perform network traffic index analysis and network fault alarm analysis based on the collected data, so as to help identify abnormal links and network load anomalies; the link monitoring and automatic alarm module is used to collect network traffic information in real time through multiple collection components of virtual machines and container environments, and automatically identify and alarm link failures.

[0015] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a step of a method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform.

[0017] Beneficial effects of the invention: The method for real-time monitoring and operation of the horizontal network performance of a microservice cloud platform provided by the invention collects traffic information between container Pods, nodes and hosts in the microservice cloud platform, builds application service topology diagrams, node topology diagrams and host topology diagrams, and realizes comprehensive visualization of the network structure of the microservice cloud platform, provides a clear network architecture view for operation and maintenance personnel, improves the accuracy of network fault diagnosis, effectively reduces misoperations caused by opaque network structure, and provides data support for network optimization and resource allocation. Based on the collected data, the system performs network traffic indicator analysis and network fault alarm analysis to help identify abnormal links and network loads. The system detects abnormal loads, enables the system to conduct in-depth analysis of network performance, improves the early detection rate of network faults, helps the operation and maintenance team quickly locate problematic links through alarm analysis, thereby shortening the fault handling time and reducing the risk of service interruptions due to network problems. Through a variety of collection components in virtual machines and container environments, network traffic information is collected in real time, and link faults are automatically identified and alarmed. The system can respond quickly to link failures, and automatic alarms can quickly locate fault points, thereby improving the fault response speed and ensuring the stable operation of the microservice cloud platform and the continuity of services. The present invention achieves better results in terms of comprehensive network monitoring, fault response speed, and service stability assurance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 An overall flow chart of a method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform provided in the first embodiment of the present invention.

[0020] Figure 2 An overall flow chart of a microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance system provided for the third embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0022] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides a method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform, comprising:

[0023] S1: By collecting traffic information between container Pods, nodes, and hosts in the microservice cloud platform, we build application service topology maps, node topology maps, and host topology maps.

[0024] Furthermore, the application service topology diagram is constructed to include link traffic information such as the upstream and downstream rates, average response latency, throughput, traffic size, packet loss rate, and packet error rate between container Pods, and high-latency, high-load, and abnormal service links are identified through screening conditions.

[0025] It should be noted that building a node topology diagram includes displaying the calling relationship between different nodes based on the application service topology diagram and the deployment relationship between nodes and containers, analyzing the performance indicators of node links through traffic data, and evaluating latency and packet loss through virtual network observation.

[0026] It should also be noted that building a host topology diagram includes displaying traffic information between hosts through node topology and the relationship between nodes and hosts, helping to analyze network packet loss and latency from the physical link level, discovering abnormally loaded links, and providing supporting data for resource allocation.

[0027] It should also be noted that the traffic information between Pods is collected, and the logical relationship topology between microservices is constructed by analyzing the traffic data. The application service topology can intuitively display the mutual calling relationship between the microservices of the application, and at the same time display the link traffic information between Pods, including the upstream and downstream rates, average response delay, throughput, traffic size, packet loss rate, and packet error rate. The service links with high latency, high load, and abnormalities can be filtered out through conditions; through virtual network observation in the cloud environment, based on the application service topology and the deployment relationship between nodes and microservices (Pods), the node topology of the application can be automatically drawn. From the node topology, the calling relationship between the various load nodes of the application can be clearly seen. By analyzing the traffic data between nodes, evaluating the performance indicators of node links, and displaying indicator data at key locations such as the client, client container node, client host, client gateway host, client gateway, server gateway, server gateway host, server host, server container node, and server, the latency and packet loss on the end-to-end path of the virtual network can be located, and high-latency, high-load, and abnormal node links can be screened out through conditions; the host topology of the application is automatically drawn through the node topology and the relationship between the node and the host. The host traffic information can be observed from the host topology diagram, and the network traffic can be analyzed from the physical link level. The location of network packet loss and latency can be quickly analyzed and located, and links with high load and latency can be found, providing data support for the reasonable allocation of resources.

[0028] It should also be noted that by collecting traffic information between container Pods, nodes and hosts in the microservice cloud platform, and building application service topology maps, node topology maps and host topology maps, a comprehensive visualization of the network structure of the microservice cloud platform is achieved, providing operation and maintenance personnel with a clear network architecture view, improving the accuracy of network fault diagnosis, effectively reducing misoperations caused by opaque network structure, and providing data support for network optimization and resource allocation.

[0029] S2: Based on the collected data, the system performs network traffic indicator analysis and network fault alarm analysis to help identify abnormal links and network load anomalies.

[0030] Furthermore, network traffic indicator analysis includes traffic indicator ranking, distribution display and historical trend analysis. Traffic ranking is displayed in the form of bar charts and pie charts, and trend analysis shows traffic changes in the form of line charts and bar charts.

[0031] It should be noted that identifying abnormal links and network load anomalies includes the system counting and displaying network fault alarm information through different alarm levels such as urgent, important, and general, drilling down layer by layer from the physical network to the virtual network, and helping to identify the fault points of network packet loss and delay.

[0032] It should also be noted that physical links and virtual links are analyzed from different dimensions of application services (pod containers), nodes, and hosts, and normal, abnormal, and high-latency links are counted separately, and detailed link lists can be viewed by drilling down; a network link overview table is provided, and the performance information of each physical link and virtual link is intuitively displayed from the list, including network rate, latency, packet loss, throughput, and traffic indicators, which are sorted and displayed according to different indicators, and queried and displayed according to name, status, high latency, and high throughput conditions; network traffic indicator analysis is performed from different dimensions of application services (pod containers), nodes, and hosts. Network traffic indicator data ranking: traffic access relationships and indicator data are displayed in a ranking manner, including bar charts, pie charts, Top line charts, stacked Top line charts, tables, etc. Network traffic indicator data distribution display: traffic access relationships and indicator data are displayed in a distributed manner, including probability distribution charts and geographic location distribution charts. Network traffic indicator data trend analysis: traffic access relationships and indicator data are displayed in a historical trend manner, including line charts, stacked line charts, trend analysis bar charts, and trend analysis stacked bar charts.

[0033] It should also be noted that, based on the collected data, the system performs network traffic indicator analysis and network fault alarm analysis to help identify abnormal links and network load anomalies, enabling the system to conduct in-depth analysis of network performance, thereby improving the early detection rate of network faults, and helping the operation and maintenance team to quickly locate problem links through alarm analysis, thereby shortening fault handling time and reducing the risk of service interruptions due to network problems.

[0034] S3: Through various collection components in virtual machines and container environments, network traffic information is collected in real time, and link failures are automatically identified and alarmed.

[0035] Furthermore, automatic identification and alarming of link failures include the system running on the KVM host through the Scapy collection component to collect virtual machine network traffic, running in the Kubernetes environment through the Cilium component to collect Pod and node network traffic, and performing traffic collection in the Calico and GlobalRouter network environments.

[0036] It should be noted that automatic link construction: the link information collected by the system is identified and analyzed, automatically managed through the system's data processing platform, and relationship links are automatically generated; virtual machine traffic collection: the Scapy collection component is run on the virtual machine of the KVM host to collect the network traffic of the virtual machine; host machine collection: the Scapy collection component is run on the host host and cloud virtual machine to collect network traffic; K8s traffic collection: by installing and running the Cilium component on the k8s container, the traffic of the pod container and node running on K8s is collected; container network collection: runs in the Calico and GlobalRouter network environment.

[0037] It should also be noted that network fault alarm information is counted according to different alarm levels such as urgent, important, and general, and displayed in charts. At the same time, detailed alarm information can be viewed by drilling down, from the source physical network, physical server to the virtual network OVS, virtual machine, and container levels; physical links and virtual links are uniformly checked link by link, and the fault point is located from the packet loss and delay data of the physical network, virtual network, and virtual network element.

[0038] It should also be noted that through various collection components of virtual machines and container environments, network traffic information is collected in real time, and link failures are automatically identified and alarmed. The system can respond quickly to link failures, and automatic alarms can quickly locate the fault point, greatly improving the fault response speed and ensuring the stable operation of the microservice cloud platform and service continuity.

[0039] Example 2 is an embodiment of the present invention, which provides a real-time monitoring and operation method for horizontal network performance of a microservice cloud platform. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0040] First, using Scapy and Cilium collection components, the network traffic between Pods, nodes and hosts in the Kubernetes cluster is collected in real time. The data collection covers the uplink and downlink rates, average response delay, throughput, traffic size, packet loss rate and packet error rate between Pods; based on the collected data, the data processing platform is used to automatically build application service topology maps, node topology maps and host topology maps. These topology maps show in detail the call relationship between services, the interaction between nodes and the traffic information of the host; the collected data is analyzed, including traffic indicator ranking, distribution display and historical trend analysis, which help identify high latency, high load and abnormal links in the network; the system issues real-time alarms for network failures according to the preset alarm level, including emergency, important and general level failures; experimental results show that the present invention can effectively monitor the network status of the microservice cloud platform, timely discover and diagnose network failures, thereby improving the stability of the network and the continuity of services. Compared with the prior art, the present invention provides a more comprehensive and intuitive network view, which improves the efficiency and accuracy of network failure diagnosis.

[0041] Example 3, reference Figure 2 , is an embodiment of the present invention, which provides a microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance system, including a topology construction module, a traffic indicator analysis and alarm module, and a link monitoring and automatic alarm module.

[0042] The topology construction module is used to build application service topology diagrams, node topology diagrams and host topology diagrams by collecting traffic information between container Pods, nodes and hosts in the microservice cloud platform; the traffic indicator analysis and alarm module is used to analyze network traffic indicators and network fault alarm analysis based on the collected data to help identify abnormal links and network load anomalies; the link monitoring and automatic alarm module is used to collect network traffic information in real time through various collection components of virtual machines and container environments, and automatically identify and alarm link faults.

[0043] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0044] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0045] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0046] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0047] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform, characterized in that: include: By collecting traffic information between container Pods, nodes, and hosts in the microservice cloud platform, we build application service topology maps, node topology maps, and host topology maps. Based on the collected data, the system analyzes network traffic indicators and network fault alarms to help identify abnormal links and network load anomalies; Through various collection components of virtual machines and container environments, network traffic information is collected in real time, and link failures are automatically identified and alarmed.

2. The microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to claim 1, characterized in that: The construction of the application service topology diagram includes link traffic information such as the upstream and downstream rates, average response delay, throughput, traffic size, packet loss rate and packet error rate between container Pods, and identifies high-latency, high-load and abnormal service links through screening conditions.

3. The microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to claim 2, characterized in that: The construction of the node topology diagram includes displaying the calling relationship between different nodes based on the application service topology diagram and the deployment relationship between nodes and containers, analyzing the performance indicators of the node links through traffic data, and evaluating the delay and packet loss through virtual network observation.

4. The microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to claim 3, characterized in that: The host topology diagram is constructed by displaying traffic information between hosts through node topology and the relationship between nodes and hosts, helping to analyze network packet loss and latency from the physical link level, discovering abnormal load links, and providing support data for resource allocation.

5. The microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to claim 4, characterized in that: The network traffic index analysis includes traffic index ranking, distribution display and historical trend analysis. The traffic ranking is displayed in the form of bar charts and pie charts, and the trend analysis shows the traffic changes in the form of line charts and bar charts.

6. The microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to claim 5, characterized in that: The identification of abnormal links and network load anomalies includes the system counting and displaying network fault alarm information through different alarm levels such as urgent, important, and general, drilling down layer by layer from the physical network to the virtual network, and helping to identify the fault points of network packet loss and delay.

7. The microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to claim 6, characterized in that: The automatic identification and alarm of link failures include the system running on the KVM host through the Scapy collection component to collect virtual machine network traffic, running in the Kubernetes environment through the Cilium component to collect Pod and node network traffic, and performing traffic collection in the Calico and GlobalRouter network environments.

8. A system using the microservice cloud platform horizontal network performance real-time monitoring and operation and maintenance method according to any one of claims 1 to 7, characterized in that: Including topology building module, traffic index analysis and alarm module, link monitoring and automatic alarm module; The topology building module is used to build an application service topology map, a node topology map, and a host topology map by collecting traffic information between container Pods, nodes, and hosts in the microservice cloud platform; The flow index analysis and alarm module is used to analyze the network flow index based on the collected data, and to perform network fault alarm analysis to help identify abnormal links and network load anomalies; The link monitoring and automatic alarm module is used to collect network traffic information in real time through various collection components of virtual machines and container environments, and automatically identify and alarm link failures.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for real-time monitoring and operation of horizontal network performance of a microservice cloud platform described in any one of claims 1 to 7 are implemented.

Images