Service end-to-end monitoring method and system based on eBPF

Abstract

The invention belongs to the technical field of computer network monitoring, and particularly relates to a service end-to-end monitoring method and system based on eBPF. Comprising a data collection module, a service topology snapshot generation module, a service template generation module and a service index calculation module, flow message data are collected and called from multiple applications and a K8S cluster through an eBPFAgent, and the collected data are transmitted to the service topology snapshot generation module; the service topology snapshot generation module receives data collected by the eBPFAgent, and obtains data from the eSpan; the service call chain template marking module manually marks nodes and paths of a panoramic topology snapshot generated by the service topology snapshot generation module; a service topology association and index calculation module obtains a panoramic topology snapshot generated by a service topology snapshot generation module, call chain data and a service template generated by a service call chain template marking module; the invention aims to provide an accurate, efficient and comprehensive end-to-end monitoring solution for an enterprise-level complex business system.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer network monitoring, and in particular relates to an eBPF-based service end-to-end monitoring method and system. Background Art

[0002] In today's information technology environment, effective monitoring of services and applications is crucial to ensuring normal system operation, optimizing performance, and quickly locating faults.

[0003] At present, application monitoring mainly uses APM (Application Performance Management) technology for data collection. Through APM technology, various types of data during the application operation can be obtained and these data can be connected into a call chain. However, in some monitoring scenarios based on eBPF technology, there are certain limitations. For example, when eBPF technology performs data association, due to the lack of Trace_id, it is necessary to combine multiple data such as Tcp_seq, IP, Port, etc. for association operations, and then calculate application performance indicators based on these associated data. This process increases the complexity and uncertainty of data processing.

[0004] Traditional end-to-end business monitoring methods usually rely on manual operations. Specifically, staff need to sort out the indicators of the business and its various links, draw the business topology on this basis, and mount the indicator data to the corresponding nodes in the topology to show the end-to-end relationship and node status of the business.

[0005] Disadvantages of existing monitoring technology 1. Separation of application and business monitoring: Existing monitoring methods such as APM and eBPF mainly focus on observing the call topology between applications, and cannot effectively combine the topology with the actual business. In this case, when there is a problem with the call chain on the application topology, it is difficult to directly locate which specific businesses these problems will affect. Similarly, when a problem occurs in the business, it is also difficult to quickly locate the call chain associated with it, which greatly increases the difficulty of troubleshooting and problem solving.

[0006] 2. Labor costs and errors in end-to-end business monitoring: For large enterprises such as operators and finance, their business scale is huge, the number of applications is large, and the business topology is complex. In this case, the traditional method of manually drawing business topology has many disadvantages. First, manually combing the business topology requires a lot of manpower and time costs. Secondly, due to the complexity of the business, omissions and errors are prone to occur during the manual combing process, resulting in low accuracy of the business topology, which in turn affects the effectiveness and reliability of end-to-end business monitoring.

[0007] Existing application monitoring methods are limited to observing the call topology between applications and cannot be closely integrated with actual business. When an abnormality occurs in the application call chain, it is difficult to determine the impact on the specific business; conversely, when a problem occurs in the business, it is impossible to quickly trace the relevant call chain, which seriously restricts the ability of enterprises to quickly locate and solve problems, affecting the normal operation of the business and customer experience. An innovative monitoring method is urgently needed to break this dilemma. Summary of the invention

[0008] In view of the shortcomings of the current related existing technologies, the present invention provides a business end-to-end monitoring method based on eBPF; the present invention is committed to utilizing the powerful kernel data collection and processing capabilities of eBPF, combined with innovative business topology construction and association mechanisms, to provide accurate, efficient and comprehensive end-to-end monitoring solutions for enterprise-level complex business systems, so as to ensure the smooth operation of business processes, optimization and improvement of performance, and rapid location and repair of faults, while ensuring business continuity and stability, helping enterprises to reduce operating costs and improve user experience, and is mainly used in IT infrastructure and business system monitoring in industries such as telecommunications and finance that have extremely high requirements for business reliability and performance.

[0009] To achieve the above object, the present invention provides a first aspect of providing an eBPF-based service end-to-end monitoring method, which at least includes the following steps: S1 Data Collection Steps Collect call flow message data from multiple applications and K8S clusters through eBPFAgent, and pass the collected data to the business topology snapshot generation module; S2 business topology snapshot generation steps The service topology snapshot generation module receives the data collected by eBPFAgent and obtains data from eSpan, processes the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port, obtains the call relationship between services by associating Span with TCP_Seq, IP, and Port, and stores it in the eBPF call list; Regularly merge and filter the call relationships between services to generate a panoramic topology snapshot, and generate call chain data to pass to the business topology association and indicator calculation module; S3 business template generation steps The business call chain template labeling module manually labels the nodes and paths of the panoramic topology snapshot generated by the business topology snapshot generation module; Set a business name for the annotation result, save it as a business template and pass it to the business topology association and indicator calculation module; S4 Business Indicator Calculation Steps The business topology association and indicator calculation module obtains the panoramic topology snapshot generated by the business topology snapshot generation module, the call chain data, and the business template generated by the business call chain template marking module; Match the data in the eBPF call list according to the service template and save the matched data to the service list; Calculate the business call duration and call success rate business indicators based on the business links and start and end times.

[0010] Furthermore, in the S1 data collection step, the eBPFAgent has a data integrity verification function to ensure that the collected call flow message data is complete and correct; the call flow message data is collected from the application nodes through eBPFAgent, and the application nodes include application NodeA, application NodeB, application NodeC and nodes in the K8S cluster, and the collected call flow messages contain TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port information.

[0011] Furthermore, in the business topology snapshot generation step, when the calling relationship between services is obtained by associating Span through TCP_Seq, IP, and Port, a fault-tolerant mechanism is provided to handle situations where data is missing or erroneous.

[0012] Furthermore, in the service template generation step, when the nodes and paths of the panoramic topology snapshot are manually annotated, the annotation operation can be traced back, and the operator, time and modification content of each annotation are recorded.

[0013] Furthermore, in the business indicator calculation step, the business topology association and indicator calculation module may generate a business indicator trend report when calculating the business indicator, for analyzing the business performance change trend.

[0014] The second aspect of the present invention provides a service end-to-end monitoring system based on eBPF, including a data acquisition module, a service topology snapshot generation module, a service template generation module and a service indicator calculation module, wherein The data collection module includes eBPFAgent, which is used to collect call flow message data from application NodeA, application NodeB, application NodeC and K8S cluster, and pass the data to the service topology snapshot generation module; The service topology snapshot generation module is used to receive data collected by eBPFAgent and data obtained from eSpan, process the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port, obtain the call relationship between services through TCP_Seq, IP, and Port association Span and store it in the eBPF call chain table, regularly merge and filter the call relationship between services to generate a panoramic topology snapshot and call chain data, and pass the call chain data to the service topology association and indicator calculation module; The service template generation module includes a service call chain template marking module, which is used to manually mark the nodes and paths of the panoramic topology snapshot, set the service name and save it as a service template to pass to the service topology association and indicator calculation module; The business indicator calculation module is used to obtain panoramic topology snapshots, call chain data and business templates, match the data of the eBPF call list according to the business template, save the matched data to the business list, and calculate the business call duration and call success rate business indicators according to the business links and start and end times.

[0015] Furthermore, the service topology snapshot generation module also includes a data preprocessing submodule for cleaning and preprocessing the collected data to remove noise data.

[0016] Furthermore, the business template generation module also includes a template version management submodule for managing different versions of business templates to facilitate backtracking and comparison.

[0017] Furthermore, the business indicator calculation module also includes an indicator abnormality analysis submodule, which performs a detailed analysis of the abnormal situation when the calculated business indicator exceeds a preset threshold.

[0018] Furthermore, it also includes a visualization display module, which is used to display the panoramic topology snapshot, business templates, business indicators and data in the business link list to the user in an intuitive graphical interface, and supports user interactive operations; including zooming, filtering, and drilling functions. This module can obtain the panoramic topology snapshot from the business topology snapshot generation module, obtain the business template from the business template generation module, and obtain the business indicators and business link list data from the business indicator calculation module.

[0019] The present invention adopts the above technical solution, which has at least the following beneficial effects: 1. Accurate integration of business and application monitoring; the present invention successfully breaks down the barriers between application monitoring data and business monitoring, and deeply integrates the information of the two. Traditional monitoring methods can often only perform application monitoring or business monitoring in isolation, while the present invention is based on the powerful data collection capabilities of eBPF to obtain rich network call flow information, and through a unique business topology construction combined with manual labeling, the call relationship at the application level is closely associated with the actual business process. For example, by processing the call flow message containing information such as TCP_Seq, Src_IP, Src_Port, Dest_IP, Dest_Port, etc., a panoramic topology snapshot and business template are generated, thereby achieving accurate mapping from the application call chain to the business chain. This enables enterprises to clearly understand the specific impact of application failures on the business during the monitoring process, as well as the application links involved behind the business problems, greatly improving the depth and breadth of monitoring, and providing enterprises with a more comprehensive and accurate insight into the business operation status.

[0020] 2. Significantly reduce labor costs and improve accuracy; in terms of business topology construction, the present invention effectively overcomes many drawbacks of traditional manual combing of business topology. In the past, the method of manually combing business and indicators of each link and drawing topology, for enterprises with large business volume, multiple applications and complex topology (such as telecommunications and financial industries), not only consumes a lot of manpower and time costs, but also easily leads to inaccurate topology structure, omissions or errors in business associations due to human negligence. The present invention adopts a combination of manual and automated modes, using eBPF to automatically collect data and generate basic topology snapshots, and then manually annotates the snapshots to generate business templates. This method greatly reduces the manual investment in the tedious data collection and collation work in the initial stage of topology construction, and at the same time, with the accuracy of automated data processing, effectively avoids the introduction of manual errors, thereby significantly improving the accuracy and completeness of the business topology, enabling enterprises to obtain more reliable business monitoring infrastructure at a lower labor cost.

[0021] 3. Efficient fault location and business optimization decision support; Based on the end-to-end business monitoring system constructed by the present invention, enterprises can achieve rapid location and accurate analysis when facing faults. When a business exception occurs, the business topology association and indicator calculation module can quickly trace the root cause of the problem based on the business template and the stored call chain data. Whether it is an application service failure or a logical error in the business process, it can be accurately identified in a short time. For example, when calculating business indicators (such as business call duration, call success rate, etc.), if the indicator exceeds the preset threshold, the system can quickly locate the relevant business links and application call links, providing a clear direction for troubleshooting. This greatly shortens the fault repair time and reduces the impact of business interruptions on enterprise operations. At the same time, the long-term accumulated accurate business monitoring data and indicator trend reports provide strong data support for enterprises to make business optimization decisions. Enterprises can deeply analyze business bottlenecks and potential risks based on these data, formulate targeted optimization strategies, and improve the overall performance and competitiveness of the business.

[0022] 4. Powerful system adaptability and flexibility; the monitoring system of the present invention has good adaptability and flexibility. In the data collection link, eBPFAgent can collect data from various types of application nodes (such as application NodeA, application NodeB, application NodeC and nodes in the K8S cluster), and can adapt to enterprise IT environments of different architectures and scales. The fault-tolerant mechanism and data preprocessing submodule in the business topology snapshot generation module enable it to cope with complex and changeable data quality issues and network environment fluctuations, ensuring the stable operation of the system under various conditions. In addition, the rich interactive functions of the template version management submodule and the visualization display module of the business template generation module facilitate enterprises to flexibly adjust monitoring strategies and view display methods according to business development and demand changes, meet the personalized monitoring needs of different user roles and business scenarios, and enable the system to continuously optimize and expand with the evolution of the enterprise business, and always maintain the effectiveness and adaptability of end-to-end business monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 It is a flow chart of the service end-to-end monitoring method based on eBPF of the present invention; Figure 2 It is a flow chart of the analysis of the business end-to-end monitoring system based on eBPF of the present invention. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0025] like Figure 1 and Figure 2 As shown, the first aspect of this embodiment provides a service end-to-end monitoring method based on eBPF, which at least includes the following steps: S1 Data Collection Steps eBPFAgent is used to collect call flow message data from multiple applications (application NodeA, application NodeB, application NodeC) and K8S clusters, and the collected data is passed to the service topology snapshot generation module; S2 business topology snapshot generation steps The service topology snapshot generation module receives the data collected by eBPFAgent and obtains data from eSpan, processes the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port, obtains the call relationship between services by associating Span with TCP_Seq, IP, and Port, and stores it in the eBPF call list; Regularly merge and filter the call relationships between services to generate a panoramic topology snapshot, and generate call chain data to pass to the business topology association and indicator calculation module; S3 business template generation steps The business call chain template labeling module manually labels the nodes and paths of the panoramic topology snapshot generated by the business topology snapshot generation module; Set a business name for the annotation result, save it as a business template and pass it to the business topology association and indicator calculation module; S4 Business Indicator Calculation Steps The business topology association and indicator calculation module obtains the panoramic topology snapshot generated by the business topology snapshot generation module, the call chain data, and the business template generated by the business call chain template marking module; Match the data in the eBPF call list according to the service template and save the matched data to the service list; Calculate the business call duration and call success rate business indicators based on the business links and start and end times.

[0026] As a preferred implementation scheme, in the S1 data collection step in this embodiment, the eBPFAgent has a data integrity verification function to ensure that the collected call flow message data is complete and correct; the call flow message data is collected from the application nodes through eBPFAgent, and the application nodes include application NodeA, application NodeB, application NodeC and nodes in the K8S cluster, and the collected call flow messages contain TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port information.

[0027] As a preferred implementation scheme, in the step of generating a business topology snapshot in this embodiment, when obtaining the calling relationship between services by associating Span through TCP_Seq, IP, and Port, a fault-tolerant mechanism is provided to handle situations where data is missing or erroneous.

[0028] As a preferred implementation scheme, in the service template generation step in this embodiment, when the nodes and paths of the panoramic topology snapshot are manually annotated, the annotation operation is traceable, and the operator, time and modification content of each annotation are recorded.

[0029] As a preferred implementation scheme, in the business indicator calculation step in this embodiment, the business topology association and indicator calculation module can generate a business indicator trend report when calculating the business indicator, which is used to analyze the trend of business performance changes.

[0030] like Figure 2 As shown, the second aspect of this embodiment provides a service end-to-end monitoring system based on eBPF, including a data acquisition module, a service topology snapshot generation module, a service template generation module and a service indicator calculation module, wherein The data collection module includes eBPFAgent, which is used to collect call flow message data from application NodeA, application NodeB, application NodeC and K8S cluster, and pass the data to the service topology snapshot generation module; The service topology snapshot generation module is used to receive data collected by eBPFAgent and data obtained from eSpan, process the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port, obtain the call relationship between services through TCP_Seq, IP, and Port association Span and store it in the eBPF call chain table, regularly merge and filter the call relationship between services to generate a panoramic topology snapshot and call chain data, and pass the call chain data to the service topology association and indicator calculation module; The service template generation module includes a service call chain template marking module, which is used to manually mark the nodes and paths of the panoramic topology snapshot, set the service name and save it as a service template to pass to the service topology association and indicator calculation module; The business indicator calculation module is used to obtain panoramic topology snapshots, call chain data and business templates, match the data of the eBPF call list according to the business template, save the matched data to the business list, and calculate the business call duration and call success rate business indicators according to the business links and start and end times.

[0031] As a preferred implementation scheme, the service topology snapshot generation module in this embodiment further includes a data preprocessing submodule for cleaning and preprocessing the collected data to remove noise data.

[0032] As a preferred implementation scheme, the service template generation module in this embodiment further includes a template version management submodule for managing different versions of service templates to facilitate backtracking and comparison.

[0033] As a preferred implementation scheme, the business indicator calculation module described in this embodiment also includes an indicator abnormality analysis submodule, which performs a detailed analysis of the abnormal situation when the calculated business indicator exceeds a preset threshold.

[0034] As a preferred implementation scheme, this embodiment also includes a visualization display module, which is used to display the panoramic topology snapshot, business templates, business indicators and data in the business link list to the user in an intuitive graphical interface, and supports user interactive operations; including zooming, filtering, and drilling functions. This module can obtain the panoramic topology snapshot from the business topology snapshot generation module, obtain the business template from the business template generation module, and obtain the business indicators and business link list data from the business indicator calculation module.

[0035] like Figure 2 As shown in the figure, the system architecture and data flow of a business end-to-end monitoring method based on eBPF are shown. The entire system involves multiple modules and different types of data, and the modules collaborate with each other to achieve business monitoring.

[0036] Application NodeA, application NodeB, application NodeC and K8S cluster; These are nodes at the application level. Application NodeA, B, and C are associated with eSpan through eBPFAgent. eBPFAgent collects data from these application nodes and passes it to subsequent modules.

[0037] eSpan and business topology snapshot generation module; eSpan is a source of data. The business topology snapshot generation module obtains eSpan from eBPFAgent and processes the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, Dest_Port and other information inside the module, including associating Span to obtain the call relationship between services and storing it in the eBPF call list, periodically merging and filtering the call relationship between services, and finally generating a panoramic topology snapshot.

[0038] Business topology snapshot, call chain and business topology association and indicator calculation module; The panoramic topology snapshot generated by the business topology snapshot generation module is transmitted to the business topology association and indicator calculation module. At the same time, the business topology snapshot generation module also generates call chain data and transmits it to the business topology association and indicator calculation module.

[0039] Business topology association and indicator calculation module, business template and business call chain template marking module The service topology association and indicator calculation module obtains the panoramic topology snapshot and call chain data from the service topology snapshot generation module. This module also obtains the service template from the service call chain template marking module. The service topology association and indicator calculation module matches the data of the eBPF call chain table according to the service template, saves the matched data to the service chain table, and calculates indicators such as service call duration and call success rate.

[0040] Business call chain template labeling module and manual snapshot labeling; The business call chain template labeling module manually labels the nodes and paths of the topology snapshot, sets the business name for the labeling result, and saves it as a business template. Manual snapshot labeling provides a manual operation link for the business call chain template labeling module.

[0041] Business topology association and indicator calculation module and business indicators; The service topology association and indicator calculation module generates service indicators, which reflect the performance of the service.

[0042] Business chain and business data; the business topology association and indicator calculation module generates the business chain, which is associated with the business data. Business data is one of the core data that the entire monitoring system focuses on.

[0043] Data flow: Data starts from the application node, collects eSpan data through eBPFAgent, and passes it to the business topology snapshot generation module. This module generates a panoramic topology snapshot and call chain data, which are passed to the business topology association and indicator calculation module respectively. The business call chain template marking module generates a business template through manual marking and passes it to the business topology association and indicator calculation module, which calculates business indicators and generates business chains, which are finally associated with business data.

[0044] Integration of application monitoring and business monitoring: The system collects data from application nodes through eBPFAgent, combines it with manual labeling to form business templates, and then connects application monitoring data with business monitoring data. For example, the business topology snapshot generation module uses the data collected by eBPFAgent to generate topology snapshots and call chains, the business call chain template labeling module generates business templates by manually labeling topology snapshots, and the business topology association and indicator calculation module calculates business indicators based on business templates and collected data, realizing the integration of application and business monitoring.

[0045] Data processing and indicator generation: There is a complete set of logic from data collection to indicator generation. First, the collected call flow messages are processed in the business topology snapshot generation module to generate topology snapshots and call chains. Then the business call chain template marking module manually marks and generates business templates. Finally, the business topology association and indicator calculation module calculates business indicators based on business templates and related data, such as business call duration and call success rate.

[0046] This system architecture solves the problems of traditional monitoring methods. Traditional application monitoring cannot be combined with business, and traditional business end-to-end monitoring relies on manual combing, which is costly and inaccurate. This system achieves a deep integration of business and application monitoring by combining eBPF collection and manual tagging, reducing labor costs and improving the accuracy and efficiency of monitoring. For example, the business topology snapshot generation module uses eBPF technology to collect data to avoid relying entirely on manual combing of the business architecture, and the business call chain template tagging module ensures the accurate integration of business and monitoring data through manual tagging.

[0047] Application scenario description: In the core business systems of telecommunications operators, such as mobile network call services, data traffic services, and value-added service platforms, there are a large number of complex applications and business processes. The eBPF-based end-to-end business monitoring method and system of the present invention can play a key role.

[0048] For call services, multiple application nodes are involved, such as user authentication applications, call routing applications, billing applications, and applications related to base station communication. By deploying eBPFAgent on these application nodes and K8S clusters (if there is a cloud native architecture part), call flow message data during the call establishment process can be collected. During the data collection process, the data integrity verification function of eBPFAgent ensures that every network data packet information related to the call (such as the IP addresses and port numbers of both parties of the call, and various protocol sequence information during the call establishment process) can be collected completely and accurately. After these data are passed to the service topology snapshot generation module, they can be processed to clearly present the service call relationship in the entire process of the call service from the user initiating the call to the end of the call, and generate a panoramic topology snapshot and call chain data. For example, by analyzing the call chain data, it can be determined whether there is an abnormality in a certain link (such as call quality optimization service) during the call process, causing the call to be stuck or interrupted, and by associating with the service template, the specific impact of the abnormality on the telecommunications service can be quickly located, such as whether it affects the billing accuracy or user experience indicators (such as call connection rate, call drop rate, etc.). The fault-tolerant mechanism in the service topology snapshot generation module ensures that a relatively complete and accurate call service topology structure can be constructed even when network fluctuations cause some data to be missing or erroneous.

[0049] In terms of value-added service platforms, such as SMS group services, mobile video services, etc., there are many types of services and they are frequently updated. The service template generation module of the present invention can easily manage different versions of value-added service templates by manually marking topological snapshots and using the template version management submodule. When launching a new value-added service, the staff can modify and mark it based on the existing template, quickly generate a monitoring template for the new service, and greatly shorten the monitoring deployment time after the service goes online. For example, for the newly launched high-definition video call value-added service, the staff only needs to mark the newly added high-definition video encoding conversion service node and related paths on the basis of the original video service template, and the monitoring template for the service can be quickly established. At the same time, the visual display module can provide telecom operators with an intuitive service monitoring view, and they can view the operation status of a value-added service in detail through functions such as zooming and filtering, such as viewing the traffic usage of mobile video services in a certain time period, the access success rate of users in different regions to video services, etc., so as to adjust resource allocation and optimize service strategies in a timely manner. In addition, when faced with a large number of users concurrently accessing value-added services, the data acquisition module can stably collect data, and the service topology snapshot generation module can efficiently process data to ensure that the real-time and accuracy of the monitoring system are not affected.

[0050] In the online banking systems, securities trading systems and payment settlement systems of financial institutions, extremely high requirements are placed on the reliability, security and performance of the services, and the present invention also has significant advantages.

[0051] Taking the online banking system as an example, it involves multiple business links such as user login verification, account query, transfer and remittance, and financial management business processing, and each link is composed of multiple application services. When the user performs a transfer and remittance operation, eBPFAgent can collect the call flow data from the user-side request to the various services in the bank's backend (such as risk assessment service, fund transfer service, transaction record storage service, etc.). The business topology snapshot generation module builds the service call topology of the transfer and remittance business by associating Span. The generated panoramic topology snapshot and call chain data can clearly show the entire transfer process. If a failure occurs during the transfer process, the business topology association and indicator calculation module can quickly locate which link (such as the docking service with the third-party payment channel) has a problem based on the business template, and deeply analyze the abnormal indicators (such as too long transfer delay time, increased transaction failure rate, etc.) through the indicator abnormality analysis submodule to determine whether it is caused by network failure, application error or business rule restriction. For example, if it is a network failure, it may be that the network connection with the third-party payment channel is unstable. The system can promptly switch to the backup channel or issue a network repair prompt. If it is an application error, the specific code segment can be quickly located for repair. If it is a business rule restriction, such as the transfer amount exceeding the daily limit, accurate prompt information can be promptly fed back to the user.

[0052] In the securities trading system, the transaction instruction processing volume is large and the speed requirement is fast during the peak trading period. The data acquisition module of the present invention can efficiently collect the massive call flow message data in the transaction process, and the data preprocessing submodule quickly cleans and preprocesses these data, removes noise data, and ensures that only valid data enters the subsequent business topology construction and indicator calculation links. Through the real-time monitoring of the panoramic topological snapshots and business indicators (such as transaction instruction processing time, transaction success rate, etc.) of the transaction business, financial institutions can promptly discover the performance bottlenecks in the transaction system, such as the slow response of a market data push service, which may affect the investor's trading decision. With the help of the present invention, the problem can be quickly located and solved, ensuring the stable operation of the securities trading business, and avoiding huge economic losses and customer confidence crises caused by transaction failures. Moreover, under the strict security and compliance requirements of the financial industry, the monitoring system of the present invention can focus on monitoring the business links involving sensitive data, such as user account information query and fund operation links, to ensure the security and compliance of data and prevent data leakage and illegal operations.

[0053] In the solution provided by this embodiment, the rich network data collected by the eBPF technology, combined with the unique TCP_Seq, IP, and Port association Span mechanism, can accurately construct the call relationship between services in a complex distributed system environment. Compared with traditional monitoring technology, it is no longer limited to the simple collection of application performance indicators, but goes deep into the call link analysis at the business level. For example, when processing massive network call flow messages, the association algorithm of the present invention can quickly and accurately integrate the scattered call information into a complete business call chain, and store it in the eBPF call chain table, providing a solid data foundation for subsequent business topology generation and indicator calculation. In the process of data association, efficient data matching technologies such as hash algorithms are used to further improve the speed and accuracy of association. At the same time, for duplicate data or conflicting data that may appear in the association process, the system has an intelligent deduplication and conflict resolution mechanism to ensure the consistency and integrity of the data.

[0054] The automated merging and filtering operations of the business topology snapshot generation module and the intelligent data cleaning of the data preprocessing submodule have greatly reduced the burden of manual data processing. At the same time, the manual labeling link of the business template generation module has fully utilized the advantages of human understanding of business logic. For example, when faced with complex financial business logic, humans can accurately label the nodes and paths in the topology snapshot in a business sense, such as labeling a node as "anti-money laundering compliance inspection service". This automated and manual collaboration method makes the monitoring system both efficient and accurate, avoiding the business understanding deviation that may be caused by pure automated monitoring and the low efficiency and high error rate of pure manual monitoring. In the manual labeling process, it supports multiple people to collaborate online at the same time, and through the traceability function of the labeling operation, the operator, time and modification content of each labeling are recorded in detail, which is convenient for subsequent review and version management.

[0055] The monitoring system of the present invention can adapt well to both traditional IT architecture and emerging cloud-native (such as K8S cluster) architecture. The eBPFAgent of the data acquisition module can be flexibly deployed on various types of application nodes, and the various modules of the system can be adaptively adjusted when facing different network environments and application loads. For example, in the business topology snapshot generation module, when the network load is light, more sophisticated merging and filtering rules can be used to obtain more detailed topology information; when the network load is heavy, it automatically adjusts to a more efficient coarse-grained processing method to ensure that the overall performance of the system is not affected and to always maintain effective monitoring of the business. In addition, the system also has good scalability. When an enterprise adds new business applications or upgrades existing applications, the monitoring system can easily expand modules or adjust configurations to quickly adapt to business changes.

[0056] In the telecommunications and financial industries, business interruptions may lead to huge economic losses and customer loss. The present invention can locate and warn the fault at the first time it occurs by real-time monitoring of each link of the business. For example, when a core routing application in a telecommunications network fails, the business topology association and indicator calculation module can quickly discover the impact of the failure on related services (such as call services and data services in multiple regions), and promptly notify the operation and maintenance personnel to repair it, which greatly shortens the business interruption time and ensures the continuity of the business. In the long run, stable business operations help to enhance customer trust and loyalty to the company. In the financial industry, for key business systems such as payment and settlement systems, the high-reliability monitoring of the present invention can ensure the accurate processing of each transaction, and even in the event of a local system failure, it can ensure the uninterrupted operation of the business through a fast fault switching and recovery mechanism, effectively maintaining the stable order of the financial market.

[0057] Based on the business indicator trend report generated by the business indicator calculation module, enterprises can deeply analyze the trend of business performance changes, find bottlenecks in business processes and optimize them. For example, by analyzing the indicator trends of transaction business, financial institutions find that the processing time of transaction instructions in a certain period is too long, which may be caused by insufficient resources in a certain middleware. Enterprises can reasonably allocate resources based on this analysis result, such as increasing server memory or optimizing database configuration, so as to improve business performance, avoid waste of resources, and achieve efficient use of resources. In the telecommunications industry, by analyzing business traffic and performance indicators in different regions and time periods, network resource allocation can be dynamically adjusted, such as allocating more bandwidth resources to areas with high data service demand during peak traffic hours, thereby improving overall network resource utilization and user experience.

[0058] On the one hand, the present invention reduces the workload of manually combing business topology and monitoring business, and reduces labor costs. For example, in the telecommunications industry, it used to take a large number of operation and maintenance personnel to spend several weeks combing business topology. Now with the help of the present invention, it can be completed in a shorter time and with higher accuracy. On the other hand, through accurate fault location and performance optimization, potential economic losses caused by business failures are avoided and the operational risks of enterprises are reduced. For example, in financial transaction systems, quickly resolving transaction failures can prevent risks such as customer claims and regulatory penalties caused by transaction delays or failures. In addition, the intuitive business monitoring view provided by the visualization display module of the present invention helps enterprise management to quickly understand the business operation status, make more informed decisions, further improve the efficiency of enterprise operation management, and indirectly reduce operating costs.

[0059] The eBPF-based end-to-end business monitoring method and system of the present invention have broad and important application value in industries such as telecommunications and finance, bringing significant benefits in many aspects to enterprises, effectively solving many problems existing in existing monitoring technologies, and promoting the development and progress of enterprise business monitoring technology.

[0060] Although the embodiments of the present invention have been shown and described above, it will be appreciated that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, substitute and vary the above embodiments within the scope of the present invention.

Claims

1. A service end-to-end monitoring method based on eBPF, characterized in that: At least the following steps are included: S1 data collection step: collect call flow message data from multiple applications and K8S clusters through eBPFAgent, and pass the collected data to the business topology snapshot generation module; S2 Business topology snapshot generation step: The business topology snapshot generation module receives the data collected by eBPFAgent and obtains data from eSpan, processes the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port, obtains the call relationship between services by associating Span with TCP_Seq, IP, and Port, and stores it in the eBPF call list; Regularly merge and filter the call relationships between services to generate a panoramic topology snapshot, and generate call chain data to pass to the business topology association and indicator calculation module; S3 business template generation steps; The business call chain template labeling module manually labels the nodes and paths of the panoramic topology snapshot generated by the business topology snapshot generation module; Set a business name for the annotation result, save it as a business template and pass it to the business topology association and indicator calculation module; S4 business indicator calculation step: the business topology association and indicator calculation module obtains the panoramic topology snapshot generated by the business topology snapshot generation module, the call chain data and the business template generated by the business call chain template marking module; Match the data in the eBPF call list according to the service template and save the matched data to the service list; Calculate the business call duration and call success rate business indicators based on the business links and start and end times.

2. The eBPF-based end-to-end service monitoring method according to claim 1, characterized in that: In the S1 data collection step, the eBPFAgent has a data integrity verification function to ensure that the collected call flow message data is complete and correct; the call flow message data is collected from the application nodes through eBPFAgent, and the application nodes include application NodeA, application NodeB, application NodeC and nodes in the K8S cluster. The collected call flow messages contain TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port information.

3. The eBPF-based end-to-end service monitoring method according to claim 1, characterized in that: In the business topology snapshot generation step, when obtaining the calling relationship between services by associating Span through TCP_Seq, IP, and Port, a fault-tolerant mechanism is provided to handle situations where data is missing or erroneous.

4. The eBPF-based service end-to-end monitoring method according to claim 1, characterized in that: In the service template generation step, when the nodes and paths of the panoramic topology snapshot are manually annotated, the annotation operation is traceable, and the operator, time and modification content of each annotation are recorded.

5. The eBPF-based service end-to-end monitoring method according to claim 1, characterized in that: In the business indicator calculation step, the business topology association and indicator calculation module may generate a business indicator trend report when calculating the business indicator, which is used to analyze the business performance change trend.

6. A service end-to-end monitoring system based on eBPF, characterized by: It includes data collection module, business topology snapshot generation module, business template generation module and business indicator calculation module. The data collection module includes eBPFAgent, which is used to collect call flow message data from application NodeA, application NodeB, application NodeC and K8S cluster, and pass the data to the service topology snapshot generation module; The service topology snapshot generation module is used to receive data collected by eBPFAgent and data obtained from eSpan, process the call flow message containing TCP_Seq, Src_IP, Src_Port, Dest_IP, and Dest_Port, obtain the call relationship between services through TCP_Seq, IP, and Port association Span and store it in the eBPF call chain table, regularly merge and filter the call relationship between services to generate a panoramic topology snapshot and call chain data, and pass the call chain data to the service topology association and indicator calculation module; The service template generation module includes a service call chain template marking module, which is used to manually mark the nodes and paths of the panoramic topology snapshot, set the service name and save it as a service template to pass to the service topology association and indicator calculation module; The business indicator calculation module is used to obtain panoramic topology snapshots, call chain data and business templates, match the data of the eBPF call list according to the business template, save the matched data to the business list, and calculate the business call duration and call success rate business indicators according to the business links and start and end times.

7. The eBPF-based end-to-end service monitoring system according to claim 6, characterized in that: The service topology snapshot generation module also includes a data preprocessing submodule, which is used to clean and preprocess the collected data to remove noise data.

8. The eBPF-based end-to-end service monitoring system according to claim 6, characterized in that: The service template generation module also includes a template version management submodule, which is used to manage different versions of service templates to facilitate backtracking and comparison.

9. The eBPF-based end-to-end service monitoring system according to claim 6, characterized in that: The business indicator calculation module also includes an indicator abnormality analysis submodule, which performs a detailed analysis of the abnormal situation when the calculated business indicator exceeds a preset threshold.

10. The eBPF-based service end-to-end monitoring system according to any one of claims 6 to 9, characterized in that: It also includes a visualization display module, which is used to display the panoramic topology snapshot, business templates, business indicators and data in the business chain table to users in an intuitive graphical interface, and supports user interactive operations; Including zooming, filtering, and drilling functions, this module can obtain panoramic topology snapshots from the business topology snapshot generation module, obtain business templates from the business template generation module, and obtain business indicators and business link table data from the business indicator calculation module.

Images