All-environment and all-scene service link monitoring method and system under multi-cloud isomerism

Abstract

The invention discloses an all-environment and all-scene service link monitoring method under multi-cloud heterogeneity, and belongs to the technical field of computers. The method comprises the steps of obtaining a request sent by an application; generating a standard mark according to the request; acquiring link monitoring data of the application; and displaying according to the corresponding standard mark and the link monitoring data. According to the method, unified link monitoring capability is carried out on service systems distributed in a traditional architecture (SkyWalking) and a cloud architecture (TSF) by breaking through link monitoring of an on-cloud and under-cloud cross-platform architecture and integrating a platform architecture, a technical mechanism and resource investment, and meanwhile, three-dimensional expansion is carried out on link monitoring in combination with service dial testing, basic resource monitoring and network monitoring, so that the link monitoring efficiency is improved. And a three-dimensional full-service link monitoring blueprint is formed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for monitoring business links in all environments and scenarios under multi-cloud heterogeneity. Background Art

[0002] With the in-depth application of modern microservice architecture, the business of a system is often decomposed into different services, and then split and implemented according to different dimensions. In addition to conventional business microservices, there are also complex link calls of various service components such as message services, data services, and cache services. These components together constitute complex distributed business units.

[0003] Small-grained service implementation ensures rapid iteration of development and robust expansion of services, but this also leads to new problems. A business request often involves multiple components. When a business request has a problem, such as a long request response time, it is difficult to locate the microservice node problem. It is necessary to check the various services and components involved and analyze their related logs, making the problem handling and location very costly and inefficient. To solve such problems, many application performance management (APM) tools have appeared on the market, such as Zinpking, Pinpoint, SkyWalking, etc.

[0004] Zipkin: An open source distributed tracing system originally developed by Twitter and contributed to the open source community. It is based on Google's Dapper paper and is designed to collect information about the execution time of distributed services to track service call links and analyze service execution delays.

[0005] Pinpoint: Open sourced by the Naver R&D team in South Korea, it is an open source application performance management (APM) tool that is mainly used to monitor and manage the performance of large-scale Java-based distributed systems. It installs an agent when the application starts, which enables monitoring without modifying the code, and with minimal performance loss.

[0006] SkyWalking: An excellent domestic APM component that focuses on link and performance monitoring. It provides distributed tracing, service mesh, performance monitoring and diagnosis functions. It is non-invasive and provides an integrated solution for distributed tracing, service mesh telemetry analysis, metric aggregation and visualization.

[0007] These tools can each monitor business links in a single system environment.

[0008] When the business scenarios involved in the business need to span different application systems of multiple manufacturers, have different technical architectures, and are deployed on different platforms, some platforms have existing link monitoring, and some platforms need to create new business link monitoring. So how to solve the link monitoring in such complex actual scenarios, understand the call link of business requests, quickly locate which vendor's service component caused the problem, and provide intuitive analysis and display of the problem, then a single business link monitoring tool cannot meet the needs.

[0009] For example, a vendor's application system is deployed in a traditional environment and uses a traditional link monitoring product, but another vendor's application system is deployed on the cloud and uses the link monitoring service provided by the cloud platform. Another vendor uses the Go architecture for development, which is another link monitoring solution. When the business needs to connect the application services of these three vendors in series, a single link monitoring cannot understand the business request link, nor can it quickly locate which vendor's service component caused the problem. Even when troubleshooting, complicated coordination and communication are required, which greatly hinders the location and processing of the problem. Summary of the invention

[0010] The purpose of the present invention is to provide an efficient method and system for monitoring business links in all environments and scenarios under multi-cloud heterogeneity. When the business scenarios involve multiple application vendors and are cross-platform, cross-environment, cross-architecture, and cross-scenario, the platform architecture, technical institutions, and resource investment are integrated to solve the problem that single link monitoring cannot provide monitoring coverage, and to perform business link monitoring in all environments and scenarios.

[0011] In order to solve the above technical problems, the present invention provides a full-environment and full-scenario business link monitoring method under multi-cloud heterogeneity, comprising the following steps:

[0012] Get the request sent by the application;

[0013] Generate standard logos upon request;

[0014] Link monitoring data collection for applications;

[0015] Displayed according to corresponding standard symbols and link monitoring data.

[0016] Preferably, obtaining the request sent by the application specifically includes the following steps:

[0017] The full-service link monitoring gateway obtains requests sent by multiple applications.

[0018] Preferably, generating a standard mark according to the request specifically includes the following steps:

[0019] The full-service link monitoring gateway generates a standard flag based on the request, adds the standard flag to the request header, and obtains a unified processed request;

[0020] The full-service link monitoring gateway will uniformly process and forward the requests to the downstream destination.

[0021] Preferably, the full service link monitoring gateway generates a standard flag according to the request, specifically comprising the following steps:

[0022] The full-service link monitoring gateway generates a corresponding standard flag based on the link monitoring tool TraceId in the request.

[0023] Preferably, the full-service link monitoring gateway forwards the uniformly processed requests to the downstream destination, specifically including the following steps:

[0024] The full-service link monitoring gateway forwards the uniformly processed request to the downstream destination according to the request address in the uniformly processed request.

[0025] Preferably, the method further comprises the following steps:

[0026] The full-service link monitoring gateway generates link logs based on the unified processed requests and sends them to the log center for storage;

[0027] The log center sends the link log to the link center.

[0028] Preferably, the link monitoring data collection of the application specifically includes the following steps:

[0029] Monitor the application through the link monitoring tool to obtain link monitoring data;

[0030] The link monitoring tool sends the link monitoring data to the link center.

[0031] Preferably, displaying according to standard signs and link monitoring data specifically includes the following steps:

[0032] The link center uniformly encapsulates the link monitoring data and link logs to obtain encapsulated data;

[0033] Display the packaging data.

[0034] Preferably, the full-service link monitoring gateway is implemented using Spring Cloud Gateway.

[0035] The log center is a log center using a monitoring platform, or a unified log center is established with ELK as the architecture.

[0036] The present invention also provides a full-environment and full-scenario business link monitoring system under multi-cloud heterogeneity, including:

[0037] A request acquisition module is used to acquire the request sent by the application;

[0038] A standard logo generation module, used for generating a standard logo according to a request;

[0039] Link monitoring data collection module, used for link monitoring data collection of applications;

[0040] The display module is used to display according to corresponding standard symbols and link monitoring data.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] By opening up the link monitoring of cross-platform architectures on and off the cloud, integrating platform architecture, technical institutions, and resource investment, we have achieved unified link monitoring capabilities for business systems distributed in traditional architecture (SkyWalking) and cloud architecture (TSF). At the same time, combined with business dialing, basic resource monitoring, and network monitoring, we have expanded link monitoring in three dimensions to form a three-dimensional full-business link monitoring blueprint.

[0043] If a problem occurs in any link, the fault can be located in time. At the same time, comprehensive basic layer monitoring and network monitoring can quickly troubleshoot and resolve the problem. In addition to supporting the location of applications and problems within applications, it also supports full-scenario analysis and location of problems outside the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0045] Figure 1 It is a flowchart of existing link monitoring;

[0046] Figure 2 It is a schematic diagram of the service call link of the existing link monitoring;

[0047] Figure 3 It is a schematic diagram of the collaborative processing of multiple business systems of multiple manufacturers for existing link monitoring;

[0048] Figure 4 It is a schematic diagram of collaborative processing of full business link monitoring of the present invention;

[0049] Figure 5 It is a flow chart of a method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity of the present invention;

[0050] Figure 6 It is a schematic diagram of the full-service link monitoring gateway request forwarding;

[0051] Figure 7 It is a schematic diagram of the link center encapsulation display;

[0052] Figure 8 This is a schematic diagram of application A and application B implementing full-environment and full-scenario business link monitoring. DETAILED DESCRIPTION

[0053] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0054] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] Taking a business of an application system as an example, when monitoring the business link, the existing link monitoring solution connects the same request distributed on each service node through a distributed globally unique ID (i.e. TraceId), thereby restoring the call relationship, tracking system problems, and then analyzing the call data and counting system indicators to monitor the application link. This is a session-level tracing capability. A single distributed request is restored to a call link, starting from the time the client initiates the request and arrives at the boundary of the system, and recording each service that the request passes through until a response is returned to the client. Figure 1 shown.

[0057] When tracing, specific identifiers are added to each request to generate the identifiers (SpanId and ParentId, representing the unique identifier of the request link, the request initiator, and the upstream source of the request) and status information required for link tracking. This information includes key data such as the service name of the initiator of the request, the function operation name, the target service access address, the processing time, the response status, and the error information. Finally, by connecting TraceId, SpanId, and ParentId in series, a single business call link is organized, such as Figure 2 As shown:

[0058] The above is the basic situation of existing link monitoring. However, the actual situation is often more complicated. When a business requires multiple business systems from multiple manufacturers to work together, from the perspective of application system A, a business has a problem. This business is started by application system A, and A's link monitoring tool quickly discovered that it was a problem with service 3. Then it was found that there was a problem with the call to the external system of service 3, and the situation was unknown. At this time, the two application systems need to coordinate with specialized personnel to analyze and connect, and then go to the external system for investigation, analysis and processing. The connection cost is high, and the problem investigation is complicated, so we return to the processing method when there is no link monitoring, such as Figure 3 shown.

[0059] To solve such problems in actual scenarios, we conducted extensive research and continuous attempts on different application vendors and different link monitoring tools, and comprehensively considered platform architecture, technical institutions, and resource investment, overcame numerous difficulties, and finally formed a full-environment, full-scenario business link monitoring solution under multi-cloud heterogeneity.

[0060] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0061] A full-environment, full-scenario business link monitoring method under multi-cloud heterogeneity, with the specific technical architecture as follows:

[0062] 1. Establish a full-service link monitoring gateway, support HTTP / RPC protocol, transfer various application requests, and add standard logos;

[0063] 2. Use the log center to collect and store key application logs;

[0064] 3. Establish a link center to retrieve the link monitoring data of each system link monitoring tool and align the cross-link monitoring data according to the standard request flag.

[0065] The specific idea is: when calling between applications, add a full-service link monitoring gateway, which adds a standard identifier to the request, and prints the source TraceId and the standard identifier in the log; each application prints the standard identifier and the TraceId of its own link monitoring tool in the log; all logs are collected to a unified log center for analysis and use, such as Figure 4 shown.

[0066] The specific implementation process of the monitoring method of this embodiment is as follows Figure 5 As shown:

[0067] The full-service link monitoring gateway is implemented using Spring Cloud Gateway. After receiving the request, it determines whether the TraceId of each link monitoring tool exists. If so, it generates a standard flag and adds it to the request header. Then, based on the request address and other information, it forwards the request to the downstream destination, such as Figure 6 shown.

[0068] Log center: Use the existing log center of the monitoring platform, or establish a unified log center with ELK (Elasticsearch, Logstash and Kibana) as the architecture. The log center collects standard format logs for link forwarding and tracking.

[0069] In the link forwarding log, TraceExchange is the log keyword, TraceId is the source trace number, and StandardId is the standard request flag. The following is an example:

[0070] [INFO]2024-02-14 17:28:54.710[TraceExchange]-[TraceId-3210]-[StandardId-7483c]

[0071] In the link tracking log, TraceTarget is the log keyword, StandardId is the standard request flag, and TraceId is the current Trace number. The example is as follows:

[0072] [INFO]2024-02-14 17:28:54.710[TraceTarget]-[StandardId-7483c]-[TraceId-6742]

[0073] The link center is connected to the link monitoring tools of each application system or the platform it relies on through interface docking and database docking. It combines the standard request flags in the log center and provides a consistent link tracking data query interface after unified packaging for use by the link monitoring display platform, such as Figure 7 shown.

[0074] Full-environment, full-scenario business link monitoring and display platform: Through the unified interface of the link center, the health status of each business is displayed with the business scenario as the main dimension; at the same time, the monitoring content of the link nodes is integrated into the basic layer monitoring, network monitoring, etc. for three-dimensional expansion.

[0075] In order to better illustrate the technical effect of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:

[0076] Embodiment 1: A method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity;

[0077] like Figure 8 As shown, different link monitoring tools are used for application A and application B. Through this embodiment, when displaying the service link, the barrier of the single application link monitoring tool is broken through, and different link monitoring tools are connected in series to realize the full environment and full scenario service link monitoring.

[0078] At the same time, when viewing details, the limitation of link monitoring tools that can only view link monitoring content is broken through, and basic layer monitoring data, network monitoring data, etc. are integrated to achieve three-dimensional monitoring of the link.

[0079] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules, modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0080] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0081] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0082] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A full-environment, full-scenario business link monitoring method under multi-cloud heterogeneity, characterized in that: include: Get the request sent by the application; Generate standard logos upon request; Link monitoring data collection for applications; Displayed according to corresponding standard symbols and link monitoring data.

2. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 1 is characterized in that: Get the request sent by the application, which includes the following steps: The full-service link monitoring gateway obtains requests sent by multiple applications.

3. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 2 is characterized in that: Generate a standard logo based on the request, including the following steps: The full-service link monitoring gateway generates a standard flag based on the request, adds the standard flag to the request header, and obtains a unified processed request; The full-service link monitoring gateway will uniformly process and forward the requests to the downstream destination.

4. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 3 is characterized in that: The full-service link monitoring gateway generates a standard flag based on the request, which specifically includes the following steps: The full-service link monitoring gateway generates a corresponding standard flag based on the link monitoring tool TraceId in the request.

5. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 4 is characterized in that: The full-service link monitoring gateway forwards the processed requests to the downstream destination, which includes the following steps: The full-service link monitoring gateway forwards the uniformly processed request to the downstream destination according to the request address in the uniformly processed request.

6. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 5 is characterized in that: The following steps are also included: The full-service link monitoring gateway generates link logs based on the unified processed requests and sends them to the log center for storage; The log center sends the link log to the link center.

7. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 6 is characterized in that: The application link monitoring data collection includes the following steps: Monitor the application through the link monitoring tool to obtain link monitoring data; The link monitoring tool sends the link monitoring data to the link center.

8. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 7 is characterized in that: Display based on standard symbols and link monitoring data, including the following steps: The link center uniformly encapsulates the link monitoring data and link logs to obtain encapsulated data; Display the packaging data.

9. The method for monitoring business links in all environments and scenarios under multi-cloud heterogeneity according to claim 8 is characterized in that: The full-service link monitoring gateway is implemented using Spring Cloud Gateway; The log center is a log center using a monitoring platform, or a unified log center is established with ELK as the architecture.

10. A full-environment, full-scenario service link monitoring system under multi-cloud heterogeneity, used to implement the full-environment, full-scenario service link monitoring method under multi-cloud heterogeneity as described in any one of claims 1-9, characterized in that: include: A request acquisition module is used to acquire the request sent by the application; A standard logo generation module, used for generating a standard logo according to a request; Link monitoring data collection module, used for link monitoring data collection of applications; The display module is used to display according to corresponding standard symbols and link monitoring data.

Images