Methods, devices, computer equipment, and storage media for end-to-end distributed monitoring

Abstract

This application relates to a method, apparatus, computer device, and storage medium for end-to-end distributed monitoring. The method includes: acquiring call data from each application in a distributed system; asynchronously reporting the call data to a backend service node pointed to by a pre-configured domain name via a first load balancing unit, obtaining processed data generated by the backend service node through aggregation of the call data, and storing the processed data in a search service unit; visualizing the processed data via a second load balancing unit to obtain the operating information of the distributed system. This method can improve the efficiency of fault location and query in a distributed tracing system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, and storage medium for end-to-end distributed monitoring. Background Technology

[0002] With the rise of microservice architecture and container technology, and the increasing volume of business, distributed tracing systems have become increasingly complex in order to enable flexible business expansion. More and more components are becoming distributed, such as microservices, distributed databases, distributed caches, distributed storage, and cross-domain calls. These components constitute a complex distributed network.

[0003] In a distributed microservice architecture, as the number of applications increases and the call relationships become more complex, when system anomalies occur, operations and development teams can only troubleshoot application anomalies one by one based on experience. For example, operations and development teams check operation logs and resource status, while DBAs check SQL, which is inefficient. This results in extremely low efficiency in analyzing and locating problems and long system recovery times. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for end-to-end distributed monitoring that can improve the efficiency of fault location and query in distributed tracing systems, addressing the aforementioned technical problems.

[0005] A full-link distributed monitoring method, the method comprising:

[0006] Obtain call data for each application in the distributed system;

[0007] The first load balancing unit asynchronously reports the call data to the backend service node pointed to by the pre-configured domain name, obtains the processed data generated by the backend service node through aggregation of the call data, and stores the processed data in the search service unit;

[0008] The processed data is visualized by the second load balancing unit to obtain the operating information of the distributed system.

[0009] In one embodiment, the call data includes interface response time, number of requests, call chain information, and virtual machine information. The process of asynchronously reporting the call data to a pre-configured backend service node pointed to by a pre-configured domain name via a first load balancing unit, and obtaining processed data generated by the backend service node through aggregation of the call data, includes:

[0010] The first load balancing unit asynchronously reports and stores the interface response time, number of requests, call chain information, and virtual machine information in the message queue corresponding to the backend service node pointed to by the pre-configured domain name.

[0011] The data in the message queue is aggregated to obtain corresponding processed data; the processed data includes inter-application relationships, inter-service relationships, and service metric data.

[0012] In one embodiment, the method further comprises:

[0013] Receive timed data processing instructions;

[0014] The processed data in the search service unit is synchronized to the transit database according to the timed data processing instruction;

[0015] The data is stream-processed through the transit database to obtain the streaming data required by the digital operation platform.

[0016] In one embodiment, the method further comprises:

[0017] Receive a link viewing instruction, the link viewing instruction carrying the application identifier of the distributed application;

[0018] Based on the application identifier, the corresponding call chain information is obtained from the processing data to obtain information on each service, execution method, execution time, and execution status that the distributed application request goes through.

[0019] In one embodiment, before obtaining the corresponding call chain information from the processing data based on the application identifier, the method further includes:

[0020] Obtain the usage permissions of the logged-in user and verify the usage permissions;

[0021] When the logged-in user's access permissions are verified, the step of obtaining the corresponding call chain information from the processing data based on the application identifier is executed.

[0022] In one embodiment, the method further comprises:

[0023] When a service metric is detected to be greater than the metric threshold, an alarm message is generated.

[0024] The alarm information is sent to the user's terminal using a preset sending method.

[0025] A full-link distributed monitoring device, the device comprising:

[0026] The acquisition module is used to acquire call data from each application in the distributed system; the data processing module is used to report the call data to the backend service node pointed to by the pre-configured domain name in an asynchronous manner through the first load balancing unit, and obtain the processed data generated by the backend service node through the aggregation processing of the call data.

[0027] The display module is used to visualize the processed data through the second load unit to obtain the operating information of the distributed system.

[0028] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0029] Obtain call data for each application in the distributed system;

[0030] The first load balancing unit asynchronously reports the call data to the backend service node pointed to by the pre-configured domain name, obtains the processed data generated by the backend service node through aggregation of the call data, and stores the processed data in the search service unit;

[0031] The processed data is visualized by the second load balancing unit to obtain the operating information of the distributed system.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0033] Obtain the call data of each application in the distributed system; report the call data to the backend service node pointed to by the pre-configured domain name in an asynchronous manner through the first load balancing unit, obtain the processing data generated by the backend service node aggregating the call data, and store the processing data in the search service unit;

[0034] The processed data is visualized by the second load balancing unit to obtain the operating information of the distributed system.

[0035] The aforementioned end-to-end distributed monitoring method, device, computer equipment, and storage medium acquire call data from each application in the distributed system; the first load balancing unit asynchronously reports the call data to the backend service node pointed to by a pre-configured domain name, obtaining the processed data generated by the backend service node through aggregation of the call data. The pre-configured domain name eliminates the need to consider changes in the backend service node, improving the efficiency of data collection and reporting; the second load balancing unit visualizes the processed data, obtaining the operating information of the distributed system; the separation of data collection and reporting from the first and second load balancing units improves high availability, thereby enhancing the efficiency of fault location and querying in the distributed tracing system. Attached Figure Description

[0036] Figure 1 This is an application environment diagram of the end-to-end distributed monitoring method in one embodiment;

[0037] Figure 2 This is a flowchart illustrating a full-link distributed monitoring method in one embodiment;

[0038] Figure 3 This is a flowchart illustrating the end-to-end distributed monitoring method in another embodiment;

[0039] Figure 4 This is a schematic diagram of the interface for setting the indicator threshold in one embodiment;

[0040] Figure 5 This is a schematic diagram illustrating the verification of logged-in user permissions in one embodiment.

[0041] Figure 6 This is a system architecture diagram of a distributed tracing system in one embodiment;

[0042] Figure 7 This is a structural block diagram of a full-link distributed monitoring device in one embodiment;

[0043] Figure 8 This is a structural block diagram of a full-link distributed monitoring device in another embodiment;

[0044] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. Detailed Implementation

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0046] The end-to-end distributed monitoring method provided in this application can be applied to, for example... Figure 1 In the application environment shown, the first terminal 102 and the second terminal 104 communicate with the server 106 via a network. The server 106 obtains the call data of each application in the distributed system; through the first load balancing unit, it asynchronously reports the call data to the backend service node pointed to by the pre-configured domain name, obtains the processed data generated by the backend service node aggregating the call data, and stores the processed data in the search service unit; through the second load balancing unit, it visualizes the processed data to obtain the operating information of the distributed system. The first terminal 102 and the second terminal 104 can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The first terminal is used to collect the call data of each application in the distributed system, and the second terminal is used to visualize the operating status of the distributed tracking system. The server 106 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0047] In one embodiment, such as Figure 2 As shown, a full-link distributed monitoring method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0048] Step 202: Obtain the call data of each application in the distributed system.

[0049] The call data refers to the data generated by each application call request in the distributed system, including various performance indicators of each call during the request processing. These performance indicators may include interface response time, number of requests, call chain information, and virtual machine (JVM) information.

[0050] Specifically, when a business request is received, the corresponding application in the distributed system is determined, and call data such as the response time of the interface corresponding to the application in the distributed system, the number of requests, the call chain information, and the virtual machine information are collected through a collection terminal (e.g., an agent probe). Among these, a domain name is configured on the collection terminal, which points to the backend service node of the distributed tracing system.

[0051] Step 204: The first load balancing unit asynchronously reports the call data to the backend service node pointed to by the pre-configured domain name, obtains the processing data generated by the backend service node after aggregating the call data, and stores the processing data in the search service unit.

[0052] The domain name refers to the domain name configured for the Nacos cluster (e.g., apm-api.sf-express.com). The backend service nodes are the data processing nodes of the distributed tracing system. The distributed tracing system is a tool used to obtain distributed system behavior and analyze performance issues. Based on the distributed tracing system, faults in the distributed system can be accurately and quickly located, and fault-solving strategies can be determined. The distributed tracing system is not limited to Skywalking. The load balancing unit is used to prevent single points of failure of the server. High availability of the server can be achieved through Keepalived in conjunction with Nginx. The load balancing unit can be, but is not limited to, Keepalived + Nginx. The load balancing unit can achieve high availability (HA) through Hyper Text Transfer Protocol (HTTP) and GPRC protocol. The first load balancing unit is used to report the application call data collected by the collection terminal in the distributed system to the backend service node (e.g., OAP Server) pointed to by the pre-configured domain name. The Search Service Unit is a Lucene-based search server that provides a distributed, scalable, real-time search and analytics engine. It enables full-text search, distributed real-time file storage of data, and indexes every field to make it searchable. It can perform real-time data analysis and scale to hundreds of servers, handling petabytes (PB) of structured or unstructured data. The Search Service Unit can be, but is not limited to, ElasticSearch.

[0053] Specifically, the system acquires call data collected by the agent probe and asynchronously reports the collected interface response time, request count, call chain information, and virtual machine information to the distributed tracing system via the first load balancing unit. The data is then sent to the SkywalkingCollector via HTTP or gRPC and stored in the message queue corresponding to the backend service node pointed to by the pre-configured domain name in the distributed tracing system. The call data in the message queue is aggregated to obtain the corresponding processed data, which includes inter-application relationships, inter-service relationships, and service metrics. This processed data is then stored in Elasticsearch, which performs distributed real-time file storage and indexes each field. The indexes are used for data retrieval, improving query efficiency and accuracy.

[0054] Step 206: The processed data is visualized through the second load balancing unit to obtain the operating information of the distributed system.

[0055] Specifically, the processing data is sent to the page management unit (e.g., a web visualization platform) through the second load balancing unit. The web visualization platform displays the processing data to obtain the operating information of the distributed system. The web visualization platform can visually display the call data of each application in the distributed system, which may include various performance metrics of each call during request processing (e.g., database response time, response time distribution, throughput, SLA, etc.), call relationships between distributed services, link call process data, and comparisons of metric data.

[0056] In the aforementioned end-to-end distributed monitoring method, call data corresponding to each application in the distributed system is acquired. This data is then collected by a collection terminal and asynchronously reported to the backend service node pointed to by a pre-configured domain name via a first load balancing unit. The backend service node then aggregates and processes the call data, generating processed data. By pre-configuring the domain name, data reporting efficiency is not affected by node changes. The processed data is then visualized by a second load balancing unit to obtain the operating information of the distributed system. By adding a first and second load balancing unit to the distributed tracing system, data collection and reporting are separated from visualization, ensuring high server availability and data processing performance, thereby improving the efficiency of fault location and querying in the distributed tracing system.

[0057] In another embodiment, such as Figure 3 As shown, a full-link distributed monitoring method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0058] Step 302: Obtain the call data of each application in the distributed system.

[0059] Step 304: The first load balancing unit asynchronously reports the interface response time, number of requests, call chain information, and virtual machine information and stores them in the message queue corresponding to the backend service node pointed to by the pre-configured domain name.

[0060] Step 306: Aggregate the data in the message queue to obtain the corresponding processed data; and store the processed data in the search service unit.

[0061] The data processed includes relationships between applications and services, as well as service metrics data, and this processed data is stored in the search service unit. Aggregation processing refers to generating a single value from each array.

[0062] Optionally, in one embodiment, a timed data processing instruction is received; the processed data in the search service unit is synchronized to the transit database according to the timed data processing instruction; and the processed data is stream-processed through the transit database to obtain the streaming data required by the digital operation platform.

[0063] Specifically, the scheduled data processing instructions can be generated by a unified scheduled task platform. Upon receiving these instructions, the system saves the processed data (i.e., the processed data) from the search service unit to a database used for storing temporary data (e.g., Kafka), synchronizes the data to a relay database (e.g., Flink), performs streaming processing on the processed data in the relay database to obtain the streaming data (i.e., real-time data) corresponding to each application's call request, and stores the streaming data in a historical database (e.g., a Hive database) to obtain the streaming data required by the digital operations platform. The digital operations platform refers to the company's internal operating system, which sets corresponding rules based on business needs, such as performing secondary classification, aggregation, and display of the streaming data stored in the Hive database (e.g., the health status of the distributed system). By periodically processing the processed data into streaming data, real-time data corresponding to each application's call request is obtained. Based on this real-time data, the operating status of the distributed system can be accurately and quickly obtained.

[0064] Optionally, in one embodiment, when the detected service metric data exceeds the metric threshold, an alarm message is generated; the alarm information is then sent to the user's terminal using a preset sending method. The service metric can be things like interface response time and request count, and the metric threshold is a pre-set threshold.

[0065] Specifically, based on business scenario requirements, the corresponding pre-configured alarm rule data is determined. When the service metric data of the alarm rule data exceeds the metric threshold, an alarm message is generated and sent to the user's terminal using a preset sending method. For example, if the interface response time exceeds 2 seconds, an alarm is triggered, an alarm message is generated, and the alarm message is sent to the user's terminal using a preset sending method. Figure 4 The image shows a threshold setting interface for an indicator in one embodiment. The interface includes alarm level, threshold setting, effective time window, alarm method, and notification personnel. Alarm methods include voice alerts, emails, and SMS messages, etc. Notification personnel can be the employee ID of the person issuing the notification. Based on the business scenario, corresponding alarm rule data is set, relevant service indicators are monitored, and alarm information is sent to the user's terminal using a preset method. This allows users to promptly handle faults in the distributed system and ensures that the user terminal can obtain operational messages from the distributed system in a timely manner, improving data timeliness and the stability of the distributed system.

[0066] Step 308: The processed data is visualized through the second load balancing unit to obtain the operating information of the distributed system.

[0067] Step 310: Obtain the login user's usage permissions and verify the usage permissions.

[0068] Specifically, when a link viewing command is received, it is necessary to verify the access permissions of the logged-in user corresponding to the link viewing command, obtain the access permissions of the logged-in user, and verify the access permissions, such as... Figure 5 The diagram shown illustrates the verification of login user permissions in one embodiment, which includes the user's login name, login password, and verification code.

[0069] Step 312: When the logged-in user's access permissions are verified, a link viewing instruction is received. The link viewing instruction carries the application identifier of the distributed application.

[0070] Step 314: Obtain the corresponding call chain information from the processing data based on the application identifier, and obtain the information of each service, execution method, execution time and execution status of each method that the distributed application request goes through.

[0071] Optionally, in one embodiment, corresponding processing data is obtained from the search service unit based on the acquired system code. This processing data is aggregated processing data. The system code is used to label different types of processing data, including service-type data (e.g., average throughput, average response time, SLA, etc.), instance-type data (e.g., instance average throughput, instance average response time, JVM information, etc.), and interface-type data (e.g., interface average throughput, average response time, etc.). Querying the corresponding processing data through the system code improves data query efficiency, thereby enhancing the fault location query efficiency of the distributed tracing system.

[0072] The following is a system architecture diagram of a distributed tracing system for end-to-end distributed monitoring in one embodiment, such as... Figure 6 As shown, when a business request is received from the business system, the agent probe collects the corresponding call data. The first load balancing unit asynchronously reports the call data to the OAP Server in the backend service node pointed to by the pre-configured domain name. The OAP Server receives the call data and aggregates the data in the message queue asynchronously to obtain the corresponding processed data, which is then stored in Elasticsearch. The second load balancing unit visualizes the processed data to obtain the operating information of the distributed system.

[0073] The system acquires the login user's access permissions, and the permissions microservice unit verifies these permissions. When the login user's permissions are verified, the system periodically synchronizes the processed data from Elasticsearch to a relay database according to received scheduled data processing instructions. After saving the processed data (i.e., monitoring data) from the search service unit to a Kafka database, the data is synchronized to a relay Flink database. In the relay database, the processed data undergoes streaming processing to obtain the corresponding streaming data (i.e., real-time data) for each distributed application call request. This streaming data is then stored in a Hive database. Based on the digital operations platform, the data in the Hive database undergoes secondary classification, aggregation, and display. When service metric data exceeds the metric threshold, an alarm message is generated through the alarm microservice unit. The alarm information is then sent to the user's terminal using a preset sending method and displayed.

[0074] In this system architecture diagram, by adding a first load balancing unit and a second load balancing unit, data collection and reporting and data visualization management are separated to ensure high availability of the server, alleviate the pressure on the backend service node OAP Server, and improve the efficiency of data processing. By visualizing the aggregated data through a web visualization platform and classifying it through a digital operation platform, system faults can be accurately located and fault query efficiency can be improved.

[0075] In the aforementioned end-to-end distributed monitoring method, the agent probe collects call data of business request applications in the distributed system, sends the call data to the server, and the first load balancing unit asynchronously reports and stores the interface response time, request count, call chain information, and virtual machine information in the message queue corresponding to the backend service node pointed to by the pre-configured domain name. The data in the message queue is aggregated to obtain the corresponding processed data; the processed data is then stored in the search service unit; and the second load balancing unit visualizes the processed data to obtain the operating information of the distributed system.

[0076] The system acquires the access permissions of logged-in users and verifies these permissions. Upon successful verification, it receives a call chain viewing command carrying the application identifier of the distributed application. Based on this command, it retrieves the corresponding call chain information from the processed data, obtaining information about each service, execution method, execution time, and execution status of the distributed application request. By adding a first and second load balancing unit, data collection and reporting are separated from data visualization management, ensuring high server availability, alleviating the pressure on the backend service node OAP Server, and improving data processing efficiency. Furthermore, by visualizing the aggregated data through a web visualization platform and categorizing it through a digital operation platform, system faults can be accurately located, and fault query efficiency can be improved.

[0077] It should be understood that although Figure 2-3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-3 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0078] In one embodiment, such as Figure 7 As shown, a full-link distributed monitoring device is provided, including: an acquisition module 702, a data processing module 704, and a display module 706, wherein:

[0079] The acquisition module 702 is used to acquire call data for each application in the distributed system.

[0080] The data processing module 704 is used to asynchronously report the call data to the backend service node pointed to by the pre-configured domain name through the first load balancing unit, and obtain the processed data generated by the backend service node after aggregating the call data.

[0081] Display module 706 is used to visualize the processed data through the second load unit to obtain the operating information of the distributed system.

[0082] In the aforementioned end-to-end distributed monitoring method, call data corresponding to each application in the distributed system is acquired. This data is then collected by a collection terminal and asynchronously reported to the backend service node pointed to by a pre-configured domain name via a first load balancing unit. The backend service node then aggregates and processes the call data, generating processed data. By pre-configuring the domain name, data reporting efficiency is not affected by node changes. The processed data is then visualized by a second load balancing unit to obtain the operating information of the distributed system. By adding a first and second load balancing unit to the distributed tracing system, data collection and reporting are separated from visualization, ensuring high server availability and data processing performance, thereby improving the efficiency of fault location and querying in the distributed tracing system.

[0083] In another embodiment, such as Figure 8 As shown, a full-link distributed monitoring device is provided, which, in addition to the acquisition module 702, data processing module 704, and display module 706, also includes: a receiving module 708, a synchronization module 710, a receiving module 712, a verification module 714, and an alarm module 716, wherein:

[0084] The data processing module 704 is also used to report and store the interface response time, number of requests, call chain information and virtual machine information asynchronously through the first load balancing unit in the message queue corresponding to the backend service node of the distributed tracing system; to aggregate the data in the message queue to obtain the corresponding processed data; the processed data includes inter-application relationship and inter-service relationship as well as service indicator data.

[0085] The receiving module 708 is used to receive timed data processing instructions.

[0086] The synchronization module 710 is used to synchronize the processed data in the search service unit to the transit database according to the timed data processing instructions.

[0087] The data processing module 704 is also used to perform streaming processing on the processed data through the transit database to obtain the streaming data required by the digital operation platform.

[0088] The receiving module 712 is used to receive the link viewing command, which carries the application identifier of the distributed application.

[0089] The acquisition module 702 is also used to obtain the corresponding call chain information from the processing data based on the application identifier, and to obtain the information of each service, execution method, execution time and execution status of each method that the distributed application request goes through.

[0090] The verification module 714 is used to obtain the access permissions of the logged-in user and verify the access permissions; when the access permissions of the logged-in user are verified, the step of obtaining the corresponding call chain information from the processing data according to the application identifier is executed.

[0091] The alarm module 716 is used to generate an alarm message when the detected service indicator data is greater than the indicator threshold; and to send the alarm information to the user's terminal in a preset sending method.

[0092] In one embodiment, the system obtains call data from each application in the distributed system; a first load balancing unit asynchronously reports and stores the interface response time, request count, call chain information, and virtual machine information in a message queue corresponding to the backend service node pointed to by a pre-configured domain name; aggregates the data in the message queue to obtain corresponding processed data; stores the processed data in a search service unit; a second load balancing unit visualizes the processed data to obtain the operating information of the distributed system; obtains the access permissions of the logged-in user, verifies the access permissions, and when the access permissions of the logged-in user are verified, receives a link viewing instruction carrying the application identifier of the distributed application; and obtains the corresponding call chain information from the processed data based on the application identifier to obtain the service, execution method, execution time of each method, and execution status information traversed by the distributed application request.

[0093] By adding a first load balancing unit and a second load balancing unit, data collection and reporting and data visualization management are separated to ensure high availability of the server, alleviate the pressure on the backend service node server OAP Server, and improve the efficiency of data processing. By visualizing the aggregated data through a web visualization platform and classifying it through a digital operation platform, system faults can be accurately located and fault query efficiency can be improved.

[0094] Specific limitations regarding the end-to-end distributed monitoring device can be found in the limitations of the end-to-end distributed monitoring method described above, and will not be repeated here. Each module in the aforementioned end-to-end distributed monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0095] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores processed data obtained by aggregating the call data corresponding to each distributed application call request. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a full-link distributed monitoring method.

[0096] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0097] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0098] Obtain call data for each application in the distributed system;

[0099] The first load balancing unit asynchronously reports the call data to the backend service node pointed to by the pre-configured domain name, obtains the processed data generated by the backend service node after aggregating the call data, and stores the processed data in the search service unit;

[0100] The second load balancing unit visualizes the processed data, thus obtaining the operating information of the distributed system.

[0101] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0102] The first load balancing unit asynchronously reports and stores the interface response time, number of requests, call chain information, and virtual machine information in the message queue corresponding to the backend service node pointed to by the pre-configured domain name.

[0103] The data in the message queue is aggregated to obtain the corresponding processed data; the processed data includes inter-application relationships, inter-service relationships, and service metric data.

[0104] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0105] Receive timed data processing instructions;

[0106] Synchronize the processed data in the search service unit to the transit database according to the scheduled data processing instructions;

[0107] The data is processed in a streaming manner through a transit database to obtain the streaming data required by the digital operation platform.

[0108] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0109] Receive a link viewing command, which carries the application identifier of the distributed application.

[0110] Based on the application identifier, the corresponding call chain information is obtained from the processing data, which yields information about each service, execution method, execution time, and execution status that the distributed application request goes through.

[0111] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0112] Obtain the access permissions of the logged-in user and verify those permissions;

[0113] When the logged-in user's access permissions are verified, the step of retrieving the corresponding call chain information from the processing data based on the application identifier is executed.

[0114] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0115] When a service metric is detected to be greater than the metric threshold, an alarm message is generated.

[0116] The alarm information will be sent to the user's terminal using a preset sending method.

[0117] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0118] Obtain the call data of each application in the distributed system; report the call data to the backend service node pointed to by the pre-configured domain name in an asynchronous manner through the first load balancing unit, obtain the processing data generated by the backend service node aggregating the call data, and store the processing data in the search service unit;

[0119] The second load balancing unit visualizes the processed data, thus obtaining the operating information of the distributed system.

[0120] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0121] The first load balancing unit asynchronously reports and stores the interface response time, number of requests, call chain information, and virtual machine information in the message queue corresponding to the backend service node pointed to by the pre-configured domain name.

[0122] The data in the message queue is aggregated to obtain the corresponding processed data; the processed data includes inter-application relationships, inter-service relationships, and service metric data.

[0123] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0124] Receive timed data processing instructions;

[0125] Synchronize the processed data in the search service unit to the transit database according to the scheduled data processing instructions;

[0126] The data is processed in a streaming manner through a transit database to obtain the streaming data required by the digital operation platform.

[0127] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0128] Receive a link viewing command, which carries the application identifier of the distributed application.

[0129] Based on the application identifier, the corresponding call chain information is obtained from the processing data, which yields information about each service, execution method, execution time, and execution status that the distributed application request goes through.

[0130] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0131] Obtain the access permissions of the logged-in user and verify those permissions;

[0132] When the logged-in user's access permissions are verified, the step of retrieving the corresponding call chain information from the processing data based on the application identifier is executed.

[0133] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0134] When a service metric is detected to be greater than the metric threshold, an alarm message is generated.

[0135] The alarm information will be sent to the user's terminal using a preset sending method.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A full-link distributed monitoring method, characterized in that, The method includes: The system acquires call data from each application in the distributed system collected by the acquisition terminal; the acquisition terminal is pre-configured with domain names pointing to backend service nodes. The first load balancing unit asynchronously reports the call data to the pre-configured backend service node pointed to by the domain name, obtains the processed data generated by the backend service node through aggregation of the call data, and stores the processed data in the search service unit; the processed data includes inter-application relationships and inter-service relationships; The processed data is visualized by the second load balancing unit to obtain the operating information of the distributed system; the first load balancing unit and the second load balancing unit are used to separate data reporting and data visualization.

2. The method according to claim 1, characterized in that, The call data includes interface response time, number of requests, call chain information, and virtual machine information. The first load balancing unit asynchronously reports the call data to the backend service node pointed to by the pre-configured domain name, obtaining processed data generated by the backend service node through aggregation of the call data, including: The first load balancing unit asynchronously reports and stores the interface response time, number of requests, call chain information, and virtual machine information in the message queue corresponding to the backend service node pointed to by the pre-configured domain name. The data in the message queue is aggregated to obtain corresponding processed data; the processed data also includes service metric data.

3. The method according to claim 1, characterized in that, The method further includes: Receive timed data processing instructions; The processed data in the search service unit is synchronized to the transit database according to the timed data processing instruction; The data is stream-processed through the transit database to obtain the streaming data required by the digital operation platform.

4. The method according to claim 2, characterized in that, The method further includes: Receive a link viewing instruction, the link viewing instruction carrying the application identifier of the application; Based on the application identifier, the corresponding call chain information is obtained from the processing data to obtain information on each service, execution method, execution time, and execution status that the distributed application request goes through.

5. The method according to claim 4, characterized in that, Before retrieving the corresponding call chain information from the processed data based on the application identifier, the method further includes: Obtain the usage permissions of the logged-in user and verify the usage permissions; When the logged-in user's access permissions are verified, the step of obtaining the corresponding call chain information from the processing data based on the application identifier is executed.

6. The method according to claim 2, characterized in that, The method further includes: When a service metric is detected to be greater than the metric threshold, an alarm message is generated. The alarm message is sent to the user's terminal using a preset sending method.

7. A full-link distributed monitoring device, characterized in that, The device includes: The acquisition module is used to acquire call data corresponding to each application in the distributed system collected by the acquisition terminal; the call data is collected by the acquisition terminal based on a pre-configured domain name; the acquisition terminal is pre-configured with a domain name pointing to the backend service node; The data processing module is used to asynchronously report the call data to the backend service node pointed to by the pre-configured domain name through the first load balancing unit, and obtain the processed data generated by the backend service node through aggregation processing of the call data; the processed data includes inter-application relationships and inter-service relationships; The display module is used to visualize the processed data through the second load balancing unit to obtain the operating information of the distributed system; the first load balancing unit and the second load balancing unit are used to separate data reporting and data visualization.

8. The apparatus according to claim 7, characterized in that, The call data includes interface response time, number of requests, call chain information, and virtual machine information. The device also includes: The data processing module is also used to report the interface response time, number of requests, call chain information and virtual machine information asynchronously through the first load balancing unit and store them in the message queue corresponding to the backend service node pointed to by the pre-configured domain name; The data in the message queue is aggregated to obtain corresponding processed data; the processed data also includes service metric data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images