Effectiveness monitoring method and system based on full stack service

Abstract

The invention discloses a full stack service-based validity monitoring method. The method comprises the following steps of: analyzing a topology of a service calling link based on a service; creating a monitoring task which is arranged in a multi-language mixed manner and covers a plurality of key nodes based on the topology of the service calling link; concurrently executing a node task at each key node in the plurality of key nodes and collecting results based on the created monitoring task, and marking the results according to node numbers and grouping the results into a success / failure state; positioning a fault root cause based on the topology depth and the link relationship of the key node; and sending the alarm information including the fault root cause to a specified terminal.

Description

Technical Field

[0001] The present application relates to the field of distributed system monitoring technology, and in particular to a full-stack service-based effectiveness monitoring method and system, which is suitable for system service effectiveness monitoring and fault warning in clustered deployment, microservice architecture and distributed database environments. Background Art

[0002] With the accelerated advancement of enterprise informatization and digitalization, modern information systems are adopting complex technical features such as clustering, microservices architecture, and distributed databases. As technical architectures shift from monolithic applications to microservices, containerization, and hybrid cloud environments, traditional manual monitoring methods are no longer able to cope with increasingly complex system environments, and existing automated inspection and dialing monitoring methods have also exposed significant shortcomings.

[0003] In operational practice, the isolation of heterogeneous systems is becoming increasingly problematic. Java services and Node.js applications use incompatible log formats, Kubernetes cluster metrics and physical server performance data are scattered across different platforms, and real-time data streams from IoT devices are difficult to correlate with cloud database anomaly alerts. This fragmented technology stack creates monitoring blind spots, a significant risk in digital transformation. Specifically, single-type inspection tasks can only identify single points of failure, making it difficult to pinpoint the root cause. Existing platforms fail to orchestrate tasks based on call chains, failing to cover key nodes in the business topology. Furthermore, they lack full-stack service monitoring, data validation mechanisms, and closed-loop alarm processing capabilities.

[0004] To solve these problems, we must overcome the barriers of technological fragmentation. Full-Stack Observability is a key solution to this challenge. Its core value lies in building a unified observation platform in complex heterogeneous environments through cross-dimensional data fusion and contextual correlation. This allows for precise root cause identification of distributed system failures, significantly reducing false alarm rates, and ultimately meeting the intelligent operations and maintenance needs of digital transformation. Summary of the Invention

[0005] This application aims to solve the problems in the prior art of difficulty in locating the root cause of faults and high false alarm rate caused by the lack of deep correlation analysis of business call links, and to provide a full-stack service effectiveness monitoring system and method based on business to parse the topological relationship of service call links, so as to achieve accurate fault diagnosis and efficient system operation and maintenance. A very necessary function in the field of intelligent operation and maintenance (AIOps) is root cause location / analysis. This function aims to use algorithms to increase the interpretability of service level agreements, so that operation and maintenance personnel can quickly understand the root cause of the problem when anomalies / failures occur, achieve rapid loss control, and thus reduce the scope of the impact of the failure.

[0006] According to one aspect of the present application, a method for monitoring the effectiveness of a full-stack service is provided, comprising: parsing the topology of a service call link based on the business; creating a multi-language mixed-orchestration monitoring task covering multiple key nodes based on the topology of the service call link; concurrently executing node tasks at each of the multiple key nodes based on the created monitoring task and collecting results, and marking the results according to the node number and grouping them into success / failure status; locating the root cause of the fault based on the topological depth and link relationship of the key node; and sending an alarm message including the root cause of the fault to a designated terminal.

[0007] According to another aspect of the present application, a full-stack service-based effectiveness monitoring system is provided, including: a construction module, configured to parse the topology of a service call link based on the business; a task orchestration module, configured to create a multi-language mixed-orchestration monitoring task covering multiple key nodes based on the topology of the service call link; a concurrent execution module, configured to concurrently execute the node task at each of the multiple key nodes based on the created monitoring task and collect the results, and mark the results according to the node number and group them into success / failure status; a fault analysis module, configured to locate the root cause of the fault based on the status of the key node and the topology depth; and a closed-loop processing module, configured to send an alarm message including the root cause of the fault to a designated terminal.

[0008] This application proposes to build a dynamic topology association model of service components based on the actual call link topology relationship of the business, establish a multi-task rule and multi-language hybrid orchestration mechanism, and achieve effective and timely monitoring service capabilities without deploying probes. Breaking through the bottleneck of single-dimensional detection, building a multi-dimensional composite monitoring mechanism, based on the multi-dimensional call rules of the call link topology structure, realize the monitoring capability of multi-task cross-level and cross-component rule orchestration, thereby effectively improving the accuracy of fault diagnosis and system operation and maintenance efficiency, and reducing business risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a service system call link topology diagram according to an embodiment of the present application;

[0010] Figure 2 is a system architecture diagram according to an embodiment of the present application;

[0011] Figure 3 is a flowchart of a method for monitoring effectiveness based on a full-stack service according to an embodiment of the present application;

[0012] Figure 4 is a full-stack effectiveness monitoring management system and method scheduling flow chart according to one embodiment of the present application; and

[0013] Figure 5 This is an example diagram of arrangement grouping according to an embodiment of the present application. Specific implementation plan

[0014] Figure 1 This is a business system call link topology diagram according to an embodiment of the present application, illustrating the call link topology structure of the financial insurance business system in one embodiment of the present application. This link topology includes key service nodes such as: ① Nginx, ② front-end web page, ③ back-end service, ④ MySQL, ⑤ Redis, etc. Each node corresponds to a monitoring detection task on the monitoring platform.

[0015] Figure 1 The topology diagram shown uses a layered architecture to clearly illustrate the call relationships and dependency levels of key service nodes in the distributed system. From a topology perspective, the system exhibits a typical five-layer service call stack: Access Layer (L1): ① Nginx nodes form the topology entry point, responsible for load balancing and traffic distribution. Monitoring and probing tasks focus on metrics such as connection count, response latency, and HTTP status codes. Presentation Layer (L2): ② Front-end page nodes, serving as a secondary topology layer, establish bidirectional communication with Nginx (solid arrows represent HTTP request / response interactions). Monitoring at this layer should cover page rendering performance, API call success rate, and front-end error logs. Application Layer (L3): ③ Back-end service nodes form the core hub of the topology, receiving front-end requests (downward arrows) and initiating calls to the downstream data layer (upward arrows). Probing tasks at this layer should correlate JVM performance, gRPC call chains between microservices, and circuit breaker status. The data storage layer (L4-L5) is divided into a structured data layer (④MySQL) and a cache layer (⑤Redis). These two layers form a star topology with the backend services. MySQL node monitoring includes query latency, connection pool status, and master-slave synchronization latency; Redis node probing focuses on cache hit rate, memory fragmentation rate, and cluster status. The topology diagram illustrates typical fault propagation paths, such as a Redis exception → backend service timeout → a 503 error on the frontend page. The dotted boxes indicate the topology subnets deployed across different zones, reflecting the need for network hop monitoring in a hybrid cloud environment. The thickness of the arrows between nodes intuitively reflects traffic weight, facilitating identification of critical paths.

[0016] This topology achieves full-stack coverage from edge nodes to data storage through a strong binding relationship between "monitoring detection tasks and service nodes." This allows for bidirectional impact analysis of any node anomalies along the topology link, such as bottom-up tracing of resource bottlenecks or top-down business impact assessment.

[0017] Figure 2It is a system architecture diagram according to an embodiment of the present application. A full-stack service-based effectiveness monitoring system 200. The monitoring system 200 may include a construction module 201, which is configured to parse the topology of the service call link based on the business. In one embodiment, the construction module 201 may include a topology parsing unit 207, which obtains the service call link through service grid traffic tracking (such as Jaeger) or configuration file parsing, and generates, for example, a directed acyclic graph (DAG) model. In another embodiment, the monitoring system 200 may include a dynamic update unit 208, which is used to adjust the topology node association relationship in real time in order to monitor Kubernetes events or service registry changes. In another embodiment, the monitoring system 200 may include a multi-language engine 211, which can support syntax parsing and parameter binding of Python, SQL, and Java tasks, such as the SQL task template {"sql":"SELECT*FROM{table}","params":{"table":"users"}}. In another embodiment, the monitoring system 200 may include a link orchestrator 212 for automatically assigning node tasks according to the topology level to ensure coverage of all critical paths (e.g., front-end → back-end → database). In another embodiment, the monitoring system 200 may include a thread pool manager 213 that dynamically adjusts the thread pool size based on the number of nodes to avoid resource contention and controls concurrency, for example, using Semaphore to prevent system overload.

[0018] The monitoring system 200 may include a concurrent execution module 203 configured to concurrently execute node tasks at each of the multiple key nodes based on the created monitoring tasks and collect results, and mark the results according to the node numbers and group them into success / failure states.

[0019] Monitoring system 200 may include a fault analysis module 204 configured to locate the root cause of a fault based on the status of the key nodes and the topological depth. Fault analysis module 204 may include a rule base 214 with built-in multi-level fault determination rules (e.g., an IF topological depth > 3THEN priority database) and support for custom rule extensions. Fault analysis module 204 may also include a root cause inference engine 215, which determines the last failed node through a depth-first search (DFS) based on the topological path of the failed node.

[0020] Monitoring system 200 may include a closed-loop processing module 205 configured to send an alarm message, including the root cause of the fault, to a designated terminal. In one embodiment, closed-loop processing module 205 may include an alarm template library 216, which predefines enterprise WeChat card templates and email body templates containing details of the fault node, the scope of impact, and recommended actions. In another embodiment, closed-loop processing module 205 may include an event tracker 217, which records the alarm processing status (notified, in progress, resolved), forming a closed-loop management log.

[0021] Figure 3 This is a flow chart of a method for monitoring the effectiveness of a full-stack service according to an embodiment of the present application. Figure 3 As shown, the effectiveness monitoring method based on full-stack services includes the following steps. In box 301, the topology of the service call link is parsed based on the business. In box 302, a monitoring task covering multiple key nodes with multi-language hybrid arrangement is created based on the topology of the service call link. In box 303, based on the created monitoring task, the node tasks at each of the multiple key nodes are concurrently executed and the results are collected, and in box 304, the results are marked according to the node number and grouped into success / failure status. In box 305, it is determined whether there is a failure state. If not, the process ends. If so, in box 306, the root cause of the fault is located based on the topological depth and link relationship of the key node. Finally, in box 307, the alarm information including the root cause of the fault is sent to the designated terminal.

[0022] Figure 4 This is a full-stack effectiveness monitoring management system and method scheduling flowchart according to an embodiment of the present application. In one embodiment, multi-language task orchestration is utilized. Create orchestration tasks based on business scenarios, and create multiple nodes in the tasks for executing different types of tasks. For different types of nodes, corresponding monitoring and detection tasks are created, such as creating a network connectivity monitoring task for Nginx nodes, creating a page response time monitoring task for front-end pages, creating an interface call monitoring task for back-end services, creating a database query performance monitoring task for Mysql databases, etc. At the same time, a multi-language mixed orchestration mechanism is adopted to support task types such as Python, SQL, and Java, and asynchronous scheduling is achieved using a strategy design pattern. The tasks (job tasks) of each node are executed asynchronously, and the strategy design pattern is used to execute different types of tasks such as SQL tasks, URL tasks, and Python tasks. The current asynchronous thread is blocked and waited, and after the tasks of all nodes are executed, the execution results of all tasks are collected.

[0023] Execute node tasks concurrently and collect results. Asynchronously execute each node's job task, using multi-threaded concurrent technology to improve monitoring efficiency. Block the current asynchronous thread until all nodes have completed their tasks, then collect all task results. For example, the following pseudo code implements multi-threaded task concurrent scheduling:

[0024] (pseudocode)

[0025] / / Multi-threaded task concurrent scheduling private List <ruletaskresultdo>

[0026] / / Asynchronously execute URL task CompletableFuture <List <ruletaskresultdo>

[0027] / / Asynchronously execute Python task CompletableFuture <List <ruletaskresultdo>

[0028] / / Asynchronously execute SQL task CompletableFuture <List <ruletaskresultdo>

[0029] / / Wait for all tasks to complete CompletableFuture <void>

[0030] / / Block and wait for all tasks to complete allOf.get();

[0031] / / Aggregate the results of all tasks... *}.

[0032] Figure 5 This is an example diagram of grouping according to an embodiment of the present application. Figure 4 like Figure 5 As shown, the basic process of providing a full-stack service and data validity monitoring system and method based on the mainstream Internet operation and maintenance architecture of this application is as follows:

[0033] 1. Create a task: Create an orchestration task based on the business scenario. Create 8 nodes in the task to execute different types of tasks.

[0034] 2. Asynchronous execution: asynchronously execute the job tasks of each node and use the strategy design pattern to execute various types of tasks;

[0035] 3. Collect results: Block the current asynchronous thread and wait for all nodes to complete the task, then collect all task results;

[0036] 4. Grouping: Group the nodes by their numbers (e.g. Figure 5 ), the purpose of grouping is to clarify the execution results of the subtasks under each group. The grouped results are as follows: (prefix 4->4:1, prefix 3->3.1:1,3.2:-1, prefix 2->2.1:1,2.2:1, prefix 1->1:-1), where 1 represents successful execution and -1 represents task failure. For example, prefix: 1->1:-1 means that the first task with prefix 1 failed.

[0037] 5. Store failure information: Find the data with execution result -1 in the sub-service from the result obtained in step 4 and store it in the failure map <String,List <objet>>, where the key of the Map is the prefix number and the value is the corresponding subtask list. For example, the result of step 5 is: (prefix: 3->3.2:-1, prefix: 1->1:-1), and all failed results are collected;

[0038] 6. Parsing failure information: traversing the Map <String,List <object>>, take out -1 and put it into the array, take out the key and put it into the array, such as: [3,1];

[0039] 7. Summarize the parsing failure information: Get the array information [3,1] obtained above from Map <String,List <object>>Get the corresponding List <object>After summarizing the information, enterprise WeChat and email notifications are pushed according to the defined template.

[0040] As mentioned above, allOf.get() is used to block and wait for all tasks to complete, and the results are summarized in List <result>A collection where each result contains the node number, task type, and execution status (1 for success, -1 for failure).

[0041] Group by node number prefix, for example, number 3.2 belongs to prefix group number 3. Example after grouping:

[0042] Prefix group number 4 → 4:1 (success)

[0043] Prefix group number 3 → 3.1: 1 (success), 3.2: -1 (failure)

[0044] Prefix group number 1 → 1: -1 (failed)

[0045] Store the task with status -1 into the Map <String,List <task>>, where the key is the prefix group number and the value is a list of failed tasks. For example: {3:[3.2 tasks], 1:[1 tasks]}.

[0046] Traverse the Map to extract the failure prefix group number (such as [3,1]), generate alarm information according to the template (including the fault node path and failure type), and push it to the designated terminal through the enterprise WeChat API and email SMTP protocol.

[0047] In one embodiment of the present application below, reference is made to Figure 1 This section describes the fault location logic. Taking the financial insurance system topology as an example, the root cause location rules are as follows:

[0048] By breaking down the topology call chain, key service nodes are divided into: ① Nginx, ② front-end pages, ③ back-end services, ④ MySQL, ⑤ Redis, etc. Each node corresponds to a monitoring detection task on the monitoring platform. If a service node fails, how to determine the type of failure through full-stack service traversal is as follows:

[0049] 1. When node ① fails and the remaining nodes are normal, the Nginx service failure at node ① is determined to be the root cause.

[0050] 2. If node ① fails, instance 1 in node ② fails, instance 2 is normal, node ③ (instance 1 and instance 2) are both normal, and nodes ④ and ⑤ are both normal, then the failure of node ②'s front-end service instance 1 is determined to be the root cause.

[0051] 3. If node ① fails, instance 1 and instance 2 in node ② fail, instance 1 in node ③ fails, instance 2 is normal, and nodes ④ and ⑤ are both normal, then the root cause is determined to be the backend service instance 1 in node ③.

[0052] 4. When node ① Nginx fails, node ② (instance 1, instance 2) fails, node ③ (instance 1, instance 2) fails, and node ④ fails, the root cause is ultimately determined to be a database failure on node ④.

[0053] 5. When Nginx node ① fails, node ② (instance 1 and instance 2) is normal, node ③ (instance 1 fails, instance 2 is normal), node ④ is normal, and node ⑤ is normal. The root cause is ultimately determined to be the failure of instance 1 on node ① and node ③.

[0054] 6. If all nodes are normal, the system call link is normal;

[0055] 7. If all nodes fail, the root cause must be a database failure.

[0056] As mentioned above, when only one node in the call link fails, an alarm is pushed to report that the current node has an abnormality; if multiple nodes fail, the last node is taken as the root cause of the failure according to the node depth and link relationship of the service topology layer and an alarm is pushed.

[0057] The previously described versions of the disclosed subject matter have many advantages that are described or apparent to those skilled in the art. Even so, these advantages or features are not necessarily present in all versions of the disclosed apparatus, system, or method.

[0058] All features disclosed in the specification, including the claims, abstract and drawings, and all steps in any disclosed method or process, may be combined in any combination, except combinations in which at least some of such features and / or steps are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in the specification, including the claims, abstract and drawings, may be replaced by an alternative feature serving the same, equivalent or similar purpose.

[0059] Furthermore, this written description may also mention specific features. It should be understood that the disclosure in this specification includes all possible combinations of these specific features. For example, where a specific feature is disclosed in the context of a particular aspect, that feature may also be used in the context of other aspects to the extent possible.

[0060] Furthermore, when this application refers to a method having two or more defined steps or operations, the defined steps or operations may be performed in any order or simultaneously, unless the context excludes such possibilities.

[0061] Although specific aspects of the present disclosure have been shown and described for illustrative purposes, it should be understood that various modifications may be made without departing from the spirit and scope of the present invention. Therefore, the present invention should not be limited except to the extent specified in the appended claims. With the benefit of the teachings presented in the foregoing description and the associated drawings, those skilled in the art will appreciate that many modifications and other embodiments of the present disclosure described herein will be apparent to those skilled in the art. Therefore, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Furthermore, although the foregoing description and the associated drawings describe example embodiments in the context of certain example combinations of elements and / or functions, it should be understood that alternative embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, as described in some of the appended claims, combinations of elements and functions that are different from the elements and / or functions explicitly described above may also be considered. Although specific terms are used herein, they are used only in a general and descriptive sense, and not for purposes of limitation.< / task> < / result> < / object> < / object> < / object> < / objet> < / void> < / ruletaskresultdo> < / ruletaskresultdo> < / ruletaskresultdo> < / ruletaskresultdo>

Claims

1. A full-stack service-based effectiveness monitoring method, comprising: Analyze the topology of service call links based on business; Creating a multi-language mixed-choreography monitoring task covering multiple key nodes based on the topology of the service call link; Based on the created monitoring task, concurrently execute the node task at each of the plurality of key nodes and collect results, and mark the results according to the node number and group them into success / failure status; Locating the root cause of the fault based on the topological depth and link relationship of the key nodes; as well as Send alarm information including the root cause of the fault to the designated terminal.

2. Based on the effectiveness monitoring method described in claim 1, it also includes obtaining the service call link through service network traffic tracking or configuration file parsing.

3. The effectiveness monitoring method according to claim 1, wherein the multiple nodes include at least one of an Nginx node, a front-end service node, a back-end service node, and a database node.

4. Based on the effectiveness monitoring method described in claim 3, the topological depth of the topology of the service call link increases layer by layer from the Nginx node to the database node.

5. The effectiveness monitoring method according to claim 1, wherein the multi-language mixed orchestration supports Python, SQL, and Java task types, and adopts a strategic design pattern to implement asynchronous scheduling.

6. Based on the effectiveness monitoring method described in claim 1, locating the root cause of the fault based on the topological depth and link relationship of the key nodes includes directly issuing an alarm when a single node fails, and taking the last failed node as the root cause when multiple nodes fail.

7. A full-stack service-based effectiveness monitoring system, comprising: A building module is configured to resolve the topology of the service call link based on the business; A task orchestration module is configured to create a multi-language mixed orchestration monitoring task covering multiple key nodes based on the topology of the service call link; a concurrent execution module configured to concurrently execute the node task at each of the plurality of key nodes based on the created monitoring task and collect results, and mark the results according to the node number and group them into success / failure status; a fault analysis module configured to locate the root cause of the fault based on the status of the key nodes and the topology depth; as well as The closed-loop processing module is configured to send alarm information including the root cause of the fault to a designated terminal.

8. The effectiveness monitoring system according to claim 7 further comprises a topology parsing unit configured to obtain the service call link through service network traffic tracing or configuration file parsing.

9. The effectiveness monitoring system according to claim 7, wherein the plurality of nodes include at least one of an Nginx node, a front-end service node, a back-end service node, and a database node.

10. The effectiveness monitoring system according to claim 9, wherein the topological depth of the topology of the service call link increases layer by layer from the Nginx node to the database node.

11. The effectiveness monitoring system according to claim 7, wherein the multi-language hybrid orchestration supports Python, SQL, and Java task types, and adopts a strategic design pattern to implement asynchronous scheduling.

12. Based on the effectiveness monitoring system described in claim 7, the fault analysis module has a built-in rule base, which defines the root cause determination logic in a multi-node failure scenario, wherein the root cause determination logic includes directly issuing an alarm when a single node fails, and taking the last failed node as the root cause when multiple nodes fail.

13. A device comprising a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the device performs the method according to any one of claims 1 to 6.

14. A computer-readable storage medium comprising instructions, wherein the instructions instruct a device to execute the method according to any one of claims 1 to 6.

15. A computer program product, wherein when the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 6.

Images