Monitoring and early warning method and device, electronic equipment, storage medium and computer program product

Abstract

The application provides a monitoring and early warning method and device, electronic equipment, a storage medium and a computer program product, and relates to the technical field of operation and maintenance monitoring. The method comprises the following steps: acquiring alarm data; matching an alarm node corresponding to the alarm data with nodes in a plurality of preset subgraph networks to determine a matching result; wherein the preset subgraph networks are determined based on key nodes in a merged topology network; the merged topology network is determined by combining the resource nodes and the business nodes in the network topology; if the matching result indicates that the alarm node belongs to a target subgraph network in the plurality of preset subgraph networks, the key nodes in the target subgraph network are detected to determine a monitoring and early warning result. As long as an alarm occurs in the target preset subgraph network, the key nodes in the target preset subgraph network are detected, which is equivalent to detecting the key nodes in advance before the key nodes alarm, and the alarm of the key nodes can be discovered in time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of operation and maintenance monitoring, and particularly relates to a monitoring and early warning method and device, electronic equipment, a storage medium and a computer program product. BACKGROUND

[0002] Currently, with the continuous adjustment and improvement of the emergency management system in China, the emergency management type business system is developing more and more richly, the business logic and deployment architecture are becoming more and more complex, and the calling relationship between the business links is relatively close. Once a problem occurs in a key node in the business link, the influence on the entire emergency related business is very disastrous, therefore, it is extremely important to monitor the entire link of the business and to give early warning to the key node. In the related technology, the method for monitoring the entire link of the business is to independently monitor and collect data of each business or each level, and then to centrally count and analyze the correlation of each business or level, so as to assist the development and operation personnel in locating the problem. However, in the actual use process, due to the huge amount of calculation, the problem is often found to be relatively lagging behind. SUMMARY

[0003] The monitoring and early warning method, device, electronic equipment, storage medium and computer program product provided by the embodiments of the present application can timely find the alarm of the key node.

[0004] The technical solution of the present application is implemented as follows:

[0005] The embodiments of the present application provide a monitoring and early warning method, comprising:

[0006] obtaining alarm data;

[0007] matching an alarm node corresponding to the alarm data with nodes in a plurality of preset subgraph networks to determine a matching result, wherein the preset subgraph networks are determined based on key nodes in a merged topology network, and the merged topology network is determined by combining the dependent resources and business nodes in the network topology;

[0008] If the matching result indicates that the alarm node belongs to a target subgraph network in the plurality of preset subgraph networks, detecting the key node in the target subgraph network to determine a monitoring and early warning result.

[0009] In the above solution, before the matching of the alarm node corresponding to the alarm data with the nodes in the plurality of preset subgraph networks to determine the matching result, the method further comprises:

[0010] determine the merged topology network based on the dependencies among the resource nodes and the service nodes in the network topology; wherein the merged topology network is used to represent the dependency relationships among the resource nodes, among the service nodes, and between the resource nodes and the service nodes in the network topology;

[0011] traverse each node in the merged topology network, and determine the key nodes in the merged topology network that satisfy a predetermined condition;

[0012] extract the preset subgraph network including the key nodes from the merged topology network.

[0013] In the above scheme, the merged topology network is determined based on the dependencies among the resource nodes and the service nodes in the network topology, comprising:

[0014] construct a resource topology network based on the obtained resource dependency related data in the network topology, and construct a service topology network based on the obtained service call chain data corresponding to the network topology; wherein the resource topology network is used to represent the dependency relationships among the resource nodes, and the service topology network is used to represent the dependency relationships among the service nodes;

[0015] merge the resource topology network and the service topology network to determine the merged topology network based on the dependencies between the resource nodes and the service nodes.

[0016] In the above scheme, the resource topology network is constructed based on the obtained resource dependency related data in the network topology, and the service topology network is constructed based on the obtained service call chain data corresponding to the network topology, comprising:

[0017] obtain the resource dependency related data; wherein the resource dependency related data includes a plurality of resource nodes, and the node type, node name, node attribute, and dependency relationship corresponding to each resource node;

[0018] form a corresponding first network node based on the node type, node name, and node attribute corresponding to each resource node, connect each first network node with a corresponding first dependency node to form the resource topology network; wherein the first dependency node is formed based on a dependent resource node determined based on the dependency relationship;

[0019] obtain the service call chain data; wherein the service call chain data includes a plurality of call service data, and the call service data includes a call service method name, a service node identifier, a parent service node identifier, and belonging resource node information;

[0020] Corresponding second network nodes are formed based on the calling business method name, the business node identifier, the parent business node identifier and the belonging resource node information in each of the calling business data, and each of the second network nodes is connected with a corresponding second dependent node to form the business topology network, wherein the second dependent node is determined based on matching the parent business node identifier with the business node identifier in the plurality of calling business data.

[0021] In the above scheme, the resource topology network and the business topology network are merged to determine the merged topology network based on the dependency between the resource nodes and the business nodes, including:

[0022] A dependent first network node including a dependent node name corresponding to each of the second network nodes is determined in the resource topology network, wherein the dependent node name is determined based on the belonging resource node information;

[0023] Each of the second network nodes is connected with the dependent first network node to form the merged topology network.

[0024] In the above scheme, the key nodes in the merged topology network that meet a predetermined condition are determined by traversing each node in the merged topology network, including:

[0025] The number of shortest paths between each pair of nodes in the merged topology network is determined;

[0026] The number of first paths passing through a first node between a first predetermined pair of nodes is determined, and the betweenness centrality corresponding to the first node is determined based on each of the first path number and the corresponding shortest path number, until the betweenness centrality corresponding to an Nth node is determined; wherein the first predetermined pair of nodes is each pair of nodes in the merged topology network that does not include the first node; the merged topology network includes N nodes; N is an integer greater than 1;

[0027] The key nodes corresponding to the betweenness centrality greater than a predetermined threshold are determined among the N nodes.

[0028] In the above scheme, the betweenness centrality corresponding to the first node is determined based on each of the first path number and the corresponding shortest path number, including:

[0029] Each of the first path number is divided by the shortest path number of the corresponding first predetermined pair of nodes to obtain a first ratio value corresponding to each of the first predetermined pair of nodes;

[0030] Each of the first ratio values is weighted and summed to determine the betweenness centrality corresponding to the first node.

[0031] In the above solution, the extracting the preset subgraph network including the key node in the merged topology network comprises:

[0032] Taking each key node as a center node and taking a preset hop number as a radius of the preset subgraph network, the preset subgraph network is extracted in the merged topology network.

[0033] Embodiments of the present application also provide a monitoring and early warning device, comprising:

[0034] a data acquisition unit configured to acquire alarm data;

[0035] a matching unit configured to match an alarm node corresponding to the alarm data with nodes in a plurality of preset subgraph networks and determine a matching result, wherein the preset subgraph networks are determined based on key nodes in a merged topology network, and the merged topology network is determined based on dependencies between resource nodes and service nodes in a network topology;

[0036] a detection unit configured to, if the matching result indicates that the alarm node belongs to a target subgraph network in the plurality of preset subgraph networks, detect the key nodes in the target subgraph network and determine a monitoring and early warning result.

[0037] Embodiments of the present application also provide an electronic device comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor implements steps in the above method when executing the computer program.

[0038] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement steps in the above method.

[0039] Embodiments of the present application also provide a computer program product comprising a computer program, and the computer program is executed by a processor to implement steps in the above method.

[0040] In the embodiment of the present application, the alarm data is obtained, the alarm node corresponding to the alarm data is matched with the nodes in the plurality of preset subgraph networks, and the matching result is determined. The preset subgraph network is determined based on the key nodes in the merged topology network. The merged topology network is determined by combining the dependencies of the resource nodes and the service nodes in the network topology. If the matching result indicates that the alarm node belongs to a target subgraph network in the plurality of preset subgraph networks, the key nodes in the target subgraph network are detected to determine the monitoring and early warning result. In this way, as long as an alarm occurs in the preset subgraph network including the key nodes, the detection of the key nodes in the preset subgraph network is started, which is equivalent to early alarm detection of the key nodes before the key nodes are alarmed. Compared with the monitoring and early warning scheme of the related art, which independently monitors and collects data of each service or each level, the calculation amount is reduced, and the alarm of the key nodes can be discovered in time. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 An optional flowchart of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0042] Figure 2 An optional flowchart of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0043] Figure 3 An optional effect diagram of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0044] Figure 4 An optional flowchart of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0045] Figure 5 An optional flowchart of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0046] Figure 6 An optional effect diagram of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0047] Figure 7 An optional effect diagram of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0048] Figure 8 An optional flowchart of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0049] Figure 9 An optional effect diagram of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0050] Figure 10 An optional effect diagram of the monitoring and early warning method provided by the embodiment of the present application is shown.

[0051] Figure 11 A structural schematic diagram of a monitoring and early warning device provided for an embodiment of the present application is shown in the figure.

[0052] Figure 12 A hardware entity schematic diagram of an electronic device provided for an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0053] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0054] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0055] If similar descriptions of "first / second" appear in the application file, the following description is added, in the following description, the terms "first\second\third" referred to only distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second\third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0057] Currently, with the continuous adjustment and improvement of the emergency management system in China, the emergency management business system is developing more and more richly, and the business logic and deployment architecture are becoming more and more complex. The calling relationship between business links is relatively close. Once a problem occurs in a key node in the business link, the impact on the entire emergency-related business is huge, so it is extremely important to monitor the entire link of the business and extract key nodes and establish corresponding early warning models.

[0058] The existing business operation and maintenance monitoring means mainly includes the following two types:

[0059] Hierarchical monitoring and correlation analysis: the monitoring system is divided into customer perception, application service, middleware, cloud platform, network equipment and other levels, and independent Agent is used to collect index data for different levels, and then the collected data is grouped and correlated to assist developers and operation and maintenance personnel in quickly locating and solving problems when faults occur;

[0060] Full-link monitoring: by analyzing the target and scope of full-link monitoring, defining the monitoring index of the application, and then installing Agent on the corresponding business machine server to collect the call chain relationship of the entire business. Full-link monitoring can monitor the entire business process, so as to more comprehensively and deeply understand the business running situation, help operation and maintenance and development personnel to provide more accurate fault positioning and diagnosis, and effectively guarantee the stability and high availability of the business.

[0061] Hierarchical monitoring and full-link monitoring are time-consuming and laborious, and problems are often found relatively late in actual use, which cannot effectively guarantee the continuity and stability of the business.

[0062] The embodiment of the application provides a monitoring and early warning method, please refer to Figure 1 An optional flowchart of the monitoring and early warning method provided by the embodiment of the application will be described in combination with Figure 1 The steps shown.

[0063] S101, acquiring alarm data.

[0064] In the embodiment of the application, the monitoring and early warning device acquires alarm data for the network topology. The alarm data can be an alarm for resources or services in the network topology.

[0065] The monitoring and early warning device can be a server device for managing the network topology. The network topology is a complex network, which mainly refers to a large-scale network with complex topological structure and dynamic behavior, which is composed of a large number of nodes connected by edges, such as virtual Internet, World Wide Web and the like.

[0066] S102, matching the alarm node corresponding to the alarm data with the nodes in a plurality of preset subgraph networks to determine a matching result; wherein the preset subgraph network is determined based on the key nodes in the merged topology network; the merged topology network is determined by the dependence combination of the resource nodes and the service nodes in the network topology.

[0067] In the embodiments of the present application, the monitoring and early warning device can establish a merged topology network including the dependency relationships between resource nodes, between service nodes, and between resource nodes and service nodes based on the dependency relationships in the resource nodes and the service nodes in the network topology. The key nodes are determined in the merged topology network, and the corresponding preset subgraph network is extracted in the merged topology network with the key nodes as the center. Since the identification of the corresponding nodes is included in the alarm data, the monitoring and early warning device can determine the alarm node based on the identification in the alarm data, match the alarm node with the nodes in each preset subgraph network, and determine the matching result. The matching result is used to represent whether the alarm node exists in the nodes of the preset subgraph network.

[0068] In the embodiments of the present application, the monitoring and early warning device can construct a resource topology network based on the dependency relationships between the resource nodes in the network topology, construct a service topology network based on the dependency relationships between the service nodes, and merge the service topology network and the resource topology network based on the dependency relationships between the service nodes and the resource nodes to obtain a merged topology network.

[0069] In the embodiments of the present application, the monitoring and early warning device applies the complex network theory to model each service node, resource node, and associated relationship in the service full link, constructs a resource topology and a service topology network model, and then merges the two to obtain a merged topology network. In combination with the betweenness centrality algorithm, the key nodes in the merged topology network are identified, the dependency relationship graph is sorted out, it is identified which are strong dependency nodes affecting the core service and which are weak dependency nodes that do not affect the core service, and the monitoring and early warning data is supplemented. Not only can the problem be quickly and accurately located when the fault occurs, but also the impact of a key node failure on the service can be predicted according to the network model before the fault occurs, relevant personnel are notified in time for processing, potential problems are eliminated at the embryonic stage, and the continuity and high availability of the service are ensured.

[0070] S103, if the matching result represents that the alarm node belongs to a target subgraph network in the plurality of preset subgraph networks, the key nodes in the target subgraph network are detected to determine a monitoring and early warning result.

[0071] In the embodiments of the present application, if the matching result represents that the alarm node belongs to a target subgraph network in the plurality of preset subgraph networks, the monitoring and early warning device detects and analyzes the index data of the key nodes in the target subgraph network to determine a monitoring and early warning result. If the index data of the key nodes in the target subgraph network is abnormal, an alarm is given.

[0072] In the embodiment of the present application, when an alarm is generated at a node on the preset subgraph network centered on a key point, an early warning notice is timely sent to relevant personnel, the alarm on the node is processed, and each index of the key node of the subgraph to which the node belongs is checked. The failure node on the subgraph is avoided from affecting the key node, thereby avoiding the large-area influence on the entire service caused by the failure of the key node.

[0073] The operation steps are as follows:

[0074] (1) Extract the node corresponding to the alarm;

[0075] (2) Query the subgraph to which the alarm node belongs, if the alarm node belongs to multiple subgraphs, query all the subgraphs;

[0076] (3) Early warning is performed on the key node that may be affected by the alarm node.

[0077] The present application analyzes and sorts out the strong dependence subgraph that affects the core business in the business and resource topology network by setting the preset subgraph network, can analyze the range of business affected from the problem node to the upstream when the alarm occurs, and can be used for risk grading of the business. And when an alarm is generated at any node in the preset subgraph network and has not been propagated through the key point, intervention and processing are performed in advance, effectively preventing the failure from being propagated to a larger range through the key node.

[0078] Please refer to Figure 2 , an optional flowchart of the monitoring and early warning method provided by the embodiment of the present application, Figure 1 S102 shown can further include S201 to S203, which will be described in combination with the steps.

[0079] S201, determining the merged topology network based on the dependencies between the resource nodes and the business nodes in the network topology.

[0080] In the embodiment of the present application, the monitoring and early warning device constructs a resource topology network based on the dependencies between the resource nodes in the network topology, constructs a business topology network based on the dependencies between the business nodes, and then merges the resource topology network and the business topology network to obtain a merged topology network according to the dependencies between the corresponding business nodes in the business topology network and the corresponding resource nodes in the resource topology network.

[0081] The merged topology network is used to represent the network of the dependencies between the resource nodes, between the business nodes, and between the resource nodes and the business nodes in the network topology.

[0082] S202, traversing each node in the merged topology network, and determining the key node in the merged topology network that meets a predetermined condition.

[0083] In the embodiments of the present application, the monitoring and early warning device traverses each node in the merged topology network, determines the importance of each node, and then filters out key nodes corresponding to the importance meeting the preset condition.

[0084] The nodes in the merged topology network are established based on resource nodes or service nodes in the network topology.

[0085] The present application can use the Betweeness Centrality algorithm to identify the key nodes in the service topology network. The Betweeness Centrality measures the number of times a node appears in all shortest paths in the entire network. If a node appears in more shortest paths, the Betweeness Centrality of the node is higher, indicating that the node bears more information transmission and control aggregation in the entire network and has greater influence, which is a key node.

[0086] S203, extracting the preset subgraph network including the key nodes in the merged topology network.

[0087] In the embodiments of the present application, after determining the key nodes in the merged topology network, the preset subgraph network corresponding to each key node is extracted in the merged topology network with the key node as the center node.

[0088] In the embodiments of the present application, the merged topology network is a complex network. The commonly used analysis tools for complex networks mainly include networkx, which is a Python software package used to draw, operate complex networks, learn and analyze network structure and function, establish network models, and design network algorithms. One of the important research directions of complex networks is key node discovery: key node discovery aims to find nodes that play a key role in the structure and function of the network. Based on the characteristics of the current emergency service business itself, such as business call and deployment complexity, in order to better achieve efficient and accurate operation and maintenance, the analysis method of complex network is introduced for the discovery and mining of business key nodes, and subgraph analysis is performed on the key nodes, so as to further improve the operation and maintenance quality and ensure the continuous stability of the emergency service. Combined with the characteristics of the emergency service business itself, such as business call and deployment complexity, the analysis method of complex network is introduced for the discovery and mining of business key nodes, and subgraph analysis is performed on the key nodes, so as to further improve the operation and maintenance quality and ensure the continuous stability of the emergency service. Figure 3In constructing the node relationship network, the application constructs the business full-link monitoring based on the complex network theory. First, the resource topology network is constructed based on the resource dependency in the configuration management database (CMDB), and then the business topology network is constructed by combining the application performance monitoring and other monitoring capabilities to obtain the business calling data. Second, the obtained resource topology network and business topology network are merged through networkx to complete the construction of the entire merged topology network. The nodes included in the merged topology network can include the customer perception layer, the application service layer, the middleware layer, the cloud platform layer, and the network device layer. In the model application, the betweenness centrality algorithm is used to identify the key points of the merged topology network, and the key point subgraph extraction is performed with the key nodes as the center. The subgraph is monitored, and the key point early warning is performed when the alarm occurs. The application extracts the strong dependency subgraph affecting the core business through the key nodes, and invests more manpower and resources in the strong dependency to establish a more perfect protection system, so as to predict the influence of the problem nodes of the strong dependency subgraph on the core business, and notify the relevant personnel to handle in time, effectively prevent the fault from spreading to a larger range through the key nodes, and kill the potential problem of the business in the bud stage.

[0089] In the embodiment of the application, the key nodes are identified in the merged topology network, and the preset subgraph network including the key nodes is extracted in the merged topology network. In this way, when the key nodes are abnormal, the problem nodes can be located in time and the influence on the business can be analyzed and warned in real time, the relevant personnel are quickly notified to take preventive measures, the problem is killed in the bud stage, and the stability and high availability of the business are further ensured.

[0090] Please refer to Figure 4 An optional flowchart of the monitoring and early warning method provided in the embodiment of the application is shown in Figure 2 The S201 shown can also be implemented through S301 to S302, which will be described in combination with the steps.

[0091] S301, based on the resource dependency related data in the obtained network topology, the resource topology network is constructed based on the dependency between each resource node included in the resource dependency relationship data, and based on the business calling chain data corresponding to the obtained network topology, the business topology network is constructed.

[0092] In the embodiment of the application, the monitoring and early warning device obtains the resource dependency related data in the network topology, constructs the resource topology network based on the dependency between each resource node included in the resource dependency relationship data, and constructs the business topology network based on the business calling chain data corresponding to the network topology.

[0093] The resource topology network is used for representing the dependency relationship between the resource nodes, and the service topology network is used for representing the dependency relationship between the service nodes.

[0094] S302, based on the dependency between the resource nodes and the service nodes, the resource topology network and the service topology network are merged to determine the merged topology network.

[0095] In the embodiments of the present application, the nodes in the resource topology network are established based on the corresponding resource nodes, the nodes in the service topology network are established based on the corresponding service nodes, and the monitoring and early warning device can determine the nodes in the resource topology network that depend on the nodes in the service topology network based on the dependency between the resource nodes and the service nodes, and then connect the nodes in the resource topology network with the dependent nodes in the service topology network to form the merged topology network.

[0096] In the embodiments of the present application, the merged topology network is used to describe and define the nodes and relationships involved in the network topology, including the classification of the nodes and relationships in the network topology, the description of attribute information, etc. The merged topology network defines the attribute information of the nodes, such as the name, type, and value. In the merged topology network, the nodes can be classified into different types, such as start point and end point. Each node has a unique name and can have multiple attributes to describe the information of the node. The relationship is represented as an edge connecting different nodes in the topology network, and the merged topology network includes the name, type, and relationship attribute of the relationship. In the relationship model, the edge represents the relationship between two nodes, which can be directed or undirected, and can have multiple attributes to describe the information of the edge.

[0097] The following is the node and relationship model defined in the present application, in which the node types are shown in Table I and the relationship types are shown in Table II:

[0098] application node application service node service middleware node middleware API node API physical machine node physical machine virtual machine node virtual machine

[0099] Table I

[0100] constitute call cluster connect logical ship / host Figure 5 Figure 4 Figure 6 Figure 7 Figure 8 Figure 2

[0101] Table II

[0102] The present application introduces the complex network theory to construct the service topology network and the resource topology network, and merges the service topology network and the resource topology network based on the dependency between the resource nodes and the service nodes to obtain the merged topology network, thereby realizing the association of the node monitoring and early warning data with the business to achieve full-link monitoring of the business. When a fault occurs in the business, the user can easily find the cause of the problem from the problem node to the downstream, thereby facilitating the positioning of the problem node.

[0103] Please refer to Figure 9An optional flowchart of the monitoring and early warning method provided in the embodiments of the present application is shown in FIG. 4. Figure 10 S301-S302 shown can also be implemented through S401-S406, which will be described in combination with the steps.

[0104] S401, acquire the resource dependency related data.

[0105] In the embodiments of the present application, the monitoring and early warning device acquires the resource dependency related data from the configuration management database of the network topology. The resource dependency related data includes a plurality of resource nodes, and the node type, node name, node attribute and dependency relationship corresponding to each resource node.

[0106] In the embodiments of the present application, the monitoring and early warning device can acquire the CMDB resource dependency related data, and the resources in the CMDB resource dependency related data include hardware, software, network equipment, application program information and related configurations. Each resource corresponds to a resource name and description, attributes and characteristics, relationships and dependencies, configuration history records, etc.

[0107] S402, form a corresponding first network node based on the node type, node name and node attribute corresponding to each resource node, connect each first network node with a corresponding first dependency node to form the resource topology network.

[0108] In the embodiments of the present application, the monitoring and early warning device can form a corresponding first network node based on the node type, node name and node attribute corresponding to each resource node in a blank directed graph, and connect each first network node with a corresponding first dependency node to form the resource topology network. The first dependency node is formed based on the dependency relationship of the dependent resource nodes.

[0109] In the embodiments of the present application, the monitoring and early warning device constructs resource topology nodes based on the resource information (physical machines, virtual machines, containers, middleware, etc.) in the CMDB, and constructs the relationship between nodes based on the dependency between resources to construct the network resource topology network. For example, first, a blank directed graph DiGraph is created, and then based on the parsed resource node information, the add_node method of DiGraph is used to add nodes to the blank directed graph. When adding each node, the related information of the node is set to form a first network node according to the corresponding node type, node name, node attribute, etc. After completing the addition of the nodes, the add_edge method of DiGraph is used to add edges based on the relationship and dependency between the resource nodes to finally form the resource topology network.

[0110] S403, acquire the service call chain data.

[0111] In the embodiments of the present application, the monitoring and early warning device acquires service call chain data in a period of time or a certain number of service calls of network topology; wherein the service call chain data includes: a plurality of call service data, the call service data includes: call service method name, service node identifier, parent service node identifier and belonging resource node information.

[0112] The service call chain data describes the call relationship between each service node (component) in the application program, and mainly includes the starting point, intermediate node, target point, time consumption and other information in a link call. The application performance monitoring is used to collect the call chain data. For example, an application performance probe can be deployed in the service node to collect the service key indicator data (response time, availability rate, etc.) and the call chain relationship to obtain the service call chain data.

[0113] In the embodiments of the present application, a complete request is a tracking record (the tracking record corresponds to service call chain data). Each call is recorded as a span (the span corresponds to call service data), including span_name, span_id, Component ID, References, Tags and parent_id, the service nodes in the call service data are connected through parent_id, so as to parse a complete call. The specific field explanation is shown in Table 3. For example, the call service data of the call service method name (span_name) 1 corresponds to the service node identifier (span_id) 1, and the parent service node identifier (parent_id) in the call service data of the call service method name (span_name) 2 is also 1, so it can be confirmed that the call service data of the call service method name (span_name) 2 is the child call service data of the call service data of the call service method name (span_name) 1. Figure 9

[0114]

[0115] Table 3

[0116] S404, based on the call service method name, service node identifier, parent service node identifier and belonging resource node information in each call service data, a corresponding second network node is formed, each second network node is connected with a corresponding second dependent node to form the service topology network.

[0117] ​In the embodiments of the present application, the monitoring and early warning device can form a corresponding second network node in the blank directed graph based on the calling business method name, the business node identifier, the parent business node identifier and the belonging resource node information in each calling business data, connect each second network node with a corresponding second dependent node to form the business topology network. The second dependent node is determined based on the parent business node identifier and the business node identifier in the plurality of calling business data.

[0118] For example, a blank directed graph DiGraph is first created, and then nodes are added to the blank directed graph using the add_node method of DiGraph according to the business nodes on the calling business data. When adding each node, the relevant information of the node is set to form a corresponding second network node according to the calling business method name, the business node identifier, the parent business node identifier and the belonging resource node information. After the addition of the second network node is completed, edges are added using the add_edge method of DiGraph based on the relationship and dependence between the business nodes. That is, the start point and the end point of each edge are set according to the parsed business node identifier and parent business node identifier, and the name and type of the edge are set according to the information in the Span Name and Tags.

[0119] S405, determining a dependent first network node including the dependent node name corresponding to each second network node in the resource topology network, wherein the dependent node name is determined based on the belonging resource node information.

[0120] In the embodiments of the present application, since the belonging resource node information includes the dependent node name of the resource node on which the calling business data corresponding to the second network node depends, the dependent first network node including the dependent node name can be found in the first network node in the resource topology network.

[0121] S406, connecting each second network node with the dependent first network node to form the merged topology network.

[0122] In the embodiments of the present application, after determining the dependent first network node corresponding to each second network node, the monitoring and early warning device connects the second network node with the dependent first network node in the resource topology network to form the merged topology network.

[0123] In the embodiments of the present application, the monitoring and early warning device merges the resource topology network and the business topology network, and merges the resource topology network and the business topology network through the deployment relationship, the dependence relationship between the business nodes and the resource nodes using networkx. Finally, the merged topology network is obtained as shown in Figure 10The dependency of the system 1 node (second network node) in the service topology network on the first network node can include a distribution of nodes (DU) 1 and a DU 2 node. Connecting the system 1 node to the DU 1 and DU 2 nodes respectively can complete the merging of the resource topology network and the service topology network, and a merged topology network is obtained.

[0124] The application proposal constructs a service topology network and a resource topology network by introducing a complex network theory, and the resource topology network and the service topology network are merged to obtain a merged topology network. The application proposal realizes the association of each resource node monitoring alarm data with the business full-link monitoring, and facilitates the user to find the cause of the problem from the downstream of the problem node when the business fails. The application proposal is mainly used for problem positioning.

[0125] Please refer to Figure 11 An optional flowchart of a monitoring and early warning method provided by the application embodiment is shown in Figure 12 S202 to S203 shown can also be implemented by S501 to S504. The steps will be described in combination.

[0126] S501, determining the number of shortest paths between each pair of nodes in the merged topology network.

[0127] In the application embodiment, after the resource topology network and the service topology network are merged, the first network node and the second network node are determined as nodes in the merged topology network. The monitoring and early warning device can use the Dijkstra algorithm to calculate the shortest path between each pair of nodes, and determine the number of shortest paths between each pair of nodes.

[0128] S502, determining the number of first paths between the first predetermined pair of nodes and passing through the first node, and determining the betweenness centrality corresponding to the first node based on each first path number and the corresponding shortest path number, until the betweenness centrality corresponding to the Nth node is determined.

[0129] In the application embodiment, the monitoring and early warning device can determine the number of first paths between the first predetermined pair of nodes and passing through the first node, divide each first path number by the shortest path number of the corresponding first predetermined pair of nodes to obtain a first ratio corresponding to each first predetermined pair of nodes. The betweenness centrality corresponding to the first node is determined by weighted summing each first ratio. Then the monitoring and early warning device starts to determine the betweenness centrality of the second node, until the betweenness centrality of the Nth node is determined. The first predetermined pair of nodes is each pair of nodes in the merged topology network excluding the first node; the merged topology network includes N nodes; N is an integer greater than 1.

[0130] For example, the betweenness centrality of node V can be determined by formula (1).

[0131]

[0132] Where v represents node V in the merged topology network, σ s,t σ represents the number of shortest paths from node s to node t (nodes S and t are nodes other than node V). s,t (v) represents the number of nodes v that pass through on the route from node s to node t. Therefore, the betweenness centrality C B (v) represents the proportion of the total number of paths that pass through node v among all the shortest paths from node v.

[0133] S503. Among the N nodes, identify the key nodes whose betweenness centrality is greater than a predetermined threshold.

[0134] In this embodiment of the application, after determining the betweenness centrality of N nodes, the key nodes with betweenness centrality greater than a predetermined threshold can be identified among the N nodes.

[0135] Among them, the predetermined threshold can be adjusted appropriately according to actual business needs, and nodes with higher betweenness centrality can be designated as key nodes.

[0136] S504. Using each of the aforementioned key nodes as the center node, and using a preset number of hops as the radius of the preset subgraph network, extract the preset subgraph network from the merged topology network.

[0137] In this embodiment, the monitoring and early warning device uses each key node as the central node and a preset number of hops as the radius to extract the corresponding preset subgraph network in the merged topology network.

[0138] After identifying key nodes, a subgraph centered on each key node is extracted. In a merged topology network, a subgraph refers to a portion of the original graph's vertices and edges that form the topological structure. A subgraph centered on a key node is a subgraph formed by connecting all vertices along the topological structure starting from the key node in the original graph. The subgraph around the key node can describe the local properties of the topology network. For example, combining... Figure 12 and ​ ,exist ​ Using the key point hjqclub-application as the central node, the corresponding preset subgraph network can be extracted from the corresponding merged topology network. ​ Using the key node application as the central node, a corresponding preset subgraph network can be extracted from the merged topology network. The steps for extracting the preset subgraph network are as follows:

[0139] 1. Determine the radius (preset hop number) of the subgraph. The radius refers to how many hops to form a subgraph with the key point as the center. According to the business requirements, the application can select 2 hops as the subgraph radius.

[0140] 2. Extract the preset subgraph network with the key node as the center and a radius of 2. The specific implementation uses a breadth-first search algorithm to traverse from the key node and add nodes within the radius to the subgraph node set.

[0141] 3. Extract the edge set of the subgraph according to the subgraph node set. Extract the edges related to the subgraph node set from the graph, which can be achieved by traversing the node set and adding the edges between the nodes to the subgraph edge set.

[0142] 4. Save the extracted preset subgraph network.

[0143] The application uses the complex network betweenness centrality algorithm to analyze the key nodes in the merged topology network and performs real-time monitoring and early warning on the key nodes in the full link. When an abnormality occurs in the preset subgraph network of the key node, the key node can be detected in a timely manner, and the abnormality can be detected before the key node is alarmed. The problem can be quickly located and preventive measures can be taken to ensure the stability and high availability of the business.

[0144] Please refer to ​ , the structure diagram of the monitoring and early warning device provided by the embodiment of the application.

[0145] The embodiment of the application further provides a monitoring and early warning device 800, which comprises a data acquisition unit 801, a matching unit 802, and a detection unit 803.

[0146] The data acquisition unit 801 is configured to acquire alarm data.

[0147] The matching unit 802 is configured to match an alarm node corresponding to the alarm data with nodes in a plurality of preset subgraph networks to determine a matching result. The preset subgraph network is determined based on key nodes in a merged topology network. The merged topology network is determined by combining resource nodes and business nodes in a network topology.

[0148] The detection unit 803 is configured to detect the key nodes in a target subgraph network if the matching result indicates that the alarm node belongs to the target subgraph network, and determine a monitoring and early warning result.

[0149] In the embodiments of the present application, the monitoring and early warning device 800 is used to determine the merged topology network based on the dependencies between the resource nodes and the service nodes in the network topology; wherein the merged topology network is used to represent the dependency relationships between the resource nodes, between the service nodes and between the resource nodes and the service nodes in the network topology;

[0150] Each node in the merged topology network is traversed to determine the key nodes in the merged topology network that meet the predetermined conditions;

[0151] The preset subgraph network including the key nodes is extracted from the merged topology network.

[0152] In the embodiments of the present application, the monitoring and early warning device 800 is used to construct a resource topology network based on the acquired resource dependency related data in the network topology, and construct a service topology network based on the acquired service call chain data corresponding to the network topology; wherein the resource topology network is used to represent the dependency relationships between the resource nodes, and the service topology network is used to represent the dependency relationships between the service nodes;

[0153] Based on the dependencies between the resource nodes and the service nodes, the resource topology network and the service topology network are merged to determine the merged topology network.

[0154] In the embodiments of the present application, the monitoring and early warning device 800 is used to acquire the resource dependency related data; wherein the resource dependency related data includes: a plurality of resource nodes, and the node type, node name, node attribute and dependency relationship corresponding to each resource node;

[0155] Based on the node type, node name and node attribute corresponding to each resource node, a corresponding first network node is formed, each first network node is connected with a corresponding first dependency node to form the resource topology network; wherein the first dependency node is formed based on the dependency relationship;

[0156] The service call chain data is acquired; wherein the service call chain data includes: a plurality of call service data, and the call service data includes: call service method name, service node identifier, parent service node identifier and belonging resource node information;

[0157] The second network node is formed based on the calling business method name, the business node identifier, the parent business node identifier and the belonging resource node information in each calling business data, each second network node is connected with a corresponding second dependent node to form the business topology network, and the second dependent node is determined based on the parent business node identifier and the business node identifier in the plurality of calling business data.

[0158] In the embodiment of the application, the monitoring and early warning device 800 is used to determine a dependent first network node including a dependent node name corresponding to each second network node in the resource topology network, and the dependent node name is determined based on the belonging resource node information.

[0159] Each second network node is connected with the dependent first network node to form the merged topology network.

[0160] In the embodiment of the application, the monitoring and early warning device 800 is used to determine the number of shortest paths between each pair of nodes in the merged topology network.

[0161] The number of first paths passing through a first node between a first predetermined pair of nodes is determined, the betweenness centrality corresponding to the first node is determined based on each first path number and the corresponding shortest path number, and the betweenness centrality corresponding to the Nth node is determined; wherein the first predetermined pair of nodes is each pair of nodes in the merged topology network that does not include the first node, the merged topology network includes N nodes, and N is an integer greater than 1.

[0162] The key node corresponding to the betweenness centrality greater than a predetermined threshold value is determined in the N nodes.

[0163] In the embodiment of the application, the monitoring and early warning device 800 is used to divide each first path number by the shortest path number of the corresponding first predetermined pair of nodes to obtain a first ratio corresponding to each first predetermined pair of nodes.

[0164] Each first ratio is weighted and summed to determine the betweenness centrality corresponding to the first node.

[0165] In the embodiment of the application, the monitoring and early warning device 800 is used to extract the preset subgraph network in the merged topology network with each key node as a center node and a preset hop number as the radius of the preset subgraph network.

[0166] It should be noted that, in the embodiments of the present application, if the monitoring and early warning method described above is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to make a monitoring and early warning device (which can be a personal computer or the like) execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various media that can store program codes. Thus, the embodiments of the present application are not limited to any specific hardware and software combination.

[0167] Correspondingly, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the method of the monitoring and early warning device side.

[0168] Correspondingly, the embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program can be executed by the processor 901 of the electronic device 900 to complete the steps in the method of the monitoring and early warning device side.

[0169] Correspondingly, the embodiments of the present application provide an electronic device 900, which includes a memory 902 and a processor 901, and the memory 902 stores a computer program that can be run on the processor 901, and the processor 901 implements the steps in the above method when executing the program.

[0170] It should be noted here that: the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects to the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0171] It should be noted that, ​ A hardware entity schematic diagram of an electronic device provided in the embodiments of the present application is shown in ​ The hardware entity of the electronic device 900 includes a processor 901 and a memory 902, wherein;

[0172] The processor 901 usually controls the overall operation of the electronic device 900.

[0173] The memory 902 is configured to store instructions and applications executable by the processor 901, and can also cache data (e.g., image data, audio data, voice communication data, and video communication data) to be processed by the processor 901 and modules in the electronic device 900. The memory 902 can be implemented by a FLASH or a Random Access Memory (RAM).

[0174] It should be understood that every feature, structure, or characteristic described throughout the specification is meant to be included in at least one embodiment of the application. Therefore, it should be understood that every embodiment described throughout the specification is not necessarily to be considered as having any order of precedence. The order of description of the embodiments of the application is meant to be a logical order of description, and should not be considered as having any order of precedence. The order of description of the embodiments of the application is merely for the purpose of description, and does not represent the merit of the embodiments.

[0175] It should be noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0176] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The described apparatus embodiments are merely schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interfaces, devices, or units, and can be electrical, mechanical, or in other forms.

[0177] The units described as separate components above can or can not be physically separate, and the components displayed as units can or can not be physical units; they can be located in one place or distributed on multiple network units; and part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0178] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0179] Those skilled in the art can understand that all or part of the steps of the above method embodiments can be completed by program instruction related hardware, and the above program can be stored in a computer readable storage medium, and the program executes the steps including the above method embodiments when executed; and the above storage medium includes mobile storage devices, read only memory (ROM), magnetic discs or optical discs and various storage program codes.

[0180] Alternatively, the integrated units of the present application, if implemented in the form of software functional modules and sold or used as independent products, can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products, and the computer software products are stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The above storage medium includes mobile storage devices, ROM, magnetic discs or optical discs and various storage program codes.

[0181] The above is only an embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A monitoring and early warning method, characterized in that, include: Obtain alarm data; A merged topology network is jointly determined based on the dependencies between resource nodes and service nodes in the network topology; wherein, the merged topology network is used to characterize the network of dependencies between resource nodes, between service nodes, and between resource nodes and service nodes in the network topology; Determine the number of shortest paths between each pair of nodes in the merged topology network; The number of first paths passing through the first node is determined as the shortest path between the first predetermined pair of nodes. Based on the number of first paths and the corresponding number of shortest paths, the betweenness centrality of the first node is determined until the betweenness centrality of the Nth node is determined. Wherein, the first predetermined pair of nodes is each pair of nodes in the merged topology network that does not include the first node; the merged topology network includes N nodes; N is an integer greater than 1. Among the N nodes, identify the key nodes whose betweenness centrality is greater than a predetermined threshold. Extract a preset subgraph network that includes the key nodes from the merged topology network; The alarm node corresponding to the alarm data is matched with nodes in multiple preset subgraph networks to determine the matching result; wherein, the preset subgraph network is determined based on key nodes in the merged topology network; the merged topology network is determined jointly by the dependencies between resource nodes and service nodes in the network topology; If the matching result indicates that the alarm node belongs to a target subgraph network among the multiple preset subgraph networks, then the key node in the target subgraph network is detected to determine the monitoring and early warning result.

2. The monitoring and early warning method according to claim 1, characterized in that, The method of jointly determining and merging the network topology based on dependencies between resource nodes and service nodes in the network topology includes: A resource topology network is constructed based on the resource dependency-related data obtained in the network topology, and a service topology network is constructed based on the service call chain data corresponding to the network topology obtained; wherein, the resource topology network is used to represent the dependency relationship between each of the resource nodes, and the service topology network is used to represent the dependency relationship between each of the service nodes. Based on the dependencies between the resource nodes and the service nodes, the resource topology network and the service topology network are merged to determine the merged topology network.

3. The monitoring and early warning method according to claim 2, characterized in that, The step of constructing a resource topology network based on the acquired resource dependency-related data in the network topology, and constructing a service topology network based on the acquired service call chain data corresponding to the network topology, includes: Obtain the resource dependency related data; wherein, the resource dependency related data includes: multiple resource nodes, and the node type, node name, node attributes and dependency relationship corresponding to each resource node; Based on the node type, node name, and node attributes corresponding to each resource node, a corresponding first network node is formed, and each first network node is connected with a corresponding first dependent node to form the resource topology network; wherein, the first dependent node is formed based on the dependent resource nodes determined by the dependency relationship; Obtain the business call chain data; wherein, the business call chain data includes: multiple call business data, the call business data including: call business method name, business node identifier, parent business node identifier and associated resource node information; Based on the name of the calling service method, the service node identifier, the parent service node identifier, and the resource node information in each of the calling service data, a corresponding second network node is formed, and each second network node is connected with the corresponding second dependent node to form the service topology network; wherein, the second dependent node is determined based on the matching of the parent service node identifier with the service node identifiers in multiple of the calling service data.

4. The monitoring and early warning method according to claim 3, characterized in that, The step of merging the resource topology network and the service topology network to determine the merged topology network based on the dependency between the resource nodes and the service nodes includes: In the resource topology network, a dependent first network node is determined, including the dependent node name corresponding to each second network node; wherein, the dependent node name is determined based on the resource node information to which it belongs; Each of the second network nodes is connected to the dependent first network node to form the merged topology network.

5. The monitoring and early warning method according to claim 1, characterized in that, Determining the betweenness centrality of the first node based on the number of each first path and the corresponding number of the shortest paths includes: The first ratio is obtained by dividing the number of each first path by the number of the shortest paths of the corresponding first predetermined pair of nodes. The betweenness centrality corresponding to the first node is determined by weighted summation of each of the first ratios.

6. The monitoring and early warning method according to claim 1, characterized in that, The step of extracting a preset subgraph network including the key nodes from the merged topology network includes: Using each of the aforementioned key nodes as the center node, and using a preset number of hops as the radius of the preset subgraph network, the preset subgraph network is extracted from the merged topology network.

7. A monitoring and early warning device, characterized in that, include: The data acquisition unit is used to acquire alarm data; A matching unit is used to jointly determine a merged topology network based on the dependencies between resource nodes and service nodes in the network topology; wherein the merged topology network is used to characterize the network of dependencies between resource nodes, between service nodes, and between resource nodes and service nodes in the network topology; The matching unit is used to determine the number of shortest paths between each pair of nodes in the merged topology network; A matching unit is configured to determine the number of first paths passing through the first node in the shortest path between the first predetermined pairs of nodes, and to determine the betweenness centrality of the first node based on each number of first paths and the corresponding number of shortest paths, until the betweenness centrality of the Nth node is determined; wherein, the first predetermined pairs of nodes are each pair of nodes in the merged topology network that does not include the first node; the merged topology network includes N nodes; N is an integer greater than 1; A matching unit is used to identify, among the N nodes, the key nodes whose betweenness centrality is greater than a predetermined threshold. A matching unit is used to extract a preset subgraph network including the key nodes from the merged topology network; A matching unit is used to match the alarm node corresponding to the alarm data with nodes in multiple preset subgraph networks to determine the matching result; wherein, the preset subgraph network is determined based on key nodes in the merged topology network; the merged topology network is determined jointly by the dependencies between resource nodes and service nodes in the network topology; The detection unit is used to detect the key node in the target subgraph network if the matching result indicates that the alarm node belongs to the target subgraph network among the multiple preset subgraph networks, and to determine the monitoring and early warning result.

8. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images