Faulty network element locating method and system, and computer-readable storage medium
Through the automated fault network element positioning method, the service call chain is obtained, the abnormal network element call pairs and call sub-chains are determined. Combined with preset rules, the problem of low fault positioning efficiency in the microservice system is solved, and the rapid and accurate fault root is achieved Due to positioning, troubleshooting efficiency has been improved.
Patent Information
- Application Number
- CN202010904574.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-01
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-09-01
AI Technical Summary
In microservice systems, manual analysis and fault location efficiency are low, especially when the call chain is complex, resulting in too long business recovery time.
By obtaining the service call chain corresponding to the service request, determine the abnormal network element call pair, and automatically locate the fault root cause network element according to the exception call sub-chain and preset rules to realize automatic positioning of the fault network element.
Improves the efficiency of troubleshooting, reduces dependence on business experts, and shortens the failure recovery time.
Smart Images

Figure CN114205222B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of microservice technology, but is not limited to it, and in particular to a method for locating a faulty network element, a system thereof, and a computer-readable storage medium. Background Art
[0002] Microservice is a design concept of system architecture. Compared with the traditional monolithic architecture, microservice is built based on business capabilities. Different microservices can be implemented through different programming technologies, or rely on different external storage. Through the collaboration of multiple microservices, business requests can be processed, which greatly simplifies the development and maintenance of the system. With the increase of business, the number of network elements and nodes carrying microservices in the system is increasing, and the call chain of microservices is becoming more and more complex. In order to facilitate tracking, distributed link tracking technology is usually used to distinguish different call chains through tracking identifiers (TraceID), and then record the parent-child call relationship of microservices in the call chain through span numbers (SpanID).
[0003] When a network element in the system fails, the abnormal time period of the service request can be quickly determined through the service golden indicator, and then the service expert can find the TraceID in the abnormal log, query the specific call chain through the TraceID, and locate the faulty network element through manual analysis. Although the fault can be located, the manual analysis method is very dependent on the experience of the service expert, and it takes a lot of time when the call chain is more complicated, resulting in a long service recovery time. Summary of the invention
[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0005] The embodiments of the present invention provide a method for locating a faulty network element and a system thereof, and a computer-readable storage medium, which can realize automatic locating of a faulty network element in a call chain and improve the efficiency of troubleshooting.
[0006] In a first aspect, an embodiment of the present invention provides a method for locating a faulty network element, including:
[0007] Get the service call chain corresponding to the business request;
[0008] Determining an abnormal network element call pair in the service call chain;
[0009] Determine an abnormal call subchain according to the abnormal network element call pair;
[0010] The root cause network element of the fault is determined according to the abnormal call subchain and preset rules.
[0011] In a second aspect, an embodiment of the present invention further provides a faulty network element locating system, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the faulty network element locating method as described above when executing the computer program.
[0012] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for executing the faulty network element locating method as described above.
[0013] The embodiment of the present invention includes: obtaining a service call chain corresponding to a business request; determining an abnormal network element call pair in the service call chain; determining an abnormal call subchain according to the abnormal network element call pair; and determining a fault root cause network element according to the abnormal call subchain and preset rules. According to the solution provided by the embodiment of the present invention, the fault root cause network element can be automatically located in the service call chain corresponding to the business request, effectively improving the efficiency of troubleshooting.
[0014] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.
[0016] Figure 1 is a flow chart of a method for locating a faulty network element provided by an embodiment of the present invention;
[0017] Figure 2 is a flowchart of determining an abnormal network element call pair in a method for locating a faulty network element provided by another embodiment of the present invention;
[0018] Figure 3 is a schematic diagram of a service call chain in an abnormal time period in a method for locating a faulty network element provided by another embodiment of the present invention;
[0019] Figure 4 It is a flow chart of determining an abnormal network element call pair according to a reference call pair in a method for locating a faulty network element provided by another embodiment of the present invention;
[0020] Figure 5 is a flowchart of determining an abnormal call subchain in a method for locating a faulty network element provided by another embodiment of the present invention;
[0021] Figure 6 is a schematic diagram of an abnormal call subchain in a faulty network element locating method provided by another embodiment of the present invention;
[0022] Figure 7 is a flow chart of determining a fault root cause network element in a fault network element locating method provided by another embodiment of the present invention;
[0023] Figure 8 is a flow chart of determining the network element chaos degree of a calling node in a method for locating a faulty network element provided by another embodiment of the present invention;
[0024] Fig. 9 It is a flow chart of determining the network element chaos degree of the calling node according to the network element proportion value in the method for locating a faulty network element provided by another embodiment of the present invention;
[0025] Fig.10 is a flow chart of determining a root cause network element in a method for locating a faulty network element provided by another embodiment of the present invention;
[0026] Fig.11 It is a flow chart of determining a root cause network element according to a preset rule in a method for locating a faulty network element provided by another embodiment of the present invention;
[0027] Fig.12 It is a flowchart of traversing the abnormal call subchain to determine the root cause network element of the fault in the fault network element location method provided by another embodiment of the present invention;
[0028] Fig.13A It is a schematic diagram of a calling node of a calling layer in an abnormal subchain in a faulty network element locating method provided by another embodiment of the present invention;
[0029] Fig. 13B It is a schematic diagram of a network element calling layer in an abnormal subchain in a faulty network element locating method provided by another embodiment of the present invention;
[0030] Fig.14 is a schematic diagram of a device of a faulty network element locating system provided by another embodiment of the present invention;
[0031] Fig.15 It is a schematic diagram of a device of a faulty network element locating system provided by another embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0033] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0034] The present invention provides a method for locating a faulty network element and a system thereof, and a computer-readable storage medium. The method for locating a faulty network element comprises: obtaining a service call chain corresponding to a service request; determining an abnormal network element call pair in the service call chain; determining an abnormal call subchain according to the abnormal network element call pair; and determining a fault root cause network element according to the abnormal call subchain and preset rules. According to the solution provided by an embodiment of the present invention, the fault root cause network element can be automatically located in the service call chain corresponding to the service request, effectively improving the efficiency of troubleshooting.
[0035] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.
[0036] like Figure 1 As shown, Figure 1 It is a flowchart of a faulty network element locating method provided by an embodiment of the present invention. The faulty network element locating method includes but is not limited to step S110, step S120, step S130 and step S140.
[0037] Step S110, obtaining a service call chain corresponding to the business request.
[0038] In one embodiment, the business request can be any request in any business, can have a continuous call chain, and the microservice module called according to the call chain can implement a specific function. For example, it can be a business request to implement a single function, or it can be a business request to implement one of the sub-businesses in a multi-functional business.
[0039] In one embodiment, the call of the microservice cannot be traced during the business execution process. Therefore, the fault network element locating method of this embodiment can be used after the business stops executing, by mining the data in the business operation log to obtain the root cause network element of the fault, thereby solving the problem of low efficiency of relying on manual fault locating in the prior art.
[0040] In one embodiment, since a business request may include any number of service call chains, in order to ensure that the root cause of the faulty network element is found, all service call chains in the business request may be obtained to form a service call chain set, and then the specific root cause network element may be located from the service call chain set. Alternatively, the service call chains may be divided into normal call chains and abnormal call chains through some existing data indicators, and only the root cause network element of the fault may be located for the abnormal call chain, thereby reducing the amount of calculation.
[0041] Step S120: determining an abnormal network element call pair in the service call chain.
[0042] In one embodiment, an abnormal network element call pair is a network element call pair whose operating indicators in a service call chain do not meet normal values. For most microservices, they are usually carried by one network element. Therefore, when a network element fails, due to the layer-by-layer calls of microservices, functions in the same call chain as the failed network element will usually be affected. Therefore, there are usually a large number of abnormal network element pairs in the service call chain. In order to ensure the accurate positioning of the network element that is the root cause of the failure, all identifiable abnormal network element call pairs in the service call chain can be obtained and formed into a collection for further analysis.
[0043] Step S130, determining an abnormal call subchain according to the abnormal network element call pair.
[0044] Based on the above embodiments, when a network element carrying a microservice fails, the microservices in the same call chain as the failed network element will be affected to a certain extent, so the abnormal network element call pair can form a continuous abnormal call sub-chain. It should be noted that the situation where multiple faulty network elements appear in the same call chain is not within the scope of discussion of the present invention. If multiple network elements fail, there will be too many factors that cause abnormalities in the network element call pair, and it is impossible to locate the only faulty network element. Therefore, the embodiment of the present invention only discusses the situation where there is one faulty network element in a service call chain, and no further details will be given later. Of course, if there are multiple service call chains in a business request, and each service call chain has a faulty network element, the fault root cause network element of each service call chain can also be located separately according to the faulty network element location method of the embodiment of the present invention.
[0045] In one embodiment, an abnormal call subchain is determined based on an abnormal network element call pair. The abnormal call subchain can be formed based on the parent-child relationship of the network element call pair, or the call chain in which the abnormal network element call pair is located can be directly determined as the abnormal call subchain. The specific determination method can be selected according to actual needs, and this embodiment does not make any limitations.
[0046] Step S140, determining the fault root cause network element according to the abnormal call subchain and preset rules.
[0047] In one embodiment, after determining the abnormal call sub-chain, it can be determined that the root cause network element is definitely located in the abnormal call sub-chain. Therefore, the root cause network element can be determined according to any preset rules, such as eliminating network elements one by one based on the order of traversal, or filtering based on data by setting multiple thresholds. The specific method can be formulated according to actual needs, and it is sufficient to determine a root cause network element from the abnormal call sub-chain.
[0048] Based on the above embodiments, the abnormality of the root cause network element will spread in the service call chain, and the direction of its diffusion is random. For example, a network element in the middle of the call chain fails. Since the network elements on both sides of the call chain will call the faulty network element in turn, the abnormal impact generated will spread as the call chain proceeds. Therefore, the network element where the abnormal impact converges can be determined in the abnormal call sub-chain according to preset rules. The specific manifestation of this can be that the number of network elements in the call node is messy, that is, the degree of confusion is high. The specific microservice where the abnormality occurs can also be determined in the abnormal sub-chain according to preset rules, and the network element that carries the microservice is set as the root cause network element. The specific method can be adjusted according to the preset rules, and this embodiment does not impose too many restrictions.
[0049] In addition, refer to Figure 2 , in one embodiment, Figure 1 Step S120 in the illustrated embodiment also includes but is not limited to the following steps:
[0050] Step S210, determining the abnormal call time period, and obtaining a call chain set in the service call chain whose call time matches the abnormal call time period;
[0051] Step S220, obtaining each network element call pair in the call chain set;
[0052] Step S230, obtaining a reference calling pair having the same calling relationship as the network element calling pair;
[0053] Step S240: determining an abnormal network element calling pair according to the network element calling pair and the reference calling pair.
[0054] In one embodiment, during the operation of the business, the same business request may be executed multiple times. Therefore, the abnormal call time period can be determined in any way, such as using the data of normal operation of the business request to train a detection model, and directly identifying the abnormal call time period through the detection model. It should be noted that the training method of the detection model is a method of the prior art, not an improvement made in this embodiment, and will not be repeated here.
[0055] In one embodiment, the abnormal time period can be of any length. For example, the detection model in the above embodiment can be used to accurately match the exact time period where the abnormality occurs. It can also be considered that the call of the network element requires time. After matching the time period according to the above detection model, a period of time is obtained forward and backward to form the abnormal time period to ensure data integrity. It should be noted that for a service request, multiple abnormal time periods can be matched through the detection model. In this case, the root cause network element of each abnormal time period can be located one by one. After locating several root cause network elements, further analysis can be performed through existing technologies to obtain the specific cause of the failure in the system, which will not be repeated here.
[0056] Based on the above embodiment, since there may be multiple abnormal time periods in the service request, the call chain set in step S210 is a call chain set within the same abnormal time period. It should be noted that the root cause network elements in different abnormal time periods may be different, and the embodiment of the present invention only locates the service request with one root cause network element. Therefore, in the subsequent embodiments, it is assumed that there is only one root cause network element in the same call chain set, which will not be repeated later. It should be noted that since the service request within the abnormal time period can have multiple service call chains, not all service call chains are abnormal. However, in order to ensure the integrity of the data, this embodiment first forms a call chain set determined within the abnormal time period, and further analyzes the call chain set to locate the root cause network element.
[0057] In one embodiment, reference Figure 3 , Figure 3 This is a schematic diagram of a service call chain in an abnormal time period in a faulty network element location method provided by another embodiment of the present invention. In the service call chain, each call layer may have a number of nodes, and each node is connected in sequence according to the parent-child relationship to form multiple call sub-chains, that is, the call chain set described in the above embodiment. It should be noted that Figure 3 The naming and arrangement of the nodes in are for illustrative purposes only and do not limit the embodiments of the present invention. It should be noted that, for the sake of ease of description, the naming method of the nodes in the embodiments of the present invention is "microservice@network element", for example, the node "csf_001@docker_003", the meaning of the node name is: the name of the microservice is "csf_001", and the name of the network element that carries the microservice is "docker_003". Unless otherwise specified, the same naming method is used in subsequent examples.
[0058] In one embodiment, after obtaining each network element call pair in the call chain set in step S220, the call chain set can be saved in the form of a list or a data dictionary, for example, according to Figure 3The call chain set shown can save the call chain set in the form shown in Table 1 to facilitate subsequent one-to-one matching. The specific recording method can be selected according to actual needs and is not limited in this embodiment.
[0059]
[0060] Table 1 Call chain collection saving example
[0061] In one embodiment, since the business request also includes a normal calling time period, the call chains within the normal calling time period are all working normally. Therefore, the network element call pairs within the normal calling time period can be obtained as reference call pairs to provide a data basis for matching abnormal network element call pairs. For example, the parent-child call relationship of the network element call pairs can be obtained, and then the call pairs with the same parent-child call relationship can be selected from the network element call pairs within the normal time period as reference call pairs.
[0062] In addition, refer to Figure 4 , in one embodiment, Figure 2 Step S240 in the illustrated embodiment also includes but is not limited to the following steps:
[0063] Step S241, obtaining a call index value of a network element call pair;
[0064] Step S242, obtaining a reference call index value of the reference call pair;
[0065] Step S243: When the difference between the call index value and the reference call index value is greater than or equal to a preset call index threshold, the network element call pair is determined as an abnormal network element call pair.
[0066] In one embodiment, the call index value can be any value that can reflect the call performance, such as common golden indicators, including the number of service requests, the number of successes, the success rate, the service time consumption, etc. The specific call index value is selected according to actual needs, and it can distinguish between normal network element call pairs and abnormal network element call pairs by value.
[0067] based on Figure 3In the embodiment shown in , since the reference call pair is a network element call pair that calls within a normal time period, the call index value of the reference call pair can be used as a reference call index value. By comparing the call index value of the network element call pair in the abnormal call time period with the reference index value, it can be determined whether the network element call pair is an abnormal network element call pair. It should be noted that the comparison between the above-mentioned call index value and the reference index value can be carried out in any statistical manner, such as average value, maximum value, minimum value, etc. For example, the call index value is compared with the average value of multiple reference index values. If the difference between the two data is greater than a preset threshold, the network element call pair is identified as an abnormal network element call pair. The specific method can be selected according to actual needs, and this embodiment does not limit it.
[0068] In addition, refer to Figure 5 , in one embodiment, Figure 1 Step S130 in the illustrated embodiment also includes but is not limited to the following steps:
[0069] Step S310, obtaining the calling parent-child relationship of each abnormal network element calling pair within the calling abnormal time period;
[0070] Step S320, determining the abnormal calling subchain according to the calling parent-child relationship.
[0071] In one embodiment, since each microservice in the service call chain is called in sequence, except for the root node and the bottom node, each network element will be called by another network element at the same time as calling one network element. It can be understood by those skilled in the art that the calling relationship is a calling parent-child relationship, in which the called calling node is the parent calling node, and the node that calls a calling node is the child calling node. The principle is the same in the calling node, and this embodiment will not be repeated. It should be noted that since each abnormal network element call pair in the service call chain has a network element with a parent-child relationship, after determining the abnormal network element call pair, the abnormal network element call pair is spliced according to the parent-child relationship to obtain an abnormal call subchain, for example Figure 6 The exception call subchain shown in , which is Figure 3 The call chain set shown in FIG is composed of some network element call pairs, that is, determining Figure 6 The network element shown in is an abnormal network element. After calling the pair, it can be spliced to get Figure 6 The exception call subchain is shown.
[0072] In addition, refer to Figure 7 , in one embodiment, Figure 1 Step S140 in the illustrated embodiment also includes but is not limited to the following steps:
[0073] Step S410, determining the calling layer in the abnormal calling subchain and the calling node in the calling layer, and obtaining the node network element information of the calling node;
[0074] Step S420, determining the network element disorder degree of the calling node in the calling layer according to the node network element information;
[0075] Step S430, determining the root cause network element of the calling layer according to the network element disorder degree and preset rules;
[0076] Step S440, determining the fault root cause network element in the abnormal call subchain according to the root cause network element of the call layer.
[0077] In one embodiment, the call layer is a call level in a call chain, and the call node can be divided into a parent call node and a child call node according to the call parent-child relationship, for example, Figure 6 The abnormal call subchain shown in the figure is distinguished by the call layer, and the call layers of the parent call node and the child call node can be obtained as follows Fig.13A As shown, the abnormal call subchain includes three call layers. For example, the parent call node of the first layer is "None", the child call node is "osb_001@os_021", and so on, which will not be repeated here.
[0078] In one embodiment, the network element disorder degree may be used to describe the number and degree of confusion of network elements in the calling node. Common network element disorder degrees may be information entropy and Gini purity. A specific type may be selected according to actual needs.
[0079] In one embodiment, since the calling nodes in each calling layer are called according to the parent-child relationship, the node causing the fault can be determined according to the network element chaos value, thereby determining the corresponding root cause network element. However, since the root cause network element is only the root cause of the calling layer, and is not the root cause that causes the abnormality of the entire calling sub-chain, after determining the root cause network element, this embodiment needs to further determine the fault root cause network element according to the root cause network element of each calling layer. It should be noted that after a network element fails, the fault is gradually propagated to other network elements in the calling chain. The network element that calls the fault root cause network element will call multiple other network elements due to the fault, thereby further causing errors in subsequent calls. Therefore, as the fault root cause network element, its network element chaos is relatively low, that is, according to the root cause network element of each calling layer, the root cause can eventually be converged to one network element, that is, the fault root cause network element.
[0080] In addition, refer to Figure 8 , in one embodiment, Figure 7 Step S410 in the illustrated embodiment also includes but is not limited to the following steps:
[0081] Step S510, determining the total number of network elements in the calling layer where the calling node is located and the number of network elements of the same type in the calling layer where the calling node is located according to the node network element information;
[0082] Step S520, obtaining a ratio of the number of network elements of the same type to the total number of network elements;
[0083] Step S530: determining the network element disorder degree of the calling node in the calling layer where the calling node is located according to the network element proportion value.
[0084] In one embodiment, since the network element chaos degree is the degree of confusion of the network elements in the calling node, in order to better reflect the degree of confusion, considering that there may be multiple network elements in the calling node, this embodiment can first calculate the network element proportion value, that is, the proportion of each type of network element in a calling node in a calling layer.
[0085] The following combination Fig. 13B The method of this embodiment is illustrated by a specific example, wherein: Fig. 13B for Fig.13A The schematic diagram of the network elements in the parent call node and the child call node is shown below. Fig. 13B Taking the parent call node in the call layer shown in the figure as an example, the calculation method of the sub-network element is similar, so I will not repeat it:
[0086] like Fig. 13B As shown, in the first layer of the parent call node, there is one network element "none", so the number of network elements of the same type is 1, the total number of network elements is 1, and the network element ratio of the first layer is 1 / 1=1. In the second layer of the parent call node, there are two network elements, both of which are "os_022", so the number of network elements of the same type is 2, the total number of network elements is 2, and the network element ratio of the second layer is 2 / 2=1. In the third layer of the parent call node, there are eight network elements, including four "docker_002" and four "docker_001", so the number of network elements of the same type is 4, the total number of network elements is 8, and the network element ratio of the third layer is 4 / 8=0.5.
[0087] In addition, refer to Fig. 9 , in one embodiment, Figure 8 Step S530 in the illustrated embodiment also includes but is not limited to the following steps, including:
[0088] Step S610, when the total number of network elements is less than or equal to the preset network element number threshold, and the maximum value of the network element proportion values is greater than or equal to the preset proportion threshold, the network element disorder degree of the calling node in the calling layer where the calling node is located is determined to be low;
[0089] or,
[0090] Step S620, when the total number of network elements is less than or equal to the preset network element number threshold, and the maximum value of the network element proportion values is less than the preset proportion threshold, the network element disorder degree of the calling node in the calling layer where the calling node is located is determined to be high;
[0091] or,
[0092] Step S630, when the total number of network elements is greater than the preset network element number threshold, obtain the mapping relationship between the network element proportion value and the preset network element chaos degree, and determine the network element chaos degree of the calling node in the calling layer where the calling node is located according to the mapping relationship and the network element proportion value.
[0093] In one embodiment, the specific values of the network element quantity threshold and the proportion threshold can be adjusted according to actual needs, and this embodiment does not limit it. It should be noted that step S610 and step S620 only need to judge by the quantity relationship, and do not need to involve other calculations other than the network element proportion value.
[0094] In one embodiment, the mapping relationship may be a formula for calculating the degree of disorder according to the network element ratio. For example, when the Gini purity is used as the degree of disorder, the mapping relationship may adopt the calculation formula of the Gini purity in the prior art: where p k is the proportion of network elements of the kth type, and k is the number of types of network elements in the calling node. It should be noted that the purer the Gini purity, the less chaotic the network element. Therefore, if the Gini purity is pure, the corresponding network element has low chaos, and if the Gini purity is impure, the corresponding network element has high chaos.
[0095] The following is a specific example Fig. 13B The calculation of the network element disorder degree of the parent call node of the call layer shown in the figure is explained. For the convenience of description, the network element quantity threshold in this example is 5, the proportion threshold is 0.8, and the network element disorder degree takes Gini purity as an example. Since the calculated Gini purity is a specific value, the purity threshold can also be set to determine the degree of disorder of the network element. In this example, 0.3 is used as the purity threshold:
[0096] It should be noted that the proportion of network elements in each call layer can be referred to Figure 8 The calculations in the embodiment shown in are not repeated here.
[0097] For the first call layer, there is only one network element, which is less than the network element quantity threshold, and the network element proportion value is 1, so the network element chaos is low;
[0098] For the second call layer, it contains two network elements, which is less than the network element quantity threshold, and the network element proportion value is 1, so the network element chaos is low;
[0099] For the third call layer, which contains eight network elements, which is greater than the network element number threshold, the above formula is used to calculate the Gini purity. There are two types of network elements in this call layer, namely "docker_001" and "docker_002", and the network element proportion value is 0.5, so the Gini purity is Among them, p1 is the network element proportion value of network element "docker_001", and p2 is the network element proportion value of network element "docker_002". Since 0.5 is greater than the purity threshold 0.3, the network element disorder degree is high.
[0100] Referring to the above calculation method, we can Fig. 13B The network element disorder degree of the sub-network element shown is further calculated and will not be elaborated here.
[0101] It should be noted that, using the above calculation method, Fig. 13B The network element chaos degree of the parent calling node and the child calling node is as follows
[0102] As shown in Table 2:
[0103] Call Layer Parent Call Node Sub-call node First floor Low Low Second floor Low high Third floor high Low
[0104] Table 2. Network element chaos of parent calling nodes and child calling nodes
[0105] In addition, refer to Fig.10 , in one embodiment, Figure 7 Step S430 in the illustrated embodiment also includes but is not limited to the following steps:
[0106] Step S710, determining the calling parent-child relationship between each calling node in the calling layer;
[0107] Step S720, determining the parent calling node and the child calling node in the calling layer according to the calling parent-child relationship;
[0108] Step S730, obtaining the network element disorder degree of the parent calling node and the network element disorder degree of the child calling node;
[0109] Step S740: determining the root cause network element of the calling layer according to the network element disorder degree of the parent calling node, the network element disorder degree of the child calling node and a preset rule.
[0110] It should be noted that the parent calling node and the child calling node can be different nodes or the same node. For example, when a calling node contains two network elements, according to the service calling chain, one network element of the calling node needs to call the microservice carried by another network element. Then, the parent calling node and the child calling node determined by the calling parent-child relationship are both the calling node. It can be understood by those skilled in the art that the calling parent-child relationship is for each network element calling pair. For example, referring to Figure 6 and Fig.13A ,for Figure 6In the abnormal call subchain shown in the figure, in the first call layer, the call node "osb_001@os_022" is the first node, so the node has no parent call node called, and the parent call node is set to "None"; in the second call layer, the call node "osb_001@os_022" is called by the call nodes "csf_001@docker_001" and "csf_001@docker_002" respectively, so in the second layer of the call layer, there are two pairs of parent-child relationships, namely: parent call node "osb_001@os_022" and child call node "csf_001@docker_001", parent call node "osb_001@os_022" and child call node "csf_001@docker_002", and the relationship of the third call layer is similar, which will not be repeated here.
[0111] In one embodiment, reference Fig.13A and Fig. 13B , the parent calling node and the child calling node can be determined according to the specific calling mode of the network element. For example, in this embodiment, the network element corresponding to the calling node "osb_001@os_022" is "os_022". Based on this, according to Fig.13A The parent-child relationship between the call nodes in the call layer shown in the figure can be obtained as follows: Fig. 13B The corresponding relationship between the parent calling node and the child calling node is shown. Those skilled in the art can understand that this embodiment takes one calling node with one network element as an example for explanation. When multiple network elements are set in the calling node, the parent calling node and the child calling node can be determined one by one according to the calling relationship between the network elements, and no further details are given here.
[0112] It should be noted that the network element chaos degrees of the parent calling node and the child calling node can be obtained separately and independently. Since the network element chaos degree is an attribute of the network element itself, it will not be affected by the parent-child calling relationship. This embodiment distinguishes between the parent calling node and the child calling node only for the subsequent update of the root cause network element through the network element chaos degree, which will not be repeated here.
[0113] In one embodiment, in step S740, the network element chaos degree of the parent calling node and the network element chaos degree of the child calling node can be calculated respectively, and then the root cause network element of the calling layer can be determined according to preset rules. If the calling layer is traversed layer by layer, the calculation can also be performed when traversing to the calling layer, which can be adjusted according to actual needs.
[0114] In addition, refer to Fig.11 In one embodiment, the preset rules may include the following rules, and any one of the following rules may be satisfied:
[0115] When the network element disorder degree of the parent calling node is low and the network element disorder degree of the child calling node is high, the root cause network element is updated to the network element in the parent calling node;
[0116] or,
[0117] When the network element disorder degree of the parent calling node is high and the network element disorder degree of the child calling node is low, the root cause network element is updated to the network element in the child calling node;
[0118] or,
[0119] When the network element chaos degree of the parent calling node and the child calling node is low, the root cause network element is updated to the network element in the child calling node;
[0120] or,
[0121] When the network element chaos degrees of the parent calling node and the child calling node are both high, the root cause network element is not updated.
[0122] In one embodiment, the preset rules can be formulated according to the adopted traversal method, and the root cause network element of the abnormal call sub-chain can be determined according to the root cause network element of each call layer. For example, the method of traversing the call layer from the top layer and updating the root cause network element layer by layer can adopt the preset rules described in this embodiment. Those skilled in the art are motivated to adjust the preset rules according to the traversal method and the network element chaos, which will not be repeated here.
[0123] For example, refer to Fig. 13B , the network element in the parent call node of the first call layer is "None", and the network element in the child call node is "os_022"; the network element in the parent call node of the second call layer is "os_022", and the network elements in the child call node are "docker_002" and "docker_001"; the network elements in the parent call node of the third call layer are "docker_002" and "docker_001", and the network element in the child call node is "docker_008". Referring to the chaos degree of the parent call node and the child call node shown in Table 2, the root cause network element of each call layer can be obtained as shown in Table 3:
[0124]
[0125] Table 3, Schematic diagram of root cause network elements at the call layer
[0126] It should be noted that according to the calculation method of the network element chaos degree in the above embodiment, the network element chaos degree of the calling node with a large number of network elements can be high by setting the chaos degree threshold. When the network element chaos degrees of the parent calling node and the child calling node are both high, the root cause network element is not updated, and further judgment can be made through the next pair of network element calls.
[0127] In addition, refer to Fig.12 , in one embodiment, Figure 7 Step S440 in the illustrated embodiment also includes but is not limited to the following steps:
[0128] Step S810, traversing the call layers in the abnormal call subchain;
[0129] Step S820: determining the fault root cause network element in the abnormal call subchain according to the root cause network element of the call layer.
[0130] In one embodiment, in a service call chain, the occurrence of a faulty network element will affect the performance of the entire call chain, so it is impossible to simply determine the root cause network element through indicator data. Based on this, it is necessary to traverse all network element call pairs of the abnormal call subchain at least once to determine the root cause network element. It should be noted that the traversal direction can be selected according to actual needs, such as starting from the top layer of the call layer, starting from the bottom layer of the call layer, or starting from the middle of the call layer. For example, using the preset rules of this embodiment, it is possible to locate the root cause network element by traversing from the top layer.
[0131] The following table 3 and Fig. 13B Here is an example of starting the traversal from the top level:
[0132] According to the root cause network elements of each call layer obtained in Table 3, after traversing the first layer, the candidate fault root cause network element is "os_022", after traversing the second layer, the candidate fault root cause network element remains "os_022", and after traversing the third layer, the candidate fault root cause network element is updated to "docker_008". At this time, the traversal is completed, so "docker_008" is determined to be the fault root cause network element in the abnormal call subchain.
[0133] It can be understood by those skilled in the art that after locating the root cause network element, it is necessary to further explore and analyze the cause of the failure of the root cause network element. The method of mining and analysis is not an improvement made in this embodiment. The technical solution of the present invention only needs to complete the positioning of the root cause network element, replacing the method of manual positioning in the prior art, thereby improving the positioning efficiency of the root cause network element.
[0134] In addition, refer to Fig.14 An embodiment of the present invention further provides a faulty network element location system, including: a service request anomaly detection module and a faulty network element location module. The service request anomaly detection module is used to detect anomalies in service requests, such as executing Figure 1 Step S110 of the illustrated embodiment may also be used to execute the detailed steps thereof, such as Figure 2Step S210 of the embodiment shown. The faulty network element location module includes an abnormal call subchain identification module and a root cause network element acquisition module. The abnormal call subchain identification module is used to identify and acquire the abnormal subchain in the service request, for example, to execute Figure 1 Step S120 and step S130 in the illustrated embodiment may also be used to execute the corresponding detailed steps, for example Figure 2 In the embodiment shown, steps S220 to S240, Figure 5 Step S310 and step S320 of the method, Fig.10 The method steps S710 and S720 in the method; the root cause network element acquisition module is used to determine the fault root cause network element in the abnormal call subchain identified by the abnormal call subchain identification module, such as executing Figure 1 Step S140 in the illustrated embodiment may also be used to execute the corresponding detailed steps, such as Figure 4 Steps S241 to S243 of the method, Figure 7 Steps S410 to S440 of the method, Figure 8 Steps S510 to S530 of the method, Fig. 9 Method steps S610 to S630.
[0135] It should be noted that the faulty network element locating module can also be connected to a faulty network element root cause diagnosis module used to diagnose the root cause of the faulty network element. The faulty network element root cause diagnosis module can be a functional module in the prior art used to further explore and diagnose the fault causes of the located faulty root cause network element. The specific method is not within the scope of improvement of this embodiment and will not be repeated here.
[0136] It should be noted that the various modules of the above-mentioned faulty network element locating system may be functional modules arranged in a terminal, server or other device that can be used for root cause diagnosis. Those skilled in the art are motivated to select specific hardware or software functional modules according to actual needs, and this embodiment does not impose any limitation thereto.
[0137] In addition, refer to Fig.15 An embodiment of the present invention further provides a faulty network element location system 1000, which includes: a memory 1020, a processor 1010, and a computer program stored in the memory 1020 and executable on the processor 1010.
[0138] The processor 1010 and the memory 1020 may be connected via a bus or other means.
[0139] The non-transient software program and instructions required to implement the faulty network element location method of the above embodiment are stored in the memory 1020. When executed by the processor 1010, the faulty network element location in the above embodiment is performed, for example, the above described Figure 1 Steps S110 to S140 of the method, Figure 2 Steps S210 to S240 of the method, Figure 4 Steps S241 to S243 of the method, Figure 5 Steps S310 to S320 of the method, Figure 7 Steps S410 to S440 of the method, Figure 8 Steps S510 to S530 of the method, Fig. 9 Steps S610 to S630 of the method, Fig.10 Steps S710 to S740 of the method, Fig.12 Method steps S810 to S820.
[0140] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0141] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by a processor or a controller, for example, by a processor in the above-mentioned faulty network element location system embodiment, so that the above-mentioned processor can execute the faulty network element location method in the above-mentioned embodiment, for example, execute the above-mentioned Figure 1 Steps S110 to S140 of the method, Figure 2 Steps S210 to S240 of the method, Figure 4 Steps S241 to S243 of the method, Figure 5 Steps S310 to S320 of the method, Figure 7 Steps S410 to S440 of the method, Figure 8 Steps S510 to S530 of the method, Fig. 9 Steps S610 to S630 of the method, Fig.10 Steps S710 to S740 of the method, Fig.12Method steps S810 to step S820 in the method. It will be appreciated by those skilled in the art that all or some of the steps and systems in the method disclosed above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0142] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.
Claims
1. A method for locating a faulty network element, comprising: Get the service call chain corresponding to the business request; Determining an abnormal network element call pair in the service call chain; Determine an abnormal call subchain according to the abnormal network element call pair; Determine the root cause network element of the fault according to the abnormal call subchain and preset rules; The determining of the fault root cause network element according to the abnormal calling subchain and preset rules includes: Determine a call layer in the abnormal call subchain and a call node in the call layer, and obtain node network element information of the call node, wherein the call layer includes a parent call node and a child call node; Determine the network element disorder degree of the calling node in the calling layer according to the node network element information, wherein the network element disorder degree is used to characterize the information entropy or Gini purity of different network element types in the calling node of the calling layer; Determine the root cause network element of the calling layer according to the network element disorder degree and preset rules; The fault root cause network element in the abnormal call subchain is determined according to the root cause network element of the call layer.
2. A method for locating a faulty network element according to claim 1, characterized in that: The determining of an abnormal network element call pair in the service call chain includes: Determine the abnormal call time period, and obtain a call chain set in the service call chain whose call time matches the abnormal call time period; Obtain each network element call pair in the call chain set; Acquire a reference call pair having the same call relationship as the network element call pair; An abnormal network element calling pair is determined according to the network element calling pair and the reference calling pair.
3. A method for locating a faulty network element according to claim 2, characterized in that: The determining the abnormal network element calling pair according to the network element calling pair and the reference calling pair includes: Obtaining a call index value of the network element call pair; Obtaining a reference call index value of the reference call pair; When the difference between the call index value and the reference call index value is greater than or equal to a preset call index threshold, the network element call pair is determined as an abnormal network element call pair.
4. A method for locating a faulty network element according to claim 2, characterized in that: The determining the abnormal call subchain according to the abnormal network element call pair includes: Obtaining the calling parent-child relationship of each abnormal network element calling pair within the calling abnormal time period; The abnormal call subchain is determined according to the calling parent-child relationship.
5. A method for locating a faulty network element according to claim 1, characterized in that: The determining the network element disorder degree of the calling node in the calling layer according to the node network element information includes: Determine the total number of network elements in the calling layer where the calling node is located and the number of network elements of the same type in the calling layer where the calling node is located according to the node network element information; Obtaining a ratio of the number of network elements of the same type to the total number of network elements; The network element disorder degree of the calling node in the calling layer where the calling node is located is determined according to the network element proportion value.
6. A method for locating a faulty network element according to claim 5, characterized in that: The determining, according to the network element proportion value, the network element disorder degree in the calling layer where the calling node is located comprises: When the total number of network elements is less than or equal to a preset network element number threshold, and the maximum value of the network element proportion values is greater than or equal to a preset proportion threshold, the network element disorder degree of the calling node in the calling layer where the calling node is located is determined to be low; or, When the total number of network elements is less than or equal to a preset network element number threshold, and the maximum value of the network element proportion values is less than a preset proportion threshold, the network element disorder degree of the calling node in the calling layer where the calling node is located is determined to be high; or, When the total number of network elements is greater than the preset network element number threshold, the mapping relationship between the network element proportion value and the preset network element chaos degree is obtained, and the network element chaos degree of the calling node in the calling layer where the calling node is located is determined according to the mapping relationship and the network element proportion value.
7. A method for locating a faulty network element according to claim 6, characterized in that: The determining the root cause network element of the calling layer according to the network element disorder degree and the preset rule includes: Determine the calling parent-child relationship between each calling node in the calling layer; Determine the parent call node and the child call node in the call layer according to the call parent-child relationship; Obtaining the network element disorder degree of the parent calling node and the network element disorder degree of the child calling node; The root cause network element of the calling layer is determined according to the network element disorder degree of the parent calling node, the network element disorder degree of the child calling node and the preset rule.
8. A method for locating a faulty network element according to claim 7, characterized in that: The preset rules include: When the network element disorder degree of the parent calling node is low and the network element disorder degree of the child calling node is high, updating the root cause network element to the network element in the parent calling node; or, When the network element disorder degree of the parent calling node is high and the network element disorder degree of the child calling node is low, updating the root cause network element to the network element in the child calling node; or, When the network element chaos degrees of the parent calling node and the child calling node are both low, the root cause network element is updated to the network element in the child calling node; or, When the network element confusion degrees of the parent calling node and the child calling node are both high, the root cause network element is not updated.
9. A method for locating a faulty network element according to any one of claims 5 to 8, characterized in that: The determining the fault root cause network element in the abnormal call subchain according to the root cause network element of the call layer includes: Traversing the call layers in the abnormal call subchain; The fault root cause network element in the abnormal call subchain is determined according to the root cause network element of the call layer.
10. A faulty network element location system, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the faulty network element locating method as claimed in any one of claims 1 to 9 when executing the computer program.
11. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the faulty network element locating method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Service fault positioning method, device, computer equipment and storage medium
CN108833184A
Fault positioning method and device
CN110351136A
Link topological graph display method and device, storage medium and equipment
CN110442641A
Fault positioning method and device, electronic equipment and storage medium
CN111597070A
Service Chain Management Method, Delivery Node, Controller, and Value-Added Service Node
US20160134465A1
